{"id":55249,"date":"2017-04-13T13:59:45","date_gmt":"2017-04-13T11:59:45","guid":{"rendered":"https:\/\/blog.wedos.cz\/?p=55249"},"modified":"2021-01-13T14:04:48","modified_gmt":"2021-01-13T13:04:48","slug":"vysvetleni-situace-ohledne-dnssec-dne-4-4-2017","status":"publish","type":"post","link":"https:\/\/blog.wedos.com\/cs\/vysvetleni-situace-ohledne-dnssec-dne-4-4-2017","title":{"rendered":"Vysv\u011btlen\u00ed situace ohledn\u011b DNSSEC dne 4. 4. 2017"},"content":{"rendered":"\n<p>Jako v\u017edy jsme otev\u0159eni a nic netaj\u00edme.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p>V \u00fater\u00fd 4. dubna do\u0161lo ke komplikac\u00edm u .cz dom\u00e9n, kter\u00e9 jsou zabezpe\u010den\u00e9 pomoc\u00ed technologie DNSSEC, co\u017e vedlo k jejich \u010d\u00e1ste\u010dn\u00e9 nedostupnosti z n\u011bkter\u00fdch s\u00edt\u00ed. R\u00e1di bychom vysv\u011btlili, co se p\u0159esn\u011b stalo a jak to ovlivnilo jednotliv\u00e9 slu\u017eby.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Co se p\u0159esn\u011b stalo<\/h2>\n\n\n\n<p>Do\u0161lo k technick\u00e9 chyb\u011b p\u0159i generov\u00e1n\u00ed kl\u00ed\u010d\u016f DNSSEC a t\u00edm do\u0161lo k probl\u00e9m\u016fm s p\u0159ekl\u00e1d\u00e1n\u00edm n\u00e1zv\u016f dom\u00e9n (a ov\u011b\u0159ov\u00e1n\u00ed dom\u00e9n u mailov\u00fdch slu\u017eeb) p\u0159i pou\u017eit\u00ed DNS server\u016f, kter\u00e9 validuj\u00ed DNSSEC. T\u011bchto server\u016f je (na\u0161t\u011bst\u00ed) zat\u00edm minimum a na dom\u00e9ny se tak nedostalo n\u011bkolik jednotek % n\u00e1v\u0161t\u011bvn\u00edk\u016f ze s\u00edt\u00ed, kde n\u00e1v\u0161t\u011bvn\u00edci pou\u017e\u00edvaj\u00ed DNS severy s validac\u00ed DNSSEC. Podle na\u0161ich m\u011b\u0159en\u00ed se to t\u00fdkalo zhruba 3&nbsp;&#8211;&nbsp;5&nbsp;% provozu (prvn\u00ed odhady byly vy\u0161\u0161\u00ed a hovo\u0159ili jsme o cca 10%). Jednalo se nap\u0159\u00edklad o \u010d\u00e1st s\u00ed\u0165\u011b O2 a s\u00edt\u011b, kde pou\u017e\u00edvaj\u00ed DNS servery Google.<\/p>\n\n\n\n<p>ZSK kl\u00ed\u010d k DNSSEC je pravideln\u011b a zcela automaticky jednou za m\u011bs\u00edc vym\u011b\u0148ov\u00e1n robotem. A to tak, \u017ee 5 dn\u00ed p\u0159ed vypr\u0161en\u00edm kl\u00ed\u010de je vygenerov\u00e1n nov\u00fd kl\u00ed\u010d a je p\u0159id\u00e1n do z\u00f3ny. Tento postup pou\u017e\u00edv\u00e1me u\u017e od roku 2011 a a\u017e do pond\u011bl\u00ed 3.4.2017 to fungovalo naprosto bez probl\u00e9m\u016f.<\/p>\n\n\n\n<p>V pond\u011bl\u00ed se bohu\u017eel stalo to, \u017ee nov\u00fd kl\u00ed\u010d vytvo\u0159en nebyl. Podle log\u016f ke spou\u0161t\u011bn\u00ed robota doch\u00e1zelo jako p\u0159edt\u00edm ani u procesu nebyla hl\u00e1\u0161ena \u017e\u00e1dn\u00e1 chyba. Technik\u016fm tak nedo\u0161lo varov\u00e1n\u00ed, \u017ee by p\u0159i vytv\u00e1\u0159en\u00ed bylo n\u011bco \u0161patn\u011b (robot nevr\u00e1til chybu) ani n\u00e1\u0161 monitoring nezachytil, \u017ee by robot nefungoval. Ten se spou\u0161t\u011bl zcela b\u011b\u017en\u00fdm zp\u016fsobem.<\/p>\n\n\n\n<p>Pro tuto situaci jsme bohu\u017eel nem\u011bli monitoring. T\u00edm p\u00e1dem do\u0161lo k vyexpirov\u00e1n\u00ed kl\u00ed\u010de a zneplatn\u011bn\u00ed \u0159et\u011bzce d\u016fv\u011bry.<\/p>\n\n\n\n<p>Resolvery poskytovatel\u016f internetu, kter\u00e9 d\u016fkladn\u011b ov\u011b\u0159uj\u00ed DNSSEC pak vracely u\u017eivatel\u016fm chybu. Co\u017e bylo zhruba 3&nbsp;&#8211;&nbsp;5&nbsp;% internetov\u00e9 populace, kter\u00e1 se po dobu, kdy m\u011bla dom\u00e9na .cz neplatn\u00fd kl\u00ed\u010d, pokou\u0161ela dostat na dom\u00e9nu.<\/p>\n\n\n\n<p>Situaci jsme okam\u017eit\u011b za\u010dali \u0159e\u0161it p\u0159egenerov\u00e1n\u00edm kl\u00ed\u010d\u016f, ke v\u0161em dom\u00e9n\u00e1m, kter\u00fdm spravujeme DNS z\u00e1znamy, co\u017e je p\u0159es 179 tis\u00edc.<\/p>\n\n\n\n<p>N\u00e1\u0161 syst\u00e9m pro generov\u00e1n\u00ed z\u00f3nov\u00fdch soubor\u016f bohu\u017eel nem\u011bl v tehdej\u0161\u00ed podob\u011b pro tuto situaci dostate\u010dn\u011b v\u00fdkonn\u00fd. Generov\u00e1n\u00ed nov\u00fdch kl\u00ed\u010d\u016f a n\u00e1sledn\u011b z\u00f3nov\u00fdch soubor\u016f je velmi n\u00e1ro\u010dn\u00e1 operace na v\u00fdkon a pr\u00e1ci s disky. &nbsp;P\u0159esto\u017ee jsme vyu\u017e\u00edvali prakticky cel\u00fd sever jen na tuto \u010dinnost, tak to nebylo dosta\u010duj\u00edc\u00ed.&nbsp;<br><br>Prim\u00e1rn\u011b jsme se tak zam\u011b\u0159ili na obnovu kl\u00ed\u010d\u016f na\u0161ich z\u00e1kazn\u00edk\u016f, kte\u0159\u00ed n\u00e1m psali na z\u00e1kaznickou podporu, d\u00e1le na\u0161ich z\u00e1kazn\u00edk\u016f, kte\u0159\u00ed u n\u00e1s maj\u00ed slu\u017eby, n\u00e1sledn\u011b parkovac\u00ed str\u00e1nky a nakonec dom\u00e9n bez A z\u00e1znamu.<\/p>\n\n\n\n<p>Abychom cel\u00fd proces urychlili, na\u0161i v\u00fdvoj\u00e1\u0159i spole\u010dn\u011b s techniky p\u0159ipravili nov\u00e9 provizorn\u00ed \u0159e\u0161en\u00ed pro obnovu kl\u00ed\u010d\u016f. V\u0161e bylo spu\u0161t\u011bno je\u0161t\u011b vedle p\u016fvodn\u00edho syst\u00e9mu a generov\u00e1n\u00ed m\u011blo vyhrazen\u00fd v\u00fdkon jen pro sebe. Po jeho spu\u0161t\u011bn\u00ed jsme se dostali na 0,7 &#8211; 2 vte\u0159iny na dom\u00e9nu ve 30 vl\u00e1knech procesoru sou\u010dasn\u011b. Toto \u0159e\u0161en\u00ed v budoucnu vylep\u0161\u00edme a nasad\u00edme jako sou\u010d\u00e1st na\u0161eho DNS. \u0158e\u0161en\u00ed jsme nasadili r\u00e1no a v\u00fdrazn\u011b tak urychlili generov\u00e1n\u00ed.<\/p>\n\n\n\n<p>P\u0159i v\u0161ech t\u011bchto zm\u011bn\u00e1ch je d\u016fle\u017eit\u00e9 nejen ud\u011blat generov\u00e1n\u00ed nov\u00fdch z\u00f3nov\u00fdch soubor\u016f, ale zajistit jejich konzistenci a bezchybnost a z\u00e1rove\u0148 v\u0161e rozdistribuovat na na\u0161e 4 DNS servery, kter\u00e9 m\u00e1me (2 v \u010cR, 1 v N\u011bmecku a 1 v Nizozem\u00ed) a potom v\u0161echny z\u00f3ny znovu na\u010d\u00edst. Vzhledem k tomu, \u017ee se jednalo o cca 179.000 dom\u00e9n, tak jist\u011b ch\u00e1pete, \u017ee podobn\u00e1 &#8222;operace&#8220; byla velmi n\u00e1ro\u010dn\u00e1.<\/p>\n\n\n\n<p>Kl\u00ed\u010de n\u00e1m vypr\u0161ely p\u0159ed 19. hodinou v \u00fater\u00fd ve\u010der. Prvn\u00ed probl\u00e9my jsme za\u010dali evidovat p\u0159ed 21:00 a ji\u017e ve 21:40 jsme prov\u00e1d\u011bli obnovu dom\u00e9n. Druh\u00fd den v cca 15:30 bylo v\u0161e vy\u0159\u00edzen\u00e9, p\u0159i\u010dem\u017e posledn\u00ed aktivn\u00ed slu\u017eby (krom\u011b parkovan\u00fdch dom\u00e9n) byly zcela funk\u010dn\u00ed kolem 13:30 (n\u00e1sledn\u011b se generovaly z\u00f3ny pro dom\u00e9ny, kter\u00e9 nemaj\u00ed slu\u017eby).&nbsp;<\/p>\n\n\n\n<p>O cel\u00e9 situaci jsme informovali jak na soci\u00e1ln\u00edch s\u00edt\u00edch (prvn\u00ed p\u0159\u00edsp\u011bvek tak byl po n\u011bkolika minut\u00e1ch od prvn\u00edch probl\u00e9m\u016f), tak i na str\u00e1nk\u00e1ch statusu na\u0161ich slu\u017eeb a z\u00e1rove\u0148 i v chatu a administraci slu\u017eeb pro na\u0161e z\u00e1kazn\u00edky. V pr\u016fb\u011bhu noci jsme v\u0161e n\u011bkolikr\u00e1t aktualizovali a pracovali jsme na tom s n\u011bkolika pracovn\u00edky sou\u010dasn\u011b.<\/p>\n\n\n\n<p>Na webu jsme m\u011bli n\u00e1vod, jak p\u0159\u00edpadn\u011b rychleji obnovit svoji slu\u017ebu. Sta\u010dilo n\u00e1m napsat nebo v z\u00e1kaznick\u00e9 administraci kliknout na aktualizaci zm\u011bn v nastaven\u00ed DNS. T\u00edm se z\u00f3na vygenerovala p\u0159ednostn\u011b a do 30 minut byla slu\u017eba pln\u011b funk\u010dn\u00ed ze v\u0161ech s\u00edt\u00ed.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Jak se v\u00fdpadek DNSSEC podepsal na na\u0161ich slu\u017eb\u00e1ch<\/h2>\n\n\n\n<p>Zasa\u017eeny byly &nbsp;dom\u00e9ny .cz s DNSSEC (tedy weby a e-maily na t\u011bchto dom\u00e9n\u00e1ch).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Emaily<\/h3>\n\n\n\n<p>Nejd\u0159\u00edve za\u010dneme s emaily. N\u00e1\u0161 mailserver ov\u011b\u0159uje z bezpe\u010dnostn\u00edch d\u016fvod\u016f, zdali byl email skute\u010dn\u011b odesl\u00e1n z dom\u00e9ny, kterou deklaruje. K tomu vyu\u017e\u00edv\u00e1 na\u0161e DNS. Jeliko\u017e nebyl validn\u00ed z\u00e1znam DNSSEC, tak mailserver odm\u00edtl email poslat. Toto se t\u00fdkalo pouze email\u016f odes\u00edlan\u00fdch p\u0159es SMTP. Emaily odeslan\u00e9 pomoc\u00ed PHP skriptu odch\u00e1zeli v po\u0159\u00e1dku.<\/p>\n\n\n\n<p>Probl\u00e9m s emaily jsme vy\u0159e\u0161ili p\u0159enastaven\u00edm ov\u011b\u0159ov\u00e1n\u00ed z na\u0161eho DNS na DNS servery CZ.NIC. &nbsp;<br><strong>Dopln\u011bn\u00ed na\u0161eho p\u016fvodn\u00edho tvrzen\u00ed &#8222;<\/strong>Ano, je p\u0159ekvapen\u00ed, \u017ee DNS servery CZ.NICu nevaliduj\u00ed DNSSEC.&#8220;&nbsp;<strong>&nbsp; a vysv\u011btlen\u00ed ze strany CZ.NIC&nbsp;<\/strong><strong>(21. 4. 2017)<\/strong>:&nbsp;<br><em>&#8222;Na\u0161e DNS samoz\u0159ejm\u011b validuj\u00ed, ve va\u0161em p\u0159\u00edpad\u011b to&nbsp;\u010d\u00e1ste\u010dn\u011b fungovalo proto, \u017ee unbound v defaultn\u00edm nastaven\u00ed (n\u00e1\u0161 p\u0159\u00edpad)&nbsp;dr\u017e\u00ed expirovan\u00e9 podpisy 10% doby platnosti podpisu&nbsp;(expiration-inception). S t\u00edm, \u017ee minim\u00e1ln\u00ed \u010das je jedna hodina,&nbsp;maxim\u00e1ln\u00ed pak 24 hodin.<\/em><\/p>\n\n\n\n<p><em>Tak\u017ee z toho m\u016f\u017eeme asi konstatovat, \u017ee opravdu po n\u011bjakou dobu na\u0161e&nbsp;ODVR nevracelo chybu, p\u0159esto\u017ee byl expirovan\u00fd DNSSEC.&#8220;<\/em><br><strong>Za nep\u0159esnou formulaci se omlouv\u00e1me. Ur\u010dit\u011b to nebylo nic proti CZ.NIC, ale popisujeme celou situaci.<\/strong><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Weby<\/h3>\n\n\n\n<p>.CZ dom\u00e9ny, kter\u00e9 u n\u00e1s maj\u00ed DNS s nastaven\u00fdm DNSSEC se pro n\u00e1v\u0161t\u011bvn\u00edka, jeho\u017e resolver ov\u011b\u0159uje platnost DNSSEC zd\u00e1li jako nedostupn\u00e9. Toto postihlo odhadem 3 &#8211; 5 % n\u00e1v\u0161t\u011bvn\u00edk\u016f. Jednalo se o n\u00e1v\u0161t\u011bvn\u00edky z \u010ceska, kte\u0159\u00ed pou\u017e\u00edvaj\u00ed DNS Google, \u010d\u00e1st n\u00e1v\u0161t\u011bvn\u00edk\u016f s\u00edt\u011b O2 a Vodafone a n\u011bkter\u00fdch dal\u0161\u00ed s\u00edt\u00ed.<\/p>\n\n\n\n<p>K tomuto \u010d\u00edslu jsme do\u0161li porovn\u00e1n\u00edm p\u0159\u00edstup\u016f z webhosting\u016f z log\u016f na\u0161\u00ed DDoS ochrany, kter\u00e1 s\u00ed\u0165 detailn\u011b monitoruje, d\u00e1le porovn\u00e1n\u00edm p\u0159enos\u016f dat od na\u0161ich dodavatel\u016f konektivity.<\/p>\n\n\n\n<p>Nejednalo se tedy o nedostupnost kompletn\u011b v\u0161eho nebo naopak o to, \u017ee by neproch\u00e1zela \u010d\u00e1st po\u017eadavk\u016f, ale jednalo se o to, \u017ee z n\u011bkter\u00fdch s\u00edt\u00ed byly skute\u010dn\u011b dom\u00e9ny nedostupn\u00e9 nebo m\u00e9n\u011b dostupn\u00e9 a to v z\u00e1vislosti na tom jak\u00e9 DNS servery konkr\u00e9tn\u00ed n\u00e1v\u0161t\u011bvn\u00edk pou\u017eil.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Porovn\u00e1n\u00ed provozu<\/h3>\n\n\n\n<p>Na n\u00e1sleduj\u00edc\u00edm grafu vid\u00edte celkov\u00fd provoz od 16:00 3.4.2017 do 16:00 4.4.2017. V horn\u00ed \u010d\u00e1sti je p\u0159\u00edchoz\u00ed provoz z internetu k n\u00e1m a v spodn\u00ed \u010d\u00e1sti odchoz\u00ed provoz od n\u00e1s do internetu (k n\u00e1v\u0161t\u011bvn\u00edk\u016fm).&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0404.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"249\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0404-1024x249.png\" alt=\"\" class=\"wp-image-55253 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0404-1024x249.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0404-300x73.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0404-768x187.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0404.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/249;\" \/><\/a><\/figure>\n\n\n\n<p>Pro porovn\u00e1n\u00ed celkov\u00fd provoz od 16:00 3.4.2017 do 16:00 6.4.2017.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0406.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"249\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0406-1024x249.png\" alt=\"\" class=\"wp-image-55259 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0406-1024x249.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0406-300x73.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0406-768x187.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r4-bits-0403-0406.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/249;\" \/><\/a><\/figure>\n\n\n\n<p>Jak vid\u00edte na grafech \u017e\u00e1dn\u00e9 velk\u00e9 propady n\u00e1v\u0161t\u011bvnosti nejsou po dobu incidentu vid\u011bt. DDoS v\u011bt\u0161ina neov\u011b\u0159uje.&nbsp;<\/p>\n\n\n\n<p>Samoz\u0159ejm\u011b celkov\u00fd provoz zahrnuje webhostingy, VPS, dedikovan\u00e9 servery a WEDOS disk. Proto jsme vybrali je\u0161t\u011b graf \u010dist\u011b 88 tis\u00edc webhosting\u016f NoLimit p\u0159es IPv4 a IPv6 podle druhu komunikace. Graf zahrnuje obdob\u00ed od 16:00 3.4.2017 do 16:00 6.4.2017. Na\u0161e DDoS ochrana dok\u00e1\u017ee v\u0161echno velmi detailn\u011b kontrolovat a vyhodnocovat v re\u00e1ln\u00e9m \u010dase.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/r10-aplications-0403-0406.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"249\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/r10-aplications-0403-0406-1024x249.png\" alt=\"\" class=\"wp-image-55265 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r10-aplications-0403-0406-1024x249.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r10-aplications-0403-0406-300x73.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r10-aplications-0403-0406-768x187.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/r10-aplications-0403-0406.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/249;\" \/><\/a><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Jednotliv\u00e9 s\u00edt\u011b<\/h3>\n\n\n\n<p>Psali jsme, \u017ee nedostupnost se dotkla jen n\u011bkter\u00fdch s\u00edt\u00ed. Pro porovn\u00e1n\u00ed v\u00e1m p\u0159in\u00e1\u0161\u00edme grafy p\u0159enos\u016f z t\u011bch nejv\u011bt\u0161\u00edch. Jedn\u00e1 se o dobu od 16:00 3.4.2017 do 16:00 6.6.2017.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Telefonica Czech Republic, a.s. (O2)<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/O2-0403-0406.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"223\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/O2-0403-0406-1024x223.png\" alt=\"\" class=\"wp-image-55271 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/O2-0403-0406-1024x223.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/O2-0403-0406-300x65.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/O2-0403-0406-768x167.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/O2-0403-0406.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/223;\" \/><\/a><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">UPC Liberty Global Operations B.V.<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/UPC-0403-0406.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"223\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/UPC-0403-0406-1024x223.png\" alt=\"\" class=\"wp-image-55277 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/UPC-0403-0406-1024x223.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/UPC-0403-0406-300x65.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/UPC-0403-0406-768x167.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/UPC-0403-0406.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/223;\" \/><\/a><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">T-Mobile Czech Republic a.s.<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/T-mobile-0403-0406.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"223\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/T-mobile-0403-0406-1024x223.png\" alt=\"\" class=\"wp-image-55283 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/T-mobile-0403-0406-1024x223.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/T-mobile-0403-0406-300x65.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/T-mobile-0403-0406-768x167.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/T-mobile-0403-0406.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/223;\" \/><\/a><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Vodafone Czech Republic a.s.<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/Vodafone-0403-0406.png\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" width=\"1024\" height=\"223\" data-src=\"https:\/\/blog.wedos.cz\/wp-content\/uploads\/2021\/01\/Vodafone-0403-0406-1024x223.png\" alt=\"\" class=\"wp-image-55289 lazyload\" data-srcset=\"https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/Vodafone-0403-0406-1024x223.png 1024w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/Vodafone-0403-0406-300x65.png 300w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/Vodafone-0403-0406-768x167.png 768w, https:\/\/blog.wedos.com\/wp-content\/uploads\/2021\/01\/Vodafone-0403-0406.png 1101w\" data-sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/223;\" \/><\/a><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Co jsme ud\u011blali proto, aby se to u\u017e nikdy nestalo a pl\u00e1novan\u00e1 vylep\u0161en\u00ed<\/h2>\n\n\n\n<p>Vylep\u0161ili jsme monitoring podobn\u00fdch situac\u00ed a bude prov\u00e1d\u011bna nejen automatick\u00e1 kontrola, ale i pravideln\u00e1 kontrola prost\u0159ednictv\u00edm \u010dlov\u011bka.<\/p>\n\n\n\n<p>Rychlej\u0161\u00ed gener\u00e1tor z\u00f3nov\u00fdch soubor\u016f dovol\u00ed prov\u00e1d\u011bt zm\u011bny v DNS rychleji.<\/p>\n\n\n\n<p>D\u00e1le m\u00e1me v pl\u00e1nu upravit na\u0161e webov\u00e9 rozhran\u00ed pro emaily tak, aby zmizela n\u011bkter\u00e1 omezen\u00ed. P\u0159eci jen pokud se u\u017eivatel u\u017e jednou do webov\u00e9ho rozhran\u00ed p\u0159ihl\u00e1s\u00ed, tak nen\u00ed nutn\u00e9 ov\u011b\u0159ovat zdali odes\u00edl\u00e1 email z dom\u00e9ny ke kter\u00e9 je p\u0159ihl\u00e1\u0161en. C\u00edlem je dos\u00e1hnout stavu, kdy p\u0159es na\u0161e webov\u00e9 rozhran\u00ed p\u016fjde odes\u00edlat email za v\u0161ech okolnost\u00ed.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Z\u00e1kaznick\u00e1 podpora<\/h2>\n\n\n\n<p>V pr\u016fb\u011bhu ve\u010dera&nbsp;byla na\u0161e z\u00e1kaznick\u00e1 podpora p\u0159et\u00ed\u017een\u00e1, proto\u017ee p\u0159ijala n\u011bkolik tis\u00edc mail\u016f a chat\u016f s po\u017eadavky, kter\u00e9 souvisej\u00ed s v\u00fd\u0161e uveden\u00fdm probl\u00e9mem.<\/p>\n\n\n\n<p>Pokud oper\u00e1tor nem\u016f\u017ee p\u0159ijmout chat v d\u016fsledku toho, \u017ee jich m\u00e1 otev\u0159en\u00fdch v\u00edce ne\u017e zvl\u00e1dne, tak po ur\u010dit\u00e9 dob\u011b &#8222;spadne&#8220; komunikace do po\u017eadavk\u016f (podobn\u011b jako zpr\u00e1vy z kontaktn\u00edho formul\u00e1\u0159e). Jakmile m\u00e1 oper\u00e1tor &#8222;voln\u00e9 ruce&#8220;, tak proch\u00e1z\u00ed po\u017eadavky a postupn\u011b na n\u011b odpov\u00edd\u00e1 emailem.&nbsp;<\/p>\n\n\n\n<p>Kolegov\u00e9 po\u017eadavky postupn\u011b vy\u0159izovali a b\u011bhem noci bylo vy\u0159\u00edzeno v\u0161e. R\u00e1no u\u017e jsme odpov\u00eddali na v\u0161echny dotazy pr\u016fb\u011b\u017en\u011b.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Z\u00e1v\u011br<\/h2>\n\n\n\n<p>Omlouv\u00e1me se za komplikaci. Mus\u00edme uznat, \u017ee to byla nep\u0159\u00edjemn\u00e1 chyba a na po\u010d\u00e1tku jsme m\u011bli strach o to, \u017ee dopad bude mnohem nep\u0159\u00edjemn\u011bj\u0161\u00ed, ne\u017e nakonec byl. Z t\u00e9to nep\u0159\u00edjemn\u00e9 zku\u0161enosti jsme se pou\u010dili a ud\u011bl\u00e1me n\u011bkolik \u00faprav, aby se to p\u0159\u00ed\u0161t\u011b neopakovalo.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Jako v\u017edy jsme otev\u0159eni a nic netaj\u00edme.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,32],"tags":[],"class_list":["post-55249","post","type-post","status-publish","format-standard","hentry","category-spolecnost","category-novinky"],"_links":{"self":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts\/55249","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/comments?post=55249"}],"version-history":[{"count":1,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts\/55249\/revisions"}],"predecessor-version":[{"id":55298,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts\/55249\/revisions\/55298"}],"wp:attachment":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/media?parent=55249"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/categories?post=55249"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/tags?post=55249"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}