{"id":55053,"date":"2014-08-26T12:46:59","date_gmt":"2014-08-26T10:46:59","guid":{"rendered":"https:\/\/blog.wedos.cz\/?p=55053"},"modified":"2021-01-13T12:47:58","modified_gmt":"2021-01-13T11:47:58","slug":"co-se-deje-v-siti-aneb-neco-malo-o-ddos-a-planovanych-upravach-site","status":"publish","type":"post","link":"https:\/\/blog.wedos.com\/cs\/co-se-deje-v-siti-aneb-neco-malo-o-ddos-a-planovanych-upravach-site","title":{"rendered":"Co se d\u011bje v s\u00edti aneb n\u011bco m\u00e1lo o DDoS a pl\u00e1novan\u00fdch \u00faprav\u00e1ch s\u00edt\u011b"},"content":{"rendered":"\n<p>Dovolte, abychom informovali o tom, co se u n\u00e1s aktu\u00e1ln\u011b d\u011bje a co p\u0159ipravujeme (z hlediska s\u00edt\u011b).<\/p>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\">Popis situace z odpoledne dne 26. srpna<\/h2>\n\n\n\n<p>N\u011bkte\u0159\u00ed z v\u00e1s (zhruba 1\/3) m\u011bla dnes odpoledne mezi 13:14 a 14:00 probl\u00e9m s p\u0159\u00edstupem na n\u011bkter\u00e9 slu\u017eby.<strong>&nbsp;Tento probl\u00e9m byl zap\u0159\u00ed\u010din\u011bn rozs\u00e1hlou poruchou IP s\u00edt\u011b u \u010cD Telematika (d\u00e1le jen \u010cDT), kter\u00e1 je jedn\u00edm z dodavatel\u016f konektivity pro na\u0161e datacentrum.<\/strong><\/p>\n\n\n\n<p>Zhruba 2\/3 klient\u016f nem\u011bly \u017e\u00e1dn\u00fd probl\u00e9m krom\u011b cca 30 sekund, kdy se p\u0159epo\u010d\u00edt\u00e1vala routovac\u00ed tabulka a m\u011bnily se cesty na tras\u00e1ch k n\u00e1m (co\u017e mohlo b\u00fdt zhruba dal\u0161\u00edch 30-60 sekund).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Update ve 20:00:<\/h3>\n\n\n\n<p>Od 19:33 na na\u0161i s\u00ed\u0165 prob\u011bhl siln\u00fd \u00fatok, kter\u00fd v dob\u011b psan\u00ed \u010dl\u00e1nku st\u00e1le prob\u00edh\u00e1. Detaily n\u00ed\u017ee.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Co se p\u0159esn\u011b stalo?<\/h3>\n\n\n\n<p>U \u010cDT Telematika do\u0161lo k poru\u0161e p\u00e1te\u0159n\u00edho routeru, kter\u00fd p\u0159ij\u00edmal v\u0161echny pakety a neroutoval je d\u00e1l (jednodu\u0161e \u0159e\u010deno &#8211; h\u00e1zel je do \u010dern\u00e9 d\u00edry). Bohu\u017eel tento router pakety p\u0159ij\u00edmal na obou stran\u00e1ch a to jak od n\u00e1s, tak i sm\u011brem z internetu. Proto nebylo mo\u017en\u00e9 hned diagnostikovat probl\u00e9m, kter\u00fd byl p\u0159\u00ed\u010dinou rozs\u00e1hl\u00e9ho v\u00fdpadku cel\u00e9 IP s\u00edt\u011b \u010cD Telematika, kter\u00e1 je jedn\u00edm z p\u0159edn\u00edch ISP v \u010cesk\u00e9 republice.<\/p>\n\n\n\n<p>Zjednodu\u0161en\u011b \u0159e\u010deno router se na v\u0161echny strany &#8222;tv\u00e1\u0159il&#8220;, \u017ee je v po\u0159\u00e1dku, ale nic nepropou\u0161t\u011bl.<\/p>\n\n\n\n<p>Z\u00e1lo\u017en\u00ed router \u010cDT Telematika, kter\u00fd m\u00e1me p\u0159ipojen\u00fd na druh\u00e9 trase (do jin\u00e9ho na\u0161eho routeru), byl p\u0159et\u00ed\u017een a tak odbavil jen malou \u010d\u00e1st p\u0159enos\u016f (zhruba 10%).<\/p>\n\n\n\n<p><strong>Update 27. srpna 20:00<\/strong><\/p>\n\n\n\n<p>Dle dne\u0161n\u00edch informac\u00ed z \u010cDT za p\u00e1d jejich router\u016f mohl \u00fatok na na\u0161i s\u00edt. Nem\u00e1me d\u016fvod tomu nev\u011b\u0159it a jen to dokl\u00e1d\u00e1 s\u00edlu a proveden\u00ed DDoS \u00fatoku, kter\u00fd prob\u011bhl. Komplikaci v jejich s\u00edti (ale ji\u017e v\u00fdrazn\u011b men\u0161\u00ed) zp\u016fsobil i ve\u010dern\u00ed \u00fatok. Celou situaci jsme dnes n\u011bkolikr\u00e1t \u0159e\u0161ili s \u010cDT a v\u011b\u0159\u00edme, \u017ee spole\u010dn\u011b najdeme cestu, jak se \u00fasp\u011b\u0161n\u011b br\u00e1nit.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Pro\u010d nepomohla z\u00e1loha?<\/h3>\n\n\n\n<p>Z\u00e1loha nepomohla pr\u00e1v\u011b z d\u016fvodu, \u017ee se prim\u00e1rn\u00ed router \u010cDT &#8222;tv\u00e1\u0159il&#8220;, \u017ee je v po\u0159\u00e1dku a ne, \u017ee m\u00e1 v\u00fdpadek nebo n\u011bjakou poruchu. Pokud by nastala porucha, tak by byl ve\u0161ker\u00fd p\u0159enos (ven i dovnit\u0159 na\u0161\u00ed s\u00edt\u011b) automaticky p\u0159esm\u011brov\u00e1n p\u0159es jin\u00e9ho dodavatele.<\/p>\n\n\n\n<p>Z\u00e1lo\u017en\u00ed router \u010cDT byl tak\u00e9 spojen, ale byl p\u0159et\u00ed\u017een a tak tak\u00e9 nedo\u0161lo k automatick\u00e9mu p\u0159esm\u011brov\u00e1n\u00ed na jin\u00e9ho dodavatele.<\/p>\n\n\n\n<p>Bohu\u017eel z na\u0161\u00ed strany nebylo nic poznat a zvenku tak\u00e9 ne. Probl\u00e9m s routerem u \u010cDT ne\u0161lo diagnostikovat. V podstat\u011b se jednalo o probl\u00e9m v internetu, zcela mimo na\u0161\u00ed s\u00ed\u0165.<\/p>\n\n\n\n<p>V\u00fdpadek jsme zaregistrovali ve 13:14 prakticky okam\u017eit\u011b na\u0161\u00edm monitoringem. My jsme nejprve ov\u011b\u0159ovali, zda nejsou probl\u00e9my na optick\u00fdch tras\u00e1ch. Tam probl\u00e9m nebyl. Kontaktovali jsme tedy dodavatele konektivity Kaora, kter\u00fd n\u00e1m tak\u00e9 potvrdil, \u017ee probl\u00e9m u nich nen\u00ed. N\u00e1sledn\u011b jsme se sna\u017eili kontaktovat podporu \u010cDT, kde n\u00e1m bylo sd\u011bleno, \u017ee maj\u00ed rozs\u00e1hl\u00fd v\u00fdpadek IP slu\u017eeb, ale \u017ee p\u0159\u00ed\u010dinu zat\u00edm neznaj\u00ed a tak jsme \u0159e\u0161ili kde je probl\u00e9m. Zkusili jsme nejprve ru\u010dn\u011b odpojit jednu z tras, co\u017e nepomohlo. N\u00e1sledovalo ru\u010dn\u00ed odpojen\u00ed druh\u00e9 trasy k \u010cDT. B\u011bhem zm\u011bn do\u0161lo k \u00faprav\u00e1m routovac\u00edch tabulek protokolu BGP, co\u017e m\u011blo za n\u00e1sledek probl\u00e9m na cca 30-60 sekund prakticky na v\u0161echnu konektivitu.&nbsp;<\/p>\n\n\n\n<p>Po rozpojen\u00ed obou peeringov\u00fdch propoj\u016f, kter\u00e9 z\u00e1rove\u0148 slou\u017e\u00ed jako upstream p\u0159ipojen\u00ed pro na\u0161\u00ed s\u00ed\u0165, kde te\u010de cca 1\/3 na\u0161eho provozu, do\u0161lo okam\u017eit\u011b k \u00fapln\u00e9mu obnoven\u00ed konektivity v cel\u00e9m rozsahu.<\/p>\n\n\n\n<p>Dne\u0161n\u00ed probl\u00e9m nebylo v na\u0161ich sil\u00e1ch mo\u017en\u00e9 n\u011bjak ovlivnit a z\u00e1rove\u0148 nebylo prakticky mo\u017en\u00e9 jej vy\u0159e\u0161it rychleji, ne\u017e za cca 40-45 minut (b\u011bhem t\u00e9to doby m\u011bla cca 1\/3 po\u017eadavk\u016f probl\u00e9m). Za komplikace se klient\u016fm omlouv\u00e1me.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Update k situaci mezi 19:33 a\u017e 20:00<\/h3>\n\n\n\n<p>V 19:33 za\u010dal siln\u00fd DDoS \u00fatok, kter\u00fd ucpal trasu k na\u0161emu poskytovateli Kaora. Vzhledem k odpoledn\u00edm probl\u00e9m\u016f s \u010cDT jsme nem\u011bli jejich s\u00ed\u0165 zapojenou. To jsme cht\u011bli ud\u011blat v noci, proto\u017ee jsme necht\u011bli riskovat b\u011bhem dne jak\u00fdkoliv probl\u00e9m.<\/p>\n\n\n\n<p>Vzhledem k okolnostem jsme byli donuceni zapnout \u010cDT a t\u00edm dopad \u00fatoku sn\u00ed\u017eit, proto\u017ee se v\u0161e rozd\u011blilo mezi dv\u011b trasy. B\u011bhem zm\u011bn byl v\u00fdpadek s\u00edt\u011b cca 3 minuty a cca 5 minut byla v\u00fdrazn\u011b sn\u00ed\u017een\u00e1 dostupnost a cca 15 minut byla dostupnost hor\u0161\u00ed (se ztr\u00e1tovost\u00ed z n\u011bkter\u00fdch sm\u011br\u016f). Situaci je\u0161t\u011b \u0159e\u0161\u00edme a z n\u011bkter\u00fdch lokalit (ze sv\u011bta) je\u0161t\u011b st\u00e1le prob\u00edh\u00e1 siln\u00fd \u00fatok, kter\u00fd ji\u017e \u0159e\u0161\u00ed dodavatel\u00e9 tras &#8222;nad n\u00e1mi&#8220;, ale m\u00e1 to za p\u0159\u00ed\u010dinu to, \u017ee z n\u011bkter\u00fdch sm\u011br\u016f je ztr\u00e1tovost paket\u016f.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">St\u00e1le se opakuj\u00edc\u00ed DDoS \u00fatoky<\/h2>\n\n\n\n<p>Od po\u010d\u00e1tku na\u0161\u00ed \u010dinnosti pravideln\u011b informujeme o DDoS \u00fatoc\u00edch, kter\u00e9 jsou vedeny proti na\u0161\u00ed spole\u010dnosti. V posledn\u00ed dob\u011b jsme informovali o rostouc\u00ed \u010detnosti a zvy\u0161uj\u00edc\u00ed se intenzit\u011b \u00fatok\u016f.&nbsp;<strong>V posledn\u00edch t\u00fddnech skute\u010dn\u011b proch\u00e1z\u00edme velkou zat\u011b\u017ek\u00e1vac\u00ed zkou\u0161kou na\u0161\u00ed s\u00edt\u011b.&nbsp;<\/strong><\/p>\n\n\n\n<p>V\u011b\u0159te n\u00e1m, \u017ee celou v\u011bc \u0159e\u0161\u00edme celou dobu s maxim\u00e1ln\u00edm z\u00e1jmem.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">S\u00edla roste<\/h3>\n\n\n\n<p>Kdy\u017e jsme loni informovali o \u00fatoc\u00edch, kter\u00e9 m\u011bly 2-3 Gbps, tak se n\u00e1m to zd\u00e1lo dost. V zim\u011b u\u017e to bylo 6 Gbps. Na ja\u0159e jsme p\u0159ekonali 10 Gbps a nyn\u00ed v l\u00e9t\u011b u\u017e m\u00e1me rekordy posunut\u00e9 k 20 Gbps. Dvakr\u00e1t jsme ji\u017e zachytili \u00fatoky, kter\u00e9 byly del\u0161\u00ed dobu p\u0159es 17 Gbps a kr\u00e1tkodob\u011b je\u0161t\u011b v\u00fd\u0161e. To jsou ji\u017e hodnoty, kter\u00e9 v \u010cR nejsou v\u016fbec obvykl\u00e9 a z\u0159ejm\u011b mo\u017en\u00e1 ani nikdy jinde nebyly&#8230;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Co se d\u011bje p\u0159i DDoS<\/h3>\n\n\n\n<p>Existuj\u00ed r\u016fzn\u00e9 druhy DDoS \u00fatok\u016f. N\u011bkter\u00e9 jsou proti jednotliv\u00fdm slu\u017eb\u00e1m, jin\u00e9 proti na\u0161im router\u016fm, jin\u00e9 proti na\u0161emu hlavn\u00edmu webu a jin\u00e9 proti na\u0161im IP rozsah\u016fm. Ka\u017ed\u00fd je jin\u00fd. Nelze to n\u011bjak jednodu\u0161e pau\u0161alizovat, ale v ka\u017ed\u00e9m p\u0159\u00edpad\u011b jde o to, \u017ee se \u00fato\u010dn\u00edk, respektive jeho po\u010d\u00edta\u010de (je\u017e ovl\u00e1d\u00e1) sna\u017e\u00ed na\u0161i s\u00ed\u0165 (nebo n\u011bkter\u00e9 slu\u017eby) znefunk\u010dnit a vy\u0159adit z provozu. Na na\u0161e routery nebo servery se doslova val\u00ed miliony paket\u016f za sekundu, kter\u00e9 ned\u011blaj\u00ed nic u\u017eite\u010dn\u00e9ho, ale jen zat\u011b\u017euj\u00ed infrastrukturu nebo ub\u00edraj\u00ed kapacitu slu\u017eb\u00e1m, kter\u00e9 ji potom nemohou m\u00edt pro sebe.<\/p>\n\n\n\n<p>N\u011bkdy dojde k tomu, \u017ee server ji\u017e nem\u00e1 voln\u00e9 kapacity na p\u0159ipojen\u00ed re\u00e1ln\u00fdch klient\u016f, proto\u017ee jej p\u0159edt\u00edm &#8222;\u017e\u00e1dalo&#8220; o spojen\u00ed n\u011bkolik milion\u016f jin\u00fdch po\u010d\u00edta\u010d\u016f b\u011bhem n\u011bkolika sekund. Jindy dojde k ucp\u00e1n\u00ed switche, respektive p\u0159\u00edpojky serveru. Jednotliv\u00e9 koncov\u00e9 servery jsou p\u0159ipojeny 1 Gbps a kdy\u017e je \u00fatok m\u00ed\u0159en na jeden konkr\u00e9tn\u00ed server, tak p\u0159i p\u0159ekro\u010den\u00ed 1 Gbps je konektivita serveru prakticky vy\u0159azena z provozu.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Co s t\u00edm?<\/h3>\n\n\n\n<p>Proti DDoS \u00fatok\u016fm se t\u011b\u017eko br\u00e1n\u00ed. Moc t\u011b\u017eko. N\u011bkter\u00e9 lze vy\u0159e\u0161it snadno, jin\u00e9 h\u016f\u0159e.<\/p>\n\n\n\n<p>My jsme prakticky v\u017edy DDoS zvl\u00e1dali s \u00fasm\u011bvem. Vzhledem k tomu, \u017ee v\u0161ak s\u00edla roste, tak jsme ji\u017e m\u011bli 2 men\u0161\u00ed probl\u00e9my a dnes t\u0159et\u00ed (ten by asi nenastal za p\u0159edpokladu, \u017ee bychom m\u011bli peering s \u010cDT).<\/p>\n\n\n\n<p>P\u0159ipravujeme z\u00e1sadn\u00ed zm\u011bny a \u00fapravy tak, aby v\u0161e fungovalo jak m\u00e1 a to i b\u011bhem DDoS \u00fatok\u016f.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Jak se br\u00e1n\u00edme nyn\u00ed?<\/h3>\n\n\n\n<p>Ono je nebezpe\u010dn\u00e9 n\u011bco prozrazovat, proto\u017ee bychom t\u00edm mohli \u00fato\u010dn\u00edk\u016fm hodn\u011b pomoci.<\/p>\n\n\n\n<p>Nyn\u00ed m\u00e1me nastaven\u00e1 r\u016fzn\u00e1 pravidla na na\u0161ich routerech a serverech. V\u0161e monitorujeme a prakticky hned (v jednotk\u00e1ch sekund). S na\u0161imi dodavateli konektivity jsme ve spojen\u00ed a ka\u017ed\u00fd \u00fatok spole\u010dn\u011b hned \u0159e\u0161\u00edme. Z\u00e1rove\u0148 na\u0161e konektivita je nyn\u00ed prim\u00e1rn\u011b p\u0159esm\u011brovan\u00e1 p\u0159es dodavatele, kter\u00fd m\u00e1 DDoS ochranu, ale ka\u017ed\u00e1 ochrana vyhodnot\u00ed \u00fatok a\u017e po ur\u010dit\u00e9 dob\u011b a z\u00e1rove\u0148 ka\u017ed\u00fd \u00fatok je jin\u00fd a tak ne v\u0161e funguje jak m\u00e1. Probl\u00e9m je nav\u00edc v tom, \u017ee \u010d\u00e1st konektivity se k n\u00e1m dostane p\u0159es druhou s\u00ed\u0165 a jin\u00e9 poskytovatele. Dal\u0161\u00ed a mnohem v\u00fdznamn\u011bj\u0161\u00ed probl\u00e9m je ten, \u017ee pokud jsou tak siln\u00e9 \u00fatoky, tak dojde (doslova) k &#8222;ucp\u00e1n\u00ed&#8220; tras u poskytovatel\u016f a to i v\u010detn\u011b poskytovatel\u016f &#8222;nad n\u00e1mi&#8220;. T\u00edm \u010d\u00e1st tras jde (nap\u0159\u00edklad z \u010cR nebo SR) a t\u0159eba Evropa, ale \u010d\u00e1st do z\u00e1mo\u0159\u00ed nebo do Asie nejde. Jindy jsou \u00fatoky t\u0159eba z Ji\u017en\u00ed Ameriky a tak jsou ucpan\u00e9 trasy tam.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Co chyst\u00e1me<\/h2>\n\n\n\n<p>Chyst\u00e1me zcela z\u00e1sadn\u00ed \u00fapravy s\u00edt\u011b. Vym\u011bn\u00edme hrani\u010dn\u00ed routery, kter\u00e9 budou m\u00edt mnohon\u00e1sobn\u011b vy\u0161\u0161\u00ed kapacity a budou m\u00edt i porty 40 Gbps. Celkov\u00e1 routovac\u00ed kapacita na hranici mezi n\u00e1mi a internetem bude 2,56 Tbps a propustnost paket\u016f&nbsp;1,904 tera paket\u016f za sekundu.&nbsp;Z\u00e1rove\u0148 nyn\u00ed z\u00edtra chceme spustit komer\u010dn\u00ed detekci DDoS \u00fatok\u016f, kterou chceme zakoupit a pokud testy dopadnou dob\u0159e, tak ji po\u0159\u00edd\u00edme (respektive si ji\u017e nech\u00e1me).<\/p>\n\n\n\n<p>Dnes jsme jednali s dodavatelem, kter\u00fd m\u00e1 Arbor \u0159e\u0161en\u00ed a p\u0159es n\u011bho\u017e n\u00e1m te\u010de zahrani\u010dn\u00ed konektivita ji\u017e nyn\u00ed o tom, \u017ee bychom spolupr\u00e1ci roz\u0161\u00ed\u0159ili a detekci zp\u0159\u00edsnili a t\u00edm v\u00fdrazn\u011b zlep\u0161ili ochranu na\u0161\u00ed s\u00edt\u011b.<\/p>\n\n\n\n<p>Z\u00e1rove\u0148 \u0159e\u0161\u00edme dal\u0161\u00ed optickou trasu a t\u00edm bychom celkovou kapacitu nav\u00fd\u0161ili na 40 Gbps.<\/p>\n\n\n\n<p>V nejbli\u017e\u0161\u00edch dnech chceme celou na\u0161i infrastrukturu p\u0159ed\u011blat. Tento krok u\u017e p\u0159ipravujeme del\u0161\u00ed dobu a ne\u0161el urychlit.<\/p>\n\n\n\n<p>Detekci \u00fatok\u016f, kterou m\u00e1me nyn\u00ed na Hlubok\u00e9 d\u00e1me p\u0159\u00edmo do Prahy. T\u00edm bychom m\u011bli zabr\u00e1nit tomu, \u017ee trasy k n\u00e1m budou doslova ucpan\u00e9.<\/p>\n\n\n\n<p>Jen pro informaci dod\u00e1v\u00e1me, \u017ee aktu\u00e1ln\u00ed situace pro n\u00e1s m\u00e1 ve v\u0161ech sm\u011brech prioritu a celkov\u00e1 investice v korun\u00e1ch bude m\u00edt sedm m\u00edst. Ka\u017ed\u00fd m\u011bs\u00edc to nav\u00edc pro n\u00e1s bude znamenat n\u00e1klady ve stovk\u00e1ch tis\u00edc korun za licence a za komer\u010dn\u00ed filtrov\u00e1n\u00ed. V<strong>\u011b\u0159\u00edme, \u017ee to pom\u016f\u017ee.<\/strong><\/p>\n\n\n\n<p>Bohu\u017eel to nejde ud\u011blat rychleji. Nap\u0159\u00edklad dod\u00e1n\u00ed hardwaru trv\u00e1 n\u011bkolik t\u00fddn\u016f a p\u0159\u00edpravy a v\u0161e souvisej\u00edc\u00ed nen\u00ed na jedno odpoledne. &nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">P\u0159ipravte se, startujeme<\/h2>\n\n\n\n<p>Ve \u010dtvrtek k n\u00e1m doraz\u00ed nov\u00fd hardware na \u00fapravy s\u00edt\u011b. Hned se na to vrhneme. Postupn\u011b to budeme m\u011bnit. Za t\u00fdden budeme m\u00edt hotovou v\u00fdraznou \u010d\u00e1st \u00faprav, ale &nbsp;odhadujeme, \u017ee to bude trvat cca m\u011bs\u00edc, ne\u017e bude kompletn\u011b v\u0161e hotovo.&nbsp;<\/p>\n\n\n\n<p>V\u0161e chceme d\u011blat v noci a bez v\u00fdpadk\u016f, ale jedn\u00e1 se o velmi z\u00e1sadn\u00ed \u00fapravy s\u00edt\u011b, \u017ee m\u016f\u017ee doj\u00edt k drobn\u00fdm a velmi kr\u00e1tkodob\u00fdm nedostupnostem (b\u011bhem noci). O v\u0161em budeme pr\u016fb\u011b\u017en\u011b informovat na na\u0161em webu.<\/p>\n\n\n\n<p><strong>Ud\u011bl\u00e1me velk\u00e9 zm\u011bny, velk\u00e9 \u00fapravy. Ji\u017e za p\u00e1r dn\u00ed za\u010d\u00edn\u00e1me&#8230;<\/strong><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Z\u00e1v\u011brem&#8230;<\/h2>\n\n\n\n<p><strong>V\u0161em klient\u016fm se velmi, ale velmi omlouv\u00e1me a na\u0161\u00ed prioritou je ud\u011blat v\u00fd\u0161e popsan\u00e9 kroky a t\u00edm omezit dopad dal\u0161\u00edch podobn\u00fdch komplikac\u00ed.<\/strong><\/p>\n\n\n\n<p>V\u011b\u0159\u00edme, \u017ee jsme v posledn\u00edch t\u00fddnech jasn\u011b uk\u00e1zali, \u017ee jsme v\u00fdrazn\u011b zlep\u0161ili krizovou komunikaci v podobn\u00fdch situac\u00edch.<\/p>\n\n\n\n<p>Je vid\u011bt, \u017ee n\u011bkomu stra\u0161n\u011b moc vad\u00edme, proto\u017ee po\u0159\u00e1dat na n\u00e1s takov\u00fd &#8222;hon&#8220; nen\u00ed zadarmo a u\u017e to mus\u00ed n\u011bkomu v\u00e1\u017en\u011b st\u00e1t za to&#8230;<\/p>\n\n\n\n<p><strong>Op\u011bt lze na z\u00e1v\u011br jen napsat, \u017ee co n\u00e1s nezabije, tak n\u00e1s pos\u00edl\u00ed.<\/strong>&nbsp;Stejn\u011b jako jarn\u00ed komplikace s elektronap\u00e1jen\u00edm, kter\u00e9 jsme vy\u0159e\u0161ili, tak i tohle vy\u0159e\u0161\u00edme. Mimochodem jen dod\u00e1me, \u017ee za p\u00e1r dn\u00ed k n\u00e1m doraz\u00ed druh\u00fd mortorgener\u00e1tor, kter\u00fd je celou dobu objedn\u00e1n a \u010dek\u00e1me na to, a\u017e bude dopraven na m\u00edsto.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Dovolte, abychom informovali o tom, co se u n\u00e1s aktu\u00e1ln\u011b d\u011bje a co p\u0159ipravujeme (z hlediska s\u00edt\u011b).<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,101],"tags":[],"class_list":["post-55053","post","type-post","status-publish","format-standard","hentry","category-spolecnost","category-udalosti"],"_links":{"self":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts\/55053","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/comments?post=55053"}],"version-history":[{"count":1,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts\/55053\/revisions"}],"predecessor-version":[{"id":55056,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/posts\/55053\/revisions\/55056"}],"wp:attachment":[{"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/media?parent=55053"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/categories?post=55053"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.wedos.com\/cs\/wp-json\/wp\/v2\/tags?post=55053"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}