As we promised last week, we are doing just that. Connectivity is crucial for any datacenter, so we set about improving our 100 Gbps routes. At the same time, our technicians prepare a set of regular sharp tests to detect any problem.
Route 3 needs to be improved and offer more options
Route 3, which also has a capacity of 100 Gbps, leads physically from Hluboká Nad Vltavou through Havlíčkův Brod to Prague. Specifically, to the T-Mobile – GTS (THP) datacenter from where we are connected to the Žižkov tower of the Czech Radio. We are running this route as a backup.
We have decided to make a change that will offer us far greater potential in the long term. That is why the management has been negotiating with ČD – Telematika for many months that we could use their new datacentre (called U2). However, due to recent events, we have decided to accelerate the negotiations.
Just for completeness, we have 2 more routes (each 100 Gbps) from CETIN, which lead through 2 different routes (one through Tábor and the other through Písek) to the Sitel datacenter (Ce-Colo).
We are connected to 4 different providers in Prague on Sitel and can add more at any time. We had only one provider on the back-up route so far, which backfired on us, and so a change had to come. We already have 2 providers there and we’re arranging more. In addition, we’re making further improvements…
What’s going on?
The current backup route was:
DC1 WEDOS Hluboká ⟶ THP ⟶ Žižkovská věž ČRa.
We decided to modify it and insert the switch into ČD – Telematika.
DC1 WEDOS Hluboká ⟶ switch ČDT (U2) ⟶ TTC.
This will allow us to use the potential of the CTD datacenter, where major ISPs have their connectivity. But more on that later. We are currently connected to the CTD network from Friday (29.11.2019).
Next week, we will be connected by leased fibre optics, which will initially be planted at 100 Gbps, to the TTC data centre where other major operators and peering nodes are located. This final status will be completed next week as we await the measurement of the leased fibre optic. In the meantime, our route:
DC1 WEDOS ⟶ ČDT (U2) ⟶ THP ⟶ Žižkovská věž ČRa
Anything goes if you want it to.
We have been preparing the whole change for more than half a year, but we were still waiting for something or someone. We have had a lot of intense negotiations with CTD over the last 11 days and during this week everything was settled. We were a bit worried that it would be a problem to get a suitable 100 Gbps switch from the suppliers, because such hardware is not usually in stock. Who in the Czech Republic needs 100 Gbps branded hardware from one day to the next, right? Except for WEDOS 🙂
However, we were lucky and one supplier had it in stock and was also able to deliver it to us the very next day. Another option was to buy the switch from a supplier in Germany who can deliver it within a few hours (or we can drive to Munich to get it ourselves).
Just for the record. Of course, we have various spare 40 Gbps and 10 Gbps switches in stock, because we use these on a regular basis. The other elements in the network that we have 100 Gbps are in the network dually. The switches we use have 6 x 100 Gbps ports. In Prague we now need a switch with 3 ports.
Preparation, configuration and arranging details took a day. The new DELL switch has 3 100 Gbps ports and 12 10 Gbps ports.
We prepared 100 Gbps QSFP28 modules and 10 Gbps (SFP+), had lunch and nothing prevented us from going to Prague on Friday.
The switch serves only as an “aggregator” for individual connectivity from individual connectivity providers and we send this connectivity to Hluboká using vlanes. There we route everything using Arista devices and filter it using our DDoS protection. But more on that next time. Just for the squeamish, we use the full BGP table. We don’t even rouse on linux… But we’ll write about that next time too.
Journey to Prague and installation
On Friday in the afternoon we went to Prague. We haven’t been to the CTD datacenter yet. So we took a few packs of WEDOS Energy to remember us by 🙂 We have been in contact with some of them for 15 years and we have never met in person…
We have to mention that we were well taken care of at CTD and the staff who helped us with the installation made a very good professional and human impression on us.
The installation of the switch itself was not difficult. After that we made some fiber optic cable reconnections and all we had to do was to arrange by phone to switch BGP routing and test if everything works. That took the most time.
In about two hours everything was ready. The final details will be finalized on Monday. Uh, like the last of the paperwork and access clearance for our staff…
What the location of the switch in the ČD – Telematika data centre means
It seems like a small thing. After all, Route 3 is a backup and virtually no traffic flows through it. Primarily, the most important thing now is that everything works even if there is a problem on the two primary routes lasting several days (from Hluboká to the SITEL (Ce Colo) data centre).
In the long term, the switch in ČDT will allow us to connect directly to other ISPs in the ČDT datacenter or through them to other datacenters in Prague. It simply serves as a signpost, for example to connect to some peering (NIX, Peering.cz). It is therefore a strategic element in our network infrastructure.
The consideration of a direct connection to a major ISP is also interesting. Their customers would then have direct access to, for example, more than 210 thousand .cz domains hosted at Hluboká. In a couple of weeks we will launch our second datacenter and we believe that the number of domains hosted by us will grow rapidly upwards.
Things have thus moved forward and we can move forward in the negotiations on Route 4 through České Budějovice. We have the optics ready and are fine-tuning the last details with CTD. Now we need to start arranging cooperation with other ISPs in České Budějovice who have connectivity there. Our goal is to be able to avoid Prague via Route 4. If something serious were to happen there, we need our customers’ services to continue to work, even if Prague has a major outage. So in CB we will now choose connections abroad. It will be a backup, but we don’t want to underestimate anything.