The basis for quality Internet services is good connectivity, i.e. connecting our network from the data centre to the Internet, and a properly set up backbone network that can cope with failures and outages.
Connecting our servers to the rest of the world involves several sub-problems:
- Build a backbone network in the datacenter itself
- Negotiate data connections from the data centre to places where it is possible to connect to other networks and further to the Internet
- Arrange with other networks for interconnection and find one or more telecom operators to provide connectivity to the rest of the Internet
Backbone network and routers
The entire backbone network will be based on Ethernet, all connections between routers will initially be 1 Gbps, but by easily upgrading switches and cabling it will be possible to quickly reach 10 Gbps Ethernet.
All our routers will be based on server hardware with Intel Xeon X5650 processor and 6 GB RAM (DDR3 1333MHz) with a number of metallic and optical Ethernet interfaces. The operating system will be Linux with kernel 2.6. As routing software we chose BIRD (more information in a future article).
All routers will be interconnected in a double star. All active elements (routers, switches) will have redundant power supplies, connected to 2 independent power branches. In the future, we also plan mutual substitutability of end routers (gateway for servers) using one of the fail-over protocols (VRRP, CARP).
Our datacentre in Hluboká nad Vltavou will be connected via a fibre optic route from Telefónica O2 to the SITEL telehouse in Prague. The connectivity will be very luxurious, because the O2 fibre optic backbone network connecting South Bohemia with Prague runs directly through our building. This means not only a high-quality connection with minimal latency (response time), but also a guarantee of fast problem solving in case of a malfunction. Due to the capacity of the O2 fiber optic routes from our building, the possible connectivity is basically unlimited, initially we will use a 1 Gbps connection, but we have already arranged an easy upgrade to a 10 Gbps connection, but there can be more connections and they can be combined freely.
As a backup route, we will either use another line from O2, which, however, will physically lead through a different route, or we will use a completely different operator, whose equipment we have to get to in some complicated way. We can dig our own fibre optic line from the centre to the outskirts of Hluboká (which is expensive, but the result will be very good, almost unlimited capacity) or we can create a wireless connection (faster and cheaper solution, but very limited capacity, completely inadequate for the future). No decision has yet been made on this matter, in particular we are dealing with the extent to which the second O2 fibre-optic route is independent of the first one. The backup link may end up in Prague in another telehouse.
Inside our autonomous system, the OSPF routing protocol will be used (without splitting into multiple areas).
Peering
The aim of the connection from our datacenter to the telehouse in Prague is of course the possibility to connect with other networks, the concentration of which is probably the highest in the SITEL telehouse.
Here we connect to NIX, which is a Czech peering node where it is possible to connect directly to all Czech networks and ISPs and some foreign ones(www.nix.cz). This is probably where most of our data will flow. We plan to become members of NIX during the month of August.
Of course, we will also connect to at least one transit operator on SITEL, which will connect our network to the rest of the Internet. Transit operators have been discussed and a decision will be made in August.
One of the 2 main routers will be placed on SITEL, and it will use the BGP routing protocol to connect to other networks. The second main router will end up in another telehouse in Prague, where it will be connected again to NIX and to some other transit operator.
The entire network will be configured using OSPF and BGP protocols so that it will not be affected by the failure of any part of it or any device.