Today, we’ll explain a few more details about TIER IV to make it clear what all you need to meet to get the internationally recognized TIER IV certification.
In the last installment of this series, we discussed what the term TIER is in general.
Fairy tales or facts
Today we will look at the various issues that you need to meet in order to obtain the relevant TIER certification. In various discussion forums you will come across many different “tales” where some individuals explain what a datacenter for TIER must and must not meet. Very often this information is fabricated and often comes from a “one lady said” agency. Our claims are based on official documents and materials from the Uptime Institute.
Each certification is based on several variables and the respective certification is issued for a specific datacenter, which is characterized primarily by its size (in square meters) and the power consumption that the datacenter must handle (which it is built to handle, which is in kW).
Since we want to focus on TIER IV in our certification, we will focus our texts primarily on this level. Unless some information is more precise, the information refers to this level – TIER IV. Firstly, we have to meet everything required for TIER I or TIER II or TIER III, but we also have to meet a few extra criteria.
TIER IV
One very important feature of TIER IV is “Fault Tolerant” or tolerance to errors. This feature extends “Concurrently Maintainable” from TIER III so that critical environments must continue to operate automatically due to unplanned outages. Even from cumulative outages. Yes, everything must be fully automatic.
For a TIER IV datacenter, you must ensure that everything is provisioned many times (at least 2 times) to full capacity (100% of the datacenter’s value). For such a datacenter we speak of “branch” A and branch “B”. All multiple systems (e.g. power, cooling, connectivity) must be completely independent and so one part must not be dependent on another. All redundant elements must not only be independent, but also physically separated (this is not only about physical separation, but also about fire safety and protection against the transmission of an accident from one system to another).
All (and ancillary) systems and distribution routes must be physically separated from all others to avoid mutual influences and impacts.
Each branch (power, cooling, connectivity) must have a capacity of at least N (so the entire datacenter is 2N or 2N+1 or 2N+2).
Any component may malfunction or be serviced and there must be no change in system operation or failure.
One flaw = everything wrong
If you want to achieve certification to the appropriate level, you must meet all the requirements. If you fail to meet any of the requirements, you can never get the grade.
This is essential because if you make one mistake in the design, the whole investment is lost. Just the mistakes that are quite common and we will cover them in the next parts of the series. However, there are also errors that are much less common and obvious, but even these mean that you will not achieve the relevant TIER.
Human errors
All systems must work automatically and eliminate human error. At first glance, you might think it’s easy, but datacentres are full of relatively complex systems and the operators need to be properly trained. In TIER IV, human error is avoided by ensuring that everything must be fully automatic and operate without human assistance. So the human element is basically just an oversight.
If we look at some historical events of problems and failures in datacenters, we still end up with the human factor failing. Sometimes the design is underestimated, sometimes the service, and sometimes regular inspection and testing is needed. As you already know from our story, we have been preparing the proposal for several years. Almost 2 years before construction and now more than a year during construction. We have been in contact with Uptime Institute for almost 2 years and we still have a lot of tuning to do.
Physical location of the datacenter
The physical location is also assessed as part of the certification. Believe me, being near a highway or major railroad is a problem. Nearby means everything within 1.5 miles, or about 2.4 kilometers. Try to find such a location in Prague. And you mustn’t even be in the way of the aircraft when you mustn’t be in the air corridor. However, if you look at where planes fly to Prague airport, there are not many locations for TIER IV datacenters in Prague. If you look at how the Czech Republic is divided by air corridors, you will find that there are really few locations in the Czech Republic where planes do not fly over datacenters. In all this, our location in Hluboká nad Vltavou is absolutely ideal.
Highways, railways or airplanes are not the only threats. The site is also being addressed in terms of seismic tremors or perhaps volcanoes. Fortunately, we don’t have to deal with this much in the Czech Republic. Nor do we have to deal with tornadoes and other such things (tropical or snow storms). We are spared that.
Believe that TIER certification also addresses flood zones and flood risks. We have therefore also thoroughly addressed this. The datacenter must not be in an area where there is a risk of flooding – for example, 100-year water. Our current datacentre is in Hluboká nad Vltavou. Tens of metres above the surface of any body of water. We are building a new datacentre in a different location, which looks different at first glance, but we have taken everything into account. We are building a new datacenter in a location where a flood occurred in 2002 on a part of the neighbouring property. There is a hydroelectric power station in the neighbourhood (whose electricity we will use for operation). The lowest point of our new building is more than 1 metre above the millennium water level and is the part of our office building where we have a lecture room. Our office is about 4.5 meters higher. The data hall is an adjacent building that is located higher up and is largely buried underground. The data hall is made of monolithic concrete and, as already mentioned, it is underground and thus secured not only against pressure but also against any water. So if the water rises by many, many metres, we are in no danger. The water would have to rise by about 7 metres (compared to the millennium water), which is a huge amount of water in the České Budějovice basin, and in that case WEDOS will not be interested in anyone, because in ČB there will be almost 9 metres of water on the square. The entire data hall is a self-contained unit, even with the motor-generators being inside, and we have secured access routes over the castle hill (60 meters high) for supplies (such as diesel). We also own the adjacent property. At the same time, we are able to operate the services in the current datacentre. We will describe it in the next parts of the series. TIER conditions have no problem with the location of our datacenters.
The surrounding area is also assessed for the site. There must be no hazards such as factories or chemical plants or plants where there is a risk of fire or explosion.
You can’t even certify a datacenter that is not in a dedicated building. So you can’t have a datacenter that is part of, say, an office or manufacturing complex. It must be a single-purpose building.
Physical security
This is for a separate chapter, and maybe we will deal with this topic some other time. It is certainly wrong if you do not have a data hall secured against someone “braking” badly in front of the building and “accidentally” driving through to the servers.
Our strength is that the current datacenter is a civil defense shelter, largely hidden underground. It is a monolithic structure with a concrete thickness of 45 centimetres. It was actually built by “comrades” in 1976 and it will only jump in the event of a nuclear explosion.
The new datacentre is again a monolithic building. The strength of the concrete (the highest grade) is 30 centimetres and it is a monolith and not some jigsaw of panels. Towards the traffic arteries there are additional protective walls and in total we have walls up to 110 centimetres thick! This data hall is also partly underground.
The door? In the new data centre we will have a real vault door protecting access to the secure areas and another vault door protecting the entrance to the data hall. We manufacture real safe doors with certification. A door that weighs hundreds of kilograms and you can’t just open it and you have to invite a “cashier” to do it. In our data centre you will also find security bars on the windows or 5 layers of special security films and window frames with increased security. That’s how you protect data. Your data. We have no other entrances to the data rooms or access for servicing or moving technology.
Cameras, access systems must be a matter of course. We use RFID cards for entry. In the new datacentre, we plan to authorise people via a bloodstream reader, which is currently considered one of the most secure solutions. In the new datacentre, we want to use a system where the entry to the datacentre will have to be authorised by 2 people at the same time – one directly in the datacentre and the other outside the datacentre. What if we were ambushed 🙂
Physical wiring and cabling
The two supply branches A and B must therefore be physically separated. For example, it is not possible to have the power cables under the double floor for branch A and branch B at the same time. This will be discussed in detail in the next parts of the series.
Interestingly, the power branches must not cross in the same space.
Both supply branches must be active simultaneously and each must be able to provide 100% operation in the event of a failure of the other. It follows that you must never have a load anywhere (in normal operation) greater than 50%.
Connectivity
Importantly, connectivity to the datacenter must also be ensured that it is physically separated. For example, the overlapping of optical routes or even the entry of optical routes into the datacenter is completely unacceptable. It MUST NOT be in one area of the building or even on one side of the building. Fiber optic routes MUST enter the building from different sides of the building, must not cross anywhere, and must not coincide anywhere.
In the building, we also design the optical routes so that one branch is physically separated from the other without crossing and concurrence.
You might be surprised how many new datacenters are being built with multiple fiber optic protectors in one trench. We saw it with our own eyes at the newly built datacenter. Similarly, we saw that the fiber optics were running along one side of the road and the other route on the other side of the road, which is obviously better than having it in one trench. But to our surprise, the two routes then met at the same junction and are now run together in one sub-route. Then it only takes one accident, or one “clever digger” and the whole datacenter is without connectivity.
How do we want to have it in our country? We currently have 2 routes to the first datacenter from Cetin (O2) because our datacenter is on their backbone. Routes do not have concurrency (except for the building). In front of the building, they divide into two different directions. This is what we know that TIER IV does not meet (the building is accessed by fiber optic routes from only one side of the building). So for several years now we have had a third route that leads completely differently and ends at the CTD. When we finish the new data centre, the situation will be a little different. The existing datacenter will have 2 independent routes from Cetin and 2 completely independent and physically separate routes to the new datacenter, where it will physically terminate from CTD. In the existing datacenter we will have 4 independent and separate routes and in the new one there will be 3 independent and separate routes (we are already planning a fourth one). Routes will be non-contiguous and will “enter” the buildings each from a different side of the building.
After the launch of the new datacenter we want to have 3 routes to Prague, each 100 Gbps (we will have 3 x 100 Gbps) and one backup route to Slovakia (to avoid problems in Prague). The route to Slovakia will not go via Prague, but physically via a completely different route. The ending in Prague will be as before – in 3 different locations, in 3 different datacentres.
Which of you dare?
Who dares to design a datacenter like this? I have only stated the basic assumptions. Already think about how the wiring diagram should look like, since nothing must cross anywhere… 🙂
What can you look forward to next?
Next time we will write down more information and details. We’ll also reveal some details from behind the scenes. Finally, you will also learn about the most common errors or the economics of operating such a datacenter. The economics of the operation will be on a separate chapter.
Since we enjoy TIER IV certification and take it as one of our biggest challenges, next time we will discuss in detail some of the technical prerequisites that a TIER IV datacenter must meet.
He will surely get to real examples that will help you understand why no datacenter in the Czech Republic (or elsewhere in Central Europe – including Germany, Poland, Hungary, Slovakia, Austria) has TIER IV certification yet. If you have any questions, please write to us and we can take the answers into account in future articles.
Why are we doing all this?
There are several reasons. We are the largest hosting company in the Czech Republic, we host the most services in our datacenter in the Czech Republic and so we are aware that we must aim everything to the maximum satisfaction of our clients. The quality of the datacenter is clearly a very important factor for our further growth and development and without a quality background it will definitely not work. )
That’s why we want to have 2 modern datacentres that will meet the most demanding criteria and we want to have the whole thing certified in this way. We are not going down the road of someone drawing something somewhere and someone somewhere quickly building it. The way we go about it is that we spend a huge amount of time preparing, we’ve been preparing for several years with a team of several people. We keep an eye on everything on the construction site and make decisions promptly. As a result, we want to handle all operational matters “in house”. For example, we have our own electricians (even 2). So we know everything in our building. We can figure it all out on our own. As quickly and as best as possible.
Both datacentres will appear as one on the outside, but in reality they will be two separate buildings that will be able to operate completely separately or complement each other in operation. It’s going to be like a RAID datacenter.
Yes, none of this makes sense economically, but we do everything we can to ensure maximum satisfaction for our clients. That is the primary objective. We do it because we enjoy it. The economic side is a secondary issue and as we say: “If the clients are happy, the profits will come”. We are a strong company that runs without credit or debt. We own the datacenter, the infrastructure and we don’t have to “report to a bank”. We are building a second datacentre for tens of millions of crowns, again without any help from third parties. Maybe you could keep the money and buy something “for fun”, but we enjoy this and we enjoy happy clients.
The planned certifications should be a guarantee to our clients that their data is in the best place and well taken care of.
Stay tuned to see how our plans are coming along.