In April, our main priority was to work on WEDOS Global and individual services. This is the largest and most costly project in the history of WEDOS, so the priorities of the individual departments correspond to this.
How we have progressed with WEDOS Global
Building the infrastructure itself is very complicated, and in addition to the technical issues, there are also a number of commercial, legal and official matters to deal with. In the first phase, we plan to build a network based on 40 to 50 points. At least 45 physical servers + 2 switches with 100 Gbps connectivity at each point.
In the next phase, we planned to start arranging peering with major local connectivity providers in individual locations and debug routing (where the traffic flows). Which takes the entire WEDOS Global network to a whole new level.
What is peering?
Peering is a direct connection between two networks. Hosting providers use peering to connect to other networks and can provide better connectivity to their customers. Direct peering between two providers allows to reduce the response time, improve stability and increase the data transfer rate.
At the end of March, we started using professional third-party tools to monitor and compare anycast networks. We wanted to know how we stack up against our global competitors. With 23 points, in 18 countries on 5 continents, we are in the Top 20 in North America and Top 10 in Europe, not bad numbers.
The overall ranking is based on measurement data from hundreds of sites around the world. Each location has an impact on the outcome. So we downloaded the measurements from all the sites, evaluated them and started thinking where to put the next points. We have found that for Europe and the US it is no longer about additional points, but just about peering and routing. Of course we need more points to get better results from the world, but it’s not as high a priority as we thought.
In April, a decision was made not to wait for phase two and to start addressing both peering and routing at selected points. For now, at the IXP (Internet Exchange Point) level.
What is an IXP (Internet Exchange Point)?
An IXP (Internet Exchange Point) is a physical place where different networks are connected. Network operators can connect and exchange data. This reduces latency and increases the speed of data transfer between their networks. It’s basically the crossroads of the local internet.
We were the first to arrange the largest IXP in Bulgaria – Bulgarian Internet eXchange (BIX.bg), which connects the largest and most important local connectivity providers. In this area, we have seen a very significant improvement in response rates and WEDOS Global has the second best response rate in the world for anycast networks.
Of course, this also had an impact on WEDOS Global’s ranking in Europe. We have confirmed that we are on the right track and it is realistic for us to gradually get among the top 10 in the world.
Theoretically, it would be possible to get into the top 5 with our solution, budget and plan in the long term, but there are locations where it is difficult to place 45 physical servers with 100 Gbps connection. For example, we have been dealing with points in Mexico and Turkey for several months. The servers there have literally “survived their own death”. But those are stories more for the after party after a conference here 🙂
We are also tuning routing with China Telecom not only in Asia. They noticed us and approached us directly at CloudFest in Germany, where we had a two-storey booth. We arrange for everything to go correctly to the nearest points in Asia and not send it to us via European points.
WEDOS Global Protection
WEDOS Global Protection is essentially working on two fronts. The development of the WEDOS Global customer administration and associated systems continues. We’d like to “stack” everything we have elsewhere to a reasonable extent, but the user might need it “within reach”. For example, linking with WEDOS OnLine, domain management and registration, etc.
At the same time, the development and tuning of protections continues. Last month we deployed a universal template to protect not only WordPress websites. Most users are happy because everything is faster and they no longer have to deal with DDoS protection, vulnerability scanning, etc. Unfortunately, we’ve also come across cases where someone is using a not-quite-appropriate solution (developers sometimes look for shortcuts that don’t quite follow WordPress standards), so we’re figuring out what to make an exception for and when.
We are proceeding cautiously and taking our time with the new filters and rules. We keep track of what is blocked and why.
CDN for static content automatically and for all
It’s worth noting that WEDOS Global Protection also deploys static content caching across all sites. So if you have the right rules in place, your content will load much faster around the world thanks to WEDOS Global Protection, and you’ll save server performance.
You can use WEDOS Global Protection even if you do not have hosting with us. It is enough if the domain points to our DNS. You can get started at client.wedos.global
L3/L4 attacks
Just for the record. We recently evaluated the activity of the forwards and it has dropped significantly compared to last year. Basically, it doesn’t even compare. L3/L4 attacks in the tens of Gbps are quite exceptional this year. I guess the attackers got tired of us and moved on.
What are L3/L4 attacks?
DDoS attacks at the L3 and L4 layer target the network and transport layers and use a variety of techniques to overwhelm target servers or devices.
Network layer (L3) – provides routing of data between different networks using logical addresses (IP).
Transport layer (L4) – provides reliable and controlled data transfer between endpoints using protocols such as TCP or UDP.
WEDOS Zone
The next service to be created at WEDOS Global will be the WEDOS Zone. This is purely an Anycast DNS. Thus, DNS will respond faster (the query goes to the closest DNS) and there will always be some available (in case of failure of the closest one, the next closest one will take over the traffic).
Due to high interest from B2B, the service will also support bulk adding of records, API, zone migration via AXFR, etc.
Although DDoS attacks on websites are the most commonly talked about, professional attackers know that they do the most damage by attacking the DNS. That’s why our resilient and decentralised DNS is in such demand.
Traffic via WEDOS Global for April
There are already more than 2,000 domains on WEDOS Global that are protected from various types of attacks. A large part of them are big sites of our customers that were added by support because they had some problem. Whether they were under attack or needed automatic content caching.
In April, a total of 1,869,131,298 requests were recorded from 8,101,233 unique IP addresses that were directed to protected domains. These are requests stripped of DDoS attacks on L3 and L4 and all traffic that we block on blacklists.
The numbers are a bit smaller than last month, but it should be added that some of what went through last month was blocked this month via blacklists containing the IP addresses of various botnets and compromised servers. We have a system for analyzing and evaluating DDoS attacks, where one of the outputs is a list of IP addresses with recommendations for blocking for different periods of time.
WAF blocked a total of 8 646 571 requests. It was mostly about finding and exploiting vulnerabilities. Of these, 4,077,260 million requests were blocked based on WAF hard rules and 4,569,311 based on WAF smart filters.
These demands are annoying. They often call uncached sites because they are trying to run some vulnerable script on them. They are far more demanding on the computing power of the hosting and can have the same impact as a DoS attack.
What is a Web Application Firewall (WAF)?
A WAF (Web Application Firewall) is a protection on our reverse proxies that is placed between the attacker and your website. It scans each request in real time, looking for specific signs of an attack or security hole exploitation. If it encounters a suspicious request, it can redirect it to a test (redirect, captcha) or block it.
WEDOS Global Protection also blocked 9,383,460 requests that were assessed as L7 DDoS attacks.
In addition, WEDOS Global Protection blocked 2,900,120 connection attempts that were assessed as L7 DDoS attacks.
What is an L7 DDoS attack?
L7 A DDoS attack is a type of cyber attack on a website or application that uses common internet requests such as GET and POST. The goal is to slow down or make a web page or an API inaccessible.
Attacks on L7 are difficult to detect and distinguish from normal traffic because they use the same protocols and methods as legitimate users. Special tools and techniques and a thorough analysis of network traffic are needed to eliminate them.
These attacks are quite expensive, so attackers don’t waste them. Often they only last for a very short time, when they try to overload the server. It starts with hundreds of queries in seconds, then tries thousands, tens of thousands, etc. Once they have or have not succeeded, they adjust the attack parameters accordingly. When they hit protection they usually end everything very quickly and withdraw. We assume that the reason is so that the IP addresses of the attacking servers do not end up on the blacklist.
Sometimes it is fascinating to watch how they launch a synchronized attack from thousands of IP addresses in 1-2 seconds, and when they hit the protection, they stop everything in 2-3 seconds.
The following chart shows the attacks during April. Each column is one day. The colours differentiate the points that checked them in.
And where are most of the attacks coming from?
We block most requests for L7 from the Czech Republic. The USA and Slovakia are a long way behind. If we divide the traffic by operators, then O2, Liberty Global B.V., T-Mobile and TS-Data s.r.o. All of them had more than 1 million requests blocked in April.
This is mostly the fault of compromised devices that have access to the Internet and perform things like vulnerability scanning.
The most powerful DDoS attacks
Also in April, we recorded a number of DDoS attacks directed at websites protected by WEDOS Global Protection. Since this article is dedicated to WAF, we list the most powerful DDoS attacks on the application (L7) layer.
1st place attack on a web application for cleaning computers – over 926 thousand requests per minute
At the end of the month, we saw an attack on a website for a computer cleaning application. WAF alone had to cope with 926,000 requests per minute at peak times. The attack was brief. In two phases it lasted a total of about three and a half minutes. These were approximately 2.5 million requests from 903 IP addresses. Something “a little” went through, but the target hosting was able to cope with only 2 status codes 502.
However, the attack was much stronger. Some of the requests that the reverse proxy blocked were in such a state that it was impossible to get where they were going, but they appeared at the time of the attack. It was more than a million requests per minute.
It was therefore a more extensive and complex attack that combined several types of attacks.
2nd place attack on wedos.cz – over 519 thousand requests per minute
There’s not a month that goes by that someone doesn’t try something on us. If we claim to have the most experience with DDoS attacks, it’s not just the strongest attacks (2021, 2022), but also hundreds of attacks of various types and intensity on our websites and thousands on our customers, which we not only encounter, but also analyze and try to learn from and improve our protections.
In April, one such attack even made it into the list of the most powerful DDoS attacks via L7. At the beginning of April, someone wanted to see what they could do and launched a short attack on our website from exactly 2000 IP addresses. Nice round number. Coincidence? We don’t think so 🙂
The total number of requests was 943,000. Our website survived without any problems, although there was a slight slowdown.
3rd place game server attack – 346 thousand requests per minute
In April we had to add another customer with a game server to WEDOS Global Protection due to attacks. Game servers, or more generally websites dedicated to online games, face DDoS attacks quite often.
This one in particular started getting some pretty strong attacks in early April, so support moved it to WEDOS Global Protection. If the customer uses our DNS, it’s not a problem and it’s quick (duplicate NS records and you’re done). As far as the peaks are concerned, the attacks were not that significant, but when they go over 100K requests per minute, it is noticeable. The advantage of WGP is that it caches static content as well as redirects (30X) and error pages (404), so even if the attack is on a non-existent site, it can handle it. Here it has worked repeatedly great.
But back to the game site. The strongest of the attacks had 3.8 million requests (which passed through additional layers of protection to the WAF) from 240 IP addresses and lasted 17 minutes. At peak times, it averaged 346,000 requests per minute.
Conclusion
WEDOS Global is growing, we are improving our attack detection and assessment on WEDOS Global Protection and we are preparing additional services on our global network. Everything is going in the right direction.
If you want more information, visit our website wedos.global.