Today we have for you a sample DDoS attack on the application layer (L7), which was directed to the websites of one of our customers several times a week. It was interesting that the attack always started at 7:00 in the morning.
Now you can also listen to the article as a Podcast.
First, a brief summary of the concepts.
Application layer attacks (L7)
These were attacks on the so-called seventh application layer. Simply put, it is the layer where the data exchange between the browser and the server already takes place. The data requested by the user is downloaded or sent directly. The previous layers took care of the connection, the form of communication, etc.
This kind of attack is very insidious because it masquerades as legitimate Internet traffic. In the access log you can see normal GET requests to PHP/HTML documents, mostly without calling other files (CSS, JavaScript, images, etc.). However, this may not always be the case, we also encounter attacks where a non-existent page is deliberately called(January 2022).
To stop such an attack, you need to see into the communication between the server and the user. Then it’s easy to reduce the faulty traffic, because you don’t worry about who goes there, but what they do and how they do it.
If you can’t see into the communication, the only solution is to restrict accesses from a given IP address, which is not ideal, as there can be a whole estate behind one IP address. In addition, we have previously encountered attacks where mobile phones from the networks of foreign mobile operators were apparently attacked due to malware(January 2021, March 2021). Such IP addresses cannot be blocked in the long term. Imagine if the only solution to protect your website was to restrict access from our mobile operators’ networks. Most legitimate traffic goes through them.
WEDOS Global Protection
And that’s exactly what our new WEDOS Global Protection service is doing, already protecting over 300 of our customers’ websites. It is built on our decentralised WEDOS Global network.
We direct the customer’s domain via DNS records to the reverse proxy WEDOS Global, through which all traffic will go. Furthermore, the attacker can only see the IP address of the reverse proxy, so he does not know where the site is actually hosted.
The reverse proxy sees the traffic and if the conditions set by the customer or us are met, it puts an obstacle in the way. It can be a cookie + redirect or a captcha. Alternatively, it can block suspicious accesses altogether or show them cached content. You can read what WEDOS Global Protection can do in the article A peek into the WEDOS Global Protection administration.
And now for the attack itself.
Who was the target
It is a successful Czech company with a number of branches in many European countries. For each country there is a separate web presentation where the company is presented, the services it offers and the product catalogue. It is not a classic e-shop.
The individual web presentations are on domains of the form mark.ccTLD (national domain).
One of the company’s websites has been regularly under attack before. They solved it by optimizing the NoLimit webhosting to handle a bigger load, we also blocked something in the standard way. Later they were one of the first I moved to test WEDOS Global Protection.
They were lucky because L7 attacks have increased significantly in strength and intensity this year. Even the new NoLimit Extra would not be able to handle that, and customers have websites with millions of requests per day. We believe this may also be a result of the war in Ukraine, where pro-Russian hacking groups have released a number of scripts for L7 attacks, including instructions on how to use them.
Since the customer has web presentations on multiple domain extensions, we had to hide them all gradually, because the attackers tried another domain as soon as they came across the active WEDOS Global Protection. However, they did not give up.
The following 5-minute chart tracks the access log with one of the sites. The attack starts at exactly 7:00 am and is so powerful that 503s (exhaustion of allocated server resources) occur within the first 5 minutes.
This was an attack on parts of WEDOS Global Protection that were not yet fully automatically protected. It’s a complicated service and we’re accelerating development towards full automation as much as possible 🙂
In the following graph of the same attack, full protection is already manually triggered and traffic is directed through the individual points.
The problem was also that not all the domains were with us. The reason for this is that we don’t offer a number of European extensions yet, so he had them from competitors and used their DNS as well. For our protections to work properly, however, you must use our DNS servers. So my colleague wrote to him to change it so we can protect everything.
The interesting thing was that he had no idea about most of the attacks. After all, if the domain was protected and everything worked as it should, the attack looked like this from the point of view of our protections.
The attack just came, the individual sites picked it up by location and put a barrier (redirect, cache, captcha) to the suspicious traffic.
Gradually the attacks became more intense, but the pattern was always the same. It started at 7:00 and after half an hour or so there was a surge. Most of the attacks were only in the morning.
The strongest and last attack on the customer’s website was recorded on 21.06.2022, when the attacker really tried hard and pulled out over 500 thousand requests per 1 domain per minute. As you can see, they also broke their normal attacking routine.
Consider also that what you see is already stripped of accesses that were blocked by the blacklisted SYN filter. It usually does most of the work (blocking up to 90% of problem traffic). Due to the nature of the filter, it is difficult to assess exactly where the attacks went and get exact numbers. The 500,000 are accesses that managed to pass the blacklisted protection. And we’re not just using the usual paid blacklists. We have our own built on data from all web hosts.
Unfortunately, we do not have centralized data from WEDOS Global Protection for all months. We only started collecting and storing them centrally at the beginning of June. Until then, the logs were on the servers in individual points and were continuously deleted. The following graph shows the attacks on this customer for the first three weeks of June. It’s a 3-hour chart. As you can see, over 134 million requests from 11,636 unique IP addresses have arrived at the protection during that time.
Conclusion
The development of both WEDOS Global and WEDOS Global Protection is progressing rapidly. We are preparing the next points. In the meantime, we are trying to analyse ongoing attacks and prepare for future ones. As you can see, we are not bored 🙂
Moving hundreds of customer sites that are often under attack or have problematic traffic (for example, they are often overloaded by bots that retrieve content from them or submit forms) has had a very positive impact on server performance. Webhostinog servers have virtually disappeared from monitoring. And thanks to wedos.online we measure every website, every customer.