Surviving an HTTP DDOS Attack
Posted by Joe Topjian on July 14, 2010 under Administration | 
Be the First to Comment
Introduction
I occasionally run into a cPanel server under a DDOS attack due to either the site itself being attacked or the site has had malicious files uploaded to it that are being used to launch an attack. This post will describe what I do to help mitigate the attack.
Table of Contents
Symptoms
The first step is to actually confirm that the server itself is under a DDOS attack. There are a few ways to do this.
The first is simply the inability to reach any websites hosted on the server. This is due to Apache processing too many requests at once. You will either be unable to reach a site at all or the site will take an extremely long time to load.
Next, log into WHM and check Apache Status. If Apache Status is not working due to Apache being completely full of connections, issue the following on the command line:
$ pkill -9 httpd & $ pkill -9 php & $ /etc/init.d/httpd restart
Once Apache has started, check Apache Status again. See anything unusual? Is there one webpage that is being requested several hundred times? If so, is this webpage being accessed by the same IP or several IPs?
If the requests are from the same IP, simply block that one IP and restart Apache.
If the requests are from several IPs, are they from the same subnet? If so, block that subnet.
If the requests are from seemingly random IPs, then your server is probably under a DDOS attack. You can try blocking individual files, but chances are that you’ll block a few hundred and still be under the same load.
Note, just because the server is under a DDOS attack does not make the scenario malicious. It is possible a customer on your server posted their site to Digg or some other popular site and there’s simply too much traffic to handle. Whether or not your believe the attack is malicious in nature or not will not change the next few steps — just how you deal with the customer once the attack is over with.
Mitigating the Attack
Remove the Zone
After you have confirmed your site is under a DDOS attack, remove the zone from your DNS servers. You can do this by editing the named.conf with vi or any other editor. Look for an entry like so:
zone "example.com" {
type master;
file "/var/named/example.com";
};
Remove all 4 lines, save, and exit the file. Then issue the command:
$ rndc reload
This will prevent any new visitors / bots from resolving the domain name to reach your server.
Suspend the Account
Next, suspend the account and notify the customer.
Unfortunately by suspending the account, you will redirect all requests to their site to /cgi-sys/suspendedpage.cgi which might cause an even higher load on your server than before. To work around this, edit the .htaccess file located in the customer’s public_html directory and comment out (add a #in front of) the line:
RedirectMatch .* /cgi-sys/suspendedpage.cgi
Limit the Amount of Connections to the Site
Finally, if you have compiled Apache with mod_bw, you will be able to limit the max number of simultaneous connections the site is allowed. I’ve found cPanel’s support of mod_bw to be buggy, so here is how I set this up:
$ /scripts/setbwlimit --domain=example.com --limit=256000
This will create the necessary mod_bw config file and limit the customer’s throughput to 256kb. This is not what we want to do, but it will create the base config file as well as configure Apache to read that config file for this customer’s domain.
Next, edit the following file:
/usr/local/apache/conf/userdata/std/2/username/example.com/cp_bw_all_limit.conf
The file should look like this:
<IfModule mod_bw.c>
MaxConnection all 1
ForceBandWidthModule On
BandWidthModule On
BandWidth all 256000
</IfModule>
<IfModule mod_bandwidth.c>
MaxConnection all 1
ForceBandWidthModule On
BandWidthModule On
BandWidth all 256000
</IfModule>
This will allow the domain to only be able to support one connection at a time. Any other request will receive a 503 Service Unavailable error.
Save the file and restart Apache:
$ /etc/init.d/httpd graceful
Now just wait until the attack subsides. I’ve seen attacks last anywhere from 12 to 48 hours.
Conclusion
Although this is not the definitive guide on handling DDOS attacks, this is the method that I have used and seems to work rather well. It mitigates the attack while still allowing other customers hosted on the server proper access to their sites.
If anyone has any comments or different methods, please let me know.
Add A Comment