POP3 DDOS Attack
 

Hi,

Currently my mail server is being targeted by POP DDOS attack. The POP packet request comes in very fast.
I have been blocking it using my iptables but my server slow to a crawl. Is there any solution to this problem.

37 1924 DROP tcp -- * * 74.63.213.126 0.0.0.0/0 tcp dpt:110

I have written to abuse@limestonenetworks.com and noc@limestonenetworks.com as they are the administrator of the IP above.

I have tried google and found that others are having the same problem. Some solutions required unplug the internet, change IP but all these are not feasible to me.

My system:
Slackware 12.2
Kernel 2.6.27.7-smp
popa3d-1.0.2-i486-2
iptables-1.4.2-i486-1

If you need any other info, do let me know.

Thanks for any help and suggestions.

Can you block it from your Gateway/Router? Blocking it at the Gateway should restore some network stability. Your best bet would be to keep the packets off your network totally.

Thanks ReaperX7,

This mail server is at the data centre which has a direct line to it, so no router or gateway.
If need to be block from gateway/router, I believe I may need to pay the data centre for firewall managed service.

Alternatively, I may have to put a firewall box (CISCO ASA5505?) in front of the mail server and NAT to it.
Do you think this will solve the problem?
Will my firewall be congested as well by the DDOS?

Any other got any solutions or suggestions?

The firewall would be a recommended addition. As far as whatever type you go with, Stateful Packet Inspect and Filtering models are what I would recommend regardless. However the CISCO ASA5510 might do a little bit better of a job, but that's just by cup of tea. Look into Barracuda Networks as well. They MIGHT be a bit more but they are one of the better solutions to get.

You say DDOS but you're blocking a specific IP...is it just a DOS attack instead then? I would just drop all packets from that IP (forget checking for POP3) at the highest level you can (and from your description it sounds like that may be the server itself). If it's a DDOS then finding the right packets to block is very difficult since it comes from a range of IPs and possibly multiple countries and discriminating between legitimate and malicious traffic becomes difficult...but if it is just a DOS attack then you're going about it the right way (I would just suggest widening the iptables rule to include all ports from that IP in case they target other ports). If you can only filter at the lowest level (the server itself) then the same number of packets will get to your box and use up available bandwidth -- you just won't use up additional bandwidth by serving responses back to the IP. Generally for a regular DOS attack this is sufficient since the speed of the connection from a single client to a single server is not sufficient to cause a massive delay for other clients...but if this is different in your case then the best you can do is filter the IP and report it to the provider (which you have done).

Hi T3Slider,

I have blocked ICMP request as well, hope to prevent other 'bots' to target my mail server.

So far only this specific IP which target POP service.
You are right, it's sensible to block all traffic from this IP, in case it target other ports.
The incoming POP request rate is estimated to be close to 100 requests per second.

I am planning to add in a CISCO5510 and NAT the traffic to the mail server.

Thanks for the suggestions, appreciate it.

If any kind souls there know of anny other solutions/suggestions, please do let me know thanks.

First check that the server itself is slow becasu of the attack and not a a bandwidth problem, becase if it a bandwidth problem (from the DOS attack) it wouldn't help you to add the Cisco firewall.

It is quite unclear how one source could bring your server to it's knees, so i would do a thorough scan of the server in order to find more culprits.

Hi hen770,

The server was up for a few years and only recently encounter this issue.
Recently my external users complaint POP timeout, but I thought it's their internet line congested.
After a few more external users having the same problem, I went to the server and check using 'iptraf'.
I only found this persistent and consistent POP attempt by a single IP.
The IP can be from different source, sometimes from Korea, China, Turkey, Middle East or USA

However, after the POP attempt subsided(could be few mins to half and hour), everything went back to normal.
Occasionally it will came back go through the POP attempt again.

I don't allow my users shell access, all setting done through web interface.
My lastlog reveals no users login access. My SSH access are only permitted from recognized/authorized IP.

You could attempt a whois or tracert and see what you can find about the IP and maybe the user. Could be a random bounce around the web, but it could be a group with multiple IP addresses that can be traced through IP address lease and account ownership.

You really should apply your filtering in layers, if it all possible. When you get your ASA or other hardware upstream of your server this will act as one layer. In the mean time, your focus should be on configuring your server to reject this malicious traffic. Of course, the required response requires that you have correctly identified the true nature of the problem. Assuming you have, the first line of defense would be to use iptables, as others have suggested. Consequently, let me start out by asking what does you current iptables firewall look like? There are a few of options to consider, ranging from blocking an IP to rate limiting the number of connection attempts allowed on a particular service. A second line of defense would be to use an application like fail2ban, which monitors the log files and will dynamically respond to these attempts to overload your system and block them for a period of time. A third layer would be to add white and black list filtering to you application.

As we like to deal with facts rather than assumptions, please provide as much detail as you can regarding the nature of the problem, such as log entries, data regarding the IP in question, what specific steps you have taken, a description of your system topology (routers, switches, etc, that are in the data stream), and so forth.

I really *wouldn't* do that. AFAIR they provide no device to protect POP traffic on port 110 (their 'spam and virus firewall' being a port 25 device and not a firewall of any kind in any case) TBH, they tend to sell boxes that are simple Linux devices filled with FOSS software glued together with some tatty Perl scripts. The only 'Firewall' part of them is IPTables. Discount this as any kind of solution.

Dropping the traffic is about the best you can hope for (or putting in a null route) - and even putting a firewall in front of it probably won't make a massive difference.

I would not call this a DDoS attack in any case as you are citing a single IP address here. Have you checked the logs to make sure this is not actually a sustained brute force password attack?

Hi ReaperX7,

I did a whois on the IP and found the admin.
Email to them about it, no response from them but the attempts stopped.

Hi Noway2,

My email servers are co-lo at data centre.

Simplified structure
data centre internet access -> switch -> both my email servers

As of now, all traffic from the IP will be drop (b4 that I was just blocking 110 traffic).
T3Slider suggested to blocked all, true enough, soon I was seeing attempts from same IP to imap services.

My iptables are by default block all incoming and allow authorized ports only.

Is this something what you meant by limiting number of connections?
Currently I create a script to monitor abnormal unsuccessful attempts and block the IP dynamically.
I am not very good in bash, but the script works for me. (email modifed to prevent abuse)
I run this every 5 mins.
If within 5 mins I got 10 unsuccessful attempts from the same IP, it will get blocked.
Currently the IP is no longer trying to brute force attempt to my email servers anymore.
Everything back to normal. I think it may not help if the culprit try again to jam up my bandwidth.
I can block all I want, but ultimately it will slow down(or timeout) my user access to the mail servers.

My various extracts of log that show the same IP.
Hi leslie_jones,

Sorry that I did not provide the full picture, I have 2 email servers but I used only 1 as example.
Both email servers are hit by the same IP.

It is good that they stopped. It is possible, though I don't know how likely, that they had some malware on their system that was responsible and your bringing it to their attention caused them to deal with it. It looks like limestone networks is a smaller hosting outfit with about 63 public addresses, at least in that block. Your report may have indeed caused them to take action.

It is unlikely that you will be able to put a physical device infront of this hardware and you will probably be limited to iptables and other software solutions, unless the problem is severe enough to warrant support from your hosting provider.

Yes, this would be rate limiting and rate limiting should certainly help, at least as long as you block the right ports. Your script looks like it is incomplete though. Here is an example for port 22 that I use:
Notice how it triggers on teh state new, and then blocks on an excessive number of new connections, which is slightly different.

Instead of using your own custom script, check out the application fail2ban. It has been around quite a long time and has a very solid reputation. Not that there is anything wrong with writing a script. In fact, I applaud the fact that you took this approach. The application developers and maintainers have had a lot of time to react to issues and threats which might give that application a bit of advantage that you could take benefit of.

The fact that it is the same IP reinforces my suspicion that someone had an infected machine with a script that was responsible rather than a deliberately dedicated attack against you. You should be able to use filtering to eliminate this kind of noise. If you are facing a true attack to where filtering methods and other host based techniques don't work, then you will be needing the assistance of the data center. Having implemented these techniques will put you in a good position to demonstrate why their assistance would be justified too.

Thanks Noway2,

I will look into the fail2ban app.

Seems like there is nothing much more I can do beside what I had already done.
Blocking from the source(originator IP) is still the best solution.

I will mark this thread as SOLVED.

Thanks everyone who have contributed.