high cpu utilization under heavy traffic load
 

I am using Mandrake 10.0 powerpack, with Shorewall as the firewall, I am observing very high CPU utilization loads due to DROP packets from Net. Someone is flooding the server at the various ports, the ports are being blocked by the firewall but the cpu utilization goes very very high. on the other hand if i disable the firewall, then the cpu utilization goes normal.

how i am supposed to handle the situation?

renice the firewall? If its being packet'd I wouldn't disable the firewall. If you renice it to 19 then everything should still run just fine . . . but if the packeting stops then restore its original niceness.

A good way to start is by using tcpdump or ethereal to capture a few packets to see what's going on, so that you can verify it is malicious in nature as well as get a idea of the specific type of traffic you're dealing with. There are various measures you can take that are dependant on what kind of traffic you're seeing.

If it is malicious, then you definitely want to notify your ISP as well as the owner of the netblock that the traffic is originating in (you can usually find an abuse@ address using a whois lookup). From there, you can try turning off connection tracking (lots of resource consumption tracking connection states) and use simple packet filtering. You can also try adjusting several of the sysctl networking parameters (shorten connection timeouts). If you're dealing with a syn flood, then turning on tcp_syncookies may help too. Seeing what type of traffic is hitting your box will help to narrow down the best options (in theory ;).

Finally, make sure that your kernel and iptables versions are current. There are several DoS vulnerabilities (including CPU resource consumption) in older versions.

I have tried to log down what type of traffic is flooding the PC, some remote IP is trying to continuously send TCP packet on 3128 (squid-port) on my server ip.

This computer is being used as a Squid-proxy server for the internal (~50 users) lan, and have a Real-IP on the Internet Interface. The Firewall is contuously dropping the incoming request on the 3128 from the Internet Interface, but the speed of the incoming packets is very very fast.....The server is connected with a 512Kbps dedicated internet.

Start out with turning tcp_syncookies on with:
echo 1 > /proc/sys/net/ipv4/tcp_syncookies
Tcp_syncookies forces any new connection attempts to be validated by the remote host before it's entered into the connection tracking table (basically a SYN-ACK is sent to the host with a special non-random sequence number that must be acknowledged). This reduces the amount of resources a SYN flood can consume. However, the syn_cookies mechanism isn't turned on until your max syn backlog is reached, so that may need to be modified as well if syncookies aren't triggered. Second, call your ISPs abuse or NOC and tell them what your experiencing. See if they can temporarlly null route traffic from that IP until it stops.

You can also try decreasing the value of /proc/sys/net/ipv4/tcp_synack_retries (should be 5 by default), but reducing it to 3 or 4 should decrease the connection timeout on un-replied syns from 180 to ~100-135 seconds. Remember to pay close attention after adjusting networking parameters as it may cause your own hosts and legitamate clients to have problems as well.

Finally, you might want to make sure that squid doesn't allow proxying to remote hosts on the internet, as the trafffic your seeing might be someone attempting to use you to proxy a large amt of traffic and not an actual attack per se.