LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 12-05-2012, 06:30 AM   #1
juice99
LQ Newbie
 
Registered: Sep 2005
Posts: 12

Rep: Reputation: 0
DDoS using up all available irq by generating interrupts


i'm being ddosed by spoofed generated IP addresses, they are generating 200k tcp packets per second, sending it to port 80

the attack causes lots of interrupts and my server starts loosing packets, and it becomes more annoying with time. server is not under heavy load durning attack, but it is using up all the interrupts that my network card (Intel 1Gb) can process, and starts loosing legitimate packets

the only thing i can do so far is to keep changing IPs, but can you please help me to find out what is the bottleneck?

ksoftirqd process is spread over to 3 cpus it seems, because two cpus are used to 100%, third one is using like 20%

atop shows 250% or so is used by irq

so far i tried to changed usecs and buffer values with ethtool, i tried installing irqbalance , i tried playing with smp_affinity although i still didn't change the kernel

my provider is refusing to help me with DDoS, they are saying it's my bussiness, besides they cannot do much since these are spoofed ip's

i also changed net.ipv4.conf.all.rp_filter and net.ipv4.conf.default.rp_filter from 2 to 1... but i don't think it will help, because even spoofed packet will probably generate interrupt

Last edited by juice99; 12-05-2012 at 07:01 AM.
 
Old 12-05-2012, 07:44 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,132
Blog Entries: 54

Rep: Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791
IMHO you should look at the bigger picture: since they seem to follow you around there's no way to predict if any measures you take now won't lead to other attacks later on and besides that there's very little an end point can do on its own (blocking bogons, rate limiting traffic at the network level, signature-based blocking may help alleviate things to some small extent) as far as DDoS attacks go. Ask yourself if your site is so important that it should remain available at all cost. If it isn't that important then just shut it down and sit out the attack. OTOH if it is then go vote with your wallet. The other question you should ask yourself is why you are under attack. Often these things happen for a reason.
 
Old 12-05-2012, 11:49 AM   #3
juice99
LQ Newbie
 
Registered: Sep 2005
Posts: 12

Original Poster
Rep: Reputation: 0
i don't care their motives, i'm running hosting service so it's probably competition

and sitting it out is not an option

my bandwidth can handle that, it's irq problem, interrupts are generated with each packet, and something is overloaded with so many packets and legitimate traffic is affected

can you please help me find out what it is and how to change it? it is technical problem and in this particular case it is problem with how kernel handles the whole thing. i won't call it a bug, but there is spare CPU power and system behaves unstable, there is something wrong there
 
Old 12-06-2012, 06:47 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,132
Blog Entries: 54

Rep: Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791Reputation: 2791
Quote:
Originally Posted by juice99 View Post
i don't care their motives, i'm running hosting service so it's probably competition
You should though. Hosting service providers don't usually DoS each other.


Quote:
Originally Posted by juice99 View Post
can you please help me find out what it is and how to change it? it is technical problem
I'll help you with suggestions that may mitigate the situation a bit but you should understand the limitations of the situation from reading documentation like the SANS Reading Room and the Network DDoS Incident Response Cheat Sheet (PDF) offer.
Let's start with some basic information gathering. If multiple public IP addresses are attacked then prepare a report for each host:
- machine type (standalone, VPS, shared host, cloud instance),
- distribution, release, kernel version, iptables version, plus an indication of the machine being able to run 'ipset',
- your firewall rule set as in 'iptables-save > /path/to/file' (attach as plain text file),
- an overview of network-related measures you already implemented.
- IDS logs if any,
- packet captures for tcp/80, see 'man tcpdump', (tcpdump -i [ethernetdevice] -n -nn -s 0 -w [/path/to/file.pcap] 'tcp and dst port 80'),
- optionally: resource utilization, preferably spannning at least a week, (SAR data like Atop, Dstat, collectl).

* And add anything else you think might be useful. Any information / files I didn't request you post or attach here should be part of a bzip2-compressed tarball, then PM or email me the location of the tarball.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] generating timer interrupts in C yaami Programming 3 07-24-2010 02:59 AM
Generating Interrupts to the USB sensor device lucky6969b Programming 1 03-31-2006 01:18 AM
generating hardware interrupts 03mcmt02 Linux - Software 1 04-04-2005 04:39 AM
How can I change interrupts (IRQ numbers) in SuSE 9.1? (Please...) Robhogg Suse/Novell 3 10-16-2004 06:47 PM
How to change interrupts / IRQ numbers in SuSE 9.1? Robhogg Linux - Hardware 0 10-12-2004 06:33 PM


All times are GMT -5. The time now is 08:35 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration