LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 03-02-2007, 04:25 AM   #1
c00kie
Member
 
Registered: Mar 2007
Location: CORE
Distribution: FC6, FreeBSD, XP?
Posts: 34

Rep: Reputation: 15
Unhappy DDoS trough a VerliHub bug, apache is dead


Okay, so.. I'm ddos'ed.

I have linux box, configured by a friend of mine and me, hosted at some ISP we have here... It's a webserver ( apache with php + mysql ).

Problem is like this, I'm gonna make it simple...:
Apache/MySQL Starts -> in 1 second they're down.
Why? Becuse there is a tool(discussed here: http://www.webhostingtalk.com/showthread.php?t=574701) that connects to DC++/oDC Hubs powered by VerliHub, and then sends a certain message to ALL the users on the hub and makes them access my site ( don't have the tool, but I have something similar.. anyway ) and my port 80 is down... all the users send a very small amount of packets to the site, and block only port 80. We have to stop this!

What should I do... cause I'm desperate, I have like 17k members that are very pissed off right now...

Thanks.
 
Old 03-02-2007, 06:32 AM   #2
live_dont_exist
Member
 
Registered: Aug 2004
Location: India
Distribution: Redhat 9.0,FC3,FC5,FC10
Posts: 257

Rep: Reputation: 30
Looked through that thread you've on the website above. There's a lot of opinions being floated out there already.Nothing much can be said that hasn't already been discussed there.

Here's my 2 cents:
1.Recreate the environment.
2.Start Ethereal/Tcpdump/Some Sniffer
3.Start the DDOS tool
4.Capture the packets
5.Find out what traffic is hitting the Main HUB.
6.Find out what Traffic the HUB is sending all the DC++ clients..all you need is the initial string.
7.Put all the DC++ clients into a group
8.Write a Snort rule(Content rule) which will detect the traffic. If you get hits on this rule you'll know that the rule is correct.
9.I havent personally done this but I believe you can configure Snort in something called inline mode where it works with iptables and drops packets if an alert is triggered.

Unless you find out what exactly this exploit does, there's no way you can block it.Thats how new exploit code/viruses are restricted. The reason why IP blocking is not working is probably becasue the code is configured to randomly choose IP's and send spoofed packets with this random source IP address.Just guessing..

The ideal thing though is for a few affected customers to send POC's of packet dumps to VerliHub/DC++(whichever one the exploit is for) and prove that there is a genuine problem so that they can come out with a patch. Everything else is Just a quick fix.

All the best and hope this helps a bit.

Cheers
Arvind
 
Old 03-02-2007, 07:01 AM   #3
nx5000
Senior Member
 
Registered: Sep 2005
Location: Out
Posts: 3,307

Rep: Reputation: 57
Quote:
Apache/MySQL Starts -> in 1 second they're down.
And how is the bandwidth? Down also?

If the packet arrives, it has already overflowed your bandwidth and the ddos is effective.
Blocking it with iptables will let the kernel breath a bit more, but won't help your connection.

The traffic has to be blackholed upstream. It's complex and cost money. Depending on your ISP and his link with his upstream providers, he's the only one who can do something.

Otherwise
Change IP?
Or change the DNS name?

Good luck!

Last edited by nx5000; 03-02-2007 at 07:03 AM.
 
Old 03-02-2007, 07:19 AM   #4
c00kie
Member
 
Registered: Mar 2007
Location: CORE
Distribution: FC6, FreeBSD, XP?
Posts: 34

Original Poster
Rep: Reputation: 15
It doesn't kill the bandwith, it kills only port 80.
I know how it works, I'll try to do what live_dont_exist recommended ( thanks m8 ), but i think it will take a while..

This tool.. doesn't work on IP, so changing IP is no good... i have to filter this, since is not killing my bandwith, it can be stopped, that's what I know...

I'm wainting for other people to gimme some info, maybe it will help.

Thanks.
 
Old 03-02-2007, 07:33 AM   #5
nx5000
Senior Member
 
Registered: Sep 2005
Location: Out
Posts: 3,307

Rep: Reputation: 57
Quote:
Originally Posted by c00kie
It doesn't kill the bandwith, it kills only port 80.
I know how it works, I'll try to do what live_dont_exist recommended ( thanks m8 ), but i think it will take a while..
ahh ok

Quote:
Apache/MySQL Starts -> in 1 second they're down.
Hum this was not clear then. Mysql on port 80?

Quote:
This tool.. doesn't work on IP, so changing IP is no good... i have to filter this, since is not killing my bandwith, it can be stopped, that's what I know...
Well if it doesn't by IP then it's by DNS, my second solution.
Anyway, your problem is different than what I thought.
Try first what is said above then.
Linux kernel also has syn flood protection but all this depends on the rate and the context of the attack.
 
Old 03-02-2007, 07:38 AM   #6
live_dont_exist
Member
 
Registered: Aug 2004
Location: India
Distribution: Redhat 9.0,FC3,FC5,FC10
Posts: 257

Rep: Reputation: 30
Quote:
Originally Posted by c00kie
This tool.. doesn't work on IP, so changing IP is no good... i have to filter this, since is not killing my bandwith, it can be stopped, that's what I know...

Thanks.
Think what nx5000 was mentioning was change your website IP / DNS not block the offending IP's. This though is probably your last resort and you'd need to work with your ISP to change routes etc.

However hold on..are you saying its hitting Port 80.Hardcoded???Are you sure???

Or is it intelligent enough to do something like a nmap -sV and find out which port Apache is running? If not then a quick change of port + a broadcast message/some other better way to tell your customers that they need to go:

Code:
www.website.com:53467 instead of www.website.com
might give you some breathing space. Not much good in the long run but buys you a bit of time.

Cheers
Arvind
p.s.... Oops .. a bit late..nx5000 had already posted .. all the best dude.

Last edited by live_dont_exist; 03-02-2007 at 07:45 AM.
 
Old 03-03-2007, 03:24 AM   #7
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
From the discussion on the linked forum, this looks like a DoS against Apache rather than a bandwidth consumption or syn_flood attack. Can you access other ports on the system? Could you post some entries from the Apache logs showing the URL request?

In the meantime you may want to take a look at the Apache page on performance tuning against DoS attacks. The default Apache settings aren't optimal in this situation, so it may help to implement a few of the tweaks. You may also want to take a look at mod_dosevasive. AFAIK, it isn't being developed anymore and the main site is down, but you may still find it useful.
 
Old 03-03-2007, 04:51 AM   #8
c00kie
Member
 
Registered: Mar 2007
Location: CORE
Distribution: FC6, FreeBSD, XP?
Posts: 34

Original Poster
Rep: Reputation: 15
@Capt_Caveman, yea, it attacks port 80, not a bandwith consumption. I can access any other ports, but this tool.. can be configured to attack any port the script kiddie wants.

Last edited by c00kie; 03-05-2007 at 03:44 PM.
 
Old 03-03-2007, 09:41 AM   #9
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
Hmmm, those kernel messages would suggest that it is a syn_flood or at least has the same effect. The fact that you don't get true connections to Apache would support that. The first kernel message is the netfilter conntrack table getting filled and subsequent packets getting dropped, along with syn_cookies getting triggered (only comes on when the conntrak table gets close to being full).

If you don't absolutely need to do statefull packet filtering, you can unload the ip_conntrack module which will get rid of the whole conntrack table altogether. Alternatively you can increase the max number of entires in the conntrack table. See this on how to find the optimal numbers to use. You can also tweak the tcp timeout values. In particular reduce the amount of time a connection can remain in SYN_RECV. Which reminds me, take a look at /proc/net/ip_conntrack and see if most of the connections are in SYN_RECV.
 
Old 03-03-2007, 05:16 PM   #10
c00kie
Member
 
Registered: Mar 2007
Location: CORE
Distribution: FC6, FreeBSD, XP?
Posts: 34

Original Poster
Rep: Reputation: 15
alot of them, like 90% are in TIME_WAIT..

Last edited by c00kie; 03-05-2007 at 03:45 PM.
 
Old 03-03-2007, 07:14 PM   #11
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
Regarding tcpdump, use -x and -X options to get a hex and ASCII dump of the payload. Also use -c to specify the number of packets to capture. So the full command would be:
tcpdump -vxX -c 50.
See if you can find anything common among the DoS packets. I have a feeling these packets don't have a payload. If these are the results of the DC++ tool, you should be seeing the DC++ clients trying to issue a logon with "$Lock <key> Pk=<pk>". I believe the attack works by spoofing an $OpForceMove command and specifying port 80 on your server as the new hub for all of the hubs clients to logon to. Most of these connections don't even appear to get passed the SYN part of the TCP handshake.

It's also probably good to get a rough number idea on the number of packets/sec that are hitting your box as well.
 
Old 03-03-2007, 07:27 PM   #12
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
Quote:
Originally Posted by c00kie
in /proc/net/ip_conntrack most of the connections are like this

PHP Code:
tcp      6 74 TIME_WAIT src=194.126.185.100 dst=194.126.185.213 sport=35785 dport=62029 packets=4 bytes=216 src=194.126.185.213 dst=194.126.185.100 sport=62029 dport=35785 packets=3 bytes=168 [ASSUREDmark=0 secmark=use=
alot of them, like 90% are in TIME_WAIT..
Yeah, I would try unloading the ip_conntrack module and see if that helps any. If it does work, you may be able to get away with using the conntrack module if you decrease the TIME_WAIT setting from the default of 2 minutes down to something shorter and then increasing the max number of conntrack entries.

Last edited by Capt_Caveman; 03-03-2007 at 08:01 PM.
 
Old 03-05-2007, 02:07 AM   #13
c00kie
Member
 
Registered: Mar 2007
Location: CORE
Distribution: FC6, FreeBSD, XP?
Posts: 34

Original Poster
Rep: Reputation: 15
ok will do tcpdump -i eth1 -vxX 50

Last edited by c00kie; 03-05-2007 at 03:45 PM.
 
Old 03-05-2007, 02:36 AM   #14
live_dont_exist
Member
 
Registered: Aug 2004
Location: India
Distribution: Redhat 9.0,FC3,FC5,FC10
Posts: 257

Rep: Reputation: 30
If you notice all the packets seem to start with either E..0 or E..< or E..( or E..4. You might be able to identify further patterns if you look closely.

If you have a Snort box set up by any chance or any other device which can look into the content of the packet you might want to write a rule which drops any packets with those specific start patterns.

Cheers
Arvind
 
Old 03-05-2007, 03:09 AM   #15
c00kie
Member
 
Registered: Mar 2007
Location: CORE
Distribution: FC6, FreeBSD, XP?
Posts: 34

Original Poster
Rep: Reputation: 15
Ohhhh, and do you know how could I ban those IP's? DROP them.. ?

Last edited by c00kie; 03-05-2007 at 03:45 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Verlihub addouns Padawan.AVT Linux - Server 0 12-12-2006 01:27 PM
VerliHub on Slackware? :-/ NightSoul Linux - Software 4 07-27-2006 05:46 PM
Trying to install verlihub z9_87 Linux - Software 0 10-19-2005 05:20 PM
Apache getenv bug? wu1821 Linux - General 1 06-29-2005 10:00 AM
Slackware 9.1 and Apache bug? xipxap Slackware 3 10-14-2003 11:09 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 04:35 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration