LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 12-19-2007, 09:01 AM   #1
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Rep: Reputation: 39
Question Apache mod_security Setup Help?


Hi all,
I've been on the mod_security website and can't say the doc's are great, i'm trying to work out if the following is possible:-
I can see that some parts are of the following like redirecting to different URL or blocking completely etc....
We have requests coming into our web servers and we get some annoying people on occasions screen scrapping us, which is basically requesting the same webpage or lots of different webpages in a short space of time, of which each request is taking cpu power and its quite obvious they are not interested in our products so we'd rather just stop them from being so annoying. We know its screen scrapping because we already have monitors that detect this behavior but currently manually just stop the ip address running the scrap we'd like to make this automatic. Therefore if a host (ipaddress) is constantly trying to access the same page over and over or various pages in a short space of time then we'd like to redirect to a holding page and then if it continues to drop their packets completely for a specified amount of time of which it would hopefully work like ip tables and have a timer in which will restart again everytime another request is sent and only after the period of the timer has waited without any requests from that host in question then to allow that host to send requests again. Now i dont know if mod_security is able to do that or perhaps if it might be better to use snort, in which case i'd like to know if there is any case study type things on the internet regarding this problem, i've been looking but not really found anything.

Thanks Regards
 
Old 12-19-2007, 09:49 AM   #2
win32sux
LQ Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Quote:
Originally Posted by helptonewbie View Post
We have requests coming into our web servers and we get some annoying people on occasions screen scrapping us, which is basically requesting the same webpage or lots of different webpages in a short space of time, of which each request is taking cpu power and its quite obvious they are not interested in our products so we'd rather just stop them from being so annoying.
The way I understand it, screen scraping is the action of having a computer read output which was designed to be read by a human. Not sure how that applies here, as what you describe sounds like an attempt to carry-out a denial of service attack. That said, I don't know how to address this with mod_security, but while you wait for an answer from someone who does, I thought I might suggest a quick temporary workaround for you:

Use iptables to prevent individual IPs from starting more than a specified number of HTTP connections in a specified time-frame. If you calculate it right, you should be able to reduce the attacker's connection rate to something only a tad higher than that which you see from your friendly users. Like, for example, let's say that you determine the attacker is starting about eight (or whatever) HTTP connections every ten seconds, while none of your friendly users ever start more than three every ten seconds. In such a case, you could implement a set of rules like this (above the actual ACCEPT rule) on the server:
Code:
iptables -A INPUT -p TCP --dport 80 -m state --state NEW -j STOP-ABUSE

iptables -A STOP-ABUSE -m recent --set

iptables -A STOP-ABUSE -m recent --update --seconds 10 --hitcount 3 -j DROP
Once again, this is only intended as an anti-DoS workaround while you figure out your per-page technique.

Last edited by win32sux; 12-19-2007 at 10:10 AM.
 
Old 12-19-2007, 10:10 AM   #3
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
hello,
Thanks for the reply i should have perhaps said i've already looked at that, but thanks anyway it is useful for anyone that doesn't know it but looks at the thread.
Thanks for the info

Yes it is sort of DoS at the same time for different reasons, not going to explain to much because its never good to give to much away in these forums could shoot yourself in the foot.

Last edited by helptonewbie; 12-19-2007 at 10:24 AM.
 
Old 12-19-2007, 10:16 AM   #4
win32sux
LQ Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Quote:
Originally Posted by helptonewbie View Post
hello,
Thanks for the reply i should have perhaps said i've already looked at that, but thanks anyway it is useful for anyone that doesn't know it but looks at the thread.
Thanks for the info
It's actually great that you have this going right now, as it drastically reduces the urgency of the mod_security solution - which was sort of the point of my post. And yes, you should have mentioned what temporary countermeasures you were currently using. It's all good. I'm sure someone familiar with mod_security will be posting soon. Hang in there.

Last edited by win32sux; 12-19-2007 at 10:23 AM.
 
Old 12-19-2007, 10:26 AM   #5
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
thanks for you help win32SUX, AS DOES 64 , i'll be hoping someone does...

Thanks or replying
 
Old 12-19-2007, 12:20 PM   #6
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
The other question i've always been curious about is i know its possible to re-write packets that are sent to servers just like NAT does basically but i know people also do it maliciously, so if the source host was re-written and then with the iptables example above, those packets sent to port 80 if reaching the limit, then i'm guessing the server would then block that source address for the specified time therefore creating a DoS on yourself, would that be the same if using snort and mod_security???? i'm guessing it would unless they could detect the packet has been tampered with which isn't possible unless using ip/sec i believe? Therefore this is a draw back to anything along these lines??

I believe that is right though isn't it, if you re-wrote the source host in the ip packet and then sent lots to the server and then it would block that source host but only for the time period even though its not from that actual source??

Cheers
 
Old 12-19-2007, 02:23 PM   #7
b1v1r
ModSecurity lead developer
 
Registered: Dec 2007
Posts: 3

Rep: Reputation: 0
I assume you are writing about someone spidering your site that is dynamically generated and that is causing you excessive CPU load?

You should be able to do this with ModSecurity using a SESSION or IP collection.

Basically, you want to keep trac of how many hits from a given SESSION/IP and then block (redirect, whatever) based on too many pages requested in a given timeframe.

Something like this will allow only 10 request in a minute from a given IP:

# Block > 10req/minute
SecAction "phase:1,pass,nolog,initcol:IP=%{REMOTE_ADDR},setvar:IP.pagecount=+1,expirevar:IP.pagecount=60"
SecRule IP:PAGECOUNT "@gt 10" "phase:1,t:none,deny,status:403,msg:'Blocking screen scraper'"


You will probably want to augment this with other checks by chaining them togeather:

# Block > 10req/minute for /cgi-bin and not in the IP whitelist
SecRule REQUEST_FILENAME "^/cgi-bin" "phase:1,log,auditlog,t:none,t:normalisePath,deny,status:403,msg:'Blocking screen scraper',chain"
SecRule REMOTE_ADDR "!^(?:127\.0\.0\.1|1\.2\.3\.4)$" "t:none,chain"
SecAction "nolog,initcol:IP=%{REMOTE_ADDR},setvar:IP.pagecount=+1,expirevar:IP.pagecount=60,chain"
SecRule IP:PAGECOUNT "@gt 10" "t:none"


This is pretty simplified and you could do a lot more, but hopefully you get the idea. A warning, though, that IP != user and you could end up blocking a proxy server's IP address with this (ie AOL, NetZero, etc), so probably best to stay away from IPs and use some sort of session or other more unique value (SESSION collection vs IP).

To get into a more detailed discussion on implementation, send a note to the ModSecurity user list (see contact page on website)

later,
-B
 
Old 12-19-2007, 02:25 PM   #8
b1v1r
ModSecurity lead developer
 
Registered: Dec 2007
Posts: 3

Rep: Reputation: 0
See also #modsecurity on freenode IRC for more help.

later,
-B
 
Old 12-19-2007, 02:45 PM   #9
win32sux
LQ Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Quote:
Originally Posted by helptonewbie View Post
The other question i've always been curious about is i know its possible to re-write packets that are sent to servers just like NAT does basically but i know people also do it maliciously, so if the source host was re-written and then with the iptables example above, those packets sent to port 80 if reaching the limit, then i'm guessing the server would then block that source address for the specified time therefore creating a DoS on yourself, would that be the same if using snort and mod_security???? i'm guessing it would unless they could detect the packet has been tampered with which isn't possible unless using ip/sec i believe? Therefore this is a draw back to anything along these lines??

I believe that is right though isn't it, if you re-wrote the source host in the ip packet and then sent lots to the server and then it would block that source host but only for the time period even though its not from that actual source??
One thing you can do to stop spoofed packets from entering the STOP-ABUSE chain (and possibly causing a denial of service for friendly users) is to activate TCP SYN cookies in order to discard any spoofed SYN packets. Then you follow this up with an iptables rule to filter any packets of state NEW which aren't proper SYN packets. This should in effect make sure that any packets getting sent to STOP-ABUSE have been checked for spoofing. Example:
Code:
echo "1" > /proc/sys/net/ipv4/tcp_syncookies
Code:
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

iptables -A INPUT -p TCP ! --syn -m state --state NEW -j DROP

iptables -A INPUT -p TCP --dport 80 -m state --state NEW -j STOP-ABUSE

iptables -A INPUT -p TCP --dport 80 -m state --state NEW -j ACCEPT

iptables -A STOP-ABUSE -m recent --set

iptables -A STOP-ABUSE -m recent --update --seconds 10 --hitcount 3 -j DROP

Last edited by win32sux; 12-19-2007 at 03:28 PM.
 
Old 12-20-2007, 09:46 AM   #10
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
I like that, some great ideas there. Still if anyone has more information in any particular websites or books even that might be particularly helpful with things along these lines then please please post away!!

Regards
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
apache mod_security module sachin1361 Linux - Enterprise 2 03-17-2007 05:51 PM
[apache] Chroot or mod_security? JockVSJock Linux - Security 1 11-26-2006 04:49 AM
LXer: Secure Your Apache With mod_security LXer Syndicated Linux News 0 07-13-2006 09:33 AM
mod_security for apache zsoltrenyi Linux - Security 0 02-08-2005 07:36 AM
Apache mod_security logging everything? ridertech Linux - Security 2 08-13-2004 02:10 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 10:28 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration