Logwatch, webserver logs, PHP malarky
Posted 10-03-2009 at 04:52 AM by unSpawn
Updated 04-01-2012 at 04:18 PM by unSpawn (//Enhanced logwatch/scripts/services/http diff, added Snort ET SID 2010920 rule and fail2ban regex example, fixed commas (thanks leslie_jones).)
Updated 04-01-2012 at 04:18 PM by unSpawn (//Enhanced logwatch/scripts/services/http diff, added Snort ET SID 2010920 rule and fail2ban regex example, fixed commas (thanks leslie_jones).)
As I'm seeing more questions about (badly coded) web applications spawning rogue processes I wonder why people don't read their logs. Attacks require reconnaissance so keeping an eye on anything that looks like a prelude enables you to take measures. And please spend time updating when updates are released, installing apps properly (like not leaving the installation files around when docs remind you not to), hardening (any IDS, mod_security, Gotroot rulesets, mod_evasive or equivalent, PHPIDS, Suhosin, GreenSQL).
If you have any questions regarding this please ask them in the LQ Linux Security forum. We'd be happy to help you along.
Finding preludes to attacks by visual inspection of logfiles is cumbersome and tedious and that's why there is Logwatch to help you as it creates reports you can actually read. Logwatch unfortunately doesn't by default include rules to filter common crack signs like gotroot's mod_security/rootkits.conf but that's easily alleviated.
Locate your scripts/services/http file (might be "/usr/share/logwatch/scripts/services/http" if it's a default installation) and patch it with this:
If there's lines in your webservers logs your Logwatch report will show:
which should be your cue to investigate things immediately.
Note this is a patch against a checkout of the Logwatch CVS. If patching fails (always try testing with --dry-run and the appropriate amount of --fuzz) then check the code at around line 300 between the line starting with "my @exploits = (" and "# Define some useful RE paterns".
Also note that Logwatch takes the --service arg so you could run Logwatch as a daily cronjob on all logs (which will be slow if you've got lots of logs) and something like 'logwatch --detail High --service http --range Today' as a hourly cronjob.
If you use Emerging Threats Snort rules then a diff for SID 2010920 ('grep 2010920 /path/to/emerging-web_server.rules > /path/to/2010920.rules') may look like this:
And if you use fail2ban you could have something like this:
* YMMV and be aware all of this reactive. Instead fixing the cause is preferable.
If you have any questions regarding this please ask them in the LQ Linux Security forum. We'd be happy to help you along.
Finding preludes to attacks by visual inspection of logfiles is cumbersome and tedious and that's why there is Logwatch to help you as it creates reports you can actually read. Logwatch unfortunately doesn't by default include rules to filter common crack signs like gotroot's mod_security/rootkits.conf but that's easily alleviated.
Locate your scripts/services/http file (might be "/usr/share/logwatch/scripts/services/http" if it's a default installation) and patch it with this:
Code:
--- http.orig 2012-03-28 00:00:01.000000000 +0000 +++ http 2012-03-28 00:00:02.000000000 +0000 @@ -334,6 +334,64 @@ 'shtml\.exe', 'win\.ini', 'xxxxxxxxxxxxxxxxxxxxxx', + '%20/tmp', + '%20/var', + '7z%20', + 'apt-get%20', + 'cat%20', + 'cc%20', + 'cd%20', + 'crontab%20', + 'curl%20', + 'cvs%20', + 'echo%20', + 'elinks%20', + 'emerge%20', + 'ftp%20', + 'GET%20', + 'gcc%20', + 'gzip%20', + 'gunzip%20', + 'HEAD%20', + 'id%20', + 'kill%20', + 'killall%20', + 'links%20', + 'ls%20', + 'lwp-download%20', + 'lwp-request%20', + 'lwp-mirror%20', + 'lwp-rget%20', + 'lynx%20', + 'mail%20', + 'mailx%20', + 'mkdir%20', + 'nc%20', + 'ncftp%20', + 'netcat%20', + 'netstat%20', + 'POST%20', + 'perl%20', + 'ps%20', + 'python%20', + 'rar%20', + 'rexec%20', + 'rm%20', + 'rpm%20', + 'ruby%20', + 'scp%20', + 'sh%20', + 'smbclient%20', + 'ssh%20', + 'svn%20', + 'tar%20', + 'telnet%20', + 'tftp%20', + 'wget%20', + 'uname%20', + 'wget%20', + 'whoami%20', + 'yum%20', ); #
Code:
Attempts to use known hacks by 5 hosts were logged 35 time(s) from: 000.0.0.0: 16 Time(s) uname 7 Time(s) tar%20 1 Time(s) cd%20 2 Time(s) perl%20 1 Time(s) wget%20 4 Time(s) rm%20 1 Time(s) 000.0.0.0: 10 Time(s) cd%20 2 Time(s) perl%20 2 Time(s) wget%20 4 Time(s) rm%20 2 Time(s)
Note this is a patch against a checkout of the Logwatch CVS. If patching fails (always try testing with --dry-run and the appropriate amount of --fuzz) then check the code at around line 300 between the line starting with "my @exploits = (" and "# Define some useful RE paterns".
Also note that Logwatch takes the --service arg so you could run Logwatch as a daily cronjob on all logs (which will be slow if you've got lots of logs) and something like 'logwatch --detail High --service http --range Today' as a hourly cronjob.
If you use Emerging Threats Snort rules then a diff for SID 2010920 ('grep 2010920 /path/to/emerging-web_server.rules > /path/to/2010920.rules') may look like this:
Code:
--- 2010920.rules.orig 2012-03-28 00:00:01.000000000 +0000 +++ 2010920.rules 2012-03-28 00:00:02.000000000 +0000 @@ -1 +1 @@ -alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SERVER Exploit Suspected PHP Injection Attack (cmd=)"; flow:to_server,established; content:"GET "; nocase; depth:4; uricontent:".php?"; nocase; uricontent:"cmd="; nocase; pcre:"/cmd=[^\x28].*(cd|\;|echo|cat|perl|curl|wget|id|uname|t?ftp)/Ui"; reference:cve,2002-0953; reference:url,doc.emergingthreats.net/2010920; classtype:web-application-attack; sid:2010920; rev:3;) +alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SERVER Exploit Suspected PHP Injection Attack (php?)"; flow:to_server,established; content:"GET "; nocase; depth:4; uricontent:".php?"; nocase; pcre:"php\?.*(cd|\;|echo|cat|perl|curl|wget|id|uname|t?ftp|/tmp|/var|7z|apt-get|cat|cc|cd|crontab|cvs|elinks|emerge|GET|gcc|gzip|gunzip|HEAD|kill|killall|links|ls|lwp-download|lwp-request|lwp-mirror|lwp-rget|lynx|mail|mailx|mkdir|nc|ncftp|netcat|netstat|POST|ps|python|rar|rexec|rm|rpm|ruby|scp|sh|smbclient|ssh|svn|tar|telnet|whoami|yum)/Ui"; reference:cve,2002-0953; reference:url,doc.emergingthreats.net/2010920; classtype:web-application-attack; sid:2010920; rev:3;)
Code:
# List of PHP shell commands # Please ensure it is your intent to block IPs driven by below shell commands. As always YMMV(VM). [Definition] badcmds = /tmp|/var|7z |apt-get |cat |cc |cd |crontab |curl |cvs |echo |elinks |emerge |ftp |GET |gcc |gzip |gunzip |HEAD |id |kill |killall |links |ls |lwp-download |lwp-request |lwp-mirror |lwp-rget |lynx |mail |mailx |mkdir |nc |ncftp |netcat |netstat |POST |perl |ps |python |rar |rexec |rm |rpm |ruby |scp |sh |smbclient |ssh |svn |tar |telnet |tftp |wget |uname |wget |whoami|yum failregex = ^<HOST> -.*"(GET|POST).*HTTP.*"(?:%(badcmds)s).*200.*"$ ignoreregex =