filtering system logs
 

Is it possible for me to tell syslog that log entries should be sent into a perl script, and the output of that script should be logged if and only if the perl script prints anything to STDOUT? I want to create a program to keep my logs clean. A lot of debugging of CGI programs I do creates really nasty entries I'd rather were sent to a special /var/log/cgi_errors and when I catch somebody scanning my box I want to automatically e-mail their ISP and try and prevent further malicious behaviour.

-Dan

I don't know the answer to your first question, but I assume it is possible. A very simple, scriptable solution would be to use something like 'grep -v' to view your logs without all the cgi errors.

As for the second, be very careful setting up any automatic-retaliation measures. TCP/IP is inherently anonymous and scanners can easily fool your computer, hide their true IP address, and even use your retaliation measures against you. For example, with the setup you've described someone could force your box to send millions of emails to any target, including your own system.

Well I was going to do a traceroute to find the main carrier (i.e. comcast.net, rr.com, whatever) and send an e-mail to abuse@domain.com . I was actually hoping to make it sophisticated enough to share -- but on second thought, perhaps by releasing the source I would enable script kiddes to circumvent the measures. :: shrugs ::

-Dan

Perhaps I should make a report of suspected abuse arrive in my inbox for approval every morning.

-Dan

Is it possible for me to tell syslog that log entries should be sent into a perl script, and the output of that script should be logged if and only if the perl script prints anything to STDOUT?
Never needed it myself, but IIRC you can have syslog pipe messages tru a filter and into a file. I tried out Ngsyslog to see if it had regex capable log filtering but I found it no solution (IMHO).

I want to create a program to keep my logs clean.
In essence the more you log the better. If you're worried about losing overview you should IMO not try to change the logs, but change your logreporting.
Did you check out Logwatch or alike?


Perhaps I should make a report of suspected abuse arrive in my inbox for approval every morning.
I agree with Mac_phil automated retalliation is one of the worst things you can do to yourself and others. Reporting single-system scans also will usually be a waste of resources. If you want your efforts to have effect on a larger scale, maybe you should join Dshield or Mywatchmen.

Well, that's the thing. When I say "logs clean" I mean that debug messages (i.e. from when I write CGI scripts) related to CGI should go to a CGI_errors log, and they should be sorted. I want to log as much information as possible just in case I ever need it.

But the other thing is, there are people out there who have attempted to hack my box. Granted, most of them are content to scan, realize I'm running a hardened version of Linux, and go away, but sooner or later I'm going to be on vacation when a particularly nasty security update comes out, etc., and what then?

So I sort of feel obligated to report these port scanners as abuse so more of them can be redirected to /dev/null.

Why? If I report to their ISP that they're doing something malicious, won't they be disconnected (and thus there will be one less hacker for me to worry about)?

-Dan

Let me redefine what I mean by "clean logs". Currently my logs are filled up with messages that make it very hard to read. If I open up my CGI error logs I see debug messages from scripts I've been testing, port scans, things of that nature, all of which make it really hard to see if there's anything bad happening. Maybe I just need to learn to read them better, but there has to be a better way for me to open up my logs, see there are problems, and respond, instead of opening them and seeing junk.

I think I understand exactly what you want, but you seem to be making it more complicated than it need be. If you indeed want to 'log as much information as possible' isn't it completely immaterial whether those logs are in one file or two? If you need a CGI error log readout, just grep the log for CGI errors. If you want the log without the CGI errors, grep -v for CGI errors.

Or is it something else?

If you're on vacation and miss a security update it doesn't make any difference if people are scanning you or not. You've been scanned in the past, so you're already on many lists. The reason people scan is to identify what services are running on what IP address. Then, when the exploit comes out, they attack. It's not that the exploit comes out and then they scan for the service. If you're paranoid when on vacation make sure you can SSH into your box to patch it.

In any case, no, they won't be disconnected. A vanishingly small number might be disconnected, but not quickly enough to make any difference. Suppose they had to scan you (which they don't). NMAP takes a few minutes. Then the exploit gives them root. Who do you think is working at their ISP? Superman?

Suppose he blocks their connection in time. Or even suppose he blocks their connection later, after they've had root for a few hours. Of those attackers, a vanishingly small number were using their own computer to attack you. If so, they can't use their own connection anymore, but they surely have a rootkit on your box and will find another way online. If they aren't using their own computer you are no better off for shutting down some compromised box in the completely wrong state, because they can just SSH somewhere else, then back to you. Even if Superman catches them during the scan they could have more compromised boxes to use. If not, well, you win. This small victory comes from the assumption that you can accurately identify who is scanning you, which is not always true. Plus it requires their ISP to have at least one employee from Krypton.

Unfortunately you win the battle but lose the war. You've been scanned by hundreds of people in the last few months, and they're busy installing rootkits.

Grep is great to filter logs when you view them, thus making them more clear.

Grep can also read filter conditions (regexps) from a file with the -f option.
So you can create predefined filter files to view your logs from different aspects.

Wait, are you saying that it's almost impossible for my system not to get hacked? Can you offer any tips on preventing attackers from gaining root kits?

I'm not saying that at all. I was merely starting from your hypothetical of being on vacation (i.e., not maintaining your box) while an exploit has been unleashed for a service you are running.

-Don't run services you don't need.(services are telnet,ssh,squid proxy,ftp,etc...)
-For services you do need, only open your firewall on that port to IP ranges that need access.
-Keep those services patched and up to date.
-Use good, long, alphanumeric, gibberish passwords.
-Don't ever login to your box with telnet or ftp. Use SSH and sftp.

If you want to get a sense of how enticing you are to crackers, nmap your box from a different computer (one not on your LAN, if you have one).
http://www.insecure.org/

If an nmap scan shows a lot of unneccessary services open to the internet, shut them down.