LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 06-10-2012, 04:04 PM   #1
vahacker
LQ Newbie
 
Registered: Jun 2012
Posts: 6

Rep: Reputation: Disabled
Cheescloth v2.0


I wrote this little script as way to stop endless sendmail calls to unknown users on my mail server trying to spam it. I then wrote a SSH brute hacking detector into it as well. Works with iptables and developed on slackware but should run on any linux flavor with very minor tweeking. If you do please let me know if you get it running on a another distro so I can added to the software project.

http://code.google.com/p/cheescloth/
 
Old 06-12-2012, 08:32 PM   #2
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,328
Blog Entries: 33

Rep: Reputation: 328Reputation: 328Reputation: 328Reputation: 328
What does this provide that I can't get from other software such as Fail2Ban? Is this something you designed to run on a personal computer or was your software intended for high availability production enterprise class servers? One feature I think might be unique to your software is writing sendmail rules when SSH gets knocked. I'm not sure if Fail2Ban can do that but it might be scriptable.

Last edited by sag47; 06-12-2012 at 08:33 PM.
 
Old 06-12-2012, 08:45 PM   #3
vahacker
LQ Newbie
 
Registered: Jun 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
I actually wrote it up originally wrote it in 2008 and put it up in 2009. I have never heard of fail2ban. I will check that out. I have it running on a few 20 user account mail servers as well as one or two mail servers that have over 200 users on them. That is a cool idea about the sendmail rules.
 
Old 06-17-2012, 08:36 AM   #4
vahacker
LQ Newbie
 
Registered: Jun 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Sag47: I checked out fail2ban but it does a lot more stuff than my program. My program strickly monitors for ssh and user mining/spam attempts. The other difference I saw between the two programs is that mine seems to be easier to install setup and run. Two entries into cron and your set all though I suggest to go through the one file script that runs ccloth and make sure there is no need for little tweeks.

As far as the idea of adding rules to sendmail. I don't think that would fit in cheescloth because once it catches you everything is cut from that IP until it is rotated out.
 
Old 06-17-2012, 10:50 AM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,529
Blog Entries: 51

Rep: Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601
- You drop your rules in the filter table INPUT chain. Blocked addresses don't need to use up conntrack resources so they should go in the raw table PREROUTING chain.
- 'cat /var/log/w_messages |sed -n '/invalid user/{p;}'' is an example of a pipeline that probably could be solved differently (more efficiently?).
- You commonly delete files and create reporting in /usr/local/chees/. According to the FSSTND, FHS and all that we hold for true this should be down in /var.
* But most importantly: just because your README says "Every minute that cheescloth runs it scans the maillog file and the message file in /var/log. After the scan the data is both of these files is copied into maillog.1 and messages.1 . Syslogd is then reset. If you have other monitoring software that relys on either of these two file you need to point them to maillog.1 and messages.1" doesn't mean it's not completely wrong. If *you* want log lines to work with (if you can't come up with checkpointing) then *you* copy and use maillog.1 and messages.1 and not the other way around! Until you have fixed this issue I suggest people avoid cheescloth.
 
Old 06-17-2012, 10:53 AM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,529
Blog Entries: 51

Rep: Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601
Moved: This thread is more suitable in the Programming forum and has been moved accordingly to help your application get the reworking it requires.
 
Old 06-18-2012, 10:08 AM   #7
vahacker
LQ Newbie
 
Registered: Jun 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
- You drop your rules in the filter table INPUT chain. Blocked addresses don't need to use up conntrack resources so they should go in the raw table PREROUTING chain.
- 'cat /var/log/w_messages |sed -n '/invalid user/{p;}'' is an example of a pipeline that probably could be solved differently (more efficiently?).
- You commonly delete files and create reporting in /usr/local/chees/. According to the FSSTND, FHS and all that we hold for true this should be down in /var.
* But most importantly: just because your README says "Every minute that cheescloth runs it scans the maillog file and the message file in /var/log. After the scan the data is both of these files is copied into maillog.1 and messages.1 . Syslogd is then reset. If you have other monitoring software that relys on either of these two file you need to point them to maillog.1 and messages.1" doesn't mean it's not completely wrong. If *you* want log lines to work with (if you can't come up with checkpointing) then *you* copy and use maillog.1 and messages.1 and not the other way around! Until you have fixed this issue I suggest people avoid cheescloth.

Thanks for moving the thread where it needs to be unspawn.

1) Concerning the Block addresses using up the conntrack resources do you think it would make that big of deal? What do you feel would be the correct syntax to use to do it with the raw tables prerouting chain.

2) Concerning the invalid user pipeline I will look at this as well. The file I am greping through is so small I did not think it was that big of deal. That will change though with #4

3) Concerning all the work being done in /usr/local/chees . When I wrote the script for the first time I was copying and pasting console commands in the script that gave the end result I wanted. I agree this should be moved to /var/run. Do you suggest that I just check to see if the directories I want to store data in are there before I run the script and create them if they arent?

4) I am was not happy about the handling of the log files either and figured it would be an issue. Originally just written for myself but I need to correct this. I like your idea about checkpoints in those files and will make this change very shortly.


I appreciate you taking the time to write these suggestions out.
 
Old 06-18-2012, 01:27 PM   #8
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,328
Blog Entries: 33

Rep: Reputation: 328Reputation: 328Reputation: 328Reputation: 328
The ideas unSpawn shot past you may seem like small issues for *your* setup. However, not everyone runs a small-midrange setup. Where I currently work we run roughly 1000 servers and over half of them are RedHat Linux. For a very large installation, cheesecloth wouldn't be suitable for the reasons mentioned by unSpawn. Think thousands or tens of thousands of users daily. We also have centralized logging though this may not be too affected by your renaming the logs it is generally bad practice to do so.

This is why I mentioned fail2ban. It may be a bit more complicated but it's battle tested in a wide variety of environments. That's not to say that your program is no good it's just something to strive for (being enterprise ready).
 
Old 06-18-2012, 03:19 PM   #9
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,529
Blog Entries: 51

Rep: Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601Reputation: 2601
Quote:
Originally Posted by vahacker View Post
1) Concerning the Block addresses using up the conntrack resources do you think it would make that big of deal? What do you feel would be the correct syntax to use to do it with the raw tables prerouting chain.
It's the same as you use with the INPUT chain except iptables defaults to using the filter table so the only thing you have to do is change PREROUTING for INPUT and explicitly add "-t raw".


Quote:
Originally Posted by vahacker View Post
2) Concerning the invalid user pipeline I will look at this as well.
Well it's not about the validity of your approach but if it could be done differently or more efficiently. Output of cat|sed, cat|grep and other such pipes could be gotten with say awk -F ' ' '/search term/ {print $fieldnumber}' /path/to/file if you're looking for fixed field values. That's using one tool only but that doesn't necessarily mean it's quicker. Gotta test different approaches. In some cases the placement of a tools switches or the way you build a regex may matter wrt speed.


Quote:
Originally Posted by vahacker View Post
I agree this should be moved to /var/run. Do you suggest that I just check to see if the directories I want to store data in are there before I run the script and create them if they arent?
If you want to cache data persistently then you could use say /var/cache/cheesecloth, data that doesn't need to survive say a reboot could go in /var/run/cheesecloth and logs and reporting should be in /var/log/cheesecloth.


Quote:
Originally Posted by vahacker View Post
I like your idea about checkpoints in those files and will make this change very shortly.
Not that it would be easy to do. However thanks to TIS / Psionic / Ci$co you're free to use logtail as a dependency and avoid having to be creative ;-p
 
Old 06-21-2012, 11:04 AM   #10
vahacker
LQ Newbie
 
Registered: Jun 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
It's the same as you use with the INPUT chain except iptables defaults to using the filter table so the only thing you have to do is change PREROUTING for INPUT and explicitly add "-t raw".



Well it's not about the validity of your approach but if it could be done differently or more efficiently. Output of cat|sed, cat|grep and other such pipes could be gotten with say awk -F ' ' '/search term/ {print $fieldnumber}' /path/to/file if you're looking for fixed field values. That's using one tool only but that doesn't necessarily mean it's quicker. Gotta test different approaches. In some cases the placement of a tools switches or the way you build a regex may matter wrt speed.



If you want to cache data persistently then you could use say /var/cache/cheesecloth, data that doesn't need to survive say a reboot could go in /var/run/cheesecloth and logs and reporting should be in /var/log/cheesecloth.



Not that it would be easy to do. However thanks to TIS / Psionic / Ci$co you're free to use logtail as a dependency and avoid having to be creative ;-p



Thanks again for your advice. I am testing the new version now.

I changed #2 to just run sed instead of cat the file through the pipe.
I put all the tmp files in /var/run/cheescloth and the reports in /var/log/cheescloth. I didnt not change any code for these directories. I only put symbolic links in where the directories use to be in /usr/local/chees. That isn't a problem is it?
The big change is that I no longer mess with the message or the maillog file in the sense of renaming and restarting syslogd. I generate a 25 chr random string now and echo that into those 2 files. cat /dev/urandom| tr -dc '0-9a-zA-Z'|head -c 25 > info/chkpoint . When I go to look the next time at these files I only look at the data from this point down and then inject a new chk point. I gave you kutos in the docs for the idea.

Did you see anything else that might need to be changed or improved before I put this thing out again?

Thanks again
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off



All times are GMT -5. The time now is 07:06 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration