LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
Reload this Page Fail2Ban regex for EXIM (TCP/IP connection count)
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-02-2014, 03:26 AM   #1
bms8197
LQ Newbie
 
Registered: Dec 2014
Location: Bucharest
Distribution: CentOS
Posts: 5

Rep: Reputation: 0
Fail2Ban regex for EXIM (TCP/IP connection count)

I am trying to create a regex condition for the exim filter of Fail2Ban. In my exim log, I have entries like this:

2014-11-27 17:09:05 SMTP connection from [42.117.255.244] (TCP/IP connection count = 1)
2014-11-27 17:09:14 SMTP connection from [118.68.249.18] (TCP/IP connection count = 5)
2014-11-27 17:09:15 SMTP connection from [113.188.85.220] (TCP/IP connection count = 2)
2014-11-27 17:09:15 SMTP connection from [113.188.85.220] (TCP/IP connection count = 7)
2014-11-27 17:09:15 SMTP connection from [113.188.85.220] (TCP/IP connection count = 4)

So I need a regex filter which analyzes the exim log, and if TCP/IP connection count > 3, then fail2ban will block that ip for the amount of time specified in fail2ban configuration.

What I have tried so far is something like this:

I came up with something like this:

[ \S]+?SMTP connection from \S+? \(TCP\/IP connection count = (?!\b1\b|\b2\b|\b3\b)\d+\) (tested on regex101.com and works perfectly but in fail2ban returns error).

I also tried something like this:

[ \S]+?SMTP connection from \[\S+\]? \(TCP\/IP connection count = ([2-9]|1[0-9])\) (to match only the entries where connection count >= 2 .

For some reason, in fail2ban-regex is not working.

What I manage to came up with in the end is:
fail2ban-regex /var/log/exim_mainlog " \[<HOST>\] .* +?SMTP connection from \[\S+\]? \(TCP\/IP connection count = ([2-9]|1[0-9])\)/gm"

I get no error from fail2ban-regex but it misses all entries. Here is the output of the command above:

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
| [244763] Year-Month-Day Hour:Minute:Second
`-

Lines: 244763 lines, 0 ignored, 0 matched, 244763 missed


Any help would be appreciated.

Thank you!
Last edited by bms8197; 12-02-2014 at 03:29 AM.
 
Old 12-02-2014, 08:35 AM   #2
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
/etc/fail2ban/filter.d/exim.conf ?
 
Old 12-02-2014, 08:54 AM   #3
bms8197
LQ Newbie
 
Registered: Dec 2014
Location: Bucharest
Distribution: CentOS
Posts: 5

Original Poster
Rep: Reputation: 0
I know about /etc/fail2ban/exim.conf and there also other exim filters like exim-relay and exim-spam. None of them has that regex.
 
Old 12-02-2014, 11:06 AM   #4
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by bms8197 View Post
I also tried something like this:

[ \S]+?SMTP connection from \[\S+\]? \(TCP\/IP connection count = ([2-9]|1[0-9])\) (to match only the entries where connection count >= 2 .
I am not sure how or if that can be done with fail2ban.
I like you, am not any good at regex (that is you, yes?)

I'd use
Code:
maxretry=3
in your jail.local to match for >= 3
and loosen the regex to scan for only failures and let the maxretry handle the match >=3

in any event the regex gurus here could use more of the actual log entries and not just some of the "entries like this" as they don't show the failures.

Good luck and let us know.
 
Old 12-02-2014, 03:39 PM   #5
bms8197
LQ Newbie
 
Registered: Dec 2014
Location: Bucharest
Distribution: CentOS
Posts: 5

Original Poster
Rep: Reputation: 0
The exim log is something like that:


2014-11-30 03:12:42 SMTP connection from [168.167.250.118] (TCP/IP connection count = 1)
2014-11-30 03:12:44 no host name found for IP address 168.167.250.118
2014-11-30 03:12:48 SMTP connection from (ANDREW) [168.167.250.118] closed by QUIT
2014-11-30 03:12:51 SMTP connection from [168.167.250.118] (TCP/IP connection count = 1)
2014-11-30 03:12:51 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:12:52 no host name found for IP address 168.167.250.118
2014-11-30 03:12:52 no host name found for IP address 168.167.250.118
2014-11-30 03:12:52 SMTP connection from [168.167.250.118] (TCP/IP connection count = 3)
2014-11-30 03:12:53 no host name found for IP address 168.167.250.118
2014-11-30 03:12:54 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:54 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:55 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:55 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:55 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:56 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:56 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:56 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:56 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:56 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:56 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=account)
2014-11-30 03:12:57 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:57 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=account)
2014-11-30 03:12:57 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:57 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:12:57 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:12:58 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:12:58 SMTP connection from [168.167.250.118] (TCP/IP connection count = 3)
2014-11-30 03:12:58 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=account)
2014-11-30 03:12:59 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:12:59 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:06 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=account)
2014-11-30 03:13:06 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:13:07 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:07 no host name found for IP address 168.167.250.118
2014-11-30 03:13:08 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:13:09 no host name found for IP address 168.167.250.118
2014-11-30 03:13:10 SMTP connection from [168.167.250.118] (TCP/IP connection count = 3)
2014-11-30 03:13:11 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 03:13:11 no host name found for IP address 168.167.250.118
2014-11-30 03:13:12 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=admin)
2014-11-30 03:13:13 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:14 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=admin)
2014-11-30 03:13:15 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:15 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:13:16 no host name found for IP address 168.167.250.118
2014-11-30 03:13:16 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=admin)
2014-11-30 03:13:17 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:21 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:13:22 no host name found for IP address 168.167.250.118
2014-11-30 03:13:22 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=admin)
2014-11-30 03:13:23 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:27 courier_login authenticator failed for (ANDREW) [168.167.250.118]: 535 Incorrect authentication data (set_id=admin)
2014-11-30 03:13:27 SMTP connection from (ANDREW) [168.167.250.118] lost
2014-11-30 03:13:30 SMTP connection from [168.167.250.118] (TCP/IP connection count = 1)
2014-11-30 03:13:31 no host name found for IP address 168.167.250.118
2014-11-30 03:13:32 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
2014-11-30 03:13:32 SMTP connection from [168.167.250.118] (TCP/IP connection count = 3)
2014-11-30 03:13:33 Connection from [168.167.250.118] refused: too many connections from that IP address
2014-11-30 11:08:54 SMTP connection from [81.181.78.25] (TCP/IP connection count = 1)
2014-11-30 11:09:20 H=(g.advertising-iq.com) [81.181.78.25] Warning: Sender rate 1.4 / 1h
2014-11-30 11:09:20 H=(g.advertising-iq.com) [81.181.78.25] X=TLSv1HE-RSA-AES256-SHA:256 F=<www-data@g.advertising-iq.com> temporarily rejected RCPT <20bampi@anywhere.ro>: host lookup deferred for reverse lookup check
2014-11-30 11:09:20 SMTP connection from (g.advertising-iq.com) [81.181.78.25] closed by QUIT
2014-11-30 11:09:20 SMTP connection from [81.181.78.25] (TCP/IP connection count = 1)
2014-11-30 11:09:49 H=(g.advertising-iq.com) [81.181.78.25] Warning: Sender rate 2.4 / 1h
2014-11-30 11:09:49 H=(g.advertising-iq.com) [81.181.78.25] X=TLSv1HE-RSA-AES256-SHA:256 F=<www-data@g.advertising-iq.com> temporarily rejected RCPT <20bampi@anywhere.ro>: host lookup deferred for reverse lookup check
2014-11-30 11:09:49 SMTP connection from (g.advertising-iq.com) [81.181.78.25] closed by QUIT
2014-11-30 11:10:01 cwd=/etc/exim/stats 2 args: /usr/sbin/exim -bp
2014-11-30 11:12:41 cwd=/ 2 args: exim -bpc
2014-11-30 11:12:41 cwd=/ 2 args: exim -bpc

Your advice with max retry = 3 is pretty cool, the ideea is that I am not able to manage fail2ban to identify these lines from exim's log file:

2014-11-30 11:08:54 SMTP connection from [81.181.78.25] (TCP/IP connection count = 1)

Basically, the regex should be able to detect this lines and ban that ip address if maxretry = 3 (just for testing purposes)
 
Old 12-02-2014, 03:56 PM   #6
bms8197
LQ Newbie
 
Registered: Dec 2014
Location: Bucharest
Distribution: CentOS
Posts: 5

Original Poster
Rep: Reputation: 0
Red face
I have tested the following regex on regexr.com and works perfectly fine:

\S+ \S+ SMTP connection from \[\S+\]? \(TCP\/IP connection count = ([2-9]|1[0-9])\)

It perfectly identifies the lines:
2014-11-30 03:12:51 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2) (where count >= 2).

If I try to validate it using fail2ban regex, I get the following error (and I don't really get it!):

29/25030# fail2ban-regex /var/log/exim_mainlog "\S+ \S+ SMTP connection from \[\S+\]? \(TCP\/IP connection count = ([2-9]|1[0-9])\)"

Running tests
=============

Use failregex line : \S+ \S+ SMTP connection from \[\S+\]? \(TCP\/IP co...
Traceback (most recent call last):
File "/usr/bin/fail2ban-regex", line 430, in ?
fail2banRegex.readRegex(cmd_regex, 'fail') or sys.exit(-1)
File "/usr/bin/fail2ban-regex", line 227, in readRegex
'add%sRegex' % regextype.title())(regex.getFailRegex())
File "/usr/share/fail2ban/server/filter.py", line 131, in addFailRegex
raise e
server.failregex.RegexException: No 'host' group in '\S+ \S+ SMTP connection from \[\S+\]? \(TCP\/IP connection count = ([2-9]|1[0-9])\)'
mem: 1232/3261 MB load: 2.01 1.20 1.37 procs: 119
root@zira in /etc/fail2ban (7 entries, 0 hidden)
30/25031#

I tried to replace \S+ \S+ with \[<HOST>\] +?, the regex returns no error, but instead it misses all the lines!

I think I'm missing something!!!
 
Old 12-02-2014, 04:16 PM   #7
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Does
Code:
fail2ban-regex /var/log/exim_mainlog /etc/fail2ban/filter.d/exim.conf
show any hits?

I used your data and stuck it in /tmp/test.log and ran the regex like so:
Code:
fail2ban-regex /tmp/test.log /etc/fail2ban/filter.d/exim.conf
and got 9 hits
Success, the total number of match is 9
Code:
Results
=======

Failregex: 9 total
|- #) [# of hits] regular expression
|  2) [9] ^( \[\d+\])? \w+ authenticator failed for (\S+ )?\(\S+\) \[<HOST>\]: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
This is on Fail2Ban v0.8.10
Since you posted data shows the error as "535 Incorrect authentication data" I think the /etc/fail2ban/filter.d/exim.conf
is the way to go and set maxretry at or near 2.

What version are you using?
Code:
fail2ban-client --version | head -1
Last edited by Habitual; 12-02-2014 at 04:19 PM.
 
Old 12-03-2014, 10:15 AM   #8
bms8197
LQ Newbie
 
Registered: Dec 2014
Location: Bucharest
Distribution: CentOS
Posts: 5

Original Poster
Rep: Reputation: 0
58/25059# fail2ban-regex /var/log/exim_mainlog.1 /etc/fail2ban/filter.d/exim.conf

Results
=======

Failregex: 52 total
|- #) [# of hits] regular expression
| 2) [12] ^( \[\d+\])? \w+ authenticator failed for (\S+ )?\(\S+\) \[<HOST>\]: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
| 3) [40] ^( \[\d+\])? H=([\w.-]+ )?(\(\S+\) )?\[<HOST>\](:\d+)? (I=\[\S+\]:\d+ )?(U=\S+ )?(P=e?smtp )?F=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (relay not permitted|Sender verify failed|Unknown user)\s*$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
| [36573] Year-Month-Day Hour:Minute:Second
`-

Lines: 36573 lines, 0 ignored, 52 matched, 36521 missed

I am running: 62/25063# fail2ban-regex --version
fail2ban-regex 0.8.14

It works, but as I was saying, the exim.conf file that comes with fail2ban is not exactly what I need.

I need to specifically search for these lines in the exim log file:
2014-11-30 03:12:51 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
 
Old 12-03-2014, 10:59 AM   #9
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by bms8197 View Post
It works, but as I was saying, the exim.conf file that comes with fail2ban is not exactly what I need.

I need to specifically search for these lines in the exim log file:
2014-11-30 03:12:51 SMTP connection from [168.167.250.118] (TCP/IP connection count = 2)
Excuse me, but if "535 Incorrect authentication data" is the error how is exim.conf not what you need?

There are other ways to rate-limit SMTP connections. Perhaps you should look into that?

I'm done here.
Have a Good Day.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] fail2ban regex help needed ! papampi Linux - Security 30 06-19-2012 10:29 AM
[SOLVED] fail2ban does not ban, maybe my regex is wrong? JeanC Linux - Server 2 03-17-2011 11:01 AM
Fail2ban regex help please wvroger Linux - Security 1 05-23-2010 08:30 PM
Fail2ban and Dovecot Regex kevinslair Linux - Software 3 05-31-2009 09:19 PM
Need help with fail2ban regex jakev383 Linux - Security 6 12-07-2008 10:35 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:16 AM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration