fail2ban pattern matching problem

gosssamer · 05-13-2014, 11:18 PM

Hi, I've been struggling with fail2ban on fedora20 for a while, and need some help. I'm trying to match the following line:

May 12 22:00:20 [MASS_MAILING] user1 (mydomain.com) from 1.2.3.4: Total 31 recipients

It is part of squirrelmail_logger, and intended to alert to when a message is sent to a large number of recipients, as might happen when an account has been compromised.

My failregex pattern is, very simply:

failregex = .*from <HOST>: Total.*$

When I run fail2ban-regex, my pattern matches, but when I cat the above line into the file being monitored, it doesn't match. If I strip off everything except for "from 1.2.3.4: Total 31" and then append it to the file being monitored, it matches.

What's wrong with my above line that causes it to not match, but something less does?

Does it have something to do with the date? I've tried setting timeregex and timepattern:

timeregex = \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}
timepattern = %%b %%d %%H:%%M:%%S

and that doesn't seem to matter either.

I'd really appreciate any input you may have.

Thanks,
Alex

Habitual · 05-14-2014, 03:16 PM

Quote:

Originally Posted by gosssamer

failregex = .*from <HOST>: Total.*$

Alex:
Try

Code:

failregex = ^<HOST> .*Total.*$

or a slight variation of that.

JJ

unSpawn · 05-14-2014, 03:46 PM

Works for me. Please return output of

Code:

fail2ban-regex /path/to/actual_log_file "MASS_MAILING.*from <HOST>:.*$"

gosssamer · 05-19-2014, 03:12 PM

Quote:

Originally Posted by unSpawn

Works for me. Please return output of

Code:

fail2ban-regex /path/to/actual_log_file "MASS_MAILING.*from <HOST>:.*$"

Code:

# fail2ban-regex /var/lib/squirrelmail/data/squirrelmail_access_log "MASS_MAILING.*from <HOST>:.*$"

Running tests
=============

Use   failregex line : MASS_MAILING.*from <HOST>:.*$
Use         log file : /var/lib/squirrelmail/data/squirrelmail_access_log
Use         encoding : UTF-8


Results
=======

Failregex: 34 total
|-  #) [# of hits] regular expression
|   1) [34] MASS_MAILING.*from <HOST>:.*$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [6312] MON Day 24hour:Minute:Second
`-

Lines: 6319 lines, 0 ignored, 34 matched, 6285 missed
Missed line(s):: too many to print.  Use --print-all-missed to print all 6285 lines

It works just fine when run this way. I haven't had an actual event occur that would provide a real-world request, so I've tried to append a similar line into the log file:

Code:

# cat /tmp/testfile
May 15 08:46:57 [MASS_MAILING] user1 (myexample.com) from 162.248.210.167: Total 76 recipients (FROM: inf
o@support.tech.net) (SUBJECT: Deactivation Of Your Webmail Account.)

# cat /tmp/testfile >> /var/lib/squirrelmail/data/squirrelmail_access_log

and I've sent it a few times, and nothing is written to the log file. However, if I remove the date info "May 15 08:46:57" then cat it into the log file, it does indeed now trigger it. It also works if I supply the filter filename to fail2ban-regex.

The full contents of my massmail.conf filter file is:

Code:

# cat massmailing.conf |egrep -v '^$|^#'
[Definition]
timeregex = \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}
timepattern = %%b %%d %%H:%%M:%%S
failregex  = .*from <HOST>: Total.*$
ignoreregex =

It has no effect if I remove the timeregex or timepattern variable.

Thanks for any ideas.
Alex

unSpawn · 05-20-2014, 01:14 AM

Quote:

Originally Posted by gosssamer

It works just fine when run this way. I haven't had an actual event occur that would provide a real-world request, so I've tried to append a similar line into the log file (..) and I've sent it a few times, and nothing is written to the log file. However, if I remove the date info "May 15 08:46:57" then cat it into the log file, it does indeed now trigger it. It also works if I supply the filter filename to fail2ban-regex.

Odd.

Quote:

Originally Posted by gosssamer

The full contents of my massmail.conf filter file is:

Code:

# cat massmailing.conf |egrep -v '^$|^#'
[Definition]
timeregex = \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}
timepattern = %%b %%d %%H:%%M:%%S
failregex  = .*from <HOST>: Total.*$
ignoreregex =

Only difference with the test I can see is you've dropped the "MASS_MAILING" part?..

Quote:

Originally Posted by gosssamer

It has no effect if I remove the timeregex or timepattern variable.

It shouldn't. These are in default includes and your log file shows the standard syslog time stamp. So there's no need to define time(regex|pattern) again.

gosssamer · 05-20-2014, 09:37 PM

Quote:

Originally Posted by unSpawn

Odd.

Only difference with the test I can see is you've dropped the "MASS_MAILING" part?..

It shouldn't. These are in default includes and your log file shows the standard syslog time stamp. So there's no need to define time(regex|pattern) again.

Thanks for your help. Looks like we'll just have to wait and see what happens next time an account is hacked. I'll try and follow up then.

Thanks,
Alex

unSpawn · 05-21-2014, 01:13 AM

Quote:

Originally Posted by gosssamer

Looks like we'll just have to wait and see what happens next time an account is hacked. I'll try and follow up then.

If the app allows for per-account settings you could create a new account (please don't name it "test" and use a simple passwd) and force the threshold to be crossed and so force a log message?

gosssamer · 06-13-2014, 10:11 AM

Quote:

Originally Posted by unSpawn

If the app allows for per-account settings you could create a new account (please don't name it "test" and use a simple passwd) and force the threshold to be crossed and so force a log message?

Following up with this from last month.

Turns out my rules worked as expected. Not sure why I couldn't create a fake entry by inserting a line into messages. It would still be really nice to know, so I can complete a full test.

unSpawn · 06-13-2014, 11:50 AM

Like I said before your rule worked for me. Since I'm not able to reproduce the error and since no new information was made available that could help shed a different light on things I'm not sure what I could help you further with?..

gosssamer · 06-13-2014, 01:15 PM

Yes, fail2ban-regex worked here too - it was just testing an actual occurrence by simulating it in the log file that I was trying to do.

What's the difference between echo'ing a pattern that should trigger the event and squirrelmail producing the same event in the log file?