LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 04-21-2017, 10:21 PM   #1
r.stiltskin
Member
 
Registered: Oct 2003
Location: USA
Distribution: Xubuntu, Arch
Posts: 231

Rep: Reputation: 31
fail2ban filters


[Arch Linux, fail2ban 0.9.6]

The default fail2ban filters do find log entries containing strings like
"not allowed because not listed in AllowUsers"
and ban the offending ip addresses when appropriate, but they're not finding multiple occurrences of entries like
Code:
Connection closed by 76.164.165.233 port 50297 [preauth]
or
Code:
Did not receive identification string from 105.106.130.52 port 52759
or
Code:
Unable to negotiate with 40.86.181.208 port 55996: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
I find many occurrences of lines like those within the specified findtime of 600 seconds in the output of journalctl -t sshd, but they don't appear in the fail2ban log.

I'm not sure what those entries represent. (Needless to say, they're not from valid users of my server.) Is there a reason why filter rules to catch them are not provided?
 
Old 04-22-2017, 01:30 AM   #2
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 16,897
Blog Entries: 10

Rep: Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026
Quote:
Originally Posted by r.stiltskin View Post
I find many occurrences of lines like those within the specified findtime of 600 seconds in the output of journalctl -t sshd, but they don't appear in the fail2ban log.
fail2ban is pretty simple - it parses log files (and i don't think it parses the systemd journal!) and looks for entries, and makes decisions based upon how often these entries appear and which IP the action originates from. configurable. has to be explicitely enabled for each application you're using.

so, 2 things to check:
- is a fail2ban configuration for sshd active?
- does sshd actually log to the files that fail2ban is looking for?
 
Old 04-22-2017, 09:22 AM   #3
r.stiltskin
Member
 
Registered: Oct 2003
Location: USA
Distribution: Xubuntu, Arch
Posts: 231

Original Poster
Rep: Reputation: 31
Sorry, I guess I didn't state my question clearly. I'll try again.

fail2ban is "working" as it is currently configured. It is monitoring sshd activity and does occasionally ban/unban an ip after a series of unsuccessful login attempts.

But in my logs I find sequences of other types of messages that I find concerning but which are not caught by any of the preconfigured fail2ban filters, such as:
Code:
Apr 21 22:23:07 darth sshd[20601]: Connection closed by 76.164.165.233 port 36709 [preauth]
Apr 21 22:25:22 darth sshd[20610]: Connection closed by 76.164.165.233 port 34574 [preauth]
Apr 21 22:27:36 darth sshd[20617]: Connection closed by 76.164.165.233 port 60747 [preauth]
Apr 21 22:29:51 darth sshd[20632]: Connection closed by 76.164.165.233 port 58626 [preauth]
Apr 21 22:32:06 darth sshd[20640]: Connection closed by 76.164.165.233 port 56484 [preauth]
Apr 21 22:34:21 darth sshd[20678]: Connection closed by 76.164.165.233 port 54383 [preauth]
Apr 21 22:36:36 darth sshd[20688]: Connection closed by 76.164.165.233 port 52297 [preauth]
Apr 21 22:38:51 darth sshd[20726]: Connection closed by 76.164.165.233 port 50297 [preauth]
and
Code:
Apr 21 15:01:45 darth sshd[18901]: Did not receive identification string from 105.106.130.52 port 52747
Apr 21 15:01:46 darth sshd[18902]: Did not receive identification string from 105.106.130.52 port 52754
Apr 21 15:01:48 darth sshd[18903]: Did not receive identification string from 105.106.130.52 port 52759
Apr 21 15:01:52 darth sshd[18906]: Did not receive identification string from 105.106.130.52 port 52775
Apr 21 15:02:00 darth sshd[18907]: Did not receive identification string from 105.106.130.52 port 52801
and
Code:
Apr 18 11:52:04 darth sshd[5571]: Unable to negotiate with 40.86.181.208 port 53698: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:05 darth sshd[5573]: Unable to negotiate with 40.86.181.208 port 53753: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:07 darth sshd[5575]: Unable to negotiate with 40.86.181.208 port 53802: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:08 darth sshd[5577]: Unable to negotiate with 40.86.181.208 port 53853: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:09 darth sshd[5579]: Unable to negotiate with 40.86.181.208 port 53900: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:10 darth sshd[5581]: Unable to negotiate with 40.86.181.208 port 53960: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:11 darth sshd[5583]: Unable to negotiate with 40.86.181.208 port 54009: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:12 darth sshd[5585]: Unable to negotiate with 40.86.181.208 port 54063: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:13 darth sshd[5587]: Unable to negotiate with 40.86.181.208 port 54117: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
Apr 18 11:52:14 darth sshd[5589]: Unable to negotiate with 40.86.181.208 port 54181: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 [preauth]
While they aren't exactly login attempts, they seem to be up to no good, but I don't understand exactly what their purpose is. So I guess I really have two questions:

Exactly what are these guys trying to do?
and
Is it important to write custom fail2ban filters to catch and ban them?
 
Old 04-22-2017, 12:13 PM   #4
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 16,897
Blog Entries: 10

Rep: Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026
they're probably trying to hack you.
do these IP addresses say anything to you?
 
Old 04-22-2017, 12:22 PM   #5
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
ssh used to be enabled by default.
Should double-check it.
Code:
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/ssh.conf
is the command-line method of testing a filter.
 
Old 04-22-2017, 12:33 PM   #6
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 16,897
Blog Entries: 10

Rep: Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026
Quote:
Originally Posted by Habitual View Post
Code:
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf
for me it was sshd...
 
Old 04-22-2017, 04:28 PM   #7
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by ondoho View Post
for me it was sshd...
Yer right. ssh would be the client side, and there's nothing to see there, move along.

d is for daemon and I missed it.
 
Old 04-22-2017, 05:52 PM   #8
r.stiltskin
Member
 
Registered: Oct 2003
Location: USA
Distribution: Xubuntu, Arch
Posts: 231

Original Poster
Rep: Reputation: 31
Quote:
Originally Posted by ondoho View Post
they're probably trying to hack you.
do these IP addresses say anything to you?
Nope. As I said in post #1, they aren't legit users of my server. I'm just trying to understand exactly what kind of attacks they represent, assuming that they are attacks.

And also wondering why there are no filter rules to pick up these messages included in the fail2ban release, given that these messages are certainly not unusual.
 
Old 04-22-2017, 08:09 PM   #9
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by r.stiltskin View Post
Nope. As I said in post #1, they aren't legit users of my server. I'm just trying to understand exactly what kind of attacks they represent, assuming that they are attacks.

And also wondering why there are no filter rules to pick up these messages included in the fail2ban release, given that these messages are certainly not unusual.
The usual "noise" for sshd, IMO. I consider it like "door bell ditch". It's annoying. Hopefully we can help.

There is one included and tons of examples, but it helps to understand the way it works first.
Code:
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd-ddos.conf
I show /etc/fail2ban/filter.d/sshd-ddos.conf has a good starting point as it has at least one of our target strings.
Code:
grep -e identification -e closed -e exchange /etc/fail2ban/filter.d/*
/etc/fail2ban/filter.d/sshd-ddos.conf:#  "Did not receive identification string".
/etc/fail2ban/filter.d/sshd-ddos.conf:failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$
Let's kick some tires!
Code:
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd-ddos.conf
You may have to edit /etc/fail2ban/filter.d/sshd-ddos.conf manually and add some regular expressions and test until it hits,
which is all I did here below.
then enable the jail after showing that it works as desired.
It does require intervention, but once you have it "dialed in", you will love it and rely on it as I do.

Now, our work...

I have modified the 0.8.1x version of /etc/fail2ban/filter.d/sshd-ddos.conf here locally and used
your supplied logs.
23 entries. 23 hits using this

Fair Warning
Warning: My RE.fu is near useless but I did manage a "result" here.

Maybe you are "that guy"?

Code:
failregex = ^%(__prefix_line)sDid not receive identification string from <HOST> .*?
            ^%(__prefix_line)sConnection closed by <HOST>.*?
            ^%(__prefix_line)sUnable to negotiate with <HOST>.*?
I see theres some delimiter? missing to match the )s, it was it looks like to this dumb bunny. Anyway...

You can create a new /etc/fail2ban/filter.d/custom.conf (for example)
and populate it using this modified copy of /etc/fail2ban/filter.d/sshd-ddos.conf Save it as /etc/fail2ban/filter.d/custom.conf
once in place, test it.

Code:
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/custom.conf
should now provide you with similar results.

Let us know.
I haz a pdf of some nuggets, if you're more than casually interested.

After successful testing, enable the jail.
I suggest
Code:
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
and stick enabled jails only in jail.local.

This file is safe from fail2ban removal or upgrade, as are all custom_named /etc/fail2ban/action.d/ and /etc/fail2ban/filter.d/ configurations.

Subscribed with interest...

Last edited by Habitual; 04-22-2017 at 08:32 PM.
 
2 members found this post helpful.
Old 04-23-2017, 02:16 AM   #10
r.stiltskin
Member
 
Registered: Oct 2003
Location: USA
Distribution: Xubuntu, Arch
Posts: 231

Original Poster
Rep: Reputation: 31
Thanks for your help. It got a little more complicated because I'm working in Arch, which has no /var/log/auth.log. Arch uses the systemd-journal.

Yeah, I realized after I posted earlier that I had a separate sshd-ddos.conf file and I hadn't enabled that jail. I have now enabled it. My sshd-ddos.conf contains just one regex -- similar to the one you found:
Code:
failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$
Notice the slightly different ending.

But it seems that fail2ban-regex ver 0.9.6 is broken for systemd-journal, so I can't test my version of sshd-ddos.conf against my actual journal. According to the manpage my syntax should be
Code:
sudo fail2ban-regex "systemd-journal" /etc/fail2ban/filter.d/sshd-ddos.conf
(or whatever the filename is that contains the filter regexes)...

but when I run it I get
Code:
Running tests
=============

Use   failregex filter file : sshd-ddos, basedir: /etc/fail2ban
Use    journal match : _SYSTEMD_UNIT=sshd.service + _COMM=sshd

Traceback (most recent call last):
  File "/usr/bin/fail2ban-regex", line 34, in <module>
    exec_command_line()
  File "/usr/lib/python3.6/site-packages/fail2ban/client/fail2banregex.py", line 599, in exec_command_line
    if not fail2banRegex.start(opts, args):
  File "/usr/lib/python3.6/site-packages/fail2ban/client/fail2banregex.py", line 538, in start
    self.process(test_lines)
  File "/usr/lib/python3.6/site-packages/fail2ban/client/fail2banregex.py", line 370, in process
    for line_no, line in enumerate(test_lines):
  File "/usr/lib/python3.6/site-packages/fail2ban/client/fail2banregex.py", line 93, in journal_lines_gen
    yield FilterSystemd.formatJournalEntry(entry)
TypeError: formatJournalEntry() missing 1 required positional argument: 'logentry'
so even though I have the right log argument for my system, it doesn't work. This is a bug that I think was fixed upstream but hasn't gotten into the Arch repo yet & I don't feel like patching my own.

So I set up a dummy auth.log containing entries like I posted earlier for testing. The regex I found in my sshd-ddos.conf file didn't catch anything -- I guess <HOST> doesn't include the port specification so the regex can't end with <HOST>/s*$

So I wrote this sshd-ddos.local file (in /etc/fail2ban/filter.d):
Code:
## sshd-ddos.local
## added regexes to catch some more exploits that I found in the systemd-journal


[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf

[Definition]

_daemon = sshd

failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>(?: port \d+)\s*$
            ^%(__prefix_line)sConnection closed by <HOST>(?: port \d+) \[preauth\]\s*$
            ^%(__prefix_line)sUnable to negotiate with <HOST>(?: port \d+): no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 \[preauth\]\s*$

ignoreregex = 

[Init]

journalmatch = _SYSTEMD_UNIT=sshd.service + _COMM=sshd
I tested that against the dummy auth.log and these regexes seem to work; time will tell if they also work against my actual journal.

I saved this file as /etc/fail2ban/jail.d/defaults-arch.conf to enable the jails:
Code:
# this file was copied from ArchWiki https://wiki.archlinux.org/index.php/fail2ban
# I changed maxretry to 3 instead of 5 and eliminated sendmail-whois
# and commented out the action line (restoring default action iptables-multiport defined in jail.conf
[sshd]
enabled  = true
filter   = sshd
#action   = iptables[name=SSH, port=ssh, protocol=tcp]
#           sendmail-whois[name=SSH, dest=your@mail.org, sender=fail2ban@mail.com]
backend  = systemd
maxretry = 3


[sshd-ddos]
enabled  = true
filter   = sshd-ddos
# not sure if backend entry is needed here - backend is defined as sshd_backend in jail.conf
# and sshd_backend is defined as default_backend in paths-common.conf
# and default_backend is defined in paths_common.conf as auto
# but I guess it can't hurt:
backend  = systemd
maxretry = 3
Restarted fail2ban and now I'll have to wait and see what happens.

Thanks again.
 
Old 04-23-2017, 03:06 AM   #11
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 16,897
Blog Entries: 10

Rep: Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026Reputation: 5026
^ so fail2ban can actually work with systemd's journal? wow, i didn't know that. good stuff.
 
Old 04-23-2017, 04:44 AM   #12
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Well, seems like your run and mine had the same result.

High volume, I sit on
Code:
tail -f /var/log/fail2ban.log
v10 now with IPv6 support.

Last edited by Habitual; 04-23-2017 at 04:45 AM.
 
Old 06-12-2017, 03:58 PM   #13
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
How did this work out, I wonder?
 
Old 06-13-2017, 07:22 AM   #14
r.stiltskin
Member
 
Registered: Oct 2003
Location: USA
Distribution: Xubuntu, Arch
Posts: 231

Original Poster
Rep: Reputation: 31
It's working well. The regexes I wrote in sshd-ddos.local, along with the built-in ones in sshd.conf find almost all of the login attempts, and fail2ban bans them when they reach my threshold of 3 attempts. The only ones that aren't found are the ones that look like this:
Code:
Jun 11 18:35:55 darth sshd[15293]: Received disconnect from 52.178.134.151 port 40378:11: Bye Bye [preauth]
Jun 11 18:35:55 darth sshd[15293]: Disconnected from 52.178.134.151 port 40378 [preauth]
They occur so infrequently that I decided not to bother adding a regex for them.

It's worth mentioning that I also ban (DROP) most of the world altogether so I only log a handful of bogus login attempts daily. I did that by adding this line
Code:
cat /etc/fail2ban/iptables-blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done
to the
actionstart section of iptables-multiport.local.

The file /etc/fail2ban/iptables-blacklist is a plain-text list of ip address ranges that cover regions where no valid logins will be coming from. What this does is to add a Chain f2b-sshd-ddos containing all the listed ip addresses to my iptables each time fail2ban service is started.
 
Old 06-13-2017, 08:39 AM   #15
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by r.stiltskin View Post
It's working well. The regexes I wrote in sshd-ddos.local, along with the built-in ones in sshd.conf find almost all of the login attempts, and fail2ban bans them when they reach my threshold of 3 attempts. The only ones that aren't found are the ones that look like this:
Code:
Jun 11 18:35:55 darth sshd[15293]: Received disconnect from 52.178.134.151 port 40378:11: Bye Bye [preauth]
Jun 11 18:35:55 darth sshd[15293]: Disconnected from 52.178.134.151 port 40378 [preauth]
They occur so infrequently that I decided not to bother adding a regex for them.

It's worth mentioning that I also ban (DROP) most of the world altogether so I only log a handful of bogus login attempts daily. I did that by adding this line
Code:
cat /etc/fail2ban/iptables-blacklist | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done
to the
actionstart section of iptables-multiport.local.

The file /etc/fail2ban/iptables-blacklist is a plain-text list of ip address ranges that cover regions where no valid logins will be coming from. What this does is to add a Chain f2b-sshd-ddos containing all the listed ip addresses to my iptables each time fail2ban service is started.
Thanks for letting us know. I used to utilize that method of persistent "bad.guys" file for restarts.
Not much traffic, so we recently rolled back a few rules. Levelled off at 6k from 9, Irritants are about the same pace.
PreAuth = Doorbell Ditch
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Fail2ban help glennbtn Linux - Software 1 11-04-2014 09:05 AM
Fail2ban help glennbtn Linux - Software 1 07-02-2014 08:40 AM
Fail2Ban Patrick032986 Linux - Software 7 09-18-2011 04:01 PM
I need help with fail2ban... trist007 Linux - Newbie 15 12-14-2009 03:22 AM
regarding filters zameer_india Linux - Newbie 1 02-05-2005 03:47 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 06:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration