LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 07-15-2018, 05:09 PM   #1
teeps
LQ Newbie
 
Registered: Jul 2018
Posts: 3

Rep: Reputation: Disabled
Not really sure what is going on, possibly compromised?


Ok, long story short, I created an SSH server to interface with my intranetwork, I was an idiot and made the password pretty simple, a couple weeks after booting up the SSH server I decided to browse my logs and there were thousands of failed password attempts in my auth.log file. After some research I setup some basic IPtables rules and the failed password attempts almost completely went away.


-N SSHATTACK
-A SSHATTACK -j LOG --log-prefix "Possible SSH Attack! " --log-level 7
-A SSHATTACK -j DROP
-A INPUT -i wlan0 -p tcp -m state --dport 22 --state NEW -m recent --set
-A INPUT -i wlan0 -p tcp -m state --dport 22 --state NEW -m recent --update --seconds 120 --hitcount 2 -j SSHATTACK


So this worked, at least it seemed, I set up the rules to block incoming traffic into port 22 and only allow for one password attempt before blocking the IP address. Problem is, I'm noticing some strange activity in the syslog file, I'm still getting hundreds of warnings from individual IP addresses... How is this possible???


Jul 15 10:38:34 pi1 kernel: [134565.202051] Possible SSH Attack! IN=wlan0 OUT= MAC=b8:27:eb:4b:d5:0f:38:d5:47:dc:60:20:08:00 SRC=61.177.172.86 DST=192.168.50.250 LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=49184 DF PROTO=TCP SPT=29499 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0


I've got at least 200 "Possible SSH Attack!" log warnings from this single IP address, with hundreds more from a small list of other IP addresses as posted above. Here's whats got so confused, there are no correlating "Failed password" attempts in the auth.log file. The IP addresses in the syslog file that are attempting hundreds of passwords every minute don't even show up in the auth.log whatsoever.

My thoughts are as follows...

Someone has access to my system and the syslog is logging command inputs as SSH traffic from an IP address that should be blocked, therefor its not logged as a failed password attempt in auth.log.

---OR---

This is something totally benign and I have nothing to worry about but I've got a lot to learn about how linux works

Either way I'm confused, and any help would be appreciated.
 
Old 07-15-2018, 05:19 PM   #2
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,727

Rep: Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211
I vote for confused Let me recap what I think you said:
1. You observed many failed login attempts on port 22
2. You added iptable rules to a. Log the attempts and b. DROP the connection.
3. You no longer see failed login attempts (because you've dropped them!)
4. You see many log entries about the dropped connections

In short, you dropped the connections and configured to log the drops. You no longer see connection attempts, but you now see the log entries you configured.

Did I get that right?
It looks to me like your iptables rule is doing what you wanted configured it to do.

Last edited by scasey; 07-15-2018 at 05:38 PM.
 
2 members found this post helpful.
Old 07-15-2018, 05:19 PM   #3
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,634

Rep: Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965
Quote:
Originally Posted by teeps View Post
Ok, long story short, I created an SSH server to interface with my intranetwork, I was an idiot and made the password pretty simple, a couple weeks after booting up the SSH server I decided to browse my logs and there were thousands of failed password attempts in my auth.log file. After some research I setup some basic IPtables rules and the failed password attempts almost completely went away.
Code:
-N SSHATTACK
-A SSHATTACK -j LOG --log-prefix "Possible SSH Attack! " --log-level 7
-A SSHATTACK -j DROP
-A INPUT -i wlan0 -p tcp -m state --dport 22 --state NEW -m recent --set
-A INPUT -i wlan0 -p tcp -m state --dport 22 --state NEW -m recent --update --seconds 120 --hitcount 2 -j SSHATTACK
So this worked, at least it seemed, I set up the rules to block incoming traffic into port 22 and only allow for one password attempt before blocking the IP address. Problem is, I'm noticing some strange activity in the syslog file, I'm still getting hundreds of warnings from individual IP addresses... How is this possible???

Jul 15 10:38:34 pi1 kernel: [134565.202051] Possible SSH Attack! IN=wlan0 OUT= MAC=b8:27:eb:4b:d5:0f:38:d5:47:dc:60:20:08:00 SRC=61.177.172.86 DST=192.168.50.250 LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=49184 DF PROTO=TCP SPT=29499 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0

I've got at least 200 "Possible SSH Attack!" log warnings from this single IP address, with hundreds more from a small list of other IP addresses as posted above. Here's whats got so confused, there are no correlating "Failed password" attempts in the auth.log file. The IP addresses in the syslog file that are attempting hundreds of passwords every minute don't even show up in the auth.log whatsoever.

My thoughts are as follows...

Someone has access to my system and the syslog is logging command inputs as SSH traffic from an IP address that should be blocked, therefor its not logged as a failed password attempt in auth.log.

---OR---

This is something totally benign and I have nothing to worry about but I've got a lot to learn about how linux works
First thing I'd suggest looking at, is any geoip database. Putting the address in, it's coming from China.
https://www.maxmind.com/en/geoip-demo

Since your box was exposed, it's probably on a list-of-things-to-break-in-to, and they keep banging on it. If you are dropping that IP address, that's why you're not seeing any further "failed password" attempts.

If you're new to this, I'd suggest a few things to do immediately:
  • Change SSH to be a different port, something other than 22
  • DO NOT expose SSH to the outside world, ever. Set up a VPN (which isn't as daunting as you'd think), and use it, and only SSH over that VPN connection.
  • If you're using a firewall appliance, get it checked immediately too.
  • If you're more comfortable with Windows/Mac, and are using those, get THOSE systems between your Linux system and the Internet. While Linux is more capable, in the time it takes you to get things locked down, damage may be done. Address the problems first with something you know.
  • If you're not comfortable with doing such things, hire someone. Seriously...security isn't something to take lightly, and someone who knows what they're doing can save you far more than you pay them.
Security isn't a simple thing, but a process that you have to continually work on. There are some basic how-to guides
https://www.cyberciti.biz/tips/linux-security.html

...but really, they're guidelines. You have to take your entire environment into account, not just one system.
 
2 members found this post helpful.
Old 07-15-2018, 05:59 PM   #4
teeps
LQ Newbie
 
Registered: Jul 2018
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thank you both for the quick responses, If I'm understanding @scasey correctly, I have blocked them properly, but they are still attempting to initiate an SSH connection with no password log, only an attempted connection log from a known blocked IP address?

I honestly feel like a fool, I totally didn't think that system would log connection attempts, I thought it was only logging password attempts, so when I didn't see any correlating password attempts I became confused.

And thanks for the suggestions @TB0ne, I have done a bit of reading on a few of your suggestions, but I hadn't thought to run it through a VPN, thank you for the words of wisdom my friend, and I can only keep learning.
 
Old 07-15-2018, 06:25 PM   #5
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,727

Rep: Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211
Quote:
Originally Posted by teeps View Post
I honestly feel like a fool, I totally didn't think that system would log connection attempts, I thought it was only logging password attempts, so when I didn't see any correlating password attempts I became confused.
Please don't feel foolish. Things like that are easy to get confused about*

To be sure you understand: It's only logging the connection attempts because you told it to:
Code:
-N SSHATTACK
-A SSHATTACK -j LOG --log-prefix "Possible SSH Attack! " --log-level 7
-A SSHATTACK -j DROP
If you removed the bolded line, then those attempts would no longer be logged...
[Caveat: I think...I really don't have my head around iptables at all...don't make any changes based on what I say, given that what you have seems to be working.]

*A story:
My mail customers were being swamped (maybe 10 per day for 3 or 4 customers) by solicitation emails from political candidates. Reporting the UCE had no effect, so we blocked the sending IP addresses. The resulting log entries about the blocked delivery attempts jumped to around 100 per hour(!)...and still no response from the sending company. That caused excessive mail log rollover, masking the real activity.
We finally moved the blocks to the firewall...no more entries in the mail logs. We didn't care about the increase in the firewall logging.
 
1 members found this post helpful.
Old 07-15-2018, 07:17 PM   #6
teeps
LQ Newbie
 
Registered: Jul 2018
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks again, now that I take a step back I see that the INPUT rules are simply detecting attempted SSH connections regardless of password attempt, and it jumps to the SSHATTACK rule once the hitcount hits 2 in a single minute, so a persistent client attempting an SSH connection over and over regardless of timeout would continue to generate SSHATTACK logs indefinitely.

This has all been a learning experience for me, I'm currently studying for my CCNA certification, and the deeper I go with hands on Cisco the more I need to work with Linux, so its a constant tug of war between the two. As an old guy attempting to learn a new trade I find myself lost in a vast ocean of never ending knowledge and it gets rather intimidating sometimes. I suppose now I've got to learn more about the Cisco Firewall I haven't done much with since I got it
 
Old 08-03-2018, 04:31 AM   #7
boombaby
Member
 
Registered: Mar 2016
Posts: 93

Rep: Reputation: Disabled

Hello, teeps, et al...


I want to add a bit of "Security" info here, for a different "view" of the problem.


PART 1

I once did some welfare-involved work at a Police Station, along with a few other people. As a "volunteer" effort we were setting up a small room at the Station with a dozen or so networked PCs of the NON-Linux kind.

One Volunteer had his own IT sales company and had some deeper knowledge about IT - and hacking. He wanted to show us his "hacking" prowess. He was at one end of the room using his variety of hacking tools, targetting them one-by-one at a PC at the other end of the room. The sequence went something like this...

"Here's one" he'd say. (Click)

Person at the other PC says, "Nup; it was blocked."

"What about this one?" (Click).

"Blocked it!"

And so on.

It took about 5 or 6 different "tools" until he got onto the PC's hard-drive and was looking around - and the other person could not detect it.

He had several dozen different hack tools - perhaps a hundred.

[Funny part here is, he told everyone to use a particular type of cleaning tool to wipe their PCs at home. I had only recently tested cleaning tools and demonstrated to him that the one he recommended DID NOT work as it described. I think it shocked him.]



PART 2

I had a similar experience to you, teeps.

My modem was being constantly hit from Internet by attacks of various kinds - each one flagged to me by email. I believed that I had my modem protection set up correctly to block them all. (However I had that other hacking demonstration always on my mind too.)

On 3 or 4 occasions I had spoken about the hacking attempts to my ISP provider. Their help-desk was being handled from OUTSIDE my country. (And it's probably not any of the main countries you might immediately think of.) Every time I mentioned it to them I was reassured there was no issue with it.

On one occasion of discussion with them I seemed to be getting a particular kind of "oblique" answer to my questions, which sounded like I was getting the run-a-round. So I used the words "...report it to the Federal Police, and let them sort it out...".

I never received another email about a hack attempt.


PART 3

Now, I know, those (true) stories are not "Linux" and you've marked your post "solved". So, points?

1. You may not be able to see "it" but it might be there anyway.
2. Be MORE careful.
3. Dig deeper maybe.
4. Be alert.
5. Listen to others about security. (I mean, listen to the "what" in what they are saying.)
6. Forums like this are, generally, a good place to understand stuff, and find out about the "new" stuff.


While not directly "Linux" I hope this has been a useful addition to the theme, for you and others.


Regards,
boombaby
 
Old 08-03-2018, 05:39 AM   #8
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 19,872
Blog Entries: 12

Rep: Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053
i understand that the discussion here is not directly about ssh, but...
i'm surprised that nobody mentioned ssh keys, instead of passwords?
i have that set up on my server, and after i made sure it works, disabled password login completely.
surely that would be much better in the long run?
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
faillog deleted? Possibly compromised? savona Linux - Security 1 05-02-2011 03:44 PM
Possibly compromised debian by running backdoor'd ProFTPD wriswith Linux - Security 4 12-05-2010 05:15 PM
possibly compromised - what to do? TreeHugger Linux - Security 4 02-04-2005 11:03 PM
System possibly compromised kloppster Linux - Security 7 07-12-2004 03:30 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 01:59 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration