Seems like Server is Compromised

tux75 · 04-11-2018, 12:02 PM

I think my VPS server running Debian 9.0 is compromised. Its a new setup with few basic applications installed, as of now only SSH and fail2ban are running on the system.

As soon as the server was up I did below changes.

1. generated a new ssh key and disabled password authentication.
2, installed and configured fail2ban for ssh
3. createad a firewall rule to block all ports except ssh

I was under the impression that I got the server fully secured, I saw that fail2ban blocked few IPs after they attempted a brute force attack or scan. Then I did a check on outgoing connections using below command.

Quote:

netstat -atn | tr -s ' '| cut -f5 -d ' ' | grep -v '127.0.0.1'

This returned an unfamiliar ip, the connect was there only for few seconds. Later I saw another ip using the same command. Further googling led me to NetHogs and I can see that there are quite few ips trying making connections.

Code:

  NetHogs version 0.8.5-2

    PID USER     PROGRAM                                       DEV        SENT      RECEIVED
   5331 wally    sshd: wally@pts/0                             ens3        0.207       0.063 KB/sec
      ? root     xxx.xx.xxx.xx:24363-77.72.85.17:56793                    0.000       0.011 KB/sec
      ? root     xxx.xx.xxx.xx:9094-5.188.11.78:44273                     0.000       0.011 KB/sec
      ? root     xxx.xx.xxx.xx:6936-77.72.85.17:56793                     0.000       0.000 KB/sec
      ? root     xxx.xx.xxx.xx:3428-5.101.66.252:57886                    0.000       0.000 KB/sec
      ? root     xxx.xx.xxx.xx:5708-5.188.11.78:44273                     0.000       0.000 KB/sec
      ? root     xxx.xx.xxx.xx:38473-191.101.167.77:59146                 0.000       0.000 KB/sec
      ? root     xxx.xx.xxx.xx:15516-191.101.167.77:59146                 0.000       0.000 KB/sec
      ? root     xxx.xx.xxx.xx:30882-5.188.11.25:42365                    0.000       0.000 KB/sec
      ? root     unknown TCP                                               0.000       0.000 KB/sec

  TOTAL                                                                    0.207       0.084 KB/sec

could some one explain to me whats going on. I can't find any process attached to these connections.

Thanks

Habitual · 04-11-2018, 01:22 PM

Quote:

Originally Posted by tux75

with few basic applications installed

Might have been "secure" when ya got it...

Your use of the term "applications" is curious to me.
Tell us, does "basic applications" include a desktop?

tux75 · 04-11-2018, 03:01 PM

Quote:

Originally Posted by Habitual

Tell us, does "basic applications" include a desktop?

by applications I meant software packages. like apache or mysql.

scasey · 04-11-2018, 03:58 PM

Code:

netstat -atnp

will add the name of the process to the display.

tux75 · 04-12-2018, 04:01 AM

There aren't any processes pertaining to those IPs

Code:

# netstat -atnp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      27435/sshd
tcp        0    404 xxx.xx.xxx.xx:22       yyy.y.yy.yyy:1727       ESTABLISHED 30137/sshd: wally [

tux75 · 04-12-2018, 04:22 AM

IPs in red, are unknown to me.

Code:

# netstat -atn | tr -s ' '| cut -f5 -d ' ' | grep -v '127.0.0.1'
and
Address
0.0.0.0:*
212.85.73.131:39528
yyy.y.yy.yyy:1157

# netstat -atn | tr -s ' '| cut -f5 -d ' ' | grep -v '127.0.0.1'
and
Address
0.0.0.0:*
151.101.28.204:80
yyy.y.yy.yyy:1157
#

scasey · 04-12-2018, 02:50 PM

Quote:

Originally Posted by tux75

There aren't any processes pertaining to those IPs

Code:

# netstat -atnp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      27435/sshd
tcp        0    404 xxx.xx.xxx.xx:22       yyy.y.yy.yyy:1727       ESTABLISHED 30137/sshd: wally [

The process is sshd in both cases...the second entry shows an established ssh connection from the yyy.y.yy.yyy IP address...user is wally

scasey · 04-12-2018, 02:59 PM

Quote:

Originally Posted by tux75

IPs in red, are unknown to me.

Code:

# netstat -atn | tr -s ' '| cut -f5 -d ' ' | grep -v '127.0.0.1'
and
Address
0.0.0.0:*
212.85.73.131:39528
yyy.y.yy.yyy:1157

# netstat -atn | tr -s ' '| cut -f5 -d ' ' | grep -v '127.0.0.1'
and
Address
0.0.0.0:*
151.101.28.204:80
yyy.y.yy.yyy:1157
#

Code:

whois 212.85.73.131

will display information about the IP address...in that case, it's a "Dynamic private network" of an ISP (Bahnhof) in Sweden.

I think the other one is showing you that you are connected to a foreign webpage...the IP resolves to a hosting company in San Francisco ...did you have a browser running?

Again... Add the -p to see the process, and maybe stop tweaking the output of netstat...it's pretty clear as it stands, and there's more information (other than the Foreign IP) than you're sharing that helps understanding what you're seeing.

tux75 · 04-17-2018, 10:57 AM

Quote:

Originally Posted by scasey

The process is sshd in both cases...the second entry shows an established ssh connection from the yyy.y.yy.yyy IP address...user is wally

Yes this is the connection I made to the server.

tux75 · 04-17-2018, 11:02 AM

Quote:

Originally Posted by scasey

Code:

whois 212.85.73.131

will display information about the IP address...in that case, it's a "Dynamic private network" of an ISP (Bahnhof) in Sweden.

I think the other one is showing you that you are connected to a foreign webpage...the IP resolves to a hosting company in San Francisco ...did you have a browser running?

Again... Add the -p to see the process, and maybe stop tweaking the output of netstat...it's pretty clear as it stands, and there's more information (other than the Foreign IP) than you're sharing that helps understanding what you're seeing.

I monitored the connection randomly for few days and didn't find anymore suspicions connections. I used the connection to browse the web and seems like these were the IPs from that connection. May be I got bit paranoid :-)

Quote:

and there's more information (other than the Foreign IP) than you're sharing that helps understanding what you're seeing

What is this information will be?

Thanks a lot for the help

scasey · 04-17-2018, 11:36 AM

Quote:

Originally Posted by tux75

What is this information will be?

Thanks a lot for the help

Code:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      27435/sshd
tcp        0    404 xxx.xx.xxx.xx:22       yyy.y.yy.yyy:1727       ESTABLISHED 30137/sshd: wally [

As the header line says, the Local IP and port, the Foreign IP and Port, the State (very useful see man netstat) and the PID and Program name...your logic is hiding all that from you.

You're most welcome.