LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 07-31-2010, 01:23 PM   #1
zaeem
Member
 
Registered: Jan 2010
Posts: 110

Rep: Reputation: 15
Server infected with scanssh, pscan2, ./sshf.


HI Gurus,

I am facing a security issue on my server. I can see many process like pscan2, scanssh and ./sshf processing on 'top'. The owner of these processes is non root account. Can anybody let me know what can be the extent of loss due to these suspicious scripts? How can I permanently remove these scripts from my server. Please note that I am using CentOS 5.5 (64bit). Any help will be greatly appreciated.

Regards,
Sherazi
 
Old 07-31-2010, 01:28 PM   #2
zaeem
Member
 
Registered: Jan 2010
Posts: 110

Original Poster
Rep: Reputation: 15
Find below output of top command. It will help you guys to suggest me a fix of it.
Code:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
10024 tivoli    25   0  1680  496  408 R 100.1  0.0   0:46.59 pscan2
 8962 root      15   0 13004 1416  816 R  0.7  0.0   0:01.72 top
 9237 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.22 scanssh
 9254 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.18 scanssh
 9257 tivoli    15   0  3700 1004  844 S  0.3  0.0   0:00.62 scanssh
 9259 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.24 scanssh
 9291 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.16 scanssh
 9303 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.18 scanssh
 9313 tivoli    15   0  3700 1060  912 S  0.3  0.0   0:00.20 scanssh
 9316 tivoli    15   0  3700 1008  856 S  0.3  0.0   0:00.36 scanssh
 9339 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.17 scanssh
 9344 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.18 scanssh
 9346 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.29 scanssh
 9351 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.18 scanssh
 9354 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.14 scanssh
 9361 tivoli    15   0  3700 1060  912 S  0.3  0.0   0:00.09 scanssh
 9382 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.22 scanssh
 9390 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.16 scanssh
 9395 tivoli    15   0  3700 1008  856 S  0.3  0.0   0:00.35 scanssh
 9406 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.16 scanssh
 9783 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.05 scanssh
 9791 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.08 scanssh
 9792 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.05 scanssh
 9794 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.05 scanssh
 9796 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.05 scanssh
 9836 tivoli    15   0  3700 1064  912 S  0.3  0.0   0:00.02 scanssh
 9851 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.03 scanssh
 9854 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.03 scanssh
 9861 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.04 scanssh
 9874 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.04 scanssh
 9900 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.05 scanssh
 9914 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.03 scanssh
 9921 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.04 scanssh
 9924 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.04 scanssh
 9931 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.04 scanssh
 9935 tivoli    15   0  3700 1068  912 S  0.3  0.0   0:00.05 scanssh
When I deleted user tivolie then top command output looks like as below

Code:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15106 503       18   0  1680  496  408 R 99.8  0.0   1:55.13 pscan2
15027 503       15   0  3700 1008  856 S  0.7  0.0   0:00.19 scanssh
15088 503       15   0  3700 1064  912 S  0.7  0.0   0:00.11 scanssh
12655 503       15   0  3700 1060  912 S  0.3  0.0   0:00.48 scanssh
12658 503       15   0  3700 1060  912 S  0.3  0.0   0:00.52 scanssh
13752 503       15   0  3700 1012  856 S  0.3  0.0   0:00.86 scanssh
13753 503       15   0  3700 1068  912 S  0.3  0.0   0:00.36 scanssh
13755 503       15   0  3700 1068  912 S  0.3  0.0   0:00.38 scanssh
13758 503       15   0  3700 1068  912 S  0.3  0.0   0:00.56 scanssh
13762 503       15   0  3700 1068  912 S  0.3  0.0   0:00.54 scanssh
13766 503       15   0  3700 1068  912 S  0.3  0.0   0:00.63 scanssh
13785 503       15   0  3700 1064  912 S  0.3  0.0   0:00.42 scanssh
13789 503       15   0  3700 1064  912 S  0.3  0.0   0:00.26 scanssh
13792 503       15   0  3700 1068  912 S  0.3  0.0   0:00.29 scanssh
13799 503       15   0  3700 1068  912 S  0.3  0.0   0:00.34 scanssh
13803 503       15   0  3700 1068  912 S  0.3  0.0   0:00.60 scanssh
13810 503       15   0  3700 1068  912 S  0.3  0.0   0:00.65 scanssh
13814 503       15   0  3700 1064  912 S  0.3  0.0   0:00.55 scanssh
13818 503       15   0  3700 1068  912 S  0.3  0.0   0:00.41 scanssh
13819 503       15   0  3700 1064  912 S  0.3  0.0   0:00.27 scanssh
13842 503       15   0  3700 1068  912 S  0.3  0.0   0:00.56 scanssh
13844 503       15   0  3700 1068  912 S  0.3  0.0   0:00.40 scanssh
13857 503       15   0  3700 1064  912 S  0.3  0.0   0:00.37 scanssh
13881 503       15   0  3700 1068  912 S  0.3  0.0   0:00.64 scanssh
13903 503       15   0  3700 1064  912 S  0.3  0.0   0:00.34 scanssh
13911 503       15   0  3700 1008  844 S  0.3  0.0   0:00.85 scanssh
13912 503       15   0  3700 1064  912 S  0.3  0.0   0:00.35 scanssh
13916 503       15   0  3700 1068  912 S  0.3  0.0   0:00.62 scanssh
14344 503       15   0  3700 1068  912 S  0.3  0.0   0:00.23 scanssh
14355 503       15   0  3700 1064  912 S  0.3  0.0   0:00.11 scanssh
14387 503       15   0  3700 1068  912 S  0.3  0.0   0:00.07 scanssh
14396 503       15   0  3700 1068  912 S  0.3  0.0   0:00.30 scanssh
14398 503       15   0  3700 1068  912 S  0.3  0.0   0:00.13 scanssh
14410 503       15   0  3700 1068  912 S  0.3  0.0   0:00.22 scanssh
14413 503       15   0  3700 1068  912 S  0.3  0.0   0:00.30 scanssh
14420 503       15   0  3700 1008  856 S  0.3  0.0   0:00.28 scanssh
14426 503       15   0  3700 1064  912 S  0.3  0.0   0:00.20 scanssh

Last edited by unSpawn; 08-02-2010 at 05:15 PM.
 
Old 07-31-2010, 02:48 PM   #3
joec@home
Member
 
Registered: Sep 2009
Location: Houston Tx
Posts: 89

Rep: Reputation: 29
What you are showing is not a simple or easy answer. I have the following article that covers in detail how to audit a server after a compromise.

Linux - HowTo - Investigate A Linux Compromise
http://sites.google.com/site/zenarst...nux-compromise

However from the information you posted you will want to first locate where the script is running from. So if you take the PID or Process ID from top and then look at the /proc/ directory.

ls -la /proc/15106

Then the next question is how did they gain access to the server in order to run the processes. The is no simple answer to that and you can expect to spend a day or two gathering information. Once you have gotten this far then you can start shutting down the compromise itself. Otherwise if you shut is down too soon you may not have gathered the correct information.

for i in `ps -elf |grep pscan2 | awk '{print $4}' ; do kill -9 $i ; done

for i in `ps -elf |grep scanssh | awk '{print $4}' ; do kill -9 $i ; done
 
Old 07-31-2010, 11:48 PM   #4
zaeem
Member
 
Registered: Jan 2010
Posts: 110

Original Poster
Rep: Reputation: 15
Thanks Joec,

I found PID from top and here is the output of the command you've mentioned.

[root@localhost tivoli]# ls -la /proc/4538
total 0
dr-xr-xr-x 5 tivoli tivoli 0 Aug 1 09:42 .
dr-xr-xr-x 187 root root 0 Jun 19 18:39 ..
dr-xr-xr-x 2 tivoli tivoli 0 Aug 1 09:45 attr
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 auxv
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:43 cmdline
-rw-r--r-- 1 tivoli tivoli 0 Aug 1 09:45 coredump_filter
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 cpuset
lrwxrwxrwx 1 tivoli tivoli 0 Aug 1 09:45 cwd -> /home/tivoli/ /game
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 environ
lrwxrwxrwx 1 tivoli tivoli 0 Aug 1 09:45 exe -> /home/tivoli/ /game/pscan2
dr-x------ 2 tivoli tivoli 0 Aug 1 09:43 fd
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 io
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 limits
-rw-r--r-- 1 tivoli tivoli 0 Aug 1 09:45 loginuid
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 maps
-rw------- 1 tivoli tivoli 0 Aug 1 09:45 mem
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 mounts
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 mountstats
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 numa_maps
-rw-r--r-- 1 tivoli tivoli 0 Aug 1 09:45 oom_adj
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 oom_score
lrwxrwxrwx 1 tivoli tivoli 0 Aug 1 09:45 root -> /
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 schedstat
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 smaps
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:42 stat
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:42 statm
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:43 status
dr-xr-xr-x 3 tivoli tivoli 0 Aug 1 09:45 task
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 wchan


[root@localhost tivoli]# pwdx 4760
4760: /home/tivoli/ /game
 
Old 08-05-2010, 06:45 PM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,166
Blog Entries: 54

Rep: Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807Reputation: 2807
Quote:
Originally Posted by zaeem View Post
I can see many process like pscan2, scanssh and ./sshf processing on 'top'. The owner of these processes is non root account.
sshf and pscan2 are some of the scanners (ELF binaries in this case) I've seen reported in systems after a non-root compromise. Commonly infection vectors are running vulnerable versions of web stack software (PHP remote file inclusions for instance) or unprotected services (VNC on Ubuntu anyone?) often in combination with incorrect permissions, net-accessible accounts with weak or no passwords. In your case it's obvious the "tivoli" user account was used and it has gained a directory "/home/tivoli/ /game" (note the space) out of which processes run.

Best approach would be to
- read the CERT Intruder Detection Checklist to know what you'll be doing,
- raise your firewall to only allow traffic to and from your management IP or range,
- verbosely list processes, open files and network connections and save those logs,
- stop web-accessible and related services (web server, database),
- list and kill rogue processes,
- disable the "tivoli" account.
Next
- check all login records, system and daemon logs using the CERT checklist and Logwatch for clues,
- check software of services you provide for versions and vulnerability fixes,
- check services you provide for configuration errors.
All that and in that order before doing anything else.


Quote:
Originally Posted by zaeem View Post
Can anybody let me know what can be the extent of loss due to these suspicious scripts?
If the "tivoli" account is an unprivileged user account and if only the "tivoli" user account was used to gain access and run processes, and if the perps did not have a need for breaching root then the damage may be limited to only (setting up an IRC bot, sending spam? and or) scanning other systems. Only an investigation of the system can provide evidence for that anything else is speculation.


Quote:
Originally Posted by zaeem View Post
How can I permanently remove these scripts from my server.
You should not "destroy evidence" until you have isolated the machine from the network, made a backup of "/home/tivoli/\ /game" and started investigating what actually happened.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Computer is INFECTED according to ClamAV davidlu766 Linux - Security 3 04-11-2010 07:12 PM
locate infected machine from dhcp server erimar77 Linux - General 2 05-20-2006 09:33 AM
rootkit: infected??? help synaptical Linux - Security 4 05-16-2005 07:11 PM
clamav: infected files provkitir Linux - Security 2 12-20-2004 01:19 AM
RH 7.3 Server infected with Linux.Jac.8759 and Linux.RST.B virus osso09 Linux - Security 10 11-17-2003 11:37 PM


All times are GMT -5. The time now is 02:23 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration