Server infected with scanssh, pscan2, ./sshf.
 

HI Gurus,

I am facing a security issue on my server. I can see many process like pscan2, scanssh and ./sshf processing on 'top'. The owner of these processes is non root account. Can anybody let me know what can be the extent of loss due to these suspicious scripts? How can I permanently remove these scripts from my server. Please note that I am using CentOS 5.5 (64bit). Any help will be greatly appreciated.

Regards,
Sherazi

Find below output of top command. It will help you guys to suggest me a fix of it.
When I deleted user tivolie then top command output looks like as below

What you are showing is not a simple or easy answer. I have the following article that covers in detail how to audit a server after a compromise.

Linux - HowTo - Investigate A Linux Compromise
http://sites.google.com/site/zenarst...nux-compromise

However from the information you posted you will want to first locate where the script is running from. So if you take the PID or Process ID from top and then look at the /proc/ directory.

ls -la /proc/15106

Then the next question is how did they gain access to the server in order to run the processes. The is no simple answer to that and you can expect to spend a day or two gathering information. Once you have gotten this far then you can start shutting down the compromise itself. Otherwise if you shut is down too soon you may not have gathered the correct information.

for i in `ps -elf |grep pscan2 | awk '{print $4}' ; do kill -9 $i ; done

for i in `ps -elf |grep scanssh | awk '{print $4}' ; do kill -9 $i ; done

Thanks Joec,

I found PID from top and here is the output of the command you've mentioned.

[root@localhost tivoli]# ls -la /proc/4538
total 0
dr-xr-xr-x 5 tivoli tivoli 0 Aug 1 09:42 .
dr-xr-xr-x 187 root root 0 Jun 19 18:39 ..
dr-xr-xr-x 2 tivoli tivoli 0 Aug 1 09:45 attr
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 auxv
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:43 cmdline
-rw-r--r-- 1 tivoli tivoli 0 Aug 1 09:45 coredump_filter
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 cpuset
lrwxrwxrwx 1 tivoli tivoli 0 Aug 1 09:45 cwd -> /home/tivoli/ /game
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 environ
lrwxrwxrwx 1 tivoli tivoli 0 Aug 1 09:45 exe -> /home/tivoli/ /game/pscan2
dr-x------ 2 tivoli tivoli 0 Aug 1 09:43 fd
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 io
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 limits
-rw-r--r-- 1 tivoli tivoli 0 Aug 1 09:45 loginuid
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 maps
-rw------- 1 tivoli tivoli 0 Aug 1 09:45 mem
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 mounts
-r-------- 1 tivoli tivoli 0 Aug 1 09:45 mountstats
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 numa_maps
-rw-r--r-- 1 tivoli tivoli 0 Aug 1 09:45 oom_adj
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 oom_score
lrwxrwxrwx 1 tivoli tivoli 0 Aug 1 09:45 root -> /
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 schedstat
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 smaps
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:42 stat
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:42 statm
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:43 status
dr-xr-xr-x 3 tivoli tivoli 0 Aug 1 09:45 task
-r--r--r-- 1 tivoli tivoli 0 Aug 1 09:45 wchan


[root@localhost tivoli]# pwdx 4760
4760: /home/tivoli/ /game

sshf and pscan2 are some of the scanners (ELF binaries in this case) I've seen reported in systems after a non-root compromise. Commonly infection vectors are running vulnerable versions of web stack software (PHP remote file inclusions for instance) or unprotected services (VNC on Ubuntu anyone?) often in combination with incorrect permissions, net-accessible accounts with weak or no passwords. In your case it's obvious the "tivoli" user account was used and it has gained a directory "/home/tivoli/ /game" (note the space) out of which processes run.

Best approach would be to
- read the CERT Intruder Detection Checklist to know what you'll be doing,
- raise your firewall to only allow traffic to and from your management IP or range,
- verbosely list processes, open files and network connections and save those logs,
- stop web-accessible and related services (web server, database),
- list and kill rogue processes,
- disable the "tivoli" account.
Next
- check all login records, system and daemon logs using the CERT checklist and Logwatch for clues,
- check software of services you provide for versions and vulnerability fixes,
- check services you provide for configuration errors.
All that and in that order before doing anything else.


If the "tivoli" account is an unprivileged user account and if only the "tivoli" user account was used to gain access and run processes, and if the perps did not have a need for breaching root then the damage may be limited to only (setting up an IRC bot, sending spam? and or) scanning other systems. Only an investigation of the system can provide evidence for that anything else is speculation.


You should not "destroy evidence" until you have isolated the machine from the network, made a backup of "/home/tivoli/\ /game" and started investigating what actually happened.