Linux - SecurityThis forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Introduction to Linux - A Hands on Guide
This guide was created as an overview of the Linux Operating System, geared toward new users as an exploration tour and getting started guide, with exercises at the end of each chapter.
For more advanced trainees it can be a desktop reference, and a collection of the base knowledge needed to proceed with system and network administration. This book contains many real life examples derived from the author's experience as a Linux system and network administrator, trainer and consultant. They hope these examples will help you to get a better understanding of the Linux system and that you feel encouraged to try out things on your own.
Click Here to receive this Complete Guide absolutely free.
I went through the entire process again and rebooted the computer to make sure nothing was running in memory. I ran avg on the entire site and it finds nothing. I run rkhunter and it finds nothing. But the index.php files keep self-modifying so I have some sort of virus running in a process somewhere. Any idea how to find it? I did set the permissions on critical index.php files to 440 so that they cannot self-modify and that seems to help. But I need to find the culprit process and stop it. ps -ax yeilds the following:
PID TTY STAT TIME COMMAND
1 ? Ss 0:01 /sbin/init
2 ? S 0:00 [kthreadd]
3 ? S 0:00 [migration/0]
4 ? S 0:00 [ksoftirqd/0]
5 ? S 0:00 [migration/0]
6 ? S 0:00 [watchdog/0]
7 ? S 0:00 [migration/1]
8 ? S 0:00 [migration/1]
9 ? S 0:00 [ksoftirqd/1]
10 ? S 0:00 [watchdog/1]
11 ? S 0:00 [events/0]
12 ? S 0:00 [events/1]
13 ? S 0:00 [cpuset]
14 ? S 0:00 [khelper]
15 ? S 0:00 [netns]
16 ? S 0:00 [async/mgr]
17 ? S 0:00 [pm]
18 ? S 0:00 [sync_supers]
19 ? S 0:00 [bdi-default]
20 ? S 0:00 [kintegrityd/0]
21 ? S 0:00 [kintegrityd/1]
22 ? S 0:00 [kblockd/0]
23 ? S 0:00 [kblockd/1]
24 ? S 0:00 [kacpid]
25 ? S 0:00 [kacpi_notify]
26 ? S 0:00 [kacpi_hotplug]
27 ? S 0:00 [ata/0]
28 ? S 0:00 [ata/1]
29 ? S 0:00 [ata_aux]
30 ? S 0:00 [ksuspend_usbd]
31 ? S 0:00 [khubd]
32 ? S 0:00 [kseriod]
33 ? S 0:00 [md/0]
34 ? S 0:00 [md/1]
35 ? S 0:00 [md_misc/0]
36 ? S 0:00 [md_misc/1]
37 ? S 0:00 [khungtaskd]
38 ? S 0:00 [kswapd0]
39 ? SN 0:00 [ksmd]
40 ? S 0:00 [aio/0]
41 ? S 0:00 [aio/1]
42 ? S 0:00 [crypto/0]
43 ? S 0:00 [crypto/1]
48 ? S 0:00 [kthrotld/0]
49 ? S 0:00 [kthrotld/1]
52 ? S 0:00 [kpsmoused]
53 ? S 0:00 [usbhid_resumer]
84 ? S 0:00 [kstriped]
158 ? S 0:00 [ttm_swap]
165 ? S< 0:00 [kslowd000]
166 ? S< 0:00 [kslowd001]
302 ? S 0:00 [scsi_eh_0]
303 ? S 0:00 [scsi_eh_1]
306 ? S 0:00 [scsi_eh_2]
307 ? S 0:00 [scsi_eh_3]
401 ? S 0:00 [kdmflush]
403 ? S 0:00 [kdmflush]
422 ? S 0:00 [jbd2/dm-0-8]
423 ? S 0:00 [ext4-dio-unwrit]
424 ? S 0:00 [ext4-dio-unwrit]
458 ? S 0:00 [flush-253:0]
507 ? S<s 0:00 /sbin/udevd -d
631 ? S 0:00 [edac-poller]
831 ? S 0:00 [kdmflush]
868 ? S 0:00 [jbd2/sda1-8]
869 ? S 0:00 [ext4-dio-unwrit]
870 ? S 0:00 [ext4-dio-unwrit]
871 ? S 0:00 [jbd2/dm-2-8]
872 ? S 0:00 [ext4-dio-unwrit]
873 ? S 0:00 [ext4-dio-unwrit]
996 ? S 0:00 [kauditd]
1005 ? S 0:00 [flush-253:2]
1290 ? S<sl 0:00 auditd
1315 ? Sl 0:00 /sbin/rsyslogd -i /var/run/syslogd.pid -c 4
1344 ? Ss 0:00 irqbalance
1358 ? Ss 0:00 rpcbind
1376 ? Ss 0:00 rpc.statd
1388 ? Ss 0:00 mdadm --monitor --scan -f --pid-file=/var/run/mdadm/mdadm.pid
1410 ? S 0:00 [rpciod/0]
1411 ? S 0:00 [rpciod/1]
1415 ? Ss 0:00 rpc.idmapd
1446 ? Ss 0:00 dbus-daemon --system
1457 ? Ss 0:00 cupsd -C /etc/cups/cupsd.conf
1482 ? Ss 0:00 /usr/sbin/acpid
1491 ? Ss 0:00 hald
1492 ? S 0:00 hald-runner
1520 ? S 0:00 hald-addon-input: Listening on /dev/input/event0 /dev/input/event1
1539 ? S 0:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
1552 ? Ssl 0:00 automount --pid-file /var/run/autofs.pid
1571 ? Ss 0:00 /usr/sbin/sshd
1579 ? Ss 0:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
1604 ? Sl 0:00 /opt/avg/av/bin//avgd
1612 ? Sl 0:01 /opt/avg/av/bin/avgavid
1651 ? S 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysq
1779 ? Sl 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/lib/mysql/pnweb-01.com.
1791 ? Sl 0:00 /opt/avg/av/bin/avgtcpd
1800 ? Sl 0:00 /opt/avg/av/bin/avgscand -c 3
1821 ? Sl 0:00 /opt/avg/av/bin/avgsched
1844 ? Ss 0:00 /usr/sbin/dovecot
1846 ? S 0:00 dovecot/anvil
1847 ? S 0:00 dovecot/log
1856 ? Ss 0:00 proftpd: (accepting connections)
1869 ? Ss 0:00 sendmail: accepting connections
1877 ? Ss 0:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue
1900 ? Ss 0:00 /usr/sbin/abrtd
1908 ? Ss 0:00 abrt-dump-oops -d /var/spool/abrt -rwx /var/log/messages
1918 ? S 0:00 /usr/bin/python /usr/bin/denyhosts.py --daemon --config=/etc/denyhosts.conf
1925 ? Ss 0:00 /usr/sbin/httpd
1933 ? Ss 0:00 crond
1944 ? Ss 0:00 /usr/sbin/atd
1957 tty1 Ss+ 0:00 /sbin/mingetty /dev/tty1
1959 tty2 Ss+ 0:00 /sbin/mingetty /dev/tty2
1961 tty3 Ss+ 0:00 /sbin/mingetty /dev/tty3
1963 tty4 Ss+ 0:00 /sbin/mingetty /dev/tty4
1965 tty5 Ss+ 0:00 /sbin/mingetty /dev/tty5
1967 tty6 Ss+ 0:00 /sbin/mingetty /dev/tty6
1973 ? S< 0:00 /sbin/udevd -d
1974 ? S< 0:00 /sbin/udevd -d
1975 ? S 0:01 /usr/sbin/httpd
1976 ? S 0:00 /usr/sbin/httpd
1977 ? S 0:01 /usr/sbin/httpd
1978 ? S 0:00 /usr/sbin/httpd
1979 ? S 0:00 /usr/sbin/httpd
1980 ? S 0:01 /usr/sbin/httpd
1981 ? S 0:01 /usr/sbin/httpd
1982 ? S 0:00 /usr/sbin/httpd
1983 ? S 0:01 /usr/sbin/httpd
2173 ? S 0:00 /usr/sbin/httpd
2182 ? S 0:00 /usr/sbin/httpd
2183 ? S 0:00 /usr/sbin/httpd
2184 ? S 0:00 /usr/sbin/httpd
2391 ? S 0:00 dovecot/config
2392 ? S 0:00 dovecot/auth
2394 ? S 0:00 dovecot/auth -w
2411 pts/0 Ss 0:00 -bash
2432 pts/0 S 0:00 su
2434 ? S 0:00 /usr/libexec/fprintd
2435 ? S 0:00 sendmail: startup with 22.214.171.124
2436 pts/0 S 0:00 bash
Anything look suspicious? I am a little concerned about the next to last line. Why would 126.96.36.199 be starting sendmail?
Last edited by unSpawn; 02-25-2012 at 05:36 PM.
Reason: //BB code tags
Hi, just a loose thought, run iotop to see what accesses the drive and how much. As soon as the clean-up is done, "it" should get back to work. At that point, it should consume a lot of disk activity...
Next, look at the /etc/passwd file to see what/who has root privz besides the root. That could be a clue to a possilbe "shadowroot"...
If you can spot where what comes from, start the system with a clean Live CD and snip where possible, Knoppix is a good one, has great tools...
And, by the way, if the IP address sendmail uses is foreign...it stands to reason that's the hacker's contact line...cutting that may alert him/her, so be careful there...
Oh, did you try a netstat of the system itself? Maybe something could come from that angle too...
Shape up the fire wall, after the clean-out, it's useless to do that now, the trojan will simply rewire it anyway.
By the way, the trojan may need root rights, deleting the entry from the passowrd file(s) can help...
From what I've been reading all day, the culprit is a compromised php file somewhere. That file is a page that is triggered remotely by accessing it's URL. It then goes and re-writes the trojan into all of the index.php files. I cleaned everything, cleaned by http access files and then started httpd waiting for it to occur. After about 30 minutes it did so I am now analyzing log fines to see if I can determine which page has the compromised code on it. What a hassle. I don't know what code to look for in the 'parent file.
Okay, let's be rational...it is possible the hacker is not even calling up the file anymore...this is automated now (real clever, granted). However, it's got to be something that is allowed to write to the local disk. I suspect that php file to call an external file: the trojan.
Okay, the trojan could be anything, but perl comes to mind. Possibly part of a rootkit (sorry, dude), so, I'd go for a grep-seek for anything a php file calls to the "outside", php has an include syntax, it goes something like this
Well, I did find some perl files with AVG that had been uploaded to the wordpress directory structure. But I deleted than and completely removed that particular blog and the trojans still came back. Is there a way to log disk writes to a file so I can inspect that?
Is there a way to log disk writes to a file so I can inspect that?
Dunnow of the top of my head....but, as you clean ship...
wait, maybe we can lure 'im out. Make a folder in the wordpress setup, only give yourself write access, make it world readable. Put a php file in there, fire op iotop and watch. There should be a process knocking at the door of that new folder...
meanwhile, back at the ranch, I want to run a script to clean all of my index.php files. I wrote this:
sed -i "s/eval(base64_decode('a-long-string-of-characters-goes-here'));//g" index.php
but it doesn't work. This isn't a very difficult command. What am I doing wrong? I basically just want to strip out that long eval. Do I need to escape some character? Also, how can I get it to recurse though all sub directories in my 'sites' folder?
Thanks if you have an answer. I have looked at various googled-results and all seem confirm my syntax.
- Stop the web server during maintenance,
- Check if your WP version is the latest version. BTW there are no compelling reasons not to run the latest version,
- Check plugin usage. Update plugins if updates are available or disable them if they are no longer updated or maintained (also see this WP/TimThumb thread.),
- Check for any files in the upload directory (for instance files having a .jpg extension doesn't mean they are JPEG files),
- Harden the machine. If you host WP yourself then the OWASP mod_security rules (CVS rules tarball) may help you beef up security.
- If there's anything else to address then would be the time, before you expose the machine to the 'net again.
* Please check prioritisation of tasks at hand. Cleaning up base64-encoded crud may seem the most important to you but it is not. If you do not combat the cause first then you'll be experiencing the symptoms again and again.