apache and/or php executing arbitrary code

bombuzal · 05-06-2010, 03:22 PM

As the subject says, my Apache has been broken into. Someone executed some exploit and gained root. PHP is secured with php-harden, but has

Code:

safe_mode=off (otherwise breaks lots of sites - it is a shared hosting)
register_globals=on
disable_functions = "shell_exec,exec,system,dbmopen,suexec,escapeshellcmd,show_source,escapeshellarg"

My question is: how can I audit through which site (lots of virtual hosts) it has been broken into?
I get something like this

sh: line 1: sysctl: command not found
sh: line 1: sysctl: command not found
sh: line 2: /tmp/back: Permission denied
sh: line 1: wget: command not found
sh: line 1: fetch: command not found

but this is in general log. When I grep for that patterns in access logs, nothing comes up. Any suggestions?
Can something be blocked through mod_security? Any quick recipes?

Hangdog42 · 05-06-2010, 04:02 PM

Don't shut down the machine, but do cut off network access. If you can pull the network plug or put up a firewall so that SSH is the only thing allowed and only from trusted IP addresses.

You'll probably want to start gathering information on what is currently running:

lsof --Pwn
netstat -anpe
ps -axfwwwe

Hopefully those may reveal a bit about who is running what, but if they gained root, tracks may have been covered. Also start looking at the CERT checklist and start developing a picture of what is going on.

Instead of grepping your access logs, you'll probably want to eyeball them and look for odd entries. If you have any idea of when the intrusion happened, that may help. Sometimes if the logs have been erased, the timestamp might be a clue. Also have a look at root's .bash_history.

Other useful information is going to include OS details, versions of Apache and PHP and if you've been keeping up to date with patches.

bombuzal · 05-07-2010, 05:08 AM

I don't think there are any rouge processes running on the server, no external nmap scanning reveals anything etc. Anyway, the server is up for a clean reinstall ASAP.

But the main question still remains: if I boot up Apache, even the reinstalled with a freshest and patched version of everything, there is still a chance that some PHP application will execute some code, even if it does no serious harm... it can send spams or erase some 777 files.

Hangdog42 · 05-07-2010, 06:59 AM

Quote:

Anyway, the server is up for a clean reinstall ASAP.

Maybe not the best decision at this point.

Quote:

But the main question still remains: if I boot up Apache, even the reinstalled with a freshest and patched version of everything, there is still a chance that some PHP application will execute some code, even if it does no serious harm... it can send spams or erase some 777 files.

And this is why your previous decision may not be a particularly good one. Unless you take the time to do an investigation, you could be setting yourself up for the same headache all over again. The bad guys already know where your server is and how to exploit it. If you just re-install they'll still know where the machine is and how to exploit it AND they'll also know that they have to be more careful next time.

There are a number of people here who have experience in analyzing cracked machines. If you're willing to do the work, you'll get some help.

jiml8 · 05-08-2010, 12:43 PM

I presume you have mysql on the server as well?

I am sure that someone like unspawn will show up with some better ideas, but to the best of my knowledge, any PHP exploit has to be based on entering something in a textbox or listbox, and these usually wind up heading for a backend database. If you deploy mod security, this is what it will watch.

You might try checking and filtering your raw HTTP logs, looking for patterns in incoming data that would match known exploits, particularly sql injection attacks. This might let you find the problem.

You also COULD deploy mod security, but that very possibly will break functioning and properly secured websites. Speaking personally, I have switched hosting services in the past when that package was deployed on me without warning, resulting in LOTS of problems for me figuring out why submit strings that USED TO work didn't work, and why a client's email address broke the system, when no other email address broke the system.

Turns out that my submit strings contained a string that matched something in mod security, and the client's email addy also contained such a string. Mod security is very unintelligent about its tests; you match the string, you're gone. No context detection and, for instance, a string like "mycmd=setrequesttoone" will match because of the string "request" in the middle, when in fact the DANGEROUS string would be " request " (note the spaces around the word request).

So, a clean install is clearly warranted, and good luck with finding the problem. Wish I could help more.

spampig · 05-08-2010, 02:22 PM

People often blame PHP and whilst it is incredibly easy to write incredibly dangerous code, it's not always guilty. Perl/CGI is often overlooked and can runs some devastating stuff and most people overlook just how much system access can be gained with Perl.

Probably stating the obvious, but don't overlook the obvious; don't rule out a weak root || privileged user password. A tool like Hydra or BruteSSH can smash that in no time at all.

The best advice is to fix it before you reinstall it, or you'll be in the same position in no time.