Help please!

My webserver has been hacked. The user logged in as php,
I have given as much info as I can think of below. Please if anyone has any advice on what other actions to take and the possible way the user gained access, please let me know. I am assuming that it was through apache or php for two reasons... the strange files below are created by www by httpd user and my ssh is on a non standard port.

Thanks for any help you can give.

"last" command output:
reboot system boot 2.4.22 Wed Apr 13 22:55 (10:26)
php pts/1 host230-97.pool8 Wed Apr 13 22:53 - crash (00:02)


"finger php" command output
Login: php Name: (null)
Directory: /home/.bash Shell: /bin/sh
Last login Wed Apr 13 22:53 (ADT) on pts/1 from host230-97.pool80181.interbusiness.it
No mail.
No Plan.

I have since changed the php users directory and shell to /dev/null
I am not sure I even need a php user as my apache runs as a different user altogether.

When I look in the /home/ directory I now see:
-rwxrwxrwx 1 www www 437367 Feb 24 2004 .php*

And when I look in the /home/.bash directory I see
drwxr-xr-x 3 root root 4096 Mar 29 22:18 ./
drwxr-xr-x 34 root root 4096 Mar 13 06:16 ../
drwxr-xr-x 11 www users 4096 Mar 29 10:12 .p/
-rwxrwxrwx 1 www www 437367 Feb 24 2004 brk*

in the .p directory is the source tree for
psyBNC 2.3.2

I have since deleted the /home/.bash directory and the /home/.php file

I am running apache 1.3.29 and php 4.3.6
With postgresql, qmail, and bind as services.
On the php server I have OpenWebMail running.
I have iptables running with the following rules:
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:CUSTOM-INPUT - [0:0]
-A INPUT -j CUSTOM-INPUT
-A FORWARD -j CUSTOM-INPUT

#manually added blocks from sketchy activity
-A CUSTOM-INPUT -s 61.109.245.145 -j REJECT

# Open the httpd port
-A CUSTOM-INPUT -p tcp -m tcp --dport 80 --syn -j ACCEPT
# Open the https port
-A CUSTOM-INPUT -p tcp -m tcp --dport 443 --syn -j ACCEPT
# Open the ftp port
#-A CUSTOM-INPUT -p tcp -m tcp --dport 21 --syn -j ACCEPT
# Open the samba port
-A CUSTOM-INPUT -p tcp -m tcp --dport 137:139 --syn -j ACCEPT
# Open the SSH port
-A CUSTOM-INPUT -p tcp -m tcp --dport 99 --syn -j ACCEPT
# Open the DNS port
#-A CUSTOM-INPUT -p tcp -m tcp --dport 53 --syn -j ACCEPT
# Open the pop port
-A CUSTOM-INPUT -p tcp -m tcp --dport 110 --syn -j ACCEPT
# Open the smtp port
-A CUSTOM-INPUT -p tcp -m tcp --dport 25 --syn -j ACCEPT
# Open the MySQL port
-A CUSTOM-INPUT -p tcp -m tcp --dport 3306 --syn -j ACCEPT
# Allow Outside PostreSQL connections
-A CUSTOM-INPUT -p tcp -m tcp --dport 5432 --syn -j ACCEPT

# Accept all from the loopback interface
-A CUSTOM-INPUT -i lo -j ACCEPT

-A CUSTOM-INPUT -p icmp -j REJECT
-A CUSTOM-INPUT -p tcp -m tcp --dport 0:1023 --syn -j REJECT
#-A CUSTOM-INPUT -p tcp -m tcp --dport 5432 --syn -j REJECT
-A CUSTOM-INPUT -p tcp -m tcp --dport 2049 --syn -j REJECT
-A CUSTOM-INPUT -p udp -m udp --dport 0:1023 -j REJECT
-A CUSTOM-INPUT -p udp -m udp --dport 2049 -j REJECT
-A CUSTOM-INPUT -p tcp -m tcp --dport 6000:6009 --syn -j REJECT
-A CUSTOM-INPUT -p tcp -m tcp --dport 7100 --syn -j REJECT
COMMIT

I have added
-A CUSTOM-INPUT -s 80.181.97.230 -j REJECT
-A CUSTOM-INPUT -s 80.181.97.0 -j REJECT
To the above to block the IP from where the offender arrived but does the .0 block all IPs in that range?

does php need a shell? cant you set it to /bin/false?

Not that I know of. I have set the home directory and shell as /dev/null (the same result I think... can't login as the user)
Thanks

its this that made me say that:

""finger php" command output
Login: php Name: (null)
Directory: /home/.bash Shell: /bin/sh
Last login Wed Apr 13 22:53 (ADT) on pts/1 from host230-97.pool80181.interbusiness.it
No mail.
No Plan."

If they were able to modify the the users shell and create a home directory then they had root access (in fact i don't believe php is a standard username). With root access they could have made a number of modifications that would be extremely difficult to detect like replace binaries with trojaned versions or put a backdoor on your system. Which means the only solution is to take the system offline immediately, backup any important files (not binaries), format the drive and reinstall the system from trusted media.

As far as how they gained access, you should check the system logs (especially the apache logs) for any abnormal messages. Also, what versions of Apache and PHP were you using? What kind of content were you hosting on the server?

The server is a web and mail server for a small company. The webserver doesn't have anything really important. The only other thing would be that they could possibly access other computers on our lan, which is more of a concern.

I checked the www user had bash access would it be possible I think that if the user was actually logged in the only had access as the www user. Checking the access logs for apache and php revealed nothing.

The version of PHP was 4.3.6 now has been upgraded to the latest 4.3.11 and apache was 1.3.29 is now 1.3.33

Would there be any other way to check if the user gains access again? Other than formatting the drive.

Is it possible it was a worm?

The webserver doesn't have anything really important. The only other thing would be that they could possibly access other computers on our lan, which is more of a concern.
The data on the machine may not be important, but using your system as a proxy for attacking other systems should be a major concern along with sniffing local traffic for sensitive info like clear-text passwords or MiM attacks. All of which can be a serious problem if not properly addressed.

I checked the www user had bash access would it be possible I think that if the user was actually logged in the only had access as the www user. Checking the access logs for apache and php revealed nothing.
If they were able to add a user, modify a users shell, and write to /home , then it's pretty clear that they gained root access. Given how old the PHP and apache versions were, it's highly likely that you had vulnerable software on the system that would allow an attacker to perform a local privilege elevation and gain root rather easily.

The version of PHP was 4.3.6 now has been upgraded to the latest 4.3.11 and apache was 1.3.29 is now 1.3.33
These are both pretty out of date and PHP has especially had a number of critical vulnerabilities recently.

Would there be any other way to check if the user gains access again? Other than formatting the drive.
Once an attacker has gained root, it becomes problematic to trust the system as commands could be replaced with trojaned versions, a rootkit or malicious kernel module could be installed. So the 'ps' command could be replaced with a version that would hide processes belonging to the cracker. With that in mind, you can see how detecting an attacker could be a problem if you can't trust the output of netstat, who, last, etc. You can try rebuilding the box as a honeypot, but that can be a dangerous game if you don't know what you're doing. Sometimes it's just better to cut your losses and chalk it up as a lesson learned. On the otherhand, doing a forensic analysis of the compromised system is a good idea. In fact, you can make an image of the compromised system and do the analysis at your convenience, that way you can get the system back online immediately

Is it possible it was a worm?
Could be, but most of the recent ones are pretty obvious as they deface everything in the server root.

Thanks for your help Capt. I guess, its not what I wanted to hear because of the time it will take. But if I have to start from scratch to be sure that's what I'll do.

One more question. It would be safe to backup images (jpg's and gifs, etc. ) from the website and just give them a scan with clam av or something?

Thanks again.