Details of DDOS attack
 

Hi,

One of our Linux servers that is running Plesk was recently compromised and was found to be launching a DOS attack. Now we would like to learn how to investigate more about what has happened.

We would like to find out:

a. Which files have been modified within a specific time frame;
b. Traffic usage within specific time frame, which domain and destination of traffic. etc
c. Failed and successful login attempts, especially plesk
d. How do we view/download any log files that could contain useful information before we reprovision the server?
e. any other useful info that will help us find how the attackers entered the system and what they did whilst they were in.

Any help would be appreciated, but please bear in mind that I only have very basic experience so please detail any advice in a step by step way.

Thanks for any help!

There's some information you can get, but some of this will be irretrievable depending upon what architecture is surrounding your server.

a. Easy. Use find with either -ctime or -mtime flags.
b. Unless you have some kind of active traffic capture going on, such as with Suricata or Bro or some device that monitors network traffic, that information is gone. Linux doesn't really do historical logging of that kind of stuff.
c. For Linux logins, use `last`. Plesk stores login information in the MySQL database known as 'psa'. If you know how to use MySQL you can go in there. Thankfully most of their tables are written with obvious names. You can also check the Plesk access log to see when the login pages were accessed and from where. I haven't worked on any control panel in a while, so the path escapes me, so if someone could chime in and refresh my memory...
d. Get /var/log/messages, /var/log/secure, /var/log/wtmp, and other web server access logs and FTP logs. xferlog is another one to check. `history` might reveal commands used, you can also check the .bash_history of other users as one of them might've been compromised (especially if they have shell access with a weak password, the former is a definite no-no without real need!).
e. Unfortunately, it was very likely one of two things:
1) Someone dropped a phpshell into a vulnerable website, irrespective of Plesk. Probably due to a vulnerable WordPress, Joomla, Drupal, etc. site. This was used to launch the DDoS.
2) Someone exploited Plesk and gained a shell onto the system. This can be difficult to combat since Parallels can be very slow to issue patches.

Thanks for your reply, I understand most of what you have said, but just a couple of points that I am unclear about


Can you explain exactly what you mean by this?

I know how to view these via cli but can you tell me how to create a logfile with these that I can export?

Sure.

find . -ctime +4 -type f

This will find all files created within the last 4 days starting at the current working directory. mtime is modified time, and the + is in days. Check the manpage for more details on those flags.

I'd recommend just logging in via CLI and running those commands. You can SCP logfiles out using a program like WinSCP and for commands like `last` and `history` just run them and copy+paste the output from PuTTY.

Don't try and do all of this through Plesk. While it's good at what it does, which is easily serve up websites and email, it's not really a system administration tool. :)

Minor clarification: ctime is time that file's status was last changed, for example a change in permissions. inodes don't store a file creation time.

Minor clarification of the previous minor clarification: the C in MAC times stands for "change" (changes to file or directory inode). In UNIX it is not possible to tell the actual creation time of a file.

Is the data for -ctime and -mtime coming from one of the log files? whats the location of this log file?

It doesn't come from a logfile. It comes from the filesystem itself.

Ok, this is useful to know but in this case nothing looks out of order so I have downloaded all of the usr/ and Var/log files and will review them when I get some spare time.

Does anybody know a good way to view them on a pc as some of the logs are 1GB in size and very slow to load using wordpad etc?


I fully agree with this and am 99& certain this is where the server was hacked, we have now patched both these areas.

For various reasons I would never use Wordpad or a machine running mcrsft for that kind of task. Linux has Logwatch (see these two posts: https://www.linuxquestions.org/quest...ogwatch&u=2450 for additional nfo).


This:
contradicts you saying:
(as in you haven't found any "evidence")?.. Makes me question what you actually checked and how?



...and if you haven't found any "evidence" then I seriously doubt that would be enough?

I'd recommend getting a VM player like VirtualBox (free) and using it to load a small Linux VM on your desktop, then put the files into that. It's sandboxed so you're protected if there's anything nefarious, and you can use the native Linux tools for text manipulation, which IMO are better than Windows'.

unSpawn........you are making incorrect assumptions,let me give you some further information:

About 18 months ago we had a report from our hosting company that our server was being used to launch a dos attack. We investigated this and were able to locate some php files that were not protected and would have allowed an sql injection attack, we saw evidence of added code in some of the files. We suspended all services from the domains and account that was effected. The problem seemed to have been resolved and appeared at that time to have been confined to these domains and that account.

Then a few days ago we received further notification of another new attack, and we were told that the attack was due to a plesk security vunribility due to the server running an unpatched versión of plesk 8.

Now fast forward to today, when we ran: and the only altered files appeared to be plesk files, I couldnt see anything else that looks unusual. This in my opinion would be consistant with a plesk attack, combine that with what happened around 18 months ago and I believe that it is fair to say that David Trests answer is very accurate certainly regarding how they seem to have got into our server.

How have we patched the server?

1.Reprovisioned the server image
2.Updated plesk
3.Removed all domains that were effected in any way

The main cause for that kind of compromise is running stale, vulnerable versions of software (including themes, plugins, etc, etc) in the web stack (and to a lesser extent leeched credentials, not adhering to the products security best practices, admin errors, etc, etc.)


Version 8 was released in 2006 and 9.0 in 2008. Version 8 and 9 went EOL in 2012 (9 having received extended support until June this year).


Apart from following advice wrt platform (distros own security documentation, SANS Reading Room, OWASP, Cisecurity benchmarks) and product security best practices, hardening (restricting access to and forcing HTTPS for admin areas and using mod_security for example), regular auditing (Logwatch, LMD, etc, etc) the real problem with both examples seems to be in keeping tabs on updates, force clients to update, when updates are released. The choice to keep running end-of-Life software versions is not a technical issue but solely dependent on human factors. Web stack compromises happen a gazillion times a day and most could have been prevented, some even easily.