Crowl, you've been a member of LQ for almost a decade, but don't have a lot of post activity. I am not sure how familiar you are with incident investigation process as we perform it here. We use an evidence gathering, fact finding approach based upon the dated CERT intruder detection check list. Here is a link:
http://www.scribd.com/doc/6398/Cert-...tion-Checklist While the checklist is dated, it still contains the fundamentals of what you need to look for.
The first step is to contain the system, which you have started to do by raising the firewall. I would suggest that you shut off all traffic except for SSH from your trusted connection. You should not try to operate this server while performing the investigation as it will only hamper your efforts and destroy potential evidence. It is also important to cut the intruder off in one swoop while still keeping processes active that could have important clues as to the nature of the intrusion. Once you have secured the system do not reboot or power down and do not try to 'clean' the system.
Next, please capture a process and network connection output. You can use the following command set as root/sudo:
Code:
(ps acxfwwwe 2>&1; lsof -Pwln 2>&1; netstat -anpe 2>&1; lastlog 2>&1; last 2>&1; who -a 2>&1 ) > /tmp/log.txt
This will create a text file called log.txt in your /tmp directory. Running as root is important because it provides key pieces of process information that helps tie things together.
Second, you will need to continue the examination of your log files. As a start, download a copy of the logwatch tool and run it.
Code:
"logwatch --detail High --service All --range All --archives --numeric --save /path/to/logwatch.log"
Note, that I have seen some variation in this command/utility and depending on your distribution it may not behave quite right in regards to producing the log output. The variation as provided I believe will work on Debian/Ubuntu systems, but on others you will get a listing to stdout that you can redirect to a file with > Again, this should be run as root as some log files are protected.
Third, you will need to make a verification of your binary files. In fact, this really should be done in part before you try to run the command set I listed above. In the interests of saving time, please download known good copies of the binaries, lsof, netstat, and ps, from you distribution repository mirrors and upload these files to the server and use the know copies in the commands.
Please post back as soon as you are able and with any questions.
Followup: you mentioned not knowing the IP address. It is from a hosting company in the Netherlands called lease web (AS number 16265).
