[SOLVED] more than one uid 0

micxz · 07-02-2011, 05:54 PM

I thought our VPS server was compromised this morning for these reasons:

- Got an email from cron. (btw: I can't find the cron script in etc or in roots crontab):

Code:

To: root@jam
Subject: Cron <root@jam> /usr/sbin/useradd -d /root -g root -G root -s /bin/sh -p "(removedpass)" sendmail && /usr/sbin/usermod -u 0 -o sendmail
(deleted headers)
Date: Sat,  2 Jul 2011 10:24:01 -0500 (CDT)

useradd: user sendmail exists

- .bash_history was missing
- maillog/messages are empty
- wtmp begins Sat Jul 2 11:29:26 2011. Shorty after the cronjob.
- a virtual terminal was open via ssh (sshd: sendmail@pts/0)
- ps outputs, "Internal error!" for some processes
- also user sendmail is at the end of the pass file (plesk was installed weeks ago):

Code:

[root@jam ~]# cat /etc/passwd | tail -1
sendmail:x:0:0::/root:/bin/sh

- And worst one I thought:

Code:

[root@jam ~]# awk -F: '($3 == "0") {print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
sendmail:x:0:0::/root:/bin/sh

My hosting company says, "The way it's working on your system with plesk is that sendmail has a UID of 0, but when it's called it has an effective UID of the account that called it, so that's how it should actually be configured. At this time I'm not seeing that your server was compromised at this time and unfortunately the logs don't have much information, however they only show that your server was accessed through SSH by us and you, even through the root login."

Steps I took to lock down:
- I disabled the user sendmail by adding a asterisk in the beginning pass field in /etc/shadow
- And killed virtual terminal process:

Code:

[root@jam ~]# ps waux | grep sendmail
Internal error!
(repeat error 4x)...
root 26589 0.0 0.2 97452 3840 ? S Jun30 0:00 sshd: sendmail@pts/0
[root@jam ~]# kill -9 26589

- added "DenyUsers sendmail" to sshd

I was under the impression there should never be another user with UID 0 right?

unSpawn · 07-02-2011, 07:49 PM

I'm sure there are exotic situations where multiple UID 0's are acceptable but
- no sane Linux distro runs their MTA as UID 0 by default.
- Adding a user using a cronjob is suspect as it's not an action one would willingly want to repeat.
* If one process still has a deleted .bash_history (or cronjob!) on a fd ('lsof -Pwln|grep dele') you can copy it out.
- When did maillog / messages become empty and do you have rotated logs slash backups (also see lsof)?
- wtmp may be rotated but the cronjob points to malicious activity. As only root can add users that's disconcerting.
- Apart from not listing which distro+release this is about user "sendmail" should not be allowed use of interactive shell let alone SSH access (hardening?).
- "ps outputs, "Internal error!" for some processes" nice, but I like details ('/bin/ps axfwwwe').
- "And killed virtual terminal process" by doing that w/o listing details (ps, netstat, lsof) first you destroy potential evidence.

Have a go at the http://web.archive.org/web/200801092...checklist.html ?

micxz · 07-02-2011, 08:38 PM

Quote:

Originally Posted by unSpawn

I'm sure there are exotic situations where multiple UID 0's are acceptable but
- no sane Linux distro runs their MTA as UID 0 by default.
- Adding a user using a cronjob is suspect as it's not an action one would willingly want to repeat.

Exactly my thoughts

Quote:

* If one process still has a deleted .bash_history (or cronjob!) on a fd ('lsof -Pwln|grep dele') you can copy it out.

Only these:

Code:

[root@jam ~]# lsof -Pwln|grep dele
syslogd    1351        0    1w   REG              0,177 13880070  294882320 (deleted) /tmp/messages
syslogd    1351        0    3w   REG              0,177  1979592  294882333 (deleted) /tmp/secure
syslogd    1351        0    5w   REG              0,177 11376025  294882324 (deleted) /tmp/maillog

Quote:

- When did maillog / messages become empty and do you have rotated logs slash backups (also see lsof)?

There is no rotated logs in /var/log/ just empty logs and still not being written to. lsof reports nothing besides what I posted above.

Quote:

- wtmp may be rotated but the cronjob points to malicious activity. As only root can add users that's disconcerting.
- Apart from not listing which distro+release this is about user "sendmail" should not be allowed use of interactive shell let alone SSH access (hardening?).

CentOS release 5.6 (Final). Besides the "Steps I took" I suppose I could harden sshd more.

Quote:

- "ps outputs, "Internal error!" for some processes" nice, but I like details ('/bin/ps axfwwwe').

Code:

[root@jay ~]# /bin/ps axfwwwe
Internal error!
Internal error!
Internal error!
Internal error!
Internal error!
Internal error!
Internal error!
  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:01 init [3]       HOME=/ TERM=linux

That way all the errors are at the top. Too much more to post here. Let me know if you want me to mesg it.

Quote:

- "And killed virtual terminal process" by doing that w/o listing details (ps, netstat, lsof) first you destroy potential evidence.

I skrewed this one up.

I'm going through the link you provided (thanks) so far no suspicious setuid files so far:

Code:

/sbin/unix_chkpwd
/sbin/pam_timestamp_check
/etc/sw/keys/restart/apskeyhandler
/etc/sw/keys/restart/plesk-key-handler
/usr/local/psa/admin/sbin/wrapper
/usr/local/psa/admin/sbin/mod_wrapper
/usr/local/psa/handlers/hooks/grey
/usr/local/psa/suexec/psa-suexec
/usr/local/psa/bin/chrootsh
/usr/libexec/openssh/ssh-keysign
/usr/sbin/suexec.saved_by_psa
/usr/sbin/suexec
/usr/sbin/userhelper
/usr/sbin/suexec.hgbak.1309396098
/usr/sbin/usernetctl
/usr/lib64/sw-cp-server/sw-suexec
/usr/lib64/plesk-9.0/autoresponder
/usr/bin/sudo
/usr/bin/sudoedit
/usr/bin/gpasswd
/usr/bin/crontab
/usr/bin/chage
/usr/bin/chsh
/usr/bin/at
/usr/bin/newgrp
/usr/bin/passwd
/usr/bin/chfn
/lib64/dbus-1/dbus-daemon-launch-helper
/bin/ping
/bin/su
/bin/mount
/bin/ping6
/bin/umount

For years I have seen you on these forums always providing the best answers and helpful tips & tricks. I really appreciate your time on this & LQ in general.

unSpawn · 07-03-2011, 07:09 AM

Quote:

Originally Posted by micxz

Only these:

Code:

[root@jam ~]# lsof -Pwln|grep dele
syslogd    1351        0    1w   REG              0,177 13880070  294882320 (deleted) /tmp/messages
syslogd    1351        0    3w   REG              0,177  1979592  294882333 (deleted) /tmp/secure
syslogd    1351        0    5w   REG              0,177 11376025  294882324 (deleted) /tmp/maillog

/tmp? What are system logs doing in /tmp? You did copy (or netcat or scp or cat|ssh) /proc/1351/fd/{1,3,5} off the system, right?

Quote:

Originally Posted by micxz

There is no rotated logs in /var/log/ just empty logs and still not being written to. lsof reports nothing besides what I posted above.

Auch.

Quote:

Originally Posted by micxz

Besides the "Steps I took" I suppose I could harden sshd more.

What do you mean by that?

Quote:

Originally Posted by micxz

Code:

[root@jay ~]# /bin/ps axfwwwe
Internal error!
Internal error!
Internal error!
Internal error!
Internal error!
Internal error!
Internal error!
  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:01 init [3]       HOME=/ TERM=linux

That way all the errors are at the top. Too much more to post here. Let me know if you want me to mesg it.

Interesting. Please use a directory like /dev/shm and append output to a file or maybe better: pipe output through netcat or ssh out of the system:
'( rpm -Vv SysVinit glibc binutils procps strace findutils lsof coreutils net-tools shadow-utils rpm grep 2>&1 | grep -v '^\.\{8\}'; )'
'( /usr/bin/strings -an1 /bin/ps 2>&1; /lib/ld-linux.so.2 --verify --list /bin/ps 2>&1; /usr/bin/strace -v -s 1024000 -f -ff /bin/ps - -o /dev/shm/ps.strace 2>&1; /usr/sbin/lsof -Pwln 2>&1; /bin/netstat -anpe 2>&1; /usr/bin/lastlog 2>&1; /usr/bin/last 2>&1; /usr/bin/who -a 2>&1; )'
'/usr/bin/find / -printf "%T@ %A@ %C@ \"%p\"\n" 2>&1'
'/bin/rpm -Vva 2>&1|/bin/grep -v "\.\{8\}" 2>&1'
* the latter two might take some time so I would appreciate it if you make the first two available as soon as possible. Please email me the D/L URI or use my designated drop-off (check your email).

Also a couple of question while you're gathering nfo:
- When did this start? Reviewing your past threads doesn't show any previous occurrences. Were there no previous anomalies or breaches of security?
- which services does the machine provide? Please include web-based management panels, statistics, web log, forum, shopping cart, plugins and other software running in the web stack if any and post exact versions,
- was the software was kept up to date?
- are SELinux and auditd active?
- Are there any setuid-root or oddly named files in writable (temp) dirs?
- did you check user shell history files?
- which access restrictions are in place and what hardening was performed?

Quote:

Originally Posted by micxz

I'm going through the link you provided (thanks) so far no suspicious setuid files so far

Good!

Quote:

Originally Posted by micxz

For years I have seen you on these forums always providing the best answers and helpful tips & tricks. I really appreciate your time on this & LQ in general.

Thanks, I do appreciate that.

micxz · 07-03-2011, 10:01 PM

Quote:

Originally Posted by unSpawn

/tmp? What are system logs doing in /tmp? You did copy (or netcat or scp or cat|ssh) /proc/1351/fd/{1,3,5} off the system, right?

Nope doing that now.

Quote:

Originally Posted by unSpawn

What do you mean by that?

I was referring to in my first post when I stated, "Steps I took to lock down:" hardening I mean by allowing only my ips or maybe just the three or so users that require ssh.

Quote:

Originally Posted by unSpawn

Interesting. Please use a directory like /dev/shm and append output to a file or maybe better: pipe output through netcat or ssh out of the system:
'( rpm -Vv SysVinit glibc binutils procps strace findutils lsof coreutils net-tools shadow-utils rpm grep 2>&1 | grep -v '^\.\{8\}'; )'
'( /usr/bin/strings -an1 /bin/ps 2>&1; /lib/ld-linux.so.2 --verify --list /bin/ps 2>&1; /usr/bin/strace -v -s 1024000 -f -ff /bin/ps - -o /dev/shm/ps.strace 2>&1; /usr/sbin/lsof -Pwln 2>&1; /bin/netstat -anpe 2>&1; /usr/bin/lastlog 2>&1; /usr/bin/last 2>&1; /usr/bin/who -a 2>&1; )'
'/usr/bin/find / -printf "%T@ %A@ %C@ \"%p\"\n" 2>&1'
'/bin/rpm -Vva 2>&1|/bin/grep -v "\.\{8\}" 2>&1'
* the latter two might take some time so I would appreciate it if you make the first two available as soon as possible. Please email me the D/L URI or use my designated drop-off (check your email).

I will send you some stuff in the drop box.

Quote:

Originally Posted by unSpawn

Also a couple of question while you're gathering nfo:
- When did this start? Reviewing your past threads doesn't show any previous occurrences. Were there no previous anomalies or breaches of security?

This started about: (Sat Jul 2 11:29:26 CDT 2011) Cron email I got at (Sat, 2 Jul 2011 10:24:01 -0500 (CDT)). Nothing previous. I did hire a company to harden/setup the server whom I've used in the past just for initial setup. Regular sys admin backups and updates I do myself.

Quote:

Originally Posted by unSpawn

- which services does the machine provide? Please include web-based management panels, statistics, web log, forum, shopping cart, plugins and other software running in the web stack if any and post exact versions,

- Plesk (web based management software)
Plesk provides DNS, HTTP, HTTPS, FTP, POP/IMAP, POPS/IMAPS, MYSQL, statistics all the goodies. I'm selling hosting to my friends and family so anyone of them could be installing forums and shopping carts. There are 79 domains I was migrating (downgrading) to this VPS server in question. I've moved about 37 of the sites when this happened. bummer'

Quote:

- was the software was kept up to date?

This is a new VPS server I just purchased last week. Plesk and CentOS are the most recent. After hardening the first time I got some errors migrating plesk so I had to reimage the VPS. Now it looks as if I need to reimage again after this incident.

Quote:

- are SELinux and auditd active?

At the time of this incident no. This is part of the hardening but was not done yet as it was just reimaged.

Quote:

- Are there any setuid-root or oddly named files in writable (temp) dirs?

Only suid files on the system are posted above.

Quote:

- did you check user shell history files?

Well the only other users with shells besides root is me and mysql oh and sendmail! mine is clean and mysql has no homedir.

Quote:

- which access restrictions are in place and what hardening was performed?

My new hosting company has root access and suppose they would anyhow even without my root pass as it's a VPS. The company doing the hardening had access.

micxz · 07-04-2011, 12:34 AM

Just to be clear:
- I purchased a VSP server
- Had it hardened by third party
- Tried to migrate my sites/mail etc with Plesk software that failed
- Because the failed migration messed up the Plesk panel/database my provider said the only way would be to re image the VPS and reinstall plesk.
- The server was not hardened yet after reimaging when this incident occured.

unSpawn · 07-04-2011, 01:02 AM

Thanks for the ordered output, well done.

From 1rpm.out:

Code:

SM5.....    /bin/ps
SM5.....    /usr/bin/top
SM5.....    /usr/bin/find
SM5.....    /usr/sbin/lsof
SM5.....    /bin/ls
SM5.....    /usr/bin/dir
SM5.....    /usr/bin/md5sum
SM5.....    /bin/netstat
SM5.....    /sbin/ifconfig

That's a traditional rootkit layout.

From 2strings.out, running '/usr/bin/strings -an1 /bin/ps 2>&1':

Code:

176 libproc.so.2.0.6
(..)
189 look_up_our_self
(..)
195 proc_hackinit
196 libc.so.6

.. same log, '/lib/ld-linux.so.2 --verify --list /bin/ps 2>&1':

Code:

	libproc.so.2.0.6 => /lib/libproc.so.2.0.6 (0xb7f63000)
	libc.so.6 => /lib/libc.so.6 (0xb7e0a000)
	/lib/ld-linux.so.2 (0xb7f78000)

.. same log, running 'strace':

Code:

1 execve("/bin/ps", ["/bin/ps", "-", "-o", "/dev/shm/ps.strace"], # etc
(..)                                = 0
9 open("/lib/libproc.so.2.0.6", O_RDONLY) = 3

libproc.so.2.0.6 being part of the cb, FuckIT, SHV4, SHV5, Tuxtendo and w00tkit rootkits. MAC times seem as you indicated time of breach of security, so let's try and find the point of entry. I'll work on it later as R/L priorities dictate I should do Something Completely Different.

micxz · 07-04-2011, 01:54 AM

Quote:

Originally Posted by unSpawn

Thanks for the ordered output, well done.

woow thanks for the confirmation. I'm not losing my mind thank god. Now I suppose I will have to do full password audit on everyone after I get this box on it's feet. And after a reimage get the hardening done right off! Man this is a first for me in my ten years + of hosting. I have only read about this incidents! And to think the hosting company said it was all good! I knew something was up.

Noway2 · 07-04-2011, 07:32 AM

Lets start with the obvious, who had access:

Quote:

My new hosting company has root access and suppose they would anyhow even without my root pass as it's a VPS. The company doing the hardening had access.

You also seem to have a pretty good idea of when the system was compromised, Sat Jul 2 11:29:26 CDT 2011. When was the work done by the 3rd party? What was it that they were supposed to be doing exactly?

Unfortunately, the log files were wiped, yet, messages, secure, and maillog were moved to /tmp too, and then deleted? Could this be part of an attempt to hide some specific activity? Does the output of netstat show any open connections? It also seems interesting that they would attempt to put a (second) mail user in place with root level access. It is almost as if their intentions were to setup something to feed information back to them at a later date.

Quote:

Plesk provides DNS, HTTP, HTTPS, FTP, POP/IMAP, POPS/IMAPS, MYSQL, statistics all the goodies.

How many of these services are web accessible from a public IP?

I would also recommend having a SERIOUS conversation with both your hosting provider and your "security" company. It is beyond absurd that the hosting provider pretty much told you to "piss off" when you presented information that your system had been compromised.

Also as a reminded, which I didn't see mentioned in the thread, if you haven't already, lock this system down. Forget about trying to migrate to your new setup, etc. You mentioned that your friends, family, and customers could be installing shopping carts and things. These need to be secured and verified, straight away.

micxz · 07-04-2011, 10:06 PM

Quote:

Originally Posted by Noway2

Lets start with the obvious, who had access:
You also seem to have a pretty good idea of when the system was compromised, Sat Jul 2 11:29:26 CDT 2011. When was the work done by the 3rd party? What was it that they were supposed to be doing exactly?

Hello Noway2 see my post above starting with, "Just to be clear:" This incident happened right after my host handed over the root password. Shortly after I got a shell open things started acting up. When I first got on there where no firewall rules set at all. The third party was responsible for implementing firewall, selinux, hardening permissions on programs, mounting /tmp nosuid,noexec your standard setup steps.

Quote:

Originally Posted by Noway2

Unfortunately, the log files were wiped, yet, messages, secure, and maillog were moved to /tmp too, and then deleted? Could this be part of an attempt to hide some specific activity? Does the output of netstat show any open connections? It also seems interesting that they would attempt to put a (second) mail user in place with root level access. It is almost as if their intentions were to setup something to feed information back to them at a later date.

I'm sure it was to hide some specific activity. netstat did not show any suspicious listens or connections. But I believe that binary was on the modified list. lsof did show a connection from someone. It's possible it was for data collecting later on or if root changed the password then he could get on later.

Quote:

Originally Posted by Noway2

How many of these services are web accessible from a public IP?

All of them well not mysql (local only).

Quote:

Originally Posted by Noway2

I would also recommend having a SERIOUS conversation with both your hosting provider and your "security" company. It is beyond absurd that the hosting provider pretty much told you to "piss off" when you presented information that your system had been compromised.

I totally agree. I plan to get in touch with them as I feel it's that type of negligence that causes these types of problems.

Quote:

Originally Posted by Noway2

Also as a reminded, which I didn't see mentioned in the thread, if you haven't already, lock this system down.

My host did not want it live on the network so it has been put on the shelf. I now have a clean new VPS and the root pass got changed with seconds of me receiving it. (last time it remained the same for many hours after first re-image)

unSpawn · 07-06-2011, 07:48 PM

Sorry for the silence, R/L kicking in. Your provider, together with a few other well-known ones, probably is on everybody's scan list for the easy targets such clusters of servers often provide but still it amazes me your server got compromised that soon. I'm not happy there's no evidence left wrt the point of entry because that leaves us without a clue as to how this happened. This means that any hardening advice will have to be as broad as possible. As you've already started configuring your new server what are your hardening plans? If you could start a new thread we'd be glad to offer advice.

micxz · 07-07-2011, 03:53 AM

Quote:

Originally Posted by unSpawn

R/L kicking in.

Forgot all about the real life. when this was all going down I was up all night checking backups and tacking a list of old scripts and I even found some files w/php3 ext. Removed old wordpress installs etc. Just soon as they handed me the pass it was run and lock down.

Quote:

Originally Posted by unSpawn

Your provider, together with a few other well-known ones, probably is on everybody's scan list for the easy targets such clusters of servers often provide but still it amazes me your server got compromised that soon.

Yep I've been watching closely logs and anything suspicious I would run a small script to block them via iptables+*.deny. Working on getting ssl setup for imap, pop & smtp.

Speaking of my provider during the re-image ticket there where several "Linux Admin" posting to the ticket. Every other post would be a different guy helping me. So I asked how many people are over there and they said they employ 600+ couldn't tell me how many where Linux admins. With that being said it could easily be internal leak, maybe the guy that said I wasn't compromised. I still need to talk to a manager about what happened.

Quote:

Originally Posted by unSpawn

I'm not happy there's no evidence left wrt the point of entry because that leaves us without a clue as to how this happened.

This is the part that is killing me. All we've got is an IP and theories (most likely order):
- internal leak from my provider or the security company
- intercepted the passwords in email transmission
- exploited software to gain priv

Quote:

Originally Posted by unSpawn

This means that any hardening advice will have to be as broad as possible. As you've already started configuring your new server what are your hardening plans? If you could start a new thread we'd be glad to offer advice.

Some but not limited to:
- Re-imaged the server completely, reinstalled the Operating System (obviously after a rootkit you can't trust anything)
- Implemented a secure firewall + additional security software
- Hardened the web services & permissions
- Requiring stronger passwords for email
- Changed all user passwords
- Policy to sms passwords without usernames or ips (for people that have login shells)
- Engaged an security firm to conduct a investigation if my provider will let me access the infected image.

I may start another thread for additional setup tips. For now I feel pretty good about the server now I've just changed glue for my domains. Logs are slowing to a stop on the old box.

Noway2 · 07-07-2011, 07:53 AM

I have been looking over the data and I have a few things to add. I will also forward this information to Unspawn and ask him to double check, but here are some things I would like to ask you to look into and / or comment on too.

1) In the file with the strings output, I am curious as to the nature of the file. I am operating under the impression that it is the output of the strings command on some of the compromised binaries and libraries. If this is the case, please look on or about line 8710, where there are references to user names and IP addresses. If, like I said, this is from the compromised file(s) the attack would seem to be awfully targeted, suggesting or implicating an information leak.

2) in the FD3 file, around line 10606, there seems to be a change in the attack style. The brute force attempts on your SSH server are happening at a level that I find astounding. The fact that your security firm didn't provide a defense against this and the fact that the hosting provider is not reacting to this kind of traffic surprises me. What is interesting is that shows what appears to be a lockout on the plesk database along with an inability to connect to the database. I am wondering if this was a potential crack in the foundation that let something in?

3) How long has this server been online? You are getting WAY too many DNS queries for 3rd party (recursive) look ups for a new system. They are almost all coming from the same Italian domain too. It makes me think that you have were sold a bad IP or bad server that had a known history and were ill prepared for it.

4) Looking in the same file, fd3, Line 17231, it Looks like proftp may have gotten hit with access from user 0 (0 = root?) Was this a potential crack?

5) File FD5, I notice the spam filter running through a socket connection in /tmp. /tmp wouldn't be my first choice for this sort of thing since it is typically 777 permission. My concern would be possible injection point.

6) Same file, look on or about 3340, it looks like the mailer system may have been compromised as it looks to be sending spam based upon the email address. If I am reading this correctly, it looks like these are outgoing messages as the SENT=OK was set. I didn't follow the SMTP ID's closely enough to confirm, but it caught my attention. Again, as with the DNS, for a server that (I think) is rather new, you are getting hit awfully hard and the defense both locally and at the hosting provider level are woefully inadequate for dealing with it.

Noway2 · 07-08-2011, 04:29 AM

Micxz, I was looking over some of the information and I came across something that was potentially interesting.
Does the user yelapan mean anything to you as well as the IP 187.173.218.171, which is dsl-187-173-218-171-dyn.prod-infinitum.com.mx. which appears to be a residential, or at least a DSL with DHCP provider in Mexico?

micxz · 07-08-2011, 11:52 AM

Quote:

Originally Posted by Noway2

Micxz, I was looking over some of the information and I came across something that was potentially interesting.
Does the user yelapan mean anything to you as well as the IP 187.173.218.171, which is dsl-187-173-218-171-dyn.prod-infinitum.com.mx. which appears to be a residential, or at least a DSL with DHCP provider in Mexico?

Noway2' I really do appreciate the efforts you and unspawn are putting into helping me find out what exactly happened. I to saw the login entry your talking about in fd3.out. Yes yelapan is the username w/bin/false for shell and is chrooted ftp account for his WP website. I just got an email from him the other day and the IP in the email matches the IP you posted. If I had to choose a different father he would be my next choice I trust him to the fullest. Unless his mac was taken over....

The only users with login shells at the time of entry was me, admin (used for Plesk), root (of course you could obtain a shell from some exploit)

I did try on the new box (well re-imaged same IPs) to replicate the adding a user via cron from the Plesk admin interface. I was able to escalate myself from admin to user micxzsendmail (uid 0) within 5 mins by adding a cron job for root. But when I tired to logging in failed as root is now only allowed key login or use su of course. Maybe Plesk should be notified?

Changing all the passwords sure does make everyone catch up on the bills. I got paid for over four hosting accounts for people wanting to login but didn't want to ask for support until they're paid up. I'm rich! he he.

Also I would like to add though maybe unrelated. There was a users pending in my billing system which is older software. I got the client signup on 06/26/2011 and even though the account was not setup (no payment or verificati0n). If that user somehow got access to that DB there are usernames and passwords for FTP accounts like above (no shells/chrooted ftp).

Then right around the time of re-imaging the tainted box I get another signup 07/03/2011 and another 07/05/2011 this time he actually paid me via paypal and emailed asking when his account will be setup. All three from different public emails and IPs in South Africa. They are suspect and I believe they wanted to get "back on".

Just for the record I do not setup new accounts right off. I have people verify email first, I call the phone #, and have them present a valid referral (e.g. Your mom told me to host my site here.)