Backdoor.Linux.Small & Linux.Plupii - Debian DSA-789-1
 

Hi all,

It seems I might of had this exploit on my server. Although, DSA-789-1 is fixed in sarge and php 4.3.10-16 (I have 4.3.10-18).



Here's what I found:

apache wasn't taking incoming connections, so I dug deep

Found this little bit in the binary /tmp/apache

Can't telnet in from external on port 9865 because its behind a router.

I can put up the the binary that was used, if anyone wants to look at it.

I am thinking it might be a variant on the known exploit. Maybe I need to report to Debian bugs ?

Thanks for any tips or information.

Backdoor running as system user, usually due to lax security settings or running bad PHP applications.


Can't telnet in from external on port 9865 because its behind a router.
Nor would you want to. It's uninteresting: take your loss and move on.
Mitigate (pull the plug), verify (start here: Intruder Detection Checklist (CERT): http://www.cert.org/tech_tips/intrud...hecklist.html), backup (start here: Steps for Recovering from a UNIX or NT System Compromise (CERT): http://www.cert.org/tech_tips/root_compromise.html), investigate, rebuild from scratch (LQ FAQ: Security references: http://www.linuxquestions.org/questi...threadid=45261)


It seems I might of had this exploit on my server.
Check your logs for clues.


I can put up the the binary that was used, if anyone wants to look at it.
I'm interested, as always.

Thanks I will follow that. I meant to mention that I ran tripwire and it found nothing changed. I have also ran bastille when I set the server up to harden it. Also have rkhunter and chkrootkit running on cron jobs.


How did this exploit get onto my system ? I haven't any xmlrpc.php like files on the system. I have changed www-data's shell from /bin/sh /bin/false. would that help prevention ?


[REMOVED]

I meant to mention that I ran tripwire and it found nothing changed.
When was that database last generated? What toplevel directories does it cover?


I have also ran bastille when I set the server up to harden it.
Bastille-Linux is a hardening tool that allows you to opt out for applying some measures, right?
So running that doesn't necessarily mean you set everything, and everything as strict as possible, right?


Also have rkhunter and chkrootkit running on cron jobs.
What versions do you run? With respect to Rootkit Hunter (abbev.: RKH), what tests did you disable? RKH-1.2.9 nor CRT-0.47a cover stuff dumped in say /tmp right now but RKH will cover a lot more in the upcoming release though, including an option to scan arbitrary directories for signs of malware (and I should know ;-p).


I have changed www-data's shell from /bin/sh /bin/false.
Shouldn't that have been an inert shell in the first place?


would that help prevention ?
It won't if PHP-based apps still allow one to perform system commands.


How did this exploit get onto my system ? I haven't any xmlrpc.php like files on the system.
Don't concentrate on one thing. The most important thing now is to mitigate the risk and "restore" (not "system restore" as in place back a backup) the situation back to normal. Here's your projected route in a slightly more elaborate way:

1. Read the Intruder Detection Checklist so you know what to do,
2. Notify anyone who has access to the system it's been compromised and to stay off it,
3. Save open files, process and network data listings first then regain full control over the situation by minimally shutting down all non-essential services and raising the firewall. If there are doubts about the effectiveness of this (vs the severity of the breach) power down the box and only boot with a Live CD,
4. After stabilising prepare /etc/and /var backups and make backups for reference (not reuse) (see the "Steps for Recovering..." doc),
5. Investigate (if necessary). Use the steps from the Intruder Detection Checklist. Inspect and report back looking minimally at these (if available):
- (changes in or logged reports about) system authentication data,
- IDS (Snort, Prelude, router logs, filesystem integrity checkers, package manager if good enough),
- all system, daemon and firewall logs,
- installed SW (and was all SW updated?),
- running services,
- unusual (setuid root) files,
- user shell histories.
When you report back separate posting exact and factual information from hints, hunches and gut feelings. What you report back will be seen as "evidence" to help your reach a "verdict" which should justify taking the next steps:
6. repartition, reformat, re-install from scratch,
7. harden (LQ FAQ: Security references).

HTH

I stuck with the Debian default. I can post the config here if you like. I kept the database on an encfs so one has to have a password to mount it and use it. It only ever mounted and umounted manually by me and never left mounted.

I pretty much turned it all one. There was one thing I remember not agreeing with for some reason, but basically I cant do much as a normal user. Its been annoying but needed so it hasn't bothered me.

server:/etc/tripwire# rkhunter --version
Rootkit Hunter 1.2.9
server:/etc/tripwire# chkrootkit -V
chkrootkit version 0.44

I don't recall disabling any tests.


inert? What do you mean ?
I would of thought Debian stable would of not given system users a shell that didn't need it. :confused:

I removed horde that I wasn't using and 2 other php wiki's that weren't being used at the moment. I also updated the one wiki I am using, PM wiki.

No mention of a security update in the change log though.

Wow that will keep me busy for weeks to come!!




Here's what I found going through log tonight.


server:/log/apache# tail -n 1* error.log.1
[Sun Feb 18 06:27:55 2007] [notice] caught SIGTERM, shutting down

server:/log/apache# tail access.log.1 -n 1
65.55.208.62 - - [18/Feb/2007:05:55:34 +1100] "GET /software/ HTTP/1.0" 200 1485 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

server:/log/apache# head access.log -n 1
10.1.1.115 - - [18/Feb/2007:15:32:15 +1100] "GET /me HTTP/1.1" 404 358 "-" "Mozilla/5.0 (compatible; Konqueror/3.5; Linux; x86_64) KHTML/3.5.5 (like Gecko) (Kubuntu)"

Down for 10 hours it seems. The SIGTERM command that it was issued is a concerned. I cant see any logs on who or how it was issued.

Other then that, nothing so far. I did a grep for apache in /var/log and nothing suspect came up.
I'll keep digging.

Would this be relevant ? A little before the SIGTERM was sent to apache

In loginlog.0

It just looks like a ssh login attempt using ipv6, no ?

In auth.log.0

It also appears that syslog is stopped and and restarted by the system (to rotate logs I am assuming) at 6:25am and generally started again by 6:28. So this SIGTERM at 6:27:55 would of been missed by syslogd I think.

I'll react to your replies later on, but meanwhile I'd like to remark that it's best to first gather all data necessary and only *then* concentrate on details. The first, crucial and guiding question should be: "did they get root account access" because that justifies later actions to take. Then the second question should be: "what attack vector was used" because it shows what measures should be taken in the hardening phase. Wrt to posting information this means investigating changes in the filesystem, auth data and such should go first and httpd logs and such later on.

Does this look normal ?

The above finds nothing.

Yeah thats exactly what I want to know.

So far I'd say no,apart from getting apache to shut down, thats the only alarming thing so far.

I am just about through the checklist; will hopefully finish that today.

I am just about through the checklist; will hopefully finish that today.
OK, I'll wait for that then.

Hectic day at work unfortunately, so haven't finished the checklist. (I think only a point to go though).

I did, however installed and ran tiger. What an awesome tool!! I love it already. Nothing too bad from the report, but I will post it when I get home tonight. I also stumbled across this:


To be continued......

So I finally got a chance to finish the checklist. Nothing else came up apart from what I have posted and ....

Running:

Returns alot of kernel source code.

It did finish with these entries

So I went into tmp to have a look and found this odd listing

so...

and..

and

Also checked out this "text" file that looks more like.

But copying it to here at work, Symantec AV picked it up straight away as
and deleted it.




And look what else I stumbled across...



Looks like my server is becoming more of a honeypot Dear lord!! :rolleyes:


I am going to run clamscan on the entire system and see what else pops up.

Obviously, this system needs to be wiped out, but what can I do right now to make the most of it and find out how this happened ?

I have run tiger and will post the output on the next post.

Should I run snort on this ? I have been wanting to install it, but haven't even had a chance to start a tutorial. I did read someone say just apt-get it, follow the prompts and watch it log. Is it that simple ?

Tiger report

Obviously lots I can fix there, but anything stand out as a way "in" to my server ?