hi there,

a few days ago i checked who was loged on
and w output was,
uptime xx days xx:xx, 4 users
and below there were only 3 users listed
where is the fourth user?
i've scanned the system with rkhunter for rootkits, none found, scaned for viruses with clamav, none found...
nothing suspicius on the system, like a strange user in /etc/shadow or anywhere
no strange processes runing...

could have anyone broke into the system?
root is disabled without a password (*), and i use sudo to access root

Users here aren't exclusive accounts, but login sessions. each xterm, ssh session etc... would be another user there, even if they are all root.

yes i know that
i don't have only 3 users on the system, but then there were curently 3 users loged into the server, me and other 2 users
so one 2much in the first line of w output

That's based on the so-called utmp information and sometimes it does have its problems.

Each time you run bash or a program runs bash, a new user process is created if im not mistaken. Is it possible you had a couple of xterm's open when you ran the command.

I use gkrellm, which tells me under the Proc tab how many users are on the system etc, and if i open a new xterm up, a new user appears.

Try closing everything except one xterm, then run the commands 'users' and 'who'. That should give you an idea of who is logged in (assuming you havent been rooted or something similarly bad).

well i changed my roots password in passwd from
root:encrypted-passwd:...
to
root:*:...

and if i would have been rooted i think that the user whould have changed this password
or create a new user for sudo or something
but there are no new users in shadow only those that i or my friend has added

Uh oh!
I'm very glad u put a password in for root, maybe try looking in /home, or using "who" every now and then? Just keep a lookout but i'm pretty sure that you will find its nothing.
p.s make sure all accounts have passwords, no password = haxor heaven.
Cheers, Luke

well i think the worst has happened
I belive i've been rooted...
yesterday the web server was shut down
today someone changed apache config file
and deleted my document root so, basicly my web page has been deleted, lucky me i have a backup...
so i ran rkhunter as soon as i saw this, nothing strange, only, apache version 2.2.4 and php version 5.2.1 is buging it
it says unkown
i've inspected /etc/shadow /etc/group and /etc/passwd
nothing unussual in there
i also checked sudoers file, nothing strange either...
so what's next for me to do?

I belive i've been rooted... (..) so what's next for me to do?
First read, then act. Use common sense.

In short the plan is this:
- mitigation,
- investigating,
- preparing backups (we'll get to that),
- repartition, reformat, re-install from scratch (later),
- harden (later).


1. Mitigation

Start by reading the Intruder Detection Checklist (CERT): http://www.cert.org/tech_tips/intrud...checklist.html.

Now save these off-site:
- full listing of processes,
- full listing of network connections,
- full listing of open files.
If the box is local reboot it. Don't run commands from the disk but boot a Live CD for investigating. If the box is in colo shut down every unnecessary network-accessable service meaning MySQL, Apache etc, etc. Use common sense. Basically you only need SSH. Raise the firewall so only you have access in and outbound access from your management IP (range).



2. Investigation

We need to build an understanding of the situation. Perform tasks from the Intruder Detection Checklist and add info below if not covered already:
- location (home, colo) and purpose of the box. If colo: (dedicated or shared hosting),
- (perceived) date of incident,
- distro+release+kernel versions,
- list of installed SW and knowing if they where updated (specifically PHP-based apps),
- list of running services and a short descr. of who has access,
- full listing of processes,
- full listing of network connections,
- full listing of open files,
- any auth data: full listings of "last", "lastb", syslogs of the "authpriv" kind (see /etc/syslog.conf),
- any audit data like running your distro's package manager in file verification mode, file integrity checker,
- system, daemon and firewall logs,
- any setuid-root files found, anything in accessable temp dirs,
- user shell histories,
- anything else idea's, hunches or gut feelings you think *could* be relevant like "obvious" configuration mistakes like allowing SSH RootLogin's etc, etc.

Please try to:
- work your way through the list methodically,
- ask before performing a task if unsure,
- save info off-site, not on the "victim" itself,
- post only *exact*information,
- if the amount of info is too much compress and tarball it and post a D/L location instead.

Please note that due to the time between the actual breach of security, your first post and your most recent post there is a good chance a lot could be scrubbed by the intruder.

@All: call for help with incident handling
While I hope my fellow Linux - Security moderators will chip in and take over when new info arrives, I think this is a good time to call everyone to the stage who wants to do incident handling to take part in this, even if you have never done incident handling in Linux - Security before.

There are a few restrictions though:
- please research and read the *whole* thread before posting,
- please follow the steps as layed out by me,
- do not offer your opinion unless it is based on presented facts,
- do not offer advice about hardening and such. It's annoying, distracting and we're not in that phase yet.

if you can't find the discipline to adhere to those restrictions please consider *not* posting at all.

TIA

i've checked the whole system...ok half of it, and nothing found
then my fellow sys admin told me he screw up with apache config file and accidently deleted htdocs directory...

only rkhunter reported a sniffer, and it was pppoe, is this posible that pppoe could act as some kind of a sniffer?

and another question
currently i have 2.4.33.3 kernel
would it be more smart/safe to compile a new cernel 2.6.20.1?

oh, okay... well, one symptom less...

yes, considering what PPPoE is, it would make perfect sense for it to be used for sniffing... if i were you, i'd bust-out a known-good ppp package and a live cd and then compare checksums with those of the files installed on your box to rule-out a false-positive...

if your 2.4 kernel is working fine (sans the security vulnerabilities) i would recommend you simply upgrade to the latest 2.4 (2.4.34.1 at the time of this post)... and after that, make sure you stay up-to-date with the latest 2.4 stable release...

Not sure how useful this would be. You would only expect the executable to match (and hence the checksum to match) if the executables being compared were compiled by identical compilers using identical libraries, and identical compiler/linker options for identical target platforms.

Such a comparison would be valid if the pppoe being used as the baseline was from a disk image of THIS SYSTEM taken immediately after installation and prior to deployment (and hence therefore was presumably clean), where pppoe had not subsequently been upgraded (therefore changed). The comparison would also be valid on a package based distro where the package being used for the baseline was the same package that had originally been used for the install.

Comparing the source checksums would be useless; wouldn't identify whether the executable on the system had been built from that source.

If there is genuine concern about a rootkit in this case (though the OP seems to have found a likely culprit), probably the best thing to do is follow the protocol for cleaning a rootkitted computer, then install something like tripwire so that in the future it will be obvious.

@jiml8: my comment assumed he was using slackware, due to his profile and signature... so yeah, the idea is to compare the checksums of the binaries in the original package with the checksums of what's currently on your box... anyhow, thanks for your input...

Yep though it wont be that useful now unless he's planning to do a complete cleanup and reinstall etc. Oh and 1 more thing , I believe Tripwire is commercial now so you might want to look at Aide just incase you plan to start from scratch.

Cheers
Arvind