LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 06-19-2006, 02:43 PM   #1
kwc5811
LQ Newbie
 
Registered: Oct 2005
Location: College Station, TX
Distribution: Mandrake
Posts: 7

Rep: Reputation: 0
Unhappy Linux network interface going down


Hello all,

Sorry I am kind of a noob and I just need some help locating information. I have a Mandriva linux setup that I have been running for about a year and just recently I am having a problem with the network interface going down. After about a week or so of working fine I will notice the machine not responding to web requests. I try to ssh or ftp or whatever and I get no response. Even my ping requests are not returned. The machine seems to be running fine otherwise. I have no monitor attached so I am just wondering what logs I could look through to see if I can find a reason why this is happening. I looked at the Apache logs but of course it just shows a bunch of requests and then a length of time with no requests. I also looked at the syslog in /var/log and it shows no entries during the time the interface is down. The only log entry that runs every minute is
Code:
(root) CMD (   /usr/share/msec/promisc_check.sh)
and that was the last command run before the large time gap until I restarted the machine. (Maybe this points to a larger issue than the network interface) I googled the command but couldn't find any good explanations. I am just straight guessing that this stands for "promiscous check" which may be causing the networking problems. Any ideas? Thanks for your help.
 
Old 06-20-2006, 07:24 AM   #2
bernied
Member
 
Registered: Mar 2006
Location: Edinburgh, UK
Distribution: debian
Posts: 304

Rep: Reputation: 30
Have you looked at the hardware? An intermittent fault sounds like it could be hardware related.

Most network cards have LEDs either side of the socket - I've always assumed that one of them means 'connected' and the other one means 'exchanging data'. At least that's how they seem to behave.

Have a look at where the network cable plugs in - are the lights on?
Try downloading a page of something from somewhere - are there more lights?
Then try giving the cable a wiggle - do the lights change?

Then, you might need to move upstream - what's the other end of the cable plugged into? Are you sure that is working properly?
etc.

That script promisc_check.sh is on an old (long disused) Mandrake install of mine too. So it is probably not a problem - unless someone has tampered with it. The first part of mine looks like this (use the cat command to look at your own):

Code:
$cat [path to my old install]/promisc_check.sh | less

#!/bin/bash

# Writen by Vandoorselaere Yoann,
# <yoann@mandrakesoft.com>

Syslog() {
    if [[ ${SYSLOG_WARN} == yes ]]; then
        /sbin/initlog --string="${1}"
    fi
}

Ttylog() {
    if [[ ${TTY_WARN} == yes ]]; then
        w | grep -v "load\|TTY" | grep '^root' | awk '{print $2}' | while read line; do
            echo -e "${1}" > /dev/$line
        done
    fi
}

LogPromisc() {
    date=`date`
    Syslog "Security warning : $1 is in promiscuous mode."
    Syslog "    A sniffer is probably running on your system."
    Ttylog "\\033[1;31mSecurity warning : $1 is in promiscuous mode.\\033[0;39m"
    Ttylog "\\033[1;31mA sniffer is probably running on your system.\\033[0;39m"
    echo -e "\n${date} Security warning : $1 is in promiscuous mode." >> /var/log/security.log
    echo "    A sniffer is probably running on your system." >> /var/log/security.log
That's not all of it, but enough for me to think that it's just a friendly script that looks for baddies on your system. It doesn't look like it touches your network settings in any way.

(NB - don't ever use that cat command on a non-text file, all sorts of bad stuff can happen to your terminal)

Last edited by bernied; 06-20-2006 at 07:27 AM.
 
Old 06-20-2006, 07:29 AM   #3
blackhole54
Senior Member
 
Registered: Mar 2006
Posts: 1,896

Rep: Reputation: 61
By syslog do you mean /var/log/messages? (Sorry, I have not used Mandrake, but I think this is pretty standard.) You are certainly looking in the correct directory for the logs. I suppose it depends on what has actually happened whether something gets logged. Your /etc/syslog.conf will determine what log files different entries go to (check man syslog.conf)

If you have not solved this before it happens again, before rebooting I would check and make sure it is actually the interface that has gone down.
Running ifconfig will list all interfaces that are currently up. (Oops. I guess w/o a monitor and without being able to talk to the box via network you can't do that. You could set up a script that monitors this periodically and logs the result. If you have a serial port available, you can also set up inittab so that you can log in with a serial terminal or emulator program.)

I am guessing that promisc_check.sh is a bash script that cron is setup to run a check to make sure interfaces aren't in "promiscuous mode," which I believe (google on the term and check security forums) could be an indication your system has been compromised. If you are setup to run this once a minute, and it stops, that is troubling. If it is a script, you can look at promisc_check.sh with any text editor or less to see what it does.

I am going to try to tag my post with "security" so that people who know more than I do about this might look at this.
 
Old 06-20-2006, 07:34 AM   #4
bernied
Member
 
Registered: Mar 2006
Location: Edinburgh, UK
Distribution: debian
Posts: 304

Rep: Reputation: 30
Sorry, I'm a total novice with bash scripts, and didn't include the main part of the code. The main part of the script starts at the 'if' command, about half-way down. Here's the whole script:
Code:
# Writen by Vandoorselaere Yoann,
# <yoann@mandrakesoft.com>

Syslog() {
    if [[ ${SYSLOG_WARN} == yes ]]; then
        /sbin/initlog --string="${1}"
    fi
}

Ttylog() {
    if [[ ${TTY_WARN} == yes ]]; then
        w | grep -v "load\|TTY" | grep '^root' | awk '{print $2}' | while read line; do
            echo -e "${1}" > /dev/$line
        done
    fi
}

LogPromisc() {
    date=`date`
    Syslog "Security warning : $1 is in promiscuous mode."
    Syslog "    A sniffer is probably running on your system."
    Ttylog "\\033[1;31mSecurity warning : $1 is in promiscuous mode.\\033[0;39m"
    Ttylog "\\033[1;31mA sniffer is probably running on your system.\\033[0;39m"
    echo -e "\n${date} Security warning : $1 is in promiscuous mode." >> /var/log/security.log
    echo "    A sniffer is probably running on your system." >> /var/log/security.log

}

# *** Main bit starts here ***
if [[ -f /var/lib/msec/security.conf ]]; then
    . /var/lib/msec/security.conf
else
    echo "/var/lib/msec/security.conf doesn't exist."
    exit 1
fi

if [[ -f /etc/security/msec/security.conf ]]; then
    . /etc/security/msec/security.conf
fi

if tail /var/log/security.log | grep -q "promiscuous"; then
    # Dont flood with warning.
    exit 0
fi

# Check if a network interface is in promiscuous mode...

if [[ ${CHECK_PROMISC} == no ]]; then
    exit 0;
fi

for INTERFACE in `/sbin/ip link list | grep PROMISC | cut -f 2 -d ':';/usr/bin/promisc_check -q`; do
    LogPromisc ${INTERFACE}
done

# promisc_check.sh ends here
So this script is just setup to run every minute to check for possibly malicious logins on your system - seems like a good idea to me. You'll probably find a cron (a task scheduler, do a man cron to find out more) entry for it somewhere.

If yours looks like mine, it's almost certainly harmless - anyone disagree?
 
Old 06-20-2006, 07:37 AM   #5
bernied
Member
 
Registered: Mar 2006
Location: Edinburgh, UK
Distribution: debian
Posts: 304

Rep: Reputation: 30
Note that the only entry in the log was
Quote:
(root) CMD ( /usr/share/msec/promisc_check.sh)
which I assume just means that the script was run, which is good.

It doesn't mean that it found any nasties, which would have been bad.
 
Old 07-20-2006, 10:14 AM   #6
kwc5811
LQ Newbie
 
Registered: Oct 2005
Location: College Station, TX
Distribution: Mandrake
Posts: 7

Original Poster
Rep: Reputation: 0
Down again

Well the system stopped responding again yesterday evening. I have a Konfabulator Widget pinging the box every 2 minutes so I noticed almost immediately when it went down. None of the network services were responding and after reboot, when I went through the logs the previous auth entry was my ssh session about 2 hours before the network interface went down. Same thing with the syslog just dead ending with the promisc check. It would appear that it is not something I am doing that is causing the system to crash. It is also interesting that it was right at 30 days since the last crash, not sure if that is coincidence or not.

I tried plugging in a USB KB and a monitor and got a blank black screen. No amount of keypressing seemed to change this. I plugged in a PS2 KB so the next time I am pretty sure the keystrokes will be making it to the system and will help verify what at this point seems like a total system lockup. Not a common occurence on a linux OS, or so I thought!

Not really sure where to go from here to start looking at the possibility of hardware failure since the problem is not reproducable by outside stimulus. Ideas?
 
Old 07-20-2006, 11:36 AM   #7
bernied
Member
 
Registered: Mar 2006
Location: Edinburgh, UK
Distribution: debian
Posts: 304

Rep: Reputation: 30
Are you confident that it's not the power supply getting interrupted?
Does the machine get hot? Any hardware monitoring?

This might be a pain to set up, but you could put a serial console on the server. Then you'd get any info dumped from the kernel during a crash. You'd need another machine nearby that could take the output and put it in a file somewhere, and a null-modem cable between the two machines. I think you also need to have the serial console enabled in the kernel. And I think you need to specify to the kernel when you boot it (if you want it to be the system console - for boot and shutdown messages).
I have this in my grub menu.lst:
Code:
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal --timeout=15 serial console

# Standard most recently installed kernel with serial console
title=Gentoo Linux with serial console
root (hd0,0)
kernel /boot/vmlinuz root=/dev/hda2 console=tty0 console=ttyS0,9600n8
boot
The top two lines put grub onto the serial port (maybe not relevant to you), and that kernel 'console=ttyS0' option puts the system console onto the serial port.
Here's a howto: http://www.vanemery.com/Linux/Serial...l-console.html

Could it just be coincidence that the promisc check is the last thing on your logs, if it's running every minute?

Last edited by bernied; 07-20-2006 at 11:42 AM.
 
Old 07-21-2006, 10:54 AM   #8
jantman
Member
 
Registered: Nov 2005
Location: New Jersey, USA
Distribution: SuSE
Posts: 492

Rep: Reputation: 31
Hardware failure is possible.

This is a stab in the dark, but:

1) Have you checked through promisc_check.sh to make sure it is the same as the one posted by bernied?

2) I find it interesting that you said the problem is exactly 30 days from the last crash. Is that to the day, or to the minute? Timing like that would make me wonder if it can be more than a conicindence. Try a crontab -l at the command line (if you can get it) and take a look at your running cron jobs. Could there be something running at a certain interval that is causing this?

Now, I don't want to drive you insane by mentioning this, but if you have been able to get into the machine (serial terminal or local console) I'd take a look at EVERYTHING in /var/log and see if anything looks out of place.

Lastly, go through your logging configuration file, usually /etc/syslog.conf or /etc/syslog-ng/syslong-ng.conf depending on which facility you use, and make sure that you boost your logging to an appropriate level. Make sure cron is logged appropriately, and I would consider boosting console and any other relevant logs to "info" level or above.
 
  


Reply

Tags
network, security


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
No network interface Pig Monkey Slackware 5 12-21-2005 02:50 PM
Network interface phopkins Linux - Networking 2 04-27-2005 09:55 AM
Need help configuring an old linux box with wireless interface connect to network itrap2003 Linux - Wireless Networking 3 06-25-2004 12:50 PM
how can a program get network interface up/down event notifications in Linux manaskb Programming 1 10-22-2002 04:37 PM


All times are GMT -5. The time now is 04:38 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration