LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Networking (https://www.linuxquestions.org/questions/linux-networking-3/)
-   -   two NICs problem (https://www.linuxquestions.org/questions/linux-networking-3/two-nics-problem-795597/)

devwatchdog 03-17-2010 06:29 AM

Is it possible to bring up the interface manually?

ifconfig eth1 up

Do the lights activate when you plug a cable into the device?

I see a MAC address in the ifcfg-eth1 file, but that address doesn't seem to match anything in the other various information your system presents. If you look at the output of lspci, you see the serial number listed for the working NIC, which matches the MAC in ifcfg-eth0. The serial number listed in lspci for the Broadcom BCM5722 is this:

Capabilities: [160] Device Serial Number 00-00-00-fe-ff-00-00-00

Whereas the MAC listed in ifcfg-eth1 is:

HWADDR=00:10:18:59:d7:23

I did a lookup on the MAC address found in ifcfg-eth1, and it seems to recognize that address as a proper Broadcom device. The serial number listed for it comes up as a Xerox device when converted to what its corresponding MAC would be.

I don't know if those serial numbers are directly related to MAC addresses or not. I've never really looked at them and am not familiar with them. It does seem odd the other card has a MAC address that matches its serial number, whereas the BCM5722 does not.

I work on a RHEL 5.3 system that doesn't show serial numbers in the output of 'lspci -vv' such as you see. This is the output I see:

Code:

02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5751 Gigabit Ethernet PCI Express (rev 01)
        Subsystem: Dell Unknown device 0180
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at dfdf0000 (64-bit, non-prefetchable) [size=64K]
        Expansion ROM at <ignored> [disabled]
        Capabilities: [48] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
                Address: 00001000000010a4  Data: 0008
        Capabilities: [d0] Express Endpoint IRQ 0
                Device: Supported: MaxPayload 128 bytes, PhantFunc 0, ExtTag+
                Device: Latency L0s <4us, L1 unlimited
                Device: AtnBtn- AtnInd- PwrInd-
                Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
                Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes
                Link: Supported Speed 2.5Gb/s, Width x1, ASPM L0s, Port 0
                Link: Latency L0s <4us, L1 <64us
                Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
                Link: Speed 2.5Gb/s, Width x1

There is another NIC on that machine, that being an Intel. Both interfaces work well enough on that system.

I would remove the MAC address from ifcfg-eth1. Just comment out the line and see if that works.

# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth1
#HWADDR=00:10:18:59:d7:23
BOOTPROTO=dhcp
ONBOOT=yes
USERCTL=no
IPV6INIT=no
PEERDNS=yes

How did you set up the configuration for eth1? Did the system do it during installation? I'm curious as to where that MAC address came from.

Does dmesg show a second card?

This is what I see:

Code:

Intel(R) PRO/1000 Network Driver - version 7.3.20-k2-NAPI
Copyright (c) 1999-2006 Intel Corporation.
ACPI: PCI Interrupt 0000:04:01.0[A] -> GSI 17 (level, low) -> IRQ 17
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
e1000: 0000:04:01.0: e1000_probe: (PCI:33MHz:32-bit) 00:0e:0c:69:8e:a4
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP,TRISTATE]
ACPI: PCI Interrupt 0000:00:1f.3[B] -> GSI 17 (level, low) -> IRQ 17
tg3.c:v3.96-1 (November 21, 2008)
ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:02:00.0 to 64
input: PC Speaker as /class/input/input2
eth1: Tigon3 [partno(BCM95751) rev 4001 PHY(5750)] (PCI Express) 10/100/1000Base-T Ethernet 00:11:11:e3:c3:ff
eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] WireSpeed[1] TSOcap[1]
eth1: dma_rwctrl[76180000] dma_mask[64-bit]

As jefro pointed out, if lspci sees the card, then it isn't disabled in the BIOS.

You might want to look through 'messages' in /var/log to see if there is anything regarding eth1 or the card itself there.

sheldonbai 03-17-2010 08:23 AM

***dmesg output after modprobe tg3***

tg3.c:v3.93 (May 22, 2008)
PCI: Enabling device 0000:09:00.0 (0000 -> 0002)
ACPI: PCI Interrupt 0000:09:00.0[A] -> GSI 17 (level, low) -> IRQ 185
PCI: Setting latency timer of device 0000:09:00.0 to 64
eth0: Tigon3 [partno(BCM95761) rev 5761100 PHY(5761)] (PCI Express) 10/100/1000Base-T Ethernet 00:25:64:a4:c4:14
eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] WireSpeed[1] TSOcap[1]
eth0: dma_rwctrl[76180000] dma_mask[64-bit]
PCI: Enabling device 0000:23:00.0 (0000 -> 0002)
ACPI: PCI Interrupt 0000:23:00.0[A] -> GSI 56 (level, low) -> IRQ 122
PCI: Setting latency timer of device 0000:23:00.0 to 64
tg3: Could not obtain valid ethernet address, aborting.
ACPI: PCI interrupt for device 0000:23:00.0 disabled
tg3: probe of 0000:23:00.0 failed with error -22
ADDRCONF(NETDEV_UP): eth0: link is not ready
tg3: eth0: Link is up at 10 Mbps, half duplex.
tg3: eth0: Flow control is off for TX and off for RX.
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready


***modinfo tg3***

filename: /lib/modules/2.6.18-128.el5/kernel/drivers/net/tg3.ko
version: 3.93
license: GPL
description: Broadcom Tigon3 ethernet driver
author: David S. Miller (davem@redhat.com) and Jeff Garzik (jgarzik@pobox.com)
srcversion: 9F10E7BFA7D69F890110EAC
alias: pci:v0000106Bd00001645sv*sd*bc*sc*i*
alias: pci:v0000173Bd000003EAsv*sd*bc*sc*i*
alias: pci:v0000173Bd000003EBsv*sd*bc*sc*i*
alias: pci:v0000173Bd000003E9sv*sd*bc*sc*i*
alias: pci:v0000173Bd000003E8sv*sd*bc*sc*i*
alias: pci:v00001148d00004500sv*sd*bc*sc*i*
alias: pci:v00001148d00004400sv*sd*bc*sc*i*
alias: pci:v000014E4d00001699sv*sd*bc*sc*i*
alias: pci:v000014E4d00001680sv*sd*bc*sc*i*
alias: pci:v000014E4d00001681sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Bsv*sd*bc*sc*i*
alias: pci:v000014E4d00001684sv*sd*bc*sc*i*
alias: pci:v000014E4d00001698sv*sd*bc*sc*i*
alias: pci:v000014E4d00001713sv*sd*bc*sc*i*
alias: pci:v000014E4d00001712sv*sd*bc*sc*i*
alias: pci:v000014E4d000016DDsv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001679sv*sd*bc*sc*i*
alias: pci:v000014E4d00001678sv*sd*bc*sc*i*
alias: pci:v000014E4d00001669sv*sd*bc*sc*i*
alias: pci:v000014E4d00001668sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Fsv*sd*bc*sc*i*
alias: pci:v000014E4d00001693sv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001674sv*sd*bc*sc*i*
alias: pci:v000014E4d00001673sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Bsv*sd*bc*sc*i*
alias: pci:v000014E4d00001672sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Asv*sd*bc*sc*i*
alias: pci:v000014E4d000016FEsv*sd*bc*sc*i*
alias: pci:v000014E4d000016FDsv*sd*bc*sc*i*
alias: pci:v000014E4d000016F7sv*sd*bc*sc*i*
alias: pci:v000014E4d00001601sv*sd*bc*sc*i*
alias: pci:v000014E4d00001600sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001677sv*sd*bc*sc*i*
alias: pci:v000014E4d00001676sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001659sv*sd*bc*sc*i*
alias: pci:v000014E4d00001658sv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Esv*sd*bc*sc*i*
alias: pci:v000014E4d00001649sv*sd*bc*sc*i*
alias: pci:v000014E4d0000170Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000170Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001696sv*sd*bc*sc*i*
alias: pci:v000014E4d000016C7sv*sd*bc*sc*i*
alias: pci:v000014E4d000016C6sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A8sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A7sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A6sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Dsv*sd*bc*sc*i*
alias: pci:v000014E4d00001654sv*sd*bc*sc*i*
alias: pci:v000014E4d00001653sv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Dsv*sd*bc*sc*i*
alias: pci:v000014E4d00001648sv*sd*bc*sc*i*
alias: pci:v000014E4d00001647sv*sd*bc*sc*i*
alias: pci:v000014E4d00001646sv*sd*bc*sc*i*
alias: pci:v000014E4d00001645sv*sd*bc*sc*i*
alias: pci:v000014E4d00001644sv*sd*bc*sc*i*
depends: libphy
vermagic: 2.6.18-128.el5 SMP mod_unload gcc-4.1
parm: tg3_debug:Tigon3 bitmapped debugging message enable value (int)
module_sig: 883f35049492f595cdc734e64d24fa112fbf609d10ffd1e3585d58cd258abb94b371f5752f3586509f5574c6dc86ad63a85a 1d52d895d00959935107f

sheldonbai 03-17-2010 08:34 AM

devwatchdog:

I configured eth1 manually. A second card did show up in dmesg as shown above. But the probe failed. The lights activate when I plug in a cable. I cannot bring up eth1 manually. The error message said there was no such device.

devwatchdog 03-17-2010 09:50 AM

Take a look through the results of this search. I think the problem is revealed with this:

tg3: probe of 0000:23:00.0 failed with error -22

Google Search

I haven't got time to look into it right now, but that search indicates there is a problem. Hopefully you can find a solution in there somewhere. If you have another gigabit ethernet card from Broadcom, I would suggest installing it to see if it works. From what I saw glancing through the results of that search, there were some bad eeproms involved.

nimnull22 03-17-2010 10:50 AM

NO no no you have to read a little bit earlier:
PCI: Enabling device 0000:23:00.0 (0000 -> 0002)
ACPI: PCI Interrupt 0000:23:00.0[A] -> GSI 56 (level, low) -> IRQ 122
PCI: Setting latency timer of device 0000:23:00.0 to 64
tg3: Could not obtain valid ethernet address, aborting.
ACPI: PCI interrupt for device 0000:23:00.0 disabled <---------------------- HERE
tg3: probe of 0000:23:00.0 failed with error -22
ADDRCONF(NETDEV_UP): eth0: link is not ready

That is why next line came up.

nimnull22 03-17-2010 11:12 AM

Suggestions, try to boot with kernel parameters - noapic nolapic
Add them to GRUB or whatever

Drakeo 03-17-2010 01:58 PM

they are both being loaded. If you would edit your net config so both come up. Like I showed in my last post. Or use your favorite GUI interface and enable eth1 . What is it in red hat Yum or something and configure the card.
Quote:

ifconfig eth1 up
is the command to get it going.
or. like I said by default when you installed it only enables one card.
You have to enable it.
Quote:

# Config information for eth1:IPADDR[1]=""
NETMASK[1]=""
USE_DHCP[1]="yes" <--- you can use "no" if there is a conflict of irq at boot time.
DHCP_HOSTNAME[1]=""
you need to configure it.

sheldonbai 03-17-2010 03:48 PM

Still didn't work. I've had it. I will try to get another NIC or just forget it. Anyway, thanks all for your time and suggestions.

devwatchdog 03-18-2010 08:39 AM

Quote:

Originally Posted by Drakeo (Post 3902157)
they are both being loaded. If you would edit your net config so both come up. Like I showed in my last post. Or use your favorite GUI interface and enable eth1 . What is it in red hat Yum or something and configure the card. is the command to get it going.
or. like I said by default when you installed it only enables one card.
You have to enable it.
you need to configure it.

Found at the bottom of post #9:

Code:

# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth1
HWADDR=00:10:18:59:d7:23
BOOTPROTO=dhcp
ONBOOT=yes
USERCTL=no
IPV6INIT=no
PEERDNS=yes

sheldonbai:

As a last few suggestion on the matter, if you have not removed the card already, I was thinking that trying to change the MAC of the NIC would be worth a try.

Found here:

Code:

# ifconfig eth1 hw ether 00:10:18:59:d7:23
# ifconfig eth1 up
# ifconfig eth1 |grep HWaddr

The MAC shown there is the one that shows up in your ifcfg-eth1 configuration file. You could use another MAC address.

Looking at the kernel you are using, it doesn't appear you have done any system updates lately. You might try running a system update. I'm maintaining a RHEL 5.3 server that is current on updates.

[jcwx@mercury ~]$ uname -a
Linux mercury 2.6.18-164.11.1.el5xen #1 SMP Wed Jan 6 14:01:18 EST 2010 i686 i686 i386 GNU/Linux
[jcwx@mercury ~]$

I'd guess your kernel is from somewhere in the range of a year ago. I've seen a number of bug reports related to the tg3 driver. My system is at release 3.96-1, so there have been some updates to the kernel module from the one you're using.

Finding another NIC is probably the easiest route to take, however. Doing a system update would be a good idea independent of whatever card you end up using.

Drakeo 03-19-2010 07:57 AM

Quote:

**/etc/sysconfig/networking-scripts/ifcfg-eth1***
# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth1
HWADDR=00:10:18:59:d7:23
BOOTPROTO=dhcp <----------why?
ONBOOT=yes <--------why ? device trying to connect at the same time.
USERCTL=no
IPV6INIT=no
PEERDNS=yes
you are asking your system to connect to the internet on a second device AT BOOT. when it is already connected to first device. this will not happen with dhcp un-less you are trying to connect to another local net. then you will have to set it up or configure it.

The first device is set to connect at boot. the second according to scripts will be void til brought up manual or if configured to come up after the first device. How by configuring a static ip not auto dhcp.

devwatchdog 03-20-2010 01:16 AM

Quote:

Originally Posted by Drakeo (Post 3904447)
you are asking your system to connect to the internet on a second device AT BOOT. when it is already connected to first device. this will not happen with dhcp un-less you are trying to connect to another local net. then you will have to set it up or configure it.

The first device is set to connect at boot. the second according to scripts will be void til brought up manual or if configured to come up after the first device. How by configuring a static ip not auto dhcp.

I don't see anything in this thread indicating that sheldonbai was attempting to configure the second interface as a public interface. Interfaces don't care if you're using public or private addressing schemes. Granted, only one default gateway is going to stick if one is requesting public addresses, and that is a problem that I imagine could be worked out -- but it would be a hell of a lot easier to simply shell out a few dollars for static IP addresses.

I have an OpenBSD device that I use for testing/basic routing. I set up a DHCP server on it, and commented out the line that defines the routers offered, which in this case was the default route. I then requested a lease from this system and it did just as I suspected -- there was no default route.

Even if sheldonbai was going to use two public interfaces on his system that wasn't cooperating, there is a strong possibility that with some scripting one could configure dual WANs with DHCP leases in conjunction with what is found here at the Linux Advanced Routing & Traffic Control HOWTO. It would be a definite PITA, but I don't see why it couldn't be done, either.

The point in what sheldonbai was having a problem with was the fact the interface never became active in the first place, so whether his intended configuration would work or not, it didn't matter because eth1 was never available. Even if the interface was improperly configured, it would still come up and show up in ifconfig. You don't have to have an interface configured to bring it up with 'ifconfig ethX up'.

I really have no doubt that if one were to load up a machine with 10 NICs, and tell them all simultaneously to request a DHCP lease, they would all do as told.

Drakeo 03-21-2010 08:11 AM

Quote:

I really have no doubt that if one were to load up a machine with 10 NICs, and tell them all simultaneously to request a DHCP lease, they would all do as told.
please read my first post. In red hat they use the same as bsd /etc/rc.d/rc.inet1.conf.
as I stated.
both of his devices are loaded do to dmesg so all he has to do is ifconfig eth1 up.
or edit his /etc/rc.d/rc.inet1.conf
I like you have multiple nic's on multiple systems and the problem should have been solved by now. Unless it is a hardware failure. or the firm ware is not being loaded on the second card.

devwatchdog 03-21-2010 12:10 PM

Quote:

Originally Posted by Drakeo (Post 3906450)
please read my first post. In red hat they use the same as bsd /etc/rc.d/rc.inet1.conf.
as I stated.
both of his devices are loaded do to dmesg so all he has to do is ifconfig eth1 up.
or edit his /etc/rc.d/rc.inet1.conf
I like you have multiple nic's on multiple systems and the problem should have been solved by now. Unless it is a hardware failure. or the firm ware is not being loaded on the second card.

On the RHEL/CentOS systems I've dealt with there are ifcfg-ethX files in /etc/sysconfig/network-scripts/, so the use of rc.inet1.conf appears to be isolated to Fedora.

I agree about if the interface can't be brought up with ifconfig, there are bigger issues.

On a side note, I looked for tg3 related firmware on a RHEL 5.3 system, and did not find anything that was mentioned in the various documentation I ran across on the web. Apparently the tg3 module will function without the firmware from Broadcom, as the firmware only adds certain functionality.


All times are GMT -5. The time now is 06:34 AM.