two NICs problem
 

I have a linux workstation running RHEL5 Client. It has two NICs, Broadcom NetXtreme BCM5761 at PCI:9:0:0 and BCM5722 at PCI:23:0:0. The drivers are both tg3 from broadcom. Is there a way to make both NICs recognizable by the system?

welcome to lq
yes it is possible, what you would do with both nics is the question since you really only need dual nics in specialized circumstances such as using your computer as a firewall/router or if you have a public web server that you want to restrict remote administration to within your physical lan
you might want to try editing /etc/network/interfaces (or whatever the equivilent is in RHEL) for something like this
or something like that
for the firewall
assuming that eth0 is the internet facing interface and eth1 is attached to your lan and then configure firewall rules to allow connections from your lan using eth0 as your gateway
otherwise you would want to configure BOTH static and configure your remote administration tools to only listen on eth1 and not allow communication between the 2 nics

note i'm not an expert in this area so i apologise for not being more specific but thats the best i can offer

Let me ask a dumb question.

Why are they not being recognized?

That was my dumb question. It seemed that only one card could be recognized at a time. Any ideas how to make both recognizable? Thanks.

Ok, that do mean by "be recognized"?
Can you post output of the command:
ifconfig -a

Thanks.

Might also see lspci results after ifconfig.

in red hat it uses the /etc/rc.d/rc.inet1.conf by default only one eth0 is enabled during install. so I would edit the /etc/rc.d/rc.inet1.conf like this
note only the first device eth0 will try to connect at boot. the other device should load but just listen. if you put yes on both they will both load and the first one will try to connect first
Please do like the others asked open a console and type lspci. and post it.
lets see if your modules and firm ware is loaded for the second device. lsmod for drivers look in your dmesg log for errors.

Here are the output of ifcontig and lspci.

***
ifcontig -a

eth0 Link encap:Ethernet HWaddr 00:25:64:A4:C4:14
inet6 addr: fe80::225:64ff:fea4:c414/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1026264 errors:67 dropped:0 overruns:0 frame:0
TX packets:61 errors:0 dropped:0 overruns:0 carrier:0
collisions:7 txqueuelen:1000
RX bytes:108833306 (103.7 MiB) TX bytes:18360 (17.9 KiB)
Interrupt:185

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1346 errors:0 dropped:0 overruns:0 frame:0
TX packets:1346 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2153428 (2.0 MiB) TX bytes:2153428 (2.0 MiB)

sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)


***
lspci | grep Ethernet

09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5761 Gigabit Ethernet PCIe (rev 10)
23:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722 Gigabit Ethernet PCI Express

***lspci -vv***
09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5761 Gigabit Ethernet PCIe (rev 10)
Subsystem: Dell Unknown device 026d
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 162
Region 0: Memory at f0ae0000 (64-bit, non-prefetchable) [size=64K]
Region 2: Memory at f0af0000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [40] Vital Product Data
Capabilities: [60] Vendor Specific Information
Capabilities: [50] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+
Address: 00000000fee06000 Data: 40a2
Capabilities: [cc] Express Endpoint IRQ 0
Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag+
Device: Latency L0s <4us, L1 unlimited
Device: AtnBtn- AtnInd- PwrInd-
Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes
Link: Supported Speed 2.5Gb/s, Width x1, ASPM L0s L1, Port 0
Link: Latency L0s <1us, L1 <64us
Link: ASPM L0s L1 Enabled RCB 64 bytes CommClk+ ExtSynch-
Link: Speed 2.5Gb/s, Width x1
Capabilities: [100] Advanced Error Reporting
Capabilities: [13c] Virtual Channel
Capabilities: [160] Device Serial Number 14-c4-a4-fe-ff-64-25-00
Capabilities: [16c] Power Budgeting

23:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5722 Gigabit Ethernet PCI Express
Subsystem: Broadcom Corporation NetXtreme BCM5722 Gigabit Ethernet PCI Express
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Interrupt: pin A routed to IRQ 122
Region 0: Memory at cfdf0000 (64-bit, non-prefetchable) [disabled] [size=64K]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Vendor Specific Information
Capabilities: [e8] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Address: 6695825ee4191df0 Data: 720f
Capabilities: [d0] Express Endpoint IRQ 0
Device: Supported: MaxPayload 128 bytes, PhantFunc 0, ExtTag+
Device: Latency L0s <4us, L1 unlimited
Device: AtnBtn- AtnInd- PwrInd-
Device: Errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 128 bytes, MaxReadReq 4096 bytes
Link: Supported Speed 2.5Gb/s, Width x1, ASPM L0s, Port 0
Link: Latency L0s <4us, L1 <64us
Link: ASPM Disabled RCB 64 bytes CommClk+ ExtSynch-
Link: Speed 2.5Gb/s, Width x1
Capabilities: [100] Advanced Error Reporting
Capabilities: [13c] Virtual Channel
Capabilities: [160] Device Serial Number 00-00-00-fe-ff-00-00-00


***/etc/sysconfig/hwconf (only network related stuff is shown. nowhere to find anything related to eth1)***
class: NETWORK
bus: PCI
detached: 0
device: eth0
driver: tg3
desc: "Broadcom Corporation NetXtreme BCM5761 Gigabit Ethernet PCIe"
network.hwaddr: 00:25:64:a4:c4:14
vendorId: 14e4
deviceId: 1681
subVendorId: 1028
subDeviceId: 026d
pciType: 1
pcidom: 0
pcibus: 9
pcidev: 0
pcifn: 0


***lsmod output***
Module Size Used by
vfat 46401 0
fat 85873 1 vfat
usb_storage 116641 1
ip6table_filter 36033 1
ip6_tables 50049 1 ip6table_filter
ip_conntrack_netbios_ns 36033 0
xt_state 35265 4
ip_conntrack 91109 2 ip_conntrack_netbios_ns,xt_state
nfnetlink 40457 1 ip_conntrack
iptable_filter 36161 1
ip_tables 55329 1 iptable_filter
libphy 53825 0
nls_utf8 35137 1
autofs4 57033 2
hidp 83521 2
rfcomm 104809 0
l2cap 89281 10 hidp,rfcomm
bluetooth 118597 5 hidp,rfcomm,l2cap
sunrpc 197897 1
ipt_REJECT 38849 1
ip6t_REJECT 38849 1
xt_tcpudp 36289 14
x_tables 50377 6 ip6_tables,xt_state,ip_tables,ipt_REJECT,ip6t_REJECT,xt_tcpudp
mptctl 116104 0
dm_multipath 55257 0
scsi_dh 41665 1 dm_multipath
video 53197 0
hwmon 36553 0
backlight 39873 1 video
sbs 49921 0
i2c_ec 38593 1 sbs
button 40545 0
battery 43849 0
asus_acpi 50917 0
acpi_memhotplug 40133 0
ac 38729 0
ipv6 424609 77 ip6t_REJECT
xfrm_nalgo 43333 1 ipv6
crypto_api 42945 1 xfrm_nalgo
lp 47121 0
snd_hda_intel 481777 1
snd_seq_dummy 37061 0
snd_seq_oss 65473 0
snd_seq_midi_event 41025 1 snd_seq_oss
snd_seq 87777 5 snd_seq_dummy,snd_seq_oss,snd_seq_midi_event
snd_seq_device 41557 3 snd_seq_dummy,snd_seq_oss,snd_seq
snd_pcm_oss 77377 0
nvidia 9656168 42
snd_mixer_oss 49985 1 snd_pcm_oss
sr_mod 50789 1
snd_pcm 116681 2 snd_hda_intel,snd_pcm_oss
cdrom 68713 1 sr_mod
snd_timer 57161 2 snd_seq,snd_pcm
snd_page_alloc 44113 2 snd_hda_intel,snd_pcm
snd_hwdep 43593 1 snd_hda_intel
serio_raw 40517 0
i2c_i801 41813 0
pcspkr 36289 0
parport_pc 62312 1
i2c_core 56129 3 i2c_ec,nvidia,i2c_i801
parport 73165 2 lp,parport_pc
snd 99433 11 snd_hda_intel,snd_seq_oss,snd_seq,snd_seq_device,snd_pcm_oss,snd_mixer_oss,snd_pcm,snd_timer,snd_hwd ep
tg3 154628 0
ohci1394 67672 0
sg 69993 0
ieee1394 132720 1 ohci1394
soundcore 41825 1 snd
dm_raid45 99025 0
dm_message 36161 1 dm_raid45
dm_region_hash 46145 1 dm_raid45
dm_mem_cache 38977 1 dm_raid45
dm_snapshot 51465 0
dm_zero 35265 0
dm_mirror 53065 0
dm_log 44865 3 dm_raid45,dm_region_hash,dm_mirror
dm_mod 100369 11 dm_multipath,dm_raid45,dm_snapshot,dm_zero,dm_mirror,dm_log
mptspi 54032 0
scsi_transport_spi 59841 1 mptspi
ahci 68809 2
libata 208721 1 ahci
shpchp 70637 0
mptsas 86288 5
mptscsih 73216 2 mptspi,mptsas
mptbase 121636 4 mptctl,mptspi,mptsas,mptscsih
scsi_transport_sas 66753 1 mptsas
sd_mod 56385 13
scsi_mod 196569 12 usb_storage,mptctl,scsi_dh,sr_mod,sg,mptspi,scsi_transport_spi,libata,mptsas,mptscsih,scsi_transport _sas,sd_mod
ext3 168017 6
jbd 94257 1 ext3
uhci_hcd 57433 0
ohci_hcd 55925 0
ehci_hcd 65741 0

***/etc/sysconfig/networking-scripts/ifcfg-eth0***
# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth0
HWADDR=00:25:64:a4:c4:14
BOOTPROTO=dhcp
ONBOOT=yes
USERCTL=no
IPV6INIT=no
PEERDNS=yes

**/etc/sysconfig/networking-scripts/ifcfg-eth1***
# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth1
HWADDR=00:10:18:59:d7:23
BOOTPROTO=dhcp
ONBOOT=yes
USERCTL=no
IPV6INIT=no
PEERDNS=yes

It can be one driver for 2 cards or it needs to 2 drivers?

Check, may be your card in black list?

Search through all LOG files for anything about ethernet cards. May be there is some information why second one does not use or load driver.

Thanks

The card is not in the blacklist. In the "messages" file, it simply said that "Device eth1 does not seem to present. delay initialization...". Does anyone experience anything like this before???

Do you have any BIOS settings regarding ethernet cards?

I cannot get the BIOS settings. The startup screen shows up only for half a second or something. None of the F# buttons works.

Since lspci sees it, I'd assume bios has it enabled but I could be wrong on that.

My guess is there isn't a native driver in your distro or a bug in the distro. Try a live cd or two to see if another config's it.

There is another thing we can do:
modprobe -r tg3
modprobe tg3

After it, please check dmesg.
Also please post output of: modinfo tg3

Thanks

Is it possible to bring up the interface manually?

ifconfig eth1 up

Do the lights activate when you plug a cable into the device?

I see a MAC address in the ifcfg-eth1 file, but that address doesn't seem to match anything in the other various information your system presents. If you look at the output of lspci, you see the serial number listed for the working NIC, which matches the MAC in ifcfg-eth0. The serial number listed in lspci for the Broadcom BCM5722 is this:

Capabilities: [160] Device Serial Number 00-00-00-fe-ff-00-00-00

Whereas the MAC listed in ifcfg-eth1 is:

HWADDR=00:10:18:59:d7:23

I did a lookup on the MAC address found in ifcfg-eth1, and it seems to recognize that address as a proper Broadcom device. The serial number listed for it comes up as a Xerox device when converted to what its corresponding MAC would be.

I don't know if those serial numbers are directly related to MAC addresses or not. I've never really looked at them and am not familiar with them. It does seem odd the other card has a MAC address that matches its serial number, whereas the BCM5722 does not.

I work on a RHEL 5.3 system that doesn't show serial numbers in the output of 'lspci -vv' such as you see. This is the output I see:

There is another NIC on that machine, that being an Intel. Both interfaces work well enough on that system.

I would remove the MAC address from ifcfg-eth1. Just comment out the line and see if that works.

# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth1
#HWADDR=00:10:18:59:d7:23
BOOTPROTO=dhcp
ONBOOT=yes
USERCTL=no
IPV6INIT=no
PEERDNS=yes

How did you set up the configuration for eth1? Did the system do it during installation? I'm curious as to where that MAC address came from.

Does dmesg show a second card?

This is what I see:

As jefro pointed out, if lspci sees the card, then it isn't disabled in the BIOS.

You might want to look through 'messages' in /var/log to see if there is anything regarding eth1 or the card itself there.

***dmesg output after modprobe tg3***

tg3.c:v3.93 (May 22, 2008)
PCI: Enabling device 0000:09:00.0 (0000 -> 0002)
ACPI: PCI Interrupt 0000:09:00.0[A] -> GSI 17 (level, low) -> IRQ 185
PCI: Setting latency timer of device 0000:09:00.0 to 64
eth0: Tigon3 [partno(BCM95761) rev 5761100 PHY(5761)] (PCI Express) 10/100/1000Base-T Ethernet 00:25:64:a4:c4:14
eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] WireSpeed[1] TSOcap[1]
eth0: dma_rwctrl[76180000] dma_mask[64-bit]
PCI: Enabling device 0000:23:00.0 (0000 -> 0002)
ACPI: PCI Interrupt 0000:23:00.0[A] -> GSI 56 (level, low) -> IRQ 122
PCI: Setting latency timer of device 0000:23:00.0 to 64
tg3: Could not obtain valid ethernet address, aborting.
ACPI: PCI interrupt for device 0000:23:00.0 disabled
tg3: probe of 0000:23:00.0 failed with error -22
ADDRCONF(NETDEV_UP): eth0: link is not ready
tg3: eth0: Link is up at 10 Mbps, half duplex.
tg3: eth0: Flow control is off for TX and off for RX.
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready


***modinfo tg3***

filename: /lib/modules/2.6.18-128.el5/kernel/drivers/net/tg3.ko
version: 3.93
license: GPL
description: Broadcom Tigon3 ethernet driver
author: David S. Miller (davem@redhat.com) and Jeff Garzik (jgarzik@pobox.com)
srcversion: 9F10E7BFA7D69F890110EAC
alias: pci:v0000106Bd00001645sv*sd*bc*sc*i*
alias: pci:v0000173Bd000003EAsv*sd*bc*sc*i*
alias: pci:v0000173Bd000003EBsv*sd*bc*sc*i*
alias: pci:v0000173Bd000003E9sv*sd*bc*sc*i*
alias: pci:v0000173Bd000003E8sv*sd*bc*sc*i*
alias: pci:v00001148d00004500sv*sd*bc*sc*i*
alias: pci:v00001148d00004400sv*sd*bc*sc*i*
alias: pci:v000014E4d00001699sv*sd*bc*sc*i*
alias: pci:v000014E4d00001680sv*sd*bc*sc*i*
alias: pci:v000014E4d00001681sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Bsv*sd*bc*sc*i*
alias: pci:v000014E4d00001684sv*sd*bc*sc*i*
alias: pci:v000014E4d00001698sv*sd*bc*sc*i*
alias: pci:v000014E4d00001713sv*sd*bc*sc*i*
alias: pci:v000014E4d00001712sv*sd*bc*sc*i*
alias: pci:v000014E4d000016DDsv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001679sv*sd*bc*sc*i*
alias: pci:v000014E4d00001678sv*sd*bc*sc*i*
alias: pci:v000014E4d00001669sv*sd*bc*sc*i*
alias: pci:v000014E4d00001668sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Fsv*sd*bc*sc*i*
alias: pci:v000014E4d00001693sv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001674sv*sd*bc*sc*i*
alias: pci:v000014E4d00001673sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Bsv*sd*bc*sc*i*
alias: pci:v000014E4d00001672sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Asv*sd*bc*sc*i*
alias: pci:v000014E4d000016FEsv*sd*bc*sc*i*
alias: pci:v000014E4d000016FDsv*sd*bc*sc*i*
alias: pci:v000014E4d000016F7sv*sd*bc*sc*i*
alias: pci:v000014E4d00001601sv*sd*bc*sc*i*
alias: pci:v000014E4d00001600sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001677sv*sd*bc*sc*i*
alias: pci:v000014E4d00001676sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001659sv*sd*bc*sc*i*
alias: pci:v000014E4d00001658sv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Esv*sd*bc*sc*i*
alias: pci:v000014E4d00001649sv*sd*bc*sc*i*
alias: pci:v000014E4d0000170Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000170Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001696sv*sd*bc*sc*i*
alias: pci:v000014E4d000016C7sv*sd*bc*sc*i*
alias: pci:v000014E4d000016C6sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A8sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A7sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A6sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Dsv*sd*bc*sc*i*
alias: pci:v000014E4d00001654sv*sd*bc*sc*i*
alias: pci:v000014E4d00001653sv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Dsv*sd*bc*sc*i*
alias: pci:v000014E4d00001648sv*sd*bc*sc*i*
alias: pci:v000014E4d00001647sv*sd*bc*sc*i*
alias: pci:v000014E4d00001646sv*sd*bc*sc*i*
alias: pci:v000014E4d00001645sv*sd*bc*sc*i*
alias: pci:v000014E4d00001644sv*sd*bc*sc*i*
depends: libphy
vermagic: 2.6.18-128.el5 SMP mod_unload gcc-4.1
parm: tg3_debug:Tigon3 bitmapped debugging message enable value (int)
module_sig: 883f35049492f595cdc734e64d24fa112fbf609d10ffd1e3585d58cd258abb94b371f5752f3586509f5574c6dc86ad63a85a 1d52d895d00959935107f

devwatchdog:

I configured eth1 manually. A second card did show up in dmesg as shown above. But the probe failed. The lights activate when I plug in a cable. I cannot bring up eth1 manually. The error message said there was no such device.

Take a look through the results of this search. I think the problem is revealed with this:

tg3: probe of 0000:23:00.0 failed with error -22

Google Search

I haven't got time to look into it right now, but that search indicates there is a problem. Hopefully you can find a solution in there somewhere. If you have another gigabit ethernet card from Broadcom, I would suggest installing it to see if it works. From what I saw glancing through the results of that search, there were some bad eeproms involved.

NO no no you have to read a little bit earlier:
PCI: Enabling device 0000:23:00.0 (0000 -> 0002)
ACPI: PCI Interrupt 0000:23:00.0[A] -> GSI 56 (level, low) -> IRQ 122
PCI: Setting latency timer of device 0000:23:00.0 to 64
tg3: Could not obtain valid ethernet address, aborting.
ACPI: PCI interrupt for device 0000:23:00.0 disabled <---------------------- HERE
tg3: probe of 0000:23:00.0 failed with error -22
ADDRCONF(NETDEV_UP): eth0: link is not ready

That is why next line came up.

Suggestions, try to boot with kernel parameters - noapic nolapic
Add them to GRUB or whatever

they are both being loaded. If you would edit your net config so both come up. Like I showed in my last post. Or use your favorite GUI interface and enable eth1 . What is it in red hat Yum or something and configure the card. is the command to get it going.
or. like I said by default when you installed it only enables one card.
You have to enable it.you need to configure it.

Still didn't work. I've had it. I will try to get another NIC or just forget it. Anyway, thanks all for your time and suggestions.

Found at the bottom of post #9:

sheldonbai:

As a last few suggestion on the matter, if you have not removed the card already, I was thinking that trying to change the MAC of the NIC would be worth a try.

Found here:

The MAC shown there is the one that shows up in your ifcfg-eth1 configuration file. You could use another MAC address.

Looking at the kernel you are using, it doesn't appear you have done any system updates lately. You might try running a system update. I'm maintaining a RHEL 5.3 server that is current on updates.

[jcwx@mercury ~]$ uname -a
Linux mercury 2.6.18-164.11.1.el5xen #1 SMP Wed Jan 6 14:01:18 EST 2010 i686 i686 i386 GNU/Linux
[jcwx@mercury ~]$

I'd guess your kernel is from somewhere in the range of a year ago. I've seen a number of bug reports related to the tg3 driver. My system is at release 3.96-1, so there have been some updates to the kernel module from the one you're using.

Finding another NIC is probably the easiest route to take, however. Doing a system update would be a good idea independent of whatever card you end up using.

you are asking your system to connect to the internet on a second device AT BOOT. when it is already connected to first device. this will not happen with dhcp un-less you are trying to connect to another local net. then you will have to set it up or configure it.

The first device is set to connect at boot. the second according to scripts will be void til brought up manual or if configured to come up after the first device. How by configuring a static ip not auto dhcp.

I don't see anything in this thread indicating that sheldonbai was attempting to configure the second interface as a public interface. Interfaces don't care if you're using public or private addressing schemes. Granted, only one default gateway is going to stick if one is requesting public addresses, and that is a problem that I imagine could be worked out -- but it would be a hell of a lot easier to simply shell out a few dollars for static IP addresses.

I have an OpenBSD device that I use for testing/basic routing. I set up a DHCP server on it, and commented out the line that defines the routers offered, which in this case was the default route. I then requested a lease from this system and it did just as I suspected -- there was no default route.

Even if sheldonbai was going to use two public interfaces on his system that wasn't cooperating, there is a strong possibility that with some scripting one could configure dual WANs with DHCP leases in conjunction with what is found here at the Linux Advanced Routing & Traffic Control HOWTO. It would be a definite PITA, but I don't see why it couldn't be done, either.

The point in what sheldonbai was having a problem with was the fact the interface never became active in the first place, so whether his intended configuration would work or not, it didn't matter because eth1 was never available. Even if the interface was improperly configured, it would still come up and show up in ifconfig. You don't have to have an interface configured to bring it up with 'ifconfig ethX up'.

I really have no doubt that if one were to load up a machine with 10 NICs, and tell them all simultaneously to request a DHCP lease, they would all do as told.

please read my first post. In red hat they use the same as bsd /etc/rc.d/rc.inet1.conf.
as I stated.
both of his devices are loaded do to dmesg so all he has to do is ifconfig eth1 up.
or edit his /etc/rc.d/rc.inet1.conf
I like you have multiple nic's on multiple systems and the problem should have been solved by now. Unless it is a hardware failure. or the firm ware is not being loaded on the second card.

On the RHEL/CentOS systems I've dealt with there are ifcfg-ethX files in /etc/sysconfig/network-scripts/, so the use of rc.inet1.conf appears to be isolated to Fedora.

I agree about if the interface can't be brought up with ifconfig, there are bigger issues.

On a side note, I looked for tg3 related firmware on a RHEL 5.3 system, and did not find anything that was mentioned in the various documentation I ran across on the web. Apparently the tg3 module will function without the firmware from Broadcom, as the firmware only adds certain functionality.