Installing and configuring two identical PCI network cards on Debian

SonJelfn · 06-16-2011, 08:52 PM

Hello,

I am using a stock Debian 6.0.1 "Squeeze" GNU/Linux system and need to install and configure 3 network cards. I need all 3 network cards up and running.

The problem I am having is that 2 of these network cards are identical PCI cards and the kernel only lets me configure one. I don't have the option of changing them.

An lspci output shows all three cards:

Code:

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller (rev 02)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)
03:01.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

The kernel shows the following via the dmesg command:

Code:

[    0.905667] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    0.905683] r8169 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    0.905715] r8169 0000:02:00.0: setting latency timer to 64
[    0.905775] r8169 0000:02:00.0: irq 26 for MSI/MSI-X
[    0.906219] eth0: RTL8102e at 0xf7f7a000, 6c:62:6d:6e:ae:fc, XID 04c00000 IRQ 26
[    0.907554] ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
[    0.908948] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    0.908959] r8169 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    0.908972] r8169 0000:03:00.0: no PCI Express capability
[    0.909435] eth1: RTL8169sb/8110sb at 0xf7fa0c00, 00:14:d1:1f:d8:df, XID 10000000 IRQ 16
[    0.910400] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    0.910408] r8169 0000:03:01.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    0.910412] r8169 0000:03:01.0: region #1 not an MMIO resource, aborting
[    0.910455] r8169 0000:03:01.0: PCI INT A disabled

All the cards seem to be using the r8169 module. I have no idea how or why the kernel isn't assigning the third card a device.

I would really appreciate any help regarding the use of two identical Ethernet cards under GNU/Linux.

Thank you all in advance!

carltm · 06-16-2011, 08:59 PM

Question: what is the output of "ifconfig -a"?

SonJelfn · 06-16-2011, 09:45 PM

Hey carltm,

ifconfig -a shows the following:

Code:

eth0      Link encap:Ethernet  HWaddr 6c:62:6d:6e:ae:fc  
          inet addr:192.168.26.237  Bcast:192.168.26.255  Mask:255.255.255.0
          inet6 addr: fe80::6e62:6dff:fe6e:aefc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:18491 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2218 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1594407 (1.5 MiB)  TX bytes:685915 (669.8 KiB)
          Interrupt:26 Base address:0xa000 

eth1      Link encap:Ethernet  HWaddr 00:14:d1:1f:d8:df  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:16 Base address:0xc00 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:560 (560.0 B)  TX bytes:560 (560.0 B)

carltm · 06-17-2011, 07:54 AM

Normally it's not a problem to have two identical cards, so I'm
wondering if one of the cards is bad.

I would try removing one of the cards and rebooting to verify that
the first card is good. Then remove the first and install the
second and reboot.

LloydRice · 06-17-2011, 02:14 PM

Assuming you get this working, --- how does it work? My question is this -- Are devices plugged into one NIC separate from devices in the other NIC? Suppose you have two independent LANs. Is this possible? Will it work? Can it be done?

Lloyd

Reuti · 06-17-2011, 02:20 PM

Sure you can have independent LANs on different cards.

To the original question: google for "region #1 not an MMIO resource, aborting", some suggest to compile to use PIO instead MMIO into the kernel or buy two cards using other chips.

jefro · 06-17-2011, 03:56 PM

Move the cards around in the slots or reset bios. Looks to be an IRQ conflict. See also lspci.

SonJelfn · 06-17-2011, 07:10 PM

Hello all,

I have tested both cards, meaning that they both work.

I turned off the machine and moved the cards around, booting it up every time I changed things, including turning off the onboard network card which doesn't use the r8169 module directly.

Then one time I rebooted the machine I got all 3 Ethernet cards working.

The new lspci is:

Code:

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller (rev 02)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)
03:01.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

The new dmesg says:

Code:

[    0.911078] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    0.911096] r8169 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    0.911127] r8169 0000:02:00.0: setting latency timer to 64
[    0.911173] r8169 0000:02:00.0: irq 26 for MSI/MSI-X
[    0.911625] eth0: RTL8102e at 0xf8024000, 6c:62:6d:6e:ae:fc, XID 04c00000 IRQ 26
[    0.914477] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    0.914488] r8169 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    0.914505] r8169 0000:03:00.0: no PCI Express capability
[    0.914982] eth1: RTL8169sb/8110sb at 0xf8058c00, 00:14:d1:1f:cf:07, XID 10000000 IRQ 16
[    0.915735] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    0.915743] r8169 0000:03:01.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    0.915756] r8169 0000:03:01.0: no PCI Express capability
[    0.916755] eth2: RTL8169sb/8110sb at 0xf805cc00, 00:14:d1:1f:d8:df, XID 10000000 IRQ 17

The ifconfig now stands:

Code:

eth0      Link encap:Ethernet  HWaddr 6c:62:6d:6e:ae:fc  
          inet addr:192.168.26.237  Bcast:192.168.26.255  Mask:255.255.255.0
          inet6 addr: fe80::6e62:6dff:fe6e:aefc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1088 errors:0 dropped:0 overruns:0 frame:0
          TX packets:163 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:95864 (93.6 KiB)  TX bytes:51529 (50.3 KiB)
          Interrupt:26 Base address:0x4000 

eth1      Link encap:Ethernet  HWaddr 00:14:d1:1f:d8:df  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:17 Base address:0xcc00 

eth2      Link encap:Ethernet  HWaddr 00:14:d1:1f:cf:07  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:16 Base address:0x8c00

There is definitely an IRQ conflict but I seriously have no idea why it suddenly worked or how to make sure the next time I shut down the machine it doesn't change.

I haven't tried the kernel recompile and this shows that there is something else in play.

Anyone have any ideas?

jefro · 06-17-2011, 08:55 PM

What is there to play with? It works. Don't play with kernel.

I suspect you swapped the two cards and the way the board timing went it configured the cards without a conflict. I guess you can do a cold boot or two and see if stable.

SonJelfn · 06-17-2011, 09:14 PM

Hey jefro,

I'm not planning on playing with the kernel at all. Like you, I think the reboots and board timing eventually landed on the configuration I needed. However I would like to know if there is anything one can do to force the IRQ, I/O settings so that I can repeat this install without having to pray and reboot when I use two identical Ethernet cards.

Thanks for the suggestions and help.

carltm · 06-17-2011, 09:31 PM

Quote:

Originally Posted by SonJelfn

There is definitely an IRQ conflict but I seriously have no idea why it suddenly worked or how to make sure the next time I shut down the machine it doesn't change.

This sounds like the BIOS is assigning the IRQs. See if there is an
option in the BIOS to let the OS assign the IRQs. If so, Linux will
assign them consistently.

jefro · 06-18-2011, 02:21 PM

It is rare to see multiple cards fail. Most distros and drivers are advanced enough to get past this. At one time it was a stinker.

Still may be an issue for odd boards and poorly written drivers.

I think one could get into trouble still on 5 or more cards with some drivers.