LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 01-15-2014, 09:22 AM   #1
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Rep: Reputation: 1
Network communication issue with lotsa interfaces (Scientific Linux 6)


I don't know the rules about linking to other forum threads, but I posted this on the Scientific Linux forums and haven't had much in the way of responses, so I thought I'd try a more Linux-generic forum with more users. Rather than re-typing or copy/pasting everything, I figured I'd just link to it:

http://scientificlinuxforum.org/inde...=ST&f=9&t=2597

I've provided most of my computer information there, but if additional information is needed let me know. The computer I'm working with is not on the network and there's no USB interfaces on it, so pasting command output results, if requested, will be difficult.

Fast forward, the issue I'm currently having (using the old version of Fedora) is this:

Computer 1 Arp Request to computer 2.
Computer 2 receives arp request.
Computer 2 sends arp reply.
Computer 1 never receives ARP reply.

When I had Scientific linux on the computer, it was communicating fine, but appeared to have a NIC limitation (would reliably communicate with approx 20 out of 24 NICs). I updated to SL previously because Fedora was no longer communicating with those NICs, and I thought it was an issue with communicating with newer link partners (protocols?).

Anyway, any help you can provide would be appreciated.
 
Old 01-15-2014, 03:34 PM   #2
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,886

Rep: Reputation: 162Reputation: 162
Too many interfaces. It could exceed NIC driver limit. You should check dmesg output whether any error in there.
By the way, does computer 1 or 2 have a lot of interface?
 
Old 01-15-2014, 03:45 PM   #3
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
Computer 2 has 6 PCI cards with 4 fiber optic NICs each for a total of 24 interfaces.

How can I find out the driver limit?

My concern though is how with Fedora 2, it could handle that number of interfaces, but with Scitentific Linux 6, it seems that it can't. So, I reimaged Computer 2 back to Fedora because at least all of the interfaces are receiving data, where with Scientific Linux, even though it would receive AND send, but seems to run into a limit of 20 interfaces.

The only thing that changed with the Fedora setup is that the "Computer 1s" are newer devices than what the setup was working with before and I don't know if there is some sort of protocol mismatch now.

Just a little while ago, I noticed that TCPDUMP on Computer 2 was seeing a message: along the lines of "ethertype Unknow (0x88cc). Googling it tells me that it's something to do with Link Layer Discovery Protocol. If this is what's causing my issue, I don't know where to begin trying to fix it.
 
Old 01-16-2014, 03:31 PM   #4
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,886

Rep: Reputation: 162Reputation: 162
How can I find out the driver limit?
1)Check dmesg output. Normally, NIC driver should report something when exceeding its limit.
2)Unplug 1 or more PCI card, whether issue is gone. You should find out how many interface Fedora 2 support?

Can you explain why you have to use Fedora 2? Fedora 2 is very old, now is Fedora 20. New Fedora could fix the issue.
 
Old 01-16-2014, 03:57 PM   #5
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
The dmesg output shows that "pcnet32: 24 cards found" with no errors indicated both on Scientific Linux and Fedora 2

To give a back story on Fedora this: This system was set up years ago, approximately 2004 when Fedora was at 2. The previous person responsible for this system had done just what you indicated, moving/removing cards to see how many interfaces Fedora could support and was able to get the system working with 24. For the system we've set up we actually needed 48 interfaces, but since only 24 could be supported we ended up with 2 identical computers, 24 ports each even though we had enough PCI slots for all 12 PCI cards (12x4 each).

This system is used to test optical networking switches in various environmental situations, which is why we're using so many individual interfaces.

The system worked with the optical network switches we were connecting to in the past. The only thing that changed is that we're connecting to a newer model switch. When we did that, we started seeing our communication issue.

Fast forward, I tried newer versions of Linux via live CDs and found that they would communicate, but don't appear to be able to handle the same number of interfaces.

It appears that something in the protocol has changed which makes our Fedora network setup incompatible, and with our Scientific Linux version it's compatible, but can't handle the number of interfaces. So, I have one system that can handle the number, but not the communication, and the other, vice versa.

What I'd like to do is either figure out how to get Scientific Linux to handle the number of interfaces, or figure out how to get Fedora to communicate with newer fiber optic link partners.

By the way, the network interface hardware is a Ramix (GE Fanuc?) 4 port fiber optic networking PMC card in a PCI carrier.

I may just go ahead and get a hold of a copy of the latest version of Fedora, or whichever is compatible with my hardware and give it a shot to see what happens. Meanwhile, if there is anything else I can try, please let me know.
 
Old 01-17-2014, 03:27 PM   #6
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,886

Rep: Reputation: 162Reputation: 162
Another way is a little hard. Newer Fedora can communicate with newer model switch. That means that new driver can talk to newer switch. If you can get Fedora 2 kernel and new driver source code, new driver can be built against Fedora 2 and then is uploaded to your system. It should solve everything.
If above way can't be done, using newer Fedora is right direction. Yes, you need resolve new issue.
 
1 members found this post helpful.
Old 01-17-2014, 04:05 PM   #7
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
The newest version of Fedora that appears to be compatible with the hardware I'm running (RAM, etc.) is Fedora 14. I'm going to give that a shot on Monday and go from there. If that doesn't work, I'll try recompiling the kernel with a newer version of the driver. I haven't recompiled a kernel in a loooooooong time, so we'll see how much I remember. I'll be sure to post results.

Thanks for the advice thus far!
 
Old 01-20-2014, 03:31 PM   #8
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
I've installed Fedora 14 on one of the two computers and after some configuration, it appears to be working as needed/expected. I'm installing on the second computer as I'm typing this.

I'm still wondering why Scientific Linux appears to have a limitation with the number of NICs.

Something I didn't try with SL6 was to install the full DVD (I installed it from a LiveCD), whereas with Fedora 14, I downloaded the entire DVD. I wouldn't think that would make much of a difference, but I could be wrong.
 
Old 01-21-2014, 01:13 PM   #9
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
Fedora 14 up on both computers and is working as needed. Thanks all for pointing me in the right direction. I don't know if I would consider this to be [SOLVED] as I wasn't able to fix the original issues with the original distributions, but at least my problem has been resolved (and my bosses will be happy).

Thanks again!
 
Old 01-21-2014, 03:50 PM   #10
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,886

Rep: Reputation: 162Reputation: 162
Each kernel and driver have limitation. Normally, they don't mention because 99% case just use maximum 4 or 5 NICs.
Newer always is better. That's why every provider keep updating their software.
 
Old 01-21-2014, 05:21 PM   #11
ericson007
Member
 
Registered: Sep 2004
Location: Japan
Distribution: RHEL9.4
Posts: 735

Rep: Reputation: 154Reputation: 154
Are you not just simply running out of irqs?

http://docs.oracle.com/cd/E24290_01/...e_perftune.htm

See 6.1.6
 
Old 01-22-2014, 06:55 AM   #12
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
I had thought that was an issue earlier on and looked at the IRQ/PIRQ/APIC stuff and did some tweaking, but noted no change in behavior.
 
Old 01-22-2014, 09:05 AM   #13
ericson007
Member
 
Registered: Sep 2004
Location: Japan
Distribution: RHEL9.4
Posts: 735

Rep: Reputation: 154Reputation: 154
That is peculiar. Why would a system digress after so many years. Have you confitrmed irq's being avalable. Have you changed hardware. Big difference between consumer and workstation class.
 
Old 01-22-2014, 10:02 AM   #14
watkinb
LQ Newbie
 
Registered: Jan 2014
Location: Virginia
Distribution: Fedora 2, Scientific Linux 6, PCLinuxOS, Mandrake 10.1, Whatever else I can play with.
Posts: 13

Original Poster
Rep: Reputation: 1
I wondered the same thing. No hardware on the computers had changed. The only hardware change was what the NICs were connected to. I'm thinking some newer protocol or something was preventing communication. The NICs are fiber-optic and circa 2004 or so, the fiber connections on the other end were manufactured in 2012.

As I said, I checked the IRQs and everything and did some tweaking. Nothing looked unusual and no tweaks changed the behavior.

Last edited by watkinb; 01-22-2014 at 10:03 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Dual screen scientific linux issue nefie Linux - Hardware 12 07-24-2013 02:17 PM
Force external communication between two interfaces on same box dlh Linux - Networking 1 05-07-2013 06:22 AM
Communication between two Network Interfaces (with IPv6 addresses) within same system peeyushpd Linux - Networking 2 03-18-2012 03:04 PM
Fault in KDE processes communication: Could not read network communication list Magnus Johansson MEPIS 0 03-30-2008 01:50 PM
Internet Connection issue with 2 Network interfaces Andrea_44 Ubuntu 1 10-05-2006 04:08 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 03:27 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration