LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 03-19-2007, 07:53 AM   #1
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Rep: Reputation: 15
Question Multiple interfaces - All traffic flows through just one...


Hello Crew!

I'm maintaining a server running Mandrake 10.0. It is being used for video editing. It is equiped with 4 (four) NIC's (interfaces). Each interface is assinged a single IP.

eth0 - 192.168.3.200 mask 255.255.255.0
eth1 - 192.168.3.201 mask 255.255.255.0
eth2 - 192.168.3.202 mask 255.255.255.0
eth3 - 192.168.3.203 mask 255.255.255.0

These four NIC's are all attached to a switch with it's own VLAN (switch IP = 192.168.3.254)

There are 3 clients attached to the server:

EDIT4 - 1932.168.3.40 mask 255.255.255.0
EDIT5 - 1932.168.3.50 mask 255.255.255.0
EDIT6 - 1932.168.3.60 mask 255.255.255.0

Each client (Win XP Pro) has a host file which points the URL [\\server] of each of the clients to a different IP.

hosts files:

[edit4]
server 192.168.3.201

[edit5]
server 192.168.3.202

[edit6]
server 192.168.3.203

When the clients are working all the traffic they generate flows through just one NIC. You can imagine that one NIC cannot handle 3 or more streams of uncompressed video.

My question: How can I force a client to use a specific NIC on the server?

Cheers
 
Old 03-19-2007, 12:49 PM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
if they're all conecting to a single switch on one vlan, then you'd do a lot better to bond the interfaces, that will then be able to more effectively load share the data out of the server to the switch. check out the docs for the bonding module, very simple stuff really.
 
Old 03-20-2007, 02:58 AM   #3
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
Cheers Chris,

does this mean I have to bond the connection on the VLAN side as well? Probably eh? I'm not sure my switch is capable of that. I don't really need load balancing. Just pointing a client to a specific NIC will do. Suggestion?
 
Old 03-20-2007, 03:03 AM   #4
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
the vlan side? if you mean the switch, no. there are bonding modes, which do prefer this, e.g. IEEE 802.3ad bonding, which enables sharing in both directions, however if it's primarily just for data leaving the box, then there's no need for the switch to do anything. if i'm wrong in assuming that this is the majority of the data flow, then yeah you would want a more capable switch for bonding.
 
Old 03-21-2007, 03:59 AM   #5
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
What I don't understand is that all traffic ends up on one interface. Why do 4 different IP addresses answer with one MAC address. (server side) I assume this is the core of the problem...
 
Old 03-21-2007, 04:06 AM   #6
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
yeah that does seem odd, but a nicer solution which has that effect is often a better way round. what do your arp tables on a client show about the registered mac addresses? (run arp -n after pinging all 4 ip addresses)
 
Old 03-21-2007, 04:38 AM   #7
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
Somthing we aready expected...

Code:
? (192.168.3.200) at 0:4:23:b9:a5:18 on en0 [ethernet]
? (192.168.3.201) at 0:4:23:b9:a5:18 on en0 [ethernet]
? (192.168.3.202) at 0:4:23:b9:a5:18 on en0 [ethernet]
? (192.168.3.203) at 0:4:23:b9:a5:18 on en0 [ethernet]
 
Old 03-21-2007, 05:51 AM   #8
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
hmm, yeah what i'd expect i guess. seems very odd though. ultimately it'll be the os replying to the arp, not just the physical nic's, so could be doign something odd within the ip stack... can you show us the full output of ifconfig?
 
Old 03-22-2007, 05:18 AM   #9
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
Code:
eth0      Link encap:Ethernet  HWaddr 00:04:23:B9:A5:18  
          inet addr:192.168.3.200  Bcast:192.168.3.255  Mask:255.255.255.0
          inet6 addr: fe80::204:23ff:feb9:a518/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:709313689 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14986 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3178330538 (3031.0 Mb)  TX bytes:2453110 (2.3 Mb)
          Base address:0x2000 Memory:dd240000-dd260000 

eth1      Link encap:Ethernet  HWaddr 00:04:23:B9:A5:19  
          inet addr:192.168.3.201  Bcast:192.168.3.255  Mask:255.255.255.0
          inet6 addr: fe80::204:23ff:feb9:a519/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4169302 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4758 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1532392960 (1461.4 Mb)  TX bytes:304590 (297.4 Kb)
          Base address:0x2040 Memory:dd260000-dd280000 

eth2      Link encap:Ethernet  HWaddr 00:30:48:75:EC:BE  
          inet addr:192.168.3.202  Bcast:192.168.3.255  Mask:255.255.255.0
          inet6 addr: fe80::230:48ff:fe75:ecbe/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21207 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4758 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3520054 (3.3 Mb)  TX bytes:304590 (297.4 Kb)
          Base address:0x3000 Memory:dd300000-dd320000 

eth3      Link encap:Ethernet  HWaddr 00:30:48:75:EC:BF  
          inet addr:192.168.3.203  Bcast:192.168.3.255  Mask:255.255.255.0
          inet6 addr: fe80::230:48ff:fe75:ecbf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:71342620 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2136643151 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:204940968 (195.4 Mb)  TX bytes:1654470405 (1577.8 Mb)
          Base address:0x3040 Memory:dd320000-dd340000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:16783 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16783 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1789687 (1.7 Mb)  TX bytes:1789687 (1.7 Mb)

sit0      Link encap:IPv6-in-IPv4  
          inet6 addr: ::192.168.3.201/96 Scope:Compat
          inet6 addr: ::192.168.3.200/96 Scope:Compat
          inet6 addr: ::127.0.0.1/96 Scope:Unknown
          inet6 addr: ::192.168.3.203/96 Scope:Compat
          inet6 addr: ::192.168.3.202/96 Scope:Compat
          UP RUNNING NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
 
Old 03-23-2007, 05:52 AM   #10
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
If you can find the time:

http://linux-ip.net/linux-ip.pdf

From page 17. The ARP Flux Problem. I don't quite understand it. I have a feeling this might bare a solution to this problem.
 
Old 03-23-2007, 01:41 PM   #11
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
yeah that does look very promising. let us know how you get on with that would you?
 
Old 04-05-2007, 05:07 AM   #12
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
Crew!

I solved my issue with a work around.

On the server I created a route to the different hosts.

Code:
route add -host 192.168.3.106 255.255.255.0 eth3
Within the route I specified what NIC should be used for traffic for a particular host.
 
Old 04-05-2007, 05:40 AM   #13
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
interesting. i should say no no that's a bad cludge, but in the real world, you're in full control of that and it presumably doesn't need to scale at all...
 
Old 04-06-2007, 03:36 AM   #14
crazyivan
Member
 
Registered: Mar 2007
Distribution: Debian, Ubuntu server
Posts: 40

Original Poster
Rep: Reputation: 15
Chris,

I know it is not the kind of solution I had in mind. The article about the ARP Flux issue didn't solve it either. I also tried to, and this one is realy odd, create a static arp entry in the arp cache on the client machines. This worked on about half the clients. The other half (all two of them ;-) ) picked a random nic on the server to send it's traffic to. I could see traffic from the server > client using a different NIC then vice versa!

The only other option I can think of is to create entrys in both the arp cache's. In that case creating a static route as posted above is easier.


Configuring switches is rather new to me. What kind of support is needed on a switch when I want to bond my 4 NIC's ? I'd like to try.

Cheers
 
Old 04-06-2007, 12:41 PM   #15
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,398

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
it depends what kind of bonding you want. only IEEE 802.3ad mandates a bidirectional protocol. this is mode 4 for the linux bonding module. all others are generally indeifferent to the capabilities of a switch, which means it's down to arps and outbound traffic as to how they get used. so great for getting rid of traffic, but not so useful for recieving.
 
  


Reply

Tags
arp, bond, bonding, cache, force, interface, mac, multiple, nic, ping, static route, switch, traffic, video, vlan


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to route traffic on a network - cannot get machine to transfer across interfaces captainpotato Linux - Networking 15 10-04-2006 08:04 AM
natting traffic between 2 interfaces nukenstien Linux - Networking 2 02-13-2005 11:12 PM
Traffic on both interfaces geomonap Linux - Networking 1 01-13-2005 02:56 PM
Red Hat 7.3 and multiple gateways on multiple interfaces bluefmc Linux - Networking 2 11-19-2004 05:01 PM
Multiple interfaces prroblem liuyangtj Linux - Networking 9 09-25-2001 03:20 AM


All times are GMT -5. The time now is 01:17 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration