Multiwan routing problem

guna_pmk · 08-16-2012, 04:01 AM

Hello,

I am trying to setup a router with a fail over mechanism.

Here is the configuration.

Router
ClearOS 6.3
eth0 - Connected to ISP1 (high speed unlimited)
eth1 - connected to ISP2 (slower speed limited)
eth2 - connected to the LAN (IP Address 192.168.55.1/24)

In clearOS I have configured multiwan so that eth0 has a weight of 4 and eth1 of 1. I have not configured the DHCP server on the router machine. The idea is to use the secondary connection (eth1) if the primary one(eth0) fails.

I have hardcoded the nameservers (in resolv.conf) to 8.8.8.8 and 8.8.4.4 and prevented overwrites of resolv.conf as the switch over does not seem to handle this properly.

Workstation
On the work station side the configuration is as follows:

OS=>Fedora16
interface=>em1(static)
ipaddress=>192.168.55.101
Subnetmask=>255.255.255.0
Default Route(Gateway)=>192.168.55.1
DNS servers=>192.168.55.1

resolv.conf entry
nameserver 192.168.55.1

Note: ClearOS handles DNS caching (I hope)

Operation
When everything is connected my workstation is connecting to the internet, it could resolve names and all.

Here are the results of the some commands from the workstation:

ip route list

Code:

default via 192.168.55.1 dev em1  proto static 
192.168.55.0/24 dev em1  proto kernel  scope link  src 192.168.55.101  metric 1

Here is the out for the above commands from the router at the same time

ip route list

Code:

default via <eth0 ip> dev eth0  proto static 
192.168.55.0/24 dev eth2  proto kernel  scope link  src 192.168.55.1
<eth1 network>/24 dev eth1  proto kernel  scope link  src <eth1 IP>
<eth0 network>/24 dev eth0  proto kernel  scope link  src <eth0 IP>

When I remove connection from eth0 on the router, it takes a bit for the routes on the router to get updated; probably after about a minute the routes on the router get updated. And the result of ip route list on the router is as follows:

Code:

default via <eth1 ip> dev eth1  proto static 
192.168.55.0/24 dev eth2  proto kernel  scope link  src 192.168.55.1
<eth1 network>/24 dev eth1  proto kernel  scope link  src <eth1 IP>

At the this point the router is able to connect to the internet (obviously through the second connection), able to resolve names and all.

But the workstation neither could resolve names nor could reach the internet through ip addresses. At the same time the workstation could connect to the router. The route ie. the result of ip route list is unchanged on the workstation. I can able to ssh into the router from the workstation. I know I am nearly there; but don't know how to proceed.

Can anybody gues what it is and could probably help?

Thanks

nikmit · 08-17-2012, 09:30 AM

How is your NAT set up?
I suspect it only translates to the address of your main connection.

guna_pmk · 08-20-2012, 03:42 AM

Hi nikmit,

Thanks very much for this. Though it looks like what you say, I am not a very advanced user (especially not when it comes to networking). So so can you please expand your answer a bit?

Or

You mean that it receives the incoming connection but routes it to the main connection? Should it not take the "default via" route? As I said, the router itself could connect to the internet through the correct interface. Should it not do the same with the incoming connection?

And

Also please be noted that I am using ClearOs which may have added an additional layer on top of the Linux's native way of routing (I think ClearOS 6.3 is based on some 6.x version of Centos).

Please let me know if I miss anything or need anymore information in this regard.

Cheers

deadeyes · 08-20-2012, 04:48 AM

Quote:

Originally Posted by guna_pmk

Hi nikmit,

Thanks very much for this. Though it looks like what you say, I am not a very advanced user (especially not when it comes to networking). So so can you please expand your answer a bit?

Or

You mean that it receives the incoming connection but routes it to the main connection? Should it not take the "default via" route? As I said, the router itself could connect to the internet through the correct interface. Should it not do the same with the incoming connection?

And

Also please be noted that I am using ClearOs which may have added an additional layer on top of the Linux's native way of routing (I think ClearOS 6.3 is based on some 6.x version of Centos).

Please let me know if I miss anything or need anymore information in this regard.

Cheers

I would think a distro is not simply "adding" a layer on the native way of routing.
It probably is a CentOS with some modifications and specific packages installed.

guna_pmk · 08-21-2012, 03:25 AM

Quote:

It probably is a CentOS with some modifications and specific packages installed.

Yes you are right. I have to be careful when using the word layer especially on a Networking forum.

Thanks

guna_pmk · 08-22-2012, 03:34 AM

Hi,

nikmit's suggestion led me to some progress.

Recap: As I have mentioned earlier, with all the connections up and alive, no problem. As soon as I unplug the main internet the workstation looses internet connection; but connection with the router is fine.

Meanwhile I have changed the secondary internet connection to a ppp0. After disconnecting the primary internet I checked the nat table. It was empty. Therefore I added nat rules as follows on the router:

Code:

iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
iptables -A FORWARD -i ppp0 -o eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i eth2 -o ppp0 -j ACCEPT

and now the workstation can connect to the internet.

If the primary internet is made alive, it automatically overwrites the nat table, which is what I want. What I am wondering is some script is run when a link is down and some script is run when it comes back which adds and removes those nat rules. What scrripts are they, so I can add the above to automate the failover. Does anybody know what it is?

Thanks

guna_pmk · 08-24-2012, 10:09 AM

I now have found out what it is.

In ClearOS there is a service called syswatch which runs a perl script /usr/sbin/syswatch. This runs an infinite loop which checks the connection status of all the network interfaces that participate in the multiwan.

The problem is, when the ppp goes down, it continuously tries to bring it up. A single attempt takes about 2 minutes, especially when there is no link at all. During this period, the default route is removed (and put back when the ppp fails to come back) and hence there is no outgoing connection. This actually kills the purpose of having multiple connections by keeping disconnecting the LAN from reaching the internet alternatively. As I have spent much time on it and I can afford loosing the internet for couple of minutes(to manually interfere to stop trying for ppp), I now have left it to be performed manually.

On the other side, though it should be through the syswatch, I did not (and have no time time to) spend anytime on finding out why the nat is not added when the primary connection goes down. I have decided to do it manually (time factor) when something like this happens.

Please let me know of any suggestions or have any questions in this regard.

Cheers