i have two internet providers that have been connected separately to the two modems (ethernet handoff). but i want to connect them together on one LAN. their gateway addresses (the address i route 0.0.0.0/0 to) and NAT addresses (private IPs my computers use) are different, so i think this is possible. both of them serve DHCP but if i set the computers up to not use DHCP (pre-configure the private IPs) i think the DHCP problems can be avoided. then i could also pre-configure the route table default gateway to which provider is selected for that computer.

now here is the hard part. i'd like to have one computer (running Ubuntu 16.04.3 with Linux kernel 4.4 use both providers at the same time with one connection going out via one provider and another connection to a different address on the internet going out via the other provider. there is only one network interface, but a 2nd one can be added if that would help. but there will only be a single LAN (three gigabit switches).

what setup can make this work on Linux? FYI, this is NOT bonding.

Hmmm... Interesting. I would solve that problem for a home network with a second (possibly virtual) interface and simple static routing. For a business I would use an Astaro Security Gateway from SOPHOS and set up multi-upstream rules. (ASG is the the easy and powerful way, but more expensive.)

The first question that comes to mind is "why have two ISPs for one home?", and the next is "what is the advantage in this, what are you really trying to achieve?"!

Before anyone can give you a viable response it would be nice to know what it is you are trying to accomplish with this setup as has already been stated.

i work entirely at home. i got the 2nd ISP in order to get more capacity to do some heavy network stuff like backups. it was running on a separate machine, so i could easily do it by dedicating that machine to the 2nd ISP. then the usage of the 1st ISP would not be bogged down. last week there was a 2 day outage of the 2nd ISP and i spent a lot of time and trouble making that machine use the 1st ISP, again, then go back to the 2nd ISP. that led me to think that it could have been the 1st ISP to go out, which would have been even harder to deal with. so that gave me the 2nd reason to have 2 ISPs (the possibility of an outage with just one ISP), the reason to keep the 2nd ISP, despite the fact that has some outage history.

the separate machine is doing backups (and a couple other things) and sends them to a couple offsite locations. it communicates with the other machines with a 2nd LAN (it gets to the 2nd ISP on the 2nd LAN), and i want to back down to just having one LAN and just one network interface on each machine. the Sun boxes only have one network interface so i had to set up some relays for them to be backed up.

for now the goal is to use a few specific sites via the 2nd ISP and everything else via the 1st ISP and do it all over a single LAN with just one network interface in each machine. everything runs Linux, OpenBSD, or Solaris. whatever is worked out on Linux i will try to replicate on the others. then i want to explore virtual-bonding of these 2 ISPs to combine the bandwidth and have quick takeover if one of them goes out. i am thinking of adding a 3rd ISP if that works. but i want to get the non-bonding one LAN and two ISPs basic setup working, first.

OK, you could setup your routing on each host to allow for multiple gateways. To do this you are going to have to set the metric of each link on your hosts.

You are fist going to have to know your gateways for both ISP's. Lets say they are as follows:

• ISP #1: 1.1.1.1
• ISP #2: 2.2.2.2

On your backup system you need to add a route with a lower cost than what is already there. If when you do a route -n and the metric is 0 then you are going to have to delete that GW and create a new one.

BACKUP MACHINE:
How you setup your routing with the original metric set to 0;
Then run your route again to see if both routes are in the table. You should see something like this;
HOSTS TO USE ISP #1:
This will get your routing setup for you. The only issue is you do not run any sort of dynamic routing protocol between you and your ISP's so your network connection migh not go off line when the ISP isn't working. You will have to disconnect the cable from the ISP that is having the outage in order for your systems to fail over to the other ISP.

the irony. soon after my last post i discovered ISP 2 went down while i was typing, then about an hour later ISP 1 went down. both remained down all night. we do have a 3rd ISP available, plus 2 phone networks served through about a dozen phone service resellers. i could get a smartphone and run that app that makes it into an access point for the final fallback reserved for limited usage.

so i should have no trouble by hard coding which ISP each machine uses while all on one LAN. ARP will find both gateways because they are different IPs and different MACs. i don't think i will use the metric 10 except on my desktop. i just need to disable or ignore or override DHCP?

I would configure them so that only ONE provided DHCP, but so that all could serve the same subnet. Then the traffic will be routed depending upon your gateway and route settings.

I'd be tempted to run a VM of some load balancing on Ubunutu. Generally when one wants to use two isp's they have to figure out how to split the use. They can select IP's to use or even programs/transport. In some advanced load balancing you might be able to switch over based on loading values. Not sure it would be easy in ubuntu.

unfortunately, i don't have access to disable DHCP on either ISP router. but at least they work with hard coded IPs in their respective /24. so i will need to have every machine not ask for DHCP or override after init is done. it looks it this will be easy on Ubuntu and Solaris.

i do plan to try out some load balancing next. i have an idea that does not involve a VM at all. what is your VM based idea? can it be done w/o a remote server?

There are a number of linux and bsd distro's that offer out of the box support. Some commercial products too. You'd install some sort of vm locally.

You'd still need to know how you want to balance. You can't just say connect two and be done with it. You have to select the way you want to share this resource.

i would probably go with round-robin balanced to about the relative bandwidths of the 2 ISPS. so far, every solution i have seen requires a server somewhere out there on the internet. a couple commercial ones put everyone on a server (selected out of many) that has other users. so it can look like there is no server because the pricing is all bundled together. using a VM is no big issue, but the idea i have does not use one.

Usually these distros are made to be perimeter sort of machines. You don't need a vm just that it is simple to test and run.

https://www.untangle.com/shop/WAN-Balancer/ thinking something like this.


Or maybe this. https://doc.pfsense.org/index.php/Multi-WAN

Using two ISP upstreams is not something home users normally do, it is something that businesses with a need for high availability do. A business can justify the expense of a perimeter device (firewall) that can balance and failover using two or more upstream paths. That is an advanced technique that even Cisco has never gotten right.

I have done it with home-grown devices, but the elegant way us using a machine MADE for that purpose. The best I have used is the Astaro Security Gateway from SOPHOS. This is NOT a cheap device, and there is an annual license update fee to keep modules active and updates current. I cannot recommend it for home use, the cost is simply not justified.

My solution, were I in need, would be to build a special machine to connect to your two ISP devices and serve DHCP to your internal network, and manage that upstream to balance routing intelligently and failover to a single upstream if the other failed. Only you can determine if your need justifies that level of investment in time and equipment.

PS. if you have a little hardware laying around, you might look at ZeroShell firewall distro. https://distrowatch.com/zeroshell