LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices



Reply
 
Search this Thread
Old 04-13-2008, 06:09 AM   #1
exodist
Senior Member
 
Registered: Aug 2003
Location: Portland, Oregon
Distribution: Gentoo
Posts: 1,372

Rep: Reputation: 46
Post ip route + ip tables + 2 lan ip's + 2 isps only 1 issue!


Problem
my lan network is 192.168.0.X/255.255.255.0
I have a server that provides dhcp, dns, and is a gateway.
The server is also a file server.

The server has 2 nic's for the lan, eth0 is 192.168.0.2 and eth1 is 192.168.0.1

I want all internet traffic to go over eth1 (also dns/dhcp)
I want all other traffic to go over eth0

What is working
Currently I have both interfaces responding to requests properly, as in if I ping 192.168.0.1 it responds from .1, if I ping .2 is responds on .2

I also have it set up so that any time I try to access a system on the lan from the server it uses eth0.

Where it breaks
HOWEVER, when I try to visit a web site with only eth1 connected it will resolve the dns and send the packets, but the packets do not return.

Testing / Trial and error
The computer I am using to test this can only be connected to the servers eth0 or eth1, not both at once, so during my testing I am 100% sure if I am, getting a response it is on the proper interface.

If I use either interface alone w/ the other disabled it works fine, but that is not what I need.

If I configure the server to use eth1 by default instead of eth0 when it accesses the lan computers then the setup works fine in every way except that connections established from the server are on the internet interface instead of the general one.

I have been playing with ip route and iptables for about 16 hours now. At one point 1 hour ago I finally found 2 commands that I ran at the shell that made it work as I wanted, so I added those lines to my config script and restarted... it does not work, so an option that I did not think was important from that shell session probably was important, but I have reviewed my bash history and none of it did it :-(

The Setup
I currently have a very complicated setup. 2 ISP's each providing a seperate internet connection. My server then takes both connections and uses 'ip route' to utilize both connections.

The server then provides dns using dnsmasq, and uses iptables to handle routing.

There are 2 seperate internal networks, one is eth4 192.168.1.1 and is bandwidth limited to 128000

the other is the lan in question w/ 2 interfaces

The script that handles it all
Code:
#The 'main' lan network interface
	/sbin/ifconfig eth0 192.168.0.2 netmask 255.255.255.0
	/sbin/ip route del 192.168.0.0/24 dev eth0
	/sbin/ip route add 192.168.0.0/24 dev eth0 src 192.168.0.2 table 1
	/sbin/ip rule add from 192.168.0.2 table 1

#The 'internet' providing interface
	/sbin/ifconfig eth1 192.168.0.1 netmask 255.255.255.0
	/sbin/ip route del 192.168.0.0/24 dev eth1
	/sbin/ip route add 192.168.0.0/24 dev eth1 src 192.168.0.1 table 2
	/sbin/ip rule add from 192.168.0.1 table 2

#The other internal network that is limited
	/sbin/ifconfig eth4 192.168.1.1 netmask 255.255.255.0

#ISP 1
	/sbin/ifconfig eth2 71.39.157.170 netmask 255.255.255.248
	/sbin/ip route add 71.39.157.168/29 dev eth2 src 71.39.157.170 table 3
	/sbin/ip route add default via 71.39.157.174 table 3
	/sbin/ip rule add from 71.39.157.170 table 3

#ISP2
	/sbin/ifconfig eth3 192.168.100.2 netmask 255.255.255.0
	/sbin/ip route add 192.168.100.0/24 dev eth3 src 192.168.100.2 table 4
	/sbin/ip route add default via 192.168.100.1 table 4
	/sbin/ip rule add from 192.168.100.2 table 4

#Switch between ISP's on new connections to utilize all bandwidth
	/sbin/ip route add default scope global nexthop via 71.39.157.174 dev eth2 weight 1\
						nexthop via 192.168.100.1 dev eth3 weight 3

#If mark 80 is present in the [packets/fames/?] use route table 2 (the internet providing interface) This is what I thought made it work when it was working for a short time.
	/sbin/ip rule add fwmark 80 table 2

# Default interface to use for 192.168.0.0 should be eth0
	/sbin/ip route add 192.168.0.0/24 dev eth0 src 0.0.0.0

# Try to route traffic from ISP 1 to eth1 if it is destined for 192.168.0.0... this did not work
	/sbin/ip route append 192.168.0.0/24 dev eth1 src 71.39.157.170

# Try to route traffic from ISP 2 to eth1 if it is destined for 192.168.0.0... this did not work
	/sbin/ip route append 192.168.0.0/24 dev eth1 src 192.168.100.2

#Set the nameserver(s)
	echo "nameserver 192.168.100.1" > /etc/resolv.conf
	echo "nameserver 71.39.157.174" >> /etc/resolv.conf

#Throttle traffic on the other internal network
	/sbin/tc qdisc add dev eth4 root handle 1: htb default 1
	/sbin/tc class add dev eth4 parent 1: classid 1:1 htb rate 128kbit burst 15k

#Set some variables for the iptables stuff.
	WIFI="eth4"
	LOCAL="eth1"
	CABLE="eth3"
	DSL="eth2"

	iptables -F
	iptables -t nat -F

	iptables -P INPUT ACCEPT
	iptables -P OUTPUT ACCEPT
	iptables -P FORWARD DROP
	#Then we lock our services so they only work from the LAN
	iptables -I INPUT 1 -i ${LOCAL} -j ACCEPT
	iptables -I INPUT 1 -i ${WIFI} -j ACCEPT
	iptables -I INPUT 1 -i lo -j ACCEPT
	iptables -A INPUT -p UDP --dport bootps -i ! ${LOCAL} -j REJECT
	iptables -A INPUT -p UDP --dport bootps -i ! ${WIFI} -j REJECT
	iptables -A INPUT -p UDP --dport domain -i ! ${LOCAL} -j REJECT
	iptables -A INPUT -p UDP --dport domain -i ! ${WIFI} -j REJECT

	#(Optional) Allow access to our server from the WAN
	for i in '80' '8080' '20' '21' '22' '24' '25' '37' '110' '113' '118' '123' '156' '194' '220' '389' '443' '465' '531' '989' '990' '993' '995'; do
		iptables -A INPUT -p TCP --dport ${i} -i ${CABLE} -j ACCEPT
		iptables -A INPUT -p TCP --dport ${i} -i ${DSL} -j ACCEPT
	done

	#Drop TCP / UDP packets to privileged ports
	iptables -A INPUT -p TCP -i ! ${LOCAL} -d 0/0 --dport 0:1023 -j DROP
	iptables -A INPUT -p UDP -i ! ${LOCAL} -d 0/0 --dport 0:1023 -j DROP
	iptables -A INPUT -p TCP -i ! ${WIFI} -d 0/0 --dport 0:1023 -j DROP
	iptables -A INPUT -p UDP -i ! ${WIFI} -d 0/0 --dport 0:1023 -j DROP

	#Finally we add the rules for NAT
	iptables -I FORWARD -i ${WIFI} -d 192.168.1.0/255.255.255.0 -j DROP
	iptables -A FORWARD -i ${WIFI} -s 192.168.1.0/255.255.255.0 -j ACCEPT
	iptables -I FORWARD -i ${LOCAL} -d 192.168.0.0/255.255.255.0 -j DROP
	iptables -A FORWARD -i ${LOCAL} -s 192.168.0.0/255.255.255.0 -j ACCEPT
	
	iptables -A FORWARD -i ${DSL} -d 192.168.0.0/255.255.255.0 -j ACCEPT
	iptables -A FORWARD -i ${CABLE} -d 192.168.0.0/255.255.255.0 -j ACCEPT
	iptables -A FORWARD -i ${DSL} -d 192.168.1.0/255.255.255.0 -j ACCEPT
	iptables -A FORWARD -i ${CABLE} -d 192.168.1.0/255.255.255.0 -j ACCEPT

	iptables -t nat -A POSTROUTING -o ${DSL} -j SNAT --to-source 71.39.157.170
	iptables -t nat -A POSTROUTING -o ${CABLE} -j SNAT --to-source 192.168.100.2
#This is the second part of what I thaught made this all work, it adds the mark 80 to the [packet/frame/?] - can someone tell me what exactly I am marking?
	iptables -t mangle -A POSTROUTING -s 71.39.157.170/255.255.255.248 -j MARK --set-mark 80
	iptables -t mangle -A POSTROUTING -s 192.168.100.2/255.255.255.0 -j MARK --set-mark 80
	
	#Tell the kernel that ip forwarding is OK
	echo 1 > /proc/sys/net/ipv4/ip_forward
	for f in /proc/sys/net/ipv4/conf/*/rp_filter ; do echo 1 > $f ; done
 
Old 04-13-2008, 10:59 PM   #2
exodist
Senior Member
 
Registered: Aug 2003
Location: Portland, Oregon
Distribution: Gentoo
Posts: 1,372

Original Poster
Rep: Reputation: 46
I have avoided the issue, I decided to just bond the interfaces together, separating internet and other services was meant to increase speed, I think bonding achieves this a lot better.

Instead of limiting the whole interface I just mangled the packets that were routed so I could limit them but not regular server use traffic.
 
  


Reply

Tags
balance, filter, ip, iptables, isp, load, network, nic, route, routing


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
IP tables -- re route a port Dan8080 Linux - Newbie 2 06-08-2006 12:43 AM
Blocking IP's with IP Tables blocks actual traffic ninjaz Linux - Security 5 04-05-2006 06:31 PM
IP Route/IP Tables depam Linux - Networking 2 10-12-2005 03:10 PM
how to route internal Networks by IP tables? quazidaniel Linux - Networking 4 10-12-2005 11:32 AM
howto relay ISPs dhcp ip's though linux router to LAN deice Linux - Networking 0 09-20-2004 12:40 PM


All times are GMT -5. The time now is 08:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration