Hello Forum:

I am working on a load balancing project. The type of routing I am implementing is direct routing (LVS-DIR). The following is a script that sets up the Linux Virtual Server (LVS) or the load balancer:

#!/bin/bash
#description: manages load balancer
#chkconfig: 35 20 80
#processname: lvsdr
. /etc/rc.d/init.d/functions

VIPeth00=192.168.2.2 #virtual ip on eth0:0
DIPeth0=192.168.2.3 #public (I know it is not public, just to get this running) real ip for lvs or load balancer on eth0
DIPeth1=192.168.1.1 #private real ip for lvs or load balancer on eth1
RIP1=192.168.1.3 #real server 1 private ip on eth0
RIP2=192.168.1.4 #real server 2 private ip on eth0

BCAST=$VIPeth0 #seeting the broadcast address to VIPeth00
NMASK=255.255.255.255

start () {
echo "Start LVS of Director Server"
#set the VIP and systcl parameter
/sbin/ifconfig eth0 $DIPeth0 up
/sbin/ifconfig eth0:0 $VIPeth00 netmask $NMASK broadcast $BCAST

/sbin/route add -host $DIPeth0 dev eth0
/sbin/ifconfig eth1 $DIPeth1 up

#ip forwarding on the lvs
echo "1" > /proc/sys/net/ipv4/ip_forward
/sbin/sysctl -p

#clear the IPVS table
/sbin/ipvsadm -C

#set LVS with web apache
# -t = TCP protocol
# -s = scheduler
# rr = round robin
# -p 120 = indicates the connection duration to 120 seconds
# -g = gatewaying
/sbin/ipvsadm -A -t $VIPeth00:80 -s rr # -p 120
/sbin/ipvsadm -a -t $VIPeth00:80 -r $RIP1:80 -g

/sbin/iptables -F

#run LVS
/sbin/ipvsadm -Ln
}

stop () {
echo "Close LVS Director Server"
echo "0" > /proc/sys/net/ipv4/ip_forward

/sbin/ipvsadm -C
#/sbin/ifconfig eth0:0 down
/sbin/ifconfig eth0 down
/sbin/ifconfig eth1 down
}


case "$1" in
start)
start
;;

stop)
stop
;;

restart)
stop
start
;;
*)
echo "Usage: $0 (start|stop|restart)"
exit 1
esac

Next is the script to set up real server1:
#!/bin/bash

#Description: Start real server 1
#chkconfig 2345
/etc/rc.d/init.d/functions

VIPlo0=192.168.2.2 #virtual ip on lo:0
RIP1eth0=192.168.1.3 #real server1 ip on eth0
GATEWAY=192.168.1.1 # is this right???

BCAST=$VIPlo0 #broadcast equal to VIPlo0
NMASK=255.255.255.255

start () {
echo "Start real server 1"
/sbin/ifconfig eth0 $RIP1eth0 up

/sbin/ifconfig lo:0 $VIPlo0 broadcast $BCAST netmask $NMASK
/sbin/route add -host $VIPlo0 dev lo:0

/sbin/route add default gw $GATEWAY

#to disable ARP for VIP:
echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

echo "1" > /proc/sys/net/ipv4/conf/eth0/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/eth0/arp_announce
}

stop () {
echo "Closing LVS of real server 1"
/sbin/ifconfig eth0 down
/sbin/route del -host $VIPlo0 dev lo:0
/sbin/ifconfig lo:0 down

echo "0" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" > /proc/sys/net/ipv4/conf/lo/arp_announce

echo "0" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" > /proc/sys/net/ipv4/conf/all/arp_announce

echo "0" > /proc/sys/net/ipv4/conf/eth0/arp_ignore
echo "0" > /proc/sys/net/ipv4/conf/eth0/arp_ignore
}


case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
*)
echo "Usage: $0 (start | stop | restart)"
exit 1
esac

On the LVS:
Interface configurations:
eth0 192.168.2.3
eth0:0 192.168.2.2 (This is the VIP as an alias)
eth1 192.168.2.1
lo 127.0.0.1

Routing tables:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.3 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0

On the real server 1 (I only have one real server setup so far):
Interface configurations:
eth0 192.168.1.3
lo 127.0.0.1
lo:0 192.168.2.2 (This is the VIP)

Routing tables:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.2 0.0.0.0 255.255.255.255 UH 0 0 0 lo
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0


On both, the LVS and Real Server 1, the firewalls (iptables) are not running.
A few questions I have are:
1) On the LVS, is apache supposed to be running?
2) If so, what would be the ServerName for apache on the LVS?
3) What would be the ServerName for apache on the real server1, is it the server real IP or the VIP?

When I try to access the web content on real server1 in a browser on the LVS, I get “Failed to Connect”. A connection to the server cannot be established.

The load balancer has two NICS, eth0 and eth1. The real servers will connect to the load balancer via eth1, and the client will connect via eth0.

Please help me find a solution to this problem.
Thanks.
--Willie

Well, I guess I will have to keep testing this thing until either I get it to work or it does not work at all.

Let's see what happens.

First to answer your questions:
1) No, the LVS is directing traffic but not acting as a proxy (Servers like NGINX function in the way you're thinking.)
2) N/A
3) All your real servers should share the same ServerName (whatever DNS is saying your VIP resolves to.)


Secondly there are a few issues with your configuration:

Your eth1 address should be in the 192.168.1.0. This should allow your LVS to route packets to your real server.

Finally once you get that fixed you can't test from the LVS to the real server:
http://www.austintek.com/LVS/LVS-HOW...O.html#gotchas


Hope this helps,
Sylvain

Thanks for your response sylvainsf. I will follow your suggestions and I will come back to post the results.

Just to clarify. The IP for DNS should be the VIP for realserver1 and realserver2? Does this apply to the load balancer?

Also, what would be the gateway of the real servers? I do not think it should be the load balancer because I am working on a LVS-DIR.

Thanks.

Correct, the site name (www.example.com) should resolve to the VIP. The VIP should be on realserver1 and realserver2 lo0:0. I didn't notice this before but you have it set to lo:0. I'm on a freebsd box at the moment so I can't check whether linux will accept that, but if things still don't work you should change it to lo0:0. The gateway for the real servers should be whatever device is routing traffic from 192.168.1.0 to 192.168.2.0 (this should not be your load balancer box.) If this doesn't work you should include the output of an ifconfig -a on both real servers and the LVS to help troubleshoot further.

1 members found this post helpful.

Hello sylvainsf:

I have the load balancer working. It is just one load balancer with two real servers for now. The next step will be to add another load balancer. The solution is a long one. I have it on this website: www.unixmultiverse.com.

I really want to thank you for helping me get this project to work.

Checked out your site and that's a pretty good writeup, one big thing to note: ifconfig will NOT show you LVS configured virtual IP's. You will need to use:
This will be important when you're testing failover of VIPs between two hosts.

I am trying to setup LVS-DR in Amazon EC2. I am planning to use LVS not amazon's elasticity loadbalancer for some reason :-)
Now, when I configure what will be my VIP.

In EC2, I have two instances, both the instances have the public IP and Private IP.