Linux HA: ipvsadm showing inconsistent routes

ptemmerman · 11-10-2008, 08:50 AM

ldirectord.cf:

Quote:

virtual=172.18.0.173:80
real=172.18.0.39:81 gate
real=172.18.0.40:81 gate
fallback=127.0.0.1:80 gate
service=http
checkport=81
protocol=tcp
request="ldirector.html"
receive="Test Page"
scheduler=rr
checktype=negotiate

ipvsadm -L -n shows:

Quote:

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.18.0.173:80 rr
-> 172.18.0.39:80 Route 0 0 0
-> 172.18.0.40:80 Route 0 0 0

Note that it says port 80 for my real servers, while I specified port 81 in the configuration file (see above).

Launching ldirectord in debug mode gives the following output:

Quote:

DEBUG2: Starting Check
DEBUG2: Starting HTTP/HTTPS
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://172.18.0.39:81/ldirector.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 10 bytes
LWP::UserAgent::request: Simple response: OK
DEBUG2: Finished HTTP/HTTPS
DEBUG2: Enabled real server=negotiate:http:tcp:172.18.0.39:81::81:1:gate:\/ldirector\.html:Test\ Page (virtual=tcp:172.18.0.173:80)
DEBUG2: check_http: http://172.18.0.39:81/ldirector.html is up
DEBUG2: Checking negotiate: real server=negotiate:http:tcp:172.18.0.40:81::81:1:gate:\/ldirector\.html:Test\ Page (virtual=tcp:172.18.0.173:80)
DEBUG2: check_http: url="http://172.18.0.40:81/ldirector.html" virtualhost="172.18.0.40"
LWP::UserAgent::new: ()
DEBUG2: Starting Check
DEBUG2: Starting HTTP/HTTPS
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://172.18.0.40:81/ldirector.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::UserAgent::request: Simple response: Internal Server Error
DEBUG2: Finished HTTP/HTTPS
DEBUG2: Disabled real server=negotiate:http:tcp:172.18.0.40:81::81:1:gate:\/ldirector\.html:Test\ Page (virtual=tcp:172.18.0.173:80)
DEBUG3: Headers Content-Type: text/plain
Client-Date: Sat, 08 Nov 2008 15:25:31 GMT
Client-Warning: Internal response
DEBUG2: check_http: http://172.18.0.40:81/ldirector.html is down

In the output it shows that it actually tries port 81, although in the ipvsadm I still get port 80, and thus my forwarding does not work.

Any clues?

ptemmerman · 11-10-2008, 10:04 AM

Found the explanation:

Quote:

For TCP and UDP (non fwmark) virtual services, unless the forwarding method is masq and the IP address of a real server is non-local (not present on a interface on the host running ldirectord) then the port of the real server will be set to that of its virtual service. That is, port-mapping is only available to if the real server is another machine and the forwarding method is masq. This is due to the way that the underlying LVS code in the kernel functions.

The entries are now shown correctly. Although, it doesn't seem to be masquerading correctly, because I get connection timeouts.

Quote:

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.18.0.173:80 rr
-> 172.18.0.39:81 Masq 1 0 0
-> 172.18.0.40:81 Masq 0 0 0