LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Networking (http://www.linuxquestions.org/questions/linux-networking-3/)
-   -   KVM virtual networks not routing properly (http://www.linuxquestions.org/questions/linux-networking-3/kvm-virtual-networks-not-routing-properly-931842/)

theillien 02-28-2012 08:57 PM

KVM virtual networks not routing properly
 
I have a physical host running RHEL 6 with KVM as a sandbox so I can study for the RHCSA exam. It gets its IP address from my home router on network 192.168.1.0/24.

There are two VMs running on two separate virtual networks via KVM. One on 192.168.122.0/24 (vibr0)and the other on 192.168.100.0/24 (virbr1).

In a previous instance of this configuration the physical host would route traffic from each VM to the home network. I could ping 192.168.1.1 from each VM and obtain DNS resolution. This doesn't seem to be working in the current iteration.

I cannot ping across subnets including to and from the home network to each of the VMs and physical host. From the physical host I can ping the gateway to each of the virtual networks and can also ping the gateway from within the corresponding virtual network. However, the physical host cannot ping either of the two VMs. Of course, I can ping other VMs within each virtual network and the home router from the physical host as well.

Not being as masterful of networking as I probably should be, I'm at a loss as to what to look for. It seems that forwarding isn't working as it should, but I can't figure out how KVM manages iptables. If I run iptables -L I get all kinds of output with no associated rules in /etc/sysconfig/iptables.

Routing table:
Code:

[root@study networks]# netstat -rn
Kernel IP routing table
Destination    Gateway        Genmask        Flags  MSS Window  irtt Iface
192.168.100.0  0.0.0.0        255.255.255.0  U        0 0          0 virbr1
192.168.1.0    0.0.0.0        255.255.255.0  U        0 0          0 eth0
192.168.122.0  0.0.0.0        255.255.255.0  U        0 0          0 virbr0
169.254.0.0    0.0.0.0        255.255.0.0    U        0 0          0 eth0
0.0.0.0        192.168.1.1    0.0.0.0        UG        0 0          0 eth0

iptables -L
Code:

[root@study networks]# iptables -L
Chain INPUT (policy ACCEPT)
target    prot opt source              destination
ACCEPT    udp  --  anywhere            anywhere            udp dpt:domain
ACCEPT    tcp  --  anywhere            anywhere            tcp dpt:domain
ACCEPT    udp  --  anywhere            anywhere            udp dpt:bootps
ACCEPT    tcp  --  anywhere            anywhere            tcp dpt:bootps
ACCEPT    udp  --  anywhere            anywhere            udp dpt:domain
ACCEPT    tcp  --  anywhere            anywhere            tcp dpt:domain
ACCEPT    udp  --  anywhere            anywhere            udp dpt:bootps
ACCEPT    tcp  --  anywhere            anywhere            tcp dpt:bootps
ACCEPT    all  --  anywhere            anywhere            state RELATED,ESTABLISHED
ACCEPT    icmp --  anywhere            anywhere
ACCEPT    all  --  anywhere            anywhere
ACCEPT    tcp  --  anywhere            anywhere            state NEW tcp dpt:ftp-data
ACCEPT    tcp  --  anywhere            anywhere            state NEW tcp dpt:ftp
ACCEPT    tcp  --  anywhere            anywhere            state NEW tcp dpt:ssh
REJECT    all  --  anywhere            anywhere            reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target    prot opt source              destination
ACCEPT    all  --  anywhere            192.168.100.0/24    state RELATED,ESTABLISHED
ACCEPT    all  --  192.168.100.0/24    anywhere
ACCEPT    all  --  anywhere            anywhere
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
ACCEPT    all  --  anywhere            192.168.122.0/24    state RELATED,ESTABLISHED
ACCEPT    all  --  192.168.122.0/24    anywhere
ACCEPT    all  --  anywhere            anywhere
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
REJECT    all  --  anywhere            anywhere            reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target    prot opt source              destination

For a different look, service iptables status:
Code:

[root@study networks]# service iptables status
Table: nat                                   
Chain PREROUTING (policy ACCEPT)             
num  target    prot opt source              destination       

Chain POSTROUTING (policy ACCEPT)
num  target    prot opt source              destination       
1    MASQUERADE  tcp  --  192.168.100.0/24    !192.168.100.0/24    masq ports: 1024-65535
2    MASQUERADE  udp  --  192.168.100.0/24    !192.168.100.0/24    masq ports: 1024-65535
3    MASQUERADE  all  --  192.168.100.0/24    !192.168.100.0/24                         
4    MASQUERADE  tcp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535
5    MASQUERADE  udp  --  192.168.122.0/24    !192.168.122.0/24    masq ports: 1024-65535
6    MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24                         

Chain OUTPUT (policy ACCEPT)
num  target    prot opt source              destination       

Table: mangle
Chain PREROUTING (policy ACCEPT)
num  target    prot opt source              destination       

Chain INPUT (policy ACCEPT)
num  target    prot opt source              destination       

Chain FORWARD (policy ACCEPT)
num  target    prot opt source              destination       

Chain OUTPUT (policy ACCEPT)
num  target    prot opt source              destination       

Chain POSTROUTING (policy ACCEPT)
num  target    prot opt source              destination       
1    CHECKSUM  udp  --  0.0.0.0/0            0.0.0.0/0          udp dpt:68 CHECKSUM fill
2    CHECKSUM  udp  --  0.0.0.0/0            0.0.0.0/0          udp dpt:68 CHECKSUM fill

Table: filter
Chain INPUT (policy ACCEPT)
num  target    prot opt source              destination
1    ACCEPT    udp  --  0.0.0.0/0            0.0.0.0/0          udp dpt:53
2    ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          tcp dpt:53
3    ACCEPT    udp  --  0.0.0.0/0            0.0.0.0/0          udp dpt:67
4    ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          tcp dpt:67
5    ACCEPT    udp  --  0.0.0.0/0            0.0.0.0/0          udp dpt:53
6    ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          tcp dpt:53
7    ACCEPT    udp  --  0.0.0.0/0            0.0.0.0/0          udp dpt:67
8    ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          tcp dpt:67
9    ACCEPT    all  --  0.0.0.0/0            0.0.0.0/0          state RELATED,ESTABLISHED
10  ACCEPT    icmp --  0.0.0.0/0            0.0.0.0/0
11  ACCEPT    all  --  0.0.0.0/0            0.0.0.0/0
12  ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          state NEW tcp dpt:20
13  ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          state NEW tcp dpt:21
14  ACCEPT    tcp  --  0.0.0.0/0            0.0.0.0/0          state NEW tcp dpt:22
15  REJECT    all  --  0.0.0.0/0            0.0.0.0/0          reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
num  target    prot opt source              destination
1    ACCEPT    all  --  0.0.0.0/0            192.168.100.0/24    state RELATED,ESTABLISHED
2    ACCEPT    all  --  192.168.100.0/24    0.0.0.0/0
3    ACCEPT    all  --  0.0.0.0/0            0.0.0.0/0
4    REJECT    all  --  0.0.0.0/0            0.0.0.0/0          reject-with icmp-port-unreachable
5    REJECT    all  --  0.0.0.0/0            0.0.0.0/0          reject-with icmp-port-unreachable
6    ACCEPT    all  --  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED
7    ACCEPT    all  --  192.168.122.0/24    0.0.0.0/0
8    ACCEPT    all  --  0.0.0.0/0            0.0.0.0/0
9    REJECT    all  --  0.0.0.0/0            0.0.0.0/0          reject-with icmp-port-unreachable
10  REJECT    all  --  0.0.0.0/0            0.0.0.0/0          reject-with icmp-port-unreachable
11  REJECT    all  --  0.0.0.0/0            0.0.0.0/0          reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
num  target    prot opt source              destination

If anyone can help me sort this out I would appreciate it.

mike_rhce 02-29-2012 11:24 PM

Hi theillen,

If you suspect that there's a problem with firewall rules, try flushing firewall rules (iptables -F).

My gut -- there might be something slightly off with the KVM network configuration. Write down the current settings for networks (or better yet, back up the contents of the /etc/libvirt/qemu/networks directory).

Use the Virtual Machine Manager to delete --both-- the default and outsider networks and then add them back in. Make sure you use NAT when re-creating the networks.

theillien 03-01-2012 08:12 PM

Thanks again. I don't know if it was flushing iptables or recreating the networks but one of the two did it. More to add to my personal knowledgebase.

joker20 04-18-2012 07:42 PM

i know this is marked as solved but i just wanted to chime in for anyone else stumbling across this...
if you have multiple KVM virtual networks the iptables that get autogenerated are incorrect and setup for failure. this is probably a default approach to keep them seperated, but if you want communication between them you'll need to modify the rules...i even go as far as removing some of the REJECT rules to make it cleaner. e.g

you'll need to add NEW to both initial rules for each virtual network, this needs to happen because if you follow it down the list nothing will match and will get rejected.
Code:

Chain FORWARD (policy ACCEPT)
target    prot opt source              destination
ACCEPT    all  --  anywhere            192.168.100.0/24    state NEW,RELATED,ESTABLISHED
ACCEPT    all  --  192.168.100.0/24    anywhere
ACCEPT    all  --  anywhere            anywhere
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
ACCEPT    all  --  anywhere            192.168.122.0/24    state NEW,RELATED,ESTABLISHED
ACCEPT    all  --  192.168.122.0/24    anywhere
ACCEPT    all  --  anywhere            anywhere
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
REJECT    all  --  anywhere            anywhere            reject-with icmp-port-unreachable
REJECT    all  --  anywhere            anywhere            reject-with icmp-host-prohibited

the following a simplified list for a cleaner look - notice the NEW state isnt required because the traffic with be matched with an accepted rule and finally at the end the 'any any -reject' will be a catch all for the other reject rules that were removed
Code:

Chain FORWARD (policy ACCEPT)
target    prot opt source              destination
ACCEPT    all  --  anywhere            192.168.100.0/24    state RELATED,ESTABLISHED
ACCEPT    all  --  192.168.100.0/24    anywhere
ACCEPT    all  --  anywhere            anywhere
ACCEPT    all  --  anywhere            192.168.122.0/24    state RELATED,ESTABLISHED
ACCEPT    all  --  192.168.122.0/24    anywhere
ACCEPT    all  --  anywhere            anywhere
REJECT    all  --  anywhere            anywhere            reject-with icmp-host-prohibited

you can use either scenario, they both allow connectivity - while the first is more restrictive i think its unnecessary imho...especially for private virtual machines

ttynjala 05-16-2012 06:29 AM

Solved
 
I believe I was experiencing this same issue and it seems my problem was solved simply by running the iptables -F on hypervisor host and the virtual machine hosts.
I did not re-create the networks again, but the problem disappeared. Great advice, thanks!


All times are GMT -5. The time now is 10:18 PM.