LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 07-29-2016, 07:46 PM   #1
GaWdLy
Member
 
Registered: Feb 2013
Location: San Jose, CA
Distribution: RHEL/CentOS/Fedora
Posts: 457

Rep: Reputation: Disabled
KVM server with bridge-need iptables assistance


So, I upgraded from my laptop KVM environment to a standalone 8 core Avoton server for my lab system. I had everything down perfectly on the laptop, so figured it would be an easy transition. It hasn't been.

I am setting up a lab environment for Openstack/Satellite/Ansible experiments, and need bridging to be working.

I have 1 primary NIC for the server, and I have configured a second NIC as a bridged interface:

~~~
# ip addr show

2: enp0s20f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 0c:c4:7a:6c:fd:fc brd ff:ff:ff:ff:ff:ff
inet 192.168.1.15/24 brd 192.168.1.255 scope global enp0s20f0
valid_lft forever preferred_lft forever
inet6 2602:306:bcb6:8a80:8b0a:ae12:d565:c071/64 scope global noprefixroute dynamic
valid_lft 2591544sec preferred_lft 604344sec
inet6 fe80::eb6d:6e14:8fe2:95ee/64 scope link
valid_lft forever preferred_lft forever
3: enp0s20f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
link/ether 0c:c4:7a:6c:fd:fd brd ff:ff:ff:ff:ff:ff
<snip>
19: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 0c:c4:7a:6c:fd:fd brd ff:ff:ff:ff:ff:ff
inet 192.168.1.73/24 brd 192.168.1.255 scope global dynamic br0
valid_lft 74524sec preferred_lft 74524sec
inet6 2602:306:bcb6:8a80:a92e:b3ae:9dfe:307c/64 scope global noprefixroute dynamic
valid_lft 2591544sec preferred_lft 604344sec
inet6 fe80::626a:4e3c:1c01:ed5b/64 scope link
valid_lft forever preferred_lft forever
~~~

enp0s20f1 is bridged to br0.

My satellite is configured to use br0. I can ping the satellite OS from my laptop, and vice-versa. I can log into the satellite server remotely from my laptop.

I cannot ping out past my gateway, 192.168.1.254, and I cannot ping the KVM host from the guest, either.

I think the routing on the KVM host is correct:

~~~
[root@zika disks]# ip route show
default via 192.168.1.254 dev enp0s20f0
default via 192.168.1.254 dev br0 proto static metric 425
172.17.13.0/24 dev virbr1 proto kernel scope link src 172.17.13.1 linkdown
192.168.1.0/24 dev enp0s20f0 proto kernel scope link src 192.168.1.15 metric 100
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.73 metric 425
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
~~~

The IP is set correctly on the Satellite:

~~~
[root@satellite ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:d6:c8:0f brd ff:ff:ff:ff:ff:ff
inet 192.168.1.73/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 2602:306:bcb6:8a80:5054:ff:fed6:c80f/64 scope global dynamic
valid_lft 2591974sec preferred_lft 604774sec
inet6 fe80::5054:ff:fed6:c80f/64 scope link
valid_lft forever preferred_lft forever
~~~

and I think the route is correct, as well:

~~~
[root@satellite ~]# ip route show
default via 192.168.1.254 dev eth0 proto static metric 100
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.73 metric 100
~~~

So I assume that there are some iptables rules that will be necessary to get this working correctly? I pose that as a question, because this is where I seem to have run past my knowledge.

Could someone please point me in the right direction?
 
Old 07-29-2016, 11:20 PM   #2
Rinndalir
Member
 
Registered: Sep 2015
Posts: 733

Rep: Reputation: Disabled
It may be sysctl that you need to use.
 
Old 07-29-2016, 11:26 PM   #3
GaWdLy
Member
 
Registered: Feb 2013
Location: San Jose, CA
Distribution: RHEL/CentOS/Fedora
Posts: 457

Original Poster
Rep: Reputation: Disabled
Hi there, and thanks for replying.

If you're thinking of the ip4.forward it's set to 1 on both host and guest.
 
Old 07-30-2016, 12:17 PM   #4
GaWdLy
Member
 
Registered: Feb 2013
Location: San Jose, CA
Distribution: RHEL/CentOS/Fedora
Posts: 457

Original Poster
Rep: Reputation: Disabled
I think I have it figured out, but I'm not sure how to resolve it, still:

Pinging a target with no answer:
~~~
[root@satellite ~]# ping redhat.com
PING redhat.com (209.132.183.105) 56(84) bytes of data.
^C
--- redhat.com ping statistics ---
153 packets transmitted, 0 received, 100% packet loss, time 151999ms
~~~

tcpdump on the KVM host br0 NIC shows ICMP requests, but no replies:
~~~
[root@zika disks]# tcpdump -nni enp0s20f1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s20f1, link-type EN10MB (Ethernet), capture size 262144 bytes
10:14:06.786099 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 123, length 64
10:14:07.786120 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 124, length 64
10:14:08.786086 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 125, length 64
10:14:09.786085 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 126, length 64
10:14:10.786077 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 127, length 64
10:14:11.786098 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 128, length 64
10:14:12.786113 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 129, length 64
10:14:13.786061 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 130, length 64
10:14:14.786101 IP 192.168.1.73 > 209.132.183.105: ICMP echo request, id 19767, seq 131, length 64
~~~

If I change the NIC to the primary server IF, I see the replies coming to that interface, instead:
~~~
[root@zika disks]# tcpdump -nni enp0s20f0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s20f0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:14:20.832846 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 137, length 64
10:14:21.832685 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 138, length 64
10:14:22.836946 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 139, length 64
10:14:23.832633 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 140, length 64
10:14:24.832732 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 141, length 64
10:14:25.832629 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 142, length 64
10:14:26.834027 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 143, length 64
10:14:27.832894 IP 209.132.183.105 > 192.168.1.73: ICMP echo reply, id 19767, seq 144, length 64
~~~

Routing and iptables are kind of weak for me. Does anyone know why I am seeing the replies on the wrong wire?
 
Old 07-30-2016, 01:35 PM   #5
Rinndalir
Member
 
Registered: Sep 2015
Posts: 733

Rep: Reputation: Disabled
If I was in your situation I would get out a piece of paper and pencil and draw my environment out. Another one I do is write a bug report. Not one that I will file but just go through all the steps as if I was writing a real bug report. Often either one of those will make the problem come out and reveal itself.
 
1 members found this post helpful.
Old 07-30-2016, 02:07 PM   #6
GaWdLy
Member
 
Registered: Feb 2013
Location: San Jose, CA
Distribution: RHEL/CentOS/Fedora
Posts: 457

Original Poster
Rep: Reputation: Disabled
I know I'm onto the issue, but still having a hard time working through it.

It seems like policy-based routing might be necessary here?

Here's what I've been trying, with no luck:

~~~
[root@zika disks]# ip rule add iif enp0s20f0 table 1
[root@zika disks]# ip rule add iif br0 table 2
[root@zika disks]# ip rule add from 192.168.1.15 table 1
[root@zika disks]# ip rule add from 192.168.1.73 table 2
[root@zika disks]# ip route show
default via 192.168.1.254 dev enp0s20f0 proto static metric 100
default via 192.168.1.254 dev br0 proto static metric 425
172.17.13.0/24 dev virbr1 proto kernel scope link src 172.17.13.1 linkdown
192.168.1.0/24 dev enp0s20f0 proto kernel scope link src 192.168.1.15 metric 100
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.73 metric 425
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
[root@zika disks]# ip route show table 1
default via 192.168.1.254 dev enp0s20f0
192.168.1.0/24 dev enp0s20f0 scope link
[root@zika disks]# ip route show table 2
default via 192.168.1.254 dev br0
192.168.1.0/24 dev br0 scope link
~~~

The only time things were working was when I accidentally broke the main routing rules by using 'ip rule flush'. As soon as I did that, the primary IF was no longer communicating, but the secondary was.

As soon as I restored the MAIN routing rule: 'ip rule add from all lookup main pref 32768'

The secondary IF was again no longer receiving traffic.
 
Old 07-30-2016, 02:20 PM   #7
GaWdLy
Member
 
Registered: Feb 2013
Location: San Jose, CA
Distribution: RHEL/CentOS/Fedora
Posts: 457

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Rinndalir View Post
If I was in your situation I would get out a piece of paper and pencil and draw my environment out. Another one I do is write a bug report. Not one that I will file but just go through all the steps as if I was writing a real bug report. Often either one of those will make the problem come out and reveal itself.
Mercifully, I don't think my environment is that complicated at this point. It really is as simple as 2 interfaces on the same network, using the same gateway/default. I just don't know how to make that work, I don't think.

I tried a Red Hat doc: How can I route network traffic such that the packets go out via the same interface they came in? https://access.redhat.com/solutions/19596

But that didn't work.
 
Old 07-30-2016, 04:37 PM   #8
GaWdLy
Member
 
Registered: Feb 2013
Location: San Jose, CA
Distribution: RHEL/CentOS/Fedora
Posts: 457

Original Poster
Rep: Reputation: Disabled
Code:
   | Lab Server     Satellite VM|
   |                       |    |
   |enp0s20f0          enp0s20f1|
   |(primary)           (br0)   |
   |____|__________________|____|
        |                  |
        |                  |
        |_____[SWITCH]_____|
                 |
                 |
              [GATEWAY]
                 |
                 |
              [Internet]
enp0s20f0 is 192.168.1.15
br0 is 192.168.1.73

Last edited by GaWdLy; 07-30-2016 at 05:32 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] KVM - Bridge, guest works until I start my iptables greenpoise Linux - Virtualization and Cloud 3 01-27-2014 08:50 AM
KVM bridge and iptables zefir_psv Linux - Security 3 04-17-2012 06:54 AM
iptables rate limiting for bridged connection (kvm created bridge) tkmsr Linux - Networking 1 10-28-2010 07:50 AM
Kvm using wireless bridge, how michaelux Linux - Virtualization and Cloud 1 06-07-2010 08:48 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 02:30 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration