LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 07-04-2014, 12:55 PM   #1
CoherentLogic
LQ Newbie
 
Registered: Nov 2010
Distribution: Slackware
Posts: 2

Rep: Reputation: 0
iptables, NAT, and kvm virtual machines


Hello all,

I find myself in a position of having to set up something that is a bit over my head from a network perspective.

We (my coworkers and I) have a server colocated in our ISP's racks. The machine is hosting a number of kvm guests. These kvm guests are being provided for free as a service to the software development community in which we are involved, but we are running out of our allotment of static IP addresses, and are unable to pay for more.

We're only providing ssh and https access to these kvm guests, and the users of these guests don't have a problem with specifying a port. What we'd like to do is give the guests private IP addresses (10.0.0.0/8), use the existing static IPs on the host machines for access to all of them, and use NAT with iptables on the host, so that, from the outside, something like this:


ssh [public ip] -p 51000


will go to the normal ssh port on the guest having IP address 10.0.0.22. We'd do something similar for https access, and just keep using higher and higher port numbers for additional kvm guests.

My difficulty is that first, I don't know whether I need bridged, NAT, or host-only networking on the KVM side. If bridged, I don't know what interface I need to bridge the guest NICs to to make iptables NAT work this way, or what interfaces to specify in the iptables NAT rules.

Been a Linux software developer for years, and did Solaris administration for years too, but this kvm networking stuff makes me feel like a dumb newbie all over again.

Any pointers? If there's documentation online, my Google-fu is failing me.

Thanks in advance,

jpw
 
Old 07-04-2014, 02:46 PM   #2
pingu
Senior Member
 
Registered: Jul 2004
Location: Skuttunge SWEDEN
Distribution: Debian preferably
Posts: 1,350

Rep: Reputation: 127Reputation: 127
The way I read your post, you already have a bunch of kvm guests up and running, and accessible over ssh and https. So you should know how to set up networking - or maybe someone else did that, but then you just need to check the config and use the same?

I haven't used kvm, only Xen, OpenVZ and VirtualBox, but I guess in kvm you use bridged networking.
I don't know anything about the host, does it have some kind of GUI - web or control panel of any kind - or is it only cli? What system is it running?
What you do is create a new nic for the guest and bridge it to hosts bridge if.
Then in iptables (running on host I assume? - dangerous, no real firewall then!) you forward from a public ip to the guests ip address.
If you need more info, please post the ip configuration of host ("ifconfig" or "ip addr") and answer the questions above.
 
Old 07-04-2014, 04:00 PM   #3
CoherentLogic
LQ Newbie
 
Registered: Nov 2010
Distribution: Slackware
Posts: 2

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by pingu View Post
The way I read your post, you already have a bunch of kvm guests up and running, and accessible over ssh and https. So you should know how to set up networking - or maybe someone else did that, but then you just need to check the config and use the same?
The host has its own public and static IP address, from a 216.xxx.xxx.xxx/27 block. Currently, the guests have public and static addresses from the same /27 block, all bridged to eth0. This configuration works, but we want to move the guests to a 10.0.0.0/8 network, and access their ssh and https ports (22 and 443) on the host's public address from higher ports (51xxx) via NAT. This is so that we can get back some of our /27 block for use by other customers who pay for us to host their virtual machines.

Quote:
Originally Posted by pingu View Post
I haven't used kvm, only Xen, OpenVZ and VirtualBox, but I guess in kvm you use bridged networking.
I don't know anything about the host, does it have some kind of GUI - web or control panel of any kind - or is it only cli? What system is it running?
What you do is create a new nic for the guest and bridge it to hosts bridge if.
Then in iptables (running on host I assume? - dangerous, no real firewall then!) you forward from a public ip to the guests ip address.
If you need more info, please post the ip configuration of host ("ifconfig" or "ip addr") and answer the questions above.
The host is running CentOS 6.4, we own the machine, and have root access. We don't have access to the networking hardware: the ISP just gives us an Ethernet cable, and that's it. This is why I'm looking for an iptables-on-the-host solution, which I realize is suboptimal. But, again, these are testbed guest machines provided to our community free of charge.

Here's the current ifconfig:

Code:
[jpw@fwbcs01 ~]$ ifconfig
br0       Link encap:Ethernet  HWaddr 6C:AE:8B:63:95:3A  
          inet addr:216.223.229.250  Bcast:216.223.229.255  Mask:255.255.255.224
          inet6 addr: fe80::6eae:8bff:fe63:953a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:189389590 errors:0 dropped:0 overruns:0 frame:0
          TX packets:175915886 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:528701426524 (492.3 GiB)  TX bytes:54684233022 (50.9 GiB)

br0:1     Link encap:Ethernet  HWaddr 6C:AE:8B:63:95:3A  
          inet addr:10.0.0.1  Bcast:10.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0      Link encap:Ethernet  HWaddr 6C:AE:8B:63:95:3A  
          inet6 addr: fe80::6eae:8bff:fe63:953a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:644367422 errors:0 dropped:0 overruns:0 frame:0
          TX packets:467453849 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:623180167736 (580.3 GiB)  TX bytes:238617117699 (222.2 GiB)
          Memory:a9a60000-a9a80000 

eth1      Link encap:Ethernet  HWaddr 6C:AE:8B:63:95:3B  
          inet addr:10.0.0.2  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::6eae:8bff:fe63:953b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:1020 (1020.0 b)
          Memory:a9a40000-a9a60000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:9322313 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9322313 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:34144833426 (31.7 GiB)  TX bytes:34144833426 (31.7 GiB)

usb0      Link encap:Ethernet  HWaddr 6E:AE:8B:63:95:39  
          inet addr:169.254.95.120  Bcast:169.254.95.255  Mask:255.255.255.0
          inet6 addr: fe80::6cae:8bff:fe63:9539/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:296233 errors:0 dropped:0 overruns:0 frame:0
          TX packets:297348 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:24197445 (23.0 MiB)  TX bytes:28383772 (27.0 MiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:0D:BD:AE  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vnet0     Link encap:Ethernet  HWaddr FE:54:00:87:F6:FB  
          inet6 addr: fe80::fc54:ff:fe87:f6fb/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4762091 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6112506 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:804771500 (767.4 MiB)  TX bytes:849642535 (810.2 MiB)

vnet1     Link encap:Ethernet  HWaddr FE:54:00:E3:CC:81  
          inet6 addr: fe80::fc54:ff:fee3:cc81/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5163792 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6449688 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:869533014 (829.2 MiB)  TX bytes:892368323 (851.0 MiB)

vnet2     Link encap:Ethernet  HWaddr FE:54:00:C4:A9:72  
          inet6 addr: fe80::fc54:ff:fec4:a972/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23286157 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31332581 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:4064818232 (3.7 GiB)  TX bytes:30281512742 (28.2 GiB)

vnet3     Link encap:Ethernet  HWaddr FE:54:00:13:B6:30  
          inet6 addr: fe80::fc54:ff:fe13:b630/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16287267 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17525475 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:10767702420 (10.0 GiB)  TX bytes:6626615448 (6.1 GiB)

vnet4     Link encap:Ethernet  HWaddr FE:54:00:9E:EC:E7  
          inet6 addr: fe80::fc54:ff:fe9e:ece7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6018395 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7395545 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:926689842 (883.7 MiB)  TX bytes:1814869027 (1.6 GiB)

vnet5     Link encap:Ethernet  HWaddr FE:54:00:2E:01:E8  
          inet6 addr: fe80::fc54:ff:fe2e:1e8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:969535 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1087637 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:212623076 (202.7 MiB)  TX bytes:857457545 (817.7 MiB)

vnet6     Link encap:Ethernet  HWaddr FE:50:56:B2:2E:59  
          inet6 addr: fe80::fc50:56ff:feb2:2e59/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15040750 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13988035 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:1631692476 (1.5 GiB)  TX bytes:2067984559 (1.9 GiB)

vnet7     Link encap:Ethernet  HWaddr FE:50:56:B2:2E:61  
          inet6 addr: fe80::fc50:56ff:feb2:2e61/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4992243 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6296006 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:840180556 (801.2 MiB)  TX bytes:884613723 (843.6 MiB)

vnet8     Link encap:Ethernet  HWaddr FE:50:56:B2:2E:60  
          inet6 addr: fe80::fc50:56ff:feb2:2e60/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4991117 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6305748 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:845643542 (806.4 MiB)  TX bytes:874211273 (833.7 MiB)

vnet9     Link encap:Ethernet  HWaddr FE:50:56:B2:2E:62  
          inet6 addr: fe80::fc50:56ff:feb2:2e62/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4959720 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6316719 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:827522143 (789.1 MiB)  TX bytes:870678790 (830.3 MiB)

vnet10    Link encap:Ethernet  HWaddr FE:50:56:B2:2E:5B  
          inet6 addr: fe80::fc50:56ff:feb2:2e5b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9265878 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13386377 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:1226553669 (1.1 GiB)  TX bytes:9913747796 (9.2 GiB)

vnet11    Link encap:Ethernet  HWaddr FE:50:56:B2:2E:5F  
          inet6 addr: fe80::fc50:56ff:feb2:2e5f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:18013971 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22901488 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:43197361376 (40.2 GiB)  TX bytes:4546391759 (4.2 GiB)

vnet12    Link encap:Ethernet  HWaddr FE:50:56:B2:2E:58  
          inet6 addr: fe80::fc50:56ff:feb2:2e58/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:44075131 errors:0 dropped:0 overruns:0 frame:0
          TX packets:71188391 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:103436476414 (96.3 GiB)  TX bytes:7326729777 (6.8 GiB)

vnet13    Link encap:Ethernet  HWaddr FE:50:56:B2:2E:5C  
          inet6 addr: fe80::fc50:56ff:feb2:2e5c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11458821 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16654356 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:1312679169 (1.2 GiB)  TX bytes:9894047046 (9.2 GiB)

vnet14    Link encap:Ethernet  HWaddr FE:54:00:10:94:F4  
          inet6 addr: fe80::fc54:ff:fe10:94f4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6989363 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8508468 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:3771533578 (3.5 GiB)  TX bytes:7169276744 (6.6 GiB)

vnet15    Link encap:Ethernet  HWaddr FE:54:00:8C:FC:0E  
          inet6 addr: fe80::fc54:ff:fe8c:fc0e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4799584 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5376816 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:6851657824 (6.3 GiB)  TX bytes:721462162 (688.0 MiB)

vnet16    Link encap:Ethernet  HWaddr FE:54:00:CD:FC:0B  
          inet6 addr: fe80::fc54:ff:fecd:fc0b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10580535 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14209631 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:11551162821 (10.7 GiB)  TX bytes:8237142019 (7.6 GiB)
 
Old 07-05-2014, 04:32 AM   #4
pingu
Senior Member
 
Registered: Jul 2004
Location: Skuttunge SWEDEN
Distribution: Debian preferably
Posts: 1,350

Rep: Reputation: 127Reputation: 127
I can see that you have created a bridge "br0:1" and assigned an ip. But eth1 also has an ip in same subnet, this is the interface you use for the bridge so it shall not have an ip.
Below is the configuration I believe should work for you. I can't test anything right now, so please be careful! There is a small risk I missed something.

The config files should look something like below: (yes I have a kvm host, had forgotten I set one up just before vacation. )

The physical interface to use for internal networks
Code:
DEVICE=em2 (=eth1 on your server)
ONBOOT=yes
BRIDGE=EXT1 (call it whatever you want)
HWADDR=d0:67:e5:f3:5b:70
NM_CONTROLLED=no
STP=no
Creating a virtual device, I want two separat internal networks (optionally)
Code:
DEVICE=em2.2
ONBOOT=yes
VLAN=yes
BRIDGE=LAN1
NM_CONTROLLED=no
STP=no
HOTPLUG=no
The bridge named "EXT1"
Code:
DEVICE=EXT1
ONBOOT=yes
TYPE=Bridge
DELAY=0
IPADDR=192.168.12.2
NETMASK=255.255.255.0
GATEWAY=192.168.12.1
BOOTPROTO=none
DEFROUTE=no
NM_CONTROLLED=no
STP=no
HOTPLUG=no
Bridge LAN1 is created the same way, should you want a second internal network.

Then you change your guests so they use bridge EXT1 instead, can't give you cli commands for this right now unfortunately.

You also need to enable ip forwarding on the host, and then configure NAT / masquerading for all the ports.
For ip forwarding:
Code:
sysctl -w net.ipv4.ip_forward=1
or 
echo 1 > /proc/sys/net/ipv4/ip_forward
To make it survive a reboot:
Edit /etc/sysctl.conf, add a line "net.ipv4.ip_forward = 1"

You might want to try this on a test server first, a small mistake in network configuration could result in loss of eth0 leaving your server unreachable.
 
Old 07-05-2014, 09:14 AM   #5
pingu
Senior Member
 
Registered: Jul 2004
Location: Skuttunge SWEDEN
Distribution: Debian preferably
Posts: 1,350

Rep: Reputation: 127Reputation: 127
There is also a package "bridge-utils" that eases the administration of bridges.
Main command to use is brctl
If it doesn't exist, "yum install bridge-utils" should install it.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Lost Then Found KVM Virtual Machines Scrapper64 Linux - Virtualization and Cloud 7 08-08-2013 06:12 AM
KVM API for starting virtual machines guardian1 Linux - Virtualization and Cloud 2 09-04-2012 11:33 AM
LXer: Embiggen Your KVM Virtual Machines LXer Syndicated Linux News 0 12-05-2011 08:50 PM
LXer: Embiggen Your KVM Virtual Machines LXer Syndicated Linux News 1 12-05-2011 04:12 PM
Virtual machines running their own KVM's bill.loney Linux - Newbie 2 05-01-2009 03:08 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 05:45 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration