LinuxQuestions.org - Creating a CentoS 7 LXC container in an EC2 instance

- Linux - Virtualization and Cloud (https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/)

- - Creating a CentoS 7 LXC container in an EC2 instance (https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/creating-a-centos-7-lxc-container-in-an-ec2-instance-4175549073/)

Creating a CentoS 7 LXC container in an EC2 instance

We've run into a situation that we've been unable to resolve. We have an AWS EC2 instance running a CentOS 7.1 AMI and under this instance we've created several LXC containers, also running CentOS 7.1. We're using virt-install to install the containers.

The containers run fine, and we can ssh into the containers from their host (the EC2 instance) and vice versa. The big problem we're hitting is we cannot access our default gateway from the containers. In fact, the external network is completely blocked within a container.

If we duplicate the setup locally by creating a CentOS 7.1 VM (using Virtual Machine Manager for example) and then creating containers under this VM, we have no issue in accessing the gateway from the containers. There a bit of difference here of course in that an EC2 instance is based on Xen whereas we're using a KVM based VM to duplicate the setup locally, but we assume this shouldn't be an issue. In both cases the VM (instance) can access the gateway of the external LAN. In the EC2 case, the containers under the VM cannot access the same gateway whereas in the KVM case the containers *can* access the external gateway.

We believe that the problem is somehow related to our AWS environment, perhaps a simple configuration issue, but we've exhausted our attempts to resolve this problem. So the question is, should this work? If a container is created under a CentOS 7 EC2 instance, should we be able to access the external gateway from the container?

what does tcpdump (in the hosting instance and in each container) show? are all the right packets going through both ways? what is missing? can you show the config of the hosting instance and one container?

My apologies for not following up on this yet, I've just been incredibly busy. Here's some of what you asked for. The network configuration for the containers all look like this:

/etc/sysconfig/network-scripts:
DEVICE=eth0
NM_CONTROLLED=no
ONBOOT=yes
BOOTPROTO=none
IPADDR=10.0.1.x
NETMASK=255.255.255.0
GATEWAY=10.0.1.1

The container host (an AWS instance) uses a bridged interface:
/etc/sysconfig/network-scripts/ifcfg-br0:
DEVICE=br0
NAME=br0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Bridge
USERCTL=no
NM_CONTROLLED=no
DEFROUTE="yes"
PEERDNS="yes"
PEERROUTES="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_PEERDNS="yes"
IPV6_PEERROUTES="yes"
IPV6_FAILURE_FATAL="no"

/etc/sysconfig/network-scripts/ifcfg-eth0:
DEVICE="eth0"
NAME="eth0"
TYPE="Ethernet"
ONBOOT="yes"
BRIDGE="br0"

This is all pretty standard stuff. The routing table on the host looks like this:

Code:

# route

Kernel IP routing table

Destination    Gateway        Genmask        Flags Metric Ref    Use Iface

default        ip-10-0-1-1.us- 0.0.0.0        UG    0      0        0 br0

10.0.1.0        0.0.0.0        255.255.255.0  U    0      0        0 br0

192.168.122.0  0.0.0.0        255.255.255.0  U    0      0        0 virbr0

whereas on one of the containers we see this:

Code:

# route 

Kernel IP routing table

Destination    Gateway        Genmask        Flags Metric Ref    Use Iface

default        10.0.1.1        0.0.0.0        UG    0      0        0 eth0

10.0.1.0        0.0.0.0        255.255.255.0  U    0      0        0 eth0

link-local      0.0.0.0        255.255.0.0    U    1032  0        0 eth0

The gateway cannot be pinged from the containers, and that's the crux of the problem. I've been installing host/container configurations like this on local hardware without problems so the issue is somehow related to AWS, just not sure how.

I'll try to get some tcpdump data in a day or two.