IPSec Tunnel AWS VPC <-> openSwan CentOS 6.6 Tunnel up, no traffic

cojafoji · 06-25-2015, 01:25 PM

Hey all,

Managed to get a tunnel up between a CentOS 6.6 box and our AWS VPC. It shows as connected both on the box, and in AWS. The problem is routing the traffic from the server. Everything I've tried doesn't seem to have worked, and I'm feeling a little beyond my reach here. Any advice?

Code:

[USER@SERVER /]# sudo service ipsec status
IPsec running  - pluto pid: 4605
pluto pid 4605
1 tunnels up
some eroutes exist

[USER@SERVER /]# ip route

Code:

10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.43 
10.4.0.0/16 via 10.0.0.1 dev eth0  src 10.0.0.43 
169.254.0.0/16 dev eth0  scope link  metric 1002 
default via 10.0.0.1 dev eth0

[USER@SERVER /]# ip xfrm policy

Code:

src 10.0.0.0/24 dst 10.4.0.0/16 
dir out priority 2352 ptype main 
tmpl src 10.0.0.43 dst 71.XX.XXX.XXX
proto esp reqid 16385 mode tunnel
src 10.4.0.0/16 dst 10.0.0.0/24 
dir fwd priority 2352 ptype main 
tmpl src 71.XX.XX.XXX  dst 10.0.0.43
proto esp reqid 16385 mode tunnel
src 10.4.0.0/16 dst 10.0.0.0/24 
dir in priority 2352 ptype main 
tmpl src 71.XX.XX.XXX  dst 10.0.0.43
proto esp reqid 16385 mode tunnel

[USER@SERVER /]# ipsec verify

Code:

Checking your system to see if IPsec got installed and started correctly:
Version check and ipsec on-path                             [OK]
Linux Openswan U2.6.32/K2.6.32-504.el6.i686 (netkey)
Checking for IPsec support in kernel                        	[OK]
 SAref kernel support                                       [N/A]
 NETKEY:  Testing for disabled ICMP send_redirects          	[OK]
NETKEY detected, testing for disabled ICMP accept_redirects [OK]
Testing against enforced SElinux mode                       [OK]
Checking that pluto is running                              	[OK]
 Pluto listening for IKE on udp 500                         [OK]
 Pluto listening for NAT-T on udp 4500                      	[OK]
Checking for 'ip' command                                   [OK]
Checking /bin/sh is not /bin/dash                           [OK]
Checking for 'iptables' command                             [OK]
Opportunistic Encryption Support                            	[DISABLED]

ipsec.conf

Code:

        protostack=netkey
        nat_traversal=yes
        virtual_private=10.0.0.0/24
        oe=off

tunnel.conf

Code:

conn aws

    type=tunnel
    authby=secret
    left=defaultroute
    leftid=171.X.XX.XX           <----Office Ext IP of Server
    leftnexthop=defaultroute
    leftsubnet=10.0.0.0/24       <----Internal subnet
    right=71.XX.XX.XXX           <----AWS Ext VPC Gateway
    rightsubnet=10.4.0.0/16      <----AWS Internal Subnet
    phase2=esp
    phase2alg=aes128-sha1
    ike=aes128-sha1
    ikelifetime=28800s
    salifetime=3600s
    pfs=yes
    auto=start
    rekey=yes
    keyingtries=forever
    dpddelay=10
    dpdtimeout=60
    dpdaction=restart_by_peer

tunnel.secrets

Code:

171.X.XX.XX  71.XX.XX.XXX: PSK "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

IPTables rules added for IPSec Traffic

Code:

iptables -A INPUT -i eth0 -p 50 -j ACCEPT
iptables -A INPUT -i eth0 -p udp --dport 500 -j ACCEPT
iptables -A INPUT -i eth0 -p udp --dport 4500 -j ACCEPT
iptables -A INPUT -i eth0 -p 51 -j ACCEPT
iptables -A INPUT -i eth0 -p udp -m policy --dir in --pol ipsec -m udp --dport 1701 -j ACCEPT

[USER@SERVER /]# ipsec verify

Code:

000 stats db_ops: {curr_cnt, total_cnt, maxsz} :context={0,2,36} trans={0,2,1536} attrs={0,2,2048} 
000  
000 "aws": 10.0.0.0/24===10.0.0.43[171.X.XX.73,+S=C]---10.0.0.1...71.XX.XX.XXX<71.XX.XX.XXX>[+S=C]===10.4.0.0/16; erouted; eroute owner: #2
000 "aws":     myip=10.0.0.43; hisip=unset;
000 "aws":   ike_life: 28800s; ipsec_life: 3600s; rekey_margin: 540s; rekey_fuzz: 100%; keyingtries: 0; nat_keepalive: yes 
000 "aws":   policy: PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW+SAREFTRACK+lKOD+rKOD; prio: 24,16; interface: eth0; 
000 "aws":   dpd: action:restart_by_peer; delay:10; timeout:60;
000 "aws":   newest ISAKMP SA: #1; newest IPsec SA: #2; 
000 "aws":   IKE algorithms wanted: AES_CBC(7)_128-SHA1(2)_000-MODP1536(5), AES_CBC(7)_128-SHA1(2)_000-MODP1024(2)
000 "aws":   IKE algorithms found:  AES_CBC(7)_128-SHA1(2)_160-MODP1536(5), AES_CBC(7)_128-SHA1(2)_160-MODP1024(2)
000 "aws":   IKE algorithm newest: AES_CBC_128-SHA1-MODP1024
000 "aws":   ESP algorithms wanted: AES(12)_128-SHA1(2)_000
000 "aws":   ESP algorithms loaded: AES(12)_128-SHA1(2)_160
000 "aws":   ESP algorithm newest: AES_128-HMAC_SHA1; pfsgroup=<Phase1>
000  
000 #2: "aws":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 1978s; newest IPSEC; eroute owner; isakmp#1; idle; import:admin initiate
000 #2: "aws" esp.6e00a5b@71.XX.XXX.XXX esp.8a07a523@10.0.0.43 tun.0@71.XX.XX.XXX tun.0@10.0.0.43 ref=0 refhim=4294901761
000 #1: "aws":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 26936s; newest ISAKMP; lastdpd=8s(seq in:2113 out:0); idle; import:admin initiate

lazydog · 06-26-2015, 07:59 AM

What does the route command show you?

Code:

route -n

cojafoji · 06-26-2015, 03:32 PM

@lazydog

Code:

[USER@SERVER tmp]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.4.0.0        10.0.0.1        255.255.0.0     UG    0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0 eth0

lazydog · 06-29-2015, 09:09 AM

Have you done a tracerouter from both ends to see where traffic gets lost?

cojafoji · 07-07-2015, 10:41 AM

@lazydog

There is no return. It's not going out over the eth0 interface, but it's not traversing the tunnel either.

lazydog · 07-07-2015, 01:04 PM

Looking at your route table it looks like you are not routing 10.4.0.0 (assuming this is AWS) over your tunnel interface.

What is the out put from

Code:

ifconfig

cojafoji · 07-07-2015, 02:10 PM

I came to the same conclusion, I've been trying to find a way to route the traffic over the tunnel.

eth0 Link encap:Ethernet HWaddr 08:00:27:F6:F7:16
inet addr:10.0.0.43 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fef6:f716/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:96248 errors:0 dropped:0 overruns:0 frame:0
TX packets:34684 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:19453550 (18.5 MiB) TX bytes:10183662 (9.7 MiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:148 (148.0 b) TX bytes:148 (148.0 b)

Also, updated /etc/ipsec.conf to try and exclude the aws subnet.

protostack=netkey
nat_traversal=yes
virtual_private=v4:10.0.0.0/24,%v4:!10.4.0.0/16
oe=off

lazydog · 07-08-2015, 11:01 AM

You do not have a tun interface listed here. You need to setup the tun interface.

Code:

ifconfig tun0 inet <IP LOCAL> <IP REMOTE> up

Then make sure your routing table shows the correct routes.

cojafoji · 07-08-2015, 04:21 PM

Should it be the cidr block 10.0.0.0/24 for ip local, and the remote cidr block for the ip remote (10.4.0.0./16)?

Edit: I'd also really like to thank you for being kind enough, and patient enough to help me with this. It is very much appreciated.

Ser Olmy · 07-08-2015, 05:49 PM

IPsec doesn't (normally) use a tunnel interface. Traffic is routed according to the regular routing table, and is encrypted/decrypted as it exits/enters the external interface of the gateways.

What does ipsec status say?

cojafoji · 07-09-2015, 08:07 AM

Since the subnets are similar for both environments behind the routers, I setup (since the first post) the virtual_private to allow only the subnet I'm trying to route traffic to (10.4.0.0/16), and excluded the other.

Quote:

000 using kernel interface: netkey
000 interface lo/lo ::1
000 interface lo/lo 127.0.0.1
000 interface lo/lo 127.0.0.1
000 interface eth0/eth0 10.0.0.43
000 interface eth0/eth0 10.0.0.43
000 interface eth0/eth0 171.X.XX.XX
000 interface eth0/eth0 171.X.XX.XX
000 myid = (none)
000 debug none
000
000 virtual_private (%priv):
000 - allowed 1 subnet: 10.4.0.0/16
000 - disallowed 1 subnet: 10.0.0.0/24
000
000 algorithm ESP encrypt: id=3, name=ESP_3DES, ivlen=8, keysizemin=192, keysizemax=192
000 algorithm ESP encrypt: id=6, name=ESP_CAST, ivlen=8, keysizemin=128, keysizemax=128
000 algorithm ESP encrypt: id=7, name=ESP_BLOWFISH, ivlen=8, keysizemin=40, keysizemax=448
000 algorithm ESP encrypt: id=11, name=ESP_NULL, ivlen=0, keysizemin=0, keysizemax=0
000 algorithm ESP encrypt: id=12, name=ESP_AES, ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=13, name=ESP_AES_CTR, ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=14, name=ESP_AES_CCM_A, ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=15, name=ESP_AES_CCM_B, ivlen=12, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=16, name=ESP_AES_CCM_C, ivlen=16, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=18, name=ESP_AES_GCM_A, ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=19, name=ESP_AES_GCM_B, ivlen=12, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=20, name=ESP_AES_GCM_C, ivlen=16, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=22, name=(null), ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=252, name=ESP_SERPENT, ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP encrypt: id=253, name=ESP_TWOFISH, ivlen=8, keysizemin=128, keysizemax=256
000 algorithm ESP auth attr: id=1, name=AUTH_ALGORITHM_HMAC_MD5, keysizemin=128, keysizemax=128
000 algorithm ESP auth attr: id=2, name=AUTH_ALGORITHM_HMAC_SHA1, keysizemin=160, keysizemax=160
000 algorithm ESP auth attr: id=5, name=AUTH_ALGORITHM_HMAC_SHA2_256, keysizemin=256, keysizemax=256
000 algorithm ESP auth attr: id=6, name=AUTH_ALGORITHM_HMAC_SHA2_384, keysizemin=384, keysizemax=384
000 algorithm ESP auth attr: id=7, name=AUTH_ALGORITHM_HMAC_SHA2_512, keysizemin=512, keysizemax=512
000 algorithm ESP auth attr: id=8, name=(null), keysizemin=160, keysizemax=160
000 algorithm ESP auth attr: id=9, name=(null), keysizemin=128, keysizemax=128
000 algorithm ESP auth attr: id=251, name=(null), keysizemin=0, keysizemax=0
000
000 algorithm IKE encrypt: id=0, name=(null), blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=0, name=(null), blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=0, name=(null), blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=0, name=(null), blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=0, name=(null), blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=0, name=(null), blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=3, name=OAKLEY_BLOWFISH_CBC, blocksize=8, keydeflen=128
000 algorithm IKE encrypt: id=5, name=OAKLEY_3DES_CBC, blocksize=8, keydeflen=192
000 algorithm IKE encrypt: id=7, name=OAKLEY_AES_CBC, blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=65004, name=OAKLEY_SERPENT_CBC, blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=65005, name=OAKLEY_TWOFISH_CBC, blocksize=16, keydeflen=128
000 algorithm IKE encrypt: id=65289, name=OAKLEY_TWOFISH_CBC_SSH, blocksize=16, keydeflen=128
000 algorithm IKE hash: id=1, name=OAKLEY_MD5, hashsize=16
000 algorithm IKE hash: id=2, name=OAKLEY_SHA1, hashsize=20
000 algorithm IKE hash: id=4, name=OAKLEY_SHA2_256, hashsize=32
000 algorithm IKE hash: id=5, name=OAKLEY_SHA2_384, hashsize=48
000 algorithm IKE hash: id=6, name=OAKLEY_SHA2_512, hashsize=64
000 algorithm IKE dh group: id=2, name=OAKLEY_GROUP_MODP1024, bits=1024
000 algorithm IKE dh group: id=5, name=OAKLEY_GROUP_MODP1536, bits=1536
000 algorithm IKE dh group: id=14, name=OAKLEY_GROUP_MODP2048, bits=2048
000 algorithm IKE dh group: id=15, name=OAKLEY_GROUP_MODP3072, bits=3072
000 algorithm IKE dh group: id=16, name=OAKLEY_GROUP_MODP4096, bits=4096
000 algorithm IKE dh group: id=17, name=OAKLEY_GROUP_MODP6144, bits=6144
000 algorithm IKE dh group: id=18, name=OAKLEY_GROUP_MODP8192, bits=8192
000 algorithm IKE dh group: id=22, name=OAKLEY_GROUP_DH22, bits=1024
000 algorithm IKE dh group: id=23, name=OAKLEY_GROUP_DH23, bits=2048
000 algorithm IKE dh group: id=24, name=OAKLEY_GROUP_DH24, bits=2048
000
000 stats db_ops: {curr_cnt, total_cnt, maxsz} :context={0,2,36} trans={0,2,1536} attrs={0,2,2048}
000
000 "aws": 10.0.0.0/24===10.0.0.43[171.X.XX.XX,+S=C]---10.0.0.1...71.X.XX.XXX<71.X.XX.XXX>[+S=C]===10.4.0.0/16; erouted; eroute owner: #2
000 "aws": myip=10.0.0.43; hisip=unset;
000 "aws": ike_life: 28800s; ipsec_life: 3600s; rekey_margin: 540s; rekey_fuzz: 100%; keyingtries: 0; nat_keepalive: yes
000 "aws": policy: PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW+SAREFTRACK+lKOD+rKOD; prio: 24,16; interface: eth0;
000 "aws": dpd: action:restart_by_peer; delay:10; timeout:60;
000 "aws": newest ISAKMP SA: #3; newest IPsec SA: #2;
000 "aws": IKE algorithms wanted: AES_CBC(7)_128-SHA1(2)_000-MODP1536(5), AES_CBC(7)_128-SHA1(2)_000-MODP1024(2)
000 "aws": IKE algorithms found: AES_CBC(7)_128-SHA1(2)_160-MODP1536(5), AES_CBC(7)_128-SHA1(2)_160-MODP1024(2)
000 "aws": IKE algorithm newest: AES_CBC_128-SHA1-MODP1024
000 "aws": ESP algorithms wanted: AES(12)_128-SHA1(2)_000
000 "aws": ESP algorithms loaded: AES(12)_128-SHA1(2)_160
000 "aws": ESP algorithm newest: AES_128-HMAC_SHA1; pfsgroup=<Phase1>
000
000 #3: "aws":500 STATE_MAIN_R3 (sent MR3, ISAKMP SA established); EVENT_SA_REPLACE in 28520s; newest ISAKMP; lastdpd=4s(seq in:1962 out:1961); idle; import:not set
000 #2: "aws":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 2834s; newest IPSEC; eroute owner; isakmp#1; idle; import:admin initiate
000 #2: "aws" esp.c86a090e@71.X.XX.XXX esp.fc92ecaa@10.0.0.43 tun.0@71.X.XX.XXX tun.0@10.0.0.43 ref=0 refhim=4294901761
000 #1: "aws":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 27793s; lastdpd=-1s(seq in:0 out:0); idle; import:admin initiate
000