[SOLVED] TCP three way handshake question for SW bridge

makarandp · 05-01-2019, 04:01 PM

Hello All,

I'm new to Linux questions and this is my first post. So pl forgive any mistakes on my part.

I'm trying to create a sw bridge using packet sockets. My set up is follows:

Machine 1
10.10.6.4/24
eth0
|
eth0 Sw bridge veth0 net1_veth0
10.10.6.1/24 10.10.6.3/24
name space: net1
Machine 2

I have coded the SW bridge using 2 threads. The SW bridge simply reads the entire L2 packet from eth0 and writes it to veth0. Similarly in the other direction.

With this setup, I can ping from 10.10.6.4 to 10.10.6.3. All the same a TCP connection fails to establish between 10.10.6.4 and 10.10.6.3.

The reason being that the three way handshake does not complete.

The server is running on 10.10.6.3.
The client is running on 10.10.6.4.

I can see the following:
10.10.6.3 receives SYM.
10.10.6.3 sends back SYN-ACK.
10.10.6.4 receives SYN-ACK.
But it does not send ACK back to 10.10.6.3.
10.10.6.4 retransmits SYN.
10.10.6.3 retransmits SYN-ACK.
The process repeats.

The tcpdump is given below.

16:50:31.086022 IP (tos 0x10, ttl 64, id 38054, offset 0, flags [DF], proto TCP (6), length 60)
10.10.6.4.38218 > 10.10.6.3.2222: Flags [S], cksum 0x2049 (incorrect -> 0x1759), seq 3623274701, win 29200, options [mss 1460,sackOK,TS val 55860331 ecr 0,nop,wscale 7], length 0
0x0000: b2b6 ea50 8c51 0800 27d8 7816 0800 4510 ...P.Q..'.x...E.
0x0010: 003c 94a6 4000 4006 85eb 0a0a 0604 0a0a .<..@.@.........
0x0020: 0603 954a 08ae d7f6 c8cd 0000 0000 a002 ...J............
0x0030: 7210 2049 0000 0204 05b4 0402 080a 0354 r..I...........T
0x0040: 5c6b 0000 0000 0103 0307 \k........
16:50:31.087097 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
10.10.6.3.2222 > 10.10.6.4.38218: Flags [S.], cksum 0x2049 (incorrect -> 0xead6), seq 2020233119, ack 3623274702, win 28960, options [mss 1460,sackOK,TS val 3991566188 ecr 55860331,nop,wscale 7], length 0
0x0000: 0800 27d8 7816 b2b6 ea50 8c51 0800 4500 ..'.x....P.Q..E.
0x0010: 003c 0000 4000 4006 1aa2 0a0a 0603 0a0a .<..@.@.........
0x0020: 0604 08ae 954a 786a 4f9f d7f6 c8ce a012 .....JxjO.......
0x0030: 7120 2049 0000 0204 05b4 0402 080a edea q..I............
0x0040: 776c 0354 5c6b 0103 0307 wl.T\k....
16:50:32.091903 IP (tos 0x10, ttl 64, id 38055, offset 0, flags [DF], proto TCP (6), length 60)
10.10.6.4.38218 > 10.10.6.3.2222: Flags [S], cksum 0x2049 (incorrect -> 0x165d), seq 3623274701, win 29200, options [mss 1460,sackOK,TS val 55860583 ecr 0,nop,wscale 7], length 0
0x0000: b2b6 ea50 8c51 0800 27d8 7816 0800 4510 ...P.Q..'.x...E.
0x0010: 003c 94a7 4000 4006 85ea 0a0a 0604 0a0a .<..@.@.........
0x0020: 0603 954a 08ae d7f6 c8cd 0000 0000 a002 ...J............
0x0030: 7210 2049 0000 0204 05b4 0402 080a 0354 r..I...........T
0x0040: 5d67 0000 0000 0103 0307 ]g........
16:50:32.091998 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
10.10.6.3.2222 > 10.10.6.4.38218: Flags [S.], cksum 0x2049 (incorrect -> 0xe6ea), seq 2020233119, ack 3623274702, win 28960, options [mss 1460,sackOK,TS val 3991567192 ecr 55860331,nop,wscale 7], length 0
0x0000: 0800 27d8 7816 b2b6 ea50 8c51 0800 4500 ..'.x....P.Q..E.
0x0010: 003c 0000 4000 4006 1aa2 0a0a 0603 0a0a .<..@.@.........
0x0020: 0604 08ae 954a 786a 4f9f d7f6 c8ce a012 .....JxjO.......
0x0030: 7120 2049 0000 0204 05b4 0402 080a edea q..I............
0x0040: 7b58 0354 5c6b 0103 0307 {X.T\k....
16:50:32.093204 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
10.10.6.3.2222 > 10.10.6.4.38218: Flags [S.], cksum 0x2049 (incorrect -> 0xe6e8), seq 2020233119, ack 3623274702, win 28960, options [mss 1460,sackOK,TS val 3991567194 ecr 55860331,nop,wscale 7], length 0
0x0000: 0800 27d8 7816 b2b6 ea50 8c51 0800 4500 ..'.x....P.Q..E.
0x0010: 003c 0000 4000 4006 1aa2 0a0a 0603 0a0a .<..@.@.........
0x0020: 0604 08ae 954a 786a 4f9f d7f6 c8ce a012 .....JxjO.......
0x0030: 7120 2049 0000 0204 05b4 0402 080a edea q..I............
0x0040: 7b5a 0354 5c6b 0103 0307 {Z.T\k....
16:50:34.096430 IP (tos 0x10, ttl 64, id 38056, offset 0, flags [DF], proto TCP (6), length 60)
10.10.6.4.38218 > 10.10.6.3.2222: Flags [S], cksum 0x2049 (incorrect -> 0x1468), seq 3623274701, win 29200, options [mss 1460,sackOK,TS val 55861084 ecr 0,nop,wscale 7], length 0
0x0000: b2b6 ea50 8c51 0800 27d8 7816 0800 4510 ...P.Q..'.x...E.
0x0010: 003c 94a8 4000 4006 85e9 0a0a 0604 0a0a .<..@.@.........
0x0020: 0603 954a 08ae d7f6 c8cd 0000 0000 a002 ...J............
0x0030: 7210 2049 0000 0204 05b4 0402 080a 0354 r..I...........T
0x0040: 5f5c 0000 0000 0103 0307 _\........

Any pointers on what the problem may be would be highly appreciated.

Thanks and Rgds,
Makarand.

smallpond · 05-02-2019, 09:40 AM

The SYN-ACK packet looks ok, but 10.10.6.4 resends SYN with the same sequence number, so it either did not receive the SYN-ACK packet or did not accept it. Run tcpdump/wireshark on the 10.10.6.4 system and see if there is something different in what's reported.

makarandp · 05-02-2019, 12:24 PM

Hello Smallpond,

Thanks for your note.

I have managed to root cause the issue. The problem was with checksum calculation virtual ethernet interface. I had to turn off checksum offloading on the virtual ethernet interface to get the checksum correct.

ip netns exec net1 ethtool --offload net1_veth0 tx off

resolved the issue.

Marking the thread as closed.

With Rgds,
Makarand.