Minutes after Bridge set up, can't connect to host anymore

somanyquestions · 08-11-2021, 07:38 AM

I have 2 little single board computers running Linux.
Let's call them the "gadget", and the "host". One is configured as a USB ethernet gadget, connected to the host's USB-host port.
That USB connection works with network interfaces called "usb0" on each computer.
The host also has a eth0 interface, connected to an external switch, to a network with a bunch of other devices.

The subnet used by devices on the switch is 10.0.*.*.

Host:
eth0 is in that net: 10.0.20.7/16 (it has an additional IP in another subnet (10.111.*.*), not related to this task)
usb0 has one IP: 69.254.200.188

Gadget:
usb0 has 2 addresses:
- 169.254.200.199/16 - connection between gadget and host
- 10.0.22.7/16 - to be on the switch's subnet, too

That's the setup currently after boot, through /etc/network/interfaces,
extept the host's usb0 IP is configured with a script that waits for the usb0 showing up, and then assigns the IP/netmask with ifconfig.

As that is, I can, from my PC (10.0.12.1) - also connected to the switch, connect to the host e.g. with SSH / SCP.

Then I start a script that sets up a bridge:

Code:

ip link add name br0 type bridge
ip link set dev br0 up
ip link set dev eth0 master br0
ip link set dev usb0 master br0

After that, I can log, from the PC, into host/eth0 by the 10.0.21.7 - just as before.
I now also can, from the PC, log into the gadget by 10.0.22.7 - so the bridging works.
Well, kinda.

Because this works for a while. Timing not exactly reproducible. Sometimes tens of seconds, sometimes 10 minutes, with me randomly opening and closing additional connections with PuTTY and WinSCP.
Then, at some point, I can still connect to the gadget, bridged by the host - but not to the host itself anymore.
I also can't, from the host, connect to the gadget, or the other way around, in the 169.254.x.x net. (it does initially work after the bridge was set up, and also fails when the other stuff fails, whether it takes 10s of seconds or minutes)

I don't exactly understand the implications of this output yet, but FWIW, here is the "ip neigh show" output (it could be less detailed than usual - it's the BusyBox implementation. I'll mention that just in case, ...)

Before running the script that sets up the bridge:

Code:

10.0.12.1 dev eth0 lladdr 68:05:ca:47:9f:32 ref 1 used 14/14/9 probes 0 REACHABLE

After bridge setup and a few connections made (host):

Code:

10.0.22.7 dev eth0  used 266/1275/264 probes 6 FAILED
10.111.0.1 dev eth0  used 28/1187/25 probes 6 FAILED
169.254.200.199 dev usb0  used 247/1200/243 probes 6 FAILED
10.0.12.1 dev br0 lladdr 68:05:ca:47:9f:32 used 1120/1180/1120 probes 0 STALE
10.111.0.1 dev br0 lladdr 00:11:32:ab:a4:e9 ref 1 used 1119/26/43 probes 0 REACHABLE
10.0.12.1 dev eth0 lladdr 68:05:ca:47:9f:32 used 232/168/134 probes 1 STALE

Or just now with some fewer connections made until it keeled over (host side):

Code:

10.0.12.1 dev eth0  ref 1 used 0/149/0 probes 4 INCOMPLETE
10.0.12.1 dev br0 lladdr 68:05:ca:47:9f:32 ref 1 used 212/0/149 probes 0 REACHABLE
169.254.200.199 dev usb0  used 153/242/152 probes 6 FAILED

Corresponding gadget side "ip neigh":

Code:

169.254.200.188 dev usb0 lladdr 16:22:33:44:55:66 STALE
10.0.12.1 dev usb0 lladdr 68:05:ca:47:9f:32 STALE

(I have serial connections to both board computers, too, in case you're wondering)

I have now disconnected all else from the switch but the PC and the host, to make sure nothing relating to other network participants is going on.

Does anyone have an idea what the source of problems may be here?
Especially, why it takes from seconds to minutes for it to fail.

Ser Olmy · 08-14-2021, 11:25 AM

You cannot assign an IP address to an interface that's part of a bridge. You need to use the bridge interface instead.

somanyquestions · 08-17-2021, 07:29 AM

Quote:

Originally Posted by Ser Olmy

You cannot assign an IP address to an interface that's part of a bridge. You need to use the bridge interface instead.

Thanks, I got it working after fixing some other problems that distracted from this.
But this apparently gives a bit of a problem, if I see it right:

Say I want to assign the "gadget" computer an IP which is known already on the host, e.g. residing in some config file there, so it can be changed with accessing the host only - no fumbling on the gadget necessary.
It seems that running the available DHCP server "dhcpd" on the host would be a tempting way to do that: just set it up to only work on the interface that the gadget is connected on, and limit the dished out IP range to the wanted IP.

But that gadget's connection is on the usb0 interface - which is also supposed to "go outside" by the bridge.
So if usb0, on the host, has no own IP - dhcpd probably won't work on it to assign connected things an IP?
I wouldn't want to set the DHCP to work on the {usb0, eth0} bridge, as that would then also potentially mess up externally (eth0) connected devices / fight against an external DHCP.
Remember, the "internal DHCP" idea here was just to have a way to configure the gadget's IP from the host - to the outside world it should look like the gadget had a static IP.

Is there a way to do that?