dhcpd complains "Failed to send 300 byte long packet over fallback interface."

spikeygg · 07-26-2015, 12:43 PM

I'm getting these messages in my /var/log/syslog occasionally. They happen immediately after the DHCPACK but sometimes the DHCPACK happens and the error messages don't appear.

Code:

Jul 26 08:16:57 routeitx dhcpd: DHCPINFORM from 192.168.0.12 via eth0
Jul 26 08:16:57 routeitx dhcpd: DHCPINFORM from 192.168.0.12 via eth0
Jul 26 08:16:57 routeitx dhcpd: DHCPACK to 192.168.0.12 (6c:f0:49:e1:78:21) via eth0
Jul 26 08:16:57 routeitx dhcpd: DHCPACK to 192.168.0.12 (6c:f0:49:e1:78:21) via eth0
Jul 26 08:16:57 routeitx dhcpd: send_packet: Operation not permitted
Jul 26 08:16:57 routeitx dhcpd: dhcp.c:1324: Failed to send 300 byte long packet over fallback interface.

another instance:

Code:

Jul 26 08:24:44 routeitx dhcpd: DHCPINFORM from 192.168.0.11 via eth0
Jul 26 08:24:44 routeitx dhcpd: DHCPINFORM from 192.168.0.11 via eth0
Jul 26 08:24:44 routeitx dhcpd: DHCPACK to 192.168.0.11 (1c:6f:65:d5:5e:c6) via eth0
Jul 26 08:24:44 routeitx dhcpd: DHCPACK to 192.168.0.11 (1c:6f:65:d5:5e:c6) via eth0
Jul 26 08:24:44 routeitx dhcpd: send_packet: Operation not permitted
Jul 26 08:24:44 routeitx dhcpd: dhcp.c:1324: Failed to send 300 byte long packet over fallback interface.

I've done some googling and most people suggest iptables is too strict but I have already opened both 67 and 68. These are the first rules on the INPUT and OUTPUT chains.

$IPTABLES -I INPUT -i $INTIF -p udp --sport 67:68 --dport 67:68 -j ACCEPT
$IPTABLES -I OUTPUT -o $INTIF -p udp --sport 67:68 --dport 67:68 -j ACCEPT

Code:

> sudo iptables -nvxL INPUT |head -n 3
Chain INPUT (policy DROP 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
     110    36431 ACCEPT     udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            udp spts:67:68 dpts:67:68
> sudo iptables -nvxL OUTPUT |head -n 3
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
     137    44936 ACCEPT     udp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            udp spts:67:68 dpts:67:68
>

I have also enabled "complain mode" in apparmor for the dhcpd process to try to get more detail on this error but nothing additional appears in the log.

Anyone know how to troubleshoot this?

Thanks,
-Greg

zhjim · 07-27-2015, 04:56 AM

Most tutorials about iptables include an allow stanca for the lo interface. Also this does not explain why it sometimes goes through lo and sometimes goes elseway.
To get rid of the error add

Code:

iptables -I INPUT 1 -i lo -d 127.0.0.1 -j ACCEPT
iptables -I OUTPUT 1 -o lo -s 127.0.0.1 -j ACCEPT

to your iptables.

To find out what is happeing install tcpdump and take a capture of the traffic. Or maybe dchp tries to update the dns record (if you have it installed on the same machine) and is not allowed though. Check your config of dhcp.

spikeygg · 07-27-2015, 11:26 PM

Quote:

Originally Posted by zhjim

Most tutorials about iptables include an allow stanca for the lo interface. Also this does not explain why it sometimes goes through lo and sometimes goes elseway.
To get rid of the error add

Code:

iptables -I INPUT 1 -i lo -d 127.0.0.1 -j ACCEPT
iptables -I OUTPUT 1 -o lo -s 127.0.0.1 -j ACCEPT

to your iptables.

To find out what is happeing install tcpdump and take a capture of the traffic. Or maybe dchp tries to update the dns record (if you have it installed on the same machine) and is not allowed though. Check your config of dhcp.

Thanks Zhjim but I've already got those in as the second item in my rules:

Code:

> sudo iptables -nvxL OUTPUT |head -n 4
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 ACCEPT     udp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            udp spts:67:68 dpts:67:68
       0        0 ACCEPT     all  --  *      lo      0.0.0.0/0            0.0.0.0/0           
> sudo iptables -nvxL INPUT |head -n 4
Chain INPUT (policy DROP 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       0        0 ACCEPT     udp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            udp spts:67:68 dpts:67:68
       0        0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
>

zhjim · 07-30-2015, 02:05 AM

Actually i misread the error message. Its fallback interface not like i read loopback interface. A search on this came up with the vague information that the fallback interface is the one with an actually ip address (server-side). Most dhcp traffic is "broadcast" as you well known.

Coming to think do you have any icmp blocks in your iptables? If i recall correctly there is an dhcpd option to ping the ip that would be used as a lease. Maybe check your dhcpd.conf for this.
To get a hold of this maybe fire up tcpdump and just log the packets.

spikeygg · 07-30-2015, 07:32 AM

Quote:

Originally Posted by zhjim

Actually i misread the error message. Its fallback interface not like i read loopback interface. A search on this came up with the vague information that the fallback interface is the one with an actually ip address (server-side). Most dhcp traffic is "broadcast" as you well known.

Coming to think do you have any icmp blocks in your iptables? If i recall correctly there is an dhcpd option to ping the ip that would be used as a lease. Maybe check your dhcpd.conf for this.
To get a hold of this maybe fire up tcpdump and just log the packets.

I do have icmp blocks in my iptables but they are only on the external interface. The internal one is pretty much wide open.

Are you talking about the "ping-check" option inside of dhcpd.conf? I have not used that before and I don't see it in my dhcpd.conf file. Do you know if it is enabled by default? I couldn't find the answer to this in the manual.

I'll have to give the tcpdump a try, I've never used that before, do you suggest any specific switches to help narrow it down? It appears to be a pretty powerful debug tool.

Thanks for the help zhjim.
-Greg

zhjim · 07-31-2015, 02:06 AM

Did you check /etc/default/dhcpd? file for the ping-check option? I don't think its enabled by default. But maybe set it to no explicitly?
Its just that I encountered "Operation not permitted" everytime that i block icmp stuff with iptables.

You can narrow down the interface tcpdump works on with the -i switch. I'd say at least include lo and your LAN interface. Depending on your traffic numbers you could record all of it to a file. And later narrow it even further giving the dhcp ports. Its a thin line between recording too much and not enough. Thus if you have some low traffic times I would try it then. Definitely use -s96 to only capture the first 96bytes of each packages. That should include all headers of ip+tcp|udp.

Something like this

Quote:

tcpdump -i eth0 -i lo -s96 -w your.log

But maybe iptables can save the day. Theres an process and owner option to it.

Quote:

iptables -I OUTPUT 1 -m owner --gid-owner $dhcp -j LOG --log-ip-options --log-prefix "DHCP: "

Might have to load a module for this. And put the right gid for the dhcpd server in.

If you can somehow reproduce the error message you might run dhcp in debug mode but yeah...

Carlos Medeiros · 11-13-2015, 08:00 AM

I solved using the following!

-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A INPUT -p udp -m udp --sport 137 --dport 137 -j ACCEPT
-A INPUT -p udp -m udp --sport 68 --dport 67 -j ACCEPT

pmackinney · 01-12-2016, 07:18 PM

Carlos, if you care to share the source of your answer, I'd be interested. I haven't been able to find a working solution yet.

My dhcp server is running iptables.
It allows all traffic on the loopback.
It allows all icmp, tcp, and udp on the local subnet

Code:

-A INPUT -p icmp -s 192.168.1.0/24 -j ACCEPT
-A INPUT -p udp -s 192.168.1.0/24 -j ACCEPT
-A INPUT -p tcp -s 192.168.1.0/24 -j ACCEPT

This should cover every thing you're doing, right?

enno.vet · 05-02-2017, 11:06 AM

Don't know if it's still of interest after such a long time. The point here however is to let the packet OUT not in. Does one need dhclient? If so, one should add a stanza like

Code:

iptables -A OUTPUT -p udp --sport 1024:65535 --dport 67 -j ACCEPT
iptables -A OUTPUT -p udp --sport 68 --dport 67 -j ACCEPT

Of course this has to be root¹). And if netfilter is compiled as modules add -m <proto>, too.
Rationale: dhclient (or similar agent) is configured to renew a dhcp lease, alas iptables doesn't let her connect to the outside's port 67 -- via which port? The documentation on bootp and dhcp unfortunately is not too clear about ports involved. Basically rfc-951 defines 67 as receiving server port and 68 as receiving client port for bootp, but does not define the originial source port. Hence the first sport stanza above. One could of course check with a tool like tcpdump what actually is going on, or search the source files of dhclient blah blah woof woof... if one had the time and energy.

So far dhclient can send its message out and don't clubber your logfiles anymore, but she won't get an answer (which doesn't necessarily matter AFAIK).

If you need the full DHCP thing you would of course have to open up the ports back inside to your box like so:

Code:

iptables -A INPUT -p udp --sport 67 --dport 68 -j ACCEPT
iptables -A INPUT -p udp --sport 1024:65535 --dport 68 -j ACCEPT

Again, the rfc does not define the answering server port.

And remember, if an error turns up in linux log files with "Operation not permitted" it is most often due to the packet filter.

Brgds, e.

¹) Possibly you could add -p tcp, as /etc/services claims the bootp ports for both protocols, even though due to the nature of the dhcp protocol tcp should be irrelevant.