Go Job Hunting at the LQ Job Marketplace
Go Back > Forums > Linux Forums > Linux - Networking
User Name
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.


Search this Thread
Old 04-01-2014, 08:58 PM   #1
LQ Newbie
Registered: Mar 2014
Posts: 7

Rep: Reputation: Disabled
Question Why is UFW/iptables blocking packets it shouldn't be?

Definitely a head-scratcher, here.

I'd been seeing some [UFW BLOCK] messages in the syslog among some torrent traffic like these (with representing my local IP address, and 18321 representing my torrent client's listening port, which *is* open in UFW/iptables):

kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=112 ID=8601 DF PROTO=TCP SPT=51124 DPT=18321 WINDOW=12500 RES=0x00 ACK FIN URGP=0 
kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=108 ID=62097 PROTO=TCP SPT=4683 DPT=18321 WINDOW=0 RES=0x00 RST URGP=0 
kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=241 ID=52202 DF PROTO=TCP SPT=65384 DPT=18321 WINDOW=0 RES=0x00 ACK RST URGP=0
And there's also blocks happening on high random ports that should be allowed through as normal passive connection traffic (or whatever the proper terminology is... sorry), though those ports are "blocked" against "incoming connections" by UFW:

kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=48 ID=0 DF PROTO=TCP SPT=16881 DPT=55480 WINDOW=0 RES=0x00 RST URGP=0 
kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=40 ID=0 DF PROTO=TCP SPT=16883 DPT=32963 WINDOW=0 RES=0x00 RST URGP=0 
kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=44 ID=0 DF PROTO=TCP SPT=16882 DPT=39586 WINDOW=0 RES=0x00 RST URGP=0
Those are almost all reset (RST) packets, but one in every couple/few hundred is an ACK RST:

kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=108 ID=4951 DF PROTO=TCP SPT=14878 DPT=59754 WINDOW=0 RES=0x00 ACK RST URGP=0
So I did some searching around, and came to this page, which seems to be quite similar, but honestly, most of it is over my head:

The TL;DR is that some packets were overly-large and getting rejected by netfilter as INVALID. On a mailing list thread linked in a comment there about a separate but similar issue, the cause of the INVALIDs were different and various ( ).

From that, I at least managed to figure out that I should turn on logging for invalid packets, by doing:

# echo 255 >/proc/sys/net/netfilter/nf_conntrack_log_invalid
**If there's someone out there who can explain the actual commands to do some of the other types of debugging they talk about there, I'd greatly appreciate it! In particular, "When enabling logging in the conntrack module for what causes the invalid packet" - I would love to know this.**

So anyway now, lo and behold, I see a bunch of packets being marked invalid in my syslog:

kernel: nf_ct_tcp: invalid RST IN= OUT= SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=44 ID=0 DF PROTO=TCP SPT=16882 DPT=48979 SEQ=2715593200 ACK=0 WINDOW=0 RES=0x00 RST URGP=0 
kernel: nf_ct_tcp: invalid RST IN= OUT= SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=41 ID=0 DF PROTO=TCP SPT=16883 DPT=43832 SEQ=1527390106 ACK=0 WINDOW=0 RES=0x00 RST URGP=0 
kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=MY:MA:CA:DD:RE:SS SRC= DST= LEN=40 TOS=0x00 PREC=0x20 TTL=41 ID=0 DF PROTO=TCP SPT=16883 DPT=43832 WINDOW=0 RES=0x00 RST URGP=0 
kernel: nf_ct_tcp: invalid packet ignored IN= OUT= SRC= DST= LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=18321 DPT=52969 SEQ=3891563822 ACK=3702885567 WINDOW=14600 RES=0x00 ACK SYN URGP=0 OPT (020405B40101040201030306) UID=1000 GID=1000
Pretty much all the inbound invalids seem to be RSTs, just like our high-port UFW blocks above, but waitasecond - that last one there is from me, outbound! But I have absolutely no UFW rules regarding outbound packets... Let's check our full set of iptables rules:

# iptables -S
-N ufw-after-forward
-N ufw-after-input
-N ufw-after-logging-forward
-N ufw-after-logging-input
-N ufw-after-logging-output
-N ufw-after-output
-N ufw-before-forward
-N ufw-before-input
-N ufw-before-logging-forward
-N ufw-before-logging-input
-N ufw-before-logging-output
-N ufw-before-output
-N ufw-logging-allow
-N ufw-logging-deny
-N ufw-not-local
-N ufw-reject-forward
-N ufw-reject-input
-N ufw-reject-output
-N ufw-skip-to-policy-forward
-N ufw-skip-to-policy-input
-N ufw-skip-to-policy-output
-N ufw-track-input
-N ufw-track-output
-N ufw-user-forward
-N ufw-user-input
-N ufw-user-limit
-N ufw-user-limit-accept
-N ufw-user-logging-forward
-N ufw-user-logging-input
-N ufw-user-logging-output
-N ufw-user-output
-A INPUT -j ufw-before-logging-input
-A INPUT -j ufw-before-input
-A INPUT -j ufw-after-input
-A INPUT -j ufw-after-logging-input
-A INPUT -j ufw-reject-input
-A INPUT -j ufw-track-input
-A FORWARD -j ufw-before-logging-forward
-A FORWARD -j ufw-before-forward
-A FORWARD -j ufw-after-forward
-A FORWARD -j ufw-after-logging-forward
-A FORWARD -j ufw-reject-forward
-A OUTPUT -j ufw-before-logging-output
-A OUTPUT -j ufw-before-output
-A OUTPUT -j ufw-after-output
-A OUTPUT -j ufw-after-logging-output
-A OUTPUT -j ufw-reject-output
-A OUTPUT -j ufw-track-output
-A ufw-after-input -p udp -m udp --dport 137 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 138 -j ufw-skip-to-policy-input
-A ufw-after-input -p tcp -m tcp --dport 139 -j ufw-skip-to-policy-input
-A ufw-after-input -p tcp -m tcp --dport 445 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 67 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 68 -j ufw-skip-to-policy-input
-A ufw-after-input -m addrtype --dst-type BROADCAST -j ufw-skip-to-policy-input
-A ufw-after-logging-forward -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-after-logging-input -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-before-forward -j ufw-user-forward
-A ufw-before-input -i lo -j ACCEPT
-A ufw-before-input -m state --state RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-input -m state --state INVALID -j ufw-logging-deny
-A ufw-before-input -m state --state INVALID -j DROP
-A ufw-before-input -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 4 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-input -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A ufw-before-input -j ufw-not-local
-A ufw-before-input -d -p udp -m udp --dport 5353 -j ACCEPT
-A ufw-before-input -d -p udp -m udp --dport 1900 -j ACCEPT
-A ufw-before-input -j ufw-user-input
-A ufw-before-output -o lo -j ACCEPT
-A ufw-before-output -m state --state RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-output -j ufw-user-output
-A ufw-logging-allow -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW ALLOW] "
-A ufw-logging-deny -m state --state INVALID -m limit --limit 3/min --limit-burst 10 -j RETURN
-A ufw-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-not-local -m addrtype --dst-type LOCAL -j RETURN
-A ufw-not-local -m addrtype --dst-type MULTICAST -j RETURN
-A ufw-not-local -m addrtype --dst-type BROADCAST -j RETURN
-A ufw-not-local -m limit --limit 3/min --limit-burst 10 -j ufw-logging-deny
-A ufw-not-local -j DROP
-A ufw-skip-to-policy-forward -j DROP
-A ufw-skip-to-policy-input -j DROP
-A ufw-skip-to-policy-output -j ACCEPT
-A ufw-track-output -p tcp -m state --state NEW -j ACCEPT
-A ufw-track-output -p udp -m state --state NEW -j ACCEPT
-A ufw-user-input -p udp -m multiport --dports 18320:18323 -j ACCEPT
-A ufw-user-input -p tcp -m multiport --dports 18320:18323 -j ACCEPT
-A ufw-user-limit -m limit --limit 3/min -j LOG --log-prefix "[UFW LIMIT BLOCK] "
-A ufw-user-limit -j REJECT --reject-with icmp-port-unreachable
-A ufw-user-limit-accept -j ACCEPT
Ok, so it will reject INVALID packets from me, too, that's fine. Except it's finding one about every 10 seconds, even when there's very little traffic overall... hmmmm... Is my wireless card borked and mangling packets, or is something else going on? I suspect that netfilter might be using an overly-strict definition of "INVALID."

So what's going on, here, and what can I do to get more information? Thanks so much to any and all who read this massive post!
Old 04-02-2014, 03:47 PM   #2
LQ Newbie
Registered: Mar 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
Some updates...

So, after a lot of reading about "invalid" packets, TCP and conntrack, I'm getting a clearer picture, but I'm still not sure how to fix this. But I've ruled some things out.

At first, I thought it might be a timeout problem - that these packets were "invalid" because they were late arrivals from peers that I was once connected to, but conntrack had decided I was done with them, even though they had not. I also read this: which talks about the ip_conntrack_tcp_timeout_close_wait setting (in /proc/sys/net/ipv4/netfilter/) potentially causing some problems.

So, I raised every one of the TCP timeouts in that /proc directory by a factor of 4 or more (after backing the originals up), and after almost two hours, I'm still getting just as many INVALIDs, perhaps even more.

Then I tried disabling conntrack's checksumming, in case these packets were getting rejected for bad checksums, by doing

# echo 0 > /proc/sys/net/ipv4/netfilter/ip_conntrack_checksum
...Which also had no effect.

What's a bit frustrating is that these options are all very poorly documented, and I know that raising some of these timeouts might have some undesireable effects, while it's possible that disabling checksumming might not actually prevent checksum failures from being marked as invalid.

Most of the invalid packets are still Resets (RST), but about one in thirty of them is an ACK, SYN, or ACK SYN. That ratio is true for both incoming and outgoing invalid packets. There are also about 30 times as many incoming invalid packets as outgoing.

I'm really mystified as to how my own outgoing packets are getting labeled invalid before they even leave the kernel... Please folks, what the heck is going on here?
Old 04-02-2014, 09:20 PM   #3
LQ Newbie
Registered: Mar 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
I'm beginning to think that this might even be an injected RST attack, like the ISP Comcast was railed against for carrying out years ago in 2007, and eventually ordered by a U.S. judge to halt in 2008 ( ). I am in fact a ComCast customer, and this might make sense... except I feel like a paranoid talking about a practice that was supposedly halted 6 years ago...

There is in fact many non-torrent anomalies going on, as well, including some invalid packets coming in from on ports 80 and 443, so I have to imagine that there's something going on in my configuration that's causing it, but I can't rule anything out at this point...
Old 04-04-2014, 04:09 PM   #4
LQ Newbie
Registered: Mar 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
So, I seem to have found a decent patrtial solution to this, inspired by this page: - specifically the bottom section, which reads:

DHT and IP connection tracking

For systems using IP connection tracking (this includes many routers as well as most operating systems, including Linux), a long-running and hence high-volume DHT server can produce an unnecessary strain because each incoming query creates a new connection, even though typically no other packets will follow after the first one.

When used in addition to firewall rules which drop invalid packets, this may cause genuine non-DHT connections to be dropped as well if the kernel runs out of memory for tracking connections.

There are several strategies to mitigate this:

Turn off the DHT server when it is not needed (default). This quickly causes incoming queries to stop when there is no response.
Reduce the DHT query volume. Since one cannot directly control incoming queries, this can be done by throttling the outgoing DHT bandwidth to discourage too many incoming queries.
Turn off connection tracking for DHT packets.

To turn off DHT connection tracking on Linux, use the NOTRACK target in the "raw" netfilter table. For instance, when using UDP port 6881 for DHT, execute these iptables commands:

iptables -t raw -I PREROUTING -p udp --dport 6881 -j NOTRACK
iptables -t raw -I OUTPUT -p udp --sport 6881 -j NOTRACK

Note that this causes all DHT packets to have a state of "UNTRACKED", keep that in mind if you have any state-based rules in the regular netfilter tables. Similarly, ICMP packets received in reply to untracked outgoing DHT packets (for instance "Host Unreachable" or "Time Exceeded") will have a state of "INVALID" because they do not belong to any known connection.
So, I added those rules as relevant to my connection:

# iptables -t raw -I PREROUTING -p udp --dport 18321 -j NOTRACK
# iptables -t raw -I OUTPUT -p udp --sport 18321 -j NOTRACK
And that did seem to cut down the number of invalid packets that were being blocked. There were still many, though, so I decided to try using that rule for the regular TCP torrent traffic, as well:

# iptables -t raw -I PREROUTING -p tcp --dport 18321 -j NOTRACK
# iptables -t raw -I OUTPUT -p tcp --sport 18321 -j NOTRACK
And now my torrent client is indeed performing much better, and far fewer torrent packets are being dropped, only about one per minute (exluding the many ICMP packets that are now being dropped, unfortunately, which I know no decent way to make a specific allowance for). And every single one that is being dropped is now a RST packet, again excluding ICMP. I still don't quite know what to draw from that, but it's info.

I would still like to have an arrangement where these RSTs are not being dropped (or perhaps not being sent in the first place, if they're not appropriate?), and where the ICMP traffic is able to do its job, but for now, this is a step up.

Last edited by d2d2; 04-04-2014 at 04:11 PM.
Old 04-10-2014, 11:44 AM   #5
LQ Newbie
Registered: Mar 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
More good news. So I was trying to work out the ICMP blocks, and found this document that's really excellent and goes in-depth:

Looking at my logs, I saw that all my ICMP blocks were types 3 (destination-unreachable) and 11 (Time Exceeded), all incoming. So, I issued the following two commands:

# iptables -I INPUT 1 -p icmp --icmp-type 3 -j ACCEPT
# iptables -I INPUT 1 -p icmp --icmp-type 11 -j ACCEPT
Still not sure if everything's working alright, but I'll keep this thread updated.

Last edited by d2d2; 04-10-2014 at 11:56 AM.
Old 04-12-2014, 12:51 PM   #6
LQ Newbie
Registered: Mar 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
I think I may have finally got it, although I had to open up the firewall a little more than I'd prefer. Fortunately for this case, doing that doesn't really open up any risks, but if I had more services running, it might be a problem.

So, I should mention that the torrent client I'm using is Deluge, which uses libtorrent-rasterbar as its back end. I finally got all my torrent traffic unblocked (DHT, peer connections, mystery "invalid" RST packets, peer ICMP packets, etc.), it seems. I essentially did this by disabling connection tracking with conntrack for all of the relevant torrent traffic, including on the ephemeral ports (or as deluge calls them, "Random ports").

I did this by disabling the "Use Random Ports" checkbox in deluge for all connections, incoming and outgoing, and specifying acceptable port ranges; for instance, 11000-11020 for incoming connections and 11021-11400 for outgoing connections. After setting the ports Deluge will use in that way, I closed Deluge and set my iptables rules - I'll include all of them here as entered commands, which will then need to be saved to preserve them on subsequent boots.

# iptables -t raw -I PREROUTING -p udp --dport 11000:11400 -j NOTRACK
# iptables -t raw -I OUTPUT -p udp --sport 11000:11400 -j NOTRACK
# iptables -t raw -I PREROUTING -p tcp --dport 11000:11400 -j NOTRACK
# iptables -t raw -I OUTPUT -p tcp --sport 11000:11400 -j NOTRACK
# iptables -I INPUT -p icmp --icmp-type 3 -j ACCEPT
# iptables -I INPUT -p icmp --icmp-type 4 -j ACCEPT
# iptables -I INPUT -p icmp --icmp-type 11 -j ACCEPT
# iptables -I INPUT -p icmp --icmp-type 12 -j ACCEPT
# iptables -I INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
It may be sloppy to just insert each rule as rule #1 over and over, but I don't know iptables well enough to do it another way. When I tried using the -A flag to append them initially, I found stuff was still getting blocked as the desired packets triggered some rule before the appended ones.

The ICMP allowances are desirable because once connection tracking is disabled on those ports, those important ICMP messages (types 3 (Destination Unreachable), 4 (Source Quench), 11 (Time Exceeded) and 12 (Parameter Problem)) would otherwise be declared INVALID themselves (as netfilter would not know of any connections that they are associated with), and they would potentially be blocked.

I don't really understand why netfilter/iptables was having issues with the RST packets on ephemeral ports, but it was, and I can only assume that there's something about the way that a torrent swarm (and especially a DHT torrent swarm) sends traffic around that's atypical, and conntrack doesn't know how to handle it. My guess is that some of the traffic is just too infrequent, and conntrack "forgets" about those peers, but I can't be sure - after all, when I raised the sysctl timeouts, I was still getting those blocks. But of course I wasn't really sure what I was doing.

So, with rules like the ones above in place, I think I can finally use torrents effectively with restrictive iptables in effect. Indeed, my torrents are seeding much more often, now - before they were hardly seeding at all, which was very obnoxious. I hope that this thread can help others with the same problem (and I imagine that almost anybody using bittorrent and iptables/netfilter is affected, which is a lot of people!). The only reason I discovered it is because of UFW's default configuration choosing to log some blocks in the syslog - hooray for logs!

Make sure to get the word out about how iptables/netfilter likes to call lots of torrent traffic "invalid," and let's keep those swarms healthy!

Last edited by d2d2; 04-13-2014 at 01:08 PM. Reason: Updated ICMP rules and reasoning
Old 05-31-2014, 09:04 PM   #7
LQ Newbie
Registered: Mar 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
A quick update. I've continued to try to learn about netfilter_conntrack and its behavior toward "invalid" packets, and I've learned some more important things. Because there seems to be EXCEEDINGLY little documentation about any of this stuff, I'd like to at least put it online here.

Firstly: when invalid packet logging is enabled, as by issuing the command mentioned above:

# echo 255 >/proc/sys/net/netfilter/nf_conntrack_log_invalid
or adding the follwing lines to /etc/sysctl.conf (or wherever your sysctl configuration file may be) to make this logging permanent across reboots:

net.netfilter.nf_conntrack_log_invalid = 255
net.ipv4.netfilter.ip_conntrack_log_invalid = 255
You may see in your syslog messages some lines like the following:

kernel: nf_ct_tcp: invalid packet ignored IN= OUT= SRC= DST= LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=9999 DPT=10101 SEQ=277160000 ACK=100670000 WINDOW=14600 RES=0x00 ACK SYN URGP=0 OPT (345405B40658456841030306) UID=1000 GID=1000
This is **NOT** a packet being dropped! I had to look at the freakin' linux kernel source code to find this out, but "ignored" in this case means that the packet is accepted! At least, I'm pretty sure, as a totally novice programmer. If anyone would like to verify, the relevant source (for a 3.2 kernel) is in the file linux-3.2.xx/net/netfilter/nf_conntrack_proto_tcp.c within the standard linux kernel source tree. It's available online at

You can search/grep for "invalid packet ignored," the section it's in concludes with "return NF_ACCEPT;" as opposed to some others that are obvious blocks that end with "return -NF_ACCEPT;" (note the - sign). So this seems to strongly suggest that these packets are in fact getting through. I really wish this log message (and even the comments in the source) were a little less ambiguous about that.

In other words, if you see a log message like the line above, DON'T PANIC! Your traffic isn't being blocked, it's just that conntrack is having a senile moment, is aware of that, and lets the traffic through.

In all this struggling with netfilter_conntrack, I am really wishing that this module had much better documentation. I may try to submit some, but it's daunting, and a lot of it is still stuff I don't really understand. If anyone does, please contribute some documentation (to the manpages and online)!


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Help Me!! UFW blocking weird IPs? mryemeni Linux - Security 3 02-06-2014 12:33 PM
UFW blocking certain incoming requests and not sure why smells_of_elderberries Linux - Security 10 04-02-2013 06:19 PM
Port forwarding: ufw/iptables not working? Weapon S Linux - Software 5 03-21-2013 03:03 AM
LXer: ufw - Iptables under Ubuntu LXer Syndicated Linux News 0 06-18-2011 04:00 AM
firewall blocking node that shouldn't be in my LAN! ming0 Linux - Security 11 03-04-2005 09:20 AM

All times are GMT -5. The time now is 11:09 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration