Netfilter marking facebook packets as 'invalid'

alpha645 · 12-27-2012, 11:40 AM

Lately, while checking the netfilter logs on an Ubuntu Precise (12.04) running v3.8rc1 (for nouveau testing) I noticed *a lot* of packets being blocked. Most of them carried the ip: 31.13.64.7 .

Whois showed that this was a family member visiting facebook.

Looking on Google only yielded results on how to effectively block facebook (along with same really bad examples btw...).

Some further digging, it seems that netfilter's 'state' module marks facebook packets as 'invalid'.

Initial connections work fine though, the problem can be reproduced most effictively by opening a fb page, like this one. Then scrolling down and wait for it to load more content. But make sure you wait random moments of time before you do so.

If you add a logfilter to catch these packets, it will trigger.

27-12-2012 16:01 .warning ::: pkt invalid: IN=eth0 OUT= MAC=00:0c:76:1c:dd:c8:00:1b:2f:a9:48:e8:08:00 SRC=31.13.64.7 DST=10.32.32.65 LEN=40 TOS=0x00 PREC=0x00 TTL=250 ID=33435 DF PROTO=TCP SPT=443 DPT=44911 WINDOW=0 RES=0x00 ACK RST URGP=0
27-12-2012 16:01 .warning ::: pkt invalid: IN=eth0 OUT= MAC=00:0c:76:1c:dd:c8:00:1b:2f:a9:48:e8:08:00 SRC=31.13.64.7 DST=10.32.32.65 LEN=40 TOS=0x00 PREC=0x00 TTL=250 ID=4902 DF PROTO=TCP SPT=80 DPT=51042 WINDOW=0 RES=0x00 ACK RST URGP=0

Does someone have any idea what's going on here?

I tripped over this since I used ESTABLISHED,UNTRACKED and NEW to sort packets in the 'filter' table. Everything else like INVALID,RELATED(don't use this) are not relevant whatsoever and are blocked (like it should be..).

Netfilter producing false positives is something new to me...

unSpawn · 12-28-2012, 07:27 AM

Quote:

Originally Posted by alpha645

Does someone have any idea what's going on here? (..) Netfilter producing false positives is something new to me...

If you strip the similarities you end up with a payload-less TCP packet, just IP + TCP headers = 20 + 20 bytes, for port HTTP and HTTPS and the only thing you can say about it is it terminates their connections. I would not conclude from these few lines that Netfilter produces false positives but would rather let a packet trace confirm or deny the connection was OK.

Quote:

Originally Posted by alpha645

I tripped over this since I used ESTABLISHED,UNTRACKED and NEW to sort packets in the 'filter' table. Everything else like INVALID,RELATED(don't use this) are not relevant whatsoever and are blocked (like it should be..).

INVALID means conntrack considers the packet out of bounds, not part of a known connection, and RELATED is used because of the way protocols like FTP work. So "INVALID,RELATED" is not a useful state combo IMHO as both states serve different purposes. Use of the common ESTABLISHED state should be clear but the only reason to use UNTRACKED is if you used a NOTRACK target earlier in the raw table. So "ESTABLISHED,UNTRACKED" is not a useful state combo IMHO as both states serve different purposes. As with most things Netfilter attaching output from running 'sudo iptables-save' would have been useful.

alpha645 · 12-28-2012, 08:11 AM

Thank you for your time and patience.

Below is my firewall, hopefully you will understand why I group ESTABLISHED,UNTRACKED and group everything else together.

Btw, NOTRACK is deprecated by '-j CT --notrack'.

And I'm not explicitly using RELATED since I don't enable any helpers (which used to be activated automatically by default).

Quote:

Originally Posted by /etc/iptables/rules.v4

*raw
:PREROUTING DROP [0:0]
:OUTPUT DROP [0:0]
:BAD - [0:0]
:BC - [0:0]
:IN - [0:0]
:FREE - [0:0]
:PROTO - [0:0]
:OUT - [0:0]
-A PREROUTING -m recent --rcheck --seconds 10 --name BAD --rsource -j BAD
-A PREROUTING -j PROTO
-A PREROUTING -m addrtype --dst-type LOCAL -j IN
-A PREROUTING -m addrtype --dst-type BROADCAST -j BC
-A PREROUTING -d 239.255.255.250/32 -p udp -m addrtype --dst-type MULTICAST -m udp --dport 1900 -j FREE
-A PREROUTING -j LOG --log-prefix "bpkt raw in: " --log-level alert
-A OUTPUT -m addrtype --src-type LOCAL --dst-type UNICAST,LOCAL -j OUT
-A OUTPUT -m addrtype --src-type LOCAL --dst-type BROADCAST -j BC
-A OUTPUT -d 239.255.255.250/32 -p udp -m addrtype --dst-type MULTICAST -m udp --dport 1900 -j FREE
-A OUTPUT -j LOG --log-prefix "bpkt raw out: " --log-level alert
-A BAD -j LOG --log-prefix "bpkt raw repeat: " --log-level alert
-A BAD -j DROP
-A BC -p tcp -m addrtype ! --src-type LOCAL -m tcp --dport 514 -j FREE
-A BC -p udp -m addrtype ! --src-type LOCAL -m udp --dport 514 -j FREE
-A BC -p udp -m udp --sport 631 --dport 631 -j FREE
-A BC -p udp -m udp --dport 137 -j FREE
-A BC -p udp -m udp --sport 138 --dport 138 -j FREE
-A BC -j LOG --log-prefix "bpkt raw broadcast: " --log-level alert
-A BC -j DROP
-A IN -i lo -j FREE
-A IN -p icmp -j FREE
-A IN -p tcp --dport 57090 -j RETURN
-A IN -s 10.32.32.64/27 -j ACCEPT
-A IN -p tcp -m multiport --dports 631,139,445 -j RETURN
-A IN -p udp -m multiport --dports 631,137,138 -j RETURN
-A IN -j ACCEPT
-A FREE -j CT --notrack
-A FREE -j ACCEPT
-A PROTO -p tcp -j RETURN
-A PROTO -p udp -j RETURN
-A PROTO -p icmp -j RETURN
-A PROTO -j LOG --log-prefix "bpkt raw proto: " --log-level alert
-A PROTO -m recent --set --name BAD --rsource -j DROP
-A OUT -o lo -j FREE
-A OUT ! -p icmp -j ACCEPT
-A OUT -p icmp -m icmp --icmp-type echo-request -j ACCEPT
COMMIT
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:IN - [0:0]
:OUT-NEW - [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state ESTABLISHED,UNTRACKED -j ACCEPT
-A INPUT -m state --state NEW -m hashlimit --hashlimit-name all --hashlimit-upto 500/second --hashlimit-burst 750 --hashlimit-mode srcip --hashlimit-srcmask 24 -j IN
-A INPUT -j LOG --log-prefix "bpkt filter in: " --log-level alert
-A INPUT -m recent --set --name BAD --rsource
-A OUTPUT -m state --state NEW -m recent --remove --name BAD --rdest
-A OUTPUT -m connmark --mark 0x1 -j OUT-NEW
-A IN -m connlimit --connlimit-above 25 -j RETURN
-A IN -m hashlimit --hashlimit-name specific --hashlimit-above 100/second --hashlimit-burst 150 --hashlimit-mode srcip -j RETURN
-A IN -p tcp -j CONNMARK --set-xmark 0x1/0xffffffff
-A IN -j ACCEPT
-A OUT-NEW -j CONNMARK --set-xmark 0x0/0xffffffff
-A OUT-NEW -p tcp -m tcp ! --tcp-flags ALL ACK,RST -j RETURN
-A OUT-NEW -j LOG --log-prefix "bpkt filter out: " --log-level alert
-A OUT-NEW -m recent --set --name BAD --rdest -j DROP
COMMIT

unSpawn · 12-28-2012, 08:30 AM

Looks cool to me. Still I suggest you use packet traces to explain any connection anomalies.

alpha645 · 12-29-2012, 04:02 AM

I did a packet trace using the TRACE function in iptables (no tcpdump, yet). Looks okay to me:

http://pastebin.com/raw.php?i=S2PKWMv2

(Ow btw, invalid packets can be detected by searching for: 'bpkt invalid'.)

I think this really is netfilter being paranoid. Maybe I should increase a timeout somewhere since Facebook is stretching the limit of some sorts?

Both packets that are marked as invalid do contain 'ACK RST'. Maybe this is my router doing fancy stuff (Netgear DG834N -> Proprietary Linux 2.4).

alpha645 · 12-29-2012, 07:02 AM

Think I nailed it. I forgot netfilter had debug options specifically for these packets:

[33529.910660] nf_ct_tcp: invalid RST IN= OUT= SRC=31.13.64.7 DST=10.32.32.65 LEN=40 TOS=0x00 PREC=0x00 TTL=250 ID=1178 DF PROTO=TCP SPT=80 DPT=49502 SEQ=1495528897 ACK=1167249799 WINDOW=0 RES=0x00 ACK RST URGP=0
[33632.451036] nf_ct_tcp: invalid RST IN= OUT= SRC=31.13.64.7 DST=10.32.32.65 LEN=40 TOS=0x00 PREC=0x00 TTL=250 ID=22279 DF PROTO=TCP SPT=443 DPT=60801 SEQ=3562358858 ACK=1445399758 WINDOW=0 RES=0x00 ACK RST URGP=0

Something is doing something ugly to my connection. And I think I'll blame my router and call it a day =)