Netfilter - analyse dropped packets
 

Hi all.
I have a headless box as the gateway to my network.
I have a bunch of specific netfilter rules with a default drop in my input chain.
I have used the log rule at the end of the chain to log packets before they are dropped.
But,
What i really want to do is capture the entire raw packets before dropping and analyse them remotely.

Is there anyway to do something like this?

Sounds like you want an IDS...

Yep, what kbp said. You won't be able to have netfilter do this. And while you can log with Snort/tcpdump, correlating the snort/tcpdump (ascii, not pcap) logs with the firewall logs can be a rather tedious process (you might be able to script that process).

A pcap app sees packets before Netfilter does. That shouldn't matter if you can create a BPF filter to compensate for accepted traffic or if you use Snort DAQ which uses NFQUEUE. Netfilter itself provides targets like ULOG / NFLOG and (NF)QUEUE to copy packets which userland apps can read or store and decide routing.

This is already going on a tangent, here, but let me explain.

I'm well aware of the BPF option/capability. And also, if there's a firewall in front of the sniffer/IDS, that sniffer/IDS will see nothing. If you're talking about running a sniffer/IDS on the same box/interface as the firewall, yeah, of course, the IDS will see what the firewall dropped. I didn't remember saying that it couldn't be done. I was just suggesting to the OP of the complications he/she may face. Correlating FW and snort logs may be easy to us folk that can use such tools in their sleep, but there are some people that will have difficulty.

IMO, ASCII files are easier to search, because you can grep the file...they're just flat files. You can't grep a pcap (well, you might be able to, but it won't look pretty and you're not going to see what you think you're gonna see). Sure, you can use Wireshark or any other pcap reader (I mentioned snort and tcpdump in my previous post), but IMO, nothing beats grep for quick-fast glimpses of files. I guess I prefer being able to view the capture with the least amount of limitation or complication. I can take an ASCII dump and view it on a system I don't own, even if that system doesn't have a pcap reader....can't do the same thing with a pcap file, though. This comes from me time-sharing computer systems at work the last 10 years (without having admin/root access to those machines). On some of my servers, I configure my snort process to do both pcap and ASCII, in case my ASCII capture is lacking. I acknowledge that in some cases, BPF is better, but again, I don't like to be stuck with a file I can't view when supporting a client on a machine I don't have control over.

I'm not entirely sure the reason for analyzing the packets. Whether it's for some kind of audit of the firewall, or to detect incidents. If it's the latter, you should understand that traffic that is blocked, probably isn't going to hurt you. It's the traffic you allow that you really need to inspect with an IDS.

Uh. I was just informing the OP wrt your "You won't be able to have netfilter do this" but thanks for the explanation.

Thanks for the suggestions,
I'm going to have a read up on bpf and Snort DAQ.
My first idea was to use wireshark remote capture and filter out packets according to the accept rules i had but, i cant get the remote capture daemon to work. It just keeps kicking the connection.

The reason i want to do this is because my filter rules are pessimistic and restrictive. Every now and then something wont work because packets are being dropped, so it would help in separating what i do want from what i don't want. Also, i'd like to be able to see in depth, what 'crap' is being fired at me from the wan side and, on the lan side, what some other os'es like to scream down the pipes for no apparent reason.

It's nothing mission critical, just a pet project at home.

Sometimes it pays to know what's being dropped. This is especially true when you've found yourself compromised. And outbound blocks are especially crucial to understand. A firewall is not the end-all security solution...it's efficient but also rather 'dumb'. You'll never be able to perform analysis of a security issue with firewall logs alone. When I see something questionable within my firewall logs, I immediately go to my IDS to gain further insight. If the IDS didn't record anything (remember, it will only alert on known badness that you've configured to alert on), that's when I'll run an adhoc pcap, hoping to record traffic from a specific host, set of hosts, or protocol(s).

In re-reading your post, I see now what you meant. I did mistakenly think your post was meant for me and not the OP (quoting might've helped in discerning who you were directing your post toward).

While I've not tried Snort DAQ, ULOG/NFLOG, or (NF)QUEUE, a simple tcpdump might be quicker (unless the OP is intending this as a long term or never-ending project). I can see the value in gathering packets only on dropped FW traffic, though...that should be much more efficient than running a full-blown instance of Snort. Would also be nice to know the performance impact of using such aforementioned tools.

Yup, I agree 100%. The OP was kind of unclear and people were talking about running an IDS... I was just saying if the OP was asking about analyzing blocked traffic, running an IDS like Snort would be more useful on the allowed traffic.

Yeah, there's that, too! In the past, I've seen the source of blocked traffic also being passed on other allowed protocols. Might be worth looking at ALL traffic, in cases like that.

Thanks for pointing me in the direction of snort. I know it's supposed to be an IDS but in this case it still helped. I put the queue filter before drop and used 'snort_inline -QXv' to dump the whole packet data to stdout which is then piped over to the remote host with netcat. It's not the most elegant solution but it is pretty much what i wanted. I don't think performance would be up to much done this way but it is only for dropped packets and i have no intention of returning them to the filter anyway.