If you never compiled a kernel the basics are
here.
*If you're installing a new tarball of netfilter/iptables you'd want to do the patch-o-matic thing first, this will patch the kernel source and add options under "netfilter config".
In the Xconfig/config menu in one of the first tabs select "code maturity" and activate it to read something like "developmental/experimental", go to Networking tab, activate "network packet filtering". Now select "netfilter Configuration" and activate all modules.
When done selecting options hanlde making/installing the kernel as usual.
HTH somehow.
/* Btw, IMO this ain't a security question but a software/config one. Please consider your forum target before posting. */