I have no idea what the "Linux layer 7 packet classifier" is, but from an architecture point of view, you certainly would want to patch and compile the kernel before you patch and compile IP tables.
If the software depends on some modifications to the kernel, it won't/can't run until the correct kernel version is in place. Best case scenario doing IP tables before the kernel is it just won't work as it is expected to.
Think about it this way - the kernel is the first floor of a house, and everything else is a higher floor. If you were going to have work done on both the 1st floor (kernel) and 2nd floor (software), would you do work on the 2nd before the 1st was done? No. Get the base correct, then build on top of it, not get the top right then try to support it (and drop and break it in the process)!