Trouble patching iptables with IMQ
 

Hi All,

I'm trying to apply the IMQ patch to iptables-1.4.6 but I'm encountering an error following the instructions given here (http://wiki.nix.hu/cgi-bin/twiki/vie...port_in_iptabl). The patch appears to apply successfully but when I try to run the chmod command (chmod +x /extensions/.IMQ-test.*) is says 'no such file or directory'.

When I check in extensions, there are no IMQ files, only one in the upper directory called libxt_IMQ.c

How can I apply this patch successfully?

Thanks
Anubis.

First of all after reading the doc you linked me to I notice two things that you may have overlooked in your assessment.
1st the command is:
Notice that it is extensions/.IMQ-test* and not /extensions/.IMQ-test* like in your mentioned command as it makes a huge difference in the path being referenced.

2nd you probably don't see any .IMQ-test files because you're not listing hidden files. In *nix files and folders that start with a period ('.') are hidden.

Double check to make sure that you use the correct commands outlined in the doc and if you're listing files you may wish to list hidden files as well. This can easily be done with this command...
Please let us know of your results....
SAM

Hi Sam,

Thanks for the reply.

Here are the directory listings:

bcg001:/kernelcomp/iptables-1.4.6# ls -h
aclocal.m4 extensions ip6tables-save.8 iptables-save.8 ltmain.sh
autogen.sh include ip6tables-save.c iptables-save.c m4
COMMIT_NOTES INCOMPATIBILITIES ip6tables-standalone.c iptables-standalone.c Makefile.am
compile INSTALL iptables.8.in iptables-xml.8 Makefile.in
config.guess install-sh iptables-apply iptables-xml.c missing
config.h.in ip6tables.8.in iptables-apply.8 iptables.xslt release.sh
config.sub ip6tables.c iptables.c libipq xshared.c
configure ip6tables-multi.c iptables-multi.c libiptc xshared.h
configure.ac ip6tables-multi.h iptables-multi.h libiptc.pc.in xtables.c
COPYING ip6tables-restore.8 iptables-restore.8 libxt_IMQ.c xtables.pc.in
depcomp ip6tables-restore.c iptables-restore.c linux
bcg001:/kernelcomp/iptables-1.4.6#

bcg001:/kernelcomp/iptables-1.4.6# ls -h extensions/
dscp_helper.c libipt_MASQUERADE.man libxt_conntrack.man libxt_rateest.c
GNUmakefile.in libipt_MIRROR.c libxt_dccp.c libxt_RATEEST.c
libip6t_ah.c libipt_MIRROR.man libxt_dccp.man libxt_rateest.man
libip6t_ah.man libipt_NETMAP.c libxt_dscp.c libxt_RATEEST.man
libip6t_dst.c libipt_NETMAP.man libxt_DSCP.c libxt_recent.c
libip6t_dst.man libipt_realm.c libxt_dscp.man libxt_recent.man
libip6t_eui64.c libipt_realm.man libxt_DSCP.man libxt_sctp.c
libip6t_eui64.man libipt_REDIRECT.c libxt_esp.c libxt_sctp.man
libip6t_frag.c libipt_REDIRECT.man libxt_esp.man libxt_SECMARK.c
libip6t_frag.man libipt_REJECT.c libxt_hashlimit.c libxt_SECMARK.man
libip6t_hbh.c libipt_REJECT.man libxt_hashlimit.man libxt_socket.c
libip6t_hbh.man libipt_SAME.c libxt_helper.c libxt_socket.man
libip6t_hl.c libipt_SAME.man libxt_helper.man libxt_standard.c
libip6t_HL.c libipt_set.c libxt_IMQ.c libxt_state.c
libip6t_hl.man libipt_SET.c libxt_iprange.c libxt_state.man
libip6t_HL.man libipt_set.h libxt_iprange.man libxt_statistic.c
libip6t_icmp6.c libipt_set.man libxt_length.c libxt_statistic.man
libip6t_icmp6.man libipt_SET.man libxt_length.man libxt_string.c
libip6t_ipv6header.c libipt_SNAT.c libxt_limit.c libxt_string.man
libip6t_ipv6header.man libipt_SNAT.man libxt_limit.man libxt_tcp.c
libip6t_LOG.c libipt_ttl.c libxt_mac.c libxt_tcp.man
libip6t_LOG.man libipt_TTL.c libxt_mac.man libxt_tcpmss.c
libip6t_mh.c libipt_ttl.man libxt_mark.c libxt_TCPMSS.c
libip6t_mh.man libipt_TTL.man libxt_MARK.c libxt_tcpmss.man
libip6t_REJECT.c libipt_ULOG.c libxt_mark.man libxt_TCPMSS.man
libip6t_REJECT.man libipt_ULOG.man libxt_MARK.man libxt_TCPOPTSTRIP.c
libip6t_rt.c libipt_unclean.c libxt_multiport.c libxt_TCPOPTSTRIP.man
libip6t_rt.man libipt_unclean.man libxt_multiport.man libxt_time.c
libipt_addrtype.c libxt_CLASSIFY.c libxt_NFLOG.c libxt_time.man
libipt_addrtype.man libxt_CLASSIFY.man libxt_NFLOG.man libxt_tos.c
libipt_ah.c libxt_cluster.c libxt_NFQUEUE.c libxt_TOS.c
libipt_ah.man libxt_cluster.man libxt_NFQUEUE.man libxt_tos.man
libipt_CLUSTERIP.c libxt_comment.c libxt_NOTRACK.c libxt_TOS.man
libipt_CLUSTERIP.man libxt_comment.man libxt_NOTRACK.man libxt_TPROXY.c
libipt_DNAT.c libxt_connbytes.c libxt_osf.c libxt_TPROXY.man
libipt_DNAT.man libxt_connbytes.man libxt_owner.c libxt_TRACE.c
libipt_ecn.c libxt_connlimit.c libxt_owner.man libxt_TRACE.man
libipt_ECN.c libxt_connlimit.man libxt_physdev.c libxt_u32.c
libipt_ecn.man libxt_connmark.c libxt_physdev.man libxt_u32.man
libipt_ECN.man libxt_CONNMARK.c libxt_pkttype.c libxt_udp.c
libipt_icmp.c libxt_connmark.man libxt_pkttype.man libxt_udp.man
libipt_icmp.man libxt_CONNMARK.man libxt_policy.c linux
libipt_LOG.c libxt_CONNSECMARK.c libxt_policy.man tos_values.c
libipt_LOG.man libxt_CONNSECMARK.man libxt_quota.c
libipt_MASQUERADE.c libxt_conntrack.c libxt_quota.man
bcg001:/kernelcomp/iptables-1.4.6#


Even with the -h I still can't see the file.

In the instructions for patching the iptables, it says to extract the patch to "iptables/" source directoy. I believe the first directory is the source directory, but perhaps I'm wrong.

Thanks again, I was starting to loose hope that anyone would answer :)
Anubis.

ls -lah is the listing command.
l for long file format
a for all files
h for calculating human readable values of the filesizes

For now I'm gonna go to bed but I'll take another crack at this tomorrow.

From what I can tell the patch is pretty much straight forward for IPTables. (it was bugging me so I couldn't sleep)
Put iptables-1.4.6-imq.diff in the iptables-1.4.6 source directory. Then run the following command sequence. Don't follow that other guide because it's crap since you're using the source directly and a different version altogether.

Your woes should be cured. Let me know how it goes...
SAM

Hi Sam,

Thanks again for the reply; and I can certainly appreciate the lack of sleep when trying to solve an issue; I've been desperately trying to solve the IMQ setup for the past week now and it looks like with your solution it's got it.

IPTables is now accepting -j IMQ which is further than it's ever got so now I've got to move to the testing stage. I'll keep you posted.

On another note (I'll create another post on this on Sunday after some testing); the reason for needing to implement IMQ is to shape our users on our network. We're a small ISP on a sat backbone and for us; every 'bit' counts.

I'm wanting to allow our users to use their full bandwidth allocation (ie: 256, 512 or 1024Kbit/s speed) on our local network and also on the internet backbone; but as in normal ISP's; we share our backbone bandwidth with a larger number of customers; so I'm wanting to implement two 'global' type rules, one which sets an overall 'parent' rule for our own IP block of 100Mbit/s then another parent rule for 'the internet' which is set to (just under) our backbone's speed.

With our current gateway (a proprietary device) it's allowing a single user to saturate our backbone which is certainly not a desired outcome!

I was going to set up 4 IMQ devices. 2 (ingress and egress) for the local-based network and 2 (ingress and egress) for 'the internet' using a configuration similar to the below:


# QDiscs attached to IMQ devices
tc qdisc add dev imq0 root handle 1:0 htb default 2
tc qdisc add dev imq1 root handle 1:0 htb default 2
tc qdisc add dev imq2 root handle 1:0 htb default 2
tc qdisc add dev imq3 root handle 1:0 htb default 2

# Parent classes for 'local' network
tc class replace dev imq0 parent 1:0 classid 1:1 htb rate 100Mbit prio 2 quantum 1500
tc class replace dev imq1 parent 1:0 classid 1:1 htb rate 100Mbit prio 2 quantum 1500

# Parent classes for 'the internet' network
tc class replace dev imq2 parent 1:0 classid 1:1 htb rate 10Mbit prio 2 quantum 1500
tc class replace dev imq3 parent 1:0 classid 1:1 htb rate 6Mbit prio 2 quantum 1500


# PER CLIENT
# Insert IPTables records classifying the client's packets
iptables -t mangle -I POSTROUTING -s {ipaddressofclient} -d {localnetworkrange} -j CLASSIFY --set-class 1:100
iptables -t mangle -I POSTROUTING -d {ipaddressofclient} -s {localnetworkrange} -j CLASSIFY --set-class 1:100
iptables -t mangle -I POSTROUTING -s {ipaddressofclient} -j CLASSIFY --set-class 1:101
iptables -t mangle -I POSTROUTING -d {ipaddressofclient} -j CLASSIFY --set-class 1:101

# Specify the client's minimum and maximum bandwidth levels
tc class replace dev imq0 parent 1:1 classid 1:100 htb rate 25kbit ceil 256kbit prio 100 quantum 1500
tc class replace dev imq1 parent 1:1 classid 1:100 htb rate 12kbit ceil 128kbit prio 100 quantum 1500
tc class replace dev imq2 parent 1:1 classid 1:101 htb rate 25kbit ceil 256kbit prio 101 quantum 1500
tc class replace dev imq3 parent 1:1 classid 1:101 htb rate 12kbit ceil 128kbit prio 101 quantum 1500


If there's anything you can see from these rules that seems odd; please do let me know. Any comments are highly appreciated!


Anyway, thanks again very, very much for your help!!

Goodnight.
Anubis.

Ok I'll take a look. And thanks for sharing the situation and some source configurations. It's much appreciated.

edit: You may also want to check out FreeBSDs default dummynet with ipfw.
http://info.iet.unipi.it/~luigi/dummynet/

SAM

In the future it's best to open the diff patch file in a text editor and view what the patch is doing. This way you know how to correctly apply it.

After typing 'man patch' into google I found how the patch command works. And chose the correct 'p' level for patch (patch -p0) for the diff file I wanted to apply.

When working with source code it's almost always best to go about it with that method.

Basically the patch you applied only created two files:
extensions/libxt_IMQ.c
include/linux/netfilter/xt_IMQ.h

So there was no need for all those other extra commands which you were attempting to use before. I got the configure, compile, and install commands from the INSTALL readme file from the iptables source code. Hopefully knowing this you won't have so much of a headache next time.

Cheers,
SAM

Hi Sam,

Thanks again for your help.

I have one other issue with IMQ you might be able to help with.


I have a perl script that is executed at boot time which runs the following:

`/sbin/iptables -t mangle -A POSTROUTING -m realm ! --realm 10 -j IMQ --todev 0`;

However this always fails with:

iptables v1.4.2: Unknown arg `(null)'
Try `iptables -h' or 'iptables --help' for more information.

However, if I run the script manually from the command line, it works and is inserted successfully. By the time this command is run, the imq devices have already been brought online using "ip link set imq0 up"

Any idea's what might be causing this?

Thanks
Anubis.

How are you starting the script in Perl?

How and when are you initiating the script in the startup process?

And what distribution of GNU/Linux are you running so that I may take you through its startup process?

It is unlikely there is a Perl script issue but I asked just in case. You might be attempting to run the script too early which is why I asked.

Hi Sam,

Thanks for the very quick reply.

While I'm trying to write and debug the script, I'm manually executing it by typing "/etc/MyFolder/MyScript.pm" at the command line once the pc has booted; so shouldn't be a timing issue.

The file 'MyScript.pm' then executes several other scripts internally using the `` characters (one script of which sets up the imq iptable rules).

As mentioned, when executing the line directly at the command line, it does function; so seems really strange to me.

I'm using Debian with the patched Kernel and IPTables.


Also, just another quick one, I hope you don't mind. While testing the IMQ integration, I can see my packets entering the IMQ0 device, but they're not getting out. Does the IMQ device re-transit the iptables using a different source address which I may be blocking?

It's after 4:30am now, so I best get 2 minutes sleep, thanks again for your help; I'm very grateful for it.

Thanks
Anubis.

The issue is it doesn't execute from directly within the script correct? I'm reading some manuals now.

edit: Do you need the output of the iptables command? If not then you should execute the iptables command like the following...

As for your question on the IMQ device, I'll have to read some more to give you a definitive answer. A lot of these questions are on subjects I've never touched which require a little bit of a learning curve for me.

Hi Sam,

I've found the problem with the unknown argument. When I use "/sbin/iptables" it throws the error, but when I use the normal command (just) "iptables" at the command line, it works.

Any idea what this could be?


I ran the whereis command and I think when I've recompiled the kernel and installed the patched version of iptables, it's not updated properly and the old iptables is stil in /sbin/iptables.

bcg001:~# /sbin/iptables -t mangle -A POSTROUTING -m realm --realm 10 -j IMQ --todev 0
iptables v1.4.2: Unknown arg `(null)'
Try `iptables -h' or 'iptables --help' for more information.
bcg001:~# iptables -t mangle -A POSTROUTING -m realm --realm 10 -j IMQ --todev 0
bcg001:~# whereis iptables
iptables: /usr/src/iptables-1.4.2/iptables.c /usr/src/iptables-1.4.2/iptables.xslt /sbin/iptables /etc/iptables.conf /usr/local/sbin/iptables /usr/share/iptables /usr/share/man/man8/iptables.8.gz
bcg001:~#


Also, I think I've found the other issue, I think I was sending the data to the wrong IMQ device. :) I'll let you know later after more testing.

Thanks
Anubis.

Ah I was going to say use the whereis command until I read your post twice...

Try using the find command to find all instances of iptables. Run either one of these commands as root...

Hi Sam,

I've ran the iptables command again (without arguments) and I'm definately getting the unpatched version using /sbin/; see below:

bcg001:~# iptables
iptables v1.4.6: no command specified
Try `iptables -h' or 'iptables --help' for more information.
bcg001:~# /sbin/iptables
iptables v1.4.2: no command specified
Try `iptables -h' or 'iptables --help' for more information.
bcg001:~#


I've ran the commands as you've mentioned above and copied the outputs below. I remember reading somewhere to uninstall the existing iptables before installing another, but I think, not only did I miss that step, I don't know how to do it in the first place. Additionally, I thought installing iptables would have replaced the old version??

Thanks again.
Anubis.


find / -type f -name iptables

Returns:
/sbin/iptables
/usr/share/lintian/overrides/iptables


find / | grep iptables

Returns:
Attached as a text file.