Rather huge IPtables chain, iptables: Memory allocation problem.
 

I'm trying to import a rather gigantic list of IP ranges into iptables.

It's 22 thousand lines.

I get to about 17K, and iptables starts spitting out:
iptables: Memory allocation problem.

I assume this is because i've exhausted some memory limit in iptables.

Is there a method of getting around this?

Thanks!

I'm not sure exactly how you are attempting to import the rules, but suspect that a change in approach or breaking the import into smaller "chunks" may prove better results.

Are you able to create multiple distinct imports?


:study:

Basically, i've taken a pre-existing list, and converted it to a series of "iptables -A ...." commands.

Each of these 22000 lines is another command similar to "iptables -A chainname -m iprange --src-range 1.2.3.4-5.6.7.8 -j DROP

this actually brings me to another question....

Is there a better way to add rules via a script?


Also, some more information.
That error was produced when i was testing the import on my workstation. Which is rather beefy, quad core, 8gb of memory. Running FC11.

The target of this whole project is a much lower end machine, running Smoothwall Express 3. I have not tried the import there yet.

hmmm, odd. I wrote a loop that created a single rule for IPs x.y.1-254.1-254 on a test box that made it from x.y.1-129.z without issue when I last checked. It is a RHEL5.x install, single cpu, maybe 512MB RAM... 128*254 would put last count above 32512 rules. I will check in later and update.

Have you tried creating 2 or 3 scripts out of your rules and running them individually?

Looks like I hit a snag. Not able to create a rule beyond rule 55399 on the filter table. I can still write to the nat table though... I just wrote a single rule for each IP x.y.1-32.1-254 as a test without any problems on nat PREROUTING chain while filter chains will not accept any more. I guess it is possible that you are hitting a similar ceiling, but sooner...

Hope this helps.


Any way to consolidate some of the rules?


:study:

I'm looking into that now.
The original list, has every singe entry listed as a range. Even if it's one IP. I think that at the very least, i should be able to hack out the non-range ranges, and replace them with single entry rules. I may also be able to take the ranges, and convert them from a range format, to an ip/subnet format. Say, change 1.2.3.0-1.2.3.255 to 1.2.3.0/24

If i change that around, i should be able to add the rules without the -m iprange option. Which may (or... may not) help with resources.

working on my importer a bit, and i realized something.
I was off by a bit on my list length. A quick look at the number of lines, made me think that i was looking at 22K lines. I looked more closely today, and it's actually... 226K lines.

I have, however, made some improvements, and i'm no longer using the iprange module. I'm testing the import now, we'll see what happens.

tried breaking the adds up into multiple lists, it was no help. is it possible to increase the available memory to iptables?

it was all in my method.
i seem to have gotten around the issue.
It wasnt iptables causing the problem, rather, bash.

I was adding all of these rules via a bash script.

I chagned my importer to output an iptables-restore formatted file, and now it imports no sweat.

Thanks for the input!

Thanks for the followup!