LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 09-10-2009, 03:20 PM   #1
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Rep: Reputation: 15
Rather huge IPtables chain, iptables: Memory allocation problem.


I'm trying to import a rather gigantic list of IP ranges into iptables.

It's 22 thousand lines.

I get to about 17K, and iptables starts spitting out:
iptables: Memory allocation problem.

I assume this is because i've exhausted some memory limit in iptables.

Is there a method of getting around this?

Thanks!
 
Old 09-10-2009, 06:34 PM   #2
rayfordj
Member
 
Registered: Feb 2008
Location: Texas
Distribution: Fedora, RHEL, CentOS
Posts: 475

Rep: Reputation: 73
I'm not sure exactly how you are attempting to import the rules, but suspect that a change in approach or breaking the import into smaller "chunks" may prove better results.

Are you able to create multiple distinct imports?


 
Old 09-10-2009, 09:55 PM   #3
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Original Poster
Rep: Reputation: 15
Basically, i've taken a pre-existing list, and converted it to a series of "iptables -A ...." commands.

Each of these 22000 lines is another command similar to "iptables -A chainname -m iprange --src-range 1.2.3.4-5.6.7.8 -j DROP
 
Old 09-11-2009, 07:10 AM   #4
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Original Poster
Rep: Reputation: 15
this actually brings me to another question....

Is there a better way to add rules via a script?


Also, some more information.
That error was produced when i was testing the import on my workstation. Which is rather beefy, quad core, 8gb of memory. Running FC11.

The target of this whole project is a much lower end machine, running Smoothwall Express 3. I have not tried the import there yet.
 
Old 09-11-2009, 07:30 AM   #5
rayfordj
Member
 
Registered: Feb 2008
Location: Texas
Distribution: Fedora, RHEL, CentOS
Posts: 475

Rep: Reputation: 73
hmmm, odd. I wrote a loop that created a single rule for IPs x.y.1-254.1-254 on a test box that made it from x.y.1-129.z without issue when I last checked. It is a RHEL5.x install, single cpu, maybe 512MB RAM... 128*254 would put last count above 32512 rules. I will check in later and update.

Have you tried creating 2 or 3 scripts out of your rules and running them individually?
 
Old 09-11-2009, 08:41 AM   #6
rayfordj
Member
 
Registered: Feb 2008
Location: Texas
Distribution: Fedora, RHEL, CentOS
Posts: 475

Rep: Reputation: 73
Looks like I hit a snag. Not able to create a rule beyond rule 55399 on the filter table. I can still write to the nat table though... I just wrote a single rule for each IP x.y.1-32.1-254 as a test without any problems on nat PREROUTING chain while filter chains will not accept any more. I guess it is possible that you are hitting a similar ceiling, but sooner...

Hope this helps.


Any way to consolidate some of the rules?


 
Old 09-11-2009, 09:13 AM   #7
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by rayfordj View Post

Any way to consolidate some of the rules?


I'm looking into that now.
The original list, has every singe entry listed as a range. Even if it's one IP. I think that at the very least, i should be able to hack out the non-range ranges, and replace them with single entry rules. I may also be able to take the ranges, and convert them from a range format, to an ip/subnet format. Say, change 1.2.3.0-1.2.3.255 to 1.2.3.0/24

If i change that around, i should be able to add the rules without the -m iprange option. Which may (or... may not) help with resources.
 
Old 09-11-2009, 10:06 AM   #8
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Original Poster
Rep: Reputation: 15
working on my importer a bit, and i realized something.
I was off by a bit on my list length. A quick look at the number of lines, made me think that i was looking at 22K lines. I looked more closely today, and it's actually... 226K lines.

I have, however, made some improvements, and i'm no longer using the iprange module. I'm testing the import now, we'll see what happens.
 
Old 09-11-2009, 12:34 PM   #9
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Original Poster
Rep: Reputation: 15
tried breaking the adds up into multiple lists, it was no help. is it possible to increase the available memory to iptables?
 
Old 09-11-2009, 03:55 PM   #10
Gangrif
Member
 
Registered: Feb 2004
Distribution: Fedora 19
Posts: 64

Original Poster
Rep: Reputation: 15
it was all in my method.
i seem to have gotten around the issue.
It wasnt iptables causing the problem, rather, bash.

I was adding all of these rules via a bash script.

I chagned my importer to output an iptables-restore formatted file, and now it imports no sweat.

Thanks for the input!
 
Old 09-11-2009, 04:30 PM   #11
rayfordj
Member
 
Registered: Feb 2008
Location: Texas
Distribution: Fedora, RHEL, CentOS
Posts: 475

Rep: Reputation: 73
Thanks for the followup!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Iptables - Chain sequence Madhushanka Linux - Security 2 12-19-2008 11:03 PM
iptables good packet chain (instead of bad packet chain) win32sux Linux - Security 6 11-06-2008 07:02 AM
Which is the chain name in iptables dkn4a1 Linux - Software 4 09-22-2008 06:23 AM
userdefined chain in iptables yawe_frek Red Hat 2 11-28-2006 07:20 AM
iptables chain modification gizza23 Linux - Networking 2 07-10-2005 06:45 AM


All times are GMT -5. The time now is 05:01 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration