LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 06-15-2012, 09:43 AM   #1
masuch
Member
 
Registered: Sep 2011
Location: /dev/null
Distribution: ubuntu 64bits
Posts: 129

Rep: Reputation: Disabled
Question how to set memory limit for egrep command


Hi,

I have a problem with egrep command which is running within some application which I did not write.

Even if I have 32 Gbytes ram it is apparently not enough for egrep - because he wants more - about 60Gbytes virtual (-> so it is swapping -> computer is unusable -> no reaction on keyboard or extremely slow ...)

Is it possible to setup max memory limit consumption for egrep
(bonus: or any other command running in terminal - not for GUI applications)

thank you very much for any kind of solution/s. I am willing to try anything.
Kind Regards,
Martin
 
Old 06-15-2012, 02:48 PM   #2
Skaperen
Senior Member
 
Registered: May 2009
Location: WV, USA
Distribution: Slackware, CentOS, Ubuntu, Fedora, Timesys, Linux From Scratch
Posts: 1,777
Blog Entries: 20

Rep: Reputation: 115Reputation: 115
In bash there is the "ulimit -v kkkkkk" command. Execute it in a subshell to affect one command:

( ulimit -v 4194304 && egrep ... )

This example sets the virtual memory limit to 4GB.
 
1 members found this post helpful.
Old 06-16-2012, 01:14 PM   #3
masuch
Member
 
Registered: Sep 2011
Location: /dev/null
Distribution: ubuntu 64bits
Posts: 129

Original Poster
Rep: Reputation: Disabled
Question

Quote:
Originally Posted by Skaperen View Post
In bash there is the "ulimit -v kkkkkk" command. Execute it in a subshell to affect one command:

( ulimit -v 4194304 && egrep ... )

This example sets the virtual memory limit to 4GB.

-Thanks a lot for this very useful hint - going to implement it in many scripts running from crontab.
-(Concerning this problem I have already created crontab script which runs each 60 seconds and kill process which consume more than 30 Gbytes ram)
- One thing - is it possible to setup such limit if I do not have possibility to execute it by subshell.
(Would it be possible to use /etc/security/limits.conf in someway for this purpose or something else ?)

thank you,
regards,
Martin
 
Old 06-16-2012, 02:53 PM   #4
masuch
Member
 
Registered: Sep 2011
Location: /dev/null
Distribution: ubuntu 64bits
Posts: 129

Original Poster
Rep: Reputation: Disabled
Hi,

I have changed ulimit -v (temporarily in terminal) to:
virtual memory (kbytes, -v) 1000

and commands:

$ sudo locate aaaa
Segmentation fault
$ free -m
Segmentation fault
ls
bash: /usr/local/bin/ls: Argument list too long

pretty ugly - what is happening ?
 
Old 06-18-2012, 08:41 PM   #5
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,261

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Pretty much what it says ... 1MB (approx) is not enough for those cmds to run with those params ...
 
Old 06-19-2012, 04:16 AM   #6
masuch
Member
 
Registered: Sep 2011
Location: /dev/null
Distribution: ubuntu 64bits
Posts: 129

Original Poster
Rep: Reputation: Disabled
(
:-) yes indeed. (could not imagine why for some simple command/s is not enough 50000*1024 ?
- where is the time I had to be happy with 64 kbytes for code segment , stack segment , data segment and extra segment all together :-)
)

(I have just moved all my activities to my cloned OS raid0 (4 partitions) on ssd revodrive - it is really extremely fast.).
1. So I can see it helped to decrease latency within this stuff when it suddenly happened.
2. I have daemon script which is killing processes with extremely big memory consumption.
3. changed my swapfile to 1 Gigabyte and location to ssd revodrive and swappiness=1.
4. more playing with ulimit for my daemons - especially for onlive rsyncing/cloning my OS.
5. setup tmpfs 7G /tmp to memory
hope it helped in future if it is happened again.

? I still do not know what application is starting this egrep - extremely memory consumption command , top and htop shows just egrep ?
do not know how to find it.

going to mark it as solved.

thank you for your help.

Does exist something like ulimit for swap consumption per process ? (ulimit does not have it if i did not overlook something)

kind regards,
M.
 
Old 06-19-2012, 07:46 PM   #7
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,261

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
As a short term thing, lookup ulimit options here http://linux.die.net/man/1/bash & also /etc/security/limits.conf.
In the long term, track down the cause; start with top and/or ps and check the owner and the PID, PPID values.
See http://linux.die.net/man/1/pstree
 
  


Reply

Tags
egrep, limit, memory


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
why different signals send to programs that exceeded memory limit set by setrlimit()? lonelycorn Programming 2 09-02-2010 09:03 PM
How to Limit suphp user memory limit? Rundi Linux - Server 2 05-14-2010 12:27 PM
Help with advanced tar, ls, and egrep command... jademan83 Linux - Newbie 1 03-05-2009 12:28 PM
how to set memory cache limit for nfs server libin88 Linux - Server 1 10-24-2008 08:23 AM
egrep to find a command not found string romainp Linux - General 1 09-07-2007 03:30 PM


All times are GMT -5. The time now is 09:44 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration