LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 01-07-2008, 03:08 AM   #1
ust
Senior Member
 
Registered: Mar 2003
Location: fasdf
Distribution: Debian / Suse /RHEL
Posts: 1,129

Rep: Reputation: 30
can't grep file


when I use grep , it pop the below message ,

"bash: /bin/grep: Argument list too long"

I know this is because there are too many files in directory, except use the command "find" , how to fix it ? is it because the memory is not enough to list all files ? if yes , how to change the parameter to increase the memory ? thx
 
Old 01-07-2008, 04:13 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,506

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
The "argument list too long" error is due to a kernel limitation relative to the number of characters that can be processed in the arguments list. I don't know if this limitation can be tweaked without recompiling the kernel, nor if it would be safe to do this. Anyway, you can also process one file at a time by the method you mentioned or by a simple for loop, e.g.
Code:
 for file in *.txt ; do grep "pattern" $file ; done
if some filename contains blank spaces, you have to set the IFS (input field separator) in order to avoid names splitted in two or more parts. A similar approach applies when you use find with the -print0 option to prevent this behavior.
 
Old 01-07-2008, 04:29 AM   #3
ust
Senior Member
 
Registered: Mar 2003
Location: fasdf
Distribution: Debian / Suse /RHEL
Posts: 1,129

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by colucix View Post
The "argument list too long" error is due to a kernel limitation relative to the number of characters that can be processed in the arguments list. I don't know if this limitation can be tweaked without recompiling the kernel, nor if it would be safe to do this. Anyway, you can also process one file at a time by the method you mentioned or by a simple for loop, e.g.
Code:
 for file in *.txt ; do grep "pattern" $file ; done
if some filename contains blank spaces, you have to set the IFS (input field separator) in order to avoid names splitted in two or more parts. A similar approach applies when you use find with the -print0 option to prevent this behavior.

thx reply , use "find" can solve my problem , however , other users have the same problem , I don't want to change the command they are used to apply , on the other hand , I hv many scripts need to change if use find , can advise other solution ? thx
 
Old 01-07-2008, 06:03 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,506

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
I'm afraid there is not any other solution. The limit is set as a kernel parameter (ARG_MAX) and in linux the only way to increase this value is to recompile the kernel itself. But not advisable, since it is related to memory allocation during the execution of commands.

You can retrieve this value for your system by issuing
Code:
getconf ARG_MAX
or looking at /usr/include/linux/limits.h

You can find a good explanation of the ARG_MAX limit in the Coreutils FAQ, here and an even more detailed one, here.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to grep all users's certain file packets Linux - Newbie 5 01-03-2008 08:35 PM
How can I grep text from file? @ngelot Linux - Software 6 06-13-2007 04:44 AM
grep output on stdout and grep output to file don't match xnomad Linux - General 3 01-13-2007 04:56 AM
grep throughout the whole file system....? vous Linux - Software 4 03-20-2005 01:30 PM
grep file in the subdirectory juno Linux - General 3 09-30-2002 11:08 AM


All times are GMT -5. The time now is 10:14 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration