LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   can't grep file (https://www.linuxquestions.org/questions/linux-software-2/cant-grep-file-611674/)

ust 01-07-2008 03:08 AM

can't grep file
 
when I use grep , it pop the below message ,

"bash: /bin/grep: Argument list too long"

I know this is because there are too many files in directory, except use the command "find" , how to fix it ? is it because the memory is not enough to list all files ? if yes , how to change the parameter to increase the memory ? thx

colucix 01-07-2008 04:13 AM

The "argument list too long" error is due to a kernel limitation relative to the number of characters that can be processed in the arguments list. I don't know if this limitation can be tweaked without recompiling the kernel, nor if it would be safe to do this. Anyway, you can also process one file at a time by the method you mentioned or by a simple for loop, e.g.
Code:

for file in *.txt ; do grep "pattern" $file ; done
if some filename contains blank spaces, you have to set the IFS (input field separator) in order to avoid names splitted in two or more parts. A similar approach applies when you use find with the -print0 option to prevent this behavior.

ust 01-07-2008 04:29 AM

Quote:

Originally Posted by colucix (Post 3014140)
The "argument list too long" error is due to a kernel limitation relative to the number of characters that can be processed in the arguments list. I don't know if this limitation can be tweaked without recompiling the kernel, nor if it would be safe to do this. Anyway, you can also process one file at a time by the method you mentioned or by a simple for loop, e.g.
Code:

for file in *.txt ; do grep "pattern" $file ; done
if some filename contains blank spaces, you have to set the IFS (input field separator) in order to avoid names splitted in two or more parts. A similar approach applies when you use find with the -print0 option to prevent this behavior.


thx reply , use "find" can solve my problem , however , other users have the same problem , I don't want to change the command they are used to apply , on the other hand , I hv many scripts need to change if use find , can advise other solution ? thx

colucix 01-07-2008 06:03 AM

I'm afraid there is not any other solution. The limit is set as a kernel parameter (ARG_MAX) and in linux the only way to increase this value is to recompile the kernel itself. But not advisable, since it is related to memory allocation during the execution of commands.

You can retrieve this value for your system by issuing
Code:

getconf ARG_MAX
or looking at /usr/include/linux/limits.h

You can find a good explanation of the ARG_MAX limit in the Coreutils FAQ, here and an even more detailed one, here.


All times are GMT -5. The time now is 04:30 AM.