grep -i is EXTREMELY slow!

CamTheSaxMan · 02-07-2014, 11:10 AM

I'm trying to search for a case-insensitive string in a directory, but grep -i is either incredibly slow or hangs. I tried to search for a simple string "document" in my home folder, but the search was still going after a whole HOUR! so I just gave up. Why is it so slow? Are there any workarounds?

Vrajgh · 02-07-2014, 11:37 AM

Can you provide the complete command used please?

If you were (for example) to use

Code:

grep -i foo

then grep would read from stdin (keyboard) rather than the directory and will never finish until interrupted. To search the directory you need to give it a list of files to search, eg

Code:

grep -i foo *

CamTheSaxMan · 02-07-2014, 12:37 PM

Yes, that's what I did. I typed:

Code:

grep -i -r documents *

and it just sits there. I did a Google search and found that this is a common issue. fgrep actually worked for me, though I don't really know what the difference between fgrep and grep is.

Tux1000 · 02-07-2014, 03:49 PM

If you tell grep to search a string in a file in a directory with or without subdirectories and there are many, many files, it will take some time of course and it may appeared to be hung but maybe not.

I did a test run on one of my directories where it has subdirectories and many files. For example:

Code:

time grep -ir -l "How do I backup a LUKS header" backup/*
backup/docs-and-media/linux-related/cryptsetuo faq

real    2m18.305s
user    2m17.685s

I use the time command to tell me how long it takes to the complete the search. It took 2m and 18 secs. The -l option tells grep to list the file the string was found in. BTW, the time command is not necessary for doing grep searches, it's used here just to demonstrate that grep can take some time on searching many files. Anyway, it found the string in backup/docs-and-media/linux-related/cryptsetuo faq

There is a another command called ack. It's like grep on steroids and very fast.

Code:

time ack -ai -l "How do I backup a LUKS header" backup/*
backup/docs-and-media/linux-related/cryptsetuo faq

real    0m19.791s
user    0m19.265s
sys     0m0.489s

Here is the same search on the backup directory, but the search is done faster by using ack, for a total of 19secs to complete.

Give ack a try.

e5150 · 02-08-2014, 02:02 PM

There are a few things you can do which might speed up greping. If you're only interested in text files that matches your search word, use the -I switch (according to man page: "-I Process a binary file as if it did not contain matching data"). Also, case insensitive searches are particularly slow when using a UTF-8 locale. If your search words only contains ascii-compatible characters (A-Z, a-z, 0-9, (),./-#! and such) try: `LC_CTYPE=C grep -i -r documents *`.

CamTheSaxMan · 02-08-2014, 05:54 PM

Odd thing is that grep -i is not slow on Linux Mint. I do have a higher-spec computer on Mint, so that may be the cause.