when i use this command it takes 2 second to find right file (i have 30 files under this directory)
I want fast search by using any command per your recommendation. I use "find" and "grep" so far and it's not fast enough because php function use grep to identify the word on result page. Is there any blazing fast system where i can use any command or software i can install anything on my server as long as it works fast i also tried ramdisk but no difference any suggestion from linux gurus highly appriciated.

(need to be exact match only -Rc works in 1 sec however doesn't come with exact match "--word regexp" catch exact match but it takes 2-3 second to result i been trying to solve this problem for 3 days


grep -Rc --word-regexp 'departmento'

/language/latest/swe.txt:0
/language/latest/ltz.txt:0
/language/latest/liv.txt:0
/language/latest/bos.txt:0
/language/latest/spa.txt:1
/language/latest/azj.txt:0
/language/latest/afr.txt:0
/language/latest/als.txt:0
/language/latest/cym.txt:0
/language/latest/oci.txt:0



grep -Rc --word-regexp 'department'

/language/latest/swe.txt:0
/language/latest/ltz.txt:0
/language/latest/liv.txt:0
/language/latest/bos.txt:0
/language/latest/spa.txt:0
/language/latest/azj.txt:0
/language/latest/afr.txt:0
/language/latest/als.txt:0
/language/latest/cym.txt:0
/language/latest/oci.txt:0

Sounds like need some indexing, have you had a look at Lucene or Glimpse?

hi thank you for your answer no i haven't install anything on my centos can i use yum to install lucene ? i am using nginx not apache btw

i found -P now little bit fast maybe not 4sec but 2sec

is there any way i can pipe 8 instance like this example any better if i do ?


1. Find the necessary files, pipe them to xargs and tell it to execute 8 instances.

time find ./ -name "*.ext" -print0 | xargs -0 -n1 -P8 grep -H "string" >> Strings_find8

real 3m24.358s
user 1m27.654s
sys 9m40.316s



can i use this way ?



time find ./ -name "*.ext" -print0 | xargs -0 -n1 -P8 grep -PRc --word-regexp "Aneurysmavuodon" /language/latest
>> Strings_find8

i found -P very fast but still far from my expectation

[root@server1 latest]# time find ./ -name "*.ext" -print0 | grep -PRc --word-regexp 'direzione' /language/latest
/language/latest/swe.txt:0
/language/latest/ltz.txt:0
/language/latest/lit.txt:0
/language/latest/bos.txt:0
/language/latest/lav.txt:0
/language/latest/som.txt:0
/language/latest/deu.txt:0
/language/latest/spa.txt:0
/language/latest/azj.txt:0
/language/latest/ita.txt:1

real 0m0.225s
user 0m0.180s
sys 0m0.044s



some of them suggest me to make 8 instance pipe by using args but not good so far


[root@server1 latest]# time find ./ -name "*.ext" -print0 | xargs -0 -n1 -P8 grep -PRc --word-regexp 'direzione' /language/latest
/language/latest/swe.txt:0
/language/latest/ltz.txt:0
/language/latest/lit.txt:0
/language/latest/bos.txt:0
/language/latest/lav.txt:0
/language/latest/som.txt:0
/language/latest/deu.txt:0
/language/latest/ita.txt:1
real 0m0.227s
user 0m0.186s
sys 0m0.040s

Why usewhen you canand I agree indexing probably makes more sense if you query often and a lot. Either find something that works with Nginx or find something that offers an independent abstraction layer.

I won't guarantee this will work in all cases, but none of the examples so far perform a parallel search - and serial file searching always takes longer:

--- note, not syntax checked...

This should provide the same list, though the order of lines will be interleaved.

The goal here is to do all the searching in parallel (hence the grep ... &). This should work as long as you don't have many thousands of files to search (you will use up your process limit quota).

The system time should increase due to context switches, but the elapsed time should decrease even more.

You are looking for a plain string, so you can use -F (it only handles plain strings so it's probably the fastest mode; although it seems like IO could be the limiting factor here, in which case there won't be any measurable improvement).

Also, from your output it appears that you never have more than 1 match per file, maybe you could use -l (--files-with-matches) instead of -c (--count)?

I think the xargs -n1 -P8 one is parallel. It might be better to try with a bit less parallelism, xargs -n5 -P2.