grep: memory exhausted
 

Command used,

grep -v -f fileToExclude.txt ContentFileList.txt > ContentFileList_out.txt

Error thrown,

grep: memory exhausted

the files has more than o.1 million lines and more over this command consumes 100% of CPU and 60-70% of memory.

and,

grep --version
GNU grep 2.6.3

rpm -qa | grep grep
grep-2.6.3-3.el6.i686

I need the solution for 'memory exhausted' and 'CPU and memory conception'.

try to sort both input files and you can implement a simple filter in perl or awk or ...

As referenced above - sort the files. Once sorted you can filter them using comm - which allows you to identify common/differences between the two files and output what you want. Refer to the manpage on comm.

Spit the files is also useful to achieve this but time consuming...

The only times I've run across the "memory exhausted" message has been when scanning a file of data extracted from a Microsoft Office file... which had one line per paragraph, with VERY long paragraphs.

But I was not using a file to hold the patterns... How many patterns are in that file (fileToExclude.txt)?