piped grep in loop returns empty file

TPinNJ · 10-12-2011, 12:17 PM

I'm a relative newbie to Linux and have a script question I'm sure is easy once its solved.. so I have a huge file (lets call it file1) that has the event log for a print server (6 months of data) I have another file (file2) that's a straight list of all the printers on the server. I figured a shell script in linux would be easier than using Windows. I tried diff and compare but they returned way too much data to be usable (all I need to know is which printers DON'T show up in the print log)

so I thought 'well, if I grep the file and pipe it.. all the files with 0k will be the ones I want' so I threw this together

Code:

#!/bin/sh
cat file2 | while read printers
do 
echo $printers
grep $printers file1 > $printers
done

It will echo it correctly but the grep returns all empty files.. I'm still learning about shell scripting, so I'm a little confused. a gentle nudge in the right direction would be appreciated.

I tried grep with single quotes and double quotes and got the same results..

millgates · 10-12-2011, 12:53 PM

Does file1 contain the printer names you are grepping for? Can you post an example of the printer name from file1 and the line in file2 containing the same name?

TPinNJ · 10-12-2011, 01:02 PM

File2 contains the list of queue names
File1 contains a dump of the printer log from a print server

File1 contains thousands of lines of this:
Document 480, 2011 Timesheet Workbook - Nething -- Merck - 30SEP2011.xls owned by nethingb was printed on SPRING_HPLJ2200_DSS-37 via port SPRING_HPLJ2200_DSS-37. Size in bytes: 179760; pages printed: 1

File2 contains this:
SPRING_HPLJ2200_DSS-37

what I'm looking for are queue names that don't match.

millgates · 10-12-2011, 01:13 PM

I just tried -- I created file1 and file2 one line each with the content you posted and ran your script in the same directory. After that, the directory contains file "SPRING_HPLJ2200_DSS-37" and the file contains the line from file1. Isn't that the desired result?

TPinNJ · 10-12-2011, 01:20 PM

That works.. is there a max file size issue? The printer log is several hundred megs of text

I tried with 10 printers and it worked too.. thanks.. I'll investigate

rknichols · 10-12-2011, 01:26 PM

By any chance was file2 made on a Windows system and thus has the DOS CR-NL end of line sequence? That carriage return character will be read into the variable and cause the match to fail. If so, you can use 'unix2dos' to convert the file or else

Code:

tr -d '\r' <file2 | while read printers
do
...

TPinNJ · 10-12-2011, 01:38 PM

yep.. it came from a windows server.. I copied and pasted the text into VI and it appeared to work.. thanks for your help.. it was driving me crazy!!

David the H. · 10-13-2011, 06:57 AM

Just to add a few scripting tips for you:

1.

Code:

#!/bin/sh

/bin/sh forces your shell to interpret the script in posix-compliant mode. This means any advanced shell features will be unavailable. Since you're using Linux, and pretty much all distributions have bash installed, then you really should use /bin/bash instead. Only use sh if you need portability to systems without bash installed.

2.

Code:

cat file2 | while read printers
do 
(commands)
done

Useless Use Of Cat! You can feed the loop with a simple file redirection instead. More complex commands can be handled with process substitution (which is one of those bash-only features I mentioned).

Code:

while read printers ; do 
	(commands)
done <file


while read printers ; do 
	(commands)
done < <( tr -d '\r' <file )

This also avoids the common pitfall of the loop running in a subshell.

http://mywiki.wooledge.org/BashFAQ/024

3.

Code:

echo $printers
grep $printers file1 > $printers

Always double-quote variable expansions, unless you want word-splitting to occur. It's vital in scripting to understand how the shell handles arguments and whitespace, so carefully read these three links:

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Code:

echo "$printers"
grep "$printers" file1 > "$printers"

When in doubt, quote!

4.
Finally, think a bit about the formatting of your script. Good, consistent use of indenting and whitespace helps make your code easier to read and to debug.

http://mywiki.wooledge.org/BashGuide...es#Readability

I also personally recommend putting your do and then keywords on the same line as the start of the loop. It helps keep down clutter and makes seeing the where the loop starts and ends a bit clearer (when combined with proper indentation, of course). See my while-loop example above.