LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-12-2011, 01:17 PM   #1
TPinNJ
LQ Newbie
 
Registered: Oct 2011
Posts: 4

Rep: Reputation: Disabled
piped grep in loop returns empty file


I'm a relative newbie to Linux and have a script question I'm sure is easy once its solved.. so I have a huge file (lets call it file1) that has the event log for a print server (6 months of data) I have another file (file2) that's a straight list of all the printers on the server. I figured a shell script in linux would be easier than using Windows. I tried diff and compare but they returned way too much data to be usable (all I need to know is which printers DON'T show up in the print log)

so I thought 'well, if I grep the file and pipe it.. all the files with 0k will be the ones I want' so I threw this together

Code:
#!/bin/sh
cat file2 | while read printers
do 
echo $printers
grep $printers file1 > $printers
done
It will echo it correctly but the grep returns all empty files.. I'm still learning about shell scripting, so I'm a little confused. a gentle nudge in the right direction would be appreciated.

I tried grep with single quotes and double quotes and got the same results..
 
Old 10-12-2011, 01:53 PM   #2
millgates
Member
 
Registered: Feb 2009
Location: 192.168.x.x
Distribution: Slackware
Posts: 840

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Does file1 contain the printer names you are grepping for? Can you post an example of the printer name from file1 and the line in file2 containing the same name?
 
Old 10-12-2011, 02:02 PM   #3
TPinNJ
LQ Newbie
 
Registered: Oct 2011
Posts: 4

Original Poster
Rep: Reputation: Disabled
File2 contains the list of queue names
File1 contains a dump of the printer log from a print server

File1 contains thousands of lines of this:
Document 480, 2011 Timesheet Workbook - Nething -- Merck - 30SEP2011.xls owned by nethingb was printed on SPRING_HPLJ2200_DSS-37 via port SPRING_HPLJ2200_DSS-37. Size in bytes: 179760; pages printed: 1

File2 contains this:
SPRING_HPLJ2200_DSS-37

what I'm looking for are queue names that don't match.
 
Old 10-12-2011, 02:13 PM   #4
millgates
Member
 
Registered: Feb 2009
Location: 192.168.x.x
Distribution: Slackware
Posts: 840

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
I just tried -- I created file1 and file2 one line each with the content you posted and ran your script in the same directory. After that, the directory contains file "SPRING_HPLJ2200_DSS-37" and the file contains the line from file1. Isn't that the desired result?
 
Old 10-12-2011, 02:20 PM   #5
TPinNJ
LQ Newbie
 
Registered: Oct 2011
Posts: 4

Original Poster
Rep: Reputation: Disabled
That works.. is there a max file size issue? The printer log is several hundred megs of text

I tried with 10 printers and it worked too.. thanks.. I'll investigate
 
Old 10-12-2011, 02:26 PM   #6
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 2,961

Rep: Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268
By any chance was file2 made on a Windows system and thus has the DOS CR-NL end of line sequence? That carriage return character will be read into the variable and cause the match to fail. If so, you can use 'unix2dos' to convert the file or else
Code:
tr -d '\r' <file2 | while read printers
do
...
 
Old 10-12-2011, 02:38 PM   #7
TPinNJ
LQ Newbie
 
Registered: Oct 2011
Posts: 4

Original Poster
Rep: Reputation: Disabled
yep.. it came from a windows server.. I copied and pasted the text into VI and it appeared to work.. thanks for your help.. it was driving me crazy!!
 
Old 10-13-2011, 07:57 AM   #8
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Just to add a few scripting tips for you:

1.
Code:
#!/bin/sh
/bin/sh forces your shell to interpret the script in posix-compliant mode. This means any advanced shell features will be unavailable. Since you're using Linux, and pretty much all distributions have bash installed, then you really should use /bin/bash instead. Only use sh if you need portability to systems without bash installed.

2.
Code:
cat file2 | while read printers
do 
(commands)
done
Useless Use Of Cat! You can feed the loop with a simple file redirection instead. More complex commands can be handled with process substitution (which is one of those bash-only features I mentioned).

Code:
while read printers ; do 
	(commands)
done <file


while read printers ; do 
	(commands)
done < <( tr -d '\r' <file )
This also avoids the common pitfall of the loop running in a subshell.

http://mywiki.wooledge.org/BashFAQ/024

3.
Code:
echo $printers
grep $printers file1 > $printers
Always double-quote variable expansions, unless you want word-splitting to occur. It's vital in scripting to understand how the shell handles arguments and whitespace, so carefully read these three links:

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Code:
echo "$printers"
grep "$printers" file1 > "$printers"
When in doubt, quote!

4.
Finally, think a bit about the formatting of your script. Good, consistent use of indenting and whitespace helps make your code easier to read and to debug.

http://mywiki.wooledge.org/BashGuide...es#Readability

I also personally recommend putting your do and then keywords on the same line as the start of the loop. It helps keep down clutter and makes seeing the where the loop starts and ends a bit clearer (when combined with proper indentation, of course). See my while-loop example above.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] [BASH] non-empty variable before loop end, is empty after exiting loop aitor Programming 2 08-26-2010 10:57 AM
How to use If Else condition when returns empty data? dwarf007 Programming 2 04-27-2010 04:00 AM
Grep in bash script returns "No such file or directory", works manually gizza23 Programming 7 02-25-2010 05:37 PM
bash: piped loop in same shell Vit77 Programming 2 04-30-2009 03:50 AM
anything piped to grep returns "(standard input)"... why? allohakdan Linux - Software 8 09-28-2008 02:02 PM


All times are GMT -5. The time now is 10:47 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration