LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Need help writing for loop shell script (https://www.linuxquestions.org/questions/linux-newbie-8/need-help-writing-for-loop-shell-script-821689/)

hsp40oz 07-23-2010 08:20 AM

Need help writing for loop shell script
 
i am trying to write a script (which I havent done since college) that will grep each line from a file named Backlist which contains the titles of about 850 ebooks against another file named HQNlist which contains about 5500 lines which are book titles plus their corresponding ISBN codes. I want to be able to find those 850 matching titles and write them to a new file because I need to make a list of those ISBNs.

I wrote this script but it just writes the entire HQNlist many times to the new file:

Code:

#!/bin/bash
for line in `cat Backlist`
do
    grep -i $line HQNlist >> results.omfg
done

Any ideas?

Thanks in advance

GrapefruiTgirl 07-23-2010 08:26 AM

Code:

cat Backlist | while read line; do
  grep '$line' HQNlist >> results.omfg
done

That should basically do it; if the stuff in the first file (which ends up in the $line variable) contains anything other than "normal" characters like letters & numbers, we may need to enhance the grep a little, but this is the basic idea.

pixellany 07-23-2010 08:30 AM

I put your code into [CODE] tags to make it easier to read.

Sasha suggested the same approach that I was going to. To see why the original approach was not working, you could try inserting an echo statement to see the value of "line" each time thru the loop.

hsp40oz 07-23-2010 08:45 AM

I tried Sasha's approach. The results were just a blank file. Some of the lines have punctuation.
Is there a way to ignore special characters or should I try to remove all the punctuation from these files?

GrapefruiTgirl 07-23-2010 08:47 AM

How about show us a snippet of what's inside each file; that'd be the best way we can ensure that the grepping will work when it encounters odd characters.

schneidz 07-23-2010 08:47 AM

whats the content of Backlist ?

could be there are spaces in each line.

maybe you need to do something like: sed -n "$i"p instead.

hsp40oz 07-23-2010 09:00 AM

Quote:

Originally Posted by schneidz (Post 4043039)
whats the content of Backlist ?

could be there are spaces in each line.

maybe you need to do something like: sed -n "$i"p instead.

A few lines from Backlist:
You've Got Male
The Darkest Facts: A Lords of the Underworld Companion
The Darkest Fire
The Amazon's Curse

From HQNlist:
Back on Blossom Street 9781426814686
Last Known Victim 9781426814693
The Healer 9781426814709
Rogue 9781426814716

does that sed command remove the spaces?

ghostdog74 07-23-2010 09:13 AM

use grep's -f option
Code:

grep -f backlist hnlist

ghostdog74 07-23-2010 09:13 AM

*delete*

GrapefruiTgirl 07-23-2010 09:15 AM

Code:

echo "$(cat Backlist)" | while read line; do
  grep -e "${line}" HQNlist >> results.omfg
done

Using the data you gave, I've adjusted it and find that the above now works for me. I suspect a problem with newlines and/or spaces, particular spaces, were causing troubles before.

See if that works for you?

SORRY - I initially left out the quotes around the `echo $(..)` which should be there -- please note that and correct..

GrapefruiTgirl 07-23-2010 09:18 AM

Quote:

Originally Posted by ghostdog74 (Post 4043067)
use grep's -f option
Code:

grep -f backlist hnlist

ghostdog, I tried this and it didn't work. Any clue why? It'd be great if it did -- nice and simple!

colucix 07-23-2010 09:22 AM

Quote:

Originally Posted by GrapefruiTgirl (Post 4043070)
Using the data you gave, I've adjusted it and find that the above now works for me. I suspect a problem with newlines and/or spaces, particular spaces, were causing troubles before.

Sasha, your previous code (post #2) could work, but you inadvertently embedded $line in single quotes, preventing shell expansion. ghostdog's solution works for me. Cheers!

GrapefruiTgirl 07-23-2010 09:25 AM

This is weird colucix :scratch:

ghost*'s -f idea does not work for me, and neither does my initial suggestion which as you suggest should work with double-quotes (I changed the quotes and still no go).

hsp40oz 07-23-2010 09:30 AM

Quote:

Originally Posted by GrapefruiTgirl (Post 4043070)
Code:

echo "$(cat Backlist)" | while read line; do
  grep -e "${line}" HQNlist >> results.omfg
done

Using the data you gave, I've adjusted it and find that the above now works for me. I suspect a problem with newlines and/or spaces, particular spaces, were causing troubles before.

See if that works for you?

SORRY - I initially left out the quotes around the `echo $(..)` which should be there -- please note that and correct..

this returned 5795 lines which is kind of strange being its a few hundred more than contained in the HQNlist

I wish grep -f worked, it was the first thing i tried last night

colucix 07-23-2010 09:31 AM

That's really weird. :scratch:

Here is my sample session:
Code:

$ Backlist
You've Got Male
The Darkest Facts: A Lords of the Underworld Companion
The Darkest Fire
The Amazon's Curse

$ cat HQNlist
Back on Blossom Street 9781426814686
The Amazon's Curse 9781426816753
Last Known Victim 9781426814693
The Healer 9781426814709
Rogue 9781426814716
The Darkest Facts: A Lords of the Underworld Companion 9781426814728

$ grep -f Backlist HQNlist
The Amazon's Curse 9781426816753
The Darkest Facts: A Lords of the Underworld Companion 9781426814728

and
Code:

$ cat test.sh
#!/bin/bash
cat Backlist | while read line; do
  grep "$line" HQNlist
done

$ ./test.sh
The Darkest Facts: A Lords of the Underworld Companion 9781426814728
The Amazon's Curse 9781426816753

:scratch:


All times are GMT -5. The time now is 01:34 AM.