How to read string in 1 file and search in another file?

ameyapandit · 07-11-2008, 05:11 AM

dear experts,

I am newbie to shell scripting and need your help writing a script.

Scenario:
I have 2 files with me.
1. source.txt : contains 10 digit number on each line
2. destination.txt : contains 10 digit number following allphanumeric
string including special characters.

What i want to do is:
Read source.txt file line by line.. and search each string (number) in destination.txt file. If the string is found in destination.txt I want to delete that entire line from this file.
and keep doing this untill end of the source file.

The string from source file may occur multiple times in destination file.

Kindly suggest.. I am lost.

ghostdog74 · 07-11-2008, 06:02 AM

if you have GNU grep, you can try the -f switch.

aldomenico · 07-11-2008, 07:08 AM

Try this:

while read string
do
# the next line searches and deletes lines that have the string
sed "/${string}/d" destination.txt > dest.tmp

# the next line moves the temporary edited file
# in place for the next iteration
mv dest.tmp destination.txt
done < source.txt

Assuming source.txt contains a list of strings, one per line

ktgajowniczek · 07-11-2008, 07:11 AM

Quote:

Originally Posted by ghostdog74

if you have GNU grep, you can try the -f switch.

that is good, but if the alphanumeric part contains numbers (since it can also be "numeric") than you could remove too much from the DESTINATION file

this, maybe not the best, solution will not:

Code:

for SEED in `cat source.txt`; do 
  grep -vE "^$SEED" destination.txt > tmp_destination.txt
  mv tmp_destination.txt destination.txt
done

rm -f tmp_destination.txt

ghostdog74 · 07-11-2008, 07:15 AM

@OP post some samples.

aldomenico · 07-11-2008, 08:22 AM

Good point Karol.

Here's a revised one to be absolutely sure:

while read string
do
sed "/^${string}/d" destination.txt > dest.tmp
mv dest.tmp destination.txt
done < source.txt

ameyapandit · 07-11-2008, 09:14 AM

Quote:

Originally Posted by aldomenico

Try this:

while read string
do
# the next line searches and deletes lines that have the string
sed "/${string}/d" destination.txt > dest.tmp

# the next line moves the temporary edited file
# in place for the next iteration
mv dest.tmp destination.txt
done < source.txt

Assuming source.txt contains a list of strings, one per line

thnx buddy,

I ran this script removing the "mv dest.tmp destination.txt"
of it so that the output file will remain as dest.tmp.
But the dest.tmp file is similar as destination.txt and no strings
has been removed from this file.

I tried below solutions as well.. but nothing works.
sol1. grep -vf source.txt destination.txt >> dest.tmp

sol2. sort -o source.txt source.txt
sort -o destination.txt destination.txt

comm -13 source.txt destination.txt >> dest.tmp

rikijpn · 07-11-2008, 12:05 PM

I think I've got what you need.

Try this

Code:

#!/bin/bash

for FROM_SRCFILE in $(cat source.txt); do
  TEMP1="sed -i '/"$FROM_SRCFILE"/ d' destination.txt"
  eval $TEMP1
done

This will work (it did on my computer?).

You may had have the same problem I had when I ran

Code:

#!/bin/bash
# doesn't work
for FROM_SRCFILE in $(cat source.txt); do
  sed -i '/"$FROM_SRCFILE"/ d' destination.txt
done

in which I case the file didn't change (if anyone knows why I would actually like to know...).

Well, the above script as you see, takes every line of output of the command "cat source.txt" , and puts line by line temporarily in a variable which we just call FROM_SRCFILE. Then puts the command which deletes the line matching this string into another variable called TEMP1 (didn't have much imagination) , and then just uses eval to correctly "evaluate" the variables and runs the command. It does it as many times as lines the source.txt file has.