Comparing lines in 2 text files?

Raveolution · 11-05-2010, 01:43 PM

Is there a way, besides writing a PERL program, to read each line one by one in file A and tell if this line also exists in file B?

Can this be done via a shell script?

Thanks!

pljvaldez · 11-05-2010, 01:55 PM

man diff

trey85stang · 11-05-2010, 01:59 PM

you could write a shell script but it wouldnt be very efficient.. At least of the top of my head I cant think how to do it without reading a file as many times as there are lines.

awk would probably be a better solution.

shell would look something like this.

Code:

#!/bin/bash
#Source file to read from
fileA=filea.txt
#File to check against
fileB=fileb.txt

num_lines=`wc -l $fileA | sed "s/\(^[0-9]*\).*/\1/"`
let cnt=1
while [ $cnt -le $num_lines ]
do
  line=`sed -n ${cnt}p $fileA`
  if grep "$line" $fileB
  then
    echo "LINE# $cnt exists in $fileB"
  fi
  let cnt++
done

GrapefruiTgirl · 11-05-2010, 02:00 PM

I agree, `diff` is the way to go. There's also `grep -f` if you like, for comparing files. Or the plain shell loop like below, which probably is not reliable if the files contain anything weird or beyond simple ascii text -- you'd maybe need a more advanced regex:

Code:

while read line; do if grep -q "$line" fileB; then echo "$line"; fi; done < fileA

trey85stang · 11-05-2010, 02:02 PM

diff would be a good choice as long as the files are not vastly different from each other.

GrapefruiTgirl · 11-05-2010, 02:15 PM

You could also (in Bash) use arrays and embedded loops...

Code:

 ARR1=( $(cat fileA) )
 ARR2=( $(cat fileB) )
 for ((x=0;x<${#ARR1[@]};x++)); do
     for ((y=0;y<${#ARR2[@]};y++)); do
         if [ "${ARR1[x]}" = "${ARR2[y]}" ]; then
             echo "Match: ${ARR1[x]}"
         fi
     done
 done

Raveolution · 11-06-2010, 03:59 PM

Quote:

Originally Posted by GrapefruiTgirl

You could also (in Bash) use arrays and embedded loops...

Code:

 ARR1=( $(cat fileA) )
 ARR2=( $(cat fileB) )
 for ((x=0;x<${#ARR1[@]};x++)); do
     for ((y=0;y<${#ARR2[@]};y++)); do
         if [ "${ARR1[x]}" = "${ARR2[y]}" ]; then
             echo "Match: ${ARR1[x]}"
         fi
     done
 done

Ah, I didn't know this task could get that complex. I can work with this script the best, but they're all good and extremely instructional. This looks like it works even for stuff with complex data.