How to get differences between two files using bash script

jfarhat · 09-09-2008, 03:52 PM

Greetings,

I am new to bash scripting and need help.

I have two text files F1 and F2. F1 contains a list of items
1 2 3 a
1 2 3 b
1 2 3 c
1 3 4 d
2 3 4 e
1 3 4 f

The file F2 contains
c
d
e

I need to write a script to show the difference between F1 and F2 where in this case the difference is
a
b
f

Your help is greatly appreciated.

I tried something as follows to store the items 2 and not in 1 but it fails:

exec < $inF1
while read line
do
cat $inF2 | grep -v "echo $line | awk '{ print $4 }" > z.txt
done

matthewg42 · 09-09-2008, 04:12 PM

It would be easy in perl, and a lot more efficient that shellscript:

Code:

#!/usr/bin/perl

my %two;
open(TWO, "<F2") || die "cannot open F2 : $!\n";
map { chomp; $two{$_} = 1; } <TWO>;
close(TWO);

open(ONE, "<F1") || die "cannot open F1 : $!\n";
map { chomp; $_ = (split(/\s+/))[3]; print "$_\n" if (!defined($two{$_})); } <ONE>;
close(ONE);

bgeddy · 09-09-2008, 04:41 PM

Heres an undoubtedly ugly bit of bash code to do it. Create a script with this as the contents,make it executable and call it with ./script_name file1 file2 as per your example.

Code:

#!/bin/bash
cut -d " " -f4 $1 | sort | sdiff --left-column  - $2 | grep -v "(" | cut -f1

I think there is a much nicer way of doing this but this is my first hack..

jan61 · 09-09-2008, 05:53 PM

Moin,

try this:

Code:

TMP_F1=`mktemp` || exit 1 # create a temp file
cut -f4 -d" " file1 | sort >$TMP_F1 # cut the 4th field from file1, sort
comm -3 $TMP_F1 file2 # show all lines not appearing in both files
rm $TMP_F1

if file2 is also not sorted:

Code:

TMP_F1=`mktemp` || exit 1 # create a temp file
TMP_F2=`mktemp` || exit 1 # create a second temp file
cut -f4 -d" " file1 | sort >$TMP_F1 # cut the 4th field from file1, sort
sort file2 >$TMP_F2
comm -3 $TMP_F1 $TMP_F2 # show all lines not appearing in both files
rm $TMP_F1 $TMP_F2

Jan

keefaz · 09-09-2008, 05:56 PM

Maybe use grep -f ?

Code:

grep -v -f F2 F1 | cut -d' ' -f4

billymayday · 09-09-2008, 06:05 PM

Not so pretty, but works

Code:

cut -d ' ' -f4 F1 | diff - F2 | grep \< | cut -d ' ' -f2