Any easy way to compare files in two different trees?

Richard Cranium · 05-18-2014, 07:24 AM

Quote:

Originally Posted by stf92

I broke the command into several lines for greater legibility:

Code:

semoi@server:~/STORE1/soft/linux/comparo$ cat comparo02a.sh
tree1="/home/semoi/STORE1/sma_"
tree2="/home/semoi/juan/sma_"

diff <(cd $tree1 
t1=$(find -type f -name \* -exec ls -l {} \;)
paste <(cut -d' ' -f5 <<<"$t1") <(cut -d' ' -f9- <<<"$t1") \
) <(cd $tree2 
t2=$(find -type f -name \* -exec ls -l {} \;)
paste <(cut -d' ' -f5 <<<"$t2") <(cut -d' ' -f9- <<<"$t2")
)

If I comment the paste lines out then I get no output. Why?

Umm, because they are the commands actually printing stuff out.

stf92 · 05-18-2014, 07:56 AM

But when I use diff at the console it always prints something whenever it finds a difference!

Paulo2 · 05-18-2014, 10:02 AM

Sorry, my fault

the way I posted it is really confusing.
I think the problem is one of the escaped new-line.

This is a very better looking version

Code:

paulo@paulobash 4.3.11~ $ cat compare-stf92.sh
#!/bin/bash
#
# Compare two directories with 'size' and 'name' taken from
# 'ls -l --time-style=long-iso', or 'date time' and 'name'.
#
# Directories are passed as $1 and $2, size or date and time as $3.



usage()
{
echo "
Usage:  ${0#*/} {dir1} {dir2} {size|date}

$(sed '/^# /! d ; s/^# /\t/' "$0")
"
exit
}



[ ! -d "$1" -o ! -d "$2" ] && usage

case "$3" in
        size) field='-f5'       ;;
        date) field='-f6-7'     ;;
           *) usage             ;;
esac


diff    <(      \
        cd "$1"
        t1=$(find -type f -name \* -exec ls -l --time-style=long-iso {} \;)
        paste <(cut -d' ' $field <<<"$t1") <(cut -d' ' -f8- <<<"$t1") \
         )      \
        <(      \
        cd "$2"
        t2=$(find -type f -name \* -exec ls -l --time-style=long-iso {} \;)
        paste <(cut -d' ' $field <<<"$t2") <(cut -d' ' -f8- <<<"$t2") \
         )

Since there is no remove, copy or move commands, any bug will be harmless.
Sorry for any English errors

By the way, using awk would be better, but I don't know how to print field $8 until
the end of line, for example.
I tested in a directory that has file names with spaces. so {print $8} prints only
the first word of file name.

stf92 · 05-18-2014, 10:21 AM

I removed the paste lines and ran diff on the two variables tree1, tree2, but diff said the variables were too long. Using input redirection seems to do things on the fly. Another advantage of your approach is that there is no limit for the length of the directory listings. Thanks for your script. I'll try to make print whenever there is a difference in length OR in the date/time.

Didier Spaier · 05-18-2014, 10:44 AM

Quote:

Originally Posted by Paulo2

By the way, using awk would be better, but I don't know how to print field $8 until
the end of line, for example.
I tested in a directory that has file names with spaces. so {print $8} prints only
the first word of file name.

Code:

ls -l --time-style=long-iso <directory>|awk '{ORS=" ";for(i=8; i<=NF; i++) print $i;print "\n"}'

EDIT Not good, but stay tuned...

EDIT2 Should work now.

Paulo2 · 05-18-2014, 11:19 AM

Quote:

Originally Posted by stf92

I removed the paste lines and ran diff on the two variables tree1, tree2, but diff said the variables were too long. Using input redirection seems to do things on the fly. Another advantage of your approach is that there is no limit for the length of the directory listings. Thanks for your script. I'll try to make print whenever there is a difference in length OR in the date/time.

I'm glad that it helped.

I guess that you are thinking in compare all the line, not just size or date and name?
This is more simple, no doubt. Just cd to directory and run find in it,
no need of vars, paste and cut. (KISS, of course

)

Code:

list_tree(){ cd "$1";find -type f -name \* -exec ls -l --time-style=long-iso {} \; ;}
diff <(list_tree tree-1) <(list_tree tree-2)

(well maybe one function

)

Paulo2 · 05-18-2014, 11:29 AM

Quote:

Originally Posted by Didier Spaier

Code:

ls -l --time-style=long-iso <directory>|awk '{ORS=" ";for(i=8; i<=NF; i++) print $i;print "\n"}'

EDIT Not good, but stay tuned...

EDIT2 Should work now.

Hi Didier, thanks for the examples.

Yep, that first one didn't work

it listed all words of the names, but broken
spaces into new lines, like this:

Code:

tree-1/Appearance
Edit
1240/Documentation.pdf

cut -d' ' -f8- cuts ok, field 8 until the end of line

Code:

tree-1/Appearance Edit 1240/Documentation.pdf

Yes, that one did the job

thanks

Code:

ls -l --time-style=long-iso <directory>|awk '{ORS=" ";for(i=8; i<=NF; i++) print $i;print "\n"}'

I'm just doing baby steps in awk yet...

stf92 · 05-18-2014, 11:40 AM

Quote:

Originally Posted by Paulo2

I'm glad that it helped.

I guess that you are thinking in compare all the line, not just size or date and name?
This is more simple, no doubt. Just cd to directory and run find in it,
no need of vars, paste and cut. (KISS, of course

)

Code:

list_tree(){ cd "$1";find -type f -name \* -exec ls -l --time-style=long-iso {} \; ;}
diff <(list_tree tree-1) <(list_tree tree-2)

(well maybe one function

)

You are right. I didn't think of it. By the way, the --time-style option was not necessary in slackware 12.0, as it was the default setting. This is one of the niceties of new versions.

eloi · 05-19-2014, 02:45 AM

Quote:

Originally Posted by stf92

That's precisely the way I don't like. Experimentation is statistically incorrect. You will precisely know what a program does by just learning what the manufacturer has to say about it, as with anything else.

Fist of all, statistics are not there to say you what is correct.

Taking in care your posts here it's evident you invest too much time in experimentation and very few in reading documentation. That's why you have to ask here simple things that you already have on man pages. Like a jazz solo, experimentation needs a base.

perbh · 05-19-2014, 09:22 PM

I have great misgivings of using the timestamp for any kind of comparison - unless you use 'cp -p', 'scp -p', 'tar ..p.' etc etc
In my own .bashrc, I always do:

Code:

alias cp='cp -p'
alias scp='scp -p'

but it's very easy to forget the 'p'-option when untarring ... unless its 'in your fingers'

stf92 · 05-19-2014, 10:18 PM

I always do the same. Could you tell what is the way generally used by Linux users, those who can't afford tapes, to make backups of their personal data? I mean, the only medium I have to do that are optical disks, and what I do is just make an iso image of the directory where I keep my data, and then burn a DVD. But it has a lot of drawbacks.

perbh · 05-20-2014, 02:22 PM

I don't know about 'others', but what _I_ do is to have an external usb-drive (say 1TB) - they can be had for about $60-$70.
Then I have a cron-script that checks if the usb-drive is present, and if so, it mounts it, does a 'rsync' and then umounts it again. Fairly painless and no problem if I forget to plug it in before I go to bed ....

Mind you - I only do it for my personal files, never bother the OS which can easily enough be replaced/reinstalled

If its auto-mounted (which I _never_ do, but 'others' might prefer that), I don't mount it but run the 'rsync'-command and always umount it at the end - no matter what!

stf92 · 05-20-2014, 03:09 PM

I am not familiar with modern terminology. By external usb-drive do you mean a flash memory drive with data transfered through USB? Or an external hard drive, hence a magnetic medium, communicating through USB?

perbh · 05-21-2014, 03:10 PM

Modern technology?
As for 'magnetic medium - even a 'flash' device is magnetic ...

DVD's can only take 4.7 GB, but even usb-sticks/flash/whatever can be had in 16GB, 32GB, 64GB ... however, their write-life is somewhat limited, but hey - DVD's you can only write once!

You can buy harddisks that resides inside a 'box'/cover/enclosure that communicates via usb. ie it is an 'external' harddrive inasmuch as it is _not_ inside your computer. These are usually referred to as 'usb-disks' and as I said, at least here in the US you can buy a 1TB usb-disk for $60-$70 ...

Usually, what I do with mine is making it bootable with slackware - this only takes away some 20 gigs out of 1TB (or 2TB) - small price to pay if anything goes wrong ;-)

As a side-note, I have found that what usually goes wrong with a usb-disk, it's the 'enclosure' that's at fault. Rip asounder or dismantle the enclosure and you have a perfectly good sata harddisk inside.

You can also buy ($20 or so) a 'converter' (for lack of a better word) that will make your 3.5"/ 2.5"/ide-disk look like a usb-storage-medium ...

stf92 · 05-21-2014, 03:22 PM

Quote:

Originally Posted by perbh

You can also buy ($20 or so) a 'converter' (for lack of a better word) that will make your 3.5"/ 2.5"/ide-disk look like a usb-storage-medium ...

I didn't know that. I said 'modern terminology', not 'modern technology'. As far as I know, flash is semiconductor memory and, as such, a purely electronic device. Nothing magnetic in it.