compare directory to .txt or .xml???

aircftech · 12-14-2010, 10:54 AM

I have a database of around 4000 files. I use rsync script to transfer from 2 separate updated hdd every month but I need a way to compare a .txt file or .xml file to the updated drive as I cannot use rsync delete because I can not mount all 3 drives at the same time??

Thanks for any help.

frieza · 12-14-2010, 11:01 AM

you cant mount all 3 drives at the same time? why not?

aircftech · 12-14-2010, 11:03 AM

they come on 2 separate hdd and I only have 1 slot. figures right.

frieza · 12-14-2010, 11:10 AM

just an idea (if you have a few bucks to spare) usb to ide/sata adapters run about $20 USD give or take
just a thought

as for synchronizing, you could do something like

Code:

ls > list.txt

once you have a list then create a text file named say, sync.sh
put in it

Code:

#!/bin/bash
for file in `cat list.txt`
do
copy $1/$file $2/$file
done

then chmod +x sync.sh and invoke it with
[code]
$./sync.sh /source/path /destination/path
assuming the list is one file name per line in the text file that is

aircftech · 12-14-2010, 11:21 AM

not really a option. I have everything all set with a rsync script I just need to remove the old file about 20 of them. but I have the compare Dir1 to a .txt or .xml file some how.

I thought "diff" could do this???

frieza · 12-14-2010, 03:25 PM

no, diff compares the contents of files line by line, useful for say, applying patches to source code, but doesn't seem to be what you are looking for, it seems at least what you need is something that provides a list of files you want updated, files you want ignored, files you want removed and then something that does a loop and acts accordingly

barriehie · 12-14-2010, 04:39 PM

Okay, this should just about do it. Was getting all tangled up reading 2 different files... Anyhow, this will read a filename from the old_file_list file and grep the new_file_list file for the file name retrieved. If it succeeds in finding the file name in the new file list then grep will return data in my_result, i.e. the my_result file will now be greater than zero size. Now test and if my_result IS zero size the file name returned from the old_file_list doesn't exist in the new file list so delete the file from the old_file_list path.

Code:

#!/bin/bash
cat old_file_list | while read
do
  grep "$REPLY" new_file_list >> my_result
  if [ ! -s my_result ]
  then
    rm /old_backup_path/"$REPLY"
  fi
  rm my_result
done

exit 0

aircftech · 12-15-2010, 07:36 AM

ok I have the output I want from creating to .txt files and running

comm -13 /home/dir1/mediaload.txt /home/dir2/mediaload.txt

returns the file names I need to delete. Any idea on how I can do that??

Thanks for the help.

barriehie · 12-15-2010, 08:39 AM

Post the code you're using!

aircftech · 12-15-2010, 12:14 PM

Sovled

I did this for both Hdd-----

zenity --question --text "Do you want to copy CRU1 to array1 ?"
if [ $? == 0 ] ; then
cp /media/cru/mediaload.txt /media/array1
rsync -a /media/cru /media/array1 | tee >(zenity --progress --title "Tranfer" --text "Coping file from CRU1 to ARRAY. Get a cup of coffee" --percentage=0 )
notify-send 'Please shut down and install CRU2 and run TranferCRU2'
else
notify-send 'Please try again something went wrong'
fi

then i did-------

zenity --question --text "Do you want to Delete old files??"
if [ $? == 0 ] ; then
comm -13 /home/dir1/mediaload.txt /home/dir2/mediaload.txt | xargs -I '{}' rm /home/dir2/'{}'
notify-send 'Please run Verify on Media'

else
notify-send 'Thank You GoodBye!'
fi

with 1tb of info I didn't have enough space to hold to copies and do a simple cp or rm.

thanks for the suggestions.