rsync skipping files

s_siouris · 04-04-2005, 08:04 PM

Hi,
I wrote a bash script for updating either my pc at home, or my pc at work (either one can be the target or source), but rsync always skips files resulting in the target directory having less files than the source. I basically don't know if there is a problem with my script, or if it's to do with rsync. I read that rsync doesn't behave properly when there are a lot of arguments, so I am only using two or three arguments, but I am stil getting that problem.

Here's the script I wrote:

Code:

#!/bin/bash

wa_zeus=warehouse
wa_home=warehouse

d[1]=bookmarks
d[2]=cics
d[3]=docs
d[4]=ga-codes
d[5]=linux
d[6]=manuals
d[7]=papers
d[8]=personal
d[9]=programming-c
d[10]=programming-fortran


zeus=xxxxxx@xxxxxxxxxx     # username and ip of the remote pc

echo ' Choose which workstation to bring up to date:'
echo ' zeus --> home box, press 1'
echo ' home box --> zeus, press 2'
echo ' anything else exits this script.'; read choice
echo ' you chose' $choice

if [ "$choice" = 1 ]
then
        for index in 1 2 3 4 5 6 7 8 9 10
        do
                echo
                echo 'now rsync-ing '${d[index]}
                rsync -r -z --delete --progress $zeus:/$wa_zeus/${d[index]}/* /$wa_home/${d[index]}/
        done
        echo 'All done!'
elif [ "$choice" = 2 ]
then
        for index in 1 2 3 4 5 6 7 8 9 10
        do
                echo
                echo 'now rsync-ing '${d[index]}
                rsync -r -z --delete --progress /$wa_home/${d[index]}/* $zeus:/$wa_zeus/${d[index]}/
        done
        echo 'All done!'
else
        echo 'You chose to exit'
fi

exit

By the way, does anybody know a quick way to write a "for" loop in bash that will recurse into directories? (wondering about this for another script of mine).

Thanks
Spiros

Technoslave · 04-05-2005, 10:10 AM

Well, the way you're doing your rsync looks right, although, that's not how I would have done it, so I can't be 100% sure of it's accuracy. Also, you might try putting a -x at the end of #!/bin/bash, that will give you more output when you run the command, you might be able to debug the issue that way.

Also, for your rsync, depending on what type of connection you have, you might want to include the --no-whole-file option.

As for your for loop, it's a kludge, but you could do something like:

for i in `ls */*.txt`; do echo $i;done

Something like that. Hopefully, someone will have a cleaner solution to that.

s_siouris · 04-05-2005, 03:10 PM

thanks for you rreply,

the reason I have done my rsync this way is because I don't want to syncronise all the directories, but only a few ones (which I can easily add/remove as values of the array), so I can't think of any other way of doing this easily. I could open and read a file containing the list of directories I want to synchronise, but this just adds complexity without real gains.

Could you please explain your suggestion for the for loop?
for i in `ls */*.txt`; do echo $i;done
I assume that the 'ls */*.txt' is a command and `pipes' to the value for `i' (correct?). I tried that ls command on a shell but it didn't work. I understand what `ls *.txt' does, but not the 'ls */*.txt'.

thanks
Spiros

s_siouris · 04-05-2005, 04:34 PM

I've just found the problem... I was making a stupid mistake...
Basically the count from rsync for the files to be considered includes folders, but these aren't actually transfered rather than `created' in the target box. This is why the actual transfer number of files was always less than the total amount of `files' (ie files+directories)

Thanks for your help anyway
Spiros

Technoslave · 04-06-2005, 11:00 AM

Sure, I know what you mean with the folders and the like.

As for my quick script.

If you were to do an ls -l */*.txt, you'd get all the files that are in one level of folder down that end in *.txt, likewise you could */*/*.txt, which would go 2 folder levels down. Like I said, a kludge, the echo $i is just to tell you what the file is named in the iterative process of the script.