rsync files accessed under 30 days

0ddba11 · 11-02-2005, 04:33 PM

Hello all...

I am setting up some disaster recovery servers accross 3 sites and I am using rsync over ssh to sync data accross our WAN.

Due to the shear volume of data, I only want to send files accessed during the last 30 days over the WAN. Since rsync doesn't seem to have any built in method for checking access or modified times I was planning on using 'find' with the -atime switch to create an exclude list that I can then suck into rsync.

Just wanted to check if this was the best way to acheive what I want or does anyone else have any other ideas?

Thanks,
Gaz.

TBC Cosmo · 11-02-2005, 05:53 PM

I think rsync's default behavior will be more efficient than what you've proposed, since it will only send the differences in changed files.

0ddba11 · 11-03-2005, 03:18 AM

Ah yes, I realise that rsync does delta changes hence why I'm using it, but, we don't have enough disk space at the remote sites to store everything.

My plan is to:

1. Use find -atime to generate a list of files accessed over 30 days ago to exclude from the rsync process
2. Rsync to the remote site using the exclude list
3. Use find -atime and -exec to bin files accessed over 30 days ago at the remote site
4. Bin any empty directories at the remote site

Thoughts?

0ddba11 · 11-03-2005, 06:59 AM

I've gone with this and it seems to be working quite well:

Code:

#!/bin/bash
#Find files modified in the last 30 days and store it in a temporary file
cd /dr/officeA
find * -type f -mtime -30 > /tmp/synclist

#rsync these files to officeB and log to a file
rsync -avz --files-from=/tmp/synclist /dr/officeA -e "ssh -i /root/my-ssh-key" root@server.officeB.domain:/dr/officeA > /dr/log/push_to_officeA_at_`date +%H.%M_on_%d.%m.%Y`.log

#Remove the temporary file
rm -f /tmp/synclist

#Bin any files at officeB older than 30 days
ssh -i /root/my-ssh-key root@server.officeB.domain 'find /dr/officeA -type f -mtime +30 -exec rm -f {} \;'

#Bin any empty directories in officeB
ssh -i /root/my-ssh-key root@server.officeB.domain 'find /dr/officeA -type d -empty -exec rmdir -p --ignore-fail-on-non-empty {} \;'

I had to do the 'Bin any files older than' and 'Bin any empty directories' as the --delete option in rsync doesn't do anything when using the --files-from option.