LinuxQuestions.org - rsync files accessed under 30 days

- Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)

- - rsync files accessed under 30 days (https://www.linuxquestions.org/questions/linux-general-1/rsync-files-accessed-under-30-days-379378/)

rsync files accessed under 30 days

Hello all...

I am setting up some disaster recovery servers accross 3 sites and I am using rsync over ssh to sync data accross our WAN.

Due to the shear volume of data, I only want to send files accessed during the last 30 days over the WAN. Since rsync doesn't seem to have any built in method for checking access or modified times I was planning on using 'find' with the -atime switch to create an exclude list that I can then suck into rsync.

Just wanted to check if this was the best way to acheive what I want or does anyone else have any other ideas?

Thanks,
Gaz.

I think rsync's default behavior will be more efficient than what you've proposed, since it will only send the differences in changed files.

Ah yes, I realise that rsync does delta changes hence why I'm using it, but, we don't have enough disk space at the remote sites to store everything.

My plan is to:

1. Use find -atime to generate a list of files accessed over 30 days ago to exclude from the rsync process
2. Rsync to the remote site using the exclude list
3. Use find -atime and -exec to bin files accessed over 30 days ago at the remote site
4. Bin any empty directories at the remote site

Thoughts?

rsync tunnelled through ssh

I've gone with this and it seems to be working quite well:

Code:

#!/bin/bash

#Find files modified in the last 30 days and store it in a temporary file

cd /dr/officeA

find * -type f -mtime -30 > /tmp/synclist



#rsync these files to officeB and log to a file

rsync -avz --files-from=/tmp/synclist /dr/officeA -e "ssh -i /root/my-ssh-key" root@server.officeB.domain:/dr/officeA > /dr/log/push_to_officeA_at_`date +%H.%M_on_%d.%m.%Y`.log



#Remove the temporary file

rm -f /tmp/synclist



#Bin any files at officeB older than 30 days

ssh -i /root/my-ssh-key root@server.officeB.domain 'find /dr/officeA -type f -mtime +30 -exec rm -f {} \;'



#Bin any empty directories in officeB

ssh -i /root/my-ssh-key root@server.officeB.domain 'find /dr/officeA -type d -empty -exec rmdir -p --ignore-fail-on-non-empty {} \;'

I had to do the 'Bin any files older than' and 'Bin any empty directories' as the --delete option in rsync doesn't do anything when using the --files-from option.
:D