LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 11-02-2005, 05:33 PM   #1
0ddba11
Member
 
Registered: Nov 2004
Location: Derby - UK
Distribution: Ubuntu at Home, RedHat Enterprise at Work
Posts: 46

Rep: Reputation: 15
Question rsync files accessed under 30 days


Hello all...

I am setting up some disaster recovery servers accross 3 sites and I am using rsync over ssh to sync data accross our WAN.

Due to the shear volume of data, I only want to send files accessed during the last 30 days over the WAN. Since rsync doesn't seem to have any built in method for checking access or modified times I was planning on using 'find' with the -atime switch to create an exclude list that I can then suck into rsync.

Just wanted to check if this was the best way to acheive what I want or does anyone else have any other ideas?

Thanks,
Gaz.
 
Old 11-02-2005, 06:53 PM   #2
TBC Cosmo
Member
 
Registered: Feb 2004
Location: NY
Distribution: Fedora 10, CentOS 5.4, Debian 5 Sparc64
Posts: 355

Rep: Reputation: 43
I think rsync's default behavior will be more efficient than what you've proposed, since it will only send the differences in changed files.
 
Old 11-03-2005, 04:18 AM   #3
0ddba11
Member
 
Registered: Nov 2004
Location: Derby - UK
Distribution: Ubuntu at Home, RedHat Enterprise at Work
Posts: 46

Original Poster
Rep: Reputation: 15
Ah yes, I realise that rsync does delta changes hence why I'm using it, but, we don't have enough disk space at the remote sites to store everything.

My plan is to:

1. Use find -atime to generate a list of files accessed over 30 days ago to exclude from the rsync process
2. Rsync to the remote site using the exclude list
3. Use find -atime and -exec to bin files accessed over 30 days ago at the remote site
4. Bin any empty directories at the remote site

Thoughts?
 
Old 11-03-2005, 07:59 AM   #4
0ddba11
Member
 
Registered: Nov 2004
Location: Derby - UK
Distribution: Ubuntu at Home, RedHat Enterprise at Work
Posts: 46

Original Poster
Rep: Reputation: 15
rsync tunnelled through ssh

I've gone with this and it seems to be working quite well:

Code:
#!/bin/bash
#Find files modified in the last 30 days and store it in a temporary file
cd /dr/officeA
find * -type f -mtime -30 > /tmp/synclist

#rsync these files to officeB and log to a file
rsync -avz --files-from=/tmp/synclist /dr/officeA -e "ssh -i /root/my-ssh-key" root@server.officeB.domain:/dr/officeA > /dr/log/push_to_officeA_at_`date +%H.%M_on_%d.%m.%Y`.log

#Remove the temporary file
rm -f /tmp/synclist

#Bin any files at officeB older than 30 days
ssh -i /root/my-ssh-key root@server.officeB.domain 'find /dr/officeA -type f -mtime +30 -exec rm -f {} \;'

#Bin any empty directories in officeB
ssh -i /root/my-ssh-key root@server.officeB.domain 'find /dr/officeA -type d -empty -exec rmdir -p --ignore-fail-on-non-empty {} \;'
I had to do the 'Bin any files older than' and 'Bin any empty directories' as the --delete option in rsync doesn't do anything when using the --files-from option.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
delete files older than 30 days using cronjob latheesan Linux - Newbie 5 06-14-2005 03:40 PM
Same permissions on two files, but only one can be accessed by web server? murr4y Linux - Software 5 06-14-2005 03:59 AM
recent attack via phpBB, log files deleted? accessed file system outside /home/ enzo250gto Linux - Security 8 12-17-2004 02:51 PM
help with a script that deletes files more than X days old BrianK Linux - General 5 06-14-2004 10:05 PM
Tracking the accessed files EdwinNameless Linux - Software 5 07-16-2003 12:00 PM


All times are GMT -5. The time now is 01:13 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration