LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Ubuntu
User Name
Password
Ubuntu This forum is for the discussion of Ubuntu Linux.

Notices


Reply
  Search this Thread
Old 02-24-2009, 03:49 AM   #1
sentiax
LQ Newbie
 
Registered: Feb 2009
Posts: 2

Rep: Reputation: 0
How to automatically detect and backup changes to folder?


Ok so here's the deal. I've got a main gaming desktop that I use all the time. I've also got an HTPC and a Fileserver. My dilemma is that I like to have my Music, Pictures, Videos, and other important data backed up onto the Fileserver (which is running Xubuntu).

I've gotten FTP, Samba, and everything setup correctly, can access it from outside the home network and stuff so I can stream music to my laptop when I'm on my college campus without having to take up so much hdd space on my laptop, as well as backup my programming and essay homework.

What I would like to do is have the fileserver automatically detect changes in my main desktops 1TB hard drive where all the Music, videos etc are originally from and then copy the new files to itself. Is there any way to have this done without it completely copying the whole folders over. I've found a guide on how to do it with rsync, but it basically copies the entire folder over again, and with 230GB worth of data, that's not worth it every time there's a 20MB change.

Just as an example and for more details in case I've made this confusing:

My terrabyte hard drive on my main Desktop is J:\. The folders I have backed up onto the Fileserver from J:\ are titled Games, Installs, ISO, Pictures, Music, Vids and are named exactly that on the fileserver as well. Let's say I buy a new Album from Itunes and save it to J:\Music\<artist name>\<album name> is there a way to have the fileserver detect that change, and then automatically copy JUST the new data?

Thanks in advance for help. Let me know if I need to give any more info.
 
Old 02-24-2009, 05:54 AM   #2
zedmelon
Member
 
Registered: Jun 2004
Location: colorado, USA
Distribution: slack, oBSD
Posts: 119

Rep: Reputation: 24
Quote:
Originally Posted by sentiax View Post
I've found a guide on how to do it with rsync, but it basically copies the entire folder over again, and with 230GB worth of data, that's not worth it every time there's a 20MB change.
Are you certain the date/time on both machines is the same? That certainly wouldn't be worth it, which is the point of rsync. In my experience, rsync takes an eternity for only the first run. Once it's made the initial sync, it will only bother with that 20M thereafter.

Timestamps can affect this as well as file size. You can alleviate that with a
cp -p.

Last edited by zedmelon; 02-24-2009 at 05:55 AM.
 
Old 02-24-2009, 06:28 AM   #3
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
I don't know what you mean by copy files to itself. You can use find to locate new files. Another option is rsync. Also tar with the -g option for incremental backups. ( See section 5.2 of the info manual for tar for examples. )
 
Old 02-25-2009, 01:42 AM   #4
sentiax
LQ Newbie
 
Registered: Feb 2009
Posts: 2

Original Poster
Rep: Reputation: 0
Sorry for the confusion. I actually figured it out using robocopy and a batch file in my gaming machine.
What I meant by copy to itself is:

On my Windows Gaming Machine there is an eSata 1TB hard drive that I keep all the aforementioned folders on Labeled as J:\
What I wanted to happen, is I wanted my Xubuntu Fileserver to automatically copy those folders too itself every night or whenever I wanted on a regular schedule.
My problem was that using a guide I found, it copied and overwrote everything on itself instead of just updating the files or copying new ones that were on J:\

After I posted on here, I actually figured out how to do it the way I wanted with rsync, but it was incredibly slow (we're talking 5KB/s and stuff).
So instead I just created a bath file on the windows machine that ignores existing files on the server and only copies new ones, and does it at around 12MB/s (so basically my servers capacity with a 100mbps nic card).

I've got a gigabit card coming in for the server (which is like an 8 year old dell I bought for 10$) which should boost the transfer speeds up exponentially.
I might try rsync again when it comes in, but I tried rsync even manually and it was still incredibly slow even after the files were already on the server. Let me know if you have any other ideas though. I would prefer it to run on the server itself rather than the Windows machine.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Detect no IPv6 on this connection therefore switch off automatically. Possible? zoubidoo Linux - Networking 2 09-02-2008 12:26 PM
New to Linux and wifi doesn't detect automatically taman Linux - Wireless Networking 0 11-25-2007 01:30 AM
How to automatically move files from one folder into another folder? xmrkite Linux - Software 6 11-05-2006 11:39 AM
Pen drive, how detect automatically dass Slackware 6 07-25-2006 07:44 AM
Detect automatically USB flashdisk intmail Slackware 1 07-20-2005 11:17 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Ubuntu

All times are GMT -5. The time now is 10:20 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration