LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-18-2011, 09:23 PM   #1
mike11
Member
 
Registered: Apr 2009
Posts: 222

Rep: Reputation: 17
Backup only the files which have changed?


Hi All,

Can someone please explain if there an application which does the following:

1. I have a laptop with an internal 200GB HD.
2. I run the application & it creates a list of all files (size & time-stamp) without actually storing them. Let's call this the "snapshot list".
3. I update some of the files on the laptop.
4. Now I run the application & it only copies the files which have changed on the laptop, that have different size/time-stamp from the snapshot list, onto some external media, such as a memory card. Of course, the files should be copied onto their proper location in the directory tree & not just pile up in one place.

Why is this useful? although the laptop has a 200GB HD I typically only update a small number of files, whose total size is maybe 10MB or so. If I could only backup those which have changed, I could do this with a tiny SD card instead of lugging around an external usb HD.

Thanks,
Mike.
 
Old 05-18-2011, 11:21 PM   #2
smturner1
Member
 
Registered: Oct 2009
Location: MI
Distribution: Arch 2.6.35
Posts: 107

Rep: Reputation: 1
If I am understanding this correctly, you would like a script that takes a list of recently updated files and have them copied to a flash drive of some kind, right?

The way you are wording this is a little confusing.

Quote:
2. I run the application & it creates a list of all files (size & time-stamp) without actually storing them. Let's call this the "snapshot list".
There is definitely a way to create a script to do this. First you would want to call out the directory, list the files with the sizes and last date changed, run a couple of conditional statements, point the output to copy to your flash media, and voila you have your application.

man ls
man cut
man cp
google if statements
google storing variables..........etc.

I am sure there are some great programmers that could give you better hints than I, but I think this might be a classroom assignment. I could be wrong, but..........?

Hope that points you into the right direction.
 
Old 05-18-2011, 11:36 PM   #3
mike11
Member
 
Registered: Apr 2009
Posts: 222

Original Poster
Rep: Reputation: 17
Thanks for replying. This is not a classroom assignment, just sometime which I think can be useful.

it is probably doable by hand but it may be incredibly inefficient, especially when examining HD of 100s of GB. I'll try to rephrase the scenario:

1. I"ve got a laptop with an internal HD, say 200GB.
2. The application scans over all HD files and stores their current size & time-stamp.
3. I work on the laptop, and update some files.
4. I re-run the application and it now compares each file to its stored size & time-stamp. any files which have been changed are copied into another media for backup. however, they are stored in their proper directory location and not just dumped into one directory.

Hope this is clearer. aren't there application out there already which do this kind of smart-backuping?.

TIA.
 
Old 05-19-2011, 12:29 AM   #4
Nylex
LQ Addict
 
Registered: Jul 2003
Location: London, UK
Distribution: Slackware
Posts: 7,464

Rep: Reputation: Disabled
rsync? Your distribution probably has a package for it, if it's not installed already.
 
Old 05-19-2011, 12:47 AM   #5
mike11
Member
 
Registered: Apr 2009
Posts: 222

Original Poster
Rep: Reputation: 17
Thanks. I am familiar with rsync though I can see an option to create a list of files with their current size and time-stamp without copying the entire file. or am I missing something?.
 
Old 05-19-2011, 01:15 AM   #6
j1alu
Member
 
Registered: Apr 2009
Distribution: debian gnu/linux
Posts: 798

Rep: Reputation: Disabled
never did it, but man says:
--list-only list the files instead of copying them

I bet if you run "-n" or "--dry-run" it will get stored somewhere.
 
Old 05-19-2011, 10:57 PM   #7
smturner1
Member
 
Registered: Oct 2009
Location: MI
Distribution: Arch 2.6.35
Posts: 107

Rep: Reputation: 1
There is another way to approach this, have you ever heard of checksum? Checksum is an algorithm that looks at files and compares their sizes to make sure you have a good copy, that there are no malicious programs attached to it, and that you have the most recent copy. Of course, the tool is only as good as the datum.

What I would do is use rsync, it uses the checksum methodology/algorithm. It will reduce the amount of commands in your shell by removing the cp, checksum, and ls, because if it is used properly it will do all of it (locally and remotely).

You are going to have to write the script, but the people on this forum are incredibly helpful. There are two reasons they are helpful, they will not give you the fish, they will teach you to fish!

Remember, have fun! There is nothing more rewarding in UNIX/Linux than creating a working tool that automates tasks.
 
1 members found this post helpful.
Old 05-20-2011, 12:10 AM   #8
astromime
LQ Newbie
 
Registered: May 2011
Posts: 26

Rep: Reputation: 13
i have an...extensive option

Code:
#!/bin/bash

function direc {
        if [ ! -d backup ]
        then
                mkdir backup
        fi
}

copy () {
if [ ! -d backup ]
then
        echo "The backup folder does not exist."
else
for file in *
do
        if [ $file -nt backup/$file.bak ] && [ -f $file ]
        then
                cp -v $file destination/$file
        else
                echo "The file $file was not modified since last copy."
        fi
done
fi
}

backup () {
direc
for files in *
do
        if [ -f $files ]
        then
                cp -v $files backup/$files.bak
        fi
done
}

while getopts cb opts
do
        case $opts in
        c) copy;;
        b) backup;;
        esac
done
obviously, this will work if you want to back the files up also, but you said you didnt want to do that...you would have to change "destination" to wherever your media is mounted on (df -h)

but if you decide to use this, just call it with the appropriate options, so if you name it "filecopy.sh" then youd call it with "filecopy.sh -b" to create the backup, or renew them, and "filecopy.sh -c" to copy the new files to your drive
 
1 members found this post helpful.
Old 05-24-2011, 02:30 AM   #9
mike11
Member
 
Registered: Apr 2009
Posts: 222

Original Poster
Rep: Reputation: 17
Thanks all for the suggestions.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Application to eliminate doubles in files and rename changed files with date? S. Chapelin Linux - Software 6 01-16-2011 03:02 AM
Changed files between two folders axelfc Linux - General 3 05-10-2010 04:24 PM
backup changed files only. deadlock Linux - Software 9 12-14-2007 12:36 PM
all my text files have changed to realmedia files erikus Linux - General 2 12-28-2006 05:43 PM
10.1 -> 10.2 which conf files changed? Vgui Slackware 3 09-21-2005 05:55 PM


All times are GMT -5. The time now is 10:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration