[SOLVED] Can you suggest a REAL simple way to backup?
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
How long it takes depends on the system. I just made a youtube video doing this process and it's video length is under twenty minutes. The system cloned was a linux distro iso image. The source iso was < 1GB and the uncompressed system was 3.1GB. From pressing record to booted into the distro was under forty minutes. The extra twenty minutes was figuring out a format to record the new distro in which lacked libx264. That was just a basic desktop with low cruft. It doesn't have to take hours, but it could if you don't take steps to ease the burden of backing up. All those movie clips from your camcorder in /home. All those packages downloaded and updated, but the .deb or .rpm files just sitting there. That mostly unused and hefty /usr/share/doc/ that many distros do not segment out as optional when installing packages. Yeah it could take hours, but it doesn't have to. And an unfragmented filesystem has it's perks.
thanks guys. It's going to take a while to sort through all this. Things I'm currently unfamiliar with (that you mentioned):
"testing any backup w/a restore" "config files" "fragmentation" and lastly "fun."
As much as I like to control my own files and stuff , I tend to back up my really really important files on both my Dropbox and Google Drive. I don't use enough space to have to pay for either and I figure they both have much more redundancy then I can afford to have locally
As much as I like to control my own files and stuff , I tend to back up my really really important files on both my Dropbox and Google Drive. I don't use enough space to have to pay for either and I figure they both have much more redundancy then I can afford to have locally
Thanks Ryanms, Well, that seems super-simple, which I like, but when I tried to download my "gregory" folder (the one below "home" and containing Documents, Downloads,Pictures etc.) Drive said there was an error and wouldn't upload it. But I was able to upload my Documents folder, which is a start! And I, like you, would never use anywhere near all the cloud storage out there. So if my computer crashes (I'm a writer) and I've got my novels safe in the cloud and on a flash drive, do I really need to do the automated sort of backup that is available (and much more complicated)? The way I see it, if I have my Documents folder and the computer crashes all I have to do is reinstall Ubuntu 13.10, which is a breeze. Is it as easy as that or am I missing something?
Only the first time. I back up my home server nightly to two external USB drives using rsync. All in all it's about 3.3TB that's being backed up, and rsync typically takes about 10 minutes to run (most of that time is just generating the file list and then comparing the current version of each file to the backed up version to check for differences).
OP - using rsync is really very simple. You can back up your entire system to an external drive with one command. The first time it will take forever (any backup will), but after that one is finished, new backups are very fast since they just transfer what has been changed since the last one. If you stick this one line command in a text file (which makes it a script), you can then add it to your cron (like Windows' "Scheduled Tasks") so that it runs automatically on a regular schedule. This is what my backup script looks like:
Code:
#!/bin/bash
if [[ ! -e /home/verify ]]; then
echo "raid not found, skipping backup"
exit 1
fi
if [[ ! -e /media/backups/home/verify ]]; then
echo "primary backup drive not found, skipping"
else
/usr/bin/rsync -aAXv --progress --delete /* /media/backups/ --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/.gvfs,/var/lib/ntp/proc/*}
fi
echo ""
echo ""
echo "Backup finished $(date)"
The first two "ifs" just check to make sure the local RAID is present and the backup drive is present before it continues, then the rsync command is what does the backup. The above code is just saved to a text file called "backup", which is made executable with "chmod +x backup", and then added to the cron. The crontab entry looks like:
Which runs it at 4:30am every day, logging the output to the backup.out file so I can inspect it later, if I want.
All in all it took about 2 minutes to set up, and now it runs forever without any user intervention, and it didn't require installing or setting up any software (any Linux distro should have rsync installed by default). You shouldn't just blindly implement what I've done here without understanding what it's doing, but it could give you a baseline to work off of.
I am very much against using the anonymous "cloud" for storage or backups. These are servers run by unknown companies in unknown locations who have unknown agendas. Who knows what they're doing with your data or if they'll even be around tomorrow or the next day. There have been countless cases of companies setting up cloud storage for people to either store data on or contribute to, then one day the company decides that everything that has been uploaded is now theirs to do with what they wish, they lock out their users, and then start charging for access. Other companies set up systems like this and then go out of business and the entire network disappears overnight. Other companies one day decide "meh, we're done" and shut down your access with no warning, permanently. Do you really want to base your entire backup system on the whims of some exec who's answering to shareholders and could decide at any moment to lock you out, delete your data, sell your data, or shut down?
Screw that.
You can buy an external hard drive for cheap, and set up a ONE LINE script to do regular backups, and never have to worry about it again (at least until the external hard drive fails some years down the road and you replace it).
Last edited by suicidaleggroll; 03-20-2014 at 05:21 PM.
Thanks Ryanms, Well, that seems super-simple, which I like, but when I tried to download my "gregory" folder (the one below "home" and containing Documents, Downloads,Pictures etc.) Drive said there was an error and wouldn't upload it. But I was able to upload my Documents folder, which is a start! And I, like you, would never use anywhere near all the cloud storage out there. So if my computer crashes (I'm a writer) and I've got my novels safe in the cloud and on a flash drive, do I really need to do the automated sort of backup that is available (and much more complicated)? The way I see it, if I have my Documents folder and the computer crashes all I have to do is reinstall Ubuntu 13.10, which is a breeze. Is it as easy as that or am I missing something?
I also do some writing and that is the bulk of what I back up. That and photos I don't want to lose. My main concern is security of my personal documents on a cloud servers but 1.) I figure Google and Dropbox are a lot better at security than I am since they are multibillion dollar businesses whose entire business involves them being able to secure their and my data 2.)Nobody wants to hack in and steal any of my writing
I like being able to wipe my entire operating system and computer and still be able to access my important stuff and also like to be able to access it from different computers with internet connections
... my "gregory" folder (the one below "home" and containing Documents, Downloads,Pictures etc.) Drive said there was an error ...
Could always compress the folder see if that goes although I don't think you will have access to the individual files unless the cloud service has the capability to look\take\add in .tar.gz or other .zip like formats? If so could store more for free.
Edit\add: google drive is grate I also email myself important files that sit in folders for years they give lots of storage there too could have lots of email accounts as well...
Last edited by jamison20000e; 03-20-2014 at 05:35 PM.
Reason: +more
Only the first time. I back up my home server nightly to two external USB drives using rsync. All in all it's about 3.3TB that's being backed up, and rsync typically takes about 10 minutes to run (most of that time is just generating the file list and then comparing the current version of each file to the backed up version to check for differences).
OP - using rsync is really very simple. You can back up your entire system to an external drive with one command. The first time it will take forever (any backup will), but after that one is finished, new backups are very fast since they just transfer what has been changed since the last one. If you stick this one line command in a text file (which makes it a script), you can then add it to your cron (like Windows' "Scheduled Tasks") so that it runs automatically on a regular schedule. This is what my backup script looks like:
Code:
#!/bin/bash
if [[ ! -e /home/verify ]]; then
echo "raid not found, skipping backup"
exit 1
fi
if [[ ! -e /media/backups/home/verify ]]; then
echo "primary backup drive not found, skipping"
else
/usr/bin/rsync -aAXv --progress --delete /* /media/backups/ --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/.gvfs,/var/lib/ntp/proc/*}
fi
echo ""
echo ""
echo "Backup finished $(date)"
The first two "ifs" just check to make sure the local RAID is present and the backup drive is present before it continues, then the rsync command is what does the backup. The above code is just saved to a text file called "backup", which is made executable with "chmod +x backup", and then added to the cron. The crontab entry looks like:
Which runs it at 4:30am every day, logging the output to the backup.out file so I can inspect it later, if I want.
All in all it took about 2 minutes to set up, and now it runs forever without any user intervention, and it didn't require installing or setting up any software (any Linux distro should have rsync installed by default). You shouldn't just blindly implement what I've done here without understanding what it's doing, but it could give you a baseline to work off of.
I am very much against using the anonymous "cloud" for storage or backups. These are servers run by unknown companies in unknown locations who have unknown agendas. Who knows what they're doing with your data or if they'll even be around tomorrow or the next day. There have been countless cases of companies setting up cloud storage for people to either store data on or contribute to, then one day the company decides that everything that has been uploaded is now theirs to do with what they wish, they lock out their users, and then start charging for access. Other companies set up systems like this and then go out of business and the entire network disappears overnight. Other companies one day decide "meh, we're done" and shut down your access with no warning, permanently. Do you really want to base your entire backup system on the whims of some exec who's answering to shareholders and could decide at any moment to lock you out, delete your data, sell your data, or shut down?
Screw that.
You can buy an external hard drive for cheap, and set up a ONE LINE script to do regular backups, and never have to worry about it again (at least until the external hard drive fails some years down the road and you replace it).
Thanks for the calm, considered explanation of how this all works. And the relatively maintenance-free running sounds fantastic. I will of course have to formulate my own backup command and buy an external hard drive. Which might take a while to do, but one of these days. And I agree with you about the vulnerability of cloud storage. I tend to think of my stuff as not being earth shatteringly valuable. But yeah, I wouldn't like any of those scenarios you suggested. I have two computers I regularly use, but I suppose I would only need to backup one of them. (One of them seems to have most of what is important to me on it.) And I see Newegg is always having external hard drives on sale. One thing I'm not entirely convinced of though, is if I'm really mostly concerned about my novels, which would be easily saved on a flash drive, do I really need to back anything else up? Couldn't it just as simple as saving the novels, and then if the computer crashes reinstalling Xubuntu 13.10 and the software I was using?
Thanks for the calm, considered explanation of how this all works. And the relatively maintenance-free running sounds fantastic. I will of course have to formulate my own backup command and buy an external hard drive. Which might take a while to do, but one of these days. And I agree with you about the vulnerability of cloud storage. I tend to think of my stuff as not being earth shatteringly valuable. But yeah, I wouldn't like any of those scenarios you suggested. I have two computers I regularly use, but I suppose I would only need to backup one of them. (One of them seems to have most of what is important to me on it.) And I see Newegg is always having external hard drives on sale. One thing I'm not entirely convinced of though, is if I'm really mostly concerned about my novels, which would be easily saved on a flash drive, do I really need to back anything else up? Couldn't it just as simple as saving the novels, and then if the computer crashes reinstalling Xubuntu 13.10 and the software I was using?
Sure, there's no reason you have to back up everything. In the rsync command, just change the "/*" to "/home/greggbell" or whatever directory you want to back up (and get rid of the unnecessary excludes in the process).
You could also have your second computer back up to the first, and then the first backs up to the external drive. I didn't mention it in my post, but that 3.3TB I'm backing up includes the regular backups from 3 other computers, one Linux, one Windows, and one OSX. So the external hard drive actually holds the backups from four machines, plus two iPhones and an iPad, all of which is being regularly synced to the external drive using that one script I posted earlier (plus similar scripts or settings on the other machines to back them up to the server).
The overall layout is:
portable devices back up to machine A
machines A-C back up to the server (machine D)
machine D backs up to the external drive using the script I posted earlier
So everybody has a backup. Some have two, and the portable devices actually have three.
Last edited by suicidaleggroll; 03-20-2014 at 05:55 PM.
I also do some writing and that is the bulk of what I back up. That and photos I don't want to lose. My main concern is security of my personal documents on a cloud servers but 1.) I figure Google and Dropbox are a lot better at security than I am since they are multibillion dollar businesses whose entire business involves them being able to secure their and my data 2.)Nobody wants to hack in and steal any of my writing
I like being able to wipe my entire operating system and computer and still be able to access my important stuff and also like to be able to access it from different computers with internet connections
Thanks Ryan, Would you clear up some of your terminology. What (in your second to last post) did you mean by: Drive and Dropbox "both having much more redundancy then I can afford to have locally." And I hear you about the convenience of accessing the cloud storage from anywhere. (I still think keeping stuff on a flash drive or two is a good idea as well.) And what did you mean by "I like being able to wipe my entire operating system and computer"? "Wipe"? Why? How? Thanks.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.