Backing up home directory
 

If I use the console command rsync to back up my entire Home directory to an external USB drive, can I do this while I am signed on to Ubuntu in my own user name (the same name as the Home directory I am copying)? The only occasion I did this before (although not with rsync), I had all sorts of problems with files not being copied because they were in use. In the end I did it using a live DVD, but when I went to use the backups recently, had more problems because some files/folders had been changed to Root ownership.

I was just worried in case backing up my Home directory while I am signed into it, will cause some files to be "skipped over" rather than copied.

Or should I run the rsync command as Root, but still signed on as me? Or would that cause more problems?

It seems such a waste to use a backup application when rsync (it would appear) can do the job just as well - if I can get it to work!

use the -a option to maintain ownership, etc.

You could use a lvm snapshot to save the momentary state of your home directory. Then you can mount that snapshot and copy it without fear that a file might be opened elsewhere.

Noted, but can I use it in a shell as an ordinary user, to copy my own Home directory while I am in it?

Sorry, my Post #4 and brianmcgee's post #3 overlapped. I'll have to look up lvm snapshots and see if they can be of use to me. Thanks.

What are you trying to back up? If you don't care that much about gnome settings or whatever, make sure you've got anything important closed and stick with rsync

Well, just the entire contents of my /home directory. I am aware that all the settings are stored in hidden files, and contain any preferences I've set for my software, so I would like to keep those in the event of a system failure, so when I reinstall the software I'll still have my settings and preferences. Does rsync copy everything in the folder, regardless of it being hidden?

Yes, unless you specify it not to.

One small point:
I always prefer to backup my files whilst they are not being used.
Think about it.

So to backup /home/tred I logout, <CTRL><ALT><F1>, login as root, make the rsync backup of /home/tred
Then logout from being root, and continue as normal.

Same goes for backing up the system files: I boot from another distro, mount the / partition I want to backup, copy the files (as root), unmount the partition etc.

The idea of not being able to copy a file because it is open is a Windows concept.

In Linux, if you have permission, you can copy the file even if it is open. So, using rsync while logged in is not a problem. The issue you might have is that rsync could immediately be out of sync, if you create or delete a file while rsync is running. Even this isn't a huge deal; you'll still get your backup.

I agree. But consider the following situation:

I am making a backup of my home whilst logged in.

An email arrives. It is filed (and backed up), but the mail program's mail.index file has already been backed up.
So now there will be an application / filesystem inconsistency when the backup is restored.
I am sure you can think of many other "race" conditions.

Maybe there is a "lock" file created, because I am logged in, and a certain application is running and busy. When the backup is restored, obviously I won't be logged in as myself, the application will not be running, and the lock file should not be there: Something will fail.

So what I have is not so much a "backup" (which I'd like to have) as a "snapshot" (which I don't).

When I restore from a backup, I want to be back exactly where I was when the backup was taken. Not with a load of inconsistencies that I have to track down and correct. I require 100% reliability from my backups, and this philosophy has provided it (it's been tested more than once) so far.

Please correct me if I have misunderstood something. Maybe there's an easier way?

As I mentioned in my earlier post, if you create/delete a file when rsync is running, your backup could be immediately out of sync.

If you want a login lock file, create one. Then have your backup script check for the existence of this login lock and behave appropriately. You can put your lock file in /tmp or in /var/run...whatever floats your boat (I would use /tmp). You also could just run the who command to see who is logged in. If your ID shows as logged in to screen 0, don't run the script.

That is exactly the situation I am trying (so far successfully), to avoid. If things are "out of sync" when I restore, I am not going to be happy.

This is why I am recommending backing up only unmounted partitions or unused directories if you want to be able to restore from your backup, and have a fully functioning system after the restore.

You said:I disagree, most of your data files (Docs, MP3s etc.) will be OK, but the integrity of your system and application configuration files are not guaranteed with your backup strategy.

YMMV: you are warned, but can do as you please.

What you are suggesting really doesn't make a lot of sense in that at any time you log in and do anything, your backup will be out of date. The only way around this would be full replication, which frankly, is pretty serious strategy for backing up a few mp3's


If you are seriously worried about interim data loss, set up a cron job (or if it's a workstation perhaps an anacron jog), and rsync your home directory every 5 minutes (I do it every hour). Rsync only copies changes, so the load won't be horrible.

The biggest issue I see with a shutdown/dismount or logout, logon as root type strategy is that you'll find that you simply don't do it that often. Would you prefer to lose an email you got 5 minutes ago or everything form the lat hour/day/week?

You can re-create mail indices and the like (if you can't, change your mail client).

Well, there's certainly been an exchange of views on this thread since I logged off yesterday. For my purposes, having an "up-to-the-minute" correct backup is less important (I'll only back up once a week maximum), and if my system goes down and I am forced to use the backup, the fact that I would not have lost everything is more important than having lost a few files that I could easily retrieve from the internet again (work files I back up straight away on removable storage anyway).

What is more important to me is not to be thinking stuff is backed up when it hasn't been. For instance, last night I did re-format my backup drive as VFAT, and backed up my home directory again. I used VFAT because before, I formatted it as ext3 and it gave me so much trouble reading the directory listing and accessing the files. I'm not sure why, unless that backup had become corrupted in some way since making it. I did notice, though, that certain file naming conventions are more flexible in ext3 than VFAT, and some files were not copied across unless I renamed them first (a colon in the filename, for instance).

So ultimately I might have to reformat as ext3 and try again.

With regard to Tredegar's comment about Ctrl-Alt-F1, login as root, to get around not copying files already in use: I've never done this "console" approach - is logging in as root in this way just involve typing sudo su, then my own user password, and that will log me in as root? Ubuntu does not have "root" accounts, as you probably know. In fact, I've yet to use a distro that does: Knoppix has a "root shell" that you go into to gain root privileges, instead of the ordinary shell.