LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-22-2008, 08:09 AM   #1
Steve W
Member
 
Registered: Mar 2007
Distribution: Ubuntu 12.04, XUbuntu 12.04
Posts: 504

Rep: Reputation: 44
Backing up home directory


If I use the console command rsync to back up my entire Home directory to an external USB drive, can I do this while I am signed on to Ubuntu in my own user name (the same name as the Home directory I am copying)? The only occasion I did this before (although not with rsync), I had all sorts of problems with files not being copied because they were in use. In the end I did it using a live DVD, but when I went to use the backups recently, had more problems because some files/folders had been changed to Root ownership.

I was just worried in case backing up my Home directory while I am signed into it, will cause some files to be "skipped over" rather than copied.

Or should I run the rsync command as Root, but still signed on as me? Or would that cause more problems?

It seems such a waste to use a backup application when rsync (it would appear) can do the job just as well - if I can get it to work!
 
Old 08-22-2008, 08:24 AM   #2
billymayday
LQ Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
use the -a option to maintain ownership, etc.
 
Old 08-22-2008, 08:28 AM   #3
brianmcgee
Member
 
Registered: Jun 2007
Location: Munich, Germany
Distribution: RHEL, CentOS, Fedora, SLES (...)
Posts: 399

Rep: Reputation: 39
You could use a lvm snapshot to save the momentary state of your home directory. Then you can mount that snapshot and copy it without fear that a file might be opened elsewhere.
 
Old 08-22-2008, 08:28 AM   #4
Steve W
Member
 
Registered: Mar 2007
Distribution: Ubuntu 12.04, XUbuntu 12.04
Posts: 504

Original Poster
Rep: Reputation: 44
Noted, but can I use it in a shell as an ordinary user, to copy my own Home directory while I am in it?
 
Old 08-22-2008, 08:32 AM   #5
Steve W
Member
 
Registered: Mar 2007
Distribution: Ubuntu 12.04, XUbuntu 12.04
Posts: 504

Original Poster
Rep: Reputation: 44
Sorry, my Post #4 and brianmcgee's post #3 overlapped. I'll have to look up lvm snapshots and see if they can be of use to me. Thanks.
 
Old 08-22-2008, 08:37 AM   #6
billymayday
LQ Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
What are you trying to back up? If you don't care that much about gnome settings or whatever, make sure you've got anything important closed and stick with rsync
 
Old 08-22-2008, 08:41 AM   #7
Steve W
Member
 
Registered: Mar 2007
Distribution: Ubuntu 12.04, XUbuntu 12.04
Posts: 504

Original Poster
Rep: Reputation: 44
Well, just the entire contents of my /home directory. I am aware that all the settings are stored in hidden files, and contain any preferences I've set for my software, so I would like to keep those in the event of a system failure, so when I reinstall the software I'll still have my settings and preferences. Does rsync copy everything in the folder, regardless of it being hidden?
 
Old 08-22-2008, 10:34 AM   #8
trickykid
LQ Guru
 
Registered: Jan 2001
Posts: 24,149

Rep: Reputation: 235Reputation: 235Reputation: 235
Quote:
Originally Posted by Steve W View Post
Does rsync copy everything in the folder, regardless of it being hidden?
Yes, unless you specify it not to.
 
Old 08-22-2008, 12:12 PM   #9
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,087

Rep: Reputation: 407Reputation: 407Reputation: 407Reputation: 407Reputation: 407
One small point:
I always prefer to backup my files whilst they are not being used.
Think about it.

So to backup /home/tred I logout, <CTRL><ALT><F1>, login as root, make the rsync backup of /home/tred
Then logout from being root, and continue as normal.

Same goes for backing up the system files: I boot from another distro, mount the / partition I want to backup, copy the files (as root), unmount the partition etc.
 
Old 08-22-2008, 12:32 PM   #10
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 115Reputation: 115
The idea of not being able to copy a file because it is open is a Windows concept.

In Linux, if you have permission, you can copy the file even if it is open. So, using rsync while logged in is not a problem. The issue you might have is that rsync could immediately be out of sync, if you create or delete a file while rsync is running. Even this isn't a huge deal; you'll still get your backup.
 
Old 08-22-2008, 01:13 PM   #11
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,087

Rep: Reputation: 407Reputation: 407Reputation: 407Reputation: 407Reputation: 407
Quote:
The idea of not being able to copy a file because it is open is a Windows concept....
I agree. But consider the following situation:

I am making a backup of my home whilst logged in.

An email arrives. It is filed (and backed up), but the mail program's mail.index file has already been backed up.
So now there will be an application / filesystem inconsistency when the backup is restored.
I am sure you can think of many other "race" conditions.

Maybe there is a "lock" file created, because I am logged in, and a certain application is running and busy. When the backup is restored, obviously I won't be logged in as myself, the application will not be running, and the lock file should not be there: Something will fail.

So what I have is not so much a "backup" (which I'd like to have) as a "snapshot" (which I don't).

When I restore from a backup, I want to be back exactly where I was when the backup was taken. Not with a load of inconsistencies that I have to track down and correct. I require 100% reliability from my backups, and this philosophy has provided it (it's been tested more than once) so far.

Please correct me if I have misunderstood something. Maybe there's an easier way?
 
Old 08-22-2008, 01:39 PM   #12
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 115Reputation: 115
As I mentioned in my earlier post, if you create/delete a file when rsync is running, your backup could be immediately out of sync.

If you want a login lock file, create one. Then have your backup script check for the existence of this login lock and behave appropriately. You can put your lock file in /tmp or in /var/run...whatever floats your boat (I would use /tmp). You also could just run the who command to see who is logged in. If your ID shows as logged in to screen 0, don't run the script.
 
Old 08-22-2008, 04:00 PM   #13
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,087

Rep: Reputation: 407Reputation: 407Reputation: 407Reputation: 407Reputation: 407
Quote:
As I mentioned in my earlier post, if you create/delete a file when rsync is running, your backup could be immediately out of sync.
That is exactly the situation I am trying (so far successfully), to avoid. If things are "out of sync" when I restore, I am not going to be happy.

This is why I am recommending backing up only unmounted partitions or unused directories if you want to be able to restore from your backup, and have a fully functioning system after the restore.

You said:
Quote:
Even this isn't a huge deal; you'll still get your backup.
I disagree, most of your data files (Docs, MP3s etc.) will be OK, but the integrity of your system and application configuration files are not guaranteed with your backup strategy.

YMMV: you are warned, but can do as you please.
 
Old 08-22-2008, 06:36 PM   #14
billymayday
LQ Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
What you are suggesting really doesn't make a lot of sense in that at any time you log in and do anything, your backup will be out of date. The only way around this would be full replication, which frankly, is pretty serious strategy for backing up a few mp3's


If you are seriously worried about interim data loss, set up a cron job (or if it's a workstation perhaps an anacron jog), and rsync your home directory every 5 minutes (I do it every hour). Rsync only copies changes, so the load won't be horrible.

The biggest issue I see with a shutdown/dismount or logout, logon as root type strategy is that you'll find that you simply don't do it that often. Would you prefer to lose an email you got 5 minutes ago or everything form the lat hour/day/week?

You can re-create mail indices and the like (if you can't, change your mail client).

Last edited by billymayday; 08-22-2008 at 06:38 PM.
 
Old 08-23-2008, 03:56 AM   #15
Steve W
Member
 
Registered: Mar 2007
Distribution: Ubuntu 12.04, XUbuntu 12.04
Posts: 504

Original Poster
Rep: Reputation: 44
Well, there's certainly been an exchange of views on this thread since I logged off yesterday. For my purposes, having an "up-to-the-minute" correct backup is less important (I'll only back up once a week maximum), and if my system goes down and I am forced to use the backup, the fact that I would not have lost everything is more important than having lost a few files that I could easily retrieve from the internet again (work files I back up straight away on removable storage anyway).

What is more important to me is not to be thinking stuff is backed up when it hasn't been. For instance, last night I did re-format my backup drive as VFAT, and backed up my home directory again. I used VFAT because before, I formatted it as ext3 and it gave me so much trouble reading the directory listing and accessing the files. I'm not sure why, unless that backup had become corrupted in some way since making it. I did notice, though, that certain file naming conventions are more flexible in ext3 than VFAT, and some files were not copied across unless I renamed them first (a colon in the filename, for instance).

So ultimately I might have to reformat as ext3 and try again.

With regard to Tredegar's comment about Ctrl-Alt-F1, login as root, to get around not copying files already in use: I've never done this "console" approach - is logging in as root in this way just involve typing sudo su, then my own user password, and that will log me in as root? Ubuntu does not have "root" accounts, as you probably know. In fact, I've yet to use a distro that does: Knoppix has a "root shell" that you go into to gain root privileges, instead of the ordinary shell.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
contents of 'home' directory found at sda5: mounted as /home, and also on sda6:/home leswatson Linux - Newbie 4 04-18-2008 05:02 PM
Backing up home folder? evilc77 Linux - Newbie 6 03-09-2008 10:56 PM
Question about backing up home directory IndyGunFreak Ubuntu 2 02-04-2007 02:19 AM
Backing Up /home carlosinfl Linux - General 3 02-16-2006 01:04 PM
Backing up /home une Mandriva 8 07-10-2005 08:38 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:44 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration