help: best ways to create and restore pseudo-disk-image backups ???
 

I'm not familiar enough with how the various backup approaches work to know whether I have a serious problem or a trivial one.

I want to create a backup image of my 1TB ubuntu64 10.04 disk drive for at least two purposes:

#1: So I can restore the drive to its current state if the filesystem gets damaged.

#2: So I can create an identical installation on another hard disk drive. Specifically, I have a new 3TB drive, and I'd like to make a functionally identical copy of the ubuntu64 installation on my 1TB drive.

Here are possible catches and unknowns for me.

#A: The 1TB drive has an "msdos" style partitions, while the new 3TB drive has "gpt" style partitions. I'm not sure whether "gpt" style partitions are required because the drive is larger than 2.2TB, but if not, I'm quite happy to go ahead and re-partition the drive with "msdos" style partitions as long as I don't lose that extra 0.8TB.

#B: The 1TB drive has 512 byte physical sectors and the 3TB drive has 4TB physical sectors. Therefore, it is obviously not acceptable to create the clone installation on the 3TB drive via a sector-by-sector copy.

#C: I don't want to make backups of empty sectors like a complete image does. Whatever does this backup should only read, save and restore real file contents. The 1TB drive is only about 30% full.

#D: It would be nice if the software can handle slight partition differences on the backup and restore drives. For example, the 1TB drive that I want to backup has these partitions ( "/", "/backup", swap) while my new 3TB (restore) drive has these partitions ("/", "/boot", "/backup", swap). But if necessary I can make the new drive have the same partitions, except the "/" partition will be vastly larger, and since the physical sector sizes are different. All the other partitions may be slightly different size [just slightly] since the physical sectors are 512B and 4096B on the 1TB and 3TB drives respectively.

#E: Finally, here is an alternative that would be great if it works, but I'm guessing there's no practical way to accomplish. That would be to create the fresh new installation of ubuntu64 on the 3TB drive from the same DVD that I created the original installation on the 1TB drive... then somehow some application figure out all the files on the 1TB drive that either don't exist yet on the 3TB drive, or have different contents --- and copy them all over to the 3TB drive. I assume that would end up creating a functionally identical installation, including all the software updates that have happened in the past 20 months since I installed ubuntu64 10.04 on the 1TB drive.

I can't believe I've been programming for 30 years without ever taking time to figure out the variations and permutations of backup applications and techniques... but there you go. Bottom line: I want something that is brain-dead simple (no matter how long it takes to execute), not something complex and fancy, or something that runs "under the covers" where I'm not aware of what's happening or when. I want it to run only when I run it, and I don't want to learn any big complex applications. Period.

So, what best fits my wish list, in your not-so-humble opinions?

Hi,

I think you are right. The mismatched sector sizes will probably give you issues using dd or similar. Most likely when the file system structure refers to offsets in the image by sector and misses. Haven't tried it though so take that with a grain of salt.

GPT is required for the table to refer beyond 2.2T. I forget the exact details, but gpt requires a small partition for some gpt boot stuff to live in, so you should see a small extra partition created if you install from DVD. Without this partition, grub will whine about not being able to put its boot up stuff anywhere on such a large drive. From memory, that error was not too clearly worded and required some google work to track down the issue.

Your idea of reinstall and then copy the differences has some merit. The utility you are after for copying the differences is called rsync. If you have a lot of hard links (millions) you can soak up all your ram and fail, otherwise it works quite well after you've worked out which options you want. You probably want to install the 3tb, then boot from the 1tb to do the rsync to avoid writing open files. For this method you don't need similar partition sizes, but you do need to avoid syncing /etc/fstab and similar files or fix them up after. If you sync /boot you probably want to redo any grub stuff and this has bitten me on several occasions This is complicated by having to tell grub that you are configuring in an alternate root location, and I spend a some time each time round trying to figure out how to do that properly so I wont guess to hard here. You are probably looking at utilities like update-grub2, grub-install, grub-probe, grub-setup. If you don't sync /boot and you have done your apt-get up(date/grade)s properly you will end up with the apt database from the 1TB thinking /boot contains the latest kernels while actually having the stock ones from the dvd, so the grub pain is a necessary evil.

HTH
John

I have never liked "image" backups.
Basically you can't have what you want if you just want to sit back with your thumb in your ear.

I prefer filesystem level backups (as per rsync above) - it has several benefits; you know the source is readable, you can selectively restore, you can restore to a different {type,size} filesystem ...
I like fsarchiver as it checksums everything as well. Safer is better.
And yes, you need to fix the boot-loader - no big deal as it's well understood procedure.
Of course all this needs to be done from a liveCD - so no files are open.