[SOLVED] Best way to do image based backups on linux
Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I would just like to know what people use as an image based solution to backing up Linux. Not really specifying a distro, just would like to know what people use / philosophies.
If I really have to make a complete image I use dd in conjunction with a compressor, like bzip2:
1. Write zeroes to the unused parts of a disk, using dd to write zeroes to a file until the disk is full.
2. Use dd to read the partition, pipe the output to the compressor and redirect it to a file on the backup medium.
Since any distro comes with dd and i can choose the compressor this is the most distro agnostic approach, I would think.
Of course there are other approaches with more functionality, like Clonezilla, but I prefer the simple things.
Never been one for image backups. I prefer to take a set of tar archives, one per filesystem, that I can use to restore the complete system state, or just individual files/filesystems as necessary. Combine those with the saved output of "sfdisk -d", "vgcfgbackup", and "cryptsetup luksHeaderBackup" and it should be possible to restore everything to its current state from scratch, using only the tools on my slackware installation media.
I can't say I have any good choice. I long for what I perceive as the good old days of ghost.
In the real world we only have a few ideas. One is a bit by bit copy that I use a lot. It has many faults and tips and tricks with it but tried and true.
Second is the file by file method. Many tools can copy off files and a few apps and live cd's work well with this method but again, not me(fool)proof
I work on so many systems and OS's that I use what I can in each place, even if it isn't the best I use what works.
Distribution: Fedora (typically latest release or development release)
Posts: 372
Rep:
I have tried with clonezilla. But, now-a-days, I do not image based backups anymore. I just rsync to my external HDs. perhaps, it helps that I run no applications that I cannot reinstall. I do backup configuration files as well.
You could try www.mondorescue.org; does a warm backup as opposed to Clonezilla which I believe requires a shutdown as it does a cold backup.
Depends on your requirements.
Databases need special handling if not done cold.
Many (most) prod systems can not be shutdown, so a combination of Having the OS install disks, then something like eg Mondo or NetBackup https://en.wikipedia.org/wiki/NetBackup, with DB hotbackup techniques is usual.
Of course if you have hot swap mirror disks, that works too
Im curious about the DD along with bzip2, how would one go about restoring?
The reason why im asking, is that in a complete disaster scenario, I would like to restore the system "at a point in time" and let it resotre an image, than take the time to copy the files, because in a disaster, there are probably other things I need to do while its restoring.
Ill also take a look at mondo rescue, seems like a flexible solution as well.
Im curious about the DD along with bzip2, how would one go about restoring?
Take the image (asuming that the partition in question is /dev/sda1 and the backu medium is /dev/sdb1):
1. Boot from a live medium.
2. Mount the partition to image:
Code:
mount /dev/sda1 /mnt
3. Fill the unused parts of the partition with zeroes, so that the compressed image is smaller:
4. Unmount the backup medium (see step 6 above) and you are done.
You may want to also make a copy of the output of
Code:
fdisk -l
so that you can have information on the exact partition size if needed.
Note that I use a blocksize of 16MB for the dd operations, because I found that blocksizes in the range between 8MB and 32MB (depending on your hardware) give me in general a better performance than the default value.
and it did the trick, even without writing zeroes, it only backed up the actual usage of the partitions.
also, i didnt specify the partitions of the disk, I just took the whole drive (sda instead of sda1)
When restoring, i ran your other command back to an un-partitioned hard drive with an Ubuntu 12.04 Desktop live CD and it worked, it restored the partitions as well.
Zeroing out the free space first allows for a smaller compressed backup. Also consider using ddrescue which might be faster. It will retry if it reaches some bad blocks. It also varies the block size depending on the health of the drive. Bad blocks are zeroed out in the image. Using dd without options will simply fail.
Do you mean you have made an image of the running system? This is not reliable, since the file-systems can change during the backup, which will leave you with an inconsistent backup. Backups that are possibly inconsistent are worthless.
Quote:
Originally Posted by jschiwal
Also consider using ddrescue which might be faster. It will retry if it reaches some bad blocks. It also varies the block size depending on the health of the drive. Bad blocks are zeroed out in the image. Using dd without options will simply fail.
I disagree. Of course you can use ddrescue, but if you have to use it because of a faulty disk it is already to late for a backup, the data is already potentially damaged. If there are already bad blocks on the disk you are not making a reliable backup anymore, you are creating an image for rescue purposes.
Distribution: Ubuntu 12.04 with KXStudio, MATE & Compiz
Posts: 46
Rep:
Thanks: I was wondering, why the zeros?
Quote:
Originally Posted by TiMMay333
I would like to restore the system "at a point in time" and let it resotre an image, than take the time to copy the files, because in a disaster, there are probably other things I need to do while its restoring.
That's not really a valid comparison. If you have a tar backup, the backup itself is one file, and can be restored with just one command. As it works at the file level, not the block device level, it gives you enormous flexibility to restore all or part, and to do so to any location (assuming that the backup was made using ./this/that/the-other relative paths) on any device or any point within the directory tree[s].
dd is a lower-level technical tool for when you need a bit-identical, not data-identical, copy. Which I suppose is what "image" means <Blush>. dd will also do stuff like read or write a given number of bits, convert block sizes, etc.
Tar is your swiss-army knife: dd is your leatherman multi-tool.
The problem with running dd on a live file system is that things can change while the backup is running e.g. changes to log files. Also you will backup the the contents of /sys and /proc which are automatically generated at boot.
Personally, I like dd images of my Windows machines. If you restore from the image after a virus invasion it is comfortable to know that the sucker has been completely eliminated from the disk.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.