How do you all deal with frequent backups of Virtual disk image files for MSWIN guests?

I've run Virtualbox on several machines with a Slackware host and MSWIN guest for a couple of years without problems using disk image files (VDI). Some machines are servers running application specific services in Microsoft Windows.

There have been some annoyances however, one is having to deal with a large files (120GB, 150GB & 300GB) on backups.

I use RSYNC for backups and typically backup (mirror) the server customer data once an hour and then every few days swap it with devices in a bank safety deposit box. Of course only the blocks changed are written so it goes quickly, but takes longer when the backup device is swapped.

This hourly RSYNC backup doesn't work well however for for the large Virtualbox disk image files where one minor MSWIN file update changes the last modification time and thus causes the entire image file to be inspected by RSYNC.

One change I'll investigate is to make is to do a better job of separating the MSWIN active data from the less active data (more image file disks). There are limits however to how small I can make a contiguous disk area and still hold the needed data file group.

Using dynamic instead of fixed size image files could help some by making the image files smaller, but would I pay a price in performance and possibly increased fragmentation?

Do you just use a MS software solution to the backup instead of the host OS? (e.g. external "Click Free" drive)

Will using Virtualbox raw disk solve the RSYNC backup problem or simply create new problems to deal with? I would only be able to give Virtualbox encrypted RAID block devices, not an an entire real "raw" disk.

I don't need/use the convenience of moving VDI files between machines. I just need to run Microsoft software on Linux machines also providing other services.

Thanks for your ideas/experiences.

Dynamically allocated virtual hard drives only suffer performance if you have a very slow drive or a loaded drive. I doubt you'd have to worry too much about fragmentation.

I'd use the Technet pages for all windows. Long gone are the days where backup is bad. Depending on the version of windows would determine how you can use a backup. Either ntbackup with system state and then incremental might work. The newer OS's use a type of virtual hard drive but in both cases you have other features like fast speeds, and live state.

Thanks for the response jefro.

I found the Technet site you mentioned. I'll have to browse the site for information. I haven't done much with MS Windows for years so I typically don't search for MS Windows solutions online. I'll look into the native MS backup solutions.

I'm using MS WIN7 now so it should be modern enough for the features you mention ... but this is not server software. Only Home Premium is needed for the tasks I use. The servers are very lightly loaded except that the file transfers (Linux not MSWIN) are often quite large.

If you bring the vm down you can also just copy the virtual hard drive but it does make it a large file. It may be a good thing to have for a golden image however.

Excellent suggestion. Every couple of months I manually copy items not automatically backed up, including stopping the VM and copying all the large VM disk image files including the MSWIN OS. I keep these files for a year along with the original when the VM guest was first installed.

It works but it's manual. Doing manual backup tasks every month or two is acceptable to me as long as the data is backed up daily.

Right now for data backups I have the application server software create a backup of the customer data (so it deals properly with the running database) to a disk (shared with Linux using the VM share option) not used for anything else. Then I have Linux automatically backup that non-VM disk when the application backup program isn't running. The VM is still active and the shared disk is still connected but nothing is happening on that disk. Because of disk space constraints on one machine I use the -inplace option with RSYNC. I'm fixing that disk space issue soon.

This all works for me ... except to backup all the other non-database-application data files I'm dealing with large files (and end up backing up stuff that doesn't change) due to the nature of handling the MSWIN files at the level of the disk image. As I mentioned I'll look into what MSWIN has to offer for backups.

I'm just looking to becoming a little more sophisticated with my use of Virtualbox and am exploring alternatives for backup and performance. I've not yet used the raw disk option with VirtualBox. In my brief experiments last week I was NOT successful in using encrypted RAID block devices as a raw device for Virtualbox (e.g. /dev/mapper/cryptdata built on top of /dev/md1). Encrypted RAID is how my machines are normally set up. I plan to start a new thread about this in the Slackware forum in a few weeks if I can't figure it out on my own.

I find that I only need a backup when something breaks. Otherwise it is a waste of time.

When you say raw do you mean you are using a vm like qemu and a raw virtual disk file as opposed to a qcow type format?

There is however a positive corelation with the severity of a computer problem and the age of the most recent backup. Recent backups keep problems away.
I've relatively inexperienced with VMs. I've only used Virtualbox so I'm not familiar with qemu.

By raw disk usage in Virtualbox I'm referring to using a real disk instead of disk image file. That's the term that Virtualbox uses. In Virtualbox the command to create a link to such a disk is something like this for full disks ...
or this for partitions ...
My understanding is that these commands create a special disk image (.vmdk) that points to the physical device.

At least that's what I understood from reading chapter 9 of the Virtualbox manual. As I mentioned I'm not yet using "raw disks" in Virtualbox. I briefly tried to create the .vmdk file pointing to an encrypted RAID block device but was unsuccessful.

I've just been cruising along the last couple of years using Virtualbox in a very basic manner but hope to expand my knowledge soon.

One thing I found with VirtualBox was that it was not possible to use a disk image from one host on another -- so simply backing up the .vid files was useless as a backup as they will not boot on another machine. I have no idea why this is the case and I was frankly appalled to find it out but I know it would prevent me from using VirtualBox in a production environment. In order to back up VMs so that they can be opened on another machine it is necessary to "export" them though that isn't guaranteed to work if you try to export a few VMs at a time.

When I read the documentation a couple of years ago I believe I came across some special import/export processing needed to move a VDI file.

My use of a VM is very simple. I don't have to move them around to different machines so I've not explored this aspect. I just need to run some MS software on a Linux machine.

EDIT: I believe this Virtualbox manual chapter discusses this disk image moving limitation.

I don't understand why you would want to keep a backup that you can't use though? If the drive you keep your VM images on dies then your backups become useless thanks to the VirtualBox developers' bizarre decision to tie them to the partition.

I see what you are saying now. I may be in for an unpleasant surprise!

I have restored a backup of a disk image before but I was resizing the disk image file size and then restoring it to the same VM on the same machine.

I'll have to investigate this more. I need to find out the limitations and maybe change my process.

Thanks for pointing this out.

No worries. A friend recently upgraded his PC and found out the hard way the limitations of VirtualBox -- thankfully he kept the old hard drives so he could export and import his old XP image. I am perhaps being a little harsh on the VB developers but to me this limitation is brain-dead.

On most vm's that I've used, it knows when you use a clone. It asks what to do.

VM's are generally portable but as with any clone or copy, uuid or disk-by-name or others need to be corrected or made generic.

Plenty of pre-made virtual appliances out there.

I've found with a Linux guest the virtual drive image seem fairly portable but for Windows that isn't the case. That makes it very problematic to back up a VM and means that the only real way to do so is to run a lengthy and space-consuming export every time you wish to back up the VM. To me, at least, that is a very very bad design decision and from what I can gather there's no official "solution" either. I've not investigated VMWare in this regard though so perhaps it is just the way things are with VMs -- seems pretty silly if it is though as it almost defeats the object.

1 members found this post helpful.