Software to clone disk on headless server?
 

I have been searching in vain for software to clone the entire HDD on a headless Fedora 10 server with six partitions. I want to be able to read files from the clone drive on another computer (likely Windows), so I don't want it compressed. It would also be nice if the clone drive was bootable, so I could install it in the server in the event that the server drive failed. I will store the backup drive offsite.

I have an external eSATA/USB drive dock attached to the server and I would prefer to use eSATA as the interface. Because the server is headless, I can't sit at a local keyboard to run the backup.

I need a disk cloning solution that could run from a VNC client or a script on a USB stick. Unfortunately I am a noob, so I have limited ability to create a script.

Suggestions?

You could use clonezilla: http://www.cyberciti.biz/tips/downlo...ne-system.html

Or, you could just use dd: http://www.brandonhutchinson.com/Disk_duplication.html

I would recommend the later. After all, you already have it. :)
One of the most frustrating and also coolest things for me about learning Linux is searching for what I need only to find that it is part of my distribution. It used to happen to me every day, and it still often does. "Oooh, that's what that does . . ." is something I never said so much.

If your source disk is /dev/MyDisk0 and destination is /dev/MyDisk1
This is an awesome quote, btw. You are an awesome noob, dude. Worry not. That thought has gone through my head SO many times, and still does. "Hmm . . . I'll bet there's a way to <insert something I don't know how to do> with <insert something I know very little about>. Unfortunately, I am a noob."

I looked at clonezilla, but because I am backing up the server, I need to use the live version. The live version needs a keyboard and display to run the disk image. I couldn't find a way to just plug in a USB flash and have it perform the backup automatically and restart the system.

I could run dd from a remote session (VNC or putty), but I read that the files would be screwed up if the system is running when the image takes place. If I unmount the filesystem, won't I lose the remote session?

Ideally, I would like to create the clone while the system is running. Is there software that will do that?

If not, is there software that will run in batch mode that will create a disk image and then restart the system?

If you were that concerned about the possibility of corruption because the system is running and accessing the disk frequently, I'd wait until the lowest traffic time possible and drop the system into single-user mode ('init 1') and then execute dd.

You should at least try doing it with dd while it is running. One thing that it would not do is hurt the source drive.

But, if he drops to init 1, isn't he going to lose networking,
and therefore his remote login session?

Create the partitions on the new drive and rsync the partitions one by one. That might require some --include= and --exclude= options. Practice this a bit on your local workstation.

Exclude the /proc and /sys directories, although you should create them on the new drive. Do not exclude /dev. You might want to exclude externally mounted drives as well.

After this is done, make the disk bootable by copying the first 446 bytes (and no more) from the old disk to the new one:
Assuming sda is the existing drive and sdb the new one.

Last time I did this was to clone a running production server to new hardware, 8000 km from where I live. Needless to say ot just worked at a reboot.

jlinkels

For Red Hat and Fedora, LVM2 is normally used. It would have a /boot partition, a swap partition and an LVM logical volume. The LVM volume could be on the remainder of the disk or composed of the remainder of the disk, plus other disks.

If you create a dump or image backup, you will want to backup the logical volume and not the physical disks. LVM has a feature where you can have a mirror of a logical volume.
This webpage has a good overview.
http://www.redhat.com/magazine/007ma...tures/storage/

If you save a snapshot or an LV image or a disk image to a file, you would want to use Linux to pull files from it, not Windows.

If you aren't using LVM, and create images of the drive to a file, you can use the "losetup" command to mount a partition that is inside the disk image. If you create an image of a Logical Volume, you can do the same. Using "fdisk -lu" to list the partition table of the disk, and printing it out, you can record what offsets to use to mount each partition. After mounting the partition, you could share the mountpoint using Samba and access files on it from a Windows machine on the network.

Ok here's an update:

I used dd to create an image of my system disk on an eSATA drive using VNC. At first it was only transferring at 10MB/s and the flat out speed of the drive is 110 MB/s. Traced the problem to the eSATA controller, changed the drive to another port, and now I get 50+ MB/s. That's probably as good as it gets for the whole process.

I found an article on the web with tips for dd and it described how to determine the best bs= by running some trial runs. Determined that 8096 was the fastest. So this created what appears to be a bootable image:

dd if=/dev/sda of=/dev/sdb bs=8096

I am able to disconnect the drive and read the ext3 partitions on my Windows machine using Ext2 Volume manager(ext2fsd). I haven't tried booting the from the image yet.

The only problem so far is that it takes over five hours to clone the drive (1TB). Since the system is primarily storing static data (music library and desktop backups), a differential backup appeals to me, or at least skipping unused blocks.

I'm not currently using LVM and it looked way too complicated for me, so I don't want to set it up. Even rsync is looking pretty complicated.

I tried rsync to copy partitions, but I can't get it to work. /dev/sda and /dev/sdb are not valid directories (tried slash on the end too). I am guessing that rsync wants the filesystem name for the partition, not the device name. Since the filesystem names on the image drive are the same as the source drive, how do I tell rsync which drive is which? I don't think the the image drive is mounted, because I don't think dd needs it mounted to work.

Your assumptions about dd vs rsync are correct. DD copies on a lower level, can use devices and devices need not to be mounted.

Rsync OTOH operates on file system level, which means it uses file names, directories names etc, and is unable to copy devices (/dev/sda) or partitions (/dev/sda1)

In order for rsync to work you must have both file systems mounted. If you want to copy the entire primary drive to the secondary (say /dev/sda to /dev/sdb), you would have to:

• partition /dev/sdb
• mount each partition in the root tree, say in /dev/sdb1 -> /mnt/sdb1, /dev/sdb5 -> /mnt/sdb5, etc
• copy each partition to the partitions of the secondary drive

Say that you have separate partitions for /, /home, /usr and /var on sda1, sda5, sda6 and sda7 /dev/ you would have to do something like this:
This is a bit awkward with excludes etc., and I am sure better solutions exist but I haven't discovered them yet. If you do it like this keep and eye on it for the first time to see if you have the exclude patterns right, otherwise you end up with copying everything on /dev/sdb1 which isn't right.

I think you are doing just fine with using DD for the initial copy. If you want to make incremental copies, rsync might be better.

jlinkels