LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
LinkBack Search this Thread
Old 07-30-2009, 10:08 PM   #1
ajaxmike
LQ Newbie
 
Registered: Dec 2008
Location: Ajax, Ontario Canada
Posts: 3

Rep: Reputation: 0
Software to clone disk on headless server?


I have been searching in vain for software to clone the entire HDD on a headless Fedora 10 server with six partitions. I want to be able to read files from the clone drive on another computer (likely Windows), so I don't want it compressed. It would also be nice if the clone drive was bootable, so I could install it in the server in the event that the server drive failed. I will store the backup drive offsite.

I have an external eSATA/USB drive dock attached to the server and I would prefer to use eSATA as the interface. Because the server is headless, I can't sit at a local keyboard to run the backup.

I need a disk cloning solution that could run from a VNC client or a script on a USB stick. Unfortunately I am a noob, so I have limited ability to create a script.

Suggestions?
 
Old 07-30-2009, 10:40 PM   #2
foodown
Member
 
Registered: Jun 2009
Location: Texas
Distribution: Slackware
Posts: 607

Rep: Reputation: 218Reputation: 218Reputation: 218
You could use clonezilla: http://www.cyberciti.biz/tips/downlo...ne-system.html

Or, you could just use dd: http://www.brandonhutchinson.com/Disk_duplication.html

I would recommend the later. After all, you already have it.
One of the most frustrating and also coolest things for me about learning Linux is searching for what I need only to find that it is part of my distribution. It used to happen to me every day, and it still often does. "Oooh, that's what that does . . ." is something I never said so much.

If your source disk is /dev/MyDisk0 and destination is /dev/MyDisk1
Code:
dd if=/dev/MyDisk0 of=/dev/MyDisk1 bs=1024k
Quote:
Unfortunately I am a noob
This is an awesome quote, btw. You are an awesome noob, dude. Worry not. That thought has gone through my head SO many times, and still does. "Hmm . . . I'll bet there's a way to <insert something I don't know how to do> with <insert something I know very little about>. Unfortunately, I am a noob."

Last edited by foodown; 07-30-2009 at 10:54 PM.
 
Old 07-31-2009, 09:13 AM   #3
ajaxmike
LQ Newbie
 
Registered: Dec 2008
Location: Ajax, Ontario Canada
Posts: 3

Original Poster
Rep: Reputation: 0
I looked at clonezilla, but because I am backing up the server, I need to use the live version. The live version needs a keyboard and display to run the disk image. I couldn't find a way to just plug in a USB flash and have it perform the backup automatically and restart the system.

I could run dd from a remote session (VNC or putty), but I read that the files would be screwed up if the system is running when the image takes place. If I unmount the filesystem, won't I lose the remote session?

Ideally, I would like to create the clone while the system is running. Is there software that will do that?

If not, is there software that will run in batch mode that will create a disk image and then restart the system?
 
Old 07-31-2009, 09:53 PM   #4
foodown
Member
 
Registered: Jun 2009
Location: Texas
Distribution: Slackware
Posts: 607

Rep: Reputation: 218Reputation: 218Reputation: 218
If you were that concerned about the possibility of corruption because the system is running and accessing the disk frequently, I'd wait until the lowest traffic time possible and drop the system into single-user mode ('init 1') and then execute dd.

You should at least try doing it with dd while it is running. One thing that it would not do is hurt the source drive.
 
Old 08-05-2009, 07:55 AM   #5
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: Tupelo, MS
Distribution: Gentoo
Posts: 6,926

Rep: Reputation: 124Reputation: 124
But, if he drops to init 1, isn't he going to lose networking,
and therefore his remote login session?
 
Old 08-05-2009, 07:10 PM   #6
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 3,975

Rep: Reputation: 476Reputation: 476Reputation: 476Reputation: 476Reputation: 476
Create the partitions on the new drive and rsync the partitions one by one. That might require some --include= and --exclude= options. Practice this a bit on your local workstation.

Exclude the /proc and /sys directories, although you should create them on the new drive. Do not exclude /dev. You might want to exclude externally mounted drives as well.

After this is done, make the disk bootable by copying the first 446 bytes (and no more) from the old disk to the new one:
Code:
dd if=/dev/sda of=/dev/sdb count=1 bs=446
Assuming sda is the existing drive and sdb the new one.

Last time I did this was to clone a running production server to new hardware, 8000 km from where I live. Needless to say ot just worked at a reboot.

jlinkels
 
Old 08-05-2009, 07:50 PM   #7
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
For Red Hat and Fedora, LVM2 is normally used. It would have a /boot partition, a swap partition and an LVM logical volume. The LVM volume could be on the remainder of the disk or composed of the remainder of the disk, plus other disks.

If you create a dump or image backup, you will want to backup the logical volume and not the physical disks. LVM has a feature where you can have a mirror of a logical volume.
Quote:
Originally Posted by http://www.redhat.com/magazine/007may05/features/storage/
LVM volumes can be mirrored so that the data on a single logical volume is copied onto up to 32 separate physical volumes. Each physical volume gets a copy of each disk block written to the LV, as shown in Figure 4, Volume mirroring. These mirrors can be used to recover from disk failures and to create point-in-time mirror copies that can be removed from a mirror set and mounted as a separate volume. Also note that combining snapshots with mirroring allows recovery from both disk errors as well as file system or database errors.
This webpage has a good overview.
http://www.redhat.com/magazine/007ma...tures/storage/

If you save a snapshot or an LV image or a disk image to a file, you would want to use Linux to pull files from it, not Windows.

If you aren't using LVM, and create images of the drive to a file, you can use the "losetup" command to mount a partition that is inside the disk image. If you create an image of a Logical Volume, you can do the same. Using "fdisk -lu" to list the partition table of the disk, and printing it out, you can record what offsets to use to mount each partition. After mounting the partition, you could share the mountpoint using Samba and access files on it from a Windows machine on the network.
 
Old 08-10-2009, 07:43 AM   #8
ajaxmike
LQ Newbie
 
Registered: Dec 2008
Location: Ajax, Ontario Canada
Posts: 3

Original Poster
Rep: Reputation: 0
Ok here's an update:

I used dd to create an image of my system disk on an eSATA drive using VNC. At first it was only transferring at 10MB/s and the flat out speed of the drive is 110 MB/s. Traced the problem to the eSATA controller, changed the drive to another port, and now I get 50+ MB/s. That's probably as good as it gets for the whole process.

I found an article on the web with tips for dd and it described how to determine the best bs= by running some trial runs. Determined that 8096 was the fastest. So this created what appears to be a bootable image:

dd if=/dev/sda of=/dev/sdb bs=8096

I am able to disconnect the drive and read the ext3 partitions on my Windows machine using Ext2 Volume manager(ext2fsd). I haven't tried booting the from the image yet.

The only problem so far is that it takes over five hours to clone the drive (1TB). Since the system is primarily storing static data (music library and desktop backups), a differential backup appeals to me, or at least skipping unused blocks.

I'm not currently using LVM and it looked way too complicated for me, so I don't want to set it up. Even rsync is looking pretty complicated.

I tried rsync to copy partitions, but I can't get it to work. /dev/sda and /dev/sdb are not valid directories (tried slash on the end too). I am guessing that rsync wants the filesystem name for the partition, not the device name. Since the filesystem names on the image drive are the same as the source drive, how do I tell rsync which drive is which? I don't think the the image drive is mounted, because I don't think dd needs it mounted to work.
 
Old 08-10-2009, 09:09 AM   #9
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 3,975

Rep: Reputation: 476Reputation: 476Reputation: 476Reputation: 476Reputation: 476
Your assumptions about dd vs rsync are correct. DD copies on a lower level, can use devices and devices need not to be mounted.

Rsync OTOH operates on file system level, which means it uses file names, directories names etc, and is unable to copy devices (/dev/sda) or partitions (/dev/sda1)

In order for rsync to work you must have both file systems mounted. If you want to copy the entire primary drive to the secondary (say /dev/sda to /dev/sdb), you would have to:
  • partition /dev/sdb
  • mount each partition in the root tree, say in /dev/sdb1 -> /mnt/sdb1, /dev/sdb5 -> /mnt/sdb5, etc
  • copy each partition to the partitions of the secondary drive

Say that you have separate partitions for /, /home, /usr and /var on sda1, sda5, sda6 and sda7 /dev/ you would have to do something like this:
Code:
rsync -uav --exclude='/home' --exclude='/usr' --exclude='/var' --exclude='/sys' --exclude='/proc' / /mnt/sdb1
rsync -uav /home/ /mnt/sdb5
rsync -uav /usr/ /mnt/sdb6
rsync -uav /var/ /mnt/sdb7
This is a bit awkward with excludes etc., and I am sure better solutions exist but I haven't discovered them yet. If you do it like this keep and eye on it for the first time to see if you have the exclude patterns right, otherwise you end up with copying everything on /dev/sdb1 which isn't right.

I think you are doing just fine with using DD for the initial copy. If you want to make incremental copies, rsync might be better.

jlinkels
 
  


Reply

Tags
dd, disk, image, rsync


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
clone hard disk with: cat /dev/sda >/media/disk/backup deathalele Linux - Security 3 04-17-2009 12:29 AM
I need to Clone a Red Hat drive and install clone in HP server drummer54 Linux - Newbie 14 03-07-2009 04:15 PM
RAID1 array down to one disk with bad blocks, clone to good disk with dd noerror? ewolf Linux - Server 2 05-10-2008 12:40 AM
Clone IDE disk to SATA disk sdem Linux - Software 1 02-16-2006 05:51 AM
headless server gravij Linux - General 2 06-06-2004 07:24 AM


All times are GMT -5. The time now is 04:44 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration