LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 10-01-2007, 07:57 AM   #361
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,592

Original Poster
Rep: Reputation: 162Reputation: 162
Post Cp -r instead of dd


Quote:
Originally Posted by Paulinux View Post
AwesomeMachine,

I've studied all 24 pages of this thread, and none of the suggestions have worked for me. Here's my story:
I thank you for reading all the replies that you did before asking questions.

Quote:
Originally Posted by Paulinux View Post
THESE ARE MY DRIVES:
********************************************************************
Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 13 104391 83 Linux
/dev/hda2 14 9729 78043770 8e Linux LVM

Disk /dev/hdb: 40.0 GB, 40060403712 bytes
16 heads, 63 sectors/track, 77622 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System
/dev/hdb1 1 208 104800+ 83 Linux
/dev/hdb2 209 77622 39016656 8e Linux LVM
********************************************************************
I created the above partitions on hdb to resemble those of hda. hda is mounted, hdb is not.

I would like to do what you recommended in one of your posts (#300) regarding small to large cloning, if it will work in the other direction as well:
"use cp -r * from the root directory of the old drive to the root of the new drive. Then, use the linux install CD in rescue mode, and reinstall the bootloader to the MBR."

My questions:

1) Will the cp -r method copy files from and to the proper partitions, or will it just dump stuff where it finds room, totally jumbling up a necessary structure?

2) How does one "...use the linux install CD in rescue mode, and reinstall the bootloader to the MBR." (my hda uses grub)

3) Is there a better way to move my system onto a smaller drive?

Thanks.

Good questions. I think there is a (STANDING)
................................................................(MISS)
Of how cp differs from dd. Cp is the Unix copy command. Cp requires partitions to have mount points, to be mounted, and be formatted. Cp is a file system aware copy utility that tells an existing file system where it put each file. Dd is a partition aware utility that bitstreams a file system to a drive or parttition, with or without an existing file system, but dd copies parts of a disk that cp will not.

So, you mount hda1, hda2, hdb1, and hdb2 to some mount points, or directories as they would normally be called, like so:

mkdir /mnt/hda1
mount -t ext3 /dev/hda1 /mnt/hda1

Use the up arrow (twice) to get the basic command lines at the prompt again, and change the 1s to 2s for mounting hda2. Use a similar fashion for the other 2 mounts, and save a lot of typing.

cp -r /mnt/hda1/* /mnt/hdb1/
cp -r /mnt/hdb2/* /mnt/hdb2/

hdb must have a file system on both partitions to use cp -r. Cp does not write to blank , or unformatted partitions. It sounds like you like doing things the manly Linux way. You would want the the mke2fs command to format. Ext3 is ext2 with the J option, which is what you want. The rest of what you need to know to format is: >man mke2fs.
 
Old 10-01-2007, 08:38 AM   #362
saikee
Senior Member
 
Registered: Sep 2005
Location: Newcastle upon Tyne UK
Distribution: Any free distro.
Posts: 3,398
Blog Entries: 1

Rep: Reputation: 112Reputation: 112
I use dd for a specific purpose and most of it for migrating the whole disk with a lot of partitions.

For migrating one of two partitions it is easier to use file-copying commands because the size of the source and the target can be different size, as long the target physically large enough to accept all the files.

Say if one has mounted the source partition is /mnt/sda2 and want it to move to the mounted /mnt/sdb3 then 2 terminal commands will copy the entire the system across, with all permission settings preserved
Code:
cd /mnt/sda2
tar cf - . | (cd /mnt/sdb3; tar xf -)
The last line can be replaced by (already suggested by AwesomeMachine)
Code:
cp -R /mnt/sda2/* /mnt/sdb3
The moved Linux in sdb3 will boot once the following have been implemented

(1) partition reference sda2 is changed to sdb3 in
(a) /etc/fstab
(b) /boot/grub/menu.lst for Grub or /etc/lilo.conf for Lilo
(2) The boot loader is restored if the Linux originally boot from the MBR (see last link of my signature for tips)
If the Linux uses Grub and sda is present as the 1st disk (hd0) and sdb as the 2nd disk (hd1), then sdb3 can be booted
(i) Manually by command in a Grub prompt
Code:
root (hd1,2)
configfile /boot/grub/menu.lst
(ii) With menu.lst by command in a Grub prompt or a Grub shell
Code:
root (hd1,2)
setup (hd0)
Using dd in such a case not only requiring the identical amount of work but the sda2 and sdb3 must be identical in size, down to the exact number of sectors. If the target partition reference is unchanged, say from sda2 to sdb2, and the sda is removed after cloning then sdb2 would be able to inherit the sda2 status and boot normally without any change.

There is no techincal problem of moving a Linux distro between partitions or between hard disks.
 
Old 10-01-2007, 09:57 AM   #363
dr_agon
Member
 
Registered: Sep 2007
Location: Poland
Distribution: Ubuntu 8.04
Posts: 77
Blog Entries: 12

Rep: Reputation: 24
dd statistics printout

Quote:
Originally Posted by AwesomeMachine View Post
I'm going to rephrase your question so people know what this reply is about. For some reason you are configuring laptops with a drive image stored remotely, inaccessible except by Internet.
Not exactly I have 2 identical laptops, I wanted to clone the entire system from one to another without installing the all stuff from start. And I had them on the same LAN without problem, booting both from live CD (Ubuntu). I created one partition for the system in the second laptop matching the partition from the first one - I mean the geometry was identical to the sector. I did not wanted to clone the entire drive, as it contained some backups on another partition. So All I needed was to transfer the system partition and MBR to identical disk in a computer connected by ethernet LAN as described on the following page (worth visiting for a newbie like me):

http://digiassn.blogspot.com/2006/01...eap-ghost.html

I did the partition cloning successfully, in spite that it was Windows on NTFS


Quote:
Originally Posted by AwesomeMachine View Post
You can use gzip, which isn't anywhere near as memory intensive.
Oh, yeah ...
The weird thing was, that
when piping dd through gzip like this:
Code:
dd if=/dev/sda1 | gzip  -cf | nc 192.168.1.120 7000 q 10
and on the target machine:
nc l p 7000 | gzip dfc | dd of=/dev/sda1
everything was fine but when I tried to do the same WITHOUT gzip in the middle,
Code:
dd if=/dev/sda1 | nc 192.168.1.120 7000 q 10
dd ended with "not enough space" error and I noticed non-zero numbers after '+', like

nnnnnn+22 records in
nnnnnn+22 records out

That's why I asked about this statistics in my previous post. And where did the extra blocks come from?
 
Old 10-01-2007, 10:17 AM   #364
saikee
Senior Member
 
Registered: Sep 2005
Location: Newcastle upon Tyne UK
Distribution: Any free distro.
Posts: 3,398
Blog Entries: 1

Rep: Reputation: 112Reputation: 112
I am sure AwesomeMachine can answer your question better but it is obvious to me in executing the dd command with gzip you compressed the data to a very small size and so it has a small foorprint when being held in any transit storage.

When you opted out the compression the number of records in the transfer will of course rocketed. Is this not logical? The much expanded data to be transferred may have become too big in the holding area.

In the general the number of records multiplied by the block size (in the bs= parameter) should exactly equal to the partition size you are transferring, assuming no compression.

If you try to clone a MS Windows from one machine and use it on another PC, even identical, it should not work because modern Windows keeps the serial numbers of the hardware too. Even the same make and same capacity different hard disks are detected. XP permits a small changes in the hardware and will lock itself up if the amount of changes exceeds a prefixed threshold.

You can clone XP if you use it on the original machine. For a laptop you can hoop the target as a USB hard disk, clone it across, remove the source, place the target in its place and XP will boot as good as the original (required a reboot for the very 1st use to update the hard disk serial number). All you need to make sure if the target disk is equal, down to the number of sectors, or just larger than the source disk. This thread may be helpful for such migration.

Last edited by saikee; 10-01-2007 at 10:28 AM.
 
Old 10-01-2007, 06:03 PM   #365
Paulinux
LQ Newbie
 
Registered: Sep 2007
Posts: 6

Rep: Reputation: 0
AwesomeMachine & saikee,

Thanks for the additional information, but it didn't help:

After two more days of experimentation, and then trying your suggestions, I found that I cannot cp -r the files from hda2 to hdb2 because I cannot mount hda2; I can only mount that which contains it, a logical volume called /dev/VolGroup00/LogVol00 (my swap area is called /dev/VolGroup00/LogVol01). That's what I get for accepting FC6 installation's default settings. LVM is a good thing, only not for me.

Fine, so I mount /dev/VolGroup00/LogVol00 to /mnt/frick, and I mount hdb2 to /mnt/frack. I see all my files on frick, so I cp them all over to frack. Happy, Happy, Joy, Joy, now I see all my files on frack !!!, but this gives me an uneasy feeling because it seems my destination should have been a container of hdb2, something with a name like /dev/VolGroup##/LogVol##. I then modify "fstab" and "menu.lst" as suggested, "cp -r" hda1 to hdb1 (not LVM => mountable), and reboot. Sure enough, the system won't come up because it can't find any LogicalVolumes, even though I told it that "/" now lives on hdb2 rather than /dev/VolGroup00/LogVol00.

It seems that one can only access a VolumeGroup which is on an active system. I believe that I must create a new VolGroup on my system containing hdb2, cp all my stuff onto it, remove it from my system without destroying anything, and then make it a stand alone bootable disk.

I even tried a fresh FC6 install on the destination disk (opting out of LVM), thinking I could cp from my LVM source to the new non-LVM destination, but that didn't boot either.

I searched in other forums for ways to remove LVM, as well as for ways to clone LVM disks because I don't want to turn this thread from "how to dd" to "what's wrong about LVM". I couldn't find anything, but I admit that I didn't read all 600 pages on cloning.

I thought I could just dd needed sectors from source to destination without regard to where they were placed, but it seems that LVM maps data onto certain sectors in a specific way, and when dd copies source sector #x to destination sector #x+y, the map is no longer valid.

Oh, Godfather, I don't know what to do. I don't know what to do . . .
 
Old 10-01-2007, 06:33 PM   #366
saikee
Senior Member
 
Registered: Sep 2005
Location: Newcastle upon Tyne UK
Distribution: Any free distro.
Posts: 3,398
Blog Entries: 1

Rep: Reputation: 112Reputation: 112
I am sorry to see you are a sufferer of LVM which I have avoided so far because of more trouble than its worth.

I also made a mistake not reading your post carefully and overlooked the reference of partition type 8e indicating it is a LVM. However the more serious mistake I now found out is your two disks have different geometries. The 80Gb has 255 heads whereas the 40Gb has only 16 heads. That alone will not work for the cloning of your LVM as a partition.

LVM is a management layer and the only way it could store several logical volumes in one partition must be by hard coding the hard disk addresses of the individual LV. Therefore you may have a technical problem bigger than you think.

The other problem that you have is when even the LVM can be correctly duplicated it will have the same name by default and so you must remove hda in order to have hdb inheriting the sda status. Leaving the two disks in the PC will likely to cause conflict.

I can't advise on how to deal with the geometry difference although by cloning the hard disk one can force the target to have the same geometry as the source. However the layout of your LVM is such that it is 78Gb large and there is no way you can dd it into a 40Gb hard drive. To certain extent you are asking the impossible. May be there is a way to copy one LVM to another but I don't have this experience. LVM is notorious difficult to resize too and not support by resizing software like Gparted as far as I am aware. Thus it may be a challenge even to bring the sda2 into a smaller size to fit the sdb disk.

My advice to cut the loss is to get another hard drive bigger than sda and clone the disk across. That should work if you remove the source leaving the cloned target to boot as sda. Hard disk newer than 80Gb will likely support LBA mode having the standard 255 heads and 63 sectors per track.

Last edited by saikee; 10-01-2007 at 06:38 PM.
 
Old 10-01-2007, 09:03 PM   #367
Paulinux
LQ Newbie
 
Registered: Sep 2007
Posts: 6

Rep: Reputation: 0
saikee,

Thanks for responding.

I was afraid that the differing geometries would be a problem. Oh well.

I wasn't planning to keep both hdds in the same box. The clone is just so I can give the system to someone who doesn't have much money. I'll keep trying -if it's Linux, there a free way to do it!!

A logical volume is easy to resize, as long as you keep it on the same drive:

http://forums.fedoraforum.org/forum/...t=lvm+commands

The example system is almost a duplicate of my system, and by following it almost exactly, I was able to shrink my system from 80G to 13G. This, I thought, would let me dd it up onto a 40G drive. That's what started me on my trip into the dark world of LVM.

I'll try a little longer to rid myself of LVM, but I think you're right, it will soon be time to cut my losses.
 
Old 10-02-2007, 06:49 AM   #368
jasee
LQ Newbie
 
Registered: Jun 2006
Posts: 27

Rep: Reputation: 15
Quote:
Originally Posted by saikee View Post
If you try to clone a MS Windows from one machine and use it on another PC, even identical, it should not work because modern Windows keeps the serial numbers of the hardware too. Even the same make and same capacity different hard disks are detected. XP permits a small changes in the hardware and will lock itself up if the amount of changes exceeds a prefixed threshold.
But this is exactly what you do with programs like Ghost and it works perfectly. I've done it many times. I've never seen it "lock itself up": it asks to be reactivated if you are using a retail version of course
 
Old 10-02-2007, 07:36 AM   #369
saikee
Senior Member
 
Registered: Sep 2005
Location: Newcastle upon Tyne UK
Distribution: Any free distro.
Posts: 3,398
Blog Entries: 1

Rep: Reputation: 112Reputation: 112
jasee,

I also used Ghost many times too. You can find it out by restoring the Ghost image on a different PC or read the WPA yourself.
 
Old 10-02-2007, 07:38 AM   #370
novelc51
LQ Newbie
 
Registered: Jul 2007
Distribution: suse 10
Posts: 9

Rep: Reputation: 0
Learn the DD command

Just an Awsome post!!!
I had used the DD command before in unix but not to the extent that is demonstrated here.
 
Old 10-02-2007, 10:14 AM   #371
gimmegamez
LQ Newbie
 
Registered: Dec 2005
Posts: 1

Rep: Reputation: 0
just want to say thank you for the useful howto.
cheers
g
 
Old 10-03-2007, 12:09 AM   #372
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,592

Original Poster
Rep: Reputation: 162Reputation: 162
Post Appreciated

Quote:
Originally Posted by novelc51 View Post
Just an Awsome post!!!
I had used the DD command before in unix but not to the extent that is demonstrated here.
It's good to hear once in a while. Your Welcome. I humbly thank my Creator for giving me the idea to do a dd thread, leading me to all the information, giving me a great mind to make sense of dd, and for Blessing this thread to be civil, useful, practical, concise, and fun.

-Awesome
 
Old 10-03-2007, 03:48 AM   #373
jasee
LQ Newbie
 
Registered: Jun 2006
Posts: 27

Rep: Reputation: 15
Quote:
Originally Posted by saikee View Post
jasee,

I also used Ghost many times too. You can find it out by restoring the Ghost image on a different PC or read the WPA yourself.
Windows product activation (as that seems to be what you are talking about doesn't apply to corporate versions of window xp. These are versions that don't need to be activated and which are supplied to large organisations. If you ghost one of these then activation simply doesn't apply.

The activated version of windows even if restored to an identical machine _may_ require activation, but I've never seen one lock up you need to change the activation key which you can do by calling M$ or can be done online if the machine has an active network connection.

However specifically if the hardware (disk controller) or processor are different it won't boot of course. It's possible to get round that using a PE disk and changing the driver(s) loading and editing the registry. For simplicity's sake I'm assuming a disk image

I work for an organization with a number of near idential machines, I've got one ghosted image of one (with suitable software installed). If what you said was true, I'd need to keep an image of every one! You can use the recovery disk supplied with each machine (which will only work with that bios) etc, this takes longer and isn't customised for your organisation as my images are, in this case you need to use the coa on the side of the machine and get it activated.

Although the restictions of product activation concern XP or later, previous versions of windows had similar hardware problems. The best version of windows for ghosting was W98, which you could ghost from a laptop to a stand alone machine after many reboots and have a fully operational machine!
 
Old 10-03-2007, 05:27 AM   #374
dr_agon
Member
 
Registered: Sep 2007
Location: Poland
Distribution: Ubuntu 8.04
Posts: 77
Blog Entries: 12

Rep: Reputation: 24
dd statistics printout

jasee and saikee,

thanks for your replies.
I didn't want to start discussion about cloning Windows, as there are many threads about it elsewhere (Saikee - I read yours among others, thanks again). The fact is, that I did clone the systems successfully using dd (but only by piping it through gzip). The reason for cloning was easy maintainance of identical systems, like in jasee's case.
My question was from pure curiosity, why dd reports sometimes different number of blocks transfered, what makes the first number in the output, and what makes the second number after the '+', and why I got "not enough space" error while the partitions have exactly the same size. Is this something I missed in documentation, a bug, or just my ignorance?
 
Old 10-03-2007, 08:47 AM   #375
saikee
Senior Member
 
Registered: Sep 2005
Location: Newcastle upon Tyne UK
Distribution: Any free distro.
Posts: 3,398
Blog Entries: 1

Rep: Reputation: 112Reputation: 112
jasee,

I note what you said and it is true that you can have one Ghost image off a corporation license that you can clone into identical machines with only small hardware difference, say increased ram of upgraded hard disk. However that Ghost image may have a problem if you install it into a different partition, other than the 1st primary, or even on to a second boot disk.

The point of using "dd" is to produce an exact mirror image of the original. To guarantee it to work is to ensure the target is used exactly in the same hardware and software position as before.

dr_agon,

dd is a very command. It reads the input file from the specified source device and write the same on the output device. The size of each record is exactly as specified in the "bs=" parameter. In the absence of one specified, as in your case, dd defaults to one sector of 512 bytes. dd starts from the first sector of the source and stops when the last sector has been reached. The "+" is reported, I believe, if the whole record isn't read.

If you compress the record as you have claimed then you can get away with a small holding area for storing the information. The "not enough space" occurs because you were transferring the uncompressed data across which proved to be too big for the holding area.
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM


All times are GMT -5. The time now is 09:01 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration