LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 12-19-2006, 01:16 AM   #256
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47

Quote:
wanted to know if it is possible for dd to clone a 80gb hdd to a 20gb hdd. Their is only about 4gb of data on the 80gb hdd
I have already asked the question, if you look in the thread. Mateer also gave some advice about resizing partition before using dd. Anyhow, still as indicated by other posters cp -R is best

AwesomeMachine,
I have heard of people having problem with the journal of ext3 when using dd, and resorting to transforming ext3 to ext2 before dd'ing. Do you know when this technique ext3->ext2 *really* need to be applied? I am not sure this is really needed (sorry I have no example at hand)
 
Old 12-21-2006, 06:13 AM   #257
yamez
LQ Newbie
 
Registered: Dec 2006
Distribution: Knoppix 5.0, KnoppMyth R5B7, IPCop 4.11
Posts: 2

Rep: Reputation: 0
I want to create an image of a USB flash drive then write that image back to another same sized USB flash drive when needed. I need to image the whole drive. Using Knoppix 5.0, I enter the command...

dd if=/dev/sdb of=/home/knoppix/usb.img bs=1M

When writing back after I swap USB flash drives, I use...

dd if=/home/knoppix/usb.img of=/dev/sdb bs=1M

The problem...

The source USB flash drive has 2816 files, 58 folders, total 94MB.
The target USB flash drive has 2190 files, 45 folders, total 64MB.

However, in Windows under drive properties, both USB flash drives report using the same space of 98MB. I've tried 2 different USB flash drives, both 128MB. No errors result when running dd.

Any ideas?
 
Old 12-26-2006, 09:18 PM   #258
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
I do not recommend using dd with RAID

Quote:
Originally Posted by saikee
I believe that is down to the kernel. If the kernel recognises the RAID disks as you want it to then you are home dry.

The kernel does have to cope with different chipsets and if it fails the two disks in the RAID may be treated as two separate disks.

You should have a pretty good idea of the hard disk recognition if you issue the "fdisk -l" command requesting all disks to be displayed with their partitions tabulated.
There are too many variations with RAID for dd to be able to reliably work with copying raid arrays to other raid arrays, unless the arrays are external. An external RAID array looks like one big disk to dd, but when you try to put two discreet RAID setups in a single PC it is very difficult. If you have four disks, any controller will try to configure this as RAID 5. If you really want (2) RAID 0 you probably won't get that.

With small arrays there are ways to copy the whole array onto a single disk, and then rebuild the array by adding disks.
 
Old 12-26-2006, 09:31 PM   #259
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
AwesomeMachine,
I have heard of people having problem with the journal of ext3 when using dd, and resorting to transforming ext3 to ext2 before dd'ing. Do you know when this technique ext3->ext2 *really* need to be applied? I am not sure this is really needed (sorry I have no example at hand)[/quote]

I hadn't heard of converting from ext3->ext2 before using dd, but there are some theoretical concerns with the superblock and journals on ext3, and use of dd. By careful guessing, I would assume this is currently a problem, or was a problem at some point. I can't find anything in any books on this, but I think if a partition is copied by itself, and it isn't the first primary partition on the HDD, and it is ext3, you need to convert to ext2 before using dd. If someone could blackbox this theory for me I would appreciate it.
 
Old 12-26-2006, 09:43 PM   #260
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Linux Hotplug

Quote:
Originally Posted by yamez
I want to create an image of a USB flash drive then write that image back to another same sized USB flash drive when needed. I need to image the whole drive. Using Knoppix 5.0, I enter the command...

dd if=/dev/sdb of=/home/knoppix/usb.img bs=1M

When writing back after I swap USB flash drives, I use...

dd if=/home/knoppix/usb.img of=/dev/sdb bs=1M

The problem...

The source USB flash drive has 2816 files, 58 folders, total 94MB.
The target USB flash drive has 2190 files, 45 folders, total 64MB.

However, in Windows under drive properties, both USB flash drives report using the same space of 98MB. I've tried 2 different USB flash drives, both 128MB. No errors result when running dd.

Any ideas?
With hotplug, /dev/sdb isn't the drive, it is the disk. If you want to copy to the drive, you use - /dev/sdb1. With conventional HDD's /dev/hda, sda are drives, but not with USB drives using hotplug. Another thing, sometimes with hotplug, if the device is detected once, and then it is unplugged, it won't be recognized again when it is plugged back in. It usually doesn't take long for /dev/sdb1 to disappear after plugging in a usb stick, so look in /dev quickly for the proper partition. With sticks that size there is probably one drive on each stick. There is a 512 MB boundary on FAT12, which is usually what small sticks are formatted with. You can also use iso9660, and put a bootable iso image file on the stick. USB boot is compliant with the el torito boot specification CDRoms use for booting.
 
Old 01-01-2007, 06:21 PM   #261
drhousbr
LQ Newbie
 
Registered: Jan 2007
Posts: 1

Rep: Reputation: 0
This was very helpful. The only extra thing I had
to do was run reiserfscks --check prior to running
resize_reiserfs. This was under SLES 9 AMD64.

Worked for me. I had a cloned disk that booted fine.

Cheers, DRHO

Quote:
Originally Posted by jlinkels
Almost.

I assume you have been partitioning hdb correctly and that 64M as first partition is enough to hold you boot stuff etc.

Then do:
dd if=/dev/hda1 of=/dev/hdb1 bs=4k conv=notrunc,noerror
to copy the first partition
and
dd if=/dev/hda2 of=/dev/hdb2 bs=4k conv=notrunc,noerror
to copy the second partition.

(You were right here, but might have forgotten to clone the first partition)

Still, the drive is not bootable, because you did not copy the master boot record yet. You can this with:

dd if=/dev/hda of=/dev/hdb bs=446 count=1

Note that you do a copy on a disk and not on a partition. The bs=446 is important because this is exactly the boot sector, without the partition table.

Then, resize both partitions:
resize_reiserfs /dev/hdb1
resize_reiserfs /dev/hdb2

And NOT /dev/hdb!

Don't forget you might want to create a swap partition if you don't have one yet. On the partition you designated as swap, do mkswap to initialize it as swap partition, no need to do this with dd.

And yes, you can do this in a live system.

jlinkels
 
Old 01-02-2007, 11:27 AM   #262
woverko
LQ Newbie
 
Registered: Nov 2005
Posts: 8

Rep: Reputation: 0
Wow those first two posts (the ones with all the dd related information) were huge. Great information though.
 
Old 01-04-2007, 11:14 AM   #263
AQG
Member
 
Registered: Jun 2005
Distribution: SuSE, Red Hat
Posts: 161

Rep: Reputation: 30
Excellent !!!!!
 
Old 01-04-2007, 08:42 PM   #264
yamez
LQ Newbie
 
Registered: Dec 2006
Distribution: Knoppix 5.0, KnoppMyth R5B7, IPCop 4.11
Posts: 2

Rep: Reputation: 0
Quote:
Originally Posted by AwesomeMachine
With hotplug, /dev/sdb isn't the drive, it is the disk. If you want to copy to the drive, you use - /dev/sdb1. With conventional HDD's /dev/hda, sda are drives, but not with USB drives using hotplug.
I have tried your recommendations but the same error occurs. I did find that it appears to be the USB flash drives. I have 100 identical flash drives and 30 have this error when using dd, the remaining do not. However, when I do a normal file copy ALL the flash drives work. I would like to use imaging though for support of bootable flash drives and different file systems/partitioning when duplicating multiple drives. Is there a way with to determine how many written blocks are on the drive and then use that number to limit how much dd writes? Thanks for your reply.
 
Old 01-04-2007, 11:52 PM   #265
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Yamez,

You weren't very detailed in your response. Were you able to locate a partition on each drive (i.e. sdb1, sdb2)? If so, did you try writing to the partition instead of the sdb device? Did you use fdisk -l to look at the partitions? You don't give a lot of information. On a USB flash drive, 512 byte logical sectors are used. You can do:

dd if=/dev/zero of=/dev/sdb1

and when it gets finished it will say how many 512 byte blocks it copied zeroes to.
 
Old 01-08-2007, 01:26 PM   #266
dwcoffin
LQ Newbie
 
Registered: Dec 2005
Posts: 4

Rep: Reputation: 0
Is it possible to dd to a SMB share using Netcat or some other network mechanism? I have been playing with DD in VM, but now want to learn how to create a gz file of the image to a network location. I'm currently using different flavors of RedHat our production servers are AS4 and my desktop VM's that I'm playing with are FC5.
 
Old 01-12-2007, 03:49 AM   #267
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Dd and block devices

Quote:
Originally Posted by dwcoffin
Is it possible to dd to a SMB share using Netcat or some other network mechanism? I have been playing with DD in VM, but now want to learn how to create a gz file of the image to a network location. I'm currently using different flavors of RedHat our production servers are AS4 and my desktop VM's that I'm playing with are FC5.
Dd is specifically designed to copy a certain amount of blocks from a block device to a file or block device. Block devices are found in '/dev' directory. I've never used samba, so I don't know if samba makes a block device of the remote share on the host. Also, dd does not work with directories. You can copy files only. Since block devices are virtual files, dd can image complete drives. Dd can also duplicate other files, but not mount points.
 
Old 01-12-2007, 03:55 AM   #268
nirmaltom
Member
 
Registered: Jun 2005
Location: India
Distribution: Redhat,Fedora,DSL,Ubuntu
Posts: 238

Rep: Reputation: 30
hi,
copied the page to my self linux helper directory
regards,
Tom.
 
Old 01-12-2007, 07:23 PM   #269
Duck2006
Member
 
Registered: Sep 2006
Distribution: Ubuntu 8.04 Hardy Heron LST
Posts: 346

Rep: Reputation: 33
Thanks a lot, this is great
 
Old 01-15-2007, 01:03 PM   #270
alisou
LQ Newbie
 
Registered: Aug 2006
Location: Québec, Canada
Distribution: Slackware based
Posts: 11

Rep: Reputation: 0
Hi all,
Verry good post!
AIR (Automated Image and Restore) is a GUI front-end at dd/dcfldd.
http://air-imager.sourceforge.net/
and this live CD use AIR:
Helix, http://www.e-fense.com/helix/
BackTrack, http://www.remote-exploit.org/index.php/BackTrack
4BAK, http://4bak.org/
AliXe, http://alixe.org

Regards!
alisou, :-)
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM


All times are GMT -5. The time now is 10:46 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration