Learn the DD command

Quakeboy02 · 05-02-2014, 06:41 PM

What has happened to the ability to use "of=/dev/sr0"? I am certain that I used to be able to burn an image directly to a CD/DVD with "dd", but now it always complains that "/dev/sr0" is a read-only filesystem. Have I lost my mind and this was never possible?

AwesomeMachine · 05-03-2014, 01:23 AM

You could try formatting a DVD-RW and then a command like this:

Code:

wodim -vv dev=/dev/sr0 -sao -fs=500m -speed=4 < dd if=/home/sam/image.iso

OR

Code:

dd if=/home/sam/image.iso | wodim -vv dev=/dev/sr0 -sao -fs=500m -speed=4

But I think wodim will fail without a filename as the last argument. I vaguely recall utter failure when I once attempted to work this out.

Quakeboy02 · 05-03-2014, 10:36 AM

OK AM, if you say it can't be done, then it can't. I guess my memory is faulty. Thanks.

rknichols · 05-03-2014, 01:22 PM

Quote:

Originally Posted by Quakeboy02

What has happened to the ability to use "of=/dev/sr0"? I am certain that I used to be able to burn an image directly to a CD/DVD with "dd", but now it always complains that "/dev/sr0" is a read-only filesystem. Have I lost my mind and this was never possible?

AFAIK, only DVD-RAM disks could be written to that simply.

sayhello_to_the_world · 01-03-2015, 08:26 AM

hello all - good day dear linux-experts

first of all : a very very happy new year to all of you!

have to save the data of a lenovo-notebook: want to achive that with knoppix.
what is aimed: how to do with knoppix a dd on windows machine with nine (9) partitions

the notebook: lenovo G 780 Notebook
the os: windows 8

see the lenovo-support-links:

https://forums.lenovo.com/t5/Lenovo-...6d69642eea6ce7
http://support.lenovo.com/us/en/documents/HT080887

so the below is only an example to show you the difficulties of the 9 partitions this beast has got!

Quote:

Disk /dev/sda: 255 haeds, 63 sectors, 38913 cylinders
Units =cylinders of 16065 + 512 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 3830 30764443+ 7 HPFS/NTFS
/dev/sda2 3831 38339 277193542+ 7 HPFS/NTFS
/dev/sda3 38340 38913 4610655 f Wind95 tild (LBA)
/dev/sda5 38340 38913 4610623+ 7 HPFS/NTFS

and others more....

Disk /dev/sdb: 255 heads, 63 sector, 60801 cylinders
Units = cylinders of 16065 + 512 bytes

how to do this - with the dd command? i want to store the stuff to external drive

well this is no easy task - but i want to do it.

Again - a happy happy new year to all of you!!

greetings

Head_on_a_Stick · 01-03-2015, 08:40 AM

Code:

# dd if=/dev/sda of=/dev/sdb bs=4096;sync

ronin21 · 02-27-2015, 04:57 AM

To senior Linux techies..
I got my PC troubleshooting bootable USB 8GB with parted magic, kali and few other isos by yumi utilities, it boots and works well. But now i want to migrate image of it to new USB flash disk 32GB.
i made image of it on my Ubuntu machine and now want to transfer it to new one.
I tried using your dd examples, but it works out 8gb capacity on new disk wasting lots of memory space but it boots. I tried to expand this partition using gparted but it ruins the bootable option.

I guess as per your words i need to use :

dd if=/dev/sda skip=1 of=/dev/sdb seek=1 bs=4k conv=noerror (good when u have two devices but i got one image and new usb left)

if yes, it all good but to further challenge, i already image as usb_8g.img on my desktop and i need to move it to bigger usb_32gb. (after making image, i formatted my usb_8gb

Regards

Ron

Quote:

Originally Posted by AwesomeMachine

The only difference between a big partition and a small partition, besides size, is the partition table. If you are copying say sda to sdb, an entire drive with a single partition, sdb being smaller than sda, then you have to do:

dd if=/dev/sda skip=1 of=/dev/sdb seek=1 bs=4k conv=noerror

You can put the skip and seek anywhere you want, I just put them there to show what they go to. Skip skips input blocks at the beginning of the media(sda). Seek skips over so many blocks on the output media before writing(sdb). By doing this, you leave the first 4k bytes on each drive the same. This is eight sectors. The first sector is the Master Boot Record (MBR). This contains the partition table. You don't want to tell a drive it is bigger than it really is by writing a partition table from a larger drive to a smaller drive. In my example, we wanted to use a byte size (bs) of 4k or 4096 bytes. This speeds up the copy from the default 512 byte bs. But, your are limited in how small of a skip or seek you can make. In this example, the smallest skip or seek is 1, or 4096 bytes. Since the first 63 sectors of the drives are empty, except sector 1, the MBR, we can skip eight sectors without any problem missing part of the boot sector. One sector on a hard drive is 512 bytes.

Now, if you are copying say sda3 to sda2, this is different. What you want to do is this:

dd if=/dev/sda3 of=/dev/sda2 bs=4096 conv=noerror

Even if these are primary partitions, they are still logical to the main partition. Do not use the conv=notrunc option. Without notrunc, multiple blocks of zeros get abbreviated with a string of asterisks, saving a lot of space. if you use notrunc, all the zeros will be written out. On a smaller partition you want to save room. If you write out all the zeros, the smaller partition won't be big enough.

fatmac · 02-27-2015, 09:11 AM

Quote:

I guess as per your words i need to use :

dd if=/dev/sda skip=1 of=/dev/sdb seek=1 bs=4k conv=noerror (good when u have two devices but i got one image and new usb left)

If that works, just substitute your disk image for /dev/sda.

moguerange · 03-20-2016, 05:08 AM

Thanks!

augustk · 03-20-2016, 05:24 AM

useful and well explained. Need more posts like this.

slac-in-the-box · 04-25-2016, 12:34 PM

Hello LQ world:

Awesome writeup of dd. I found it when trying to use dd to make a sparse file, when cloning MBR and first partition of disk. When I scroll through results from

Code:

man dd

I see a list of symbols that can be used with conv=, and one of the symbols listed is sparse:

Code:

sparse try to seek rather than write the output for NUL input blocks

. However, when I try to make use of this, I get error:

Code:

dd: invalid argument 'sparse' to 'conv'

I am trying to build a custom slackware64-14.1 image for google cloud compute engine, following documentation at https://cloud.google.com/compute/doc...uilding-images. I started up an extra laptop with slackware usb-installer, and used fdisk to create one 10G partition; I used setup and installed all the slackware sets, except for KDE, KDEI, X, XAP, and Y...; I configured grub instead of lilo, and booted into new installation. Then I rebuilt kernel and configured system according to google documentation above, and system reboots into new kernel: so far so good.

Now I started back up from the slackware USB installer, which has 9GB free space left on the thumb drive, and I want to create a sparse image from my 10GB partition, and I want the image to include the MBR, I want it to be called disk.raw, and I want to save it on the USB drive.

I mounted the thumb drive to /mnt.

The command I concocted that doesn't work is as follows:

Code:

dd if=/dev/sda of=/mnt/disk.raw bs=1048 count=10485760 conv=sparse

dd: invalid argument 'sparse' to 'conv'

Does dd actually work with conv=sparse? How do I do it?

rayburn · 04-28-2016, 02:50 PM

I have just tried using dd to image a partition and then compress it using this command as in the first post of this thread:

Code:

dd if=/dev/sdb2 ibs=4096 | gzip > partition.image.gz conv=noerror

However, I found that the correct command should be this:

Code:

dd if=/dev/sdb2 ibs=4096 conv=noerror | gzip > partition.image.gz

The option for the dd command should of course be at the end of the dd command rather than at the end of the gzip command.

I mention this only to assist others who may use the command, and it is in no way a criticism of an outstanding guide to the dd command which I have referred to on many occasions. Thank you!

Updated code:

Code:

dd if=/dev/sdb2 ibs=4096 conv=noerror,sync | gzip > partition.image.gz

rknichols · 04-28-2016, 09:31 PM

Quote:

Originally Posted by rayburn

However, I found that the correct command should be this:

Code:

dd if=/dev/sdb2 ibs=4096 conv=noerror | gzip > partition.image.gz

And that is still wrong. If there is any error reading the source, the rest of the stream will just be shifted over to fill in the unreadable gap, and the result will be a badly corrupted filesystem image. (I'm tempted to say "hopelessly corrupted," but careful forensic detective work could create a sane image.) You absolutely need the "sync" conversion too so that the missing block will be filled in with zeros.

Code:

dd if=/dev/sdb2 ibs=4096 conv=noerror,sync | gzip > partition.image.gz

AwesomeMachine · 04-28-2016, 10:18 PM

Quote:

Originally Posted by slac-in-the-box

Hello LQ world:

Awesome writeup of dd. I found it when trying to use dd to make a sparse file, when cloning MBR and first partition of disk. When I scroll through results from

Code:

man dd

I see a list of symbols that can be used with conv=, and one of the symbols listed is sparse:

Code:

sparse try to seek rather than write the output for NUL input blocks

. However, when I try to make use of this, I get error:

Code:

dd: invalid argument 'sparse' to 'conv'

I am trying to build a custom slackware64-14.1 image for google cloud compute engine, following documentation at https://cloud.google.com/compute/doc...uilding-images. I started up an extra laptop with slackware usb-installer, and used fdisk to create one 10G partition; I used setup and installed all the slackware sets, except for KDE, KDEI, X, XAP, and Y...; I configured grub instead of lilo, and booted into new installation. Then I rebuilt kernel and configured system according to google documentation above, and system reboots into new kernel: so far so good.

Now I started back up from the slackware USB installer, which has 9GB free space left on the thumb drive, and I want to create a sparse image from my 10GB partition, and I want the image to include the MBR, I want it to be called disk.raw, and I want to save it on the USB drive.

I mounted the thumb drive to /mnt.

The command I concocted that doesn't work is as follows:

Code:

dd if=/dev/sda of=/mnt/disk.raw bs=1048 count=10485760 conv=sparse

dd: invalid argument 'sparse' to 'conv'

Does dd actually work with conv=sparse? How do I do it?

Hi,

'sparse is not one of the 'dd' convert (conv) arguments. A sparse file is like a place marker on the disk. It is a file that at least some of it contains nothing.

To write a 45MB sparse file, try:

Code:

dd if=/dev/zero of=/home/sam/sparse.file count=45 bs=1M

But that way wastes disk space, because you have to make the file as big as it will ever need to be.

Sparse file that grows from zero Bytes to 45MB as you fill it:

Code:

dd if=/dev/zero of=/home/sam/sparse.file count=0 seek=45 bs=1M

makes a file of 0B that can be filled up to 45MB, but only consumes the disk space of the current size of its contents. That's how the virtual machine disks usually are created.

If you simply want to write one disk partition to another, try:

Code:

dd if=/dev/sda of=/thumbdrive_root/partition_image.bin bs=4096

AwesomeMachine · 04-28-2016, 10:30 PM

Quote:

Originally Posted by rayburn

I have just tried using dd to image a partition and then compress it using this command as in the first post of this thread:

Code:

dd if=/dev/sdb2 ibs=4096 | gzip > partition.image.gz conv=noerror

However, I found that the correct command should be this:

Code:

dd if=/dev/sdb2 ibs=4096 conv=noerror | gzip > partition.image.gz

The option for the dd command should of course be at the end of the dd command rather than at the end of the gzip command.

I mention this only to assist others who may use the command, and it is in no way a criticism of an outstanding guide to the dd command which I have referred to on many occasions. Thank you!

The

Code:

conv=noerror

is superfluous to the subject. Usually dd will stop on an error. If the

Code:

conv=noerror

conversion flag is used, if dd encounters an error it continues to run while outputting a message each time it encounters an error. Either command should work on a healthy drive.

If it only works with the 'noerror' flag, then the compressed image may be flawed. It's always a bad sign when you have to use 'noerror' to get 'dd' to work.

I have not found a good use for this conversion flag since the program

Code:

ddrescue

should be run on drives that produce dd errors.