AwesomeMachine,

There has been an ineresting debate on dd in this thread.

The debate is on whether dd needs to read one full sector of 512 bytes to give us one byte when we set bs=1.

This is probably based on reading/writing a file the smallest unit is one full sector of 512 bytes and that has been taken as the minimum unit in any read/write operation of a hrd disk. dd does not care about the filing structure but does it still need to read one full sector? It has to carry some kind of additional intelligence to strip out the 511 bytes to give the exact one byte we want if it does read one sector at a time.

Care to comment?

Hi All,

I would like to know how to capture the messages displayed by dd when it is done transfering data (number of records in and out, and data transfer rate). I need to do this in my script, then grep the capture to find out the data rate. I tried the following:

dd if=/dev/zero of=datafile bs=32k count=32768 | grep 'bytes/sec'

But it did not work. Seems the messages from dd were not piped to grep. I then tried:

dd if=/dev/zero of=datafile bs=32k count=32768 > output.txt

It still did not work. output.txt ended with 0 byte in it.

What is the proper way to capture the output from dd?

Any help will be greatly appreciated.

Thanks.

Dd is a little bit tricky here. The command sends the error output to STDOUT, which is the terminal. When you pipe dd to another command you are piping the bitstream dd creates to another command. For example:

dd if=/dev/sda | hexdump -C | grep 'I love you, Thor.'

Will check all of:

/dev/sda

on all file systems, regardless of what security is used, to check if your girlfriend is in love with someone else named 'Thor', and she used her computer to convey this information to someone, possibly Thor himself. But, don't ever use a computer without permission. And, trust me, if a girl cheats on you and you find out, pretend like you don't know, and suck it in. Walk away, and never look back. But, leave with your dignity. If she thinks you never knew she will respect you, even though you left her. Never let a woman know what you are thinking, and tell her everything you think she wants to hear. Like if she asks, "If I was burned over 90% of my body in a car wreck, would you still love me?", say yes. If she looks at you in the morning with her make-up all smeared, caustic breath, and a rats nest for hair you don't need to be honest with her when she asks, "Do you still think I'm beautiful without your beer goggles on?". Women always look beautiful, no matter what.

Ok, try this page to redirect STDOUT to a file using:

'exec'

and then direct it back to the terminal after you capture the dd output:

http://tldp.org/LDP/abs/html/x15794.html

on tldp.org

I made my comments. I think the whole concept will be easier to understand now.

Yes you did. Great stuff and essential to the general understanding of the subject.

I know we can leave the problem to be sorted out between the OS and the hard disk controller but a little lecture by the expert comes in handy.

I using dd to backup ntfs partitions.
Up untill now i was cloning a partition to a partition but i decided to dump to a file this time.

so to backup sda1 (50gb partition with 3gb used space):
dd if=/dev/sda1 of=/.../9gbfile bs=512
but:
mkfs.ntfs -f /dev/sda1
dd if=/.../9gbfile of=/dev/sda1 bs=512 conv=notrunc
or
dd if=/.../9gbfile of=/dev/sda1 bs=512
(I used also bs=4b)
18106848+0 records in
18106848+0 records out
9270706176 bytes (9.3 GB) copied, 11600.9 seconds, 799 kB/s

don't recreate sda1 as it was. I can mount the 9gbfile fine as a loop device (loop device content and original sda1 content match).
Can i use dd to restore directly from the 9gbfile or first mount the 9gbfile and then use dd? 9gbfile is in a reiserfs 3.6 filesystem.

FYI:
I stuck this thread (I thought it was stuck before?) because I think it's a valuable resource. It is certainly a long thread, but the data is mostly pertinent, and is therefore also valuable.

Cool

Hi,

Dd copies file systems or files. If you make the file system smaller, and bitstream it to a file, when you bitstream it back to a bigger partition, the file system will shrink. I don't know how you could dd a 50 GB partition to a 9 GB file using dd. Dd will not ordinarily do that. Dd writes a file system. If you only want to back up what is used use:

cp -r /* /wherever-you-copy-to

That will copy files. You can also pipe dd to gzip and the resulting file will be quite a bit smaller. That method is contained somewhere in this thread.

And I've un-sticked it per our e-Mail communication,
thanks mate. :}

I still think that awesome machine should submit it as
a linux-answer, and that dd isn't sought by newbies
often enough to warrant the sticky thread for it.


Cheers,
Tink

This thread was designed from the ground up to be used by newbies. Everything is taught by example. Dd provides an insurance policy against losing a good install, and allows the newbie to experiment without fear of having to reinstall. This marks a remodeling an renovation in which a 1980's look was updated to third millennium, and obsolete, little used information was removed from the post. I hope the new motif is more suited to LinuxQuestions.org, and users can now get the information they need, quickly and easily.

Nice change. The new look is *much* more pleasing and sensible,
easier to read. Well done.




Cheers,
Tink

I wanted to start out by saying thank you for taking so much time and effort to document such a useful and complicated command. I've read through most of the previous posts on this thread and I'm not sure if I can do what I need to do with dd or not. I'm made a new thread here so that I wouldn't hijack anyone's thread, but you guys in here really seem to know your stuff, and if I can get you to look at my problem, I'm hopeful that one of you can give me a clue. I'm not sure if it is doable with dd or not, though some part of it will probably include dd. If you have a moment, please come by and check it out, and forgive the intrusion on such a wonderfully cared-for thread.

I did install the debian based insert.iso into my memory stick, unfortunatelly my machine is not able to boot from a usb device.
Now my usb memory appears to be 59.2MB instead of the 256 MB that it really is.
How can i restore my memory stick? i.e., erase the insert.iso file and restore the original size?

Thanks in advance

AwsomeMachine, can I ask you consider adding one specific example to your first page: i.e. copy the MBR and first partition from one drive to another without wrecking the partition table on the second drive. I think I can do this now, but it wasn't clear to me a few months ago, and I've badmouthed "dd" as a result of not being able to add it all up to make a working method. I doubt I'm the only one that would have trouble putting this particular 1+1 together to make 2.

256 MB is 1024 * 1000 * 256 bytes = 256 MB:

dd if=/dev/zero of=/dev/hdb1 bs=1024 count=256000

Make your file system of choice with the appropriate command:

mkisofs
mak2fs
makreiserfs
format

You might need to partition first. Or you can use gui tools like:

gparted