LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 09-18-2005, 04:42 AM   #16
aus9
Guru
 
Registered: Oct 2003
Posts: 5,056

Rep: Reputation: Disabled

Awesome

1) great posting

2) any chance you edit your own first post to include all the juicy stuff so us beginners can read it in one hit? heh heh

3) Can you put a link in your signature like mine, so us beginners can click on the link if they feel the need to?

cheers
 
Old 09-19-2005, 02:07 AM   #17
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
More functionality for DD

I had to add this to the post.

To write dd output to the screen, leave out the "of" part.

dd if=/home/sam/myfile.txt bs=512 conv=notrunc

will write /home/sam/myfile to the terminal window. You might pipe through hexdump for a binary file:

Code:
dd if=/home/sam/myfile | hexdump -C | less

Last edited by AwesomeMachine; 08-22-2011 at 10:50 AM.
 
Old 09-19-2005, 06:04 AM   #18
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
Why not submit to LinuxQuestions.org > LinuxAnswers as a howto

I know about this
http://www.linuxjournal.com/article/1320
http://www.geocities.com/tipsforlinux/articles/036.html

Your post is comprehensive, adding external links can
help with other examples, other phrasing

Great work anyway
 
Old 09-19-2005, 06:13 AM   #19
scuzzman
Senior Member
 
Registered: May 2004
Location: Hilliard, Ohio, USA
Distribution: Slackware, Kubuntu
Posts: 1,851

Rep: Reputation: 47
Bookmarked?! You mean printed

Very well done

Have you thought of contributing this to the LQ Wiki or submitting it as a Linux Answer?

Last edited by scuzzman; 09-19-2005 at 06:14 AM.
 
Old 09-30-2005, 01:31 AM   #20
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
This post is largely covered in the OP. The reader can safely skip it.

I added a few things:

Learn the DD command


The basic command is structured as follows:

dd if=<source> of=<target> bs=<byte size>(usually some power of 2, not less than 512 bytes(ie, 512, 1024, 2048, 4096, 8192, 16384, but can be any number.) skip= seek= conv=<conversion>.

Source is the data being read. Target is where the data gets written. If you mess up, and accidentally reverse the source and target, you can wipe out a lot of data.

Examples::

Copy one hard disk partition to another hard disk:

dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=noerror

sda2, sdb2 are partitions. You want to copy sda2 to sdb2. If sdb2 doesn't exist, dd will start at the beginning of the disk, and create it. Be careful with order of if and of. You can write a blank disk to a good disk if you get confused.

Make an iso image of a CD:

dd if=/dev/hdc of=/home/sam/mycd.iso bs=2048

CD sectors are 2048 bytes, so this copies sector for sector. The result will be a hard disk image file of the CD. You can use "chmod a+rwx mycd.iso" to make the image writable. You can mount the image with "mkdir /mnt/mycd", this line in fstab: "/home/sam/mycd.iso /mnt/mycd iso9660 rw,user,noauto 0 0", save fstab, "mount -o loop /mnt/mycd". Then the file system will be viewable as files and directories in the directory /mnt/mycd. You can edit the image as you wish, and the new file will be "/home/sam/mycd.iso" dd does not write to CD's. You need to use a burning utility, or the cdrdao command

Copy a floppy disk:

dd if=/dev/fd0 of=/home/sam/floppy.image bs=2x80x18b

or

dd if=/dev/fd0 of=/home/sam/floppy.image

The 18b specifies 18 sectors of 512 bytes, the 2x multiplies the sector size by the number of heads, and the 80x is for the cylinders--a total of 1474560 bytes. This issues a single 1474560-byte read request to /dev/fd0 and a single 1474560 write request to /home/sam/floppy.image. This makes a hard drive image of the floppy, with bootable info intact. The second example uses default "bs=" of 512, which is the sector size of a floppy.

To format a series of floppies: Take one empty, never been used, formatted floppy; and:

dd if=/dev/fd0 of=/home/sam/floppy.bin

and make a hard disk image of a new formatted floppy, then:

load one of the floppies you want to format into the floppy drive, and:

dd if=/home/sam/floppy.bin of=/dev/fd0

This floppy will end up exactly like the never been used floppy you started with

To make a trusted DOS boot floppy

Get DOS from http://www.freedos.org/

dd if=/dev/fd0 of=/home/sam/floppy.bin

Open the image in a hex editor and change all references of "DRIVESPACE" to "XXXX.XXXXX" and "DOUBLESPACE" to "XXXXX.XXXXX". Change all references of "C:" to "A:".

This is now a trusted boot floppy. Trusted boot floppies are used to insure no writes are made to the hard drive on a floppy boot.

Copy a hard drive image of a floppy to a floppy:

dd if=/home/sam/floppy.image of=fd0 bs=2x80x18b

Copy just the MBR and boot sector of a floppy to hard drive image:

dd if=/dev/fd0 of=/home/sam/MBRboot.image bs=512 count=2

This copies the first 2 sectors of the floppy

Fix a floppy hacked by a DRM trojan.

Insert the floppy

dd if=/dev/null of=/dev/fd0
dd if=/home/sam/floppy.image of=/dev/fd0 conv=noerror

Normally, writing null to the first two sectors of a floppy renders the floppy totally unusable. It cannot even be formatted after that. Thanks to the image of the new, unused floppy, floppy.image, you can write the first two sectors back properly.

Cloning an entire hard disk:

dd if=/dev/sda of=/dev/sdb conv=noerror

In this example, sda is the source. sdb is the target. Do not reverse the intended source and target. Surprisingly many people do. notrunc means to not truncate. noerror means to keep going if there is an error. Normally dd stops at any error. if you have a question about a hard drive on whether or not it works, you can try to use it as the source drive for the dd command. You should get an error if it is not working. target drives need to be really messed up to give an error in dd.

Copy MBR only of a hard drive:

dd if=/dev/sda of=/home/sam/MBR.image bs=446 count=1

this will copy the first 446 bytes of the hard drive to a file. If you haven't already guessed, reversing the objects of if and of, in the dd command line reverses the direction of the write.

Wipe a hard drive of all data (you would want to boot from a cd to do this)

http://www.efense.com/helix is a good boot cd

The helix boot environment contains the DoD version of dd called dcfldd. It works the same way, but is has a progress bar.

dd if=/dev/zero of=/dev/sda

This is useful for getting rid of viruses, DRM trojans and the like.

It would be useful, at this time, to interject a tip:

I have several machines, but on the one I use a lot I have two SATA hard drives. They are exactly the same. Before I do anything dangerous, I boot from the helix CD, run

dcfldd if=/dev/sda of=/dev/sdb bs=4096 conv=noerror

and copy my present working sda drive system to the sdb drive. If I wreck the installation on sda, I just boot with the helix cd and

dcfldd if=/dev/sdb of=/dev/sda bs=4096 conv=noerror

Please note: bs=4096 works fast for machines with at least 128 MB of ram. Dd uses a lot of buffers. At bs=4096, on modern machines, the optimal transfer rate is reached for hard drives.

To make a file of 100 random bytes

dd if=/dev/urandom of=/home/sam/myrandom bs=1 count=100

Here, urandom is the linux random byte device. myrandom is a file. Byte size equals 1 and there are 100 of them. Gpg requires a random seed to generate keys. Generating a file of say 4096 random bytes, which can be passed to Gpg, will allow a truly random seed.

Write random data over a file before deleting it:

first do an ls -l to find filesize. In this case it is 3769

ls -l afile
-rw------- ... 3769 Nov 2 13:41 <filename>

dd if=/dev/urandom of=afile bs=3769 count=1

This will write random characters over the entire file.

Copy a disk partition to a file on a different partition. Do not copy a partition to the same partition.

dd if=/dev/sdb2 of=/home/sam/partition.image bs=4096 conv=noerror

This will make a file that is an exact duplicate of the sdb2 partition. You can substitue hdb, sda, hda, or whatever the disk is called.

Restore a disk partition from an image file.

dd if=/home/sam/partition.image of=/dev/sdb2 bs=4096 conv=noerror

This way you can get a bazonga hard drive and partition it so you can back up your root partition. If you mess up your root partition, you just boot from the helix cd and restore the image.

To covert a file to all uppercase:

dd if=filename of=filename conv=ucase

Copy ram memory to a file:

dd if=/dev/mem of=/home/sam/mem.bin bs=1024


The device /dev/mem is your system memory. You can actually copy any block or character device to a file with dd. Memory capture on a fast system, with bs=1024 takes about 60 seconds. Copying a 120 GB HDD takes about an hour. Copying a CD to hard drive takes about 10 minutes. Copying a floppy to a hard drive takes about 2 minutes. With dd, your floppy drive images will not change at all. If you have a bootable DOS diskette, and you save it to your HDD as an image file, when you restore that image to another floppy it will be bootable. dd is an excellent way to make an image of MS Windows XP install CD's. When you make a copy of such a cd, there is one sector that is of nonstandard length. It is the last sector. dd doesn't pad this sector, making the copy indistinguishable from the original. If you burn the CD using cdrdao, the resulting disk will be an absolutely exact copy of the original.

dd will print to the terminal window if you omit the “of=” part.

dd if=/home/sam/myfile

will print the file myfile to the terminal window.

If you are just curious about what might be on you disk drive, or what an MBR looks like, or maybe what is at the very end of your disk:

dd if=/dev/sda count=1 | hexdump -C

Will show you sector 1, or the MBR. There is the beginning of the loader code and the partition table in there. To see the end of the disk you have to know the total number of sectors for the disk, and the disk has to be set up with Maximum Addressable Sector equal to Maximum Native Address. The helix CD has a utility to set this correctly. In the dd command your seek value will be one less than MNA of the disk.

for a 120 GB Seagate SATA drives

dd if=/dev/sda of=home/sam/myfile skip=234441646

default bs=512, so this reads sector for sector, and writes the last sector to myfile.

Disks, even though there is LBA addressing now, still secretly are read in sectors, cylinders, and heads. There are 63 sectors per cylinder, and 255 heads per cylinder. Then there is a total cylinder count for the disk. You multiply out 512x63x255=bytes per cylinder. 63x255=sectors per cylinder. With dd you usually want to work with sectors per cylinder. With 234441647 total sectors, and 16065 sectors per cylinder, you get some trailing sectors which do not make up an entire cylinder, 14593.317584812. This leaves you with 5102 sectors which cannot be partitioned because to be in a partition you have to be a whole cylinder. Part cylinders do not count. It's like having part of a person. That doesn't really count as a person. So, what happens to these sectors? They become surplus sectors after the last partition. This a perfect place for sneaky programs to play, because you can't ordinarily read in there with an operating system. But, dd can.

It is really a good idea to check for anything writing to surplus sectors. For our Seagate 120 GB drive you subtract total sectors(234441647)-(5102) which don't make up a whole cylinder=234436545 partitionable sectors. Remember, native HDD sectors are 512, or 1b. If you don't specify “bs” in dd it defaults to 512.

dd if=/dev/sda of=/home/sam/myfile skip=234436545

this writes the last 5102 sectors to myfile. Launch “mc” to view the file. I swear, half the time Windows XP has left a weird, mutated MBR there. It like marks the disk for life that XP was there.

If there is something in there, you do not need it for anything. In this case you would write over it with random characters. Many digital rights management programs use surplus sectors to operate from, while enforcing DRM. These trojans, which are corporate trojans, are meant to enforce the security measures in copyrighted software. There are other various means to conceal such a trojan. One of these is a hidden partition. There is an undocumented type of partition which is called hidden. It is not visible to any operating system.

dd if=/dev/urandom of=/dev/sda bs=512 seek=234436545

Will overwrite the 5102 surplus sectors on our 120 GB Seagate drive.

If you want to check out some random area of the disk:

dd if=/dev/sda of=/home/sam/myfile bs=4096 skip=2000 count=1000

will give you 8,000 sectors in myfile, after the first 16,000 sectors. You can open that file with a hex editor, edit some of it, and write the edited part back to disk:

dd if=/home/sam/myfile of=/dev/sda bs=4096 seek=2000 count=1000

So there you got yourself a disk editor. It's not the best, but it works.

You can make a boot floppy: with the boot.img file, which is pretty easy to get. You just need a program that will literally start writing at sector 1.

dd if=boot.img of=/dev/fd0 bs=1440k

This makes a bootable disk you can add stuff to.


If you want to make a partition image on another machine:

on source machine:

dd if=/dev/hda bs=16065b | netcat targethost-IP 1234

on target machine:

netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

Netcat is a program, available by default, on almost every linux installation. It is like a swiss army knife of networking. In the preceding example netcat and dd are piped to one another. One of the functions of the linux kernel is to make pipes. The pipe character looks like two little lines on top of one another, both vertical. Here is how this command behaves:

On the source machine dd is told to read /dev/hda with a byte size which is kind of weird. This byte size is a cylinder. bs=16065b equals one cylinder on an LBA drive. Ok, then the dd command is piped to netcat, which takes as its arguments the IP address of the target(like 192.168.0.1, or any IP address with an open port) and what port you want to use(1234). Don't hit enter yet. Type the command for the target machine first, hit enter on the target machine, hit enter on the source machine. Now the bit stream copy will take place. This is kind of how Norton Ghost works to image a drive to another machine. All you have to do is boot both machines with the helix CD, and don't confuse the source machine with the target machine.


Ok, say you want to find out if your girlfriend or wife is cheating on you, having cyber sex, or just basically misbehaving with her computer. Even if the computer is secured with a password, you can boot with the:

http://www.efense.com/helix

CD and search the entire drive partition for text strings:

dd if=/dev/sda2 bs=16065 | hexdump -C | grep 'I really don't love him anymore.'

Will search the whole drive partition for the text string specified between the single quotes. Searching an entire disk partition several times can be quite tedious. This particular command string prints the search results to the screen, with the offset where it is located in the partition. dd works in the decimal system. Disk offsets work in hexidecimal.
Say you found that text string in your partition at offset 020d0d90. You convert that to decimal with one of the many calculators found in linux. This is decimal offset 34409872. Dividing by 512b per sector we get 67206.78125. now we know, to read the rest of what ever it is, and these numbers are just guestimates:

dd if=/dev/sda2 bs=16065 skip=2140 count=3 | less

This will put the output to the screen so you don't accidentally write a file over what you want to read. Piping dd to less will give you one screen at a time of output. With this method you search all the deleted files, any chat activity, and emails. It works no matter what security is being employed on the machine. It works with NTFS, ext2, ext3, reiserfs, swap, and FAT partitions. The helix CD is not fussy, and neither is the dd command.

On a related note, you can write the system memory to a CD. This is useful for documenting memory contents without contaminating the HDD. I recommend using a CD-RW so you can practice a little. This doesn't involve dd, but it's cool.

cdrecord dev=ATAPI:0,1,0 -raw tsize=700000000 driveropts=burnfree /dev/mem

to find the cdwriter

cdrecord -scanbus=ATAPI

This method records raw, so you have to do a

dd if=/dev/hdd | less

to view the recorded memory. Searching the recorded memory is as above

dd if=/dev/hdd | hexdump -C | grep 'string'

string is any ascii sequence, hex sequence (must be separated with a space: '55<space>aa<space>09' searches for the hex string '55aa09'), list:

'[[:alnum:]]' any alphanumeric characters
'[[:alpha:]]' any alpha character
'[[:digit:]]' any numeric character
'[[:blank:]]' tabs and spaces
'[[:lower:]]' any lower case alpha characters
'[[:upper:]]' any uppercase alpha character
'[[:cntrl:]]' ASCII characters 000 thru 037, and 177 octal
'[[:graph:]]' [:alnum:] and [unct:]
'[[unct:]]' any punctuation character
` ! ' # $ % ' ( ) * + - . / : ; < = > ? @ [ \ ] ^ _ { | } ~
'[[:space:]]' tab, newline, vertical tab, form feed, carriage return, and space
'[[:xdigit:]]' any hex digit
ranges('[a-d]' = any, or all abcd, '[0-9]' = any, or all 0123456789)

You can back up your MBR

dd if=/dev/sda of=mbr.bin count=1

Put this on a floppy you make with

dd if=boot.img of=/dev/fd0

Along with dd. Boot from the floppy and

dd if=mbr.bin of=/dev/sda count=1

Will restore the MBR.

I back up all my floppies to HDD. Floppies don't last forever, so I do

dd if=/dev/fd0 of=/home/sam/floppies/backup.bin

If my floppy fails, I can make unlimited copies

dd if=/home/sam/floppies/backup.bin of=/dev/fd0

Last edited by AwesomeMachine; 08-22-2011 at 11:04 AM. Reason: update
 
1 members found this post helpful.
Old 10-03-2005, 02:20 AM   #21
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
AwesomeMachine

Very good, as said before

There is also a use of dd to create a swap file
(as opposed to a swap partition)
Example in
http://www.pathogenomics.sfu.ca/brinkman/l-faq.pdf
look for swapon then the page just before

Last edited by Emmanuel_uk; 10-03-2005 at 04:21 AM.
 
Old 10-03-2005, 03:53 AM   #22
bruse
Member
 
Registered: Feb 2005
Location: internet
Distribution: Debian
Posts: 821

Rep: Reputation: 30
yes really a good help.
but still i am thinking is it possible from dd command.
we can extract or copy a portion of the content from CD-RW/ROM using dd command.
now my question is.. I have one cd-rom with full content .Can we copy a content of cd-rom from 300MB to 600MB.
Because remaning portion my cd-rom have full of input output error's.
Or is there any software for linux like for windows "virtual cd"?
-----thanks.
 
Old 10-03-2005, 05:56 AM   #23
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
dd does not do a reread on error, so this is the advantage

But maybe you want to look at readcd instead, which will do rereads on error
(No personal experience to help you more with)
 
Old 10-04-2005, 12:44 AM   #24
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
To recover a damaged CD: search the OP for foremost.

Last edited by AwesomeMachine; 08-22-2011 at 11:06 AM.
 
Old 10-04-2005, 03:06 AM   #25
bruse
Member
 
Registered: Feb 2005
Location: internet
Distribution: Debian
Posts: 821

Rep: Reputation: 30
good help.but still i need more.

as i said earlier i need to copy the disc content from 400MB to 600MB or whatever.

the reason why i am asking this is because of when i run the dd command with noerror,when the error comes it takes long time to skip that block.
it takes very very very long time.

and also i want to know the option " notrunc " what is that?what does it do?
 
Old 10-04-2005, 07:49 AM   #26
Imon
LQ Newbie
 
Registered: Aug 2004
Location: Washington, NE England
Distribution: Machine 1 = Slackware 10, Machine 2 = Slackware 10.1, Debian (Sarge). Machine 3 = Ubuntu 5.04
Posts: 10

Rep: Reputation: 0
Excellent thread AwesomeMachine.....most useful
 
Old 10-04-2005, 09:39 AM   #27
johny42
LQ Newbie
 
Registered: Oct 2005
Location: Kosice, Slovakia
Distribution: Fedora Core 5
Posts: 9

Rep: Reputation: 0
Re: Learn the DD command

Quote:
Originally posted by AwesomeMachine
dd if=/dev/zero of=/dev/sda conv=notrunc

This is useful for getting rid of viruses, DRM trojans and the like.
*laughing* Definitely, I'll use this next time I find a virus on my computer

But otherwise, a very good howto, found some useful things there I didn't know...
 
Old 10-04-2005, 05:15 PM   #28
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
AwesomeMachine

I like your tutorial, so here we go, back to this thread to bump it to the top ;-)

From http://www.columbia.edu/~ariel/acpi/acpi_howto.txt
another example of dd to dump some memory, and what it can be used for

Code:
If you want to look through memory yourself, and you have 32-bit hardware
   which is not EISA/MCA based, you could try looking for "RSD PTR" in 0e0000h
   through 0fffffh by grepping it out of /dev/mem, like this:
        # dd if=/dev/mem of=blot bs=64K skip=14 count=2
        # od -c -A x blot | grep 'R   S   D'
        01c9b0   R   S   D       P   T   R     312   D   E   L   L          \0

   If you see output like this, you know you have the root table stricture for
   ACPI, which means that you have at least some degree of support.
 
Old 10-05-2005, 03:10 AM   #29
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,978
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
How about something like ...
Code:
dd if=/dev/hda bs=2048 | gzip | ssh user@host "cat - | gunzip | dd of=/dev/hda bs=2048"
to clone a HDD over a network? :}


Cheers,
Tink
 
Old 10-06-2005, 02:27 AM   #30
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 61
This should work, although I've always thought that dd is far from the best method to clone a disk, because (unless there's something I misunderstand) dd only applies if both disk partitions are exactly the same number of raw bytes.

Yves.
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM


All times are GMT -5. The time now is 05:47 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration