I read man and info and searched here but could not find definite answers.

Reason for asking is this:

/dev/hda2 is WinNT with NTFS. Can dd read any and all bits from that NTFS-partition for a complete backup? (It's been absolutely dastardly to set up, don't want to have to redo it when or if necessary).

I want to do

Will each and every bit of /dev/hda2 be back not only in the files, but also the partition table, system information of NTFS etc. so that I have a complete "new" and useable install on /dev/dha2?

PS.: I formatted but forgot to set /dev/hda2 on binary zeroes before installing, so bzip2 won't work with maximum achievable compression. Anybody know a tool unter windows (like "dd if=/dev/null of=/file/with /zeroes) to create a large file containing binary zeroes?

PPS.: Sometimes dd gives in a summary information like 12345+1 records written ... huh? what "+1" record (or sometimes "+0" at that) ... any ideas?

the +1 means part of a record.
dd will only get what you give it access to. if you tell it to get hda2, it will get all and only hda2. the partition table is at the beginning of the disk, not in the partition itself, except in logical drives, sort of. and you would have to extract it to a partition that was exactly the same size or bigger, or it wouldn't work. only bootloaders within the partition itself would be restored.

you can use dd under windows with cygwin, and the command is
dd if=/dev/zero of=/filename

a little cleaner

dd if=/dev/hda2 | bzip2 > /filename.dd.bz2

use split if you need to if anything gives you grief from over > 2 gig files

dd if=/dev/hda2 |bzip2 |split --bytes 2000m - /1/hda2.dd.bz2-

bzip2 -cd /filename.dd.bz2 > /dev/hda2

Ah thank you.

Even though the data reside on NTFS?

Oh sh*t! No writing to NTFS. My whole scheme down the drain ...
Now I'll have to use partimage, and that's so inartistic .

That's why I would use bzip2.

Good point. Boot loader and boot.ini file to be added to backup.

I'll have to look into cygwin, then.

Thanks whansard. Anybody else? Please?

quote:Originally posted by whansard
dd will only get what you give it access to. if you tell it to get hda2, it will get all and only hda2. the partition table is at the beginning of the disk, not in the partition itself, except in logical drives, sort of.

Even though the data reside on NTFS?

no, no,
the filesystem information is backed up by dd also, just not the partition table. no format or mount. just write to a partition that's big enough to hold it all. when you access a partition with dd, it doesn't care what the filesystem is. it just copies bytes. that includes everything there. you can copy any filesystem with it, as long as it's the same size place you restore it to. and the windows boot loader inside the partition requires that you restore it to the same numbered partition.

Okay, so I'd use (learned that just today ). And for the rest, bzip2 compresses the partition contents after all, so size is not so bad a restriction. Lastly, I'd restore to the original partition with the original size of course.

And /devhda2 is a primary partition.

And the boot loader resides on /dev/hda1 (C: ).

[Edit]
Um, come to think of it, using dd if=/dev/hda2 really should include the partition table in its input, shouldn't it?!?

you could get the partition table with dd if=/dev/hda of=/table.dat bs=512 count=1

say your hda2 goes from cylinder 2000-4000
dd if=/dev/hda2 of=backup, gets EVERYTHING from cylinder 2000-4000, nothing else. the partition table is in the first cylinder, first sector of the drive. that's before hda1 starts.

As I understand the disk layout, dd if=/dev/hda of=/table.dat bs=512 count=1 should give me the MBR (Master Boot Record). Partition table should be in the next record(s?), shouldn't it?

Now, to the read-only state of NTFS partitions, what would happen should I re-format /dev/hda2 as ext2 and then restore with dd my old WinNT backup, originally from NTFS. Would that change the file system type too?

[edit]
Is the mount status relevant at all with the dd command? Or can I write/output to any device using dd without prior mounting?

the partition table is part of the master boot record. i actually have a 180 gig hard drive that i keep 2 full uncompressed backups of my 80 gig hard drive on. i use 2 different partition tables for that and i restore them that way. dd or cat, it doesn't matter. like cat table1>/dev/hde puts the first partition table on that drive.

http://www.boot-us.com/gloss04.htm

yes, the dd restore would put hda2 back as ntfs. i have my copies of solaris 8 and freebsd and netbsd and openbsd backed up just that way. they all exist on my hard drive from cylinder 261-999. ( or they did anyway. not right now.) and i just dd the backup to a partition existing there, and it's back to any of those OS's with their filesystems intact.

don't forget you could actually mount the backup file as a loopback filesystem. if you mounted something the kernel could write to ok, like ext2 or reiserfs, then you could write to the big file and make changes in it with it mounted.
mount -o loop /backup.file /mountpoint
cd /mountpoint
ls
as long as the filesystem isn't compressed anyway, i think. i'll show you.

/# fdisk -l
Disk /dev/hda: 81.9 GB, 81964302336 bytes
255 heads, 63 sectors/track, 9964 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 260 2088418+ 6 FAT16
/dev/hda2 261 1500 9960300 c W95 FAT32 (LBA)
/dev/hda4 1501 9964 67987080 85 Linux extended
/dev/hda5 1501 3500 16064968+ 83 Linux
/dev/hda6 3501 3700 1606468+ 83 Linux
/dev/hda7 3701 4000 2409718+ 83 Linux
/dev/hda8 4001 9964 47905798+ 83 Linux
/# dd if=dev/hda6 of=/hda6.file bs=1M
1568+1 records in1568+1 records out
/# mount -o loop hda6.file /3
/# cd /3
/3# ls
total 326
4 . 4 Win4Lin 4 floppy 4 nap 4 usr
2 .. 4 bin 4 hda1 4 nuke 4 util
4 1 0 bmp 4 hda2 4 opt 4 var
4 10 68 bookmark.htm 4 hda3 4 proc 4 varia
4 2 4 boot 4 home 4 root 4 will
16 2.2.14 4 burnup 4 jpg 4 sbin 4 win1
16 2.2.16 4 cdrom 4 lib 4 scdrom 4 work
12 2.4 32 dev 4 linux 4 tangeman 4 xc
4 21 4 dos 4 list 4 test
4 22 4 download 4 lost+found 4 test.txt
4 3 4 etc 4 mnt 4 tmp
/3# df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda5 16064472 9683024 6381448 61% /
none 452364 0 452364 0% /dev/shm
/dev/hda1 2088128 1228192 859936 59% /hda1
/dev/hda2 9950560 1167216 8783344 12% /hda2
/hda6.file 1581200 1322232 178648 89% /3

/3# cd /
/# umount /3
/# fsck /hda6.file
fsck 1.33 (21-Apr-2003)
e2fsck 1.33 (21-Apr-2003)
/hda6.file: clean, 62957/200928 files, 336875/401617 blocks
/#

crazy stuff huh.



you do not want to have a filesystem mounted when you are overwriting it with dd. you can mess it up, and will likely not be able to umount it either. you don't want it mounted when you are backing up either.

you can actually use a disk editor, like norton disk editor, if you had to for this type of stuff. it's just harder since you don't have the linux devices to write to, which make this stuff easy. you would have to figure out the start and ending sectors for that.

Didn't know that. I thought the MBR just starts a known boot loader. What happens with a big partition table causing an overflow of the 512 bytes of the MBR? Or is there a limit to the size of the partition table? I'll read the link you provided to find the answers there... thanks by the way.

Huh? Wouldn't that set your system to wobbling -- I mean, the partition table better reflects the physical layout of your drive(s) or you can't successfully mount...

Slick trick

Won't help with NTFS, though...

Thanks again crazy stuff indeed.

Oh. I'll keep that in mind.

Thank you for an excellent and helpful discussion.

it could help with ntfs. you could mount it read-only and copy all the files out of it if you had to. you could make a fat32 partition, mount the ntfs image as read-only, and copy all the files from the image to the new partition.

yes, there is a limit to the size of the partition table. that link explains it well, under partition table format i think.

To sum it up: dd definitely is "low level", enough so that it doesn't even need a device to be mounted to be able to write to it. It doesn't care what file system exists for read and write operations. (It can read / write to mounted devices / file systems though, using a file name for the "if=" or "of=" parameter(s).) I conclude from this, that dd can get direct access to the devices with the blessings of the kernel.

Powerful and dangerous -- and capable of doing exactly what I want it to do.

Thanks for a most instructive discussion, whansard .

yeah. it's really just a data reader-copier-manipulator. it's the linux devices that let you do the cool stuff with it. you can even copy a hard drive with "cat" . cat /dev/hda > /dev/hdb.

also, i like to be able to watch dd's progress, so i edited the source code, and added a counter, and i changed the default block size to 1 meg, so i could watch the meg's copied.
it was only a few lines of code, and a quick compile. i named that version ddd, and i have the standard and my version both. sometimes you can be copying stuff, and you need to know how long it's going to take, or how fast it's going. maybe dma mode got turned off or something, but you can't tell, because dd doesn't tell you anything until it's done.
this is the area in dd.c i changed. about line 475. the part commented wills hacks.
i can't remember exactly what i changed, but i found where dd was keeping track of the number of blocks copied, and outputted that variable to the screen.

if (nwritten != output_blocksize)
{
error (0, errno, _("writing to %s"), quote (output_file));
if (nwritten > 0)
w_partial++;
quit (1);
}
else
w_full++;
fprintf (stderr, _("%s records out\r"), /* wills hacks */
human_readable (w_full, buf[0], 1, 1));
oc = 0;