LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 07-16-2006, 04:27 AM   #166
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162

Quote:
Originally Posted by Copy
Hello again.

This thread continues to become better and better with time. Cheers, Awesome!

I have another question. Here goes:
I'm trying to clone Windows 9x partitions, and, as I mentioned earlier in the thread, I don't have any large hard disks available so I'm trying to backup my partitions to files instead of to the raw disk itself. The disk I want to clone is 120GB in capacity, but only has 18 or so GB of used space. I've run into a problem, however. I'm using the 'count=x' option to control the size of the files I'm making so that I can split the partitions into pieces that will fit nicely onto DVDs... But instead of truncating the empty space on the partition, dd just obediently copies as many blocks as I tell it to, even if they're empty. I know this is true, because dd didn't stop cloning even after it had passed all the used space on the partition, and all the files I made were exactly the same size--none were smaller. I definetly am not using the 'conv=notrunc' option... So my question is, if I use the 'count=x' option, does dd simply not truncate empty space? And if so, how do I use dd (or is there a better way) to clone my partitions but split the clones up into files of an arbitrary size that I specify? Here's the exact command I'm using:

dd if=/dev/hda of=/mnt/mydir/hda1_piece1_07-14-2006.ddclone bs=4096 count=1147460 conv=noerror

dd if=/dev/hda of=/mnt/mydir/hda1_piece2_07-14-2006.ddclone bs=4096 count=1147460 skip=1147460 conv=noerror

...and so on...

Thanks a million in advance for your time and help!
Cheers!
If you are using count=, that overrides leaving out notrunc. So then, you would make a complete backup file, and chop it up in small pieces with the count option.
 
Old 07-16-2006, 04:32 AM   #167
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Quote:
Originally Posted by OlivierB
Hi,

I am trying to copy my old 40Gb disk to a brand new 80Gb one, so as to keep my old OSes (winXP & Mandrake9.1) and add a brand new Ubuntu.
Here is the process I followed:

1- I partitionned the new disk with Partition Magic (PM) to make sure the partitionning is the same for the copies of old partitions.
Note: I tried to copy partitions with PM, but it fails (error 510 whatever the partition).

2- Then I install Ubuntu on the free space.

3- From Ubuntu, I use dd to copy the old partitions to my new disk.

It seems to have worked properly (good number of blocks read, and files are accessible), but each time dd was ending with a segmentation fault.
Any idea why dd was segmenting ?

Regards,
olivier
It is probably something in the surplus sectors at the end of the disk that is dd resistant. Read about surplus sectors in my original post. Then, overwrite them with random characters. If that doesn't work, make a file the size of the surplus sectors, and write the file in one block to the surplus sectors.
 
Old 07-16-2006, 09:47 AM   #168
Copy
LQ Newbie
 
Registered: Jun 2006
Posts: 9

Rep: Reputation: 0
Quote:
Originally Posted by AwesomeMachine
If you are using count=, that overrides leaving out notrunc. So then, you would make a complete backup file, and chop it up in small pieces with the count option.
Thanks for the tip! I'm just a little unclear as to how this will work since (as I said before) I'm trying to backup 18GB of data to my 10GB HDD, so would I use dd like this:

First to make the full clone files without the empty space:

dd if=/dev/hda1 of=/mnt/mydir/hda1_piece1_7-15-2006.ddclone bs=4096 conv=noerror

dd if=/dev/hda1 of=/mnt/mydir/hda1_piece2_7-15-2006.ddclone bs=4096 skip=[however many records already written] conv=noerror

And then to split up the clone files to fit them on (in this case) DVDs:

dd if=/mnt/mydir/hda1_piece1_7-15-2006.ddclone of=/mnt/otherdir/hda1_chunk1_7-15-2006.ddclone bs=4096 count=[4.7GB] conv=noerror

dd if=/mnt/mydir/hda1_piece1_7-15-2006.ddclone of=/mnt/otherdir/hda1_chunk2_7-15-2006.ddclone bs=4096 count=[4.7GB] skip=[4.7GB] conv=noerror

...and so on...

Would that do it?

Thanks for all your help, Awesome!
 
Old 07-16-2006, 12:50 PM   #169
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
Quote:
but each time dd was ending with a segmentation fault.
Did you try to dd from a live distro instead from inside ubuntu.
This may avoid the seg fault (I think it did at home once, not 100% sure)
 
Old 07-17-2006, 04:22 AM   #170
OlivierB
LQ Newbie
 
Registered: Jul 2006
Posts: 3

Rep: Reputation: 0
Quote:
Originally Posted by AwesomeMachine
It is probably something in the surplus sectors at the end of the disk that is dd resistant. Read about surplus sectors in my original post. Then, overwrite them with random characters. If that doesn't work, make a file the size of the surplus sectors, and write the file in one block to the surplus sectors.
Problem solved !

In fact windows was identifying my new disk as F: because its disk signature appears in the registry to refer to F:.

The solution was to get the signature of my old disk and replace the signature of my new disk with it in the MBR. A bit more on disk signature can be found on wikipedia's article on MBR (I don't put the link because I'm too "young" on this forum to have this right).

Thanks again.
I've learned a lot on partitionning and windows boot process this week-end

olivier
 
Old 07-18-2006, 04:31 PM   #171
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Post

Quote:
Originally Posted by Copy
Thanks for the tip! I'm just a little unclear as to how this will work since (as I said before) I'm trying to backup 18GB of data to my 10GB HDD, so would I use dd like this:

First to make the full clone files without the empty space:

dd if=/dev/hda1 of=/mnt/mydir/hda1_piece1_7-15-2006.ddclone bs=4096 conv=noerror

dd if=/dev/hda1 of=/mnt/mydir/hda1_piece2_7-15-2006.ddclone bs=4096 skip=[however many records already written] conv=noerror

And then to split up the clone files to fit them on (in this case) DVDs:

dd if=/mnt/mydir/hda1_piece1_7-15-2006.ddclone of=/mnt/otherdir/hda1_chunk1_7-15-2006.ddclone bs=4096 count=[4.7GB] conv=noerror

dd if=/mnt/mydir/hda1_piece1_7-15-2006.ddclone of=/mnt/otherdir/hda1_chunk2_7-15-2006.ddclone bs=4096 count=[4.7GB] skip=[4.7GB] conv=noerror

...and so on...

Would that do it?

Thanks for all your help, Awesome!
First:
dd if=/dev/hda1 of=/some-directory-not-on-hda1/complete-image-of-hda1.bin

Then:
dd if=/some-directory-not-on-hda1/complete-image-of-hda1.bin bs=1M count=4200 of=/some-directory/DVD-image-of-hda1-1.bin conv=noerror
dd if=/some-directory-not-on-hda1/complete-image-of-hda1.bin bs=1M skip=4200 count=4200 of=/some-directory/DVD-image-of-hda1-2.bin conv=noerror
dd if=/some-directory-not-on-hda1/complete-image-of-hda1.bin bs=1M skip=8400 count=4200 of=/some-directory/DVD-image-of-hda3-1.bin conv=noerror...

If you can't figure this out from my previous post, which said, "First make a complete image, and then chop it up in DVD size pieces with count, I don't think it is safe for you to have your data rely on dd. You might be better of with a backup program that is made for imaging onto DVD's.
 
Old 07-18-2006, 04:51 PM   #172
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Tell us more.

Quote:
Originally Posted by OlivierB
Problem solved !

In fact windows was identifying my new disk as F: because its disk signature appears in the registry to refer to F:.

The solution was to get the signature of my old disk and replace the signature of my new disk with it in the MBR. A bit more on disk signature can be found on wikipedia's article on MBR (I don't put the link because I'm too "young" on this forum to have this right).

Thanks again.
I've learned a lot on partitionning and windows boot process this week-end

olivier
You're pretty smart, and a fairly good writer. What you cited as the cause and solution don't add up, though. Where you using two different size disks, and overwrote the partition table of one with the other, and then did you redo the entire MBR again? Or did you patch in just four bytes at 0x1b8 as the post implies? I would like everyone to know how this actually worked, because if it is as you imply, that you patched in four bytes and this solved the whole problem, it is worthy of further investigation. Why don't you tell us, in your own words, the steps you took to solve this problem, in a little more detail.

Tom
 
Old 07-18-2006, 09:45 PM   #173
OlivierB
LQ Newbie
 
Registered: Jul 2006
Posts: 3

Rep: Reputation: 0
Quote:
Originally Posted by AwesomeMachine
You're pretty smart, and a fairly good writer. What you cited as the cause and solution don't add up, though. Where you using two different size disks, and overwrote the partition table of one with the other, and then did you redo the entire MBR again? Or did you patch in just four bytes at 0x1b8 as the post implies? I would like everyone to know how this actually worked, because if it is as you imply, that you patched in four bytes and this solved the whole problem, it is worthy of further investigation. Why don't you tell us, in your own words, the steps you took to solve this problem, in a little more detail.

Tom
I'm not sure my previous emails were clear, as the story has been cut into pieces.
Not knowing exactly what was confusing, here is the complete story:

- Hard drives:
1- old 40Gb formatted/partitionned with partition magic
hosting Windows XP home and Mandrake 9.1
starts making noise

2- new 80Gb formatted/partitionned using the same partition magic from my old windows
(40Gb is main disk, 80Gb plugged through usb)
I create identical partitions to make things simple,
plus one big ext3 for future use (Ubuntu).

- After partitionning:

1- Ubuntu installed on 80Gb (which now replaces 40Gb inside my laptop).

2- Use dd to copy each partition separately from 40Gb (temporarily connected through usb).
All my files are there. No problem (even if dd terminates with a segfault).

3- Update my Grub config file to tell how to start with copy of windows and copy of Mandrake.

4- Try rebooting under Mandrake and Windows (40Gb unplugged):
- Mandrake fine.
- Windows stops with blue "login" screen, a working mouse, but no user.

5- Investigate a lot on windows startup sequence and stuff.
Try cleaning "surplus sectors" as advised.
This only leads to windows telling me that F: needs to be checked.
But why F: ?...

6- The new windows partition was F: when formatted by old windows, meaning that something not modified by dd tells windows my first partition is F: (something on the partition table or MBR ?).

7- Reading some documentation on the MBR, I discover the "disk identifier" and the fact that windows uses it for various stuff. I guess that "various stuff" covers "assigning drive letters to partitions".

8- So, I get the disk identifier of 40Gb and replace the disk identifier of 80Gb with it.
Now windows works from scratch as on my old 40Gb.


Does that answer your question ?
I guess this is not a usual problem since not everyone partitions under windows (to get the exact same partitions) before copying partitions with dd (because partitionning magic couldn't do it).

olivier
 
Old 07-19-2006, 05:40 AM   #174
Copy
LQ Newbie
 
Registered: Jun 2006
Posts: 9

Rep: Reputation: 0
Quote:
Originally Posted by AwesomeMachine
First:
dd if=/dev/hda1 of=/some-directory-not-on-hda1/complete-image-of-hda1.bin
I understand that you can't clone a partition or device back onto itself. If you were confused by my post, let me assure you that what I meant by "/mnt/mydir" was "/some-directory-not-on-hda1/".

Quote:
Originally Posted by AwesomeMachine
Then:
dd if=/some-directory-not-on-hda1/complete-image-of-hda1.bin bs=1M count=4200 of=/some-directory/DVD-image-of-hda1-1.bin conv=noerror
dd if=/some-directory-not-on-hda1/complete-image-of-hda1.bin bs=1M skip=4200 count=4200 of=/some-directory/DVD-image-of-hda1-2.bin conv=noerror
dd if=/some-directory-not-on-hda1/complete-image-of-hda1.bin bs=1M skip=8400 count=4200 of=/some-directory/DVD-image-of-hda3-1.bin conv=noerror...
I understand all this too, and, except for the values of 'bs' and 'skip' you've specified in these commands, it's the same as in my post.

Quote:
Originally Posted by AwesomeMachine
If you can't figure this out from my previous post, which said, "First make a complete image, and then chop it up in DVD size pieces with count, I don't think it is safe for you to have your data rely on dd. You might be better of with a backup program that is made for imaging onto DVD's.
Forgive me for repeating myself, but I'm finding your answer a little hard to comprehend. Are you saying that there's no way to backup 18GB of raw data in uncompressed form to a 10GB HDD? Do I have to go out and buy a HDD with a capacity equal to or greater than the amount of data I want to backup? Is this process (which I tried to describe in my previous post and therefore won't show the commands here) possible or impossible to do:

Copy as much raw data to the smaller HDD as it can hold; burn DVDs of that data; skip over whatever data from 'if' that was already copied and copy the rest; burn the remaining data to DVDs.

I'm just trying to avoid having to buy another HDD. Can I do what I'm suggesting here (again, for the commands see my previous post)?

Thanks for your patience and help. I appreciate it.
 
Old 07-21-2006, 01:31 PM   #175
Copy
LQ Newbie
 
Registered: Jun 2006
Posts: 9

Rep: Reputation: 0
Hey again. While waiting for an answer to my previous question, I went ahead and decided to test if I could backup the partition on my other PC using dd to backup only the used space on the partition. My 2nd PC is a Windows 98 PC with 2 10GB HDDs: /dev/hda which is my Windows partition w/ FAT32 and uses 4.5GB of space, and /dev/hdb which I use for backups and is empty at the moment. Unfortunately, I was unable to copy only the used space from /dev/hda1 to /dev/hdb1--every time I tried, I found that the file I was dumping /dev/hda1 to was 10GB instead of 4.5GB. Here's what I did:

First I simply tried

dd if=/dev/hda1 of=/some_directory_on_hdb1/clone.file bs=4096 conv=noerror

which copied all 10GB (empty space obviously included) into my clone file.

Next, I read back in this thread and saw that if that doesn't work, I should try creating a file in the empty space on my partition and fill it with zeroes. So, since FAT32 has a filesize limitation (which I forget) I made a series of 1GB zero-filled files until the empty space on /dev/hda1 was completely full, like this:

dd if=/dev/zero of=/some_directory_on_hda1/zerofil1.bin count=2097152

dd if=/dev/zero of=/some_directory_on_hda1/zerofil2.bin count=2097152

dd if=/dev/zero of=/some_directory_on_hda1/zerofil3.bin count=2097152

...and so on...

When the empty space was filled up, I deleted these zero-filled files and then attempted the same command I listed above... but again, dd copied every sector in the partition, not just the 4.5GB of actual data. I don't know what I did wrong, if anything, or if there may be some configuration file or something that presets dd's operations somehow, but could you tell me why this didn't work? I'd greatly appreciate any help.

Cheers!
 
Old 07-21-2006, 02:59 PM   #176
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Copy, please go find a backup program that will image your partition onto DVD's for you. Dd isn't going to work. I don't what you are actually doing, but it's not working. I can't help you. It's probably not your fault, so don't worry.

Last edited by AwesomeMachine; 07-21-2006 at 03:04 PM.
 
Old 07-21-2006, 03:04 PM   #177
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
Olivier,

Thanks for the clarification.
 
Old 07-21-2006, 03:22 PM   #178
Copy
LQ Newbie
 
Registered: Jun 2006
Posts: 9

Rep: Reputation: 0
Could it be the Linux distribution I'm using? I use a live-CD called Sysresccd version 0.2.19. Maybe I should try using a different live-CD. Which do you use and/or recommend? I *really* don't want to buy any more software or hardware if I don't have to (budget is an issue with me).

Thanks again.
 
Old 07-22-2006, 12:16 AM   #179
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,593

Original Poster
Rep: Reputation: 162Reputation: 162
I was wrong

Quote:
Originally Posted by Copy
Could it be the Linux distribution I'm using? I use a live-CD called Sysresccd version 0.2.19. Maybe I should try using a different live-CD. Which do you use and/or recommend? I *really* don't want to buy any more software or hardware if I don't have to (budget is an issue with me).

Thanks again.
Dd actually does copy all the zeroes even without conv=notrunc.
 
Old 07-22-2006, 01:15 AM   #180
bangoram
Member
 
Registered: Jan 2006
Posts: 61

Rep: Reputation: 15
Talking Great Contribution

Its really a very good contribution!! I found it very usefull, any just now I have started working on this, I will be back whenever I have queries..
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 05:22 AM
The best way to learn? iz3r Programming 7 02-07-2005 12:00 AM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 04:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 02:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 02:02 AM


All times are GMT -5. The time now is 09:40 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration