LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 03-09-2007, 05:31 AM   #286
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654

I haven't read all 15 pages of comments, so forgive me if this has already been mentioned. I would suggest changing the example where you erase a hard drive.
Instead of using /dev/random, use /dev/urandom. The random device collects entropy information for the random numbers. It's source of entropy will run out after 1 or 2 K. The urandom device will supply a large number of psuedo-random numbers.
 
Old 03-09-2007, 05:33 AM   #287
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Smile Copy bootstrap code, but not partition table

Quote:
Originally Posted by Quakeboy02
AwsomeMachine, add one specific example to your first page:

Copy the bootstrap code from the MBR and the first partition from sda to sdb without overwriting the partition table on sdb.
If I knew linux well enough I could make waffles with it.

dd if=/dev/sda bs=1 count=446 of /dev/sdb
dd if=/dev/sda1 skip=63 of=/dev/sdb1 seek=63 bs=1b

If sdb doesn't boot when you get done install grub from a rescue CD and it will. But, it should work with those two dd command lines. Let me know how it goes. I have faith in you. You can badmouth dd all you want. It won't care.
 
Old 03-09-2007, 05:45 AM   #288
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Smile I have to respond to this.

Quote:
Originally Posted by jschiwal
I've read all 15 pages of comments. I would suggest changing the example where you wipe a hard drive. Instead of using /dev/random, use /dev/urandom. The random device collects entropy information for the random numbers. It's source of entropy will run out after 1 or 2 K. The urandom device will supply a large number of psuedo-random numbers.
When a user overwrites a hard drive it doesn't matter if a wide pattern can be established for the sequence used to overwrite the disk. If a user is making crytographic keys they would preferably use /dev/urandom to gather true randomness, but if /dev/urandom runs out of entropy it sits there and waits until it can gather more, whereas /dev/random will spit out the same sequence over and over. Trust me, /dev/urandom will never contain 250 GB of random bytes, and if the user leaves the machine it will take forever to generate enough truly pseudo-random bytes to wipe a drive.

But, your heart is in the right spot. I know you wanted to help this thread, so thanks.
 
Old 03-09-2007, 07:37 AM   #289
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
I think you have it backwards. /dev/random will run out quickly, but /dev/urandom goes on like the energizer bunny.

You need to move the mouse or use the keyboard to continue using /dev/random.

From the "man urandom" man page
Code:
       When  read,  the  /dev/random  device will only return random bytes within the estimated number of
       bits of noise in the entropy pool.  /dev/random should be suitable for uses that  need  very  high
       quality  randomness such as one-time pad or key generation.  When the entropy pool is empty, reads
       from /dev/random will block until additional environmental noise is gathered.

       A read from the /dev/urandom device will not block waiting for more  entropy.   As  a  result,  if
       there is not sufficient entropy in the entropy pool, the returned values are theoretically vulner‐
       able to a cryptographic attack on the algorithms used by the driver.  Knowledge of how to do  this
       is  not  available in the current non-classified literature, but it is theoretically possible that
       such an attack may exist.  If this is a concern in your application, use /dev/random instead.
 
Old 03-09-2007, 11:13 AM   #290
Sleepmore
LQ Newbie
 
Registered: Dec 2005
Posts: 2

Rep: Reputation: 0
Question Just want to make sure

Thanks for this thread, it has helped a lot. I just want to make sure I have everything correct. Here is my situation. I have 40GB drive which I want to backup onto a new 250GB drive. From what I have gleamed from the post all I need to do is:

dd if=/dev/hda of=/dev/hdb bs=4k conv=noerror

when the copy is finished, I:

resize2fs /dev/hdb

to resize the new drive to take up the rest of the new drive's space.

Now the questions:

1. First and foremost, is the commands I have above correct for making a copy of the old drive and resizing the new drive?

2. Do I need to place an entry into /etc/fstab for the new drive?

3. Do I need to mount the new drive?

3. Is there anyway to "see" the progress of the backup? Any command-line option?

Thanks for you help and I'm sorry if I'm the 10,000th person who has asked these questions.
 
Old 03-09-2007, 02:20 PM   #291
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,965
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
Hi, and welcome to LQ!

Quote:
Originally Posted by Sleepmore
Thanks for this thread, it has helped a lot. I just want to make sure I have everything correct. Here is my situation. I have 40GB drive which I want to backup onto a new 250GB drive. From what I have gleamed from the post all I need to do is:

dd if=/dev/hda of=/dev/hdb bs=4k conv=noerror

when the copy is finished, I:

resize2fs /dev/hdb

to resize the new drive to take up the rest of the new drive's space.

Now the questions:

1. First and foremost, is the commands I have above correct for making a copy of the old drive and resizing the new drive?
No. You don't want to copy the raw-device but the
partitions; same with the resizefs - unless of course
you actually *didn't* partition the old drive, and
created a file-system on the raw-device.

Quote:
Originally Posted by Sleepmore
2. Do I need to place an entry into /etc/fstab for the new drive?
Not for dd to work.

Quote:
Originally Posted by Sleepmore
3. Do I need to mount the new drive?
Not for dd to work.

Quote:
Originally Posted by Sleepmore
3. Is there anyway to "see" the progress of the backup? Any command-line option?
No, not to the best of my knowledge.



Cheers,
Tink
 
Old 03-09-2007, 02:27 PM   #292
Quakeboy02
Senior Member
 
Registered: Nov 2006
Distribution: Debian Squeeze 2.6.32.9 SMP AMD64
Posts: 3,238

Rep: Reputation: 121Reputation: 121
Posted by AwsomeMachine:
Quote:
If sdb doesn't boot when you get done install grub from a rescue CD and it will. But, it should work with those two dd command lines. Let me know how it goes. I have faith in you. You can badmouth dd all you want. It won't care.
AwsomeMachine,

I posted a howto on moving from IDE to SATA. I had hoped to use "dd" in penance, but the receiving partition is much larger than the origin partition, and I could only figure out how to do it using "cp -axu". If I understand dd correctly, when copying from one partition to another, the filesystem in receiving partition will be sized the same as that from the sending filesystem. If I've got that wrong, could you correct me? That's the part of "dd" that I'm just not sure of.

Added:
OH! I think that jschiwal in the following post has also answered my question: use resize2fs to match the copied filesystem to the partition it resides in.

Added again:
So, your example would be modified as follows? Or do I still have it wrong?
Code:
dd if=/dev/sda bs=1 count=446 of /dev/sdb
dd if=/dev/sda1 skip=63 of=/dev/sdb1 seek=63 bs=1b
resize2fs /dev/sdb1

Last edited by Quakeboy02; 03-09-2007 at 02:37 PM.
 
Old 03-09-2007, 02:29 PM   #293
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Quote:
Originally Posted by Sleepmore
Thanks for this thread, it has helped a lot. I just want to make sure I have everything correct. Here is my situation. I have 40GB drive which I want to backup onto a new 250GB drive. From what I have gleamed from the post all I need to do is:

dd if=/dev/hda of=/dev/hdb bs=4k conv=noerror

when the copy is finished, I:

resize2fs /dev/hdb

to resize the new drive to take up the rest of the new drive's space.

Now the questions:

1. First and foremost, is the commands I have above correct for making a copy of the old drive and resizing the new drive?

2. Do I need to place an entry into /etc/fstab for the new drive?

3. Do I need to mount the new drive?

3. Is there anyway to "see" the progress of the backup? Any command-line option?

Thanks for you help and I'm sorry if I'm the 10,000th person who has asked these questions.
1. The dd command is correct. You need to change the size of the partition using the fdisk command. Then use resize2fs to increase the size of the filesystem to match. The resize2fs command is missing the size to resize to. Also, it will only work for ext2 or ext3 filesystems. It would probably be a better idea A) clone the old drive to the new drive. B) resize it using gparted or your distro's partitioner tool. Many have a resize option. When you copy /dev/hda to /dev/hdb, you will be copying the MBR as well which contains the partition table of the disk drive.

2. When you are done, you will need to check the contents of /etc/fstab. These are the partitions you want mounted at boot. Since you are replacing the drive, it probably will be /dev/hda after removing your old /dev/hda and replacing it with the new drive. You will need to configure the drive for the ide cable connection, e.g. master, auto, lone master.

3. Don't mount the new drive at first. You are writing on the device for the entire drive, /dev/hdb and not a partition on the drive. You mount a partition's filesystem and not a drive.

4. Read the man pages for dd and resize2fs and fdisk before you start. The dd manpage and the intial posting show how to send a USR1 signal to the process to have it print out the progress on stderr. If you have SuSE, or Mandriva, or others, you may be able to do all of these steps graphically using the respective distro's partitioner program.

It also might be a good idea to backup important data on the source drive first in case something goes wrong. Sometimes backing up /home and reinstalling will give you a fresh start. Especially if you have done several distro version upgrades. A fresh install might work better. Some features of xgl for example worked after I did a fresh install versus the previous upgrade. Plus, installing an OS fresh is kind of fun!
 
Old 03-09-2007, 03:38 PM   #294
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Post Make that /dev/urandom

Quote:
Originally Posted by jschiwal
I think you have it backwards. /dev/random will run out quickly, but /dev/urandom goes on like the energizer bunny.

You need to move the mouse or use the keyboard to continue using /dev/random.

From the "man urandom" man page
Code:
       When  read,  the  /dev/random  device will only return random bytes within the estimated number of
       bits of noise in the entropy pool.  /dev/random should be suitable for uses that  need  very  high
       quality  randomness such as one-time pad or key generation.  When the entropy pool is empty, reads
       from /dev/random will block until additional environmental noise is gathered.

       A read from the /dev/urandom device will not block waiting for more  entropy.   As  a  result,  if
       there is not sufficient entropy in the entropy pool, the returned values are theoretically vulner‐
       able to a cryptographic attack on the algorithms used by the driver.  Knowledge of how to do  this
       is  not  available in the current non-classified literature, but it is theoretically possible that
       such an attack may exist.  If this is a concern in your application, use /dev/random instead.
Instead of using /dev/random to generate random bytes, use /dev/urandom for the above reasons. I will change the original post.
 
Old 03-10-2007, 03:29 AM   #295
iframe
LQ Newbie
 
Registered: Mar 2007
Posts: 15

Rep: Reputation: 0
AwsomeMachine:
Thanks for your advise.

Documenting what I did: The output of

dd if=/dev/zero of=/dev/sdb bs=1024 count=256000

was:

dd: writing `/dev/sdb': No space left on device
251905+0 records in
251904+0 records out
257949696 bytes (258 MB) copied, 67.7849 seconds, 3.8 MB/s


and built the file system with:

mkdosfs -I -F 32 -n KINGSTON -v /dev/sdb

and the output was:

mkdosfs 2.11 (12 Mar 2005)
/dev/sdb has 8 heads and 62 sectors per track,
logical sector size is 512,
using 0xf8 media descriptor, with 503808 sectors;
file system has 2 32-bit FATs and 1 sector per cluster.
FAT size is 3876 sectors, and provides 496024 clusters.
Volume ID is 45f27263, volume label KINGSTON .
 
Old 03-10-2007, 03:43 PM   #296
Tortanick
Member
 
Registered: Jul 2006
Distribution: Debian Testing
Posts: 299

Rep: Reputation: 30
just out of curiosoty dose dd create byte for byte perfect copies? bit for bit copies? imperfect copies?

Last edited by Tortanick; 03-10-2007 at 03:54 PM.
 
Old 03-10-2007, 04:58 PM   #297
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Dd is a bitstream duplicator, so it copies bit for bit. It has been tested by The United States Department of Justice computer forensics lab and found to be an exact duplicator.
 
Old 03-11-2007, 03:10 AM   #298
Tortanick
Member
 
Registered: Jul 2006
Distribution: Debian Testing
Posts: 299

Rep: Reputation: 30
thank you
 
Old 03-13-2007, 02:17 PM   #299
Sleepmore
LQ Newbie
 
Registered: Dec 2005
Posts: 2

Rep: Reputation: 0
To recap, I'm attempting to replace a 40GB drive with a new 150GB drive. I want the remaining space on the new drive to be given to the / partition.

I did:

dd if=/dev/hda of=/dev/hdb bs=4k conv=noerror

it seemed to work. I did a

fdisk -l /dev/hda

and compared it to

fdisk -l /dev/hdb

and the results are the same. Here is the output from the list of hdb:


/dev/hdb1 * 1 6 486136+ 83 Linux
/dev/hdb2 7 260 2040255 82 Linux swap
/dev/hdb3 261 4865 36989662+ 83 Linux


I then tried (because it looks like the / partition, which is the rest of the disk):

resize2fs /dev/hdb3

and got:

resize2fs: Filesystem has unsupported feature(s) (/dev/hdb3)


Here is my /etc/fstab (just in case it's an issue, I have only included line pertaining to the hard drive):


LABEL=/ / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
/dev/hda2 swap swap defaults 0 0


My questions:

1.) Did I need to format the new drive before I did the dd? I don't think I need to do this, but just checking. If I do need to "format" the new drive, what command do I use?

2.) What does the error I got from resize2fs mean? I have done the "goggle" search and the responses are unclear.

Thanks for your help.
 
Old 03-19-2007, 09:02 PM   #300
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
From the end of the thread look backwards until you get to the part where I discuss moving smaller partitions to larger ones. You do not need to format before dd'ing. If you want to keep the same system, partition the new drive, format it, and use cp -r * from the root directory of the old drive to the root of the new drive. Then, use the linux install CD in rescue mode, and reinstall the bootloader to the MBR.
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM


All times are GMT -5. The time now is 12:20 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration