LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 03-21-2007, 02:14 PM   #301
sidder
LQ Newbie
 
Registered: Feb 2007
Distribution: sid64
Posts: 2

Rep: Reputation: 0

From tfm about

Sending a USR1 signal to a running ‘dd’ process makes it print I/O
statistics to standard error and then resume copying.
$ dd if=/dev/zero of=/dev/null& pid=$!
$ kill -USR1 $pid

I still haven't found how to dd a partition out of a sparse file
 
Old 03-21-2007, 07:43 PM   #302
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Sparse file

Quote:
Originally Posted by sidder
From tfm about

Sending a USR1 signal to a running ‘dd’ process makes it print I/O
statistics to standard error and then resume copying.
$ dd if=/dev/zero of=/dev/null& pid=$!
$ kill -USR1 $pid

I still haven't found how to dd a partition out of a sparse file
For everyone who doesn't know, a sparse file is a placeholder of a certain size on a hard drive. It doesn't take up as much disk space as its size. It just reserves space for a large file. One must populate the sparse file with something, like zeroes for instance. I have tried dd with everything to do with sparse files and find it works fine. I don't know how or why one would need to dd a partition out of a sparse file. If the sparse file has a file system on it it could be mounted as a loop back device partition, but to use dd on such a thing you wouldn't need to mount it, but only:

dd if=/home/sam/sparsefile.img of=/dev/sda3

Sparse files withing a partition also dd to the new partition perfectly.

A useful example of a sparse file is this:

dd if=/dev/zero of=/dev/ram7 bs=4096 count=4096

to make a ramdisk into a ramdrive. If you put a file system on this device you can mount it just like a HDD, and write files to it. In the OP there is an example of this, with the proper command line to format and mount the drive. A ramdrive has throughput comparable to HDD cache speeds (950 MB/s Ramdrive vs 50 MB/s Sequential disk reads vs 950 MB/s HDD cache transfers) If you are using
a web browser it can be helpful to place the browser caches in a ramdrive so it doesn't need to search 400,000 cached jpg files off the disk platters every time you go to a new web site. The ramdrive is also useful for working on large documents, or programming projects which require many compiles.
 
Old 03-28-2007, 08:06 PM   #303
CyberCod
Member
 
Registered: Jul 2006
Posts: 35

Rep: Reputation: 15
Hey, very informative thread you have here.

One thing you've impressed upon me is how dangerous this command can be if misused. So I'd like to verify that I understand you correctly before I attempt to use the snippet of code that I've gotten from you.

Code:
dd if=/dev/hda of=/hda3/backup/MBR.image bs=446 count=1
This (from what I gather) will back up the MBR of hda to a file called MBR.image, in the /hda3/backup directory... yes?

and to restore the mbr would be the opposite...
which would be

Code:
dd if=/hda3/backup/MBR.image of=/dev/hda bs=466 count=1

I'm trying to use this in a restoration script I'm working on at http://www.linuxquestions.org/questi...d.php?t=535223

My other question is, would this also back up the grub portion of the MBR? would restoring this restore grub (assuming it is installed) as well?

Thanks for your time.

Last edited by CyberCod; 03-28-2007 at 10:00 PM.
 
Old 04-09-2007, 12:37 AM   #304
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
I was on vacation for a week

Quote:
Originally Posted by CyberCod
Hey, very informative thread you have here.

One thing you've impressed upon me is how dangerous this command can be if misused. So I'd like to verify that I understand you correctly before I attempt to use the snippet of code that I've gotten from you.

Code:
dd if=/dev/hda of=/hda3/backup/MBR.image bs=446 count=1
Files need to be placed in a directory, within a file system. Files cannot be directly written to devices. The way to write to a file is to specify a path, starting with the mount point of the drive. For the / directory, which is mounted as root you begin with /. I always use /home/sam in my examples.

Quote:
Originally Posted by CyberCod
This (from what I gather) will back up the MBR of hda to a file called MBR.image, in the /hda3/backup directory... yes?
Quote:
Originally Posted by CyberCod
and to restore the mbr would be the opposite...
which would be

Code:
dd if=/hda3/backup/MBR.image of=/dev/hda bs=466 count=1
your reasoning is correct, but again, use a valid path.

Quote:
Originally Posted by CyberCod
I'm trying to use this in a restoration script I'm working on at http://www.linuxquestions.org/questi...d.php?t=535223

My other question is, would this also back up the grub portion of the MBR? would restoring this restore grub (assuming it is installed) as well?
You can always reinstall grub from an install CD/DVD, which would be the easy, correct method, I have found, when cloning to different sized partitions, especially if there are multiple os's involved, and then you might still need to edit /boot/grub/menu.lst. I strongly do not suggest using a script to restore data, unless it is a production system that doesn't change very often. For a consumer desktop I recommend, and this is my latest recommendation:

Make /home on a separate partition from /. Make / about 15 GB. Then you can backup / to /home with dd. If you trash / with some slip of the finger as the root user, or like to experiment, no matter what you do you can always boot with the http://www.efense.com/helix CD and write the backed up, known good / back from /home/sam to /. If you always do a backup:

dd if=/dev/sda1 of=/home/sam/rootpart.img bs=4096 conv=notrunc,noerror

and /dev/sda1 is root, and /home is on a different partition to restore:

dd if=/home/sam/rootpart.img of=/dev/sda1 bs=4k conv=notrunc,noerror

Quote:
Originally Posted by CyberCod
Thanks for your time.
 
Old 06-03-2007, 04:04 AM   #305
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Dd is good!! :-)
 
Old 06-09-2007, 05:58 AM   #306
dodjie60
LQ Newbie
 
Registered: Aug 2005
Location: Equatorial Guinea
Distribution: OpenSuSE 10.2/Fedora 7/Ubuntu
Posts: 27

Rep: Reputation: 15
Stupid me, I should have read this thread yesterday before I "cp -a /mnt/sda7/* /mnt/sda5".

Anyway just came across this thread a while ago and it was great. Goes straight to my bookmark. Really awesome.
 
Old 06-10-2007, 12:14 AM   #307
thegeekster
Member
 
Registered: Dec 2003
Location: USA (Pacific coast)
Distribution: Vector 5.8-SOHO, FreeBSD 6.2
Posts: 513

Rep: Reputation: 33
Rejuvenating hard drive script

Very nice thread.....

I like the idea of being able to use dd to regenerate/rejuvenate a hard drive, so I whipped up this little script to automate the process a bit, with some basic sanity checks......Comments/feedback welcome....

regenhdd
Code:
#! /bin/sh
#
# regenhdd : Regenerate/rejuvenate hard drive by rewriting all data back onto
#            itself.  Restores possible data loss on unused drives.
#

### SANITY CHECKS
  # Commandline check:
if ! [ $# -eq 1 -o -b "${1}" ] ;then
	echo -e "\tUSAGE: ${0} <block_device>" >&2
	echo -e "\t NOTE: Device must be a valid block device, such as '/dev/hda'." >&2
	exit 1
else
	DEV=$1
fi

  # Check for mounted partitions:
if grep -q "${DEV}.\+rw" /proc/mounts ;then
	echo -e "Please make sure all partitions on ${DEV} are not mounted, or at least mounted as read-only (ro) to avoid possible filesystem corruption." | fold -s >&2
	echo -e "If this is the root filesystem ( / ), then drop into single-user mode (runlevel 1) and make sure the filesystem is mounted as read-only.  Even better, reboot to a live cd/usb stick and run this program from there." | fold -s >&2
	exit 1
fi

### Time to go to work
echo "Regenerating block device, ${DEV}.  This may take a while..."

nice dd if=${DEV} of=${DEV}  ||  exit $?

echo 'Done!'

exit 0

---thegeekster
 
Old 06-11-2007, 08:19 PM   #308
SSVegito888
LQ Newbie
 
Registered: Jun 2007
Posts: 1

Rep: Reputation: 0
Can you use the dd command on Live CDs or Live DVD's such has Knoppix 5.1.1?

If so, is it any different than using dd on installed distros?

Do devices mount in a different folder on Live CDs/DVDs?

If so, in what folder do the devices mount?

I ask all of these questions because I am trying to image my HDD hda1, MBR and all, to my external HDD "sdb2" and save the output as an image(.img).

Again, I am using Knoppix 5.1.1 DVD

Thanks

Last edited by SSVegito888; 06-12-2007 at 09:49 PM.
 
Old 06-13-2007, 03:57 AM   #309
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Dd on live CDs

Quote:
Originally Posted by SSVegito888
Can you use the dd command on Live CDs or Live DVD's such has Knoppix 5.1.1?

If so, is it any different than using dd on installed distros?

Do devices mount in a different folder on Live CDs/DVDs?

If so, in what folder do the devices mount?

I ask all of these questions because I am trying to image my HDD hda1, MBR and all, to my external HDD "sdb2" and save the output as an image(.img).

Again, I am using Knoppix 5.1.1 DVD

Thanks
Dd works the same way on a live CD as it does from a hard drive installation. You didn't mention any problems, but I'm guessing your external drive might not show up in Knoppix. You can try hotplugging it, and then do:

dmesg | tail

pretty quickly after you plug in the external hard drive. The other problem might be file system dependant. Certain file systems only allow 2 GB files, and certain other ones only 4 GB files.
 
Old 06-18-2007, 02:24 PM   #310
fw12
Member
 
Registered: Mar 2006
Distribution: Fedora core, Ubuntu
Posts: 168

Rep: Reputation: 30
Quite helpful post. Thank you.

The only problem is that the page is stretched out horizontally.
The first DIV contains:
dd if=<source> of=<target> bs=<byte size>("USUALLY" some power of 2,...

So to read each line, I have to keep scrolling back and forth. This is very annoying and distracting.

I suggest the width of the DIV be reduced to 790px.
 
Old 06-19-2007, 02:54 PM   #311
fw12
Member
 
Registered: Mar 2006
Distribution: Fedora core, Ubuntu
Posts: 168

Rep: Reputation: 30
Well, I finally tested this DD thing to clone a hard drive. The command I used was:

# dd if=/dev/sda of=/dev/sdb conv=notrunc,noerror

Once the cloning was done, I unplugged the original drive, and left the newly cloned drive in.
It booted up great, and everything seemed ok, like the original drive.

Unfortunately the command,
# crontab -e
gave me the error:

cron/root: Input/output error

I don't know what this error means, or how to fix it.

So, I assumed the drive was bad.

So I removed the supposedly bad drive, put in another blank drive, hooked up the original drive, and repeated:

# dd if=/dev/sda of=/dev/sdb conv=notrunc,noerror

This time, the cloning didn't give the error above. The cloned drive seems to be an exact copy of the original.
 
Old 06-20-2007, 02:54 AM   #312
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
A very useful suggestion

Quote:
Originally Posted by fw12
Quite helpful post. Thank you.

The only problem is that the page is stretched out horizontally.
The first DIV contains:
dd if=<source> of=<target> bs=<byte size>("USUALLY" some power of 2,...

So to read each line, I have to keep scrolling back and forth. This is very annoying and distracting.

I suggest the width of the DIV be reduced to 790px.
I believe this property is browser dependent.
 
Old 06-20-2007, 02:59 AM   #313
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Post I/O error is almost always the drive hardware.

Quote:
Originally Posted by fw12
Well, I finally tested this DD thing to clone a hard drive. The command I used was:

# dd if=/dev/sda of=/dev/sdb conv=notrunc,noerror

Once the cloning was done, I unplugged the original drive, and left the newly cloned drive in.
It booted up great, and everything seemed ok, like the original drive.

Unfortunately the command,
# crontab -e
gave me the error:

cron/root: Input/output error

I don't know what this error means, or how to fix it.

So, I assumed the drive was bad.

So I removed the supposedly bad drive, put in another blank drive, hooked up the original drive, and repeated:

# dd if=/dev/sda of=/dev/sdb conv=notrunc,noerror

This time, the cloning didn't give the error above. The cloned drive seems to be an exact copy of the original.
The only thing dd can do for a drive with I/O errors is write zeroes to every sector. Then one needs to use a hard drive utility from the drive manufacturer's web site to relocate the bad sectors.
 
Old 06-21-2007, 07:03 AM   #314
spindles
Member
 
Registered: May 2005
Location: New Zealand
Distribution: Suse 9.1 Linux 2.6.4-52-default, Kubuntu Feisty
Posts: 136

Rep: Reputation: 15
Bookmarked.
Thanks.
I have been thinking about hard drive health. I'll be looking this up again.
 
Old 06-22-2007, 04:03 AM   #315
alibeheshti
LQ Newbie
 
Registered: May 2007
Posts: 8

Rep: Reputation: 0
very good explanations.

Hi,
I found the answers to many of my questions. thank you very much.you are a very good teacher.


Here is one of my problems with my home network and its internet connection sharing. Maybe you could help me to solve my problem.


1- I have a xp(windows) which is connected to internet using a dialup modem,I owuld like to set it up as a gateway (proxy) for my linux (redhat9) machine to access the internet.

2- my second question is exaxctly vise versa, I have a redhat9 installed pc which is connected to internet using a dialup modem and I would like to set it up as a gateway for my xp(windows) machine to access the internet.

thank you in advance

best regards
alibeheshti (linux newbie)
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM


All times are GMT -5. The time now is 01:11 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration