LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 11-20-2005, 05:10 AM   #46
arjaya
LQ Newbie
 
Registered: Nov 2005
Posts: 4

Rep: Reputation: 0

Thank you for your help! It is much appreciated!
 
Old 11-28-2005, 07:06 AM   #47
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
AwesomeMachine, what happens if one copies a given partition onto a smaller partition? I was going to test this because I want to backup a partition which is far from full onto a smallish partition that I know is big enough to take all the data.
dd if=/dev/hda5 (30gb but only 3 gb of data) of=/dev/hdb14 (5gb of space)
Will this work? Will dd spit an error and stop when the destination is full? (wihthout trying to write to the next partition for example).

I'd rather know beforehand than being sorry

Last edited by Emmanuel_uk; 11-28-2005 at 07:07 AM.
 
Old 11-29-2005, 01:38 PM   #48
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Bigger Partition to Smaller Partition With DD

Quote:
Originally posted by Emmanuel_uk
AwesomeMachine, what happens if one copies a given partition onto a smaller partition? I was going to test this because I want to backup a partition which is far from full onto a smallish partition that I know is big enough to take all the data.
dd if=/dev/hda5 (30gb but only 3 gb of data) of=/dev/hdb14 (5gb of space)
Will this work? Will dd spit an error and stop when the destination is full? (wihthout trying to write to the next partition for example).

I'd rather know beforehand than being sorry
The only difference between a big partition and a small partition, besides size, is the partition table. If you are copying say sda to sdb, an entire drive with a single partition, sdb being smaller than sda, then you have to do:

dd if=/dev/sda skip=1 of=/dev/sdb seek=1 bs=4k conv=noerror

You can put the skip and seek anywhere you want, I just put them there to show what they go to. Skip skips input blocks at the beginning of the media(sda). Seek skips over so many blocks on the output media before writing(sdb). By doing this, you leave the first 4k bytes on each drive the same. This is eight sectors. The first sector is the Master Boot Record (MBR). This contains the partition table. You don't want to tell a drive it is bigger than it really is by writing a partition table from a larger drive to a smaller drive. In my example, we wanted to use a byte size (bs) of 4k or 4096 bytes. This speeds up the copy from the default 512 byte bs. But, your are limited in how small of a skip or seek you can make. In this example, the smallest skip or seek is 1, or 4096 bytes. Since the first 63 sectors of the drives are empty, except sector 1, the MBR, we can skip eight sectors without any problem missing part of the boot sector. One sector on a hard drive is 512 bytes.

Now, if you are copying say sda3 to sda2, this is different. What you want to do is this:

dd if=/dev/sda3 of=/dev/sda2 bs=4096 conv=noerror

Even if these are primary partitions, they are still logical to the main partition. Do not use the conv=notrunc option. Without notrunc, multiple blocks of zeros get abbreviated with a string of asterisks, saving a lot of space. if you use notrunc, all the zeros will be written out. On a smaller partition you want to save room. If you write out all the zeros, the smaller partition won't be big enough.
 
Old 11-30-2005, 04:20 PM   #49
Emmanuel_uk
Senior Member
 
Registered: Nov 2004
Distribution: Mandriva mostly, vector 5.1, tried many.Suse gone from HD because bad Novell/Zinblows agreement
Posts: 1,604

Rep: Reputation: 47
Thumbs up

Thanks. This was eductional. And my little test went half ok

As expected
Code:
dd: writing `/dev/hdb15': No space left on device
3957708+0 records in
3957707+0 records out
This was from ext3 to ext3, different size partitions (source 12Gb onto 4 Gb destination, on purpose,
although I want to test 12 Gb onto 14 Gb as well)

Fdisk says the destination was 15830829 blocks
3957708 * 4=15830832
So about ok

#ls -l
ls: blablafolder: Input/output error
some files and folder are visible are catable good
Filesystem Size Used Avail Use% Mounted on
/dev/hdb15 51G 12G 39G 24% /mnt/toto
Look really wrong because hdb15 is only 4 Gb/ So it took the info from the source directory

Not good
Code:
# e2fsck -f /dev/hdb15
e2fsck 1.36 (05-Feb-2005)
/dev/hdb15: recovering journal
e2fsck: unable to set superblock flags on /dev/hdb15
I went reading a bit more about superblocks
http://www.win.tue.nl/~aeb/linux/lk/lk-7.html
I am afraid for now, I need to read more about debug2fs etc.
 
Old 11-30-2005, 09:17 PM   #50
arjaya
LQ Newbie
 
Registered: Nov 2005
Posts: 4

Rep: Reputation: 0
Angry Stumped!

Hi,

I'm trying to extract the contents of a 4MM tape using dd.

The tape contains backup data from a novell netware 6.5 server.

Can I use dd to extract the contents of the tape?

I've tried using this command

dd if=/dev/st0 of=tapedata

But LINUX responds

dd: /dev/st0: Input/output error
0+0 records in
0+0 records out

I checked that all was well with the tape drive by successfully extracting
data from another tape containing linux backup data.

We are using Fedora Core 4, and an internal 4MM tape.

I have tried numerous block sizes to extract the data from this tape. I have also tried the command:
Code:
mt -f /dev/st0 setblk 0
Is there any other way to determine the block size of the tape?


Can anyone point me in the right direction on what I am doing wrong?


Any help is much appreciated!!!

Here are some details on the scsi st0 device.

Code:
[root@localhost /]# dmesg | grep scsi
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
scsi1 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
scsi1:A:0:0: Tagged Queuing enabled.  Depth 4
(scsi1:A:0): 6.600MB/s transfers (16bit)
(scsi1:A:0): 160.000MB/s transfers (80.000MHz DT, offset 127, 16bit)
Attached scsi disk sda at scsi1, channel 0, id 0, lun 0
scsi1:A:1:0: Tagged Queuing enabled.  Depth 4
(scsi1:A:1): 6.600MB/s transfers (16bit)
(scsi1:A:1): 160.000MB/s transfers (80.000MHz DT, offset 127, 16bit)
Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0
(scsi1:A:6): 6.600MB/s transfers (16bit)
(scsi1:A:6): 80.000MB/s transfers (40.000MHz, offset 32, 16bit)
scsi2 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
Attached scsi tape st0 at scsi1, channel 0, id 6, lun 0
[root@localhost /]#
[root@localhost /]#
[root@localhost /]# dmesg | grep st0
Attached scsi tape st0 at scsi1, channel 0, id 6, lun 0
st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA 1048575
[root@localhost /]#
Code:
[root@localhost /]# mt -f /dev/st0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 512 bytes. Density code 0x26 (DDS-4 or QIC-4GB).
Soft error count since last status=0
General status bits on (41010000):
 BOT ONLINE IM_REP_EN
[root@localhost /]#

Code:
[root@localhost /]# dd if=/dev/st0 of=/st0.img
dd: reading `/dev/st0': Input/output error
0+0 records in
0+0 records out
[root@localhost /]#
Code:
Nov 22 14:44:57 localhost kernel: st0: Block limits 1 - 16777215 bytes.
Nov 22 14:47:17 localhost kernel: st0: Incorrect block size.
[root@localhost log]#
 
Old 12-02-2005, 12:31 AM   #51
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
This seems to indicate one of two things:

When dd gives an i/o error, it is always because of the device being read from, in this case, /dev/st0. What you probably have is an unreadable tape.

Try this:

dd if=/dev/st0 of=tape.img bs=1b conv=notrunc,noerror

If this fails, try looking toward the end of the post for a link to dd_rescue. dd_rescue can start at the end of the tape, and restore backwards. You can also try "skip=1" to skip the first sector of the tape. If the tape is bad, at least the first sector is bad because you are getting 0 + 0 records. That means the tape fails before sector 1. If this fails, Sanderson Forensics sells a utility to get data off damaged tape media. Once you buy it, you can just download it. I've never used it, but I have used other programs made by Paul Sanderson and they are good.

I have also heard, but not verified, that dd has trouble with QIC-80 drives.
 
Old 12-25-2005, 02:07 PM   #52
microsoft/linux
Senior Member
 
Registered: May 2004
Location: Sebec, ME, USA
Distribution: Debian Etch, Windows XP Home, FreeBSD
Posts: 1,445
Blog Entries: 9

Rep: Reputation: 45
I'd like to back up my root partition, so as to be able to resize the partition, and I want to make sure I'd be doing this right before I mess anything up.

I'd need to run
Code:
dd if=/dev/hda3 of=/backup<an HD mounted across a network> bs=4096 conv=notrunc,noerror
to backup my root
and
Code:
dd if=/backup skip=1 of=/dev/hda3<being the new, larger, partition> seek=1 bs=4k conv=noerror
to put the backed up / on the new partition.

is this right?
 
Old 12-27-2005, 04:09 AM   #53
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Cool

Quote:
Originally Posted by microsoft/linux
I'd like to back up my root partition, so as to be able to resize the partition, and I want to make sure I'd be doing this right before I mess anything up.

I'd need to run
Code:
dd if=/dev/hda3 of=/backup<an HD mounted across a network> bs=4096 conv=notrunc,noerror
to backup my root
and
Code:
dd if=/backup skip=1 of=/dev/hda3<being the new, larger, partition> seek=1 bs=4k conv=noerror
to put the backed up / on the new partition.

is this right?
If you are going across a network, there are instructions in my original post for using netcat with dd. Just go back to the first page and do a search for "netcat" with you browser. I don't think dd works with NFS mount points. With netcat you can specify the actual device file on the remote backup machine. Also, you can back up root to a file, and even mount the file as a file system, which is a good test to see if the backup file is any good, on the remote machine. If you want the new partition smaller, don't use notrunc. Notrunc copies all the zeroes in empty sectors. Without notrunc, sections of the disk which are solid zeroes are abbreviated with a string of asterisks. So if you have 3000 sectors in a row, which are all blank, without notrunc, all 3000 sectors only consume 10 bytes. Any sector that has anything in it besides zeroes will not be truncated. I use notrunc for copying complete drives where everything has to end up in the right spot. For a single partition you don't need notrunc. If you want to see how much is on the root partition now do a <df -h>. Then you'll have some idea of the actual space needed. I really appreciate these replies because they give me all kinds of new ideas.
 
Old 12-28-2005, 09:59 PM   #54
microsoft/linux
Senior Member
 
Registered: May 2004
Location: Sebec, ME, USA
Distribution: Debian Etch, Windows XP Home, FreeBSD
Posts: 1,445
Blog Entries: 9

Rep: Reputation: 45
So in order to back up my / partition to a NFS mounted HD I'd have to do

on my laptop(whose root would be backed up)
Code:
dd if=/dev/hda3 bs=16065b | netcat <ip goes here> <port goes here>
where the ip is that of the remote machine?

Code:
netcat -l -p <port goes here> | dd of=/dev/hdb1(the HD I want the backup to go to) bs=16065b
is this correct? Would I just reverse the process to copy it back?
 
Old 12-29-2005, 12:15 AM   #55
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Lightbulb dd netcat

Yes, that is how you do it. The "bs=16065b" is an example of how to specify one cylinder per block. In reality, smaller bs values, such as 2048 or 2k, 4096 or 4k, 8192 or 8k are good for network backups. In fstab, make sure the NFS drives specify "rsize=8192,wsize=8192". This will make the backup work better.
 
Old 12-29-2005, 12:30 AM   #56
microsoft/linux
Senior Member
 
Registered: May 2004
Location: Sebec, ME, USA
Distribution: Debian Etch, Windows XP Home, FreeBSD
Posts: 1,445
Blog Entries: 9

Rep: Reputation: 45
what does all of that mean? A block is just a section of an HD right? Why do I need to be worried about the blocks?
 
Old 01-08-2006, 04:15 AM   #57
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian jessie/sid; OpenSuSE; Fedora
Posts: 1,591

Original Poster
Rep: Reputation: 162Reputation: 162
Lightbulb Blocks

Quote:
Originally Posted by microsoft/linux
what does all of that mean? A block is just a section of an HD right? Why do I need to be worried about the blocks?
Optimal block size, specified in the dd command by bs= gives optimal transfer rates. On a machine with even a minimal amount of ram, dd's default bs=512 is quite slow. 512 is one sector at a time. I have found this to be slow. I prefer larger block sizes to speed transfers. These larger block sizes, specified with bs=, enable less read/writes, because the dd program reads and writes larger chuncks of data. Of course, too big of a block size will slow things down because flushing the dd buffers does take some time.
 
Old 01-08-2006, 08:07 AM   #58
Lotharster
Member
 
Registered: Nov 2005
Posts: 144

Rep: Reputation: 15
Quote:
Originally Posted by AwesomeMachine
With urandom it is possible to read a byte size of 100. With /dev/random you need to do bs=1. urandom does not have this restriction.
This is not correct. Both random and urandom can in theory produce output of any byte size. They use the kernel's entropy pool to compute the random numbers. This entropy pool is filled with random noise gathered from device drivers and other sources. The important difference between the two is that /dev/random produces only as many random bits as the entropy pool contains. This results in high quality randomness, which is neccessary for kryptographic keys. In case the user demands more random bytes than currently contained in the entropy pool, the process stops until the entropy pool is refilled (waggling your mouse helps). /dev/urandom, however, does not have this restriction. If the user demands more bits than currently in the entropy pool, it produces them using a pseudo random number generator. These pseudo random numbers should not be used as cryptographic keys.[/quote]
 
Old 01-08-2006, 08:02 PM   #59
jon_flanders
LQ Newbie
 
Registered: Mar 2004
Location: troy, ny
Distribution: Mandriva 2006
Posts: 21

Rep: Reputation: 15
Clone Windows Partition with DD?

I would like to clone my 13 gb windows 98 drive, divided into c: programs and d: data to a 100 gb drive via usb which I would pre-partition to say, c: 20gb , d: 30 gb.

The goal would be to then simply pop the 100 gb drive into the ide ribbon in place of the 13 gb drive. My Linux drive is currently 30gb which is sufficient for my purposes. Here's the output of df currently:

[jon@localhost ~]$ df
Filesystem Size Used Avail Use% Mounted on
/dev/hdb5 4.9G 3.6G 1.1G 78% /
/dev/hdb3 5.7G 3.3G 2.3G 60% /home/data
/dev/hda1 5.8G 5.1G 663M 89% /mnt/win_c
/dev/hda5 6.9G 4.2G 2.8G 60% /mnt/win_d
[jon@localhost ~]$

I have unmounted partitions and unused space on the 30gb drive.

Is this doable? I know I can partition the hard drive in the usb via Mandriva's Harddrake. I've done that already with an old 3gb drive.

Jon Flanders

Last edited by jon_flanders; 01-08-2006 at 08:05 PM.
 
Old 01-10-2006, 02:52 AM   #60
tonnnn
LQ Newbie
 
Registered: Jan 2006
Posts: 1

Rep: Reputation: 0
greate tutorial !!!
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM


All times are GMT -5. The time now is 09:48 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration