LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-29-2011, 05:46 AM   #691
oso_togari
Member
 
Registered: Jan 2008
Distribution: SOLARIS 9, RHEL4
Posts: 39

Rep: Reputation: 0

Hi all

very good post...is bookmarked since right now!

I have a question that maybe is answered in some of the 46 pages, but sincerely i dont read al the posts.

I fill the disk with zeros: dd if=/dev/zeros of=/file.zeros
i remove /file.zeros
i do a dd of my sda disk: sda1: /boot
sda2: LVM
lvm1: /
lvm2: swap

the compressed dd file is 3 gb but the dd itself its 136 gb

Now i want to use the image to create a kvm guest, but i dont want to create a 136 gb file; i.e. only a 15 gb file with the files and some extra space.

(i understand that i need some kind of defrag to move all data to the begining and then maybe i have to resolve some problems related to the geometry of the disk.)

Is this posible?

Regards

Last edited by oso_togari; 08-29-2011 at 06:06 AM. Reason: explain
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 09-08-2011, 02:42 PM   #692
ChocolateChicken
LQ Newbie
 
Registered: Oct 2010
Posts: 3

Rep: Reputation: 0
I've referred back to this thread at least 10 times. Very helpful, thanks.
 
Old 09-13-2011, 07:12 AM   #693
chadr
LQ Newbie
 
Registered: Sep 2011
Posts: 2

Rep: Reputation: Disabled
dd status check

I logged into my server via ssh and ran dd in the background. My ssh session terminated when my laptop went to sleep. I logged back in to the server to check status but don't know how to do it

[root@pogo tmp]# ps -ef | grep dd
root 2 0 0 Aug30 ? 00:00:00 [kthreadd]
root 31073 1 97 Sep12 ? 09:02:31 dd if=/dev/urandom of=/dev/sdb bs=1M
root 32246 32090 0 08:05 pts/2 00:00:00 grep dd
[root@pogo tmp]# killall -USR1 dd
[root@pogo tmp]# killall -USR1 31073
31073: no process found
[root@pogo tmp]#

Are there any options for me to see the status? Thanks.
 
Old 09-13-2011, 07:34 AM   #694
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fedora-35
Posts: 5,313

Rep: Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918
Quote:
Originally Posted by chadr View Post
I logged into my server via ssh and ran dd in the background. My ssh session terminated when my laptop went to sleep. I logged back in to the server to check status but don't know how to do it

[root@pogo tmp]# ps -ef | grep dd
root 2 0 0 Aug30 ? 00:00:00 [kthreadd]
root 31073 1 97 Sep12 ? 09:02:31 dd if=/dev/urandom of=/dev/sdb bs=1M
root 32246 32090 0 08:05 pts/2 00:00:00 grep dd
[root@pogo tmp]# killall -USR1 dd
[root@pogo tmp]# killall -USR1 31073
31073: no process found
[root@pogo tmp]#

Are there any options for me to see the status? Thanks.
Code:
kill -USR1 31073
 
Old 09-13-2011, 07:45 AM   #695
chadr
LQ Newbie
 
Registered: Sep 2011
Posts: 2

Rep: Reputation: Disabled
kill -USR1 31073

I don't get any output with this one either. Thanks
 
Old 09-17-2011, 12:20 AM   #696
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524

Original Poster
Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
Smile Compressed Partition Image Files

Quote:
Originally Posted by oso_togari View Post
Hi all

I fill the disk with zeros: dd if=/dev/zeros of=/file.zeros
i remove /file.zeros
i do a dd of my sda disk: sda1: /boot
sda2: LVM
lvm1: /
lvm2: swap

the compressed dd file is 3 gb but the dd itself its 136 gb

Now i want to use the image to create a kvm guest, but i dont want to create a 136 gb file; i.e. only a 15 gb file with the files and some extra space.

(i understand that i need some kind of defrag to move all data to the begining and then maybe i have to resolve some problems related to the geometry of the disk.)
Hi oso,

The dd command duplicates files as a bitstream. In UNIX and Linux, disks and file systems are available as virtual files in the /dev directory.

You didn't list the exact command lines you used, so I don't really know what you did. But if you want to get the files out of a compressed disk image file, you have to uncompress it, mount it.

Then, make an image file to hold just the files, but not the free space.
Code:
dd if=/dev/zero of=/file_system_image.img bs=1MB count=15000
Then, make a file system, say e2fs.
Code:
mkfs.ext3 /file_system_image.img
Mount the empty image file.
Code:
mkdir /mnt/image && mount -o loop /file_system_image.img /mnt/image
To transfer files, use
Code:
rsync -aHv <orig_image_mount_point/*> </mnt/image/>
Change root to the new image mount point. and install a boot loader and do everything else you need to customize.
Code:
chroot /mnt/image
Type exit to exit the chroot shell. I didn't actually try this one, so there might be a few gotchas you have to work around.

I use VMware, so I've not yet used KVM directly.
 
Old 09-17-2011, 12:47 AM   #697
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524

Original Poster
Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
Post killall USR1 is not the same as kill USR1

Quote:
Originally Posted by chadr View Post
I logged into my server via ssh and ran dd in the background. My ssh session terminated when my laptop went to sleep. I logged back in to the server to check status but don't know how to do it

[root@pogo tmp]# ps -ef | grep dd
root 2 0 0 Aug30 ? 00:00:00 [kthreadd]
root 31073 1 97 Sep12 ? 09:02:31 dd if=/dev/urandom of=/dev/sdb bs=1M
root 32246 32090 0 08:05 pts/2 00:00:00 grep dd
[root@pogo tmp]# killall -USR1 dd
[root@pogo tmp]# killall -USR1 31073
31073: no process found
[root@pogo tmp]#

Are there any options for me to see the status? Thanks.
I think using killall and specifying a pid might create a problem. I would just use the command line found in the OP. You have to open 2 ssh sessions. In the first one type:

Code:
<CTRL>z
fg
the dd command output should appear, which will be nothing until you type the command
Code:
ps aux | awk '/bin\/dd/ && !/awk/ {print $2}' | xargs kill -s USR1 $1
In the place of bin, just put something that differentiates between the dd processes, i.e. every ps aux | grep <command> will contain output of the ps command itself. So you have to tell awk something it can use to grab the correct line.

Then, if you switch back to the first of the 2 ssh sessions, it should have spit out a progress report on the dd process.
 
1 members found this post helpful.
Old 09-17-2011, 01:16 AM   #698
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524

Original Poster
Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
Post

Quote:
Originally Posted by deanksalinux View Post
Hi there

Is it possible to use dd to copy on used space while ignoring free space.

I can then extend the partitions if need be.

Thanks
Clonezilla will image one disk to another, copying only used space. But I'm a little confused as to the limitation that the destination partition must at least as large as the source partition. http://clonezilla.org/clonezilla-live.php
 
Old 09-21-2011, 02:49 AM   #699
psrdotcom
LQ Newbie
 
Registered: Sep 2011
Posts: 5

Rep: Reputation: Disabled
Nice Article

Very good thread
 
Old 09-21-2011, 11:23 AM   #700
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fedora-35
Posts: 5,313

Rep: Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918
@ awesome-machine, using dd is there a way to copy a 2 gb drive (boot sector + 1 partition) so that when it is restored to a 100 gb drive, it can be expanded to read/write the whole drive ?
 
Old 09-23-2011, 01:00 AM   #701
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE Tumbleweed-KDE, Mint 21, MX-21, Manjaro
Posts: 4,629

Rep: Reputation: Disabled
Did you already try gparted?
 
Old 09-23-2011, 06:48 AM   #702
archtoad6
Senior Member
 
Registered: Oct 2004
Location: Houston, TX (usa)
Distribution: MEPIS, Debian, Knoppix,
Posts: 4,727
Blog Entries: 15

Rep: Reputation: 234Reputation: 234Reputation: 234
schneidz,

You haven't given many details of the background behind your question, so this may or may not be an appropriate suggestion. You use the word "restore", & to me this implies that your source (2 GB) will be copied to an intermediate medium, some time may pass, & then the intermediate medium will be copied to your final target (100 GB). Would Clonezilla work for you? (Please note, dd is one of several tools that Clonezilla uses.)

Meanwhile, especially if my inference above is wrong, expect the details of a "pure" dd method to be forthcoming.
 
Old 09-23-2011, 07:49 AM   #703
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fedora-35
Posts: 5,313

Rep: Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918
Quote:
Originally Posted by JZL240I-U View Post
Did you already try gparted?
yeah good advice, but it came up unbootable... maybe i'll try again if i need to reinstall (maybe the boot flag wasnt ticked).

Last edited by schneidz; 09-23-2011 at 07:59 AM.
 
Old 09-23-2011, 07:59 AM   #704
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fedora-35
Posts: 5,313

Rep: Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918
Quote:
Originally Posted by archtoad6 View Post
schneidz,

You haven't given many details of the background behind your question, so this may or may not be an appropriate suggestion. You use the word "restore", & to me this implies that your source (2 GB) will be copied to an intermediate medium, some time may pass, & then the intermediate medium will be copied to your final target (100 GB). Would Clonezilla work for you? (Please note, dd is one of several tools that Clonezilla uses.)

Meanwhile, especially if my inference above is wrong, expect the details of a "pure" dd method to be forthcoming.
hi, this was for my fc-11 backup but since i got fc-15 now it is kinda' moot.
my premise is to install the os on a 2 gb drive and install all the packages i want (and configure things like compiz-fusion how i like).
so then if i want to install it on another pc (since it is easier to carry around a 2 lbs weight rather than 100 pounds) i can reimage the disk and expand the partitions.

i havent tried clonezilla... i'll look into it if there is a next time.



edit: i'll also mention that i like imaging the whole disk (i.e. dd if=/dev/sda) rather than a partition (/dev/sda1) because its more comprehensive. (i dont like needing to chroot mountpoint; grub-install ...)
but lets say i want to loop mount one of the partitions of the disk image and save changes ?
i think buried in this thread are instructions how to do this but i dont remember where.

Last edited by schneidz; 09-23-2011 at 08:20 AM.
 
Old 09-26-2011, 01:30 AM   #705
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE Tumbleweed-KDE, Mint 21, MX-21, Manjaro
Posts: 4,629

Rep: Reputation: Disabled
Quote:
Originally Posted by schneidz View Post
yeah good advice, but it came up unbootable...
There is a gparted bootable CD available: http://gparted.sourceforge.net/livecd.php HTH.
 
  


Reply

Tags
backup, best, clonezilla, cloning, command, data, dd, disk, drive, duplicate, erase, explanation, formatting, ghost, hard, image, iso, memory, ping, popular, recover, recovery, rescue, search, security, stick, upgrade, usb, wipe


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Learn The DD command AwesomeMachine Linux - Newbie 17 08-17-2006 04:22 AM
The best way to learn? iz3r Programming 7 02-06-2005 11:00 PM
Best way to learn Linux from the command line patpawlowski Linux - General 2 03-01-2004 03:37 PM
I want to learn C. KptnKrill Programming 14 12-18-2003 01:03 PM
Best way to learn.... InEeDhElPlInUx Linux - Newbie 5 10-11-2003 01:02 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration