Create partition compressed image with dd & gzip and see the progress with pv.
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Create partition compressed image with dd & gzip and see the progress with pv.
What would be the correct way to create a partition compressed image with dd and gpzip and see the progress with pv?
In the hard disk I have several partitions and I only want to clone the mbr and the first two partitions. With fdisk -l i get the End block of the second partition. So XXX is the obtained end block + 1
fdisk will report the end block using a block size of 512 bytes.
dd will use a block size of whatever the heck you tell it. In this case, 4096 bytes. The number you use here is unimportant unless you are using a tape backup device or something like that. Basically, it just tells dd how many bytes to process at a time. Using a higher number helps performance, up to a point of diminishing returns.
But here's where you likely stumbled-you put in the block count based on what fdisk told you, but the disk block size isn't what dd uses or thinks of when you specify "count". dd doesn't care about the hard drive's block size. It only cares about the bs value you give it.
So, to get the desired count=XXX, you need to do some arithmetic. I like to use python for this arithmetic, but you can use a pencil and paper to calculate it if you like. Basically, you want to multiply by 512 to get the desired byte count, and then you divide by bs.
I see now. By dividing the number of blocks obtained with fdisk by 8 (bs: 4096/ fdisk logical sector size: 512) it works as expected.
Thanks for your help.
Nice! I have used dd for years, and I never even looked for that option. I have just adopted an attitude of, "It takes as long as it takes." But no more, thanks!
[edit added:]
Oh well, the version of dd in Debian Stable does not have the progress option. No big deal. It takes as long as it takes.
What would be the correct way to create a partition compressed image with dd and gpzip and see the progress with pv?
In the hard disk I have several partitions and I only want to clone the mbr and the first two partitions. With fdisk -l i get the End block of the second partition. So XXX is the obtained end block + 1
Have you tried
1. just the MBR+partition table (first 512 bytes of the disk):
There are also file by file ways that may do just as well or better.
dd is great to make snapshots of a whole partition, so that in case of disk crash, by just restoring on another hd you get quickly a working system.
To backup the data I agree that solutions that work by files such as rsync are more adequate.
Quote:
Could look at changing default compression level.
I did some tests with bzip2 or changing the compressing level, but the compression time increased a lot. The scope was in my case to save some space but not to obtain the smallest possible file, so that the gz default was good for me.
Yes, you could use a number of compression tools that would suit your data better. I meant more of an option number behind the compression. If you want speed I think it is 1 and full space is 9.
Just thoughts, you may know all this already, can't tell.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.