Create partition compressed image with dd & gzip and see the progress with pv.
 

What would be the correct way to create a partition compressed image with dd and gpzip and see the progress with pv?

In the hard disk I have several partitions and I only want to clone the mbr and the first two partitions. With fdisk -l i get the End block of the second partition. So XXX is the obtained end block + 1

I have tried:

but then dd does not stop to the second partition but seems to continue and backup the complete hd. What I am doing wrong?

dd will do as it is told - implies that your arithmetic is wrong.

fdisk will report the end block using a block size of 512 bytes.

dd will use a block size of whatever the heck you tell it. In this case, 4096 bytes. The number you use here is unimportant unless you are using a tape backup device or something like that. Basically, it just tells dd how many bytes to process at a time. Using a higher number helps performance, up to a point of diminishing returns.

But here's where you likely stumbled-you put in the block count based on what fdisk told you, but the disk block size isn't what dd uses or thinks of when you specify "count". dd doesn't care about the hard drive's block size. It only cares about the bs value you give it.

So, to get the desired count=XXX, you need to do some arithmetic. I like to use python for this arithmetic, but you can use a pencil and paper to calculate it if you like. Basically, you want to multiply by 512 to get the desired byte count, and then you divide by bs.

I see now. By dividing the number of blocks obtained with fdisk by 8 (bs: 4096/ fdisk logical sector size: 512) it works as expected.
Thanks for your help.

FYI you do not need pv any more, dd can show progress with status=progress option.

Nice! I have used dd for years, and I never even looked for that option. I have just adopted an attitude of, "It takes as long as it takes." But no more, thanks!

[edit added:]

Oh well, the version of dd in Debian Stable does not have the progress option. No big deal. It takes as long as it takes.

Sorry, forgot to mention it is a new feature.

Thoughts.

Could look at changing default compression level. The default is middle, your use may vary.

There are also file by file ways that may do just as well or better.

Have you tried
1. just the MBR+partition table (first 512 bytes of the disk):
2. images of 1st and 2nd partitions:

Thank you for your comments.

dd is great to make snapshots of a whole partition, so that in case of disk crash, by just restoring on another hd you get quickly a working system.
To backup the data I agree that solutions that work by files such as rsync are more adequate.

I did some tests with bzip2 or changing the compressing level, but the compression time increased a lot. The scope was in my case to save some space but not to obtain the smallest possible file, so that the gz default was good for me.

Yes, you could use a number of compression tools that would suit your data better. I meant more of an option number behind the compression. If you want speed I think it is 1 and full space is 9.

Just thoughts, you may know all this already, can't tell.