LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 02-16-2019, 09:21 AM   #1
battles
Member
 
Registered: Apr 2014
Distribution: Debian GNU/Linux 7.5 (wheezy)
Posts: 258

Rep: Reputation: Disabled
Trouble calculating an image


I have made a Debian image that I need to upload to my server. The image was created on a local computer that is around 244GB.

Device Boot Start End Blocks Id System
/dev/sda 0 244198584
/dev/sda1 * 2048 175783297 87890625 83 Linux
/dev/sda2 175783934 185548799 4882433 5 Extended
/dev/sda5 175783936 185548799 4882432 82 Linux swap / Solaris

The image I need to upload to the VPS can only be 100GB or less. What I am trying to do is use dd to make a smaller copy of the disk. I am going to try to limit the dd to copying less than 100GB using this:

dd if=/dev/sda of=/debian711 bs=512K count=189782

512 * 1024 = 524,288
99,500,000,000 / 524,288 = 189,782 = < 100GB

Does this look correct? Thanks

Last edited by battles; 02-16-2019 at 12:18 PM.
 
Old 02-16-2019, 01:18 PM   #2
Brains
Senior Member
 
Registered: Apr 2009
Distribution: All OS except Apple
Posts: 1,591

Rep: Reputation: 389Reputation: 389Reputation: 389Reputation: 389
Not sure if the image will be of any use with a 512K byte size, if the file system has 4K blocks, that should be the maximum byte size.

You'll need to shrink the partitions so they add up to 100GB or less if you are to use a simple dd command. DD does not sift out used space leaving out unused space, whatever formula you drum up to get the size down to 100GB with DD will give you a useless image. As partition sizes won't match what the partition table reports, and you'll be missing a lot of stuff etc.

Other than that, if you want the restored image to replicate exact size of the original, you need to use a back up utility that only copies used space like Clonezilla or Image for Dos, these utilities will also compress the image making it 10GB or less. And the restored image will have same partition layout, same size.

NOTE: When there are only three partitions, there is no need for an extended and logical partition, just 2 primary partitions.
 
Old 02-16-2019, 01:32 PM   #3
battles
Member
 
Registered: Apr 2014
Distribution: Debian GNU/Linux 7.5 (wheezy)
Posts: 258

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Brains View Post
Not sure if the image will be of any use with a 512K byte size, if the file system has 4K blocks, that should be the maximum byte size.

You'll need to shrink the partitions so they add up to 100GB or less if you are to use a simple dd command. DD does not sift out used space leaving out unused space, whatever formula you drum up to get the size down to 100GB with DD will give you a useless image. As partition sizes won't match what the partition table reports, and you'll be missing a lot of stuff etc.

Other than that, if you want the restored image to replicate exact size of the original, you need to use a back up utility that only copies used space like Clonezilla or Image for Dos, these utilities will also compress the image making it 10GB or less. And the restored image will have same partition layout, same size.

NOTE: When there are only three partitions, there is no need for an extended and logical partition, just 2 primary partitions.
The extended was crated automatically. I did add the swap. So I need to change the 512K to 4K. I am using an Atom controller to create this. I want to shrink the disk to 100GB and then do a simple dd if/of, but I can't figure out how to shrink it with the various Linux tools. The debian is netinit only, so I am not sure that I could use Clonezilla. Any further information on how to shrink the 244GB disk down to 100GB would be appreciated. I am assuming that dd would handle the image creation correctly then. Thanks.

P.S. Just found partimage. Any knowledge on it?

Last edited by battles; 02-16-2019 at 01:44 PM.
 
Old 02-16-2019, 01:55 PM   #4
Brains
Senior Member
 
Registered: Apr 2009
Distribution: All OS except Apple
Posts: 1,591

Rep: Reputation: 389Reputation: 389Reputation: 389Reputation: 389
Quote:
Originally Posted by battles View Post
The extended was crated automatically. I did add the swap. So I need to change the 512K to 4K. I am using an Atom controller to create this. I want to shrink the disk to 100GB and then do a simple dd if/of, but I can't figure out how to shrink it with the various Linux tools. The debian is netinit only, so I am not sure that I could use Clonezilla. Any further information on how to shrink the 244GB disk down to 100GB would be appreciated. I am assuming that dd would handle the image creation correctly then. Thanks.
Get Gparted live, burn it to optical disk or USB key and boot it up, hit enter to select defaults at the questions. In the top rectangular window showing the partitions, highlight the swap and extended partitions and delete them from menus, from the edit menu, click "apply changes", you may need to do one at a time. Then highlight the root partition, select "resize" in menus, grab the right side of the root partition and slide it to the left to bring it down to desired size, something like 80GB, then highlight the free space and select "create" to create a 15-20GB primary swap partition in the free space, ensure to select format swap, then click "apply changes" and wait till complete before exiting Gparted.

Then you will need to boot into Debian, it should hang for a couple minutes because it can't mount swap but will eventually get to the desktop where you need to edit /etc/fstab file to change the UUID of the swap partition. Run command: sudo blkid /dev/sda2 to find the new UUID and replace the existing UUID for the swap partition in /etc/fstab file as sudo or root and save it. It should then boot normally.
Then you will need to add proper "count=" value in the DD command to only copy the two partitions and not the leftover free space.

Last edited by Brains; 02-16-2019 at 02:10 PM.
 
Old 02-16-2019, 02:01 PM   #5
battles
Member
 
Registered: Apr 2014
Distribution: Debian GNU/Linux 7.5 (wheezy)
Posts: 258

Original Poster
Rep: Reputation: Disabled
Thank you very much. First time to do this outside of Windows.
 
Old 02-16-2019, 02:28 PM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,133

Rep: Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121
dd uses the bs as a logical blocksize - it is irrelevant what the natural blocksize of the device (or filesystem for that matter) is. I typically use 4M for no better reason than I always have ...
Makes the math easier too.
 
Old 02-16-2019, 02:32 PM   #7
Brains
Senior Member
 
Registered: Apr 2009
Distribution: All OS except Apple
Posts: 1,591

Rep: Reputation: 389Reputation: 389Reputation: 389Reputation: 389
Quote:
Originally Posted by syg00 View Post
dd uses the bs as a logical blocksize - it is irrelevant what the natural blocksize of the device (or filesystem for that matter) is. I typically use 4M for no better reason than I always have ...
Makes the math easier too.
Good to know, I'm accustomed to no larger than block size by habit from a forensics background.
 
Old 02-16-2019, 02:32 PM   #8
battles
Member
 
Registered: Apr 2014
Distribution: Debian GNU/Linux 7.5 (wheezy)
Posts: 258

Original Poster
Rep: Reputation: Disabled
So bs=4M rather than bs=4K.
 
Old 02-16-2019, 02:46 PM   #9
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,133

Rep: Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121
Your choice, but it simply reduces the amount of I/O's issued by dd. Seems faster, but may be just imagination.
You must include all of the source disk that has any part of a partition in the image. So if you make it total 95G, maybe copy 99G to ensure you get it all - doesn't matter if you copy more, it is merely a "safety factor". The 99G is arbitrary to make sure it is under your 100G limit.
 
Old 02-16-2019, 03:03 PM   #10
battles
Member
 
Registered: Apr 2014
Distribution: Debian GNU/Linux 7.5 (wheezy)
Posts: 258

Original Poster
Rep: Reputation: Disabled
Unfortunately, gparted is not working past the initial start window on the controller. The loading of the gparted system reverts to a partitioning program at the first, so I'll reload and use it. Actually, I think I am ready to go now with the way it is.

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 175783297 87890625 83 Linux
/dev/sda2 175783934 185548799 4882433 5 Extended
/dev/sda5 175783936 185548799 4882432 82 Linux swap / Solaris

dd if=/dev/sda conv=sync,noerror bs=4M count=23603 | gzip -c > debian711.gz
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Slackware 14.1 32bit system tarball/stage3 image/chroot image/bootstrap image vinipsmaker Slackware 20 05-14-2014 01:41 AM
calculating download time/speed juanb Linux - Networking 4 03-19-2004 02:49 PM
need perl help calculating fibonacci numbers WorldBuilder Programming 5 12-17-2003 01:41 AM
Calculating Surface Distance! john23 Programming 1 11-27-2003 01:45 PM
calculating server load hardigunawan Programming 5 06-27-2003 11:07 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 07:47 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration