[SOLVED] Backup, shrink backup and modify MBR of backup

jps1x2 · 12-16-2013, 05:42 AM

Hi all,

after introducing me, this is my first post.

I am experimenting with RaspberryPi, installed Raspbian on a 16Gb SD and expanded to use all SD Card. I use this device, but i think this question is related to General Linux, not for these type of device.

Now that all my Rasp is configured (Deluge, noip, vnc server, Wifi SSID,...) i would like to backup the SD card and i would like to be able to restore it to small card (4 Gb, 8 Gb, ...) this backup and i would like to use only Raspberry, not other computer in linux or windows, and then fully or partially automate this process.

Have read very much posts in Raspberry forums and in other places (dd, losetup, e2fsck, resize2fs, extract MBR and modify it, etc.) but i don't understand image structure of SD generated with dd tool or from original SD Card.

Tested this on Raspberry:

a) Dump of all 16Gb SD card to external usb drive:

Code:

$ sudo dd bs=1M if=/dev/mmcblk0 of=/media/TOSHIBA/backupRaspRaw.img

b) Mounted image as a loop device for testing:

Code:

$ sudo losetup -f --show /media/TOSHIBA/backupRaspRaw.img
loop0

c) fdisk info of original SD Card:

Code:

$ sudo fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 15.6 GB, 15611199488 bytes
4 heads, 16 sectors/track, 476416 cylinders, total 30490624 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f06a6

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/mmcblk0p2          122880    30490623    15183872   83  Linux

d) fdisk info of mounted backup image (is not equal to original SD Card):

Code:

$ sudo fdisk -l /dev/loop0

Disk /dev/loop0: 15.6 GB, 15611199488 bytes
255 heads, 63 sectors/track, 1897 cylinders, total 30490624 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f06a6

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop0p2          122880    30490623    15183872   83  Linux

e) Extracted MBR of original SD card:

Code:

sudo dd if=/dev/mmcblk0 of=/media/TOSHIBA/originalSD.mbr bs=512 count=1
hexedit /media/TOSHIBA/originalSD.mbr
...
000001C0                                                00 00  ....... ........
000001D0   C1 80 83 03  10 FF 00 E0  01 00 00 60  CF 01        ...........`....
...

f) Extracted MBR of loop device and studied structure of its second partition to modify it (it's equal to SD Card):

Code:

sudo dd if=/dev/loop0 of=/media/TOSHIBA/backupRaspRaw.mbr bs=512 count=1
hexedit /media/TOSHIBA/backupRaspRaw.mbr
...
000001C0                                                00 00  ....... ........
000001D0   C1 80 83 03  10 FF 00 E0  01 00 00 60  CF 01        ...........`....
...

g) Using following links:

http://www.raspberrypi.org/phpBB3/vi...p?f=91&t=19468
http://sirlagz.net/2012/06/20/how-to...an-image-file/
http://en.wikipedia.org/wiki/Master_boot_record#PTE

I understood how to resize second partition of raspbian in loop device (e2fsck, resize2fs) and how to calculate MBR structure and ready to modify it and overwrite in .img file with dd, but i don't understand why information of extracted MBRs is not correct, see following analisys of first 4 bytes of MBR structure of second partition:

Code:

Using formula from indicated links:

LBA = (((C x HPC) + H) x SPT) + S - 1

Code:

000001C0                                                00 00  ....... ........
000001D0   C1 80 83 03  10 FF 00 E0  01 00 00 60  CF 01        ...........`....

00          = Partition status
00 C1 80    = HSC (head, sector, cylinder) address of the first absolute sector in the
              partition (00h C1h 80h)

    00h   = (00 dec) is the head
    C1h   = (11000001b) indicates sector in bits 5–0; bits 7–6 are high bits of cylinder, C1h
            is 11000001b and in bits 5-0 we have 000001b which is 1 decimal
    80h   = indicates low bits of cylinder, 80h is 10000000b, adding two high bits (11)
            from previous address we have 1110000000b which is 896 dec

so the first sector is at cylinder 896, sector 1 head 0

and its fdisk data (heads and sectors/track):

Disk /dev/loop0: 3355 MB, 3355443200 bytes
255 heads, 63 sectors/track, 407 cylinders, total 6553600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f06a6

Device Boot            Start         End      Blocks   Id  System
/dev/loop0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop0p2          122880    30490623    15183872   83  Linux   
      
LBA = (((896 x 255) + 0) x 63) + 1 - 1 = 14394240

Formula result is 14394240 and not 122880 like indicated in fdisk info of second partition for starting position of second partition.

Tested the same process mounting as a loopback device image of raspbian installation (2013-09-25-wheezy-raspbian.img), studied its MBR and it is correct:

Code:

sudo fdisk -l /dev/loop0

Disk /dev/loop0: 2962 MB, 2962227200 bytes
255 heads, 63 sectors/track, 360 cylinders, total 5785600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002c262

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop0p2          122880     5785599     2831360   83  Linux

000001C0                                                00 A5  ....... ........
000001D0   1F 07 83 22  7A 68 00 E0  01 00 00 68  56 00        ..."zh.....hV...

00          = Partition status
A5 1F 07    = HSC (head, sector, cylinder) address of the first absolute sector in the
              partition (A5h 1Fh 07h)
    A5h (165 dec) is the head
    1Fh (00011111b) indicates sector in bits 5–0; bits 7–6 are high bits of cylinder, 1Fh
        is 00011111b and in bits 5-0 we have 11111b which is 31 decimal
    07h indicates low bits of cylinder, 07h is 00000111b, adding two high bits (00)
        from previous address we have 0000000111b which is 7 dec

so the last sector is at cylinder 7, sector 31 head 165

Using formula from indicated links:

LBA = (((7 x 255) + 165) x 63) + 31 - 1 = 122880

The result is correct.

My Raspbian is working perfect and its disk structure is not modified by me, only expanded using raspi-config to use all SD Card.

What do you think that is wrong in MBR of used SD Card or backup .img file? Should I have something else in mind when i do the backup with dd?

Thanks in advance and sorry for this long post!

jps1x2 · 12-17-2013, 06:03 AM

Hello,

after more testing in a Linux VM creating and managing .img disks found steps to solve problem:

a) Complete backup of 16 Gb SD Card:

Code:

$ sudo dd bs=4M if=/dev/mmcblk0 of=/media/TOSHIBA/16Gb.img

b) Mount image as a loopback device and shrink second partition to 3 Gbytes.:

Code:

$ sudo losetup -f --show /media/TOSHIBA/16Gb.img

Mounted as loopback device /dev/loop0

Code:

$ sudo fdisk -l /dev/loop0

Disk /dev/loop0: 2962 MB, 2962227200 bytes
255 heads, 63 sectors/track, 360 cylinders, total 5785600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002c262

     Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop0p2          122880     5785599     2831360   83  Linux

Second partition starts in sector 122880

Byte position 122880 * 512 = 62914560

Code:

$ sudo losetup -f --show -o 62914560 /media/TOSHIBA/16Gb.img

Mounted as loopback device /dev/loop1

Check Disk :

Code:

$ sudo e2fsck -p -y -f /dev/loop1

Shrink :

Code:

$ sudo resize2fs -p /dev/loop1 3G

Delete loopback devices:

Code:

$ sudo losetup -d /dev/loop0 /dev/loop1

d) We will create one image disk with similar properties (sectors/track and heads) like original Raspbian image, but greater in size. Information of raspbian image disk (fdisk can obtain/manipulate directly info from image file):

Code:

$ sudo fdisk -l /media/TOSHIBA/16Gb.img

Disk /media/TOSHIBA/16Gb.img: 2962 MB, 2962227200 bytes
255 heads, 63 sectors/track, 360 cylinders, total 5785600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002c262

     Device Boot           Start         End      Blocks   Id  System
/media/TOSHIBA/16Gb.img1    8192      122879       57344    c  W95 FAT32 (LBA)
/media/TOSHIBA/16Gb.img2  122880     5785599     2831360   83  Linux

We will use cylinder number, that contain 255 x 63 = 16.065 sectors each one or 16.065 x 512bytes = 8.225.280 bytes per cylinder.

We will create one image disk with 4 MBytes (at beginning) + 56 MBytes (Fat32 partition) + 3 GBytes (3.072 Mb system partition) that it 3.132 MBytes or 3.284.140.032 bytes, we divide it by bytes per cylinder and obtain number of cylinders to create on disk:

3.284.140.032 / 8.225.280 = 399,27 cylinders

We add 1 and we will use 400 that will be 3.137 MBytes approximately

Code:

$ dd if=/dev/zero of=/media/TOSHIBA/3Gb.img bs=8225280c count=400

e) Using fdisk we create directly to the image the disk structure with correct Cylinders, Sectors and Tracks and create directly partitions in this fdisk session:

Code:

$ fdisk -u -C400 -S63 -H255 /media/TOSHIBA/3Gb.img

We create the first partition in file in same fdisk session and change partition type to W95 FAT32 (LBA):

Code:

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First sector (2048-204799, default 2048): 8192
Last sector, +sectors or +size{K,M,G} (8192-204799, default 204799): +56M

Command (m for help): p
...
      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1            8192      122879       57344   83  Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): c
Changed system type of partition 1 to c (W95 FAT32 (LBA))

We create the second partition in file in same fdisk session and change partition type to Linux:

Code:

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First sector (2048-204799, default 2048): 122880
Last sector, +sectors or +size{K,M,G} (122880-204799, default 204799): +3072M

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 83

We see created partitions in same fdisk session:

Code:

Command (m for help): p

Disk /media/TOSHIBA/3Gb.img: 3290 MB, 3290112000 bytes
255 heads, 63 sectors/track, 400 cylinders, total 6426000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbfe1ca60

Device Boot              Start   End      Blocks   Id  System
/media/TOSHIBA/3Gb.img1  8192    122879   57344    c   W95 FAT32 (LBA)
/media/TOSHIBA/3Gb.img2  122880  6414335  3145728  83  Linux

Save disk information and exit this fdisk session:

Code:

Command (m for help): w
The partition table has been altered!


WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

h) We will use dd to "copy" first partition from 16Gb image file to 3Gb image file skipping from input file 4 MBytes (8192 sectors of 512 bytes) and seeking 4 MBytes (8192 sectors of 512 bytes) on output file to copy 56 MBytes (114688 sectors of 512 bytes) of first partition:

Code:

$ dd if=/media/TOSHIBA/16Gb.img of=/media/TOSHIBA/3Gb.img bs=512 count=114688 skip=8192 seek=8192

Mon Dec 16 16:27:09 UTC 2013
114688+0 records in
114688+0 records out
58720256 bytes (59 MB) copied, 13.9752 s, 4.2 MB/s

i) We will use dd to "copy" second partition from 16Gb image file to 3Gb image file skipping from input file 4 MBytes + 56 MBytes (122880 sectors of 512 bytes, 60 MBytes) and seeking 4 MBytes + 56 MBytes (122880 sectors of 512 bytes, 60 MBytes) on output file to copy 3072 MBytes (6291456 sectors of 512 bytes) of second partition:

Code:

$ dd if=/media/TOSHIBA/16Gb.img of=/media/TOSHIBA/3Gb.img bs=512 count=6291456 skip=122880 seek=122880

Mon Dec 16 16:28:20 UTC 2013
6291456+0 records in
6291456+0 records out
3221225472 bytes (3.2 GB) copied, 762.765 s, 4.2 MB/s
Mon Dec 16 16:41:02 UTC 2013

j) We compared with hexedit 16Gb and 3Gb file going to start of each partition and end of each partition and all data is equal, backup seems correct.

k) Copied 3Gb file with ftp/scp to windows and used win32diskimager to restore image to 4 Gb SD Card, Raspbian boots and works perfect!