[SOLVED] fdisk/cfdisk size discrepancy

catkin · 12-10-2012, 03:09 AM

sda3 and sdb7 have the same number of cfdisk sectors but different numbers of fdisk blocks, by ~0.0002%. Why would that be? It matters because they are intended to be exacly the same size to allow using dd to back up one partition to the other in whatever direction is convenient.

Code:

root@CW8:~# cfdisk -Ps /dev/sda         
Partition Table for /dev/sda

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
[snip]
 3 Primary    12498570    38491739      0    25993170 Linux (83)           None
[snip]
root@CW8:~# cfdisk -Ps /dev/sdb         
Partition Table for /dev/sdb

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
[snip]
 7 Logical  1224268445* 1250261614*    63    25993170 Linux (83)           None
root@CW8:~# fdisk -l /dev/sda /dev/sdb

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7b381e77

   Device Boot      Start         End      Blocks   Id  System
[snip]
/dev/sda3        12498570    38491739    12996585   83  Linux
[snip]

Disk /dev/sdb: 640.1 GB, 640133946880 bytes
255 heads, 63 sectors/track, 77825 cylinders, total 1250261615 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000bec02

   Device Boot      Start         End      Blocks   Id  System
[snip]
/dev/sdb7      1224268508  1250261614    12996553+  83  Linux

Versions:

OS: Slackware64 14.0
cfdisk and fdisk: from util-linux 2.21.2
sda: Seagate Barracuda 7200.12, model ST31000524AS, firmware: JC4B (1 TB)
sdb: Western Digital, Caviar Green, model WD6400AACS-00G8B1(640 GB)

rknichols · 12-10-2012, 09:06 AM

In the cfdisk output for /dev/sdb7, note the starting offset of 63. Every logical partition starts with a secondary partition table, so you lose at least that sector, and lose that entire track if partitioning was done in (deprecated) "DOS compatibility" mode. The fdisk "Blocks" column reflects that lost space. The cfdisk "Length" column does not.

So, your partitions are not the same size, even though they occupy the same amount of space. There are 63 fewer usable sectors in /dev/sdb7.

catkin · 12-11-2012, 05:46 AM

Thanks rknichols

Makes perfect sense, now you have explained it. If I have it right ... the extended partition and each logical partition has an Extended Boot Record and it is this which requires one sector (or one track).

rknichols · 12-11-2012, 09:20 AM

Close, but there is no separate record for the extended partition itself.

catkin · 12-13-2012, 01:40 AM

Got it, thanks

Useful to learn about EBRs for "partition" backup, restore and copy.

If I have rightly understood the implications ...

The only time an EBR would be wanted along with a partition copy or restore would be if the target partition's position and size were the same as the source's and any subsequent partition were identical.

Otherwise there is no benefit in backing up or copying the source's EBR. So the aim is not to backup or copy a partition but "the file system space within a partition". The difference is only important for logical partitions; for primary partitions and LVM volumes they are the same thing.

If the dd command is used, the input partition's EBR is ignored by using skip=<input offset>; the output partition's EBR is left untouched by using seek=<output offset>.

rknichols · 12-13-2012, 10:08 AM

Quote:

Originally Posted by catkin

If the dd command is used, the input partition's EBR is ignored by using skip=<input offset>; the output partition's EBR is left untouched by using seek=<output offset>.

No, and no. The "Start of Data" offset is already taken into account when you reference, for example, "/dev/sda5". The only way to access the sectors occupied by the MBR and EBRs is to access the whole device (/dev/sda) and calculate the absolute LBAs. If you just do "dd if=/dev/sda5 of=whatever" you will not pick up either the EBR for sda5 or any reserved "DOS Compatibility mode" sectors that might follow it.

You are correct, though, that the only time you need that partition table information is when you want to restore an entire drive to match the layout of the original. Otherwise, you can generally restore a non-boot partition's content to any other partition of the same or larger size. (Boot loaders are a special case, and the early stages, at least, depend on data located at exact LBAs, which might include the reserved sectors following the MBR or EBR. When such partitions are moved, the boot loader needs to be reinstalled.)

catkin · 12-16-2012, 09:30 AM

Thanks rknichols

I did some tests today and the results, as I understand them, suggest you are wrong.

First the partition details:

Code:

root@CW8:~# cfdisk -Ps /dev/sda
Partition Table for /dev/sda

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
[snip]
14 Logical  1432516050  1458509219     63    25993170 Linux (83)           None
[snip]
root@CW8:~# fdisk -l /dev/sda
[snip]
   Device Boot      Start         End      Blocks   Id  System
[snip]
/dev/sda14     1432516113  1458509219    12996553+  83  Linux

The Last Sector/End values are identical. The First Sector/Start values differ by the 63 sector offset which accommodates the EBR plus DOS compatibility sectors in accordance with your reply on 10 December.

The Length of 25993170 is Last Sector - First Sector + 1 (1458509219-1432516050+1=25993170). AIUI that means 25993170 includes the EBR plus DOS compatibility sectors.

Then I ran dd and it reported copying 25993107 records.

Code:

root@CW8:~# dd if=/dev/sda14 of=/dev/null
25993107+0 records in
25993107+0 records out

As a further test, I ran dd using the skip and seek options I suggested above and then booted from the partition named in the of=. It booted OK.

???

rknichols · 12-16-2012, 01:50 PM

Quote:

Originally Posted by catkin

Code:

root@CW8:~# cfdisk -Ps /dev/sda
Partition Table for /dev/sda

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
[snip]
14 Logical  1432516050  1458509219     63    25993170 Linux (83)           None
[snip]
root@CW8:~# fdisk -l /dev/sda
[snip]
   Device Boot      Start         End      Blocks   Id  System
[snip]
/dev/sda14     1432516113  1458509219    12996553+  83  Linux

Then I ran dd and it reported copying 25993107 records.

Code:

root@CW8:~# dd if=/dev/sda14 of=/dev/null
25993107+0 records in
25993107+0 records out

Notice that those two numbers are not identical. They differ by 63.

Quote:

Originally Posted by catkin

As a further test, I ran dd using the skip and seek options I suggested above and then booted from the partition named in the of=. It booted OK.

???

I suggest that the partition identified by "of=" was not cleared before you did that, and that the sectors left untouched by your "seek=" still had their content from your previous test. Try this:

Code:

dd if=/dev/zero of=/dev/{your_target_partition} seek=63 count=100
file -s /dev/{your_target_partition}

Notice that the file command still sees the beginning of a file system in that partition.

Now try this on your source disk:

Code:

dd if=/dev/sda skip=1432516050 count=1 | hexdump
dd if=/dev/sda14 count=1 | hexdump

Notice that those are not displaying the same data. /dev/sda14 does not begin at what cfdisk claims is the starting sector, but in fact begins 63 sectors later:

Code:

dd if=/dev/sda14 count=4 | hexdump
dd if=/dev/sda skip=$((1432516050+63)) count=4 | hexdump

Note that I used "count=4" in those because an ext{2/3/4} filesystem does not use the first two sectors of the partition, and 1024 bytes of all-zeros isn't very unique.

catkin · 12-17-2012, 02:30 AM

Quote:

Originally Posted by rknichols

Notice that those two numbers are not identical.
...
Note that I used "count=4" in those because an ext{2/3/4} filesystem does not use the first two sectors of the partition, and 1024 bytes of all-zeros isn't very unique.

Many thanks rknichols

Appreciation for your patience; attention to what I wrote; and clear, detailed refutations and exercises.

Sorry about the ...107/...170 numeric dyslexia on my part.

EDIT: for JFS2, it was necessary to change count=4 to count=65 to display the first not-all-zeroes sector.

catkin · 12-17-2012, 04:11 AM

EDIT: solved this question; there are notionally 63 sectors per track, not the 64 I had wrongly assumed. Presumably -- since 63 would be hard to manufacture -- there were 64 but the 64th was used for error detection and correction for internal use only and hidden from the OS.

I've been summarising what I have learned in this thread but in doing so found a discrepancy of one sector while deriving the exact logical partition size from fdisk and cfdisk displays. Clearly my understanding is not yet 100%. The discrepancy is at the end, noted in red.

Quote:

Each logical partition is preceded by an Extended Boot Record (EBR) a.k.a. Extended Partition Boot Record (EPBR) of one sector followed by a number of unused sectors (typically 63 because Windows XP and older required 63, notionally to fill a complete track of 64 sectors).

When cfdisk is used to create a logical partition, the size entered (which may be specified in sectors, to be precise) includes the EBR and any following unused sectors. cfdisk displays the number of unused sectors as the Offset.

When fdisk is used to create a logical partition, the size entered (which may be specified in sectors, to be precise) does not include the EBR and any following unused sectors.

The device files /dev/sd[a-z][1-9][0-9]* reference the partitions, excluding the EBR and any following unused sectors. If the EBR and any following unused sectors are to be accessed – for example by the dd command – it must be done via /dev/sd[a-z].

The partition size shown by fdisk ("Blocks") is rounded; the accurate partition size must be calculated by Last Sector - First Sector + 1. For example:

Code:

root@CW8:~# fdisk -l /dev/sdb
[snip]
   Device Boot      Start         End      Blocks   Id  System
[snip]
/dev/sdb7      1224268508  1250261614    12996553+  83  Linux

1250261614 - 1224268508 + 1 = 25993107.

The partition size shown by cfdisk by default is in MB and is rounded; for an accurate partition size switch the units to sectors (in full screen, keep pressing u; at command line, use -Ps) and one sector (the EBR) subtracted and the offset subtracted. For example:

Code:

root@CW8:~# cfdisk -Ps /dev/sdb
Partition Table for /dev/sdb

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
[snip]
 7 Logical  1224268445* 1250261614*    63    25993170 Linux (83)           None

25993170 - 1 - 63 = 25993106. <== why not the same as the fdisk calculation?

rknichols · 12-17-2012, 12:13 PM

Yes, the offset is one 63-sector track, one sector of which is occupied by the EBR.

The reason for that 63 sectors per track is historical. Back when drives actually used CHS addressing, sector numbering in the control register ran from 1 to 256, and if you put 0 in an IDE drive's sector register you got sector 256. The reason for that is unknown -- the other address parameters all started from 0. The sector number in the partition table is a 6-bit field, so valid sector numbers there are 1..63. AFAIK that never had anything to do with error detection/correction.

catkin · 12-18-2012, 03:07 AM

Thanks again rknichols

That's it for now.