SOLVED: Trying to "reverse engineer" a calculation: any number hackers out there?
 

Here is the solution for calculating the last sector used in the partition table, usefull for when the number of Heads on the disk is not 255. This is handy if you are ever need to write to the partition table manually (such as in a perl script).

note: I am not an expert so I can't guarantee this is correct, but it did work for me.

================

first use 'sfdisk -l' to get your disk geometry.

let H = the number of Heads on the disk
let C = the number of cylinders on the disk
let T = 63 (I believe this is a constant)
let L = the Last sector written on the partition table

the equation to find the last sector you should use is as follows:


================

That's all there is to it.

If you want to see how this is derived, read on. Otherwise, happy disk witting!

first, let's find out which sector sfdisk used by dumping the partition table:

We can get the last disk used by adding up the total sectors:
63 + 16595082 + 1076355 + 1959930 = Last Sector Used = 19631430

Now let's find the actual number of sectors on the disk:

The total sectors is found with:
C*H*T = 19640880

note the variance between the two (remember this number)
19640880 - 19631430 = 9450 sectors

Now let's also note that the two numbers both share a common factor of the number of Tracks (63)
19640880/T=311760

19631430/T=311610

Hmm, this is interesting. Now lets try getting all of the factors to see what we can come up with.

factor(19631430)
2 * 3^3 * 5 * 7 * 13 * 17 * 47

Lets try and get 3 usefull numbers from this (Tracks, Cylinders, and Heads)
Now we know 63 is a factor so let's pull it out. We can also see 255 as a factor, so let's pull that out as well. From what's left, we can see the geometry that was actually written to the disk 1222 cylinders, 255 heads, and 63 Tracks.
(3^2 * 7)(3 * 5 * 17)(2 * 13 * 47)

This is all well and good, but what if we need to find out the last sector we should write to when it is unknown? So we derive the information we just used into an equation.

In order to figure out the last sector that will be written, we need to find out how many cylinders there are based on a 255 head disk geometry.

Consider the Following proof:


Hopefully this will be of some use to someone. I figured since I spent all the time doing this, I might as well share it with anyone who needs it.

regards,
...drkstr


note: everything form here to post # 09 was written before the above text

** original question **
I have been racking my brain over the past 3 hours trying to "reverse engineer" (for lack of a better term) the way sfdisk calculates the last sector it uses in the partition table. Any of you number hackers out there feel like giving me a hand? Here is an example output from a 'sfdisk -d /dev/hda' on my computer.
and here is the disk geometry What I need to know is how sfdisk comes up with the last sector sfdisk writes to since it is not the last sector of the disk

Now I don't really know a lot about disk geometry so I've just been plugging numbers in the calculator to see if I can reproduce the numbers with an equation.

I've noticed that both the total sectors on the disk, and the last sector written by sfdisk have a common factor of 63 (Number of tracks) Does this have anything to do with how the last sector written to is calculated, or am I just way off? Is it possible to compute the number I need with an equation?

Any input will be greatly appreciated!
...drkstr
** original question **

Hi

Doesnt the first sector belong to the boot sector and so it is not counted at all?
therefore the sectors will be the last -1

Hi Aaron, I don't understand what you're asking here. There's a lot of info but the question isn't clear (to me). Please spell out what you are trying to understand.

P.S. You see that sfdisk counts starting from sector 63, right (in the first example you posted) but from 0 in the second. In the first example this means the the first track (sectors 0 - 62, inclusive) is bypassed. This is the MBR, as synapse alluded to, but what he said about sectors - 1 isn't true- it's sectors - 63.

sorry, this was a bit confusing. I realize I didn't really explain well what I was trying to do.

What I am need to be able to do is edit the output of the partition table dump (the first example) so I overwrite the existing partition table by piping it back through sfdisk. In order to do this I need to be able to calculate how they come up with each number in the dump. All of them are pretty strait forward except for the "total number of sectors included in the partition table".

note: I know over writing the partition table "live" sounds like a bad idea, but it actually works. I've tested it by using cfdisk to change the partition table, then dumped it with sfdisk and piped it back in. When I was done I was able to restore the partition table back from the original (which I dumped to a file before changing anything). This all worked without rebooting or losing any data.

The very last sector it uses is not (total disk size) - (63 sectors) it is actually (total disk size - 9450 sectors) I strongly belive that the number of sectors used by the partition table can be figured out mathmaticly since it is not an arbitrary number (it can be devided evenly by both the number of heads or the number of tracks).

The first example is the actual partition table dumped by sfdisk, where the second example is the total disk size (MBR included).
But then why is the varience from total sectors written to total sectors on the disk 9450 sectors instead of 63?


thanks for everyones interest!
...drkstr

Well I think I might have figured it out after a little bit of sleep.

Sfdisk wrote the partition table by a factor of 255 heads instead of 16.

Sfdisk calculation:
LastSector = 63+16595082+1076355+1959930 = 19631430.
Heads = 255.
SectersPerTrack = 63.
LastSector / Heads / SectersPerTrack = 1,222. Cylinders

Actual disk geometry (correct terminology?):
Heads = 16.
SectersPerTrack = 63.
Cylinders = 19485.
ActualSectors = Heads * SectorsPerTrack * Cylinders = 19640880.

When converted to a 255 Head geometry
ActualSectors / 255. / SectersPerTrack = 1222.58823 WrongCylinders… Uh oh!

Therefore sfdisk wrote to the last whole cylinder based on a 255. Head geometry.

To confirm: (WrongCylinders – 1222.) * 63. * 255. = 9450. Sectors

Which is the exact variance I was getting in Sectors.

In order to use the full hard drive space, it looks like I will need to read in the # of heads from the disk, and specify this number before calling sfdisk.


Thanks everyone for the help! Your comments were able to get me thinking in the right direction.

regards,
…drkstr

Good going. I was going to point out that your second set of numbers was in cyls but your first set was in sectors. You can change this in fdisk (don't know about sfdisk/cfdisk, since I don't use those) by going into "expert" mode.

So it's possible I wasn't getting any varience at all, just comparing apples to oarenges?

...drkstr

I'm sorry, I don't have time to go over the calculations at the moment, but I did notice from the two examples you pasted that the units were different. Also I think most people find the geometry defined in terms of 255 heads like you mentioned later. I don't know why it's shown as 16 in the initial example you pasted.

That's ok, I think I have the calculations pretty much worked out. I'm pretty sure you just confirmed in words what I found in numbers. Which is a big help since I'm always trying to ask why.

I think the problem might have been with the fact that I was working with an old hard drive on a laptop from 2001. The same 'fdisk -l' command run on my server gives the units as 255 Heads which is what led me to think that sfdisk was not using the correct geometry for my disk.

I'm just happy to have it figured out. Now I can get back to writing code again.

Thanks for all your help!
...drkstr

Looks like I can mathmaticly determain the last sector used by the partition table. I wrote a brief proof in hopes it can be of use to anyone else.

regards,
...drkstr

Nice job. Maybe put it in lq bookmarks......

As requested, thread title changed

Several things I see:

You have a null number of sectors for the first partition.
The partitions are mixed physically and logically -as if some had been created, then some removed or resized or recreated.

Logical cylinder boundaries. One of the things that sucks about DOS fdisk is that you can't be very specicfic about partition sizes -some sizes are uncreatable. Linux tools are better of course, they'll warn you sometimes like this:
'number of heads/tracks/sectors doesn't match logical cylinder boundaries'

If the point is to 'occupy' all the available space on the disk, then you need to get down to block-size also so you can ask for the proper number of K's for the partition. Most of choose a 'rounded' size like 100MB or 128MB. The logical cylinder boundaries may not match these fiures. When they don't, the last unused sectors are skipped. The next partition starts at a new logical cylinder boundary.

If you want to see how sfdisk or whatever does it you could try reading the sources.

Thanks gnashley.

Logical partitions has presented a problem for me which I have chosen not to deal with at the moment. Currently, I will be able to delete a "group" of logical partitions (IE. the primary extended partition), or one of the true primary partitions. Furthermore, the new partitions will only be created on one of the 4 primary slots, and fail if there are not enough available.

This will be the focus down the road, but right now, I am just trying to modify the partition table dump with numbers that will be safe to pipe back into sfdisk.

I'm not quite sure I've achieved this yet, but I will find out shortly ;)

thanks!
...drkstr

For all Newbies reading this thread:
Don't forget the following:

• Messing with partition info can corrupt your data!
• If you do mess it up, odds are it is unrecoverable without a high degree of skill.
• Sacrifice a chicken to the hdd gods first. KFC will do it for you for a fee...

(Last was done tongue in cheek. You don't need to sacrifice a chicken. Pizza works just as well. However, I've had mixed results w/lasagna. Use at your own risk...)

A question. When you overwrite the partition table while running, do the changes show up in /proc/partitions ?

Note that logical cylinder boundary is not the same as logical partition.
If you are using an extended partition then you need to take into account that the partition table in the MBR will only hold the address of the sector where the extended partition info is kept. This extra sector must be counted in the total.
The cylinder boundaries affect the final sector even if the disk is not full. They come from the mismatch between the byte capacity of one or more full cylinders. If you choose a partiton 1MB smaller, say, the last two sectors would be 'skipped' as partition capacity but counted in the total number of sectors. The new partition begins on a cylinder boundary not on the next unclaimed sector.

I would add to this that you should make a point of defining new partitions on integral cylinder boundaries and avoid using every last bit of space.

Thanks, cwwilson721!

I added your warning to the top post.

Also, I made a pretty large mistake in my proof by evaluating when I shouldn't have. The resulting equation was one that only worked for my specific disk. I have fixed both the equation and the proof.

Also, thanks gnashley and Randux!

I will head this advice. Although this is probably more math then I feel like doing at the moment. Maybe in a few days I will give it a shot.

...drkstr

regards,
...drkstr

Thanks. I just thought that needed to be said for such low-level hdd mucking about.

But was serious about the pizza....lol

No, the host system cache's the partition table as you said, which is fine. I don't need the host system to see the new table, as long as I can access it. All I need to do is force the change (while live), do a few things, restore the original, then reboot (hopefully back to the original working system). I'm not sure if the reboot is necessary, but couldn't hurt.

regards,
...drkstr

Did somebody say something about pizza? I'll bring the brewskies.

here's some output from a couple of disks. If you are writing code that is gonna process output from sfdisk. You need to understand that all these programs write the info in hexcode and that linux uses a DOS-type partition table. So, the tools have to follow some rules that are governed by that and write info which is useful to other programs which may not do the same thing. I mean, you can write a DOS table with partition info that can be read and used by a linux program, but not by DOS or Windows or MacOS.
Finding out which sector is the last used is easy enough. But knowing why that is, or should or must be is fifferent. Some tools only create partitions the way they 'should' be. Others create them the they 'must' -letting you specify. Others do the 'best thing' for you if you leave them alone -this means that for creating partitions there is no better tool than linux fdisk. But that doesn't help if you are working with already created partitions.

bash-3.00# sfdisk -l /dev/hdc
Disk /dev/hdc: 9733 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End #cyls #blocks Id System
/dev/hdc1 * 0+ 126 127- 1020096 b W95 FAT32
/dev/hdc2 127 9732 9606 77160195 f W95 Ext'd (LBA)
/dev/hdc3 0 - 0 0 0 Empty
/dev/hdc4 0 - 0 0 0 Empty
/dev/hdc5 127+ 381 255- 2048256 b W95 FAT32
/dev/hdc6 382+ 1034 653- 5245191 b W95 FAT32
/dev/hdc7 1035+ 3358 2324- 18667498+ 83 Linux
start: (c,h,s) expected (1023,254,63) found (1023,1,1)
/dev/hdc8 3359+ 9732 6374- 51199123+ 83 Linux
start: (c,h,s) expected (1023,254,63) found (1023,1,1)
bash-3.00# sfdisk -d /dev/hdc
# partition table of /dev/hdc
unit: sectors

/dev/hdc1 : start= 63, size= 2040192, Id= b, bootable
/dev/hdc2 : start= 2040255, size=154320390, Id= f
/dev/hdc3 : start= 0, size= 0, Id= 0
/dev/hdc4 : start= 0, size= 0, Id= 0
/dev/hdc5 : start= 2040318, size= 4096512, Id= b
/dev/hdc6 : start= 6136893, size= 10490382, Id= b
/dev/hdc7 : start= 16627338, size= 37334997, Id=83
/dev/hdc8 : start= 53962398, size=102398247, Id=83
bash-3.00# fdisk -l /dev/hdc

Disk /dev/hdc: 80.0 GB, 80060424192 bytes
255 heads, 63 sectors/track, 9733 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hdc1 * 1 127 1020096 b W95 FAT32
/dev/hdc2 128 9733 77160195 f W95 Ext'd (LBA)
/dev/hdc5 128 382 2048256 b W95 FAT32
/dev/hdc6 383 1035 5245191 b W95 FAT32
/dev/hdc7 1036 3359 18667498+ 83 Linux
/dev/hdc8 3360 9733 51199123+ 83 Linux


bash-3.00# sfdisk -d /dev/hda
# partition table of /dev/hda
unit: sectors

/dev/hda1 : start= 63, size= 2000817, Id= c, bootable
/dev/hda2 : start= 2000880, size= 4305168, Id=83
/dev/hda3 : start= 0, size= 0, Id= 0
/dev/hda4 : start= 0, size= 0, Id= 0
bash-3.00# sfdisk -l /dev/hda |grep Disk
Warning: The partition table looks like it was made
for C/H/S=*/16/63 (instead of 782/128/63).
For this listing I'll assume that geometry.
Disk /dev/hda: 782 cylinders, 128 heads, 63 sectors/track
bash-3.00# fdisk -l /dev/hda

Disk /dev/hda: 3228 MB, 3228696576 bytes
16 heads, 63 sectors/track, 6256 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 1985 1000408+ c W95 FAT32 (LBA)
/dev/hda2 1986 6256 2152584 83 Linux

Thanks a bunch gnashley!

I have a lack of disks to test this on so your output is very usefull. This got me thinking, it would probably be pretty usefull to start a thread asking for as many people who are willing to just post their output. I would like to be able to look at as many different partition tables that I can.

I'm going to go pop one in the software forum, it will be interesting to see how many people respond.

thanks!
...drkstr