MBR and Boot Loader

I think we need some hard evidence to be convinced that every read and write from a hard disk must be conducted in one complete sector unit of 512 bytes.

There are plenty of threads showing that we can dd with bs=1 byte or bs=446 bytes, including one suggested here by the moderator in whom I do have faith.

We have to bear in mind that we are talking about Linux and dd here and whatever MS systems do with the hard disk should be ignored. In Linux we can impose different DMA to slow down the hard disk speed which I doubt it is practised in the MS systems.

It is possible that at the hardware level the hard disk always read/write 512 bytes at each action and throw away 511 bytes when we specify bs=1 but that sounds a bit stupid to me. As far as dd is concrerned whatever we specify it is just a "record" to it and the size doesn't matter.

"I think we need some hard evidence to be convinced that every read and write from a hard disk must be conducted in one complete sector unit of 512 bytes."

So, you're unconvinced by the mathematics of LBA-28? Would you consider maxtor.com to be a reliable source of information?

http://www.maxtor.com/en/documentati...ite_papers.pdf

The 137-gigabyte barrier is the result of the original design specification for the ATA interface that provided only 28 bits of address for data. This specification means a hard disk can have a maximum of 268,435,456 sectors of 512 bytes of data which puts the ATA interface maximum at 137.4 gigabytes.

Read this quote carefully and understand the implications of using 28 bits of data to address a single byte within a 137.4 gigabyte space. As I have already shown, and as it's clear from the quote above, it doesn't work. It's possible that they could, in the future, come up with a new hard disk specification that would allow for byte-addressability using added firmware. But, why waste the 8 bits of address space on something as trivial as that, in addition to having to add still another transfer mode and use up more command space.

I think mixing bs=1 byte is possible in dd with the 137Gb barrier limit in a hard disk is not helpful here.

Everybody knows the 137Gb limit is due to the traditional way of addressing a harddisk by Cylinder/Head/sector which is now superseded by the Linear addressing. That problem has no relevance in our understanding why a hard disk cannot read a single byte.

What is the point of using 137Gb limit to compare with the operation of ready just 1 byte?

There are plenty of Linux operations which involving the alteration of a single byte in the hard disk. The one that comes to mind is Grub's hide command and parttype command. They only change the partition ID which is the 4th byte of the 16-byte register defining a primary partition table. It is hard to swallow that many Linux commands like these of changing a single byte must involve 512 bytes in every operation. This can make Linux very uncompetitive to spend more time to strip off unwanted information than pick up the core information as required.

I can understand if a data bus is 28 bits or 32 bits long then every bit must be filled but 512 bytes or 4096 bits just sounds excessive in the modern day computing.

I for one can do with some enlightment here.

"I can understand if a data bus is 28 bits or 32 bits long then every bit must be filled but 512 bytes or 4096 bits just sounds excessive in the modern day computing.

I for one can do with some enlightment here."

I can lead you to the calculator, but I can't make you press the keys. I give up.

Can I argue with Maxtor's math?

268435456 x 512 = 137438953472 bytes / 1024 = 134217728 KiloBytes /1024 = 131072 MegaBytes / 1024 = 128 GigaBytes

Though I realize all the HDD manufacturers cheat by using 1000 instead of 1024.

Originally Posted by Quakeboy02 In summary, there is no way to address a single byte on a hard disk. What we do is to read a sector change the byte we're interested in, and then write the sector to the disk.

Ya learn something new every day. That would explain why Gentoo blew away my LBA partition even though I installed to a different partition and didn't change the partition layout at all. I guess it must not have understood the filesystem type # and wrote something else back when I installed GRUB. At the time I couldn't understand why it touched my partition table at all... makes sense now.

This is getting interesting...

Originally Posted by Quakeboy02 In summary, there is no way to address a single byte on a hard disk. What we do is to read a sector change the byte we're interested in, and then write the sector to the disk.

Actually, if I am not mistaken, the physical process involves changing the magnetization of a small spot for every BIT.
Is it physically possible to only change one BIT? We had better hope so, or getting the correct data pattern would be really hard....
I am not smart enough to quibble with everything that is being said here, but I believe we are talking about the limitations of specific firmware or software.

"Is it physically possible to only change one BIT? We had better hope so, or getting the correct data pattern would be really hard...."

Actually, it's not. For the reason, look at: http://www.storagereview.com/guide20...m/dataRLL.html

I have no idea what the technology is now, but I doubt they've gone back to a simple N/S representation for a single bit of data. That would be too wasteful of space and time.

"I am not smart enough to quibble with everything that is being said here, but I believe we are talking about the limitations of specific firmware or software."

We are talking about the limitations of hard disks. If you think about it, it's not much of a limitation. In the real world of computing, it's a very very rare thing that we want to address only a single byte on a disk; whether to read it or write it. It's much more common, and much faster to access data in larger chunks.

I imagine perpendicular recording uses a similar method. They'd have to read/write the entire "stack" of bits in one shot.

"I imagine perpendicular recording uses a similar method. They'd have to read/write the entire "stack" of bits in one shot."

We call that "stack" a sector.

Quakeboy02's assertion certainly makes many of us think again.

I think there can be a subtle difference that makes us arguing about potentially two different operations.

Operation A - hard disk access for a file

It is likely that most of the read/write operations are indeed conducted in a sector by sector basis. We do talk about bad sectors on a hard disk but never say bad bytes, right? dd command man page does indicated the default block size is 512 bytes. So that conceievable the normal or normal operation of reading a hard disk in accessing a file.

Hard disk operation B - specifically read/write operation for dd command

This operation makes us think the read/write can be smaller than 512 bytes beacuse dd does not deal with any filing system. It is well established that we can get dd to read a single byte. We thought that was it until Quakeboy02 said the normal standard of filing-related read/write still applies. In order to obtain one byte dd still has to read the whole 512 bytes of a full sector and then internally carries additional intelligence to strip out all but the very byte we instructed.

I can't dispute Quakeboy02's assertion but equally we do not want a filing system handling instruction, in which the smallest common denominator is one sector, is "assumed" for dd operations which are unrelated to any filing system but specifically for block size of any length including just one byte. It would be nice if we can get more information on the read/write operation of dd and to see it is different from a standard read/write operation for a file.

Well,
just to summarize:

1st point, the original one) I seem to understand that, no matter if we can physically change 1 byte at a time or we have to read a chunk of 512 byte and change the one we need and rewrite. Method apart, GRUB does not overwrite the entire MBR with its own unaltered stage1 (as for a floppy), but only the first part, preserving the original partition table (or "rewriting it the same as it was", if you want).

2nd point, derived from the original and the same interesting, if not more) No doubt that we can change every single byte of a hard disk (as saikee says). But the matter is "how" that change happens. Dear Saikee, if I understood, what you and Quake are saying, are both possible and not in conflict. In fact you are simply saying that if Quacke sentence is true, we could not write a single byte (i.e. with dd bs=1). But he's only saying that we can do that in a different "way" than you imagine. That is, not directly, but the way he already explained. Now, I still don't know if the Quacke description is what really happens or only his guess (also if based on valid observations). Quacke, let's suppose your theory is right, do you (or anybody else) know if the action to "read 512 bytes - change one of them in cache - rewrite the entire 512 modified bytes" is a task demanded to the high level program (i.e. GRUB install process, dd ...) or otherwise to the low level driver, or furthermore at a lower level again, i.e. to the firmware of the HD? If the driver did that, here it could be the solution. When we say "we", referring to the high level program (dd), we can say that we alter 1 single byte, (no need to pay particular attention if we were writing the code of the program (dd)), and someone other (the driver) do the dirtywork for "us". I don't know, it's only a guess.

p.s. sorry if I don't succeed to be really clear but I have problem also with my english , not only whit my HD :-)

"Quacke, let's suppose your theory is right,"

Not a theory.

"do you (or anybody else) know if the action to "read 512 bytes - change one of them in cache - rewrite the entire 512 modified bytes" is a task demanded to the high level program (i.e. GRUB install process, dd ...) or otherwise to the low level driver,"

I don't know the answer to this. But, I will point out, once again, that there is no way to address a particular byte in a sector. There is no direct mechanism to the hardware for you to address sector zero byte 17. There is also no direct mechanism for the hardware to allow you to write anything other than multiples of 512 bytes to the disk. I haven't looked under the code, so all I can do is speculate. But, I will point out that it would be a pretty lousy set of software that allowed you to corrupt the partition table just to change the MBR.

"or furthermore at a lower level again, i.e. to the firmware of the HD?"

If you can show me a way to tell the harddisk to write something other than a multiple of 512 bytes, then I will seriously consider this proposal.

"If the driver did that, here it could be the solution. When we say "we", referring to the high level program (dd), we can say that we alter 1 single byte, (no need to pay particular attention if we were writing the code of the program (dd)), and someone other (the driver) do the dirtywork for "us". I don't know, it's only a guess."

The one thing that needs to be considered is that not only does dd know the blocksize of the device you're writing to, it also knows the blocksize you've told it to use. It doesn't require any great math skill to compare those two and do the proper reads and writes to make sure that blocks aren't broken on the disk. Think about it, this way: if you wrote to a disk using one blocksize and read back using another, would you expect to see different data?

Hey guys,

Consulted my "village elders" here.

The information substantiates Quakeboy02's assertion and even says in Linux the disk controller may be asked to read/write at least 4096 bytes at a time to serve the kernel. Terminal command like dd then selects the part that are only relevant to the user's request.

We should thank Quakeboy02 for filling in our gaps of knowledge.

Still doesn't address the issue of writing to sector(s) that apparently have not been pre-read. I'll try and get my test-box back in order over the weekend and try the dd issue with the I/O scheduler set to NO - this disables the batching of I/O saikees elders mentioned.
I think I can control the general I/O well enough to see what physical I/O takes place.

Guys, this is first year CS degree stuff.