LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-04-2007, 12:03 PM   #1
veeruk101
Member
 
Registered: Mar 2005
Distribution: Ubuntu 12.04 LTS
Posts: 249

Rep: Reputation: 16
Conceptual questions about partitions


I have several questions about disk partitions, that'll help me understand this area that's new to me. Hopefully some of the Linux guru's out there could help to clarify:

1) Can you ever create an extended partition if you have used less than three primary partitions? (i.e. 1 or 2 primary and then add an extended?)

2) Can Linux manipulate (e.g. write bytes to, read bytes from) unpartitioned space, or is only partitioned space visible to Linux?

3) Can you ever create 4 primary/extended partitions that don't take up the entire disk? Why might anyone ever do this? Isn't that space lost to any OS that ever tries to use the disk until someone makes the remaining space viewable to the OS?

4) Is the way that a disk is formatted with its partitions completely independent of (i.e. standard across) which tool or OS does it, and which filesystem goes on top of it?

5) Are there any performance or other differences between making a partition primary vs logical, or can they be considered identical for all purposes? (Or are there cases when you should definitely make it one or the other - e.g. production environment, database server, etc?)

6) Can there ever be more than one extended partition on a disk? (I've read the term nested partitions a lot - does this mean that an extended can contain other extended partitions?)

It's a lot of questions, but it'll definitely be some good learning for me. Thanks.
 
Old 12-04-2007, 12:39 PM   #2
indienick
Senior Member
 
Registered: Dec 2005
Location: London, ON, Canada
Distribution: Arch, Ubuntu, Slackware, OpenBSD, FreeBSD
Posts: 1,853

Rep: Reputation: 65
Quote:
Originally Posted by veeruk101 View Post
1) Can you ever create an extended partition if you have used less than three primary partitions? (i.e. 1 or 2 primary and then add an extended?)
Yes, but the device nodes will still be starting at /dev/*d4 for any extended (logical) partitions.
Quote:
Originally Posted by veeruk101 View Post
2) Can Linux manipulate (e.g. write bytes to, read bytes from) unpartitioned space, or is only partitioned space visible to Linux?
No - the space needs to have a filesystem present as when data is written, it is not passed to the data blocks directly. All data flow is either passed to, or intercepted by, the filesystem, which then shuffles the data around accordingly.
Quote:
Originally Posted by veeruk101 View Post
3) Can you ever create 4 primary/extended partitions that don't take up the entire disk? Why might anyone ever do this? Isn't that space lost to any OS that ever tries to use the disk until someone makes the remaining space viewable to the OS?
Yes, yes you can. The unformatted space will just be as though it doesn't exist (unless you run Windows, which will set up a "drive" for it, but the moment you try to access it, will attempt to format it with a filesystem). You may want to do this if you need to leave space for another operating system that does not have publicly-available tools that work under the primary OS (ie. ZFS for Solaris).
Quote:
Originally Posted by veeruk101 View Post
4) Is the way that a disk is formatted with its partitions completely independent of (i.e. standard across) which tool or OS does it, and which filesystem goes on top of it?
The tool used to format the partition will be from the developer and/or maintainer of the filesystem. In the case of a filesystem like NTFS or HPFS, the tools are built into the OS that primarily uses them. While NTFS formatting tools do exist for Linux, they aren't "official." The whole idea of third-party filesystem tools is that they try and imitate the original as best as possible without adding features the original doesn't have.
Quote:
Originally Posted by veeruk101 View Post
5) Are there any performance or other differences between making a partition primary vs logical, or can they be considered identical for all purposes? (Or are there cases when you should definitely make it one or the other - e.g. production environment, database server, etc?)
While I don't know for sure if there are any performance differences, I highly doubt there would be. A partition is a partition is a partition. The whole "primary" vs. "extended/logical" forum is probably a convention left over from around the time they discovered dirt, and had some completely solidified reason as to why it was this way. If this is the case, the reason has long since been lost, and yet still maintained due to the infrastructure around it (think meta-headers in files - the POSIX method - versus the extension method inherent in DOS). DOS started the whole file extension scheme so as to know filetype A is run by program A. Whereas the POSIX method relies on file encoding and meta-information in the file headers (think MIME types). The only reason the DOS-method of file naming still exists - IMHO - is because it makes it marvelously easy to parse long lists of files.
Quote:
Originally Posted by veeruk101 View Post
6) Can there ever be more than one extended partition on a disk? (I've read the term nested partitions a lot - does this mean that an extended can contain other extended partitions?)
Primary partitions exist in numbers 1 through 3. Extended always resides in partition 4. Anything with a parition number > 4 is a subset - "nested" as you put it - within partition 4. For a better idea of how this works, try installing FreeBSD (or any SVR4 UNIX for that matter). They get you working around the whole idea between "partitions" and "slices."

In Windows you will only ever see C:, D:, ..., Z:. In Linux you will see /dev/{h,s}d{a,b,c,d}N. But in an SVR4 UNIX, you will see /dev/hd{a,b,c,d}NsM.

Last edited by indienick; 12-04-2007 at 12:46 PM.
 
Old 12-04-2007, 01:05 PM   #3
veeruk101
Member
 
Registered: Mar 2005
Distribution: Ubuntu 12.04 LTS
Posts: 249

Original Poster
Rep: Reputation: 16
Thanks a lot for your answers - some follow-up questions below:

Quote:
No - the space needs to have a filesystem present as when data is written, it is not passed to the data blocks directly. All data flow is either passed to, or intercepted by, the filesystem, which then shuffles the data around accordingly.
So if I have this right, Linux will be able to view that space is available, but cannot read/write it? It would have to do the following:
- format it with a filesystem
- mount it
- now it can read/write?

New question:

If you need a filesystem to write to a disk, how do tools like fdisk work, which write bytes directly to a disk without any knowledge or use of filesystems?

Quote:
Yes, yes you can. The unformatted space will just be as though it doesn't exist (unless you run Windows, which will set up a "drive" for it, but the moment you try to access it, will attempt to format it with a filesystem). You may want to do this if you need to leave space for another operating system that does not have publicly-available tools that work under the primary OS (ie. ZFS for Solaris).
Could you expand on this, I can't understand the example of when you'd not partition all the space - can't OSs not read unpartitioned space? So the other OS (Solaris, like you described) would still be out of luck? I'm a bit confused - it's all new to me.

Quote:
The tool used to format the partition will be from the developer and/or maintainer of the filesystem. In the case of a filesystem like NTFS or HPFS, the tools are built into the OS that primarily uses them. While NTFS formatting tools do exist for Linux, they aren't "official." The whole idea of third-party filesystem tools is that they try and imitate the original as best as possible without adding features the original doesn't have.
Because disks partitioned using one tool (e.g. fdisk) on one OS (e.g. Linux) can be recognized, used, and manipulated by another OS (e.g. Windows), does this mean that the protocol for partitioning a disk (e.g. all that stuff about MBRs, partition sectors, etc) is a standard that relates to all disks, regardless of in what environment they'll be used?

Thanks.
 
Old 12-04-2007, 02:28 PM   #4
indienick
Senior Member
 
Registered: Dec 2005
Location: London, ON, Canada
Distribution: Arch, Ubuntu, Slackware, OpenBSD, FreeBSD
Posts: 1,853

Rep: Reputation: 65
Quote:
So if I have this right, Linux will be able to view that space is available, but cannot read/write it? It would have to do the following:
- format it with a filesystem
- mount it
- now it can read/write?
Yes - but you will have to partition the space first using gparted, cfdisk, fdisk, or the like so Linux knows to create a node to that space in /dev.
Quote:
If you need a filesystem to write to a disk, how do tools like fdisk work, which write bytes directly to a disk without any knowledge or use of filesystems?
When you parition a drive, the first X sectors of a drive a reserved for a partition table. Fdisk and the like do nothing more than read and write partition tables which, at the very least, contain the total size of the drive, start and end sectors for each partition, and the filesystem which they are inteded for regardless of whether or not they've been formatted yet. In fdisk and cfdisk, you can see code lists of all the different types of partition types. Any filesystem you would find in use on a Linux install (ext2, ext3, reiserfs, xfs, etc.) are all classified under the "Linux" and "Linux swap" codes (82 and 83, respectively, I believe).
Quote:
Could you expand on this, I can't understand the example of when you'd not partition all the space - can't OSs not read unpartitioned space? So the other OS (Solaris, like you described) would still be out of luck? I'm a bit confused - it's all new to me.
I can only expand on this slightly as I don't have an intrinsic knowledge on the inner workings of Windows (dim the lights, add dark, sinister chanting and steady drum beats). It appears as though Windows will read the partition table and create a drive for the unpartitioned space. It won't format the space until you try to access it through the drive link it provides. However, Windows cannot (and will not) read non-FAT or non-NTFS partitions and will not set up drive letters for them.

Essentially, to access a paritioned and formatted drive, the OS needs to have - at the very least - installed drivers that provide read capabilities for that filesystem.

In the Linux world (since it is the most pertinent to know about), Linux will NOT attempt to create half-assed drive links (in Linux terms, "nodes") to unpartitioned spaces on a hard drive; it will leave well enough alone. It will only recognize drives which have been paritioned with a particular filesystem code as described in the parition table.

As an additional note, you don't need to format the drive for Linux to recognize it, it just needs to have an appropriate filesystem code in the partition table. After every parition table modification, you must restart your computer for Linux to read the partition table, and create appropriate nodes.
Quote:
Because disks partitioned using one tool (e.g. fdisk) on one OS (e.g. Linux) can be recognized, used, and manipulated by another OS (e.g. Windows), does this mean that the protocol for partitioning a disk (e.g. all that stuff about MBRs, partition sectors, etc) is a standard that relates to all disks, regardless of in what environment they'll be used?
Yes. Reading and writing parition tables is standardized. However, an OS cannot read and/or write to a partition unless it knows how to, through drivers and such.

NOTE: MBR (Master Boot Records) have little, to nothing, to do with this discussion. I'm not slamming down, I'm just letting you know. But yes, MBRs are standardized as well. While writing to the MBR isn't standardized, HOW it is written to, is. Reading the MBR (at least from the standpoint of your computer booting up) is handled - practically - exclusively by the BIOS.

Last edited by indienick; 12-04-2007 at 02:36 PM.
 
Old 12-04-2007, 03:51 PM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
Just a couple of hair-splitting nuances:

Using low-level tools like dd you can read and write anywhere on the disk---even if it has no partitions.

I like to think of extended/logical partitions as a linked list. What actually happens is that the first logical requires the first extended to be created. (Many tools do this automatically.) Think of the extended as a pointer to the first logical. The logical partition contains a "boot sector" with space for the same 4-entry partition table that lives in the MBR. When the 2nd logical is created, an extended partition is created in that 1st boot sector---pointing to logical #2......etc.

Can you use all four entries in these "logical" partition tables? No way to do it that I have seen.
 
Old 12-04-2007, 04:17 PM   #6
veeruk101
Member
 
Registered: Mar 2005
Distribution: Ubuntu 12.04 LTS
Posts: 249

Original Poster
Rep: Reputation: 16
Reading your responses, I realize that things may be more complicated than I thought, with there being a difference between partitioning, formatting, and adding a filesystem to a disk! Could somebody please elaborate on these differences as it relates to this discussion?

Based on my understanding, I would guess that the dd tool can write to a disk where none of the above 3 things have been performed, but only parts of the disk that have been partitioned (though not necessarily formatted or made to contain filesystem) are visible to Linux?

Just when I thought I had it...

Quote:
Using low-level tools like dd you can read and write anywhere on the disk---even if it has no partitions.
So do tools like fdisk that write out partition tables build upon the dd tool (or use the same underlying system calls, if I've used the term correctly, as dd)?
 
Old 12-04-2007, 04:24 PM   #7
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
Quote:
Originally Posted by veeruk101 View Post
Reading your responses, I realize that things may be more complicated than I thought, with there being a difference between partitioning, formatting, and adding a filesystem to a disk! Could somebody please elaborate on these differences as it relates to this discussion?

Based on my understanding, I would guess that the dd tool can write to a disk where none of the above 3 things have been performed, but only parts of the disk that have been partitioned (though not necessarily formatted or made to contain filesystem) are visible to Linux?

Just when I thought I had it...



So do tools like fdisk that write out partition tables build upon the dd tool (or use the same underlying system calls, if I've used the term correctly, as dd)?
You've got me on the last one---One sure way to find out would be to get the source code for fdisk. Great project for a student: Recreate fdisk using BASH + all the built-ins (which includes dd)

Anything which is a filesystem function will only work in areas of the disk that have a filesystem--this is not unique to Linux. So, what is the definition of "filesystem function"? I would guess anything that can operate on filenames and directory names.

Last edited by pixellany; 12-04-2007 at 04:25 PM.
 
Old 12-04-2007, 05:17 PM   #8
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 116Reputation: 116
Quote:
So if I have this right, Linux will be able to view that space is available, but cannot read/write it? It would have to do the following:
- format it with a filesystem
- mount it
- now it can read/write?
The dd command could read/write the bare space, placing anything in that space. You could, in fact, use dd to construct your own "file system" on otherwise bare and unformatted HD space. Wouldn't be efficient and would have no error tolerance, but you could do it. Actually, dd can read/write to any part of the disk, regardless of whether or not that space is partitioned and in use.

Quote:
So do tools like fdisk that write out partition tables build upon the dd tool (or use the same underlying system calls, if I've used the term correctly, as dd)?
dd, fdisk, and every other tool that talks to a disk, communicates through a device driver which is a piece of code that translates a set of universal Linux system calls (such as open, close, read, write, seek) into device specific commands.

fdisk accesses the disk through this driver and implements a set of rules that exist throughout the x86 world for historical reasons, to build a disk partition structure that is recognizable by a large number of different operating systems. This need not be done; this built-in structure could be completely ignored by any OS but the ability of that OS to interoperate with - well - with a lot of things - would be greatly impaired. You can think of a partition as being a "logical disk", in that the system will treat it pretty much like a separate disk, while it is in fact only a part of a disk.

The file system also accesses the hard drive through this driver, working within one partition, but the file system has built a complete structure on the partition which enables files to be stored, found, and maintained. Basically, the filesystem defines and implements a substantial set of rules by which data is placed in the partition, while the dd command just puts anything you tell it to put anyplace you tell it to put it.

So, dd ignores all the file system and partitioning rules. This makes it very powerful, very useful, and very dangerous.

Last edited by jiml8; 12-04-2007 at 05:23 PM.
 
Old 12-04-2007, 06:38 PM   #9
David1357
Senior Member
 
Registered: Aug 2007
Location: South Carolina, U.S.A.
Distribution: Ubuntu, Fedora Core, Red Hat, SUSE, Gentoo, DSL, coLinux, uClinux
Posts: 1,302
Blog Entries: 1

Rep: Reputation: 107Reputation: 107
Quote:
Originally Posted by veeruk101 View Post
1) Can you ever create an extended partition if you have used less than three primary partitions? (i.e. 1 or 2 primary and then add an extended?)
Yes. An extended partition uses only one entry in a partition table to point at an partition table entry in the first "logical" partition. That partition table entry can be unused, or it can point at the second "logical" partition. In this manner, you can create arbitrary number of partitions, limited only by the amount of disk space you have.

Quote:
Originally Posted by veeruk101 View Post
2) Can Linux manipulate (e.g. write bytes to, read bytes from) unpartitioned space, or is only partitioned space visible to Linux?
Only partitioned space is visible to Linux. Linux reads the partition table to get the disk addresses it uses to access space on the drives. It will not access access unallocated space or partitions it does not recognize unless an error occurs or you install a custom device driver that allows direct access to the hard drive interface.

The exception to this (as other posters have mentioned) is the fact that a hard drive looks like a file to the operating system (e.g. /dev/hda). You can open that file and seek to any position in it and read and write it. The tool most commonly used to do this type of operation is "dd". However, you could write a C program or even a Perl script to do the same thing. In all cases, you risk data loss.

Quote:
Originally Posted by veeruk101 View Post
3)a) Can you ever create 4 primary/extended partitions that don't take up the entire disk? b)Why might anyone ever do this? b)Isn't that space lost to any OS that ever tries to use the disk until someone makes the remaining space viewable to the OS?
a) Yes
b) One example: Some OEMs use this technique to hide system restoration information from the operating system so that it does not get corrupted by the user.
c) Yes

Quote:
Originally Posted by veeruk101 View Post
4) Is the way that a disk is formatted with its partitions completely independent of (i.e. standard across) which tool or OS does it, and which filesystem goes on top of it?
I do not know the answer to this question. I do know that an NTFS partition formatted using Linux works under Windows, but Windows often adds some extra housekeeping files to the disk (e.g. "System Volume Information" and "RECYCLER" directories).

Quote:
Originally Posted by veeruk101 View Post
5) Are there any performance or other differences between making a partition primary vs logical, or can they be considered identical for all purposes? (Or are there cases when you should definitely make it one or the other - e.g. production environment, database server, etc?)
There should be no performance differences. The operating system most likely reads the partition table information (including the partition tables of extended partitions) and saves the information in a cache. Once it has this information, it can calculate drive addresses without incurring any extra overhead no matter what the partition type is.

Quote:
Originally Posted by veeruk101 View Post
6) Can there ever be more than one extended partition on a disk? (I've read the term nested partitions a lot - does this mean that an extended can contain other extended partitions?)
No. In my previous response, I was incorrect. You only need one extended partition. Inside of that, you can create an arbitrary number of "logical" partitions.

Last edited by David1357; 12-04-2007 at 10:00 PM. Reason: Edited to fix erroneous information about extended and logical partitions. Please accept my apologies.
 
Old 12-04-2007, 08:16 PM   #10
wraithe
Member
 
Registered: Feb 2006
Location: Australia
Distribution: Linux... :-)
Posts: 241
Blog Entries: 1

Rep: Reputation: 50
Quote:
Originally Posted by veeruk101 View Post
Reading your responses, I realize that things may be more complicated than I thought, with there being a difference between partitioning, formatting, and adding a filesystem to a disk! Could somebody please elaborate on these differences as it relates to this discussion?

Based on my understanding, I would guess that the dd tool can write to a disk where none of the above 3 things have been performed, but only parts of the disk that have been partitioned (though not necessarily formatted or made to contain filesystem) are visible to Linux?

Just when I thought I had it...



So do tools like fdisk that write out partition tables build upon the dd tool (or use the same underlying system calls, if I've used the term correctly, as dd)?

ok, patitioning relates to the harddrive, physical layout...

formatting is the way that an OS sets the flags for its filesystem to use that partition...

filesystem is the method that files are located on space and indexed...

these are the simple answers, and can be elaborated upon in more detail but then it becomes more complicated...

dd is a tool for low level work on a drive, and unless you have lots of drive knowledge, dont touch...
lowlevel is the actual viewing of data on the drive, irrelevant of drive fs and is how you can recover data thats erased(thats why high security organisations rewrite over deleted data several times, so it cant be recovered)...a lowlevel format will destroy any data on the surface of a drive, this is not recommended as it can sometimes even destroy the drive it self...

fdisk sets partition tables so that an OS can format that space for its own filesystem...
dd is another app that performs a task at a lower level of the drive...
 
Old 12-04-2007, 09:15 PM   #11
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
Quote:
Yes. An extended partition uses an entry in a partition table to point at another partition table on the disk where you get 4 more partition table entries. You can make any of those new partition table entries an extended partition. By repeating this process, you can create an arbitrary number of partitions. All you need is at least one extended partition entry in the MBR partition table.
I don't think so---the "secondary" partition table has the standard four entries, but only two get used. I don't know any partitioning tool that will use all 4.

(In the output of fdisk -l, you'll never see more than one extended partition.)

Last edited by pixellany; 12-04-2007 at 09:18 PM.
 
Old 12-04-2007, 09:48 PM   #12
rootboy
Member
 
Registered: Oct 2001
Distribution: Mint 15
Posts: 770

Rep: Reputation: 51
Quote:
Originally Posted by jiml8 View Post
The dd command could read/write the bare space, placing anything in that space. You could, in fact, use dd to construct your own "file system" on otherwise bare and unformatted HD space. Wouldn't be efficient and would have no error tolerance, but you could do it. Actually, dd can read/write to any part of the disk, regardless of whether or not that space is partitioned and in use.
Actually, I can see where this would be extremely efficient (no slack space*).

Leave a chunk of disk unpartitioned and use dd to read and write from it.

You tell dd where & how much to read (Sectors? We don't need no stinkin' sectors!)

Nice way to keep private things private...

As for error tolerance, it's no worse than losing a sector in almost any other filesystem. Once it's gone it's gone.


Quote:
Originally Posted by jiml8
So, dd ignores all the file system and partitioning rules. This makes it very powerful, very useful, and very dangerous.
Yup.


*http://en.wikipedia.org/wiki/Slack_space
 
Old 12-04-2007, 09:51 PM   #13
David1357
Senior Member
 
Registered: Aug 2007
Location: South Carolina, U.S.A.
Distribution: Ubuntu, Fedora Core, Red Hat, SUSE, Gentoo, DSL, coLinux, uClinux
Posts: 1,302
Blog Entries: 1

Rep: Reputation: 107Reputation: 107
Quote:
Originally Posted by pixellany View Post
I don't think so---the "secondary" partition table has the standard four entries, but only two get used. I don't know any partitioning tool that will use all 4.

(In the output of fdisk -l, you'll never see more than one extended partition.)
I apologize for posting an erroneous reply.

pixellany is correct. The extended partition table entry is used to point to the first "logical" partition. The first sector (512 bytes) of the first logical partition is used to point to the next logical partition. In this manner, you can create an arbitrary number of partitions. I just verified this using fdisk on an IDE drive.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: gOS: The Conceptual Google OS Screenshots LXer Syndicated Linux News 0 11-03-2007 10:21 PM
Java: A Conceptual Question timmit Programming 7 10-03-2007 09:12 PM
[conceptual] Network Disk setup chief_officer Linux - Networking 1 02-05-2007 01:44 AM
Ldap neophite with basic conceptual question(s)??? BobMCT Linux - Server 0 11-16-2006 07:43 AM
conceptual server design question Bostonian Programming 3 12-27-2004 02:22 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:07 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration