Linux file structure question about i-nodes and allocation
 

I'm on files and file systems with UNIX Academy training DVDs and I'm preparing for my first exam. I have a question to which I can't find an answer. I learned from the training and accompanying books that file name and a pointer to its actual i-node are stores in directory record. I have no problem with that, as I can clearly verify all of it.
My question is, though, most files are much larger that one i-node can accommodate, so I logically conclude that it takes many more i-nodes to store a file.
Here's the question: how file knows where rest of its parts allocated? It is not in directory records, I checked. Apparently it is not in superblock. So where is the magic place that stores all file' pieces?

Try here.

syg00, thanks! I don't know how I missed this article on Wikipedia.

Would the design like that, as I see it, create very fragmented files?

Can somebody to advise/comment about file system fragmentation in Linux and to deal with it?

This is not really a Linux (kernel) question, it's more a filesystem
question. Most distributions use ext2, ext3 or ext4 by default, and
these filesystems use inodes. Most other filesystems don't use them.

An inode is a reference to the first sector on the harddrive where
the file is stored. At the end of each sector is the address of the
next sector or an indicator that there are no more sectors. This means
that each file has only one inode, regardless of how many sectors are
used to store the file's contents.

Generally you don't need to be concerned about fragmentation on Linux
filesystems. When the kernel writes a file it automatically finds a
place that requires minimal fragmentation.

I see it this way, the fragmentation must occur as different file' pieces aren't written consequently, one after another. And as far as I understand from my training, the fragmentation is inevitable as files are many, and changing in size, updating etc. The way Linux handles the writing probably improves the situation, but it is incapable of preventing the fragmentation. The fragmentation occurs.

More from Wikipedia...

See "ext3"

Play Bonny! :hattip:

Does it mean there's no standard de-fragmentation tool in Linux? So how people deal with the fragmentation, or it is just ignored? In windows it is very efficient and useful tool.

Fragmentation does not become significant enough to affect performance in native Linux file systems.

The inode system allows the kernel to find all the pieces of the file almost instantaneously. I have read (can't find the source) that, even with defragmentation, Linux file systems tend to maintain a steady, not an increasing rate of fragmentation because the file systems police themselves.

Frankly, fragmentation is not that much of an issue with NTFS, either. In FAT and FAT32, the system had to load the first block of a file to find the pointer to the second block; the second block to find the pointer to the third block; and so on. This is why fragmentation would slow performance of a machine as fragmentation increased over time.

In NTFS, my Windows 2000 instructor said that the pointers to all blocks for a file are contained in the MFT. A fragmentation map that might bring a FAT or FAT32 system to its knees will have only a marginal effect on an NTFS system--even though the map may seem scary to someone who cut his teeth on FAT, as I did--because the MFT works in the same general way as the inode system.

frankbell, as I understand it from my Unix Academy DVD training, problem of fragmentation isn't that kernel has a difficulty finding i-nodes or data blocks, but that data block allocated all over the drive and takes many drive spins to read them (istead of one), so it's becoming inefficient and slow.

Actually, it can be both. The longer seek times mean more drive spins during the seeking.

Fabrication is not an issue with the major Linux filesystems.

The kernel and the filesystems together police the system so that the percent of fragmentation remains well within limits and does not grow. Third party tools would be redundant.

Edit: Another way of looking at it would be to consider that fragmentation control (not elimination, but control) is built-in.

How can it remain within limits? Imagine the system that has many data files (aka database) and the files often rewritten and their size is changing both ways. It is inevitable that fragmentation will occur and it will be growing. And it would be happening not because something is wrong with Linux file system, I think it is just the way it is. As I see it (may be I'm wrong), kernel has limited ability to control how and where it puts new data. Otherwise the data writing alone would become huge issue and entire disk map to be kept in memory and constantly calculate location/relocations. May be on some expensive disk controllers it would be possible (my wild guess). As I researched there are third party programs that admit and deal with fragmentation issue.

Basically, the filesystem service called by the kernel is fairly clever and is designed to resist fragmentation in general.
Historically it's been so successful that no defrag tool has been supplied by the core code writers.
Don't forget that by default when you create a new fs on a disk it reserves 5% space (tunable) for exclusive use by root.
http://linux.die.net/man/8/mkfs.ext3

In extreme cases eg continually at 95% full and large files being created/deleted/updated, you will get some fragmentation, but by then you should allocate a bigger disk anyway.