LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-02-2016, 01:25 AM   #1
Joy Stick
Member
 
Registered: Dec 2015
Distribution: RHEL 4.7
Posts: 127

Rep: Reputation: 0
File system and cooked file system


Hi all

I read some articles.
I do not understand some points (for a layman)

1) Why cooked file system IO is slower than raw device ?
2) If raw devices gives better performance rather than file systems then why we need cooked file system ?

Raw devices are bypassing OS layer

3) What is OS layer? Where do we find it ?
4) Buffering may good for video files, why buffering is required to retrieve contents from OS files.

Thanks

Last edited by Joy Stick; 01-02-2016 at 01:49 AM.
 
Old 01-02-2016, 04:11 AM   #2
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,842

Rep: Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472
Quote:
Originally Posted by Joy Stick View Post
Hi all

I read some articles.
I do not understand some points (for a layman)

1) Why cooked file system IO is slower than raw device ?
The "cooked" filesystem contains data that describes the storage of application data. To read or write some data requires reading/writing this description data first - thus slowing down the rate application data may be read or written.
Quote:
2) If raw devices gives better performance rather than file systems then why we need cooked file system ?
The problem with raw devices is that there is no control - data can be written arbitrarily by any process - thus allowing one process to overwrite data belonging to another process. The simplest (and fasted) type of storage requires the disk to have "assigned" storage for every file... Unfortunately, that also means the file can't grow. Such pre-allocation is done when the computer is used for a single dedicated purpose (or the disk is). Such things happen with embedded systems where everything about the data is known ahead of times, or when the disk is assigned to be used by a single function - such as a database. Unfortunately for the data base, that also means that the database system must perform the operations of allocation/deallocation/garbage collection/secure access control/IO scheduling... and they are not that good at doing that.
Quote:
Raw devices are bypassing OS layer

3) What is OS layer? Where do we find it ?
Most systems (I could say ALL systems, but that is carrying it a bit too far) have kernels that control all aspects of the system. The Linux kernel sits between the hardware, and user applications:
Code:
hardware  - the physical components of memory, computation units (CPUs/GPUs), and
            peripherals such as disks, keyboards, mice, ...
the kernel- controls access to the hardware to ensure sharing the CPU/memory/peripherals such that one application
            doesn't interfere with other applications.
user applications - the programs that carry out the users wishes
Most of your questions are better suited to a classroom or tutorial. Some introductory text for operating systems:
https://en.wikipedia.org/wiki/Operating_system
Quote:
Thanks
Not a problem.

Last edited by jpollard; 01-02-2016 at 04:14 AM.
 
Old 01-02-2016, 06:46 AM   #3
Joy Stick
Member
 
Registered: Dec 2015
Distribution: RHEL 4.7
Posts: 127

Original Poster
Rep: Reputation: 0
1) Raw devices are bypassing OS layer

Quote:
- Did you mean OS layer is KERNEL
2) Why cooked file system IO is slower than raw device ?

-
Quote:
To read or write some data requires reading/writing this description data first
- Description - What do you mean by description data
 
Old 01-02-2016, 07:34 AM   #4
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,842

Rep: Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472
Quote:
Originally Posted by Joy Stick View Post
1) Raw devices are bypassing OS layer
From your usage, yes.
Quote:

2) Why cooked file system IO is slower than raw device ?

-

- Description - What do you mean by description data
A UNIX (or most operating systems) filesystem has multiple users that are not necessarily allowed access to some data. To implement that requires the users to go through a "cooked" filesystem.

The UNIX filesystem is divided into parts:

1. a superblock. This block identifies the filesystem type, and locates where a file identification list is, and how large it is (In UNIX and unix like systems this is an inode table), it also identifies the blocks that are available for use (the free list; in UNIX and unix like systems this is an inode table). An inode contains information about who owns the file, what data blocks belong to the file (or if the file is really small, the file data itself). The list of data blocks used in the file can be organized in different ways. The classic method is that the inode contains the first 10 (or so) block numbers that contain the beginning of the file. The last block number is used to locate a block that contains an additional list of blocks. This block is the "first level indirect" because the list of block numbers are for blocks that contain additional block numbers (that hold the data). The last block number entry is for yet another block. This block is the "second level indirect", because the block identifies contains a list of block numbers that list more blocks that are also lists of blocks... And it goes three levels deep.

This tree has to be managed - every block of data written has to be accounted for, when the file is deleted, the data blocks have to be tracked... and any/all indirect blocks have to be tracked.

To allocate a block for either data or pointer blocks requires additional management (at its simplest, the list of free blocks is a bitmap - one bit for every block. If the block is allocated the bit is set to 1, if it is available it is set to 0.

Creating the filesystem requires initializing the inode list, and marking the bits in the free list that are used by the inode table. This prevents an inode block from being used for data (which would then lose the data that used to be there).

The inode list is numbered from 0 to however many files can be created (again, this varies as different filesystems have different ways to specify the inode list). The classic implementation has inode 0 being identified as a directory. The data within the directory is a list of inode number, file name pairs. The inode 0 directory has a first entry "0,." where the "." is the file name. This represent the root file system. To maintain a tree, the second entry in the inode 0 directory fiel is "0,..", the .. is the name of the parent directory. Because this is the first directory in the filesystem (the "root" directory), both names identify the same inode.

Any additional files created within this directory will be assigned an inode, and the name will be added to the directory. Thus if a file "data" is being created - the entry would have "1,data", because the data file would get a free inode (and the next one would be inode 1).

This is only a brief (and necessarily incomplete) introduction.

For a user to access the data, first the filesystem has to be "mounted". Mounting requires the filesystem superblock to be read, and the information in it gets associated with a mountpoint. During the boot process access to the system tools requires the filesystem holding those tools to be mounted. The kernel achives this by initializing a memory block that contains the necessary pointers to the inode 0 block (it actually copies it into memory so that it doesn't have to keep re-reading the block every time it gets referenced). Now when a command is given, the kernel takes the path specified by the system call and matches it against a mounted list. Assuming the command is "/bin/ls", the first "/" character causes the kernel to search the directory of the first mounted filesystem for the name "bin". That requires the filesystem to read the first data block of the directory. If the name isn't found there, it must read more blocks (these data blocks are specified by the pointer blocks in the inode, or the indirect pointer blocks, or the third level pointer blocks). Thus a read for one block of data gets translated into reading two or more blocks... When the name "bin" is found, the kernel now has the inode number associated with the directory bin. And the process repeats with that inode until it finds the file "ls".

Now the inode number of "ls" is found, another part of the kernel determines if it is executable (specified by flags in the inode header), and will (either directly or indirectly) start reading the data blocks... To read data blocks into memory requires the kernel to allocate memory for it. Once all the data blocks associated with the "ls" file are in memory the kernel can then create a process (allocating CPU time...) that will give the CPU access to the memory (and which memory blocks).. to execute the program.

There are entire text books on how all the parts of the kernel interact to accomplish shared access to resources.

This has only been a VERY SHALLOW exposure of only a few of the procedures involved.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] after install how to keep file system safe and use another partion for my file system highrolla Linux - Newbie 25 04-08-2013 02:22 PM
Linpus Fedora after saving files to file system, CAN'T open FILE SYSTEM from desktop! remiwa Fedora 2 01-07-2009 07:28 AM
when does linux switch from realmode file system to protectedmode file system sitthar Linux - General 3 04-08-2006 08:05 AM
figuring out 'file system' and 'swap file system' types TrulyTessa Linux - Newbie 3 09-26-2005 06:46 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:45 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration