LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 05-17-2003, 08:14 PM   #1
akidd
LQ Newbie
 
Registered: Jan 2003
Posts: 23

Rep: Reputation: 15
why not? - file system fragmentation on Linux


A very common newbie question (including myself) is "how do I defrag my linux fs?".

This question is usually answered with vague responses, uninformed opinions, throw away one-liners or just plain rudeness. I've searched the usual places for answers and haven't found anything substantial. I'm just not satisfied with the answers "you don't have to". MS used to say that about NTFS as well - and it is widely understood now that defragging NTFS does improve performance.

All filesystems on sequential access media will create "fragmentation" over time. As files are added and removed of varying sizes - it is impossible to have all files contiguous on a disk of a "normally" used system. It is completely understandable that more efficient algorithms and storage methods can reduce the problem ( like creating files in contiguous free space first, or reordering during system inactivity).

Regardless - it seems logical that any disk based filesystem should benefit from defragmentation - even if very infrequently.


Can anyone direct me to a technical answer to the question?

Thanks for your help
 
Old 05-17-2003, 08:21 PM   #2
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
Quote:
Disk fragmentation in the Windows file systems, whether it be the traditional FAT file system of Windows 3/95A, the FAT32 file system of Windows 95B/98, or the NTFS file system of Windows NT, causes significant performance degradation.

Disk fragmentation cuts directly across the integrity of your system. Files fragmented into 200 pieces take 200 times longer to access. Files shattered into 200,000 pieces will take 200,000 times longer, and so on.

The ext2 file system which is native to Linux fragments very little because of the way the file system is designed. Fragmentation on a typical ext2 disk is usually between zero and three percent no matter how much file system activity occurs. By default, the Linux swap area on disk is on its own disk partition and does not affect normal files.
 
Old 05-17-2003, 08:33 PM   #3
akidd
LQ Newbie
 
Registered: Jan 2003
Posts: 23

Original Poster
Rep: Reputation: 15
Thanks for the reply - I'm looking for the technical discussion/description around what this paragraph is saying - how does the ext2, ext3, reiserFS accomplish 0-3% typical fragmentation. The swap file is obviously not the only file that grows and shrinks on a FS so this alone isn't the answer.
--------------

The ext2 file system which is native to Linux fragments very little because of the way the file system is designed. Fragmentation on a typical ext2 disk is usually between zero and three percent no matter how much file system activity occurs. By default, the Linux swap area on disk is on its own disk partition and does not affect normal files.
 
Old 05-17-2003, 08:44 PM   #4
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
This is more technical

http://e2fsprogs.sourceforge.net/ext2.html

this site also has links to other filesystems for linux

Last edited by DavidPhillips; 05-17-2003 at 08:50 PM.
 
Old 05-17-2003, 08:52 PM   #5
akidd
LQ Newbie
 
Registered: Jan 2003
Posts: 23

Original Poster
Rep: Reputation: 15
David - do you ever rest?

Linux is Linux because of people like you David. Thanks for all your help and time.
 
Old 05-17-2003, 08:54 PM   #6
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
You may also want to see if you have defrag or defragfs for jfs installed

Last edited by DavidPhillips; 05-17-2003 at 08:56 PM.
 
Old 05-17-2003, 08:57 PM   #7
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
Sometimes it does seem that I am always looking at this monitor.
 
Old 05-17-2003, 09:05 PM   #8
akidd
LQ Newbie
 
Registered: Jan 2003
Posts: 23

Original Poster
Rep: Reputation: 15
hate to be annoying

That was a good article. This section is the most germaine to my question:

Ext2fs also contains many allocation optimizations. Block groups are used to cluster together related inodes and data: the kernel code always tries to allocate data blocks for a file in the same group as its inode. This is intended to reduce the disk head seeks made when the kernel reads an inode and its data blocks.

When writing data to a file, Ext2fs preallocates up to 8 adjacent blocks when allocating a new block. Preallocation hit rates are around 75% even on very full filesystems. This preallocation achieves good write performances under heavy load. It also allows contiguous blocks to be allocated to files, thus it speeds up the future sequential reads.

These two allocation optimizations produce a very good locality of:

related files through block groups
related blocks through the 8 bits clustering of block allocations.

---------------

Where did you get the text that claims fragmentation of only 0-3%? I found a blurb with the same text GOOGLING, but no description of test methodology or how that figure was derived.

Even after this article, I have to wonder if defragging wouldn't eventually be useful? The 0-3% fragmentation claim seems astronomical and I'd really like to read how that can be validated. Always writing to contiguous blocks is smart, and it claims a 75% hit rate in doing this even on full disks - but that seems very data size, free space and write frequency dependent.
 
Old 05-17-2003, 09:11 PM   #9
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
I found it by searching for "how ext2 works"

There is a defrag tool, it's just not needed as much as windows defrag, and the way the filesystem is accessed it appears that the benefit would not be near the same as the benefit gained with windows.

I think it would make a very slight difference, maybe not even enough to notice.

Last edited by DavidPhillips; 05-17-2003 at 09:32 PM.
 
Old 05-17-2003, 09:16 PM   #10
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
ok here is where I saw it

http://www.biznix.org/whylinux/windows/fragment.html

also I think maybe by checking the filesystem fragmentation on systems that have been running for years would give a good idea of what to expect.
 
Old 05-17-2003, 09:17 PM   #11
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
http://www.biznix.org/whylinux/windows/index.html
 
Old 05-18-2003, 02:49 AM   #12
Robert0380
Guru
 
Registered: Apr 2002
Location: Atlanta
Distribution: Gentoo
Posts: 1,280

Rep: Reputation: 47
it said Kernel panics are rare, i had a kernel that paniced often once. but i keep like 3 or 4 kernels on my system so i just stopped using the one that paniced all the time.

lol, my kernel had "panic attacks"
 
Old 05-18-2003, 06:39 AM   #13
whansard
Senior Member
 
Registered: Dec 2002
Location: Mosquitoville
Distribution: RH 6.2, Gen2, Knoppix, 98,2000 + various
Posts: 3,171

Rep: Reputation: 51
when i want to defrag, i copy everything off to a different
partition, delete everything, then copy back.
i do that for reiserfs because i run with --notails, but
i copy the stuff onto the drive without --notails. it takes
up less room that way, but helps with performance.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Linux filesystems and fragmentation Frogular Linux - General 1 04-29-2005 03:47 AM
Check Non Continuous File Status (fragmentation)? carl0ski Mandriva 0 02-07-2005 08:34 AM
Does Linux ext2 and 3 really never suffer fragmentation? MadCactus Linux - General 4 11-03-2004 02:36 AM
Linux File System Jerrac Linux - Newbie 4 12-16-2003 04:26 AM
Does Slackware/Linux suffer from fragmentation? CodeWarrior Slackware 2 07-27-2003 09:30 PM


All times are GMT -5. The time now is 07:03 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration