Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
you don't have to defrag in linux .. the file system is organized and stored more efficiently than a windows machine. If you really want to then i can
suggest you go to something like www.google.ca/linux and search for
"linux defrag" or something, and grab a utility .. they do exist, but unnessesary..
2. Nope, don't worry about it. Even in Windows, you should only worry about defragging your filesystem if its 20% fragmented or more.. which is rare in most cases. The ony time I ever saw any type of performance increase on any OS after defragging was on Win98 and Win95 systems.. which used a horrible filesystem.
Originally posted by Boow well if your so obsessed with defragging in linux give fsck -a a try. you'll need a livecd to fsck the / partition since its mounted all the time.
That's why you can tell shutdown to perform a fsck on the / filesystem upon reboot while it mounts it as ro.. no need for a live cd.. pah!
Now imagine your hard disk is a huge file cabinet, with millions of drawers (thanks to Roberto Di Cosmo for this comparison). Each drawer can only contain a fixed amount of data. Therefore, files that are larger than what such a drawer can contain need to be split up. Some files are so large that they need thousands of drawers. And of course, accessing these files is much easier when the drawers they occupy are close to one another in the file cabinet.
Now imagine you're the owner of this file cabinet, but you don't have time to take care of it, and you want to hire someone to take care of it for you. Two people come for the job, a woman and a man.
* The man has the following strategy : he just empties the drawers when a file is removed, splits up any new file into smaller pieces the size of a drawer, and randomly stuffs each piece into the first available empty drawer. When you mention that this makes it rather difficult to find all the pieces of a particular file, the response is that a dozen boys must be hired every weekend to put the chest back in order.
* The woman has a different technique : she keeps track, on a piece of paper, of contiguous empty drawers. When a new file arrives, she searches this list for a sufficiently long row of empty drawers, and this is where the file is placed. In this way, provided there is enough activity, the file cabinet is always tidy.
Without a doubt, you should hire the woman (you should have known it, women are much better organized ). Well, Windows uses the first method ; Linux uses the second one. The more you use Windows, the slower it is to access files ; the more you use Linux, the faster it is. The choice is up to you!
I frequent about 20 or so Linux and Unix newsgroups, and the question of
linux defrag has come up so often in these groups that I've put together
a stock answer that tries to explain what 'fragmentation' is and what
linux does about it. However, although my explanation is detailed in
some respects, it lacks a lot of information in others. I think that I
need to include more information on (1) how linux filesystems (ext2,
ext3, afs, etc.) manage file data block arrangement, in the light of
'file fragmentation', and what performance exposures _are_ present in
the filesystems.
So, I'm asking for suggestions; does anyone here have a good (simple)
explanation of how our filesystems work, and where their weaknesses are?
I'll take anything I can get, and credit you with the information.
FWIW, what follows is my 'stock defrag' answer; enjoy...
In a single-user, single-tasking OS, it's best to keep all blocks for a
file together, because _most_ of the disk accesses over a given period
of time will be against a single file. In this scenario, the read-write
heads of your HD advance sequentially through the hard disk. In the same
sort of system, if your file is fragmented, the read-write heads jump
all over the place, adding seek time to the hard disk access time.
In a multi-user, multi-tasking, multi-threaded OS, many files are being
accessed at any time, and, if left unregulated, the disk read-write
heads would jump all over the place all the time. Even with
'defragmented' files, there would be as much seek-time delay as there
would be with a single-user single-tasking OS and fragmented files.
Fortunately, multi-user, multi-tasking, multi-threaded OSs are usually
built smarter than that. Since file access is multiplexed from the point
of view of the device (multiple file accesses from multiple, unrelated
processes, with no order imposed on the sequence of blocks requested),
the device driver incorporates logic to accomodate the performance hits,
like reordering the requests into something sensible for the device
(i.e elevator algorithm).
In other words, fragmentation is a concern when one (and only one)
process access data from one (and only one) file. When more than one
file is involved, the disk addresses being requested are 'fragmented'
with respect to the sequence that the driver has to service them, and
thus it doesn't matter to the device driver whether or not a file was
fragmented.
To illustrate:
I have two programs executing simultaneously, each reading two different
files.
The files are organized sequentially (unfragmented) on disk...
[1.1][1.2][1.3][2.1][2.2][2.3][3.1][3.2][3.3][4.1][4.2][4.3][4.4]
As you can see, the accesses are already 'fragmented' and we haven't
even reached the disk yet (up to this point, the access have been
against 'logical' addresses). I have to stress this, the above
situation is _no different_ from an MSDOS single file physical access
against a fragmented file.
So, how do we minimize the effect seen above? If you are MSDOS, you
reorder the blocks on disk to match the (presumed) order in which they
will be requested. On the other hand, if you are Linux, you reorder the
_requests_ into a regular sequence that minimizes disk access using
something like an elevator algorithm. You also read ahead on the drive
(optimizing disk access), buffer most of the file data in memory, and
you only write dirty blocks. In other words, you minimize the effect of
'file fragmentation' as part of the other optimizations you perform
on the _access requests_ before you execute them.
Now, this is not to say that 'file fragmentation' is a good thing. It's
just that 'file fragmentation' doesn't have the *impact* here that it
would have in MSDOS-based systems. The performance difference between a
'file fragmented' Linux file system and a 'file unfragmented' Linux
file system is minimal to none, where the same performance difference
under MSDOS would be huge.
Under the right circumstances, fragmentation is a neutral thing, neither
bad nor good. As to defraging a Linux filesystem (ext2fs), there are
tools available, but (because of the design of the system) these tools
are rarely (if ever) needed or used. That's the impact of designing up
front the multi-processing/multi-tasking multi-user capacity of the OS
into it's facilities, rather than tacking multi-processing/multi-tasking
multi-user support on to an inherently single-processing/single-tasking
single-user system.
All "fragmented" drives are better than "unfragmented" ones on a
multiuser multitasking o/s. The point is that the machine is doing
many things simultaneously, so it has to jump arround even if one task
is interested in only one file. Tehre will be up to a hundred tasks
doing i/o simultaneously.
Yes, all disk drivers use elevator algorithms, in any o/s.
But to answer your question, ext2s spreads blocks out evenly through
the disk, using various strategies (well, a single mixed strategy)..
This reduces the average seek time on a single elevator pass.
"Linux filesystems" is a little misleading. e2fs doesn't generally
have fragmentation issues, for certain definitions of "fragmentation."
The short answer is this: e2fs splits the disks up into block groups,
which are contiguous regions of blocks. The group will contain a
certain number of inodes and (data) blocks. When you create an inode,
Linux probably chooses the group with the largest number of free
(data) blocks. When you write to an inode, Linux will preferentially
allocate (data) blocks in the same group as the inode. When it has
to, it will move on to another (later) group, but will still try to
keep the blocks together.
The end result of this is that data is generally fragmented by only a
few blocks, and almost always travels in the same direction. That's
as opposed to the front-to-back fragmentation which could, and
frequently did, occur in FAT and its derivatives.
The above works great until the file system is nearly full, at which
point free blocks are scattered all across the disk is discontiguous
locations. This is why, on a nearly-full file system (above 95% or
so), e2fs performance will degrade _substantially_.
Other file systems (HPFS in particular) are similar, but call groups
"bands" or "stripes" instead. HPFS is actually worse than e2fs when
nearly full, because it uses pseudo B-trees for the directory
structure which periodically need to be rebalanced. The problem there
is that, when the file system is nearly full, directories may need to
be rebalanced into many different groups, which will obviously cause
enormous slowdowns. e2fs uses a crummy, paleolithic array for its
directories, which results in far worse performance overall, but wins
out in this one narrow case (or can, depending on what's done to the
directory).
Sorry, but most people on this group know better than to mention "file
systems" and "explain" in the same sentence when I am around.
Now, this is not to say that 'file fragmentation' is a good thing. It's just that 'file fragmentation' doesn't have the *impact* here that it
would have in MSDOS-based systems. The performance difference between a
'file fragmented' Linux file system and a 'file unfragmented' Linux
file system is minimal to none, where the same performance difference
under MSDOS would be huge.
All the answers re Linux and defragging mention ext2/ext3/hpfs/etc - i.e all are Linux/Unix native file systems. What about VFAT file systems under Linux? Is the VFAT fs as clever as ext2fs or is it a straight port of the FAT16/FAT32 MS-DOS fs? Many of us have such drives in our system as that is for the most part the only sane way to share data in a multiboot system with MS products. As an example, I have quite a large USB drive with all my music, photos and videos on, and with all the re-organising and deleting cleaning up going on, it does become very fragmented over time. Now, I can always defrag by booting into Windows and running their defragger. Alternatively I can move all the files to some empty space on another drive and re-format/clear the multimedia drive, but that takes time and I do not always have 160 GB spare capacity handy. So it would be really nice to have a Linux defragger for the odd Windows/MS-DOS drive.
There is a Windows driver available that will let you use an ext2 or ext3 file system from Windows NT or XP. (Since the driver doesn't have a "Microsoft approved" signature, it's unlikely that it could be used with Vista.)
Using the driver, you can use a native Linux file system for your Windows storage, and eliminate the need for an FAT storage.
Thanks, I will look into the Windows fs driver. There is of course the problem of portability - I would have to install that driver on all of the PCs where that USB disk may be plugged in. I suppose that the best way to do this would be to split the disk into 2 partitions, one being a FAT32 and the other a suitable Linux native partition. The FAT32 partition can then be mounted anywhere and can be used to hold the fs driver installation file for the other partition. The only other drawbacks will then be Vista, and having the necessary permissions on the PC to install the driver.
well if your so obsessed with defragging in linux give fsck -a a try. you'll need a livecd to fsck the / partition since its mounted all the time.
The fsck utility is a tool for checking the consistency of a file system, not to defrag a file system; fsck is equivalent to the scandisk and chkdsk programs in Windows.
Hi,
do a "shutdown -F"
and it will tell you how much it is fragmented when the system reboots. It will be less than 5% unless the disk is full. The only file systems that seem to get fragmented are: ms-dos fat 16, fat32, ntfs. And vms. Since the person who wrote most of VMS is now working for Microsoft it probably has the some bugs. When I used to fix hard drives on DEC systems it was common to back the systems to tape, initialize the original drive, restore it all back again.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
). Well, Windows uses the first method ; Linux uses the second one. The more you use Windows, the slower it is to access files ; the more you use Linux, the faster it is. The choice is up to you!
Windows fs driver
