LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 04-03-2007, 09:06 PM   #1
james2b
Member
 
Registered: Feb 2007
Location: Washington state, USA
Distribution: Fedora 13, SUSE 11, Ubuntu 10.04, Mandriva 2010, Mint 8
Posts: 336

Rep: Reputation: 34
Question disk defrag and temp files


Does Linux have a need for and system tools for hard drive disk defrag and disk clean up for temporary files such as is in the Windows XP ? Useing Ubuntu 6.06 and OpenSUSE 10.2with the ext3 file system.
 
Old 04-03-2007, 09:30 PM   #2
St.Jimmy
Member
 
Registered: Jun 2006
Location: Boaz,Alabama
Distribution: Ubuntu 10.10 / Windows 7 Pro 64-Bit / Snow Leopard 10.6.4 64-Bit
Posts: 152

Rep: Reputation: 30
there are some defraggers out there, but you don't need them. As for logs, most distro's delete them at boot, but check your docs for information.
 
Old 04-04-2007, 10:20 AM   #3
SlowCoder
Member
 
Registered: Oct 2004
Location: Southeast, U.S.A.
Distribution: Fedora (Desktop), CentOS (Server), Knoppix (Diags)
Posts: 934

Rep: Reputation: 38
What allows file systems like ext3 to not require defragmenting? Does it auto-defragment on the fly?
 
Old 04-04-2007, 12:06 PM   #4
jay73
Guru
 
Registered: Nov 2006
Location: Belgium
Distribution: Ubuntu 11.04, Debian testing
Posts: 5,019

Rep: Reputation: 130Reputation: 130
ext3 is one of those file systems that is poorly supported when it comes to defragging. Then again, it's true that it suffers far less fragmentation than, say, fat32 because of the different approaches these systems have to storing data.

To put it simply, a file system can be represented as a notebook. Ext 3 will use only lines that are completely free, while fat32 tends to fit things in wherever it finds a spot - and since these spots are frequently too small to hold all of a piece of data, the remaining bit(s) will be placed elsewhere on the disk. Data being spread out all over the disk, that's precisely what fragmentation is all about. The more your disk heads need to move to gather data, the poorer the performance.

Now, if you know this, you"ll understand that ext3 can fragment as badly as fat 32: when there is little space left on a partition (=no more fresh lines),it will effectively be forced to break up data and store them wherever it finds "lines" that are not full yet. Under such conditions, it behaves like fat32 - in fact, worse, considering it's more difficult to defragment again.

Last edited by jay73; 04-04-2007 at 12:21 PM.
 
Old 04-04-2007, 01:25 PM   #5
jrbuergel
LQ Newbie
 
Registered: Mar 2004
Location: Everett, WA.
Distribution: Open SUSE 11, Ubuntu 9.04, Fedora 11, Slackware 12.1
Posts: 7

Rep: Reputation: 0
Smile

I see then that in Linux a defragmenter is less needed. This is also partially because there are most allways a lot less programs installed and or uninstalled in Linux as in Windows. Thanks for the information on that.
 
Old 04-04-2007, 02:09 PM   #6
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
ext3 does not need defrag. It uses a journal instead of writing directly to the disk in sequence. Hence, FAT and NTFS. ext3 will first put all writes onto a hidden journal or unofficial section of files on the hard disk. It will then write the journaled file onto memory and then onto the true disk blocks, but in linear mode, taking it's time, in the background. Because it uses a journal files are written in linear mode instead of sequentially in little bits and pieces all over the place. The tradeoff is that the journal causes a lot more delays in writting to disk. However, reads are faster because the file is all in one piece within neighboring blocks. It makes sense, faster reads are more important than writes slower. I don't care how long it takes for my computer to finally write the file to disk but I do care that I can open it quickly in one swop without the disk head having to look for the file in several different areas. Hence FAT and NTFS. So ext3 writes all files already defraged. You don't need defrat in ext3 - defrag always sounded silly to me. Somehow it reminds me of the parts in really old movies where the guy has to hand-crank his Ford Model-T's to start.
 
Old 04-04-2007, 07:46 PM   #7
SlowCoder
Member
 
Registered: Oct 2004
Location: Southeast, U.S.A.
Distribution: Fedora (Desktop), CentOS (Server), Knoppix (Diags)
Posts: 934

Rep: Reputation: 38
Quote:
Originally Posted by ramram29
ext3 does not need defrag. It uses a journal instead of writing directly to the disk in sequence. Hence, FAT and NTFS. ext3 will first put all writes onto a hidden journal or unofficial section of files on the hard disk. It will then write the journaled file onto memory and then onto the true disk blocks, but in linear mode, taking it's time, in the background. Because it uses a journal files are written in linear mode instead of sequentially in little bits and pieces all over the place. The tradeoff is that the journal causes a lot more delays in writting to disk. However, reads are faster because the file is all in one piece within neighboring blocks. It makes sense, faster reads are more important than writes slower. I don't care how long it takes for my computer to finally write the file to disk but I do care that I can open it quickly in one swop without the disk head having to look for the file in several different areas. Hence FAT and NTFS. So ext3 writes all files already defraged. You don't need defrat in ext3 - defrag always sounded silly to me. Somehow it reminds me of the parts in really old movies where the guy has to hand-crank his Ford Model-T's to start.
ramram,

I'm from Windows, so please bear with me ...
From what you described here, it sounds like ext3 simply caches the to-be-written files to the hidden journal location, then writes them to the disk storage when it has time. But I don't understand how that helps keep the files defragmented. I don't understand journalling, but from what I've read elsewhere it sounds like its more of a data redundancy facility, rather than for defragmentation.

So, I've got this scenario: 10 byte hard drive (sounds rediculous, but keeps it simple). There is a 5 byte file sequentially written to the drive starting at byte #1, and a 3 byte file sequentially written starting at byte #7. This leaves only bytes #6 and #10 available for writing.

How Windows does it: The file is saved in fragments into the #6 and #10 byte locations.

How does ext3 handle this type of situation? Does it automatically defragment in the background, then save the new, perhaps journalled, files defragmented?
 
Old 04-04-2007, 08:05 PM   #8
Quakeboy02
Senior Member
 
Registered: Nov 2006
Distribution: Debian Squeeze 2.6.32.9 SMP AMD64
Posts: 3,245

Rep: Reputation: 121Reputation: 121
From this page: http://www.salmar.com/pipermail/wftl...ch/000603.html

Quote:
if you are Linux, you reorder the
_requests_ into a regular sequence that minimizes disk access using
something like an elevator algorithm. You also read ahead on the drive
(optimizing disk access), buffer most of the file data in memory, and
you only write dirty blocks. In other words, you minimize the effect of
'file fragmentation' as part of the other optimizations you perform
on the _access requests_ before you execute them."

...

Yes, all disk drivers use elevator algorithms, in any o/s.

But to answer your question, ext2s spreads blocks out evenly through
the disk, using various strategies (well, a single mixed strategy)..
This reduces the average seek time on a single elevator pass.
IOW, Linux doesn't rely simply on accessing the disk in a linear fashion. It looks at everything that's likely to be needed on the disk for the accesses it has pending and caches that to memory. In Linux, the data would be read in disk order, but not in file order, and then reordered properly before handing it to the requesting process. The data is actually fragmented on the disk, but because the of the more efficient kernel access, it doesn't matter.
 
Old 04-05-2007, 05:30 AM   #9
hand of fate
Member
 
Registered: Nov 2003
Distribution: Mandriva
Posts: 441

Rep: Reputation: 30
Quote:
Originally Posted by jrbuergel
This is also partially because there are most allways a lot less programs installed and or uninstalled in Linux as in Windows. Thanks for the information on that.
What are you basing that claim on?
 
Old 04-05-2007, 08:06 AM   #10
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
Quote:
Originally Posted by SlowCoder
How does ext3 handle this type of situation? Does it automatically defragment in the background, then save the new, perhaps journalled, files defragmented?
ext3 will save the files in fragments if it has no other choice. That is why it is important to keep some free space on your hard disk. You should never have your partitions filled more than 90%, especially if they are small partitions.

I have noticed a lot less thrashing, I would even say, almost no noticeable thrashing in Linux, even when I use a lot of big multimedia GUI apps.
 
Old 04-05-2007, 12:43 PM   #11
SlowCoder
Member
 
Registered: Oct 2004
Location: Southeast, U.S.A.
Distribution: Fedora (Desktop), CentOS (Server), Knoppix (Diags)
Posts: 934

Rep: Reputation: 38
Thank you. That clarifies some things for me.

For me, sometimes reading threads raises more questions for me, whether they be technical or philosophical. I appreciate your willingness to answer.
 
Old 04-11-2007, 05:27 PM   #12
changedsoul
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Rep: Reputation: 0
Question I have been woundering. If indeed the way ext2 seems to access data on the drive and allows for faster reads, why then does it seem to take forever long for apps to load in linux?

Not bashing Linux, but Windows XP and all programs I use in Windows load light years faster than Linux and apps in Linux. I am using openSuse 10.2 and I am new to linux in general so I am sure there are tweaks to be had to speed things up, but was just woundering if part of the slow application load times could be caused from not so fast disk reads as was claimed above.

Also...probably not the thread for this, but I have been having programs acts as if they were loading ( clicked the icon / mouse cursor changed showing something was going on) but after maybe 30 secs, nothing loads and the cursor changes back as if I never clicked the link to start the program in the first place.
Does anyone know what could cause this?
 
Old 04-11-2007, 06:49 PM   #13
stress_junkie
Senior Member
 
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 331Reputation: 331Reputation: 331Reputation: 331
Quote:
Originally Posted by SlowCoder
What allows file systems like ext3 to not require defragmenting? Does it auto-defragment on the fly?
I see that a lot of people have put forward numerous theories about this. The popular opinion is, as you asked about, that ext3 and other file systems don't need to be defragmented. The same claim is made about ext2 and even ufs.

When I was taking my first class on Solaris system administration I asked the teacher the same question. I was used to Windows and VMS and they both required a lot of defragmenting. The teacher said that it was due to the inode structure of the file system. Frankly I didn't buy it then and I don't buy it now. On the other hand I don't see Unix or Linux systems performance degrade over time as would be evident if the disks were getting very fragmented.


Last edited by stress_junkie; 04-11-2007 at 06:51 PM.
 
Old 04-11-2007, 09:02 PM   #14
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
Quote:
Originally Posted by changedsoul
Question I have been woundering. If indeed the way ext2 seems to access data on the drive and allows for faster reads, why then does it seem to take forever long for apps to load in linux?
There are thousands of possibilities why your system may be so slow to load programs, including misconfiguration, a bad disk, a bad installation. I don't thing it has anything to do with ext2 or ext3. I would suggest that you read about benchmarks performed comparing ext3 to NTFS. ext3 outperforms NTFS in many tests. Here's a good link http://m.domaindlx.com/LinuxHelp/res...benchmarks.htm
 
Old 04-11-2007, 10:01 PM   #15
jay73
Guru
 
Registered: Nov 2006
Location: Belgium
Distribution: Ubuntu 11.04, Debian testing
Posts: 5,019

Rep: Reputation: 130Reputation: 130
Performance actually involves more than just hard drives, of course. MSware tends to pre-load a lot of stuff in RAM - which does make for faster access - but it also tends to waste resources unless you are aware of all that is going.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
disk utilities, scandisk, defrag, etc BCBruce Linux - Newbie 8 03-09-2006 05:31 PM
Where is Disk Defrag? Boffy Linux - General 3 08-28-2004 11:30 PM
defrag/temp files mindseye Linux - Newbie 1 01-18-2004 11:28 AM
Defrag...and ext2/3 files systems. oicdn Linux - General 1 01-09-2004 02:10 PM
temp files Daggelicious Linux - Newbie 2 08-21-2003 10:00 PM


All times are GMT -5. The time now is 03:32 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration