ext3 lost space???
 

Hi, I have formatted my 120GB hdd to ext3 and now I have 6GB of space and I don't know where it went. Here is my df output:

Filesystem Type Size Used Avail Use% Mounted on
/dev/hda5 reiserfs 4.1G 3.2G 897M 79% /
tmpfs tmpfs 363M 0 363M 0% /dev/shm
/dev/hda7 reiserfs 3.4G 2.3G 1.2G 68% /home
/dev/hda1 ntfs 20G 15G 4.9G 76% /windows/C
/dev/sda1 ext3 113G 33M 107G 1% /media/Maxtor <---- WTF???


The drive is actually USB and I need it to be ext3 (my choise would be reiser or xfs, but I am forced to use ext3).

Is this behavior normal? Or am I doing smth wrong?


Thanks

So-called 120GB drives are actually smaller (around 120,000,000,000 bytes)
Everything is OK.

Yes, sure, they are smaller, because they are 120 giga bytes, so that is somewhere around 113GB, and that is what df command shows, but what puzzles me is that why is there used space 33MB and free space 107 of 113GB?
Is there 6GB of FS data? Reiser/xfs or even NTFS for that matter use only 100-200MB of space for the file system, so I am confused about where did 6GB go.

I have exactly the same thing: 107/113GB free.

I have looked at several ext3 filesystems, and there always seems to be a lost 5%.
I have also looked at a NetBSD filesystem (not sure about the name, but it's the standard BSD-like filesystem used by NetBSD), and, here too, there was a lost 5%.
On solaris, it's always 10%.

After reading df, mkfs, mkfs.ext3,mkfs.ext2, newfs and several other manpages, my guess is that df does not include reserved blocks in the Avail size.
I say this because filesystems with 5% reserved blocks have a 5% 'lost space', and filesystems with 10% reserved blocks (Solaris default) have a 10% 'lost space'

Read the reserved block count with tune2fs to be sure.

EDIT: If you need the extra space, use tune2fs to lower the reserved block count. Only do this on drives that only provide extra storage (ex. a hard drive to store pictures, or other user data only, like /home). DON'T do this on a partition that provides storage for essential system applications (like /tmp, /usr, /, /etc)

But hey, reiserfs doesn't do that. Look: on same hardware:

Filesystem Type Size Used Avail Use% Mounted on
/dev/hda5 reiserfs 4.1G 3.2G 897M 79% /
tmpfs tmpfs 363M 0 363M 0% /dev/shm
/dev/hda7 reiserfs 3.4G 2.5G 907M 74% /home
/dev/hda1 ntfs 20G 15G 4.9G 76% /windows/C
/dev/sda1 reiserfs 115G 33M 115G 1% /media/Maxtor

No to mention, reiserfs is about 15x faster, then ext3: Look here:

http://www.tud.ttu.ee/~t030726/reiserfs.png

and

http://www.tud.ttu.ee/~t030726/ext3.png

Damn, and I am forced to use ext3, I would be much happier with reiser. So my advice is to always user reiser whenever possible.

Thanks for the reply, now at least i know I am not alone, who has lost 6 GB :)

It is not exactly lost. By default when you format an ext2/ext3 filesystem 5% is reserved for root use. In your case 5% is 6GB. This is space is supposed to reduce fragementation and allow root to login in case the filesystem becomes full.

And as stefan_nicolau suggests you can use tune2fs to reduce the amount of reserved space.

Yes, but what about performance? I use a Maxtor diamond max9, which is one of the industrie's fastest performing IDE ATA-133 drives, and operating at 2 MB/s is crazy, I will start tearing my hair out if I see this speed again (My SD-Card works faster on FAT32).

I see with man tunefs, that I can tweak the size of blocks, to improve performance and, there is some time optimization, is there something else, that can help me speed-up the rate?

Use hdparm to test the hard drive speed before jumping to conclusions.

hmm, strange, you are right, hdparm shows better results, actually here are the results:

Maxtor 7200RPM Reiser:


/dev/sda1:
Timing buffered disk reads: 124 MB in 3.06 seconds = 40.54 MB/sec

Maxtor 7200 ext3:

/dev/sda1:
Timing buffered disk reads: 78 MB in 3.03 seconds = 25.77 MB/sec
darkstar:/home/ntadmin # hdparm -t /dev/hda1

Seagate Laptop HDD 4200RPM with Reiser:

/dev/hda1:
Timing buffered disk reads: 82 MB in 3.06 seconds = 26.76 MB/sec

So ext3 is still significantly slower, then reiser (not 15x though, wich kind of makes me happy), well at least I got my 6 GB back and increased the speed to 25MB/s after reformatting with -m 0 option.

However I will now create some dumb files on the 120GB drive to test it at 70% used and 90% used space to see how it reacts to it.

Thanks, you made me feel better, I thought Red Hat's ext3 was really outdated and slow.

hdparm tests hard drive speeds, not filesystems. Your ext3 hard drive seems slower than your reiser hard drive.

You can use hdparm to try to tune disk access.

It could still be true, what version do you use?
If you use a 2.4.16 or later kernel, post the kernel version.

Linux darkstar 2.6.11.4-21.7-default #1 Thu Jun 2 14:23:14 UTC 2005 i686 i686 i386 GNU/Linux

hmm, but how can it monitor disk speed if I used the same drive for ext3, then reformatted it with reiser and got different results. Then I tested on another drive with NTFS and reiser partitions, and NTFS and reiser sections of /dev/hda got different speeds. Besides: why would you need to enter the partition ID if it scanned the drive speed? it would just need /dev/hda, not /dev/hda1, so it does meter FS speed too.

A) You have the latest version of ext3 (check which journal mode you are using if performance seems low)
B) I have no clue what is going on with hdparm
C) http://forums.gentoo.org/viewtopic-t-322898.html
D) Repeat the above tests on your computer if you don't trust the author (or want to confirm his results).

The Journal size is around 2GB (auto-selected) and the block size is 4k, which is the best for performance.
The point is, that I don't need a universal system, because versatility makes us compromise performance, it's kind of like it is with planes for example: you can't make a good bomber and a figher at the same time.
I am planning to use the HDD as a ftp server folder, so I need fast concurrent transfer rates, and that's it.

And I have found the cause of low transfers, but I don't have a solution to it yet:
if it auto-mounts on hotplug, the speed is < 2MB/s, but if I remount it again myself, the speeds are ok.
Do you have any idea what causes this?

Thanks

When the ext3 file sytem is created. It creates a lot of item called superblocks and inodes. These are the areas which are used to store things like location, size, permission and filenames on your file system. It creates them all at the start and they are what are taking up all that space on your hard drive. The journal and reserved spaces added to this gives you a reasonable size chunk of you drive. Without these you have no file system.
In other words it takes a percentage of the drive space to create a file system. This applies to all file systems.

Your internet connection will be slower than your hard drive.

Not sure about this... If you use your hard drive only to store large files, I think larger block sizes are better.