[SOLVED] nilfs2 / How2 check disk space & eventually free it

Pearlseattle · 06-17-2010, 04:49 PM

Hi!

I did some benchmarking on a USB-stick (flash memory) of nilfs2 vs. ext2 & ext3 and nilfs2 seems to be the best.

I know that nilfs2 does checkpoints and that's ok, but I don't fully understand how it handles free/used disk space:

1)
How can I check how much unused space (excluding checkpoints) is left on the device?
E.g. after writing and deleting a file I still see that the space that was used by the file is still allocated even if no files are present:

Code:

nas1 memstick # ls -ltrh
total 0
nas1 memstick # df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/sde3              14G  2.9G   11G  22% /mnt/memstick

2)
Does that allocated space get freed up somehow automatically when no space is left (garbage collector?) or do I have to free it up manually?

Don't find those answers in the docs - perhaps they're too obvious or I just missed them...?

Thanks a lot!

rweaver · 06-17-2010, 05:14 PM

Has something recently changed in nilfs2? Last I ran benchmarks and most of them I've seen online show it has pretty dismal performance comparative to just about everything except in a couple specific circumstances (and even then ext3 out performed it.)

Nilfs2 is interesting imo but it's to new to be trusted with anything you want to keep reliably stored (ditto on ext4, btrfs, etc.)

http://www.phoronix.com/scan.php?pag...s_nilfs2&num=1 is one example of what I mean... note where bigger is better and where lower is better specifically.

Andrew Benton · 06-17-2010, 06:32 PM

I don't know how to check how much space is free on the device other than to see how much is on it (with du) and then subtract that from however big the device is. When you've made a change to the file system and deleted some files it takes a while for the nilfs_cleanerd (a process that runs in the background) to reclaim the freed space. Sometimes it can take a while, particularly if the filesystem is almost full.

Pearlseattle · 06-18-2010, 04:15 PM

Thanks a lot for your replies!!

rweaver:
Well, I remember that some time ago I saw some benchmarks against nilfs2 and I agree with you remembering that the results for NILFS2 weren't good. I didn't test it back then, but what I tested now gave me a good impression.
Here are the tests I ran yesterday evening & their results:
http://www.blah-blah.ch/Mra/Nilfs2performance

The link to the tests you posted are interesting.
I programmed some time ago an app which was using an interface to sqlite and I know that if not instructed differently it uses a "sync"-approach and that result matches what I saw in my tests as the final "sync" when using nilfs2 didn't almost take any time in any occasion, contrarily to ext3.
For the 4GB-write&read performance I now get different results - perhaps something changed in the meanwhile in the code as the article is dated June 2009.
The other tests, like the 32-thread read tests are not relevant for me as I don't have e.g. an Oracle DB or a cluster using that FS - it's just a tiny, shy OS...

Andrew Benton:
Hehe..., I was hoping not to get such an answer but it looks like that's the only way to see the real used space.

Even ignoring that yesterday evening I was waiting for at least 10 minutes without seeing the garbage collector/cleanerd doing anything, today after remeounting the partition the garbage collector woke up and cleared 2.8GB and it stopped there (I know that in reality the free space should be around 13GB out of the 16)......and now?
Anyway I started writing a 4GB file and the limit seems to be at 1.4GB free space - I am writing and writing and the free space is stuck at 1.4GB (still with "df -h"). Now it finished writing the 4GB-file and I started writing one more - the free space is slowly decreasing down to 800MB and now I interrupted the write. Now (I'm doing nothing) the free space varies (funny, or?) between 864MB and 840MB and I see the LED activity on the USB-stick... . I see an increase in CPU-time of "nilfs_clearnerd" and everything still blinking but no changes to the free space... - "lscp" still shows all old checkpoints (never did any manual ones). It started blinking on the 65th minute of England-Algeria and it's still blinking at the 81st minute......CPU-time of "nilfs_clearnerd" increasing..... no differences with "lscp" nor "df -h".... boring.

As an experiment I will now unmount the partition to see if that happens immediately......yes - unmounted.

Now I will remount it....done. "df -h" now shows 1.2GB free space (?)....waiting (as the USB-stick is blinking)... . At 84th of Englang-Algeria it shows 1.5GB free...... at 85th it shows 1.7GB free and it stopped blinking - and I see that the old checkpoints have vanished.

Well, quite a funny FS. I think I'll try to use it.

Thanks guys for your posts!!
Just go on posting answers if you don't agree with anything or have anything more to add!!!