The ext2/ext3 filesystem automatically allocate blocks when you write a sparse file, but when I no longer want some blocks of them, I found no ways to do it. It feels like using malloc() without free().
Is it possible to "free" some blocks of a sparse file? If it is, how?
Don't tell me to copy it to a new file. It's too boring and needs a lot of disk space.

Good question, I can imagine situations where it would be useful.

I don't think it is possible in ext2/ext3. Because the suspended allocation is an optimization rather than part of the posix filesystem 'model' that the api describes, it doesn't necessarily have to be accessible, and any posix file system that provides access must do so through flags to the system calls or non-standard extensions.

In the XFS they provide access to freeing sections through xfsctl, using the XFS_IOC_RESVSP and XFS_IOC_UNRESVSP flags.

And in some Unix systems eg SVr4 you could use 'fcntl' with the FREESP flag.

This doesn't leave you with good options. Copying the file is an expensive operation if there is a lot of data (and as you say, boring).

You could implement your own 'block' layer on top of an ext file (with a block index such that when a block becomes empty, it can be copy-swapped with the last block in the file so that 'truncate' can be used to deallocate it. But this would be like writing your own "malloc/free" on top of the file system, a lot of extra work to gain the feature you want.

Thank you, neonsignal, although I didn't find a 'fcntl' with FREESP support on my system.

Your advice of creating my own 'block layer' seems interesting. However, a problem still exists:
After swapping the blocks, the content of the file is changed. So all other data related to the swapped tail block must be modified. That's terrible...

Anyway I won't really try to implement it. Just for discussing.

When I was talking about a block index over the top, everything has to go through that index: read, write, seek, etc. So swapping blocks underneath just means swapping the index pointers to match. This is transparent to the program, as long as it uses your interface and does not directly access the file.

But you end up reimplementing the whole interface over the top, just so you can gain one additional feature.

Oh, sorry, I misunderstood it.

Then that seems to be a huge project.
You are right, a lot of extra work. That's not worthy.

It can't be hard for the filesystem to implement such a useful feature. Since it's already working in that way, all needed is to provide an interface. But nobody does...

Oh, good! You're volunteering, then?

Don't think so highly of me. I am far far away from that level. I just expressed some of my feeling...

I suspect WJE's point is that it is not as easy as it might seem. It might be reasonably trivial in a file system that only supports a single thread and has no performance constraints. But in a complex multi-user system you have to consider a lot of issues like race conditions, journalling, fragmentation, and so on.

Your best chance of getting the feature is to make a request to those developing new filesystems so that they can gauge how much demand there is for this feature. There was work on a FA_DEALLOCATE flag in ext4, but I don't know the status of this (I haven't used ext4 yet). It is unlikely to be added to older file systems such as ext2, because the possible consequence of breaking existing applications far exceeds any advantage.

I can buy a 1.5TB hard-drive for about $200 at a local department store, so...

I had to admit I sometimes do make mistakes, thinking things too simple. At that time I was thinking that NTFS had implemented a similar feature for a long time, so it couldn't be so challenging. Now it seems too optimistic.

I upgraded my system last month and I have been using ext4 for a few weeks. But I have no idea of the FA_DEALLOCATE flag. At many sites they say it's used with the 'fallocate()' function. However, the document said there's only one flag to use with it: 'FALLOC_FL_KEEP_SIZE'. I even searched the headers of my kernel(2.6.28). Still, nothing was found. I doubt whether it's still in their plan.

Thanks for all the ideas above!

Have a look at FUSE: http://fuse.sourceforge.net/ .