LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 01-12-2014, 02:58 PM   #1
kubuntu-man
Member
 
Registered: Oct 2011
Posts: 36

Rep: Reputation: Disabled
After deleting lots of large files, free space only increases partially


Hi,

I just have deleted 71 GB of files.
Free Space before: 117 GB
Free space after: 126 GB

So, instead of having 71 GB of additional free space, I only have 9 GB. I have double-checked that I really have deleted 71 GB and free space increased only 9 GB.

I also tried to sync, but no effect.

This is not the first time it happens. When this happens, I can free the space by umounting and then re-mounting the filesystem. In these cases, umounting takes several seconds instead of being immediate.

Nowadays, I can not easily umount and remount the filesystem, because it is permanently busy from my video recorder, the owncloud server for my family, and a few more services I did not have so far. And I don't want to get up at 3 o'clock in the night just to umount and remount :-)
(no, the 'at' utility won't do because one of the services needs a manual state check to find a good moment when it can be shut down)

So far, I only noticed this behaviour when deleting a large amount of data. In the other hand, I am not sure if it happens on other cases too and the difference is just not noteable.

The filesystem was created with 0% reserved for root (mkfs -m 0). According to fsck -f, the filesystem is not corrupted, and according to S.M.A.R.T. diagnostics extended test, the hardware is also OK.

So here are my 2 questions:
1. What is happening here ? Why is space freed at umount instead of at delete ?
2. Is there something I can do to trigger freeing the space without umount ?
 
Old 01-12-2014, 05:10 PM   #2
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
It sounds like you're deleting files that are currently open by running processes. In this case, you're simply deleting the reference to the file, but the space won't be freed until the process is stopped or restarted.
 
Old 01-12-2014, 10:10 PM   #3
sag47
Senior Member
 
Registered: Sep 2009
Location: Raleigh, NC
Distribution: Ubuntu, PopOS, Raspbian
Posts: 1,899
Blog Entries: 36

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477
Quote:
Originally Posted by suicidaleggroll View Post
It sounds like you're deleting files that are currently open by running processes. In this case, you're simply deleting the reference to the file, but the space won't be freed until the process is stopped or restarted.
I agree with this assessment.

Code:
lsof | grep 'deleted'
 
Old 01-12-2014, 11:00 PM   #4
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
also did you do this via the GUI or via CLI?
 
Old 01-13-2014, 01:46 PM   #5
kubuntu-man
Member
 
Registered: Oct 2011
Posts: 36

Original Poster
Rep: Reputation: Disabled
It does not matter if I delete the files in the GUI or in the terminal. I know about the recycle bin ;-)

I forgot to mention in my question that I also checked that none of the files are open.
Not by any program, and I also stopped the NFS server (no SAMBA server installed). And even if I had not checked, I think it's not very likely to have open files with a total of 62 GB ;-)

But indeed, I have copied the files to another PC via NFS before deleting them. If the files would have still been open after stopping the NFS server, this would be a bug, wouldn't it ?
In the other hand ... why does umount work without complaining about "filesystem is busy", just take unusually long ? This means there must be some sort of cleanup going on.

But I have noticed something else:
Yesterday, I could not do the umount-remount cycle because of the runnning services (my statemet about "umount is working" was from the previous cases when the problem occured).
This morning, the space was freed - without any user action. So it looks like the space is freed with some delay. Maybe it's the same cleanup I suspect to be going on during umount.
 
Old 01-13-2014, 05:45 PM   #6
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,976

Rep: Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623
lleb asked a good question, he could not have known.

Just for grins boot to a live cd and see what is says.
 
Old 01-13-2014, 09:05 PM   #7
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,657
Blog Entries: 4

Rep: Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938Reputation: 3938
I predict that it would be more accurate to say that there is, indeed quite a bit of "background activity going on," which blocked the umount request until it could be completed.

You are, after all, deleting "71 gigabytes worth of" disk files here ... and simple reason says that it's going to take quite a bit of time to readjust all of the relevant data structures when you do that. While the filesystem, quite graciously, might not oblige any given user-process to wait around while it (the filesystem) "gets its paperwork in order," it certainly would (have to ...) block any request to un-mount the device.
 
Old 01-13-2014, 09:14 PM   #8
docbop
LQ Newbie
 
Registered: Sep 2009
Location: Woodshed, CA
Distribution: Debian
Posts: 11

Rep: Reputation: 0
Here's what an old Linux Journal article said.....

>>>>
rm is the command used, in Linux terminology, to unlink a file. What this means is that the directory entry for the file is removed. A side effect (and the effect that we generally expect) is that the file is deleted. But this may not be the case.

The Linux file system makes it possible for a file to have more than one name or directory entry. The ln command allows you to create these additional names or links. If these links are hard links, links created with the ln command without the -s option, you have a file that can be accessed by these multiple names.

By using the rm command on one of these names, you only delete the name, not the actual file. When the last name pointing to the file is removed, the file is finally removed.

<<<<
 
Old 01-14-2014, 01:08 PM   #9
kubuntu-man
Member
 
Registered: Oct 2011
Posts: 36

Original Poster
Rep: Reputation: Disabled
@jefro
I did not mean to disrespect lleb, I just wanted to state I already have thought about this.

As the umount, re-mount cycle (most times with a fsck in between) always helped, and the space was freed overnight this time, I would say that booting a live CD will probably not give more information. Additionally, the PC needs to run nearly 24/7 because of all the services, and it is always difficult to find a good moment to do this. Except, maybe, 3 o'clock in the night ...

@sundialsvcs
When deleting so much data, I pay attention to choose a moment when the disk is not (or not very) busy. Formerly, I ensured that the disk was completely idle.
Since I have all my new services runnning, I can not totally ensure this any more. But still, I choose a moment with minimal possible I/O load.
But indeed, I do not know if the filesystem driver still chooses to handle user processes' I/O with higher priority than the "internal paperwork".
BTW: the umount was never really blocked (like it is when having open files), it was only very slow.

@docbop
Yes, I know about the relation between directory entry and inodes. Also that an inode can have multiple hard links.
But if multiple hard links was the problem, neither the umount, re-mount cycle nor waiting over night would have helped. I have no services, cron-jobs and scripts that perform 'rm' in by /bigdata file system.

Up to now, the "internal paperwork" explanation of sundialsvcs appears the most reasonable one to me.
 
Old 01-20-2014, 04:25 PM   #10
selfprogrammed
Member
 
Registered: Jan 2010
Location: Minnesota, USA
Distribution: Slackware 13.37, 14.2, 15.0
Posts: 635

Rep: Reputation: 154Reputation: 154
This discussion has gone on as if your FS was ext2.
What if the FS is ext4 with the journalling enabled.
Ext4 also has Delayed Allocation. You can do your own web search, I got 94000 hits.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Deleting files in Ubuntu doesn't free up space on Linux partition. kechlion Linux - General 7 12-28-2009 02:57 PM
Deleting files does not increase amount of free space? frznchckn Linux - General 6 12-19-2008 06:23 AM
Find Large files or How to free up space. sipsipi Linux - Software 3 02-25-2006 02:34 PM
No free Space even after deleting files (ext3) jeffUK Linux - General 6 08-21-2005 07:13 PM
no free space on / after deleting files wishbone42 Linux - General 3 11-21-2004 12:09 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 03:26 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration