File deletion not recovering the space on the disk.
 

I have done rm -rf nohup.out file from the linux server in-order to make some disk-space because of lack of space in the server. But I surprised by seeing the following,

ll -h nohup.out
-rw------- 1 user1:user1 54G Dec 13 02:29 nohup.out

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 226G 219G 0 100% /
/dev/sda1 99M 19M 76M 20% /boot
tmpfs 2G 0 2G 0% /dev/shm

rm -rf nohup.out

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 226G 219G 0 100% /
/dev/sda1 99M 19M 76M 20% /boot
tmpfs 2G 0 2G 0% /dev/shm

but file nohup.out got deleted.

I later able to make space in the disk by deleting other files. Is ll -h was showing wrong? Any one can explain why it has been happened.

The scenario you describe is possible when a running process is still writing to the nohup.out file. Even though the file is not visible, it is still kept open and the space it occupies isn't freed until the process writing to it stops.

You might be able to find which process is keeping the file open by running (you might need root privileges):

If stopping the process is not an option, you can try just emptying the file:

This will free up the space. The process will still write to the file, so you have to expect that the file will start growing again.

@Jostekk: I'm not sure you can do that if the nohup.out file has already been deleted by the user. I suspect that > nohup.out will create a new file which isn't associated with the process that holds open the "original" nohup.out file.

Your solution will work if the file hasn't been deleted yet.

If you determine what process is holding the file open (lsof will show deleted files), you can read and write the file via its link in the /proc/{PID}/fd/ directory.

Yes, that will work.

I would think though that one would prefer to stop the process and restart it (with or without the nohup.out file). After a while one might forget that the actual file was deleted and the token used in /proc/PID/fd/ needs to be used.