df reports wrong results
 

I had posted the note below on the Red Hat News Group but no one replied. Since that time I have increased available space but am still missing ~3 gigabytes. The server is still running at 90% used but df reports:

df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/ida/c0d0p2 67709088 57431780 6837868 90% /

Where can I look for the missing space?


The other evening I tried some new SQL queries via PlainBlack's WebGUI where I was trying to left join two tables. Apparently something was wrong with the query and it never completed because it exhausted disk space on the server. I have a Proliant 6500 quad Xeon 500 with 65 GB total disk. In /tmp I found some large MySQL temp files which I deleted with rm -f <filenames>. The files are gone but I'm still left with almost no disk space. I am receiving hourly warnings like below. Note how it states 61G of 65G used but only 821M remaining. I'm missing 3.1 gigabytes somehow. Can someone tell me where I might look to remove any other temporary files which may have been created by mysqld?

Oh, I did try emptying the trash can on the GUI and I rebooted the server.

/dev/ida/c0d0p2 (/) is 99% full -- 61G of 65G used, 821M remain

maybe you should do this

touch /forcefsck
reboot

This forces a fsck at the begining of the next boot. You may have many orphan inodes hanging around. That sometimes happens when you fill your disk up.

Thanks for your reply.

I created the /forcefsck and rebooted. The file system check started, ran to 100%, said there were errors and asked me for the root password to run manual maintenance. I ran e2fsck on each partition but it said no errors were found. I rebooted but the /forcefsck was still there which started the file system check again. This time it completed without error and the system booted normally (from that point on). When I checked the /forcefsck had been removed.

Unfortunately I'm still sitting with incorrect results. The way I calculate it the system has 86.5% in use, not 90%. Available should be 10107784, not 6668344.

Any other suggestions?

[root@vrod root]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/ida/c0d0p2 67709088 57601304 6668344 90% /

Maybe sql dropped big temporary files in you /tmp, /usr/tmp or /var/tmp

http://dev.mysql.com/doc/refman/5.0/...ary-files.html

I assume that this is either an ext2 or ext3 filesystem. By default 5% is reserved for root's use which calculates to ~3.2GB in your case. This space does not show up in the results of df so it appears that it is missing. Reserved space allow root to access the filesystem in case it becomes full and to reduce fragmentation. You can use tune2fs to reduce the amount of reserved space.

Do you know how much space was in use prior to running the queries?