LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 03-13-2007, 05:39 AM   #1
jovie
Member
 
Registered: May 2006
Posts: 54

Rep: Reputation: 15
Disk space - du and df discrepency


Hi

Can someone explain this to me?

I started off with
Code:
[root@zs]# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/hda6              48G   45G     0 100% /var
So I did some investigating and found these three files
6.6G /var/log/httpd/error_log.1
7.3G /var/log/httpd/error_log.2
6.7G /var/log/httpd/ssl_error_log.2

I had a look in them, saw which php was causing the trouble, fixed it and deleted the log files. Then I had...
Code:
[root@zeldia tmp]# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/hda6              48G   25G   21G  55% /var
But whithin about an hour the used space (according to df) had gone up to 39G, the newly used 14G were not in the log files, infact I can't find them anywhere and du is reporting only 27G used. So why the descrepancy? Where have my 14Gs gone?

Code:
[root@zs]# df -h /var
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda6              48G   39G  6.0G  87% /var
[root@zs]# du -sh /var
27G     /var
[root@zs]# du -sh /var/*
12K     /var/account
22M     /var/cache
16K     /var/crash
8.0K    /var/cvs
28K     /var/db
16K     /var/empty
28K     /var/gdm
146M    /var/lib
8.0K    /var/local
124K    /var/lock
2.9G    /var/log
20K     /var/lost+found
4.0K    /var/mail
80K     /var/named
8.0K    /var/nis
8.0K    /var/opt
8.0K    /var/preserve
8.0K    /var/racoon
288K    /var/run
1.6M    /var/spool
8.0K    /var/tmp
8.0K    /var/tux
24G     /var/www
24K     /var/yp
 
Old 03-13-2007, 06:04 AM   #2
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
This is a guess. Look at the output of lsof. If a program has a lock on one of the files, it might not be dropped until that program releases the lock.
 
Old 03-13-2007, 10:45 AM   #3
jovie
Member
 
Registered: May 2006
Posts: 54

Original Poster
Rep: Reputation: 15
Thanks for the suggestion but no significantly large files show up in the list.

I'm now back to ...
Code:
[root@zs]# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/hda6              48G   45G     0 100% /var
[root@zs]# du -sh /var
27G     /var
So I cat'd a file into /var until I got the 'cat: write error: No space left on device' message. The file was 2.5G when it stopped writing - conclusion df is reporting the correct amount of used space but what ever is using up my free space (21G in a matter of hours) is not recognized by du. And that's the problem, if I can't find out what's using it how can I stop it?

These figures haven't changed since my first post but 6G of free space on /var have gone...

Code:
[root@zs]# du -sh /var/*
12K     /var/account
22M     /var/cache
16K     /var/crash
8.0K    /var/cvs
28K     /var/db
16K     /var/empty
28K     /var/gdm
146M    /var/lib
8.0K    /var/local
124K    /var/lock
2.9G    /var/log
20K     /var/lost+found
4.0K    /var/mail
80K     /var/named
8.0K    /var/nis
8.0K    /var/opt
8.0K    /var/preserve
8.0K    /var/racoon
288K    /var/run
1.6M    /var/spool
8.0K    /var/tmp
8.0K    /var/tux
24G     /var/www
24K     /var/yp
 
Old 03-14-2007, 04:23 AM   #4
win32sux
Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
hi, just wanted to post some new info, posted by jovie in the Linux - Security forum, related to this (the thread was closed for being essentially the same issue):
Quote:
Originally Posted by jovie
Could someone tell me if this behavior is like to be as a result of a hack and if so is there a better way of dealing with it than reformatting the hard drives and starting again with better security?

Symptoms
Disapearing disk space from the /var partition only. First noticed about 36hrs ago
Code:
[root@zs]# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/hda6              48G   45G     0 100% /var
[root@zs]# du -sh /var
27G     /var
[...]

while /var/log/http/access_log was still being up dated /var/log/http/error_log last changed ~36 hrs ago and I would have expected a few php warnings to appear in there since then.
ssh connections refused, first noticed about 12hrs ago.
No unrecognized processes were seen running, no malicious damage to web pages and nothing unusual on any other partition, just /var.

I disconnect it from the network about 12 hrs ago so now we have no web server running. There are the usual failed logins in /var/log/secure but the successful ones (that remain in the log) are accounted for.

I read the first thing to do is make sure I'm in control of the machine. How can I do that if I don't know where the disk space went and what its being used for?

The data are backed up so a rebuild is not out of the question but I'd rather not do it if this has some other cause that I can fix.

Thanks for your opinions.

Last edited by win32sux; 03-14-2007 at 04:38 AM.
 
Old 04-06-2007, 04:46 AM   #5
jtmoon
LQ Newbie
 
Registered: Jun 2006
Location: WA, USA
Distribution: Fedora Core 10
Posts: 15

Rep: Reputation: 5
Question df reports plenty of space yet I get 'no space left' errors

Hi,
I'm having similar problems.
According to df I have 77 Gigabytes of space.
However, all attempts to create a file fail with the error 'no space left on device'.
These examples are done as user root.

$ df -Tah
Filesystem Type Size Used Avail Use% Mounted on
...
/dev/hdh ext3 466G 367G 77G 83% /media/.Backup


and here's the error (it's consistent no matter which user tries to create a file).
$ touch /media/.Backup/foo
touch: cannot touch `/media/.Backup/foo': No space left on device


Permissions are set to allow rwx by everyone (for the sake of this example).
If I remove some files from /media/.Backup then the call to touch /media/.Backup/foo succeeds.

Here's some more information about the disk and filesystem.
$ parted -s /dev/hdh print
Model: Maxtor 6H500R0 (ide)
Disk /dev/hdh: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Number Start End Size File system Flags
1 0.00B 500GB 500GB ext3

$ tune2fs -l /dev/hdh
tune2fs 1.38 (30-Jun-2005)
Filesystem volume name: MAXTOR500GB
...
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 477056
Block count: 122096646
Reserved block count: 6104832
Free blocks: 26151205
Free inodes: 4
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 1024
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 128
Inode blocks per group: 4
...
Mount count: 20
Maximum mount count: 10
Last checked: Thu Nov 16 06:11:43 2006
Check interval: 1728000 (2 weeks, 6 days)
Next check after: Wed Dec 6 06:11:43 2006
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: tea
...
Journal backup: inode blocks


I also tried lsof to list all open files. There are no files open on /media/.Backup
My best idea is there is 77GB of filesystem overhead (superblocks,journal). But I would guess any smart space reporting tool would only report the amount of possible space since users aren't interested in disk space they can never use (most of the time)...

Any ideas on why there is plenty of free space reported yet 'no space' errors?

-J_Tom_Moon_79

Last edited by jtmoon; 04-06-2007 at 04:51 AM.
 
Old 04-10-2007, 02:46 AM   #6
jovie
Member
 
Registered: May 2006
Posts: 54

Original Poster
Rep: Reputation: 15
Sorry I should have posted the solution when I found it. jschiwal was on the right lines I just didn't understand the implications.

I'd deleted the log file used by apache but didn't restart apache. Apache had a lock on the file and kept using it. Thus 'du' considered the file deleted and didn't count the space but 'df' considered the file in use and did count the space. Every time I made free space on the partition apache used it to write to the deleted log file which 'du' wasn't reading so it appeared that the space was mysteriously disappearing.

If you know which service you've been messing with restart it or else you could reboot the machine.
 
  


Reply

Tags
df, ext3, filesystem, freespace, parted


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Cedega "Disk space required for the installation exceeds available disk space" Solved Spewdemon LinuxQuestions.org Member Success Stories 1 10-18-2007 06:19 PM
rsync size discrepency kav Linux - Software 11 11-15-2006 03:27 PM
No DISK SPACE NOW! PLEASE HELP king of penguin Linux - Hardware 3 02-17-2005 03:41 PM
3Gb of disk space lost! Disk space problem or mother board conflicts with HDD Mistreated Linux - Hardware 4 12-06-2004 03:58 PM
Disk space wastage 73 GB Hard disk rajgopalhg Linux - Hardware 2 10-18-2002 03:41 PM


All times are GMT -5. The time now is 02:19 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration