LinuxQuestions.org - Commands fail with disk full messages, but df says I've got plenty of space

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Commands fail with disk full messages, but df says I've got plenty of space (https://www.linuxquestions.org/questions/linux-newbie-8/commands-fail-with-disk-full-messages-but-df-says-ive-got-plenty-of-space-788035/)

Commands fail with disk full messages, but df says I've got plenty of space

I've been getting error messages indicating my disk might be full, but I don't really think it is. It is so bad that I could not log on via Gnome, but fortunately I was able SSH into my box. Once in, however, I can't even create a directory!

So I did the df command:

Code:

vree:~/tmp$ df -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/sdb1            132G  57G  69G  46% /

tmpfs                505M    0  505M  0% /lib/init/rw

varrun                505M  124K  505M  1% /var/run

varlock              505M    0  505M  0% /var/lock

udev                  505M  2.7M  502M  1% /dev

tmpfs                505M  24K  505M  1% /dev/shm

lrm                  505M  2.0M  503M  1% /lib/modules/2.6.27-11-generic/volatile

Hmm... Doesn't look full to me, as df reports 69G available! Or am I confused?

Any ideas?

TIA,
-Bob

fsck the partitions

Get full info on the relevant partitions incl inodes

ext4?

Exactly what are the error messages saying? Also, just for kicks and giggles, post the results of:
fdisk -l
Is this a new install? What distro are you using?

Also, post the output of mount and df -i.

You may have a read-only filesystem, which the fsck could probably evaluate and repair. (Likely cause.) Or... you could be out of inodes. (Unlikely.)

Definitely post the exact error msgs. You can also check the inodes

df -i

your df command shows / mounted on /dev/sdb1. What is on /dev/sda?

What "error" are you receiving

Quote:

Originally Posted by rsgrimes (Post 3858007)

I've been getting error messages indicating my disk might be full

What is the error you are getting?

Wow! Thanks for all the responses! Here are my replies...

1. Sorry I wasn't clear on ther actual error messages. Here is one example; basically, I get the same "No space left on device" regardless of what I try:

Code:

mkdir: cannot create directory `testdir': No space left on device

2. It is an old installation (Ubuntu 8.10), about a year, pretty much just used as a Subversion and Trac server...

3. After rebooting, I was able to log on in a fail-safe terminal, and by deleting a few files, I was able to do the following, as suggested by several posts I've found:

Code:

sudo touch /forcefsck

Rebooting again, I still have the problem.

4. fdisk -l and mount results

Code:

vree:~$ fdisk -l

Cannot open /dev/sda

Cannot open /dev/sdb



rsg@#vree

vree:~$ mount

/dev/sdb1 on / type ext3 (rw,relatime,errors=remount-ro)

tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)

/proc on /proc type proc (rw,noexec,nosuid,nodev)

sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)

varrun on /var/run type tmpfs (rw,nosuid,mode=0755)

varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)

udev on /dev type tmpfs (rw,mode=0755)

tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)

devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)

fusectl on /sys/fs/fuse/connections type fusectl (rw)

lrm on /lib/modules/2.6.27-11-generic/volatile type tmpfs (rw,mode=755)

securityfs on /sys/kernel/security type securityfs (rw)

rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

nfsd on /proc/fs/nfsd type nfsd (rw)

binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)

5. df -i results

Code:

vree:~$ df -i

Filesystem            Inodes  IUsed  IFree IUse% Mounted on

/dev/sdb1            8773632 8773616      16  100% /

tmpfs                129117      4  129113    1% /lib/init/rw

varrun                129117      59  129058    1% /var/run

varlock              129117      3  129114    1% /var/lock

udev                  129117    4941  124176    4% /dev

tmpfs                129117      1  129116    1% /dev/shm

lrm                  129117      17  129100    1% /lib/modules/2.6.27-11-generic/volatile

So it's an INode problem! Okay, I know what INodes are, but I don't know how to manage them. FWIW, I wouldn't think I have that many files, so I would welcome suggestions on the usual suspects, you know, where to look, what generally causes lots of extraneous inodes, etc.

Thanks!

RESOLVED Commands fail with disk full messages, but df says I've got plenty of space

I found my problem! I have been using BackInTime (a poor man's Time Machine) on my workstation, and it was taking snapshots on the server; I haven't constrained it a bit, and so it has generated quite a few files - I'm currently counting them, and it's up to almost 4 million - and counting!

That's what you get with "set and forget" tools! (No, that's not a criticism of BackInTime!).

Thanks,
-Bob

An inode is a data structure that is associated with each file. With ext3 the max number of inodes is set when the filesystem is created and the only way to increase the number is to reformat. You can not manage them i.e if you run out of inodes you can not create more files. There are other filesystems that can dynamically allocate inodes like jfs, xfs or ext4.

Quote:

Originally Posted by michaelk (Post 3859094)

An inode is a data structure that is associated with each file.

Yeah, this I know, but...

Quote:

Originally Posted by michaelk (Post 3859094)

With ext3 the max number of inodes is set when the filesystem is created and the only way to increase the number is to reformat. You can not manage them i.e if you run out of inodes you can not create more files. There are other filesystems that can dynamically allocate inodes like jfs, xfs or ext4.

...this I didn't - thanks for the info!

Cheers,
-Bob

Also, there's not much point in backing up to the same disk; what happens if the disk goes bad?
:)

Quote:

Originally Posted by chrism01 (Post 3859692)

Also, there's not much point in backing up to the same disk; what happens if the disk goes bad?
:)

No, it was another computer backingup to this computer - of course, what you say makes sense!

Quote:

Originally Posted by rsgrimes (Post 3859088)

I haven't constrained it a bit, and so it has generated quite a few files - I'm currently counting them, and it's up to almost 4 million - and counting!

That's what you get with "set and forget" tools! (No, that's not a criticism of BackInTime!).

Not being familiar with that tool, I'm wonder whether it is possible to configure it so that you can be emailed a summary of what was backed up/saved? If so, maybe you could put the number of files acted on right into the "Subject:" line you'd get that "In Your Face" reminder to keep track of things.

Later...

--
Rick

You can add a cron job to check the disk space, both filespace & inode cnt and email you if you exceed specified values.
You can even use quotas, eg soft limit at warning level, hard limit at full disk.
Quotas have both diskspace and inode settings.