LinuxQuestions.org - [SOLVED] Root system directory is full

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Root system directory is full (https://www.linuxquestions.org/questions/linux-newbie-8/root-system-directory-is-full-911486/)

Root system directory is full

Hi all,

Hope this is the right place to post this silly question. I was asked to check the available space of a server which is running Fedora Core release 3 (Heidelberg)
LSB_VERSION="1.3" and when I type df -h command, it shows the below info:

Code:

Filesystem            Size  Used Avail Use% Mounted on

/dev/hda1            4.0G  2.4G  1.5G  62% /

/dev/md2              58G  43G  13G  77% /name/test1

/dev/md1              58G  54G  1.3G  98% /name/test2

/dev/md3              58G  33G  23G  59% /name/test3

/dev/md0              48G  37G  8.0G  83% /name/test4

none                  506M    0  506M  0% /dev/shm

/dev/hda3            4.0G  41M  3.8G  2% /tmp

/dev/hda4              10G  2.2G  7.4G  23% /usr

Could someone please tell me how to clean this up and what files do I need to delete? Sorry if this is a silly question, I'm very new to this.

Thanks in advance

Hi!

It seems that you still have 1.5GB available on the partition your root is mounted on:

/dev/hda1 4.0G 2.4G 1.5G 62% /

Two good places to look are /tmp and /var/log (although your /tmp doesn't look all the big). Note that /var/log may vary from distribution to distribution -- look for the logs if you don't have /var/log. If you do not have logrotate running, your system logs will be mammoth -- and, yes, you can simply empty them with

Code:

cat > /var/log/logname                (the name of a Really Big Log file)

Ctrl-D

Although it doesn't appear that your user directories are in the root, look at the sizes of user directories, probably in /usr. Users do tend to gobble space and there may be some Really Large Files sitting in their directories; you can use find to locate them:

Code:

cd /usr

find . -type f -size M10

that will give a list of files larger than 10 MB (and then you go holler are folks, suggesting politely that the gzip utility is quite useful).

Hope this helps some.

Thanks tronayne and Bfresh for your replies.

Here is everything in my /var/log directory:

Code:

total 4056

drwx--S---  2 amanda disk        4096 Jan 29  2010 amanda

-rwxr-xr-x  1 root  root      15013 May 29  2005 anaconda.log

-rwxr-xr-x  1 root  root      31044 May 29  2005 anaconda.syslog

-rw-------  1 root  root        2570 Nov  2 10:40 boot.log

-rw-------  1 root  root        195 Oct 30 04:02 boot.log.1

-rw-------  1 root  root        2615 Oct 23 04:02 boot.log.2

-rw-------  1 root  root        2702 Oct 16 04:02 boot.log.3

-rw-------  1 root  root      11494 Oct  9 04:03 boot.log.4

-rw-------  1 root  root      631967 Nov  2 13:19 cron

drwxr-xr-x  2 lp    sys        4096 Oct 30 04:02 cups

-rw-------  1 root  root        157 Feb 25  2010 DEADJOE

-rw-r--r--  1 root  root      15283 Sep 29 22:29 dmesg

drwxr-xr-x  2 root  root        4096 May 29  2005 gdm

drwx------  2 root  root        4096 Oct  7 10:51 httpd

drwx------  2 root  root        4096 May 29  2005 iptraf

-r--------  1 root  root    19136220 Nov  2 13:19 lastlog

drwxr-xr-x  2 root  root        4096 May 29  2005 mail

-rw-------  1 root  root    1198262 Nov  2 13:19 maillog

drwxrwsr-x  2 root  mailman    4096 Oct 30 04:02 mailman

-rw-------  1 root  root    1269058 Nov  2 13:19 messages

-rw-r-----  1 mysql  mysql          0 Oct 30 04:02 mysqld.log

-rw-r-----  1 mysql  mysql          0 Oct 23 04:02 mysqld.log.1

-rw-r-----  1 mysql  mysql          0 Oct 16 04:02 mysqld.log.2

-rw-r-----  1 mysql  mysql          0 Oct  9 04:03 mysqld.log.3

-rw-r-----  1 mysql  mysql          0 Oct  2 04:03 mysqld.log.4

drwx------  2 root  root        4096 Nov  2  2004 ppp

-rw-r--r--  1 root  root      30820 Oct 30 04:02 prelink.log

-rw-r--r--  1 root  root      25808 Nov  2 04:02 rpmpkgs

-rw-r--r--  1 root  root      25808 Oct 29 04:02 rpmpkgs.1

-rw-r--r--  1 root  root      25808 Oct 22 04:04 rpmpkgs.2

-rw-r--r--  1 root  root      25808 Oct 15 04:02 rpmpkgs.3

-rw-r--r--  1 root  root      25808 Oct  8 04:03 rpmpkgs.4

drwxr-xr-x  2 root  root        4096 Nov  2 00:00 sa

drwx------  2 root  root      73728 Nov  2 09:14 samba

-rw-r--r--  1 root  root      144582 Jan 16  2006 scrollkeeper.log

-rw-------  1 root  root        1089 Nov  2 13:19 secure

-rw-------  1 root  root        513 Oct 28 10:28 secure.1

-rw-------  1 root  root        1518 Oct 21 12:40 secure.2

-rw-------  1 root  root        410 Oct 14 10:33 secure.3

-rw-------  1 root  root        1445 Oct  7 10:27 secure.4

-rw-------  1 root  root          0 Oct 30 04:02 spooler

-rw-------  1 root  root          0 Oct 23 04:02 spooler.1

-rw-------  1 root  root          0 Oct 16 04:02 spooler.2

-rw-------  1 root  root          0 Oct  9 04:03 spooler.3

-rw-------  1 root  root          0 Oct  2 04:03 spooler.4

drwxr-x---  2 squid  squid      4096 Jun 26  2006 squid

-rw-r--r--  1 root  root          0 Oct 30 04:02 up2date

-rw-r--r--  1 root  root          0 Oct 23 04:02 up2date.1

-rw-r--r--  1 root  root          0 Oct 16 04:02 up2date.2

-rw-r--r--  1 root  root          0 Oct  9 04:03 up2date.3

-rw-r--r--  1 root  root          0 Oct  2 04:03 up2date.4

drwxr-xr-x  2 root  root        4096 Oct  5  2004 vbox

-rw-rw-r--  1 root  utmp        4992 Nov  2 13:19 wtmp

-rw-rw-r--  1 root  utmp      27648 Oct 31 10:11 wtmp.1

-rw-r--r--  1 root  users      42945 Aug  6  2008 Xorg.0.log

-rw-r--r--  1 root  root      43021 Mar 16  2008 Xorg.0.log.old

-rw-r--r--  1 root  root        3247 Aug 24  2006 yum.log

So if I understand correctly, I can just delete the .log files? Active files should not be deleted just truncate it if I'm not mistaken (I was doing a rm -rf for the .1 .2 .3,... files).

I found these under /usr:

Code:

drwxr-xr-x    2 root root 69632 Jan 27  2010 bin

drwxr-xr-x    2 root root  4096 Aug 12  2004 etc

drwxr-xr-x    2 root root  4096 Aug 12  2004 games

drwxr-xr-x  110 root root 20480 Jan 16  2006 include

drwxr-xr-x    6 root root  4096 Jun 29  2005 kerberos

drwxr-xr-x  115 root root 69632 Oct 22 04:04 lib

drwxr-xr-x  14 root root  4096 Jan 17  2006 libexec

drwxr-xr-x  12 root root  4096 May  5  2008 local

drwx------    2 root root 16384 May 29  2005 lost+found

drwxr-xr-x    2 root root 20480 Jan 27  2010 sbin

drwxr-xr-x  238 root root 12288 Oct  5  2005 share

drwxr-xr-x    3 root root  4096 May 29  2005 src

lrwxrwxrwx    1 root root    10 May 29  2005 tmp -> ../var/tmp

drwxr-xr-x    7 root root  4096 Oct  6  2004 X11R6

But I found a directory /depot/server1/home in which I see directories corresponding to each user. When I ran that command you typed, it was complaining about the format:

Code:

[root@server home]# find . -type f -size 10M

find: invalid -size type `M'

Many users didn't have any files in those directories though, maybe I'm looking in the wrong place?

Thanks in advance for your help

Quote:

So if I understand correctly, I can just delete the .log files? Active files should not be deleted just truncate it if I'm not mistaken (I was doing a rm -rf for the .1 .2 .3,... files).

After a week or two, yes you can delete the .1, .2., .3, etc. files if you're short on space. You may however want to add one or two to logrotate -- there are some that are being rotated, those with .1, .2, .3, etc. extensions. The messages log is a good candidate (it's just under 20 MB right now) that is not being rotated.

Go look in /etc. There should be a file, logrotate.conf, and a directory, logrotate.d. If you get into /etc/logrotate.d there should be one or more files that are used to rotate the logs every week to keep the individual sizes down. One of them may be named syslog and you can change it to add /var/log/messages so that log file will get rotated.

Bear in mind that your system may vary; see man logrotate to find out the location for these files if you do not immediately see a /etc/logroate.d directory.

Here's what the file /etc/logrorate.d/syslog looks like on my system:

Code:

/var/log/cron /var/log/debug /var/log/maillog /var/log/messages /var/log/secure /var/log/spooler /var/log/syslog {

    sharedscripts

    postrotate

        /bin/kill -HUP `cat /var/run/syslogd.pid 2>/dev/null` 2>/dev/null || true

    endscript

}

Again, yours may vary and you might need to look around; if your file looks like that, you could simply insert /var/log/messages in the list as above and over the coming weekend it should get rotated and compressed. Don't just copy the above to your system, work with what your system has.

maillog is another good candidate for rotation.

I would leave the files and directories in /usr alone.

Quote:

But I found a directory /depot/server1/home in which I see directories corresponding to each user. When I ran that command you typed, it was complaining about the format:

Looks like /depot/server/home is where your user directories are? Different, that.

It seems that your find utility just doesn't know about the -size 10M argument, so let's kill the same bird with a different stone. Take a look at the manual page for find, particularly the section on size.

You probably want to find files in /depot/server1/home that are bigger that 1 GB:

Code:

find /depot/server1/home -type f -size +10485760c -exec ls -l {} \;

that will show you files larger than 10 MB and their actual size (you may get a large list) to discuss with users. If you add an additional zero (+104857600c), you'll get files larger than 100 MB.

Hope this helps some.

tronayne,

Thanks much for your very helpful reply.

This is the info in my syslog:

Quote:

/var/log/messages /var/log/secure /var/log/maillog /var/log/spooler /var/log/boot.log /var/log/cron {
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
endscript
}

So looks like the messages are already a part of the rotation.

I found the users who happen to have large files on the network and contacted them all. So you are suggesting if they all clean up the space, I will have a clean root directory? This is the output of df -h:

Quote:

Filesystem Size Used Avail Use% Mounted on
/dev/hda1 4.0G 2.4G 1.5G 63% /
/dev/md2 58G 32G 23G 59% /depot/server1
/dev/md1 58G 54G 1.3G 98% /depot/server2
/dev/md3 58G 33G 23G 59% /depot/server3
/dev/md0 48G 37G 8.0G 83% /depot/server4
none 506M 0 506M 0% /dev/shm
/dev/hda3 4.0G 41M 3.8G 2% /tmp
/dev/hda4 10G 2.2G 7.4G 23% /usr

These are everything I have under the root directory:

Quote:

drwxr-xr-x 3 root root 4096 Jan 5 2011 boot
drwxr-xr-x 6 root root 4096 May 29 2005 depot
drwxr-xr-x 8 root root 6400 Sep 29 22:30 dev
drwxr-xr-x 91 root root 12288 Nov 4 09:20 etc
drwxr-xr-x 13 root root 4096 Sep 28 2010 home
drwxr-xr-x 2 root root 4096 Aug 12 2004 initrd
drwxr-xr-x 10 root root 4096 Jan 20 2010 lib
drwx------ 2 root root 16384 May 29 2005 lost+found
drwxr-xr-x 3 root root 4096 Sep 29 22:30 media
drwxr-xr-x 2 root root 4096 Mar 23 2005 misc
drwxr-xr-x 2 root root 4096 Aug 12 2004 mnt
drwxr-xr-x 3 root root 4096 Jun 16 2005 opt
dr-xr-xr-x 90 root root 0 Sep 29 18:28 proc
drwxr-x--- 21 root root 4096 Aug 19 08:15 root
drwxr-xr-x 2 root root 12288 Jan 14 2010 sbin
drwxr-xr-x 1 root root 0 Sep 29 18:28 selinux
drwxr-xr-x 3 root root 4096 Jul 24 2008 share
drwxr-xr-x 2 root root 4096 Aug 12 2004 srv
drwxr-xr-x 10 root root 0 Sep 29 18:28 sys
drwxrwxrwt 4 root root 8192 Nov 4 04:02 tmp
drwxr-xr-x 15 root root 4096 Aug 13 2009 usr
drwxr-xr-x 24 root root 4096 Jul 24 2008 var

Thanks in advance

Well, your root partition is a little on the small side -- you do have 1.5G free, but over time that ain't a whole lot of wiggle room. You may -- may -- want to consider a larger drive and copying or reinstalling your existing system to it (which is not a trivial task). I typically make the root partition at least 10G (and I do not use a separate partition for /tmp); that results in about 7.5G of disk use. However, as long as you keep an eye on free space, you can probably get away with leaving well enough alone.

It's always a good thing (always) to keep an eye on the space users gobble up -- folks do tend to get and hang on to big stuff "just in case I might need it some day" and it never hurts to monitor that. Their use has nothing to do with your root space, but the day will come that somebody's going to fill up a partition and all heck will break loose (trust me: been there, they did that more than once). Also, have a look-see at what's in /tmp -- folks may be stashing big stuff in there and let it sit there forever. Oh, yeah, teach them about the benefits of gzip.

One thing you can do that will save you a little space is look at /etc/logrotate.conf. Mine looks like this (your will probably vary):

Code:

# /etc/logrotate.conf

#

# logrotate is designed to ease administration of systems that generate large

# numbers of log files.  It allows automatic rotation, compression, removal, and

# mailing of log files.  Each log file may be handled daily, weekly, monthly, or

# when it grows too large.

#

# logrotate is normally run daily from root's crontab.

#

# For more details, see "man logrotate".



# rotate log files weekly:

weekly



# keep 4 weeks worth of backlogs:

rotate 4



# create new (empty) log files after rotating old ones:

create



# uncomment if you want to use the date as a suffix of the rotated file

#dateext



# uncomment this if you want your log files compressed:

#compress



# some packages install log rotation information in this directory:

include /etc/logrotate.d



# Rotate /var/log/wtmp:

/var/log/wtmp {

    monthly

    create 0664 root utmp

        minsize 1M

    rotate 1

}



# Rotate /var/log/btmp:

/var/log/btmp {

    monthly

    create 0600 root root

    rotate 1

}



# Note that /var/log/lastlog is not rotated.  This is intentional, and it should

# not be.  The lastlog file is a database, and is also a sparse file that takes

# up much less space on the drive than it appears.



# system-specific logs may be also be configured below:

Of particular interest are the lines

Code:

# uncomment this if you want your log files compressed:

#compress

If yours is like this, uncomment (remove the #) from compress. That will cause the rotated log files to be compressed with gzip; ain't much, but every little bit counts.

You know, disk drives are cheap -- a 500G drive for the system and another 500G drive for users is not a huge investment (and, no, you don't want to use terabyte drives yet; too new for longevity and reliability statistics to be meaningful). You now have a good statistical base to work from (your existing disk use as given by df -h) to give you a good starting place for adding or changing a drive or two.

Hope this helps some.

Hi tronayne,

Thanks again for your helpful reply. I looked at my /etc/logrotate.conf and it reads #compress, so I will remove the # sign.

Since users are using that machine all the time and I have never done anything like this, I'm not sure how I can replace the drive with a bigger one and not sure how to re-partition the existing drive. I know the person who put this box in service said you have to keep cleaning the root directory but he didn't tell me how I can do that. I constantly truncate the active logs and remove the .x files but that doesn't empty much space.

I would really appreciate it if you help me with this

Thanks

http://www.cyberciti.biz/faq/how-do-...sd-filesystem/
du : Disk usage

This shows the disk usage for all the files and directories (folders) in /var:

du -a /var | sort -n -r | head -n 10
1008372 /var
313236 /var/www
253964 /var/log
192544 /var/lib
152628 /var/spool
152508 /var/spool/squid
136524 /var/spool/squid/00
95736 /var/log/mrtg.log
74688 /var/log/squid
62544 /var/cache

You might need to sudo or su in order to see some directories.
Good luck.

Thanks tollingalong,
Here's the output I'm gettting:

1040600 /var
870180 /var/spool
862408 /var/spool/clientmqueue
99196 /var/lib
65952 /var/lib/rpm
45012 /var/lib/rpm/Packages
41332 /var/log
21484 /var/log/cups
16516 /var/www
16100 /var/lib/imap

Hi tronayne and everyone,

I have asked all ths user to move their files from the /name/test2. But the root directory % didn't go down and it even went up. Is there any way someone please help me and tell me what else to delete to empty up free space on the root directory?

This is my biggest fear that one day the root directory gets to 100% and the machine (therefore the LDAP server, DHCP server and everything else) goes down for good. Thank you in advance

Filesystem Size Used Avail Use% Mounted on
/dev/hda1 4.0G 2.7G 1.2G 70% /
/dev/md2 58G 13G 43G 23% /name/test1
/dev/md1 58G 54G 1.3G 98% /name/test2
/dev/md3 58G 33G 23G 59% /name/test3
/dev/md0 48G 37G 7.9G 83% /name/test4
none 506M 0 506M 0% /dev/shm
/dev/hda3 4.0G 41M 3.8G 2% /tmp
/dev/hda4 10G 2.2G 7.4G 23% /usr

Do the following:
du -a /name/test2 | sort -n -r | head -n 10
Please note the da -a <directory> means the directory you wish to find the largest files.

That should tell you which files are largest in the specific location you're in need.
If you cannot find enough files there is always the possibility a process is holding onto a file.

Hiya.

You do realize that the content of /name/test1 (test2, test3 and test4) have nothing to do with what's in the root partition -- they're mounted partitions and have wads of space available.

If I remember correctly, your user's home directories are in /home? That's your best candidate for relocation if that's so -- the size of user's home directories never goes down, only up and it's only a matter of time...

One candidate you have for moving /home is /dev/hda3, a 4.0G partition currently mounted as /tmp which appears to only be using 41M with 3.8G free. To consider doing that, you would want to do this (your size will be different, of course); the -sh shows a summary in human-readable form:

Code:

su -

du -sh /home

2.6G    /home

If this sounds good to you, let us know and we can walk you through how to do it; it's not difficult but has to be done step-by-step.

Bottom line, however, is that you're going to have to bite the bullet and add a drive or two and/or redo your system with larger partition sizes sooner or later. You do have time for now but the day will come when things get filled up, sorry.

Hope this helps some.

Thanks tronayne and tollingalong for your replies.

Tronayne,
I ran the df -h command again:

Code:

[root@home]# df -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/hda1            4.0G  2.7G  1.2G  71% /

/dev/md2              58G  13G  43G  23% /name/test1

/dev/md1              58G  486M  55G  1% /name/test2

/dev/md3              58G  33G  23G  59% /name/test3

/dev/md0              48G  37G  7.9G  83% /name/test4

none                  506M    0  506M  0% /dev/shm

/dev/hda3            4.0G  41M  3.8G  2% /tmp

/dev/hda4              10G  2.2G  7.4G  23% /usr

I understand that those drives are not parts of the root partition, but in /name/test2 there are all the users listed. Not sure why the root directory gets full if the users are in a different mounted drive.

The command you asked me to run for /home returned only 6.0M so I don't think it's the correct user directory:

Code:

[root@home]# du -sh /home

6.0M    /home

Where else should I be looking for large files?

Thanks in advance

I've attached a file, biggest.sh, that you can use to find... uh, big files.

Download the file then enter

Code:

make biggest

you'll have an executable utility named biggest.

Running it without arguments will show how to use it:

Code:

Usage: biggest.sh -fHh -l <nn> -s <nnn> -t <dir> -v fs\n

                  -f = follow links

                  -H = Full documentation

                  -h = Usage brief

                  -l = Displays <nn> lines

                  -s = Minimum file size is <nnn>

                  -t = Temp/work directory, <dir>

                  -v = Edit (vi) file list

                  fs = Required filesystem argument.

What you can do is

Code:

su -

biggest /

and that will show you large files throughout the file system (you can modify the minimum size with the -s option). You'll need to play with it a little to get a feel for what it's going to show you.

You really do have to realize that you're going to have to bite the bullet and work on reinstalling your system with a larger partition for root (more on the order of 10G or more) -- you don't have to do that as an emergency (hey, 71% is still 30%-ish free, eh), but you will have to do it sooner or later and it might be a good time to learn how before you hit the wall. I give root 15G and have 7G used (and /home is a mounted partition).

Hope this helps some.

Ah, nuts! Guess it didn't attach so here

Code:

#!/bin/sh

#ident  "$Id: biggest.sh,v 1.2 2011/12/03 15:16:31 trona Exp $"

#

#      Name:          $Source: /usr/local/cvsroot/utils/biggest.sh,v $

#      Version:        $Revision: 1.2 $

#      Modified:      $Date: 2011/12/03 15:16:31 $

#      Purpose:

#      Author:        Bob Orlando

#      Date:          8 Apr 1995

#      $Log: biggest.sh,v $

#      Revision 1.2  2011/12/03 15:16:31  trona

#      minor edit

#

#      Revision 1.1.1.1  2009/10/07 18:06:40  trona

#      initial installation Slackware 13.0

#

#      Revision 1.1.1.1  2009/01/29 15:03:25  trona

#      inital

#

#      Revision 1.1  2007/08/19 15:31:34  trona

#      initial installation

#

  #----------------------------------------------------------------#

  # Script_name assignment is necessary if there exists the        #

  # possibility that this process may be run by the 'at' command.  #

  # Run via 'at' and $0 simply returns '/bin/sh' or 'sh' (hardly  #

  # desirable if you run that into basename).                      #

  #----------------------------------------------------------------#

  script_name="biggest.sh"

  [ $0 = "/bin/sh" -o `dirname $0` = "." ] \

    && script_home=`pwd` || script_home=`dirname $0`

  bin=/usr/bin # Default



  #----------------------------------------------#

  # Use awk, nawk or gawk, depending on the OS.  #

  #----------------------------------------------#

  OZ=`uname -s 2> /dev/null | tr '[a-z]' '[A-Z]' 2> /dev/null`

  if  [ ."$OZ" = ."HP-UX" ]; then

    AWK=awk

  elif [ ."$OZ" = ."LINUX" ]; then

    bin_dir=/bin

    AWK=gawk

  elif [ ."$OZ" = ."SUNOS" ]; then

    AWK=nawk

  else # Unknown OS, see if there's any kind'a Awk available.

    if  [ -f $bin/gawk ]; then AWK=gawk

    elif [ -f $bin/nawk ]; then AWK=nawk

    elif [ -f $bin/awk  ]; then AWK=awk

    elif [ `expr "\`awk 2>&1\`" : 'Usage: '` -gt 0 ]; then AWK=awk

    else # This is really getting awkward :-o

        echo "Unable to locate [gn]awk program! $0 terminating." 1>&2

        exit 1 # Well behaved here

    fi

  fi





#======================================================================#

#                    L O C A L    F U N C T I O N S                    #

#                      (in alphabetical order)                        #

#----------------------------------------------------------------------#

EXIT_USAGE()

#----------------------------------------------------------------------#

{

  echo "Usage: biggest.sh -fHh -l <nn> -s <nnn> -t <dir> -v fs\n" 1>&2

  echo "                  -f = follow links"                      1>&2

  echo "                  -H = Full documentation"                1>&2

  echo "                  -h = Usage brief"                      1>&2

  echo "                  -l = Displays <nn> lines"              1>&2

  echo "                  -s = Minimum file size is <nnn>"        1>&2

  echo "                  -t = Temp/work directory, <dir>"        1>&2

  echo "                  -v = Edit (vi) file list"              1>&2

  echo "                  fs = Required filesystem argument."    1>&2

  echo ""                                                        1>&2

  exit 1

}



#----------------------------------------------------------------------#

SHOW_DOCUMENTATION() # Function documentation located at bottom.      #

#----------------------------------------------------------------------#

{

  #----------------------------------------------------------------#

  # If the following variables are not set, use these as defaults. #

  #----------------------------------------------------------------#

  : ${script_name:=`basename $0`}

  : ${script_home:=`dirname  $0`}

  SD_script_home=`echo $script_home | sed 's/\/*$/\//'`



  #------------------------------------------------#

  # User wants help, so find the documentation    #

  # section and print everything from there down.  #

  #------------------------------------------------#

  $AWK -v script_name=$script_name \

    'BEGIN { n=0 }



    { #------------------------------------------#

      # Until we find the documentation section, #

      # keep looking at each line.              #

      #------------------------------------------#

      if (n == 0)

      {

        if ($0 ~ /^# +D O C U M E N T A T I O N/)

        {

          n = NR

          print line

          print $0

        }

        else

        {

          line = $0

        }



        next

      }    #-------------------------------------#

      else # Once we find it, print until EOF.  #

      {    #-------------------------------------#

        print

      }

    }



    END {

          if (n == 0) # Means there is no documentation section.

          {

            "date +%Y-%m-%d" | getline yyyy_mm_dd

            print yyyy_mm_dd" NO DOCUMENTATION",

              "section found for "script_name".\a" | "cat 1>&2"

            exit 1 # Exit failure

          }

          exit 0 # Else exit success

        }' ${SD_script_home}$script_name



  exit $?

} # "SD_" prefix identifies this function's variables





#======================================================================#

#                    I N I T I A L I Z A T I O N                      #

#======================================================================#

  opt_v=0 # Default 'vi' option (0 = Do NOT vi the file list)

  tmp=/var/tmp

  follow=""

  size="499999" # Default minimum filesize

  lines="500"  # Default maximum lines



  while getopts fHhl:s:t:v opt 2> /dev/null

  do

    case "$opt" in

        f ) follow='-follow'  ;;

        H ) SHOW_DOCUMENTATION;;

        h ) EXIT_USAGE        ;;

        l ) lines="$OPTARG"  ;; # Max number of lines to display.

        s ) size="$OPTARG"    ;; # Minimum file size.

        t ) tmp="$OPTARG"    ;; # Temp directory (if /var/tmp full)

        v ) opt_v=1          ;;

        * ) echo "Ignoring invalid option, $1.";;

    esac

  done

  #----------------------------------#

  # Shift past options to arguments. #

  #----------------------------------#

  shift `expr $OPTIND - 1`





#======================================================================#

#                                M A I N                              #

#======================================================================#



  [ $# -eq 0 ] && EXIT_USAGE



  #-----------------------------------------------------------------#

  # Ensure we have write-access to temp/work directory.            #

  #-----------------------------------------------------------------#

  if [ ! -d $tmp ]; then

    echo "Temp/work directory, $tmp not found!" \

          "\n$script_name terminated."

    exit 1

  elif [ ! -w $tmp ]; then

    echo "No write access to temp/work directory, $tmp!" \

          "\n$script_name terminated."

    exit 1

  fi



  #----------------------------------------------------------------#

  # File lists of remote filesystems is problematic, so we limit  #

  # our operations to local filesystems only.                      #

  #----------------------------------------------------------------#

  df -lk $1

  if [ $? -ne 0 ]; then

    echo "$1 MUST be a local filesystem--it is not!" \

          "\n$script_name terminated."

    exit 1

  fi



  #----------------------------------------------------------------#

  # Build a 'find' command with the necessary options/arguments.  #

  # Be sure to exclude anything with cdrom in it and include -xdev #

  # -xdev if the filesystem being searched is root (/).            #

  #----------------------------------------------------------------#

  outfile=$tmp/$LOGNAME"_biggest.files" # Formatted 'find' output

  include='-size +'"$size"'c -exec ls -lc {} \;'

  exclude='-o -fstype nfs -prune -o -name cdrom\* -prune'

  [ ."$1" = ."/" ] && find_opt="-xdev $follow" || find_opt="$follow"



  date "+%D %T"

  find_cmd="find $1 $find_opt $include $exclude -print"



  #----------------------------------------------------------------#

  # Display find command before running it.  Use [gn]awk to format #

  # the output and sort it in descending order (biggest on top).  #

  #----------------------------------------------------------------#

  echo "$find_cmd 2> /dev/null | $AWK"

  eval  $find_cmd 2> /dev/null | $AWK \

    'BEGIN \

    {

      i  = 0

      own = 3

      siz = 5

      mmm = 6

      day = 7

      yyy = 8 # This may actually be yyyy or hh:mi

      Mon = "^(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)$"

    }

  # "! /^[bcd]/" skips block and character devices and directories

    ! /^[bcd]/ \

    {

      #-----------------------------------------------------------#

      # If it looks like owner and group fields are concatonated, #

      # try backing up the field ($n) list and work with that.    #

      #-----------------------------------------------------------#

      if ($siz !~ /[0-9]+/ && !match($mmm,Mon))

      {

        if ($(siz-1) ~ /[0-9]+/ && match($(mmm-1),Mon))

        {

          siz = 4 # 4th field

          mmm = 5 # Etc.

          day = 6

          yyy = 7

        }

      }



      gsub(/[\t ]+/," ")  # Squeeze whitespace.

      gsub(/./,"& ",$siz) # Isolate each digit,

      q=split($siz,a," ") #  then split the $siz into an array.

      $siz=""            # Clear $siz.

      for (p=1;q>0;q--)  # Insert commas into $siz.

      {

        $siz=a[q]""$siz

        if ((p%3) == 0 && q != 1) $siz=","$siz # Insert commas here

        p++

      }



      printf("%13s %-8s %s %02d %-5s %s\n",

        $siz, $own, $mmm, $day, $yyy, $NF)



      #-----------------------------------------------#

      # if size value is not 5, then reset it, et al. #

      #-----------------------------------------------#

      if (siz != 5)

      {

        siz = 5

        mmm = 6

        day = 7

        yyy = 8

      }

    }' |  sort -r -k 1,2 | head -$lines  > $outfile



  #----------------------------------------------------------------#

  # Unless 'vi' option was given, simply cat our file list.        #

  #----------------------------------------------------------------#

  if [ `wc -l < $outfile` -eq 0 ]; then

    echo "No files found in $1 > $size bytes in size."

  else

    [ $opt_v -eq 0 ] && cat $outfile || vi $outfile

  fi



  exit $?





#======================================================================#

#                      D O C U M E N T A T I O N                      #

#======================================================================#

#                                                                      #

#      Author: Bob Orlando (Bob@OrlandoKuntao.com)                    #

#                                                                      #

#        Date: April 8, 1995                                          #

#                                                                      #

#  Program ID: biggest.sh                                              #

#                                                                      #

# Code Contrl: aphrodite:~dmc/SCCS.                                    #

#                                                                      #

#      Usage: biggest.sh -fHh -l <nn> -v -t <dir> -s <nnn> fs        #

#                                                                      #

#                        -f = Follow links                            #

#                        -H = Displays detailed documentation        #

#                        -h = Provides usage brief                    #

#                        -l = Displays <nn> lines (default is 500)    #

#                        -s = Minimum file size is <nnn>              #

#                              (default is 500K)                      #

#                        -t = Use <dir> as temp/work directory        #

#                              (default is /var/tmp)                  #

#                        -v = Edit (vi) file list                    #

#                        fs = Required filesystem argument.          #

#                                                                      #

#    Purpose: List biggest files in a given filesystem (files        #

#              appear in descending order).                            #

#                                                                      #

# Description: Using the find command, descend through the specified  #

#              file system (fs) listing all files whose sizes exceed  #

#              either the default minimum size (500K) or the minimum  #

#              value provided via size (-s) option.  The filelist      #

#              is created in /var/tmp by default as it usually much    #

#              larger than /tmp.  However, in the event that /var      #

#              is the filesystem that's full (or is not writable to    #

#              the user), the temp dir (-t) option is available to    #

#              redirect the output elsewhere.                          #

#                                                                      #

#              When root is the directory being searched, -xdev is    #

#              supplied as a find argument so only root, and none      #

#              of its subdirectories, is searched.                    #

#                                                                      #

#              With the 'vi' option (-v) the user can edit the        #

#              normally cat'd file list.                              #

#                                                                      #

#    Modified: 2005-03-02 Bob Orlando                                  #

#                v1.6  * Add code to parse the correct fields when    #

#                        the owner and group fields are concatonated, #

#                        effectively making two fields, one (really  #

#                        messes up AWK processing).                  #

#                                                                      #

#----------------------------------------------------------------------#

Hi tronayne,

Thanks much for your very helpful reply. I ran the command and the results were almost all looking like this:

Code:

862 mailprint Sep 30 23:44 qfp913i2wb023649

89 mailprint Dec 01 04:33 dfpB19Xgfu017333

Tried to find the files (i.e. dfpB19Xgfu017333) and noticed they all live inside /var/spool/clientmqueue directory.

Code:

[root@servername]# du -s  /var/spool/clientmqueue

1441968 /var/spool/clientmqueue

And

Code:

[root@servername]# du -ah /var/spool/clientmqueue

8.0K    /var/spool/clientmqueue/dfp9F411ua002981

8.0K    /var/spool/clientmqueue/dfp9QMJ2Wh004013

8.0K    /var/spool/clientmqueue/qfpAQ8n1cx001885

8.0K    /var/spool/clientmqueue/dfp9VGg1sW014417

8.0K    /var/spool/clientmqueue/qfpAQ341Sr030239

8.0K    /var/spool/clientmqueue/qfp944u2sZ003347

8.0K    /var/spool/clientmqueue/dfpA5AK1vn009286

8.0K    /var/spool/clientmqueue/qfpA6Jx1g6022690

8.0K    /var/spool/clientmqueue/qfpAL4D2do005340

8.0K    /var/spool/clientmqueue/dfpA4Cq1lL020613

1.4G    /var/spool/clientmqueue

I think we're getting close! Would the files in this directory be somethings I can just go ahead and delete?

Thanks,
tezarin

Well, I've got to admit that I did not have any idea what the heck /var/spool/mqueue was for so I did a little Google search and turned up this: http://www.the-art-of-web.com/system/mqueue/; appears to explain the what's, why's and wherefore's.

So, give it a read.

Hope this helps some.

Thanks for the thread. Me either...I was sitting here scratching my head and at the same time looking at the size of that directory...

Have been reading up on it since this morning:

http://groups.google.com/group/comp....49feda8bd3bad3

http://www.unix.com/solaris/152772-n...entmqueue.html

http://www.unix.com/solaris/152772-n...entmqueue.html

They all say it's safe to delete everything found in that clientmqueue directory...

Has anyone else done this before? The server is not even a mail server...Very strange

This is the machine's operating system info: 2.6.12-1.1381_FC3smp

Update - I ran the following command and got an error:

Code:

 

[root@servername clientmqueue]# rm -f *

-bash: /bin/rm: Argument list too long

Hi Tronayne,

OK, I successfuly deleted clientmqueue directory and went from 71% to 53%. Created that directory again in case it's needed by something, mkdir clientmqueue...Was this a bad approach?

Checked to make sure sendmail is stopped which looks like it is so not sure why I would have enteries in that directory to begin with, some of the enteries were dating today.

Now, ran your code (biggest.sh) and this is the output I'm getting. What would you think I should be deleting?

Thanks in advance

Code:

[root@servername tezarin]# ./biggest /

Filesystem          1K-blocks      Used Available Use% Mounted on

/dev/hda1              4134900  2049488  1875364  53% /

12/05/11 13:10:47

find / -xdev  -size +499999c -exec ls -lc {} \; -o -fstype nfs -prune -o -name cdrom\* -prune -print 2> /dev/null | gawk

      948,578 root    May 02 2006  /root/.mozilla/firefox/4kgrrgn9.default/XUL.mfasl

      936,960 root    Nov 22 2005  /lib/libcrypto.so.0.9.7a

      903,393 root    May 29 2005  /etc/gconf/schemas/gnomemeeting.schemas

  887,935,163 root    Nov 16 22:41 /etc/bacula/bacula-fd.log

      872,812 root    Dec 05 13:10 /var/log/cups/access_log

      845,636 root    Nov 22 2005  /lib/tls/i686/libdb-4.2.so

      845,636 root    Nov 22 2005  /lib/tls/i586/libdb-4.2.so

      845,636 root    Nov 22 2005  /lib/tls/i486/libdb-4.2.so

      843,996 root    May 29 2005  /lib/libdb-4.2.so

      839,680 root    May 02 2006  /root/.mozilla/firefox/4kgrrgn9.default/Cache/_CACHE_003_

      825,328 root    May 29 2005  /sbin/restore

      821,989 root    May 29 2005  /etc/gconf/schemas/gok.schemas

      816,171 root    Jun 05 2006  /var/cache/yum/base/primary.xml.gz

      795,578 root    Oct 23 2005  /boot/System.map-2.6.12-1.1380_FC3smp

      795,578 root    Nov 21 2005  /boot/System.map-2.6.12-1.1381_FC3smp

      795,114 root    Aug 31 2005  /boot/System.map-2.6.12-1.1376_FC3smp

      792,857 root    Oct 23 2005  /etc/termcap

      790,080 root    Aug 14 2005  /boot/System.map-2.6.12-1.1372_FC3smp

      774,827 root    May 29 2005  /boot/System.map-2.6.11-1.27_FC3smp

      766,856 root    May 29 2005  /lib/modules/2.6.11-1.27_FC3/kernel/drivers/isdn/hisax/hisax.ko

      766,296 root    Aug 14 2005  /lib/modules/2.6.12-1.1372_FC3/kernel/drivers/isdn/hisax/hisax.ko

      766,244 root    Oct 23 2005  /lib/modules/2.6.12-1.1380_FC3/kernel/drivers/isdn/hisax/hisax.ko

      766,244 root    Nov 21 2005  /lib/modules/2.6.12-1.1381_FC3/kernel/drivers/isdn/hisax/hisax.ko

      766,244 root    Aug 31 2005  /lib/modules/2.6.12-1.1376_FC3/kernel/drivers/isdn/hisax/hisax.ko

      765,359 root    Oct 23 2005  /boot/System.map-2.6.12-1.1380_FC3

      765,359 root    Nov 21 2005  /boot/System.map-2.6.12-1.1381_FC3

      764,895 root    Aug 31 2005  /boot/System.map-2.6.12-1.1376_FC3

      760,344 root    Nov 21 2005  /lib/security/pam_userdb.so

      760,080 root    Aug 14 2005  /boot/System.map-2.6.12-1.1372_FC3

      747,077 root    May 29 2005  /boot/System.map-2.6.11-1.27_FC3

      742,506 root    May 29 2005  /boot/System.map-2.6.9-1.667smp

      740,729 root    Dec 05 13:10 /var/log/maillog

      727,788 root    May 29 2005  /sbin/cryptsetup

      722,472 root    May 29 2005  /lib/libnss_wins.so.2

      714,266 root    May 29 2005  /boot/System.map-2.6.9-1.667

    7,128,600 root    Dec 05 04:03 /var/lib/slocate/slocate.db

      710,168 root    May 29 2005  /lib/security/pam_smbpass.so

      708,427 root    May 29 2005  /etc/gconf/gconf.xml.defaults/schemas/apps/nautilus/preferences/%gconf.xml

      686,604 root    Nov 22 2005  /lib/libasound.so.2.0.0

      677,612 root    May 29 2005  /lib/modules/2.6.11-1.27_FC3smp/kernel/drivers/isdn/hisax/hisax.ko

      677,288 root    Aug 14 2005  /lib/modules/2.6.12-1.1372_FC3smp/kernel/drivers/isdn/hisax/hisax.ko

      677,236 root    Oct 23 2005  /lib/modules/2.6.12-1.1380_FC3smp/kernel/drivers/isdn/hisax/hisax.ko

      677,236 root    Nov 21 2005  /lib/modules/2.6.12-1.1381_FC3smp/kernel/drivers/isdn/hisax/hisax.ko

      677,236 root    Aug 31 2005  /lib/modules/2.6.12-1.1376_FC3smp/kernel/drivers/isdn/hisax/hisax.ko

      667,152 root    May 29 2005  /sbin/dump

      663,552 cyrus    May 29 2005  /var/lib/imap/db/__db.002

      663,244 root    Aug 14 2005  /lib/modules/2.6.12-1.1372_FC3/kernel/fs/xfs/xfs.ko

      663,168 root    Oct 23 2005  /lib/modules/2.6.12-1.1380_FC3/kernel/fs/xfs/xfs.ko

      663,168 root    Nov 21 2005  /lib/modules/2.6.12-1.1381_FC3/kernel/fs/xfs/xfs.ko

      663,168 root    Aug 31 2005  /lib/modules/2.6.12-1.1376_FC3/kernel/fs/xfs/xfs.ko

      662,748 root    May 29 2005  /lib/modules/2.6.11-1.27_FC3/kernel/fs/xfs/xfs.ko

      651,624 root    Aug 14 2005  /etc/gconf/schemas/gedit.schemas

      633,496 root    May 29 2005  /etc/gconf/gconf.xml.defaults/schemas/apps/gnome-terminal/keybindings/%gconf.xml

      629,219 root    May 29 2005  /etc/firmware/microcode.dat

      621,164 root    May 29 2005  /lib/modules/2.6.9-1.667/kernel/drivers/isdn/hisax/hisax.ko

      616,620 root    Aug 14 2005  /lib/modules/2.6.12-1.1372_FC3smp/kernel/fs/xfs/xfs.ko

      616,544 root    Nov 21 2005  /lib/modules/2.6.12-1.1381_FC3smp/kernel/fs/xfs/xfs.ko

      616,544 root    Aug 31 2005  /lib/modules/2.6.12-1.1376_FC3smp/kernel/fs/xfs/xfs.ko

      616,540 root    Oct 23 2005  /lib/modules/2.6.12-1.1380_FC3smp/kernel/fs/xfs/xfs.ko

      616,312 root    Aug 09 2008  /bin/bash

      615,692 root    May 29 2005  /lib/modules/2.6.11-1.27_FC3smp/kernel/fs/xfs/xfs.ko

      609,372 root    Oct 23 2005  /sbin/fsck.ext3

      609,372 root    Oct 23 2005  /sbin/fsck.ext2

      609,372 root    Oct 23 2005  /sbin/e2fsck

      595,281 root    Oct 23 2005  /boot/initrd-2.6.12-1.1380_FC3.img

      595,278 root    Nov 21 2005  /boot/initrd-2.6.12-1.1381_FC3.img

      595,162 root    Aug 31 2005  /boot/initrd-2.6.12-1.1376_FC3.img

      595,154 root    Aug 14 2005  /boot/initrd-2.6.12-1.1372_FC3.img

      590,232 root    May 29 2005  /lib/modules/2.6.9-1.667/kernel/fs/xfs/xfs.ko

      589,885 root    May 29 2005  /boot/initrd-2.6.11-1.27_FC3.img

      588,764 root    Dec 05 13:10 /var/log/messages

      579,309 root    Nov 21 2005  /boot/initrd-2.6.12-1.1381_FC3smp.img

      577,698 root    Aug 31 2005  /boot/initrd-2.6.12-1.1376_FC3smp.img

      577,681 root    Aug 14 2005  /boot/initrd-2.6.12-1.1372_FC3smp.img

      577,653 root    Oct 23 2005  /boot/initrd-2.6.12-1.1380_FC3smp.img

      574,717 root    May 29 2005  /boot/initrd-2.6.11-1.27_FC3smp.img

      570,400 root    Nov 21 2005  /sbin/ldconfig

    5,681,152 rpm      Aug 24 2006  /var/lib/rpm/Basenames

    5,593,374 mailprint Dec 05 13:10 /home/mailprint/mailprint.log

      558,391 root    Jan 14 2010  /sbin/bacula-fd

    5,566,464 rpm      Aug 24 2006  /var/lib/rpm/Dirnames

    5,532,228 root    Jun 05 2006  /var/cache/yum/base/primary.xml.gz.ce7bdfd5a6066a4f18c498b693faa16597a0733c.pickle

    5,501,431 root    Dec 04 04:02 /var/log/cups/access_log.3

      549,060 root    May 29 2005  /lib/modules/2.6.9-1.667smp/kernel/fs/xfs/xfs.ko

      548,068 root    May 29 2005  /lib/libnss_db.so.2.0.0

      545,540 root    May 29 2005  /sbin/dmraid.static

      541,716 root    Oct 23 2005  /sbin/udev.static

      541,540 root    May 29 2005  /lib/modules/2.6.9-1.667smp/kernel/drivers/isdn/hisax/hisax.ko

      522,116 root    May 29 2005  /bin/ash.static

      519,405 root    May 29 2005  /etc/gconf/schemas/panel-toplevel.schemas

      518,089 root    Jun 16 2005  /opt/ymessenger/lib/libgtkhtml.so.6

    5,170,210 root    Dec 04 04:02 /var/log/cups/access_log.2

      508,182 root    May 29 2005  /etc/gconf/gconf.xml.defaults/schemas/apps/gnome-terminal/profiles/Default/%gconf.xml

  46,043,136 rpm      Aug 24 2006  /var/lib/rpm/Packages

    4,517,673 root    Dec 04 04:02 /var/log/cups/access_log.1

    4,340,206 root    Dec 04 04:02 /var/log/cups/access_log.4

    4,123,734 root    Oct 24 2009  /root/zot

    2,818,460 root    May 29 2005  /lib/libnss_ldap-2.3.3.so

    2,648,166 root    Jun 05 2006  /var/cache/yum/updates-released/primary.xml.gz.d0eecb98e9a164da7dccea13c6fd6acab7823d81.pickle

    2,647,550 root    Sep 27 04:03 /var/spool/mail/root

  22,568,960 cyrus    May 29 2005  /var/lib/imap/db/__db.004

    2,096,844 root    Jun 10 2005  /root/.mozilla/plugins/libflashplayer.so

    1,954,668 root    Jun 16 2005  /opt/ymessenger/bin/ymessenger.bin

  19,136,220 root    Dec 05 09:31 /var/log/lastlog

    1,660,612 root    Oct 23 2005  /boot/vmlinuz-2.6.12-1.1380_FC3

    1,660,241 root    Nov 21 2005  /boot/vmlinuz-2.6.12-1.1381_FC3

    1,659,818 root    Aug 31 2005  /boot/vmlinuz-2.6.12-1.1376_FC3

    1,652,295 root    Aug 14 2005  /boot/vmlinuz-2.6.12-1.1372_FC3

    1,636,782 root    May 29 2005  /boot/vmlinuz-2.6.11-1.27_FC3

    1,583,651 root    Oct 23 2005  /boot/vmlinuz-2.6.12-1.1380_FC3smp

    1,582,919 root    Nov 21 2005  /boot/vmlinuz-2.6.12-1.1381_FC3smp

    1,581,871 root    Aug 31 2005  /boot/vmlinuz-2.6.12-1.1376_FC3smp

    1,575,017 root    Aug 14 2005  /boot/vmlinuz-2.6.12-1.1372_FC3smp

    1,552,968 root    May 29 2005  /boot/vmlinuz-2.6.11-1.27_FC3smp

    1,525,032 root    Nov 22 2005  /lib/tls/libc-2.3.6.so

    1,508,968 root    Nov 21 2005  /lib/libc-2.3.6.so

    1,500,500 root    Nov 21 2005  /lib/i686/libc-2.3.6.so

    1,318,912 root    Sep 30 04:04 /var/lib/rpm/__db.002

    1,164,630 root    May 29 2005  /etc/gconf/schemas/gnome-terminal.schemas

    1,089,168 root    May 29 2005  /sbin/lvm.static

    1,080,872 root    May 29 2005  /etc/gconf/schemas/metacity.schemas

    1,055,308 root    May 29 2005  /etc/gconf/schemas/apps_nautilus_preferences.schemas

    1,047,083 root    Dec 05 04:02 /var/cache/man/whatis

  10,420,224 rpm      Aug 24 2006  /var/lib/rpm/Filemd5s

    1,037,232 root    Nov 14 12:03 /var/spool/cups/d10703-001

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

                        00

Well, /etc/bacula/bacula-fd.log looks like a good candidate, eh? You can, if you'd like, empty that file with

Code:

su -

>/etc/bacula/bacula-fd.log

Ctrl-D

Think I'd leave the rest of it alone.

Good time to start thinking about a new drive....

Hope this helps some.

Hi tronayne,

Wow, we think a like now! Yesterday before I come back to see this thread, I ran the same command you posted and emptied that large log file! The root directory is now 31% full! You have been more than helpful, I really appreciate it. We should mark this thread fixed.

Regards,
t