For you sysadmins, what are you doing to track what directories are slowly creeping up in size? Currently, we're just placing du in a cron job that's ran every other day, but I'm hoping others might chime in with a better solution. We're using xymon as our monitoring solution, but it obviously doesn't show that level of detail for which directories are increasing. JDiskReport is a pretty handy tool that shows a pie chart of exactly which directories are taking up what amount of space, but one would have to run that almost daily to get the trending I'm after.

Munin monitoring, nagios with PNP4Nagoos (or Icinga instead of nagios), cacti, or collectd with graphite would be my goto for disk trends. du is a bit heavy, do you mean your scripts use df?

here my hax:

That's a pretty disk IO intensive script for passive disk usage monitoring. Also, it's odd your ksh script references bash in the shebang. You should consider using one of the solutions I mentioned. They automatically visualize the data using a round robin database.

sag, thanks for the info. I'm trying to figure out which directories are the culprits over, say, a 6-month period. The line graphs within xymon tell me the trending for the total disk, but I need something more granular that can track which subdirectory or subdirectories are on the rise.

If you have a gui jdisk report is a nice java graphical tool for viewing disk usage. It gives you a clickable pie chart to drill down a directory tree. Cross-platform too, works on windows as well.

http://www.jgoodies.com/downloads/jdiskreport/

Doug, appreciate your response, but I already mentioned we've used JDisk Report in the past and it wasn't exactly the automated solution we were after.

Oops, missed that. Sorry.

Why do you feel you need that much granularity vs monitoring the overall disk usage? I typically monitor the overall disk usage and if maintenance needs to be performed or space hogs need to be found then that's when I manually go into more granularity. A script like that doesn't scale well into the terabytes and orders of magnitude larger.

sag -> The main purpose behind this is that we have a plethora of users who I would consider to be "storage abusers". There are a few directories they're writing to that seem to fill and, instead of being reactive, I prefer to proactively monitor exactly which subdirectories are being impacted. We have a budget that prevents us from buying additional storage for the moment, so knowing when and how much data is being written to which subdirectory would greatly benefit us. And you're right, du is I/O intensive which is why we were hunting for something a little less intrusive. Ideally, we'd run du every few weeks to get this info, but since our users aren't consistent regarding WHEN they're creating the data, then I'm forced to babysit the server's storage to prevent total fillup.

This is all being done improperly from my perspective. What SHOULD'VE happened is disk quotas should've been enforced from the get-go, but I was hired after decisions like these should've been made. So now these users new norm is willy nilly disk writes with no regard for other's ability to write data. We're integrating groups at our job so disk quotas will go into effect, but this likely won't happen for another year. I was hoping I could find something in the meantime.

The only other thing I can think of is to perhaps cron a find / -cmin every day or so, but it just seems like too much administrative overhead.

Thanks for giving it a shot. None of the other admins care for Nagios, but it wouldn't do what we're after anyway. Xymon is pretty dang good for its simplistic and intuitive nature and creating custom monitoring tools, complete with graphing, is extremely helpful and fairly straightforward.

@loadedmind: your problem would easily be solved using quotas. That should prevent users from abusing their diskspace. Depending on the exact platform and filesystem used there are various solutions to achieve this.

As for trending: yes it's reactive. Yet the granularity and heavy I/O that comes from running your script will probably bring new, other problems. Quotas would prevent the need for that. Zabbix is a general tool to get trending results for various metrics all over your system, but comes with another cost: storage of that database, which may run into the gigabytes, especially if not set-up with the data size in mind.

have you bump to this one:

http://xdiskusage.sourceforge.net/