listing file size by percentage?

abolishtheun · 03-25-2008, 04:56 PM

is there a program like du(1) but can list the sizes in terms of the percentage of the total size in a directory? imagine something like this:

Code:

% foo ~/dir-to-measure

13.00 /home/user/dir-to-measure/subdirA
20.00 /home/user/dir-to-measure/subdirB
30.00 /home/user/dir-to-measure/subdirC
100.00 /home/user/dir-to-measure

this would tell you that subdirA contains 13% of the total of ~/dir-to-measure. I occasionally need something like this to find out what's taking up so much space on a disk. Any help?

BlueC · 03-25-2008, 05:26 PM

you could try "baobab" which is part of gnome-utils package - its a GUI utility and does exactly what you want, and more.

i'm pretty sure that feeding the right options to du can do what you want too... just not sure how

colucix · 03-25-2008, 06:18 PM

Or you can try to write a simple script which performs the calculation, e.g. something like

Code:

#!/bin/bash
total=$(du -s $1 | gawk '{print $1}')
du --max-depth=1 $1 | while read line
do
   size=$(echo $line | gawk '{print $1}')
   dir=$(echo $line | gawk '{print $2}')
   percent=$(echo "scale=4; $size / $total * 100" | bc)
   printf "%6.2f%%   %s\n" $percent $dir
done

the output will be something like

Code:

$ foo ~/doc/
 44.78%   /home/colucix/doc/shell
  0.08%   /home/colucix/doc/help
  2.04%   /home/colucix/doc/articles
 51.97%   /home/colucix/doc/manual
100.00%   /home/colucix/doc/

Tinkster · 03-25-2008, 08:19 PM

Nice :}

A little suggestion:

Code:

du --max-depth=1 $1 | sort -g | while read line

Cheers,
Tink

abolishtheun · 03-25-2008, 09:17 PM

That looks very nice! The only down side I see is that it calculates the size of every file twice, doesn't it?

Tinkster · 03-25-2008, 10:30 PM

no it doesn't

abolishtheun · 03-25-2008, 11:24 PM

does it not calculate the total size, then again in the while loop?

Tinkster · 03-26-2008, 12:52 AM

nope :}

Show me which line you think does that?

Cheers,
Tink

abolishtheun · 03-26-2008, 01:21 AM

Every file is counted once here:

Code:

#!/bin/bash
total=$(du -s $1 | gawk '{print $1}')

and then again here:

Code:

du --max-depth=1 $1 | while read line

does du cache?

ak_random · 03-26-2008, 01:25 AM

Quote:

Originally Posted by abolishtheun

does du cache?

I doubt it does any caching between invocations. However, a lot of the file system metadata loaded by du should still be in the buffer cache, so depending on how many files you have under that directory, two runs of du might still execute pretty quickly.

Tinkster · 03-26-2008, 01:58 AM

Quote:

Originally Posted by abolishtheun

Every file is counted once here:

Code:

#!/bin/bash
total=$(du -s $1 | gawk '{print $1}')

and then again here:

Code:

du --max-depth=1 $1 | while read line

does du cache?

OK. I wouldn't have referred to a run of du as
a calculation. I suppose if you were concerned
with running du twice (and no, it doesn't cache,
but your file-system most likely does) you could
use a different approach, e.g. run du ones, stick
the results in an array with awk, add them up, and
then calculate the percentages on a second run.

Cheers,
Tink

colucix · 03-26-2008, 04:10 AM

Ok. Here's a solution which uses du only once, to avoid problems with the buffer cache:

Code:

#!/bin/bash
du --max-depth=1 $1 | sort -gr | while read line
do
   test -z "$total" && total=$(echo $line | cut -f 1 -d \ )
   size=$(echo $line | cut -f 1 -d \ )
   dir=$(echo $line | cut -f 2- -d \ )
   percent=$(echo "scale=4; $size / $total * 100" | bc)
   printf "%6.2f%%   %s\n" $percent "$dir"
done | tac

also introduced the sort -g command as suggested by Tinkster. Furthermore I substituted the gawk statements with cut to manage directories containing blank spaces in their name, by extracting from the second field to the end of the line:

Code:

dir=$(echo $line | cut -f 2- -d \ )

pwc101 · 03-26-2008, 06:22 AM

Nice script!

I added in a section to give the directory size as a second column. There's probably a more elegant way, but it seems to work:

Code:

#!/bin/bash

du --max-depth=1 $1 | sort -gr | while read line
do
   test -z "$total" && total=$(echo $line | cut -f 1 -d \ )
   size=$(echo $line | cut -f 1 -d \ )
   if [ $size -lt 1000 ]; then
      printsize="$size"K
   elif [ $size -ge 1000 ] && [ $size -lt 1000000 ]; then
      printsize=$(echo "scale=1; $size/1024" | bc)M
   elif [ $size -ge 1000000 ] && [ $size -lt 1000000000 ]; then
      printsize=$(echo "scale=1; $size/1024000" | bc)G
   fi
   dir=$(echo $line | cut -f 2- -d \ )
   percent=$(echo "scale=4; $size / $total * 100" | bc)
   printf "%6.2f%%\t%s\t%s\n" $percent $printsize "$dir"
done | tac

You can add in tera and petabytes if you need them!