Total space occupied by all files above 1G
 

Hi,
I am doing something like this.

[user01@tmelbld01 dropzone]$find . -size +1G -exec ls -larth {} \;|awk '{print $5}'
find: ./DM/lost+found: Permission denied
2.7G
4.7G
2.0G
find: ./1.9.0.6/Hotfix/QC16841/patch_16841: Permission denied
find: ./1.9.0.6/Hotfix/QC16847/CA: Permission denied
4.1G
1.5G
2.9G
1.8G
1.8G
1.8G
1.9G
4.2G
1.9G
4.2G
1.9G
4.3G
2.0G
2.0G
find: ./lost+found: Permission denied
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
2.0G
1.5G
1.1G
1.1G
1.1G
1.1G
1.2G
1.1G
1.8G
1.8G
1.8G
1.9G
1.8G
1.8G
1.8G
1.8G
1.8G
1.8G
1.8G
1.8G
1.8G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.1G
1.5G
1.5G
1.1G
2.4G
1.1G
[user01@tmelbld01 dropzone]$

What I want is "no entries where it says permission denied" + add sum of first colums mena s 1.1G, 2.4G etc total GB should be dispalyed at last colum with Total:

Is this possible?

Please educate.

Many thanks

[edit]
I should probably explain why I took that 'h' off from ls'
output .... rounding errors. Using size in bytes, adding
up and dividing as I did yields a result out by > 4 GB on
my machine, thought that's worthwhile considering[/edit]


Cheers,
Tink

Thank you very much for your reply, that indeed worked and gave me result at 106GB, that amount of space we are "wasting".

Now I am trying to understand the logic behind your one line.

Please help me understand.

First bit,
find . -size +1G -exec ls -lart {} \;
this is list all files in long format and sorting as per rt.
then,
2>/dev/null
here you are re-directing output "standard output" to null device.

the last bit I am unable to comprehend,
awk '{size+=$5}END{print size/1024/1024/1024"G"}'
here you are printing 5th column and start evaluation from 0 I think
you wrote END but where is BEGIN?

Also on a side note when I do find for some files I do get "Permission denied" your one liner is ignoring that? I believe.

Please educate.

Many Thanks in advance.

Yup.. That said, I should have trimmed the fat; since find will
run the ls against individual files sorting is pretty pointless
in this case. Make that "ls -la"

Nope, standard error. You didn't want to see the "permission denied",
and that's what we suppress.

BEGIN is not required in this case. awk operates "per line" of
text/output/... so what we do is: we create a variable "size".
For each line of find's ls output we add the 5th column to it.
After all lines have been processed (END block) we print the
added up values.




Cheers,
Tink

Two suggestions, if I may add:
1) using 'stat -c "%s" {} ' instead of 'ls -lart {}'. The output of 'ls' might differ on different systems. Using 'stat' makes a small modification to 'awk' necessary:
awk '{size+=$0}END{print size/1024/1024/1024"G"}

2) adding '-type f' to find. This does not have any immediate impact if find operates on this scale ( > 1G ). But if one wishes to change that, e.g. to > 1k then also directories would be found. And especially with 'ls' this would yield highly erroneous results.

Can't say I've come across an ls on any posix compliant OS that would
yield different byte sizes for files from stat, your mileage may vary.


Point 2 is quite valid.


Cheers,
Tink

My concern was not so much that the size itself would be reported wrongly but that field 5 *might* not always contain file size.
I once read here
http://www.grymoire.com/Unix/Awk.html#uh-2

that on Linux systems the output differs from unix systems. Not sure if size is affected but it seems that there were (still are?) different implementations of 'ls' at some time.

Thanks to both of you for your help.