LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 11-07-2007, 09:28 PM   #1
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Rep: Reputation: 51
gawk/awk/tcsh - how to convert large number to human readable (Mb,Gb, etc)?


in a script, I run through a file system finding certain files and accumulating their size.

At the end of the script, it generates a report, but the accumulated sizes are HUGE.

I'm woefully ignorant when it comes to [g]awk. Is there a simple way to convert the large number into something more human redable?
 
Old 11-07-2007, 10:48 PM   #2
angrybanana
Member
 
Registered: Oct 2003
Distribution: Archlinux
Posts: 147

Rep: Reputation: 21
example code:

Code:
awk 'BEGIN{sum=456456;
hum[1024**3]="Gb";hum[1024**2]="Mb";hum[1024]="Kb"; 
for (x=1024**3; x>=1024; x/=1024){ 
        if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x];break }
}}'
output:
Code:
445.76 Kb

Last edited by angrybanana; 11-07-2007 at 11:12 PM. Reason: changed to printf for output (for rounding)
 
Old 11-08-2007, 05:28 AM   #3
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,314

Rep: Reputation: 175Reputation: 175
Code:
du -sh *
94M     HOME.tgz
32K     noughts.zip
146M    solaris_home_20070814.tgz
linux only
 
Old 11-08-2007, 01:29 PM   #4
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by angrybanana View Post
example code:

Code:
awk 'BEGIN{sum=456456;
hum[1024**3]="Gb";hum[1024**2]="Mb";hum[1024]="Kb"; 
for (x=1024**3; x>=1024; x/=1024){ 
        if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x];break }
}}'
output:
Code:
445.76 Kb
Factastic, thanks!

I had to remove the "BEGIN" in order to be able to access data passed in through $1, otherise, this works great.

Thanks again.

Last edited by BrianK; 11-08-2007 at 01:31 PM.
 
Old 11-08-2007, 02:22 PM   #5
angrybanana
Member
 
Registered: Oct 2003
Distribution: Archlinux
Posts: 147

Rep: Reputation: 21
Quote:
Originally Posted by BrianK View Post
Factastic, thanks!

I had to remove the "BEGIN" in order to be able to access data passed in through $1, otherise, this works great.

Thanks again.
Oh yea, used "BEGIN" for demonstration sake (no input). You'll probably use it in "END". Also, I forgot to add in 'bytes', but that should be trivial.
 
Old 08-13-2010, 10:23 AM   #6
Ronald
LQ Newbie
 
Registered: Aug 2010
Posts: 8

Rep: Reputation: 0
Quote:
Originally Posted by angrybanana View Post
example code:

Code:
awk 'BEGIN{sum=456456;
hum[1024**3]="Gb";hum[1024**2]="Mb";hum[1024]="Kb"; 
for (x=1024**3; x>=1024; x/=1024){ 
        if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x];break }
}}'
output:
Code:
445.76 Kb
This is great !!!

I do have one question

My source is a find and the total size of all found files is greater than 1 Gb. This is ofcourse no problem for this code.. But here is the thing. When the total size is above 1Mb the result is given in Mb's as wel as Kb's . Even when the total size is above 1Gb the results are in Gb's Mb's and Kb's

Any idea how to filter this so that it will only display the correct one?
 
Old 08-18-2010, 05:53 AM   #7
Ronald
LQ Newbie
 
Registered: Aug 2010
Posts: 8

Rep: Reputation: 0
ok no one with any suggestion(s)
 
Old 08-18-2010, 06:34 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Could you post an example of what you actually tried to do? The above awk code shows 1 format, be it Gb, Mb or Kb. It does not show Mb and Kb or Gb, Mb and Kb.

One more thing that could be of interest: You posted this from a machine running OS-X. Assuming that the code you tried is also run on that machine: What flavour/version of awk are you using? (awk --version should show both).

Hope this helps.
 
Old 08-18-2010, 07:04 AM   #9
Ronald
LQ Newbie
 
Registered: Aug 2010
Posts: 8

Rep: Reputation: 0
Quote:
Originally Posted by Ronald View Post
ok no one with any suggestion(s)

Hmm Turns out I must have an error in my format..

When I use a big number in your test script it works just fine.. So the script is good

Nevermind me asking
 
Old 08-18-2010, 07:06 AM   #10
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Glad to read you got it sorted out
 
Old 08-18-2010, 07:34 AM   #11
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
Code:
#!/usr/bin/awk -f

BEGIN{	x = 456456

        split("B KB MB GB TB PB",type)

	for(i=5;y < 1;i--)
	    y = x / (2**(10*i))

	print y type[i+2]
}
 
1 members found this post helpful.
Old 08-18-2010, 07:35 AM   #12
Ronald
LQ Newbie
 
Registered: Aug 2010
Posts: 8

Rep: Reputation: 0
Quote:
Originally Posted by druuna View Post
Glad to read you got it sorted out
Hi,

I do need some help.. I am quite a noob on Bash and awk

Code:
find ./ -name "*.jpg" -exec ls -l {} \; | awk '{for (I=9;I<=NF;I++){ printf "%s",$I }print " :",$5}{ x++ } { size=size+$5 } END { print "\n" x " Files Found""\n\n" } END { hum[1024**3]="Gb"; hum[1024**2]="Mb"; hum[1024]="Kb"; for (x=1024**3; x>=1024; x/=1024) { if (size>=x) print "\n Total Size= " size/x,hum[x]; } }'
This is the script I am working on.. So I put the code for size in Human readable form at the end (with some minor changes). This works but it gives the 3 outputs (Gb, Mb, Kb) When I add the 'break' at the end (see original code) there is no output of the size.

So I do need some help with this. Maybe you can push me in the right direction..

2nd question: I use print is stead of printf cause I want to print some normal text with the output. I can't seem to fix that with printf..

MAny thanks

Edit: it seems it has something to do with total size.. In my test dir I have only 2 .jpg's with a total size of 228850 bytes. When I run the script (with the ';break' at the end, there is no result. When I creat a fake .jpg with a size of 1Gb the output works good. Also when I make a fake file of 25Mb no output is displayed.

Edit2: ok, I am a bit further.. It has something to do with using print in stead of printf..... when I use printf the output works great.. But How can i put some additional text in the output? So it displays Total Size = ......

Last edited by Ronald; 08-18-2010 at 08:04 AM.
 
Old 08-18-2010, 08:13 AM   #13
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Using grail's code posted in post #11:
Code:
#!/bin/bash

find . -name "*.jpg" -exec ls -l --time-style=long-iso {} \; | \
awk 'BEGIN { totalAmount = "0" }
{ totalAmount = totalAmount + $5 }
{ for (I=8 ; I<=NF ; I++ ) { printf "%s",$I } { print " :",$5} }
END { print "-----------------------------------------------------" ;
printf("Total amount of files : %s - total size : ",NR) ; 
split("B KB MB GB TB PB",type)
  for(i=5;y < 1;i--)
    y = totalAmount / (2**(10*i))
    print y type[i+2]
}'
Because the output of ls -l is not always the same when it comes to the date field, I changed it to ls -l --time-style=long-iso, which does give you a consistent output and a fixed starting field (8).

Hope this helps.
 
1 members found this post helpful.
Old 08-18-2010, 08:48 AM   #14
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
Whilst I like druuna's solution I have been reading a little about the vagaries of ls and why it should not be used, part of which is pointed out by druuna.
So I would submit this slight variation on the same theme:
Code:
#!/bin/bash

find . -name "*.awk" -exec stat {} \; | 
    awk '/Size:/{sum+=$2}
         END{split("B KB MB GB TB PB",type);
             for(i=5;y < 1;i--)
                 y = sum / (2**(10*i));

             print "Total amount of files : "(NR/7)" - total size : "y type[i+2]
         }'
 
Old 08-18-2010, 08:50 AM   #15
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 16

Rep: Reputation: 233Reputation: 233Reputation: 233
I think you can easily get the file size with this command:
Code:
find -type f -name '*.jpg' -printf '%s\n'
I also suggest using -iname instead.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Static awk or gawk binary rbautch Linux - Software 5 06-14-2011 02:41 AM
LXer: Get started coding with GAWK and AWK LXer Syndicated Linux News 0 09-26-2006 03:54 PM
Webalizer - howto convert the trafic into human readable? sys7em Linux - Server 3 09-25-2006 05:48 AM
Deleting a line with gawk/awk caps_phisto Linux - General 4 11-06-2004 03:31 PM
awk strange behaviour in tcsh flyingalex Linux - Software 5 10-17-2003 11:27 AM


All times are GMT -5. The time now is 09:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration