LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 04-03-2011, 03:20 PM   #16
Ramurd
Member
 
Registered: Mar 2009
Location: Rotterdam, the Netherlands
Distribution: Slackwarelinux
Posts: 703

Rep: Reputation: 111Reputation: 111

because awk is not built for speed; it's in fact quite slow, but does support alot of otherwise useful features. :-) nice script!
 
Old 04-03-2011, 04:05 PM   #17
kjhambrick
Senior Member
 
Registered: Jul 2005
Location: Round Rock, TX
Distribution: Slackware64 15.0 + Multilib
Posts: 2,159

Rep: Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512
Thanks Ramurd.

What I found baffling was that I expected the get-pkgsize
script to take longer than the simple awk one-liner because
get-pkgsize is a gawk script in a shell-wrapper with which
pipes output to sort then to another gawk script.

Anyhow ...

-- kjh(<G> it's not like the real delta-t's are significant in any case <G>)
 
Old 04-03-2011, 04:09 PM   #18
kjhambrick
Senior Member
 
Registered: Jul 2005
Location: Round Rock, TX
Distribution: Slackware64 15.0 + Multilib
Posts: 2,159

Rep: Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512
p.s. I use gawk every day for some serious data conversions( GB's of Data )
and I have found that the speed is close enough to perl's speed that I choose
gawk over perl because I can use code-generators I've developed over the past
30-years
 
Old 04-04-2011, 05:37 AM   #19
mRgOBLIN
Slackware Contributor
 
Registered: Jun 2002
Location: New Zealand
Distribution: Slackware
Posts: 999

Rep: Reputation: 231Reputation: 231Reputation: 231
Nice (g)awk kjhambrick.

Good to see someone use awk for something other than '{ print $1 }' for a change.
 
Old 04-04-2011, 06:43 AM   #20
kjhambrick
Senior Member
 
Registered: Jul 2005
Location: Round Rock, TX
Distribution: Slackware64 15.0 + Multilib
Posts: 2,159

Rep: Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512
Thanks mRgOBLIN !

I really do like gawk's syntax. It's quite capable of any text processing that I have ever thrown at it.

I did figure out why get-pkgsize was so much faster than audriusk's original awk one-liner ...

The get-pkgsize script invokes nextfile as soon as it finds the /^UNCOMPRESSED/ pattern in the stream

OTOH, the awk one-liner scans the rest of each the files in /var/log/packages, looking for more instances
of the pattern in each line.

These are the times again. Note that the new one-liner with nextfile is about as fast as the grep pipeline.

-- kjh

p.s. not that the times of 0.02 secs vs 0.14 secs mean anything in the real world where the one-liner will be invoked


Code:
# gnashley's fgrep suggestion:

[konrad@kjhlt5 compat32pkg]$ time fgrep UNCOMPRESSED /var/log/packages/* | awk -F: '{print $3,$1}' | LC_ALL=C sort -rh > /dev/null

real    0m0.020s
user    0m0.009s
sys     0m0.017s

# fskmh's grep approach with audriusk's LC_ALL=C and sort -rh 

[konrad@kjhlt5 compat32pkg]$ time grep UNCOMPRESSED /var/log/packages/* | awk -F: '{print $3,$1}' | LC_ALL=C sort -rh > /dev/null

real    0m0.022s
user    0m0.015s
sys     0m0.016s

# audriusk's original awk one-liner

[konrad@kjhlt5 compat32pkg]$ time awk -F: '/UNCOMPRESSED/ {print $2,FILENAME}' /var/log/packages/* | LC_ALL=C sort -rh > /dev/null

real    0m0.137s
user    0m0.131s
sys     0m0.014s

# audriusk's original awk one-liner with nextfile after match

[konrad@kjhlt5 compat32pkg]$ time awk -F: '/^UNCOMPRESSED/ {print $2,FILENAME ; nextfile }' /var/log/packages/* | LC_ALL=C sort -rh  > /dev/null

real    0m0.022s
user    0m0.011s
sys     0m0.016s

# get-pkgsize (also invokes nextfile )

[konrad@kjhlt5 compat32pkg]$ time get-pkgsize  > /dev/null

real    0m0.040s
user    0m0.028s
sys     0m0.019s

# since we're being anal about runtimes <G>, here's grep with the -m1 arg:

[konrad@kjhlt5 compat32pkg]$ time grep -m1 UNCOMPRESSED /var/log/packages/* | awk -F: '{print $3,$1}' | LC_ALL=C sort -rh > /dev/null

real    0m0.014s
user    0m0.005s
sys     0m0.015s

# and here's fgrep with the same -m1 flag

[konrad@kjhlt5 compat32pkg]$ time fgrep -m1 UNCOMPRESSED /var/log/packages/* | awk -F: '{print $3,$1}' | LC_ALL=C sort -rh > /dev/null

real    0m0.018s
user    0m0.009s
sys     0m0.022s
 
2 members found this post helpful.
Old 04-04-2011, 09:51 AM   #21
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware64-15.0
Posts: 6,357

Rep: Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739Reputation: 2739
Quote:
# since we're being anal about runtimes
OMG, not me. Just posted my observation after a few trials on an idle whim.
Thankyou for the insightful investigation and explanation.
Quote:
not that the times of 0.02 secs vs 0.14 secs mean anything in the real world where the one-liner will be invoked
So true! I cannot blink that fast.
 
Old 04-04-2011, 03:05 PM   #22
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694
Quote:
Originally Posted by prol View Post
pkgtool just lists it in alphabetical order.

Thanks
If slakware had RPM, you could use:

Code:
rpm -qa --queryformat="%{size} %{name}-%{version}-%{release}\n" | sort -rn
 
Old 04-04-2011, 03:09 PM   #23
brixtoncalling
Member
 
Registered: Jul 2008
Location: British Columbia
Distribution: Slackware current
Posts: 403

Rep: Reputation: 67
Quote:
Originally Posted by szboardstretcher View Post
If slakware had RPM, you could use:

Code:
rpm -qa --queryformat="%{size} %{name}-%{version}-%{release}\n" | sort -rn
And if ArchLinux used apt-get then you could ... Talk about unhelpful posts.
 
Old 04-04-2011, 04:24 PM   #24
kjhambrick
Senior Member
 
Registered: Jul 2005
Location: Round Rock, TX
Distribution: Slackware64 15.0 + Multilib
Posts: 2,159

Rep: Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512Reputation: 1512
And I wouldn't have had all the fun I had writing get-pkgsize
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I view all of my packages and dependencies? ktek Linux - Software 2 09-25-2008 04:13 AM
Treemap view for installed packages? grayFalcon Linux - Software 5 07-05-2007 10:42 PM
View directories size rickylim Linux - General 3 07-18-2006 11:24 PM
how to view which packages are from which repo? kushalkoolwal Debian 6 05-11-2006 09:59 PM
what packages to view chinese? ralphb Debian 2 09-24-2004 09:58 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 03:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration