LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 10-11-2010, 08:57 AM   #16
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Original Poster
Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550

Quote:
Originally Posted by konsolebox View Post
I think by default awk closes all file descriptors that are still open when it exits.
That would make sense, and be the expected behavior: when exiting, close all fd's..

However, before the program fully exits, I think it does not close the fd - which I think explains why I had to close it myself in order to re-open it again while looping - otherwise, since it was already open, it couldn't open again for the next loop iteration until it had been closed first.

In short: w/o a name that one can use with close() then fd may remain open indefinitely, until exit. But with a name, one can close(name) and reset the fd so it can be opened and used again.

Make sense?

Quote:
@GrapefruiTgirl I sometimes call associative arrays as hashes or vice versa. So with internal hashes in awk, any command string can be associated with a single value.. a single fd that is.
Thanks for that too. Maybe when I read "hashes" some of the time, arrays are what's being referred to... But I think there are other things referred to as "hashes" also, are there not? Like, I'm thinking about encryption and md5's and things like that.

EDIT: konsolebox - that other post you linked to, is informative. Thanks for the reference. Why do you say file2 "necessarily needs" to be closed? Why not leave it hanging and let awk close it when it exits?

Last edited by GrapefruiTgirl; 10-11-2010 at 09:02 AM.
 
Old 10-11-2010, 04:56 PM   #17
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
Quote:
Originally Posted by GrapefruiTgirl View Post
That would make sense, and be the expected behavior: when exiting, close all fd's..

However, before the program fully exits, I think it does not close the fd - which I think explains why I had to close it myself in order to re-open it again while looping - otherwise, since it was already open, it couldn't open again for the next loop iteration until it had been closed first.

In short: w/o a name that one can use with close() then fd may remain open indefinitely, until exit. But with a name, one can close(name) and reset the fd so it can be opened and used again.

Make sense?
Yup.

Quote:
Thanks for that too. Maybe when I read "hashes" some of the time, arrays are what's being referred to... But I think there are other things referred to as "hashes" also, are there not? Like, I'm thinking about encryption and md5's and things like that.
That's because in hashing as related with encryption is also similar since in associative arrays the string are calculated with a shorter hash string then compared to many strings that matches the hash string. The data with an associated string that matches the original string is the true data owned of the original string.

Quote:
EDIT: konsolebox - that other post you linked to, is informative. Thanks for the reference. Why do you say file2 "necessarily needs" to be closed? Why not leave it hanging and let awk close it when it exits?
Because we need to read it again from the beginning if that line is met.
 
Old 10-11-2010, 06:33 PM   #18
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Original Poster
Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
Code:
time_dash (df | awk): CPU=70-80% avg
w/o               fflush(): 20.62 , 21.14 , 21.51 , 21.66 , 
w/  fflush("") after print: 20.68 , 20.68 , 20.86 , 21.22


time_bash (df | awk): CPU=70-80% avg
w/o             fflush(): 22.70 , 23.04 , 23.20 , 23.35
w/  fflush() after print: 22.56 , 23.00 , 23.01 , 23.08


time_awk (cmd |& getline): CPU50-60% avg
w/               close(cmd): 28.96 , 29.03 , 29.37 , 29.49
w/  fflush(cmd); close(cmd): 28.86 , 28.88 , 29.15 , 29.15


time_awk (cmd | getline): CPU=50-60% avg
w/                         close(cmd): 28.74 , 28.77 , 28.98 , 29.06
w/               fflush(); close(cmd): 28.88 , 29.04 , 29.22 , 31.39
w/             fflush(""); close(cmd): 28.75 , 28.79 , 28.87 , 28.99
w/ fflush("") after print; close(cmd): 28.71 , 28.76 , 28.83 , 28.91
#NOTE: using fflush(cmd) == "warning: pipe `df` opened for reading, not writing."
grail,

above are some more highly scientific test results in reply to your post #12. It's a loop of 10,000 iterations that print field $2 from the `df` input. Times are in seconds.
Any difference in execution time from using fflush(), if there is one and I'm not just seeing a placebo effect, is very small. Basically I figure, if I don't need it, I wouldn't use it.
The biggest difference, and I'm still surprised by it, is when piping `df` into awk using the shell, instead of having awk open `df` from within itself. Using the shell pipe takes 6-8 seconds off of the total elapsed time.

You also asked, "what if I leave the close(cmd) out?".
What happens is, in code that otherwise runs the 10,000 loops and works fine, will only run 1 iteration and exit completely, if the close(cmd) is removed.
 
  


Reply

Tags
awk stdin syntax


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Problem with Pango Install, make giving error 2 luisfong11 Linux - Software 4 12-29-2009 04:25 PM
Hello, I am entered here, Make me comfortable by giving support bitras LinuxQuestions.org Member Intro 1 12-13-2009 01:54 PM
'make' giving errors... bharathp Linux - Hardware 9 11-21-2009 11:12 PM
How to make 2 variables from one variable value in awk intikhabalam Linux - General 1 07-30-2008 04:32 AM
How to make extra stdin input in awk ? khaan Programming 3 07-30-2007 05:04 AM


All times are GMT -5. The time now is 12:11 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration