What is the Pythonic equivalent to Perl's "$| = 1" ?

rnturn · 07-14-2018, 05:33 PM

I'm pretty sure that the subject line pretty much is all you need, but just in case:

I have a Python (v3.4.5) that runs for a good long time and I've included status messages that I'd like to see displayed during execution that wind up not being displayed until some time when the system sees that you have a buffer full of text to display. So instead of seeing

Code:

Beginning data collection phase... (20 seconds pass)  Done. (Took 00:20)

I see

Code:

(20 seconds pass)
Beginning data collection phase...  Done. (Took 00:20)

(Actually, it's worse than that. I might see a couple of more "Beginning... Done" messages before the OS buffering decides to finally spit text onto the terminal.)

In ages past, we'd do something like this in our C programs:

Code:

printf( format, args);
fflush(STDOUT);

to get immediate output. In Perl, I can control this with "$| = 1" near the top of the script which causes immediate output rather than letting Perl and the OS wait until there was a buffer full of stuff to display.

Surely Python has a mechanism to force this without having to write a wrapper around prints that (somehow) does a manual flush. Right?

TIA for any tips.

UPDATE: I'm not sure how I missed this when I was poking around in the Python docs but there is a command line switch that is supposed to do, basically, what Perl's "$| = 1" statement does. I've modified my "shebang" line to be:

Code:

#!/usr/bin/python3 -u

Even after that, though, I haven't been able to make it work in my testing. Still plugging away on this.

dugan · 07-14-2018, 05:37 PM

Code:

sys.stdout.flush()

Example:

https://www.linuxquestions.org/quest...5/#post5350284

hydrurga · 07-14-2018, 05:44 PM

See here for various options:

https://stackoverflow.com/questions/...print-function

rnturn · 07-15-2018, 07:06 PM

Quote:

Originally Posted by hydrurga

See here for various options:

https://stackoverflow.com/questions/...print-function

Thanks...

One of the suggestions at that link did the trick though it seems like it's going to be a PITA to have to have to add "flush=True" to every intermediate print that's showing status when one global switch could (should, IMHO) have done the trick. BTW, my testing involving the "-u' command line switch only seemed to work with Python 2.7---anything I tried with Python3 was a bust...except the "flush=". Maybe that got fixed in 3.4.6 and later. My test script is/was:

Code:

#!/usr/bin/python3 -u

import time

def current_tod():
    return time.strftime("%H:%M:%S")

print("Starting at %s ..." % current_tod(), end=" ", flush=True)
time.sleep(5)
print("Finished at %s." % current_tod())

Remove the "flush=" parameter and nothing is seen until the "Finished" message is displayed. For now, that "-u" switch seems to be nothing but a nop.

Now I'm off to doing python3<-->psql testing. Should be, uh, "fun". That's it "fun".

syg00 · 07-15-2018, 07:31 PM

You can't use stdbuf ?

rnturn · 07-15-2018, 08:01 PM

Quote:

Originally Posted by dugan

Code:

sys.stdout.flush()

OK... that'd work but having to go that route is what I was hoping to avoid. I can think of a case where I may employ that when I translate some module I've written for Perl to a Python equivalent. Using those lower level functions really cries out for enclosing them in a black box-type module where the gorier code is hidden. I'd hate to be sprinkling all the "sys.*" function calls in higher level code. To me, anyway, it hinders understanding the processing flow---sort of like dropping into inline assembler.

pan64 · 07-16-2018, 04:21 AM

what you want is: unbuffered write (I think), probably this helps: https://stackoverflow.com/questions/...in-the-program

rnturn · 07-16-2018, 09:49 AM

Quote:

Originally Posted by pan64

what you want is: unbuffered write (I think), probably this helps: https://stackoverflow.com/questions/...in-the-program

Which, from my reading of the manpage, is what the "-u" switch was supported to provide so that you wouldn't have to include all this extra verbiage in each and every print() invocation. I'm wondering if the "-u" switch isn't being deprecated which force all these extras to be added to each use of print(). Just my HO but this is the sort of thing that makes the language ugly. I'm still learning Python (dabbled with it for some time but really digging in now) but from my limited experience, the way I/O gets done in Python is crazy: sys.*, io.* who-knows-what-else.*.

dugan · 07-16-2018, 11:34 AM

One of Python's design principles was "explicit is better than implicit".

pan64 · 07-17-2018, 12:53 AM

the unbuffered i/o is much slower. That's why we use buffers.
But in the example I sent you can reopen stdout unbuffered, therefore you do not need to modify your print statements.