LinuxQuestions.org - Shell pipeline q'n

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Shell pipeline q'n (https://www.linuxquestions.org/questions/programming-9/shell-pipeline-qn-4175701773/)

Shell pipeline q'n

Hi. This command writes a status line every 5 seconds:

Code:

$ zpool iostat -n -q 5

              capacity    operations    bandwidth    syncq_read    syncq_write  asyncq_read  asyncq_write  scrubq_read  trimq_write

pool        alloc  free  read  write  read  write  pend  activ  pend  activ  pend  activ  pend  activ  pend  activ  pend  activ

----------  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----

zroot        167G  473G    398    209  17.8M  5.14M      0      0      0      0      0      0      0      0      0      0      0      0

zroot        167G  473G      0    42  19.3K  281K      0      0      0      0      0      0      0      0      0      0      0      0

^C

I'm not interested in the pool name, capacity, scrub and trim operations, so I decide to use colrm to get rid of them. First the left hand side:

Code:

$ zpool iostat -n -q 5 | colrm 1 26

 operations    bandwidth    syncq_read    syncq_write  asyncq_read  asyncq_write  scrubq_read  trimq_write

 read  write  read  write  pend  activ  pend  activ  pend  activ  pend  activ  pend  activ  pend  activ

-----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----

  398    209  17.8M  5.14M      0      0      0      0      0      0      0      0      0      0      0      0

^C

So far so good. Now for the right hand side:

Code:

$ zpool iostat -n -q 5 | colrm 1 26 | colrm 85

This doesn't output anything and hangs until Ctrl+C is pressed!

I should note that it doesn't matter what the last command is, even cat -u, even head -n1 - it will hang. But if the "feeder" is killed, the expected output is delivered:

Code:

$ timeout 15 zpool iostat -n -q 5 | colrm 1 26 | colrm 85

 operations    bandwidth    syncq_read    syncq_write  asyncq_read  asyncq_write  

 read  write  read  write  pend  activ  pend  activ  pend  activ  pend  activ  

-----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  

  398    209  17.8M  5.14M      0      0      0      0      0      0      0      0  

    0    33      0  214K      0      0      0      0      0      0      0      0  

    0      0      0      0      0      0      0      0      0      0      0      0

What the $#$% is going on? Thanks for any advice.

P.S.: this doesn't hang, so it must be something in colrm (and cut -b):

Code:

$ zpool iostat -n -q 5 | cat | cat

This is a very common problem. The output from the command is being buffered. If you wait for a while, a sudden rush of output will appear when the buffer is full (which is clearly not what you want).

Try running the zpool command via stdbuf or unbuffer, like this:

Code:

stdbuf --output=L zpool iostat -n -q 5 | colrm 1 26



unbuffer -p zpool iostat -n -q 5 | colrm 1 26

I'm sure you can find other ways of doing it by Googling something like "forcing line buffering in pipes".

Quote:

Originally Posted by Ser Olmy (Post 6290691)

Code:

stdbuf --output=L zpool iostat -n -q 5 | colrm 1 26



unbuffer -p zpool iostat -n -q 5 | colrm 1 26

I'm sure you can find other ways of doing it by Googling something like "forcing line buffering in pipes".

Damn! You're right! Thanks a lot! It was colrm doing the buffering. This helped:

zpool iostat -n -q 5 | stdbuf -o L colrm 1 26 | colrm 85

Code:

$ zpool iostat -n -q 5 | stdbuf -o L colrm 1 26 | colrm 85

 operations    bandwidth    syncq_read    syncq_write  asyncq_read  asyncq_write  

 read  write  read  write  pend  activ  pend  activ  pend  activ  pend  activ  

-----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  

  396    208  17.7M  5.12M      0      0      0      0      0      0      0      0  

    0      0      0      0      0      0      0      0      0      0      0      0  

^C