LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-13-2016, 02:33 PM   #1
tynix
LQ Newbie
 
Registered: Nov 2016
Posts: 3

Rep: Reputation: Disabled
Process does not close on broken pipe


I am starting following command from a bash script (arguments omitted for simplicity)

avconv | sox | nc

I am starting about 150 such commands on the same box at the same time.

The last nc command sends the stream to another host. When that host dies, nc dies but avconv and sox can stay alive. I don't know exactly why there is no sigpipe to kill the whole chain. Maybe the system is highly contended and all buffers are full and waiting for some reason? How can I set up this pipe chain so that all processes terminate if any of the 3 processes terminate?
 
Old 11-13-2016, 02:39 PM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,122

Rep: Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270
that should be done "automatically", so I would say you do not need to do anything.
From the other hand I think (so it is my own opinion) if you start 150 chains of commands you can be confused about the result.
(just a comment, nc is not trivial, it can be netcat, nedit client or something else, I assumed it was something like netcat)
If you need more help you need to give more information (like what's happened on the other side).
 
Old 11-13-2016, 03:03 PM   #3
tynix
LQ Newbie
 
Registered: Nov 2016
Posts: 3

Original Poster
Rep: Reputation: Disabled
The problem is that it does not happen automatically. I'm not sure no data is being written or there is another issue, but when the target host goes down, I see 0 instances of nc (netcat) and lots and lots of instance of avconv and sox left. They don't die until I do a killall.

My problem is that my re-try mechanism does not work when the target host goes down because only nc terminates, but the other processes don't. Therefore, the parent process stays alive and there is no re-start.
 
Old 11-13-2016, 04:58 PM   #4
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 2,961

Rep: Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268
Are those processes actively writing anything to their output pipes? Each process must discover its own broken pipe (the pipes are entirely separate), but a broken pipe is only discovered when a process attempts to write to that pipe.
 
Old 11-14-2016, 12:35 AM   #5
tynix
LQ Newbie
 
Registered: Nov 2016
Posts: 3

Original Poster
Rep: Reputation: Disabled
They should be, but even if they don't, I would like them to close. Is there a way to bind the lifetimes of these processes together so that they exit together without strings attached?
 
Old 11-14-2016, 02:01 AM   #6
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,122

Rep: Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270Reputation: 2270
in that case I think you can reproduce that problem easily, you only need to "cut the wire". I do not know how was nc configured, but actually nc should catch the event (something like connection lost) and you can also handle it:
Code:
set -e
set -o pipefail
# call them in a subshell
avconv | sox | nc
# check "$PIPESTATUS[@]"
if returncode != 0 kill all of the child processes
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
SSH broken pipe kills foreground process xeon123 Linux - General 3 04-20-2015 01:41 PM
PHP shell_exec() always gives error "Broken pipe" when using pipe burek Linux - Server 1 01-19-2012 07:04 AM
[SOLVED] How to handle a broken pipe exception (SIGPIPE) in FIFO pipe? zyroot998 Programming 5 03-03-2011 09:10 PM


All times are GMT -5. The time now is 12:47 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration