In a thread about a specific problem, pan64 raised a potential general issue with pipes in BASH. I'm suggesting that this is a better place to discuss such a concern, beyond the specific problem that was considered solved in its own thread.

pan64, in what way do you believe that a pipe with no data is not handled properly? How should it be handled, and why?

Might work with all Bourne-derived shells. Still it is a misuse.
In tcsh you get
"Ambiguous output redirect."
That's what it is.
Correct is
The ; (or a newline) chains two commands.
The | chains two commands by connecting the stdout of the first command with the stdin of the second command.

From talking with pan64, AFAIK, both pan64 and I are well aware of what a pipe is, and neither pan64 nor I, would choose to use a pipe in a meaningless way, in our own code.

From pan64's comment which I quoted, I get the idea that pan64 believes the situation is not handled properly within BASH.

For me, if the operator works the way the manual says it's supposed to work, then it's handled properly. So if the BASH man page, or info file, doesn't say BASH should complain about that situation, and when a pipe is used with data with BASH, the data flows through the pipe properly, to me it's handled properly, for BASH.

If I'm being paid to build a computer program for someone, and I being told to do something that seems wrong to me, but what I'm being told to do, is in the specification, then if I do something different than the specification, strictly speaking, my code has a bug. However wrong, what the specification says, might seem to me, the specification defines how the code is supposed to work. If the code works contrary to the specification, the code has a bug.

In a sense, I'm somewhat glad that tcsh detects the potential ambiguity, and complains about it. But if the official BASH documentation does not say that BASH will complain about it, etc., then it's not necessarily wrong for BASH.

Of course, if the situation is an oversight on the part of the BASH developers, then obviously it could be a architectural or design flaw. :-) In such a case, if and when, the manual were to be changed, then what would be proper for BASH, would change.

what you missed it is not "pure" bash. Using pipe you will have a construct like this:
bash will start all the commands, will create all the pipes and passes those file descriptors to the commands. But the commands will take care about their own stdin/stdout/stderr and it only depends on how they were implemented.
For example the command mv (most probably) simply ignores stdin (as far as I know), but in case of any other implementation it may cause an error. Also cat will not send anything to its stdout and I do not really know how a pipe can handle if the process just terminates without handling that pipe. And how the next command will handle the possible sigpipe event.
Again, it is not related to bash, bash will not take care about it (how the children will use the pipes). pipe is not an operator, like || and &&

Is mv fixed now so it won't overwrite a directory with a file. It used to be that 'mv file dir' would just change dir into a file. So, you had to use 'mv file dir/'.

edit, I tried it and the slash isn't necessary. But what if you actually want to overwrite a directory with a file?

It *could* be implemented, but it is complicated because the mv command needs to recursively delete the directory.
A similar complex thing is already implemented in most mv commands: move across file systems. This is a recursive copy with restoring all attributes, plus a recursive deletion of the source, and all is an "atomic" operation.

The workaround is easy, two mv commands: 1. rename the directory then 2. move the file. Then 3. rm -r the directory.

pan64,

I feel as if we seem to be getting into an a debate about semantics, if that wasn't already the case. If you are suggesting that pipe is not an operator at all within BASH, then please note, from the BASH manual page: so according to the BASH manual page, to BASH, the "pipe character" i.e. | is an operator. If you are suggesting that pipe is not conditional, in the same sense that || and && are, I never suggested that pipe is a conditional. That distinction seemed to me, as if it might be the focus of your original concern. You indicated that something wasn't handled properly; I thought you meant that BASH wasn't establishing the pipe properly; I may have misinterpreted what you said, as I feel you misinterpreted something that I said. In the following comment of yours, exactly what is it that you think is not being handled properly, and what is it that's not handling the case properly?

ok, it looks like pipe is an operator, just we don't know what does it really mean (also please do not mix pipe and pipeline).
So yes, you are right, probably that was not the correct word. pipe is not a || or && like operator, but something different.
bash is not a language used alone, but a lot of external command executed (like sed, find, grep, awk, cat, cut, and several thousands of others) inside a script (but obviously you can write a pure bash script without invoking external binaries - || and && may work like this), but | will/can work only on external commands.
What you still missed is: bash has no influence on how the commands work and actually bash does not take care about that at all. It will just set stdin/stdout/sterr and an environment to the child process and waits for the exit code. Nothing more. So the execution of a pipeline depends on how those commands work and how they handle these goodies. If you construct a pipe but nothing will be put into it and nothing will be read from it the pipe is not used properly (the process which should send data does not care at all if there is a pipe here, the process which should read data does not care if there is anything in the pipe). It may cause strange things, like sigpipe or other signals (like hangup), but actually "nothing strange" may happen too, but this case is more or less unpredictable.

so constructing a pipe with commands which does not handle pipe will not only waste resources but is wrong and unsafe.

OK, we've accomplished something.

If you check the BASH manual page, you should discover that it refers to BASH's builtin's as commands. So where exactly does it say that pipe will work only with external commands? It seems to me that I've used the "echo" builtin, with a pipe.


Quite apart from the idea of my missing anything, I keep trying to apply what you are saying to the specific situation, and it does not seem to apply. So I keep guessing as to what specifics about which you are thinking, you might not have mentioned.

Once a pipe is established, the next step in the sequence of events necessary for a SIGPIPE to occur, is for something to write to a pipe. In this specific case: since |& is not used, stderr of the cat command is not redirected to the pipe. Since > redirects stdout of the cat command to file2, not the pipe, cat's stdout is not going to write to the pipe. Since nothing will write to the pipe, SIGPIPE will not occur, in this specific case.


Surely the kernel is the authority on what's right and what's wrong. If it doesn't restrict a person from doing something, who are we to say it is wrong? The kernel seems to have no problem with the use of the pipe in this particular case. Also, BASH doesn't complain about it; so it seems consistent with the "language"/etc., which BASH considers right.

How then can you justify claiming that it is wrong?

Use of the pipe in this specific case may be pointless, it does seem to waste some resources, but to say it's wrong, seems just plain incorrect.

Is it unsafe?

The "std" in stdin, stdout, and stderr, represents the word "standard" for a good reason; multiple computer programming languages arrange for a program written in those languages to automatically have those three file descriptors without the programmer having to explicitly create those descriptors. If you check, I believe you'll find that a rather large percentage of the commonly available Linux commands, external to BASH, are written in one of those languages.

In this specific case, unless in the future, someone goes to the trouble of deliberately closing some of those descriptors in some way that affects this specific case, then the use of the pipe in this specific case, is not unsafe.

If instead you are thinking of things in the abstract, in much more general terms, yet you don't seem to have said so, then it would also appear to me that you are artificially restricting that much more general situation, in some way which isn't clear.

If you are thinking in terms of someone using some command other than mv, considering a very large percentage of the commonly available Linux commands, external to BASH, would by default, have the three standard file descriptors, what's the probability that there would be a situation such that, if a command other than mv were used, a problem would occur? It would seem quite small; not particularly unsafe.

You've also talked about what BASH handles and doesn't handle. What you said is more or less valid in the specific type of situation presented in the original newbie forum thread, from which this current discussion derives. Yet if you are thinking about a much more general case, what prevents someone from using the type builtin, in which case BASH does care-about/handle things which you say it does not.

So, unless overall you are thinking in some much more general context and not describing it, if what you are thinking applies to this specific situation, it seems hard to see how it applies correctly.

This is simply false.
From one side obviously there are built-in commands which produces output and that can be piped into something, but in any case if you use pipe you will unconditionally run new processes, every and each command in a pipechain will run independently from all of the others (the only exception can be the very first command, which can use the current shell).

I don't realyl understand what are you talking about, what does not seem to apply?

this is simply false again. SIGPIPE means a process wants to write into a pipe, but there is no process [reading it] at the other end. In the example pipe was never established propery, because noone reads the content.

In this specific case I have no idea what will be sent into the pipe, most probably nothing, but it will be definitely created. From the other hand a command like this: will definitely write something into a pipe which is not handled at all and based on the definition a SIGPIPE could have been acceptable here. But actually the implementation of mv is wrong, it simply does not handle/ignores stdin and therefore unable to send sigpipe.

This is false again. kernel has no any right to decide if your code is ok or not. It will simply do what was requested - if possible.

Kernel will allow to do a lot of things which has no real meaning. Programmatically correct, but logically useless/meaningless/pointless.

This logic is false. bash does not complain, because bash cannot decide. But that does not mean consistency. It only means there is no correctly implemented error handling. Nothing more.

because it is wrong.


As I explained it works only because pipe handling in mv is a bit strange. It may change in the future and (although it is unlikely) it may break this script.


as I tried to explain, it is neither related to stdin/out/error itself, nor the pipe, nor the kernel, but the implementation of pipe handling in a command which actually does not handle pipe at all.

pan64,

Although I'm using the rules of the English language when I'm structuring my comments, and when I interpret responses, Human language can be ambiguous at the best of times. I try to find something that is helpful in almost anything that anyone says. Since I'm only Human, I make mistakes, plenty of them. If at first something that someone else says doesn't seem accurate to me, maybe I'm misinterpreting what was said. Or perhaps I didn't understand the perspective from which it was said.

But ultimately we are dealing with a technical discipline, so if we are talking about something of fairly limited scope, as we are now, we ought to be able to check what we say to be sure that what we say is accurate, because everyone makes mistakes. I typically use LQ to try to help people. We are not helping people if we make false claims. So in a situation such as this, I generally check everything, before I comment; I checked everything in this situation before I commented. But, we might be using very different versions of BASH, etc. If I'm interpreting things correctly, I seem to be using BASH version 4.3. Maybe it would help if we focus on just one sequence of comments and responses.

At one point in the discussion, you said this:
this was my response to that:
and this seems to be your response to those comments of mine:
To clarify, when I've been using the phrase "external command" I mean a command that is not a BASH built-in. I'm not talking about whether or not the command might run in a sub-shell.

If I understand you correctly, you seem to think that the first command in what the BASH manual page calls a pipeline ( which you seem to call a pipechain ) runs in the main BASH shell, rather than a sub-shell. In the BASH I'm using, the first command in a "pipechain" seems to run in a sub-shell. Here's the BASH program that appears to establish that:

Here an output from a run of that trivial program with the BASH I'm using:

So, are you a using different version of BASH? If you run the BASH program I've included with the BASH you're using, does it instead output the same PID twice? If it outputs two different PID's, that seems to mean you are incorrect about the first command in a "pipechain" using the current shell. If the BASH program produces two different PID's with the BASH you are using, then what made you think that the first command in a "pipechain" runs in the main shell?

So what you say doesn't seem to apply to the version of BASH that I am using, and in general the Linux environment that I am using; if I understand you correctly, it would seem that I get different results than what you seem to claim that I should.

I think you still miss some parts:
running a subshell means it is a new process. Actually it is a bash, not a grep/sed/whatever, but it is another process anyway.
What I wanted to say: pipe cannot work within a single bash script, it will unconditionally start another process even if the commands on the right side are shell built-ins.
So a bash built-in can be (and will be) an external command if it was called using another bash process. In short we say "subshell".

the version of bash is irrelevant here.

Please read and consider this thoroughly.

Since I've written some of the operating system kernel code for some forms of Unix, I'm well aware of the relationships between processes and sub-processes. Although Linux is not Unix, the source code is different ( apart from a few elements of some distro's that may have adapted some portions of the so-called BSD 4.4 "lite" ), it would certainly seem that many Unix concepts apply quite well to Linux. After all, there are good reasons why Linux has many times been referred to as "Unix-like". So no, I'm not missing the relationships between processes and sub-processes.

That's completely outside the question I asked, the issue I raised.

I specifically stated how I was using the term "external". Did you read my statement, which I've quoted below?

I also understand that you are using the term "external command" very literally, and in a manner that to me, does not seem consistent with the way the BASH manual uses it. At least not the version of the BASH manual which I am using.

Regardless of all that, right now I'm focusing on a particular portion of one particular comment that you made:

even more specifically, I'm focusing on the particular exception that you claimed here:


If I understand that particular claim that you've made, it would seem that you think the first command in what you call a "pipechain" can run in the main BASH shell. If that's what you've been saying, then it would seem that either you are just plain wrong, or the version of BASH which you are using works differently, than the version I'm using.

If you believe that the first command in what you call a "pipechain" runs in the main BASH shell, then can you indicate what made you think that?

If that is not what you meant, then in what way do you think the first command in the "pipechain" can "use" the "current shell" any differently than any other commands in the "pipechain"???

I think you are completely lost. You need to understand what I was writing (and do not try to explain me that I misunderstood you).
1. bash creates the pipe itself.
2. pipe (as file descriptor) is passed to the [sub-]processes. The only exception can be the very first command in a pipe chain which can be the current bash shell, but otherwise all the commands specified in a pipe chain are new processes, not the [current] bash itself.
3. pipes are handled by these subprocesses, not by the shell, so this is the answer to the original question (Does BASH handle piping properly when there is no data to pipe?). The current shell can only catch the exit code and signals of those subprocesses.
4. in your original question the first command is a cat with a redirection which will not write into the pipe anything and the second command is a mv which does not even try to handle stdin. Therefore neither cat nor the mv command will inform the parent process about the fact: here is a pipe which wasn't used properly.
5. as it was explained in post #2 it is a misuse.

Sorry about that, but I have been attempting to understand what you are writing, from the beginning in the original newbie forum thread. If my interpretation of what you have written, has matched what you intended to say, your comments about SIGPIPE and some exception for the first command in a "pipe chain", seem just plain wrong, when I've actually tested them on the Linux I am using. That's why I have focused on possible differences between Linux environments, or potential miscommunication.

If the title I set for the thread was too confining, given what I wished to accomplish, I do apologize. To try to clarify, my main reasons for starting this thread are because I thought you had claimed that in this situation, or a similar situation:

1) a SIGPIPE might be received by one of the commands in the "pipe chain", due to the way the pipe is being used.
2) Some other unexpected problem might occur.
3) Using BASH to create a pipe between commands which do not know how to handle pipes, is wrong.

I had wished to discuss such things, and comments you made relating to those ideas.

I thought, given the way you worded things, that you had originally associated SIGPIPE with BASH handling pipes.

To try to make very clear the distinction between what's understood/agreed versus not understood or not agreed, I'll go through the nice summary you provided.


We agree on that much, and have from the beginning.


Are you suggesting that the BASH process itself is part of the "pipe chain" ??? If so, in what way do you feel that the BASH process is part of the "pipe chain", and not just the parent process?



Here we might have a linguistic issue. "handling a pipe" seems to me to be a rather general concept. I would tend to think that a program which does not know how to "handle a pipe", could not create a pipe; that creating a pipe is part of handling a pipe. In that sense, I would tend to say that BASH does handle pipes, even if only to create pipes.


That's pretty much what I've been saying.



cat and mv do not inform the parent about what fact?


As has already been agreed, in the particular code example being used, the use of a pipe seems to accomplish nothing positive. It seems to waste resources, so overall seems rather pointless.

But beyond that, some of the comments that you've made, seemingly have expressed the idea that a pipe shouldn't be used with commands that don't know how to handle pipes, due to some risk for a SIGPIPE, etc., which you seem to feel exists in such a situation.

If I write a program that knows how to handle pipes, I would tend to have it create a pipe; the program would typically not need BASH to create a pipe for it. So I suspect that the vast majority of commands for which BASH has actually been used to create pipes, do not know how to "handle pipes"; that they do not make pipe specific calls themselves, but just general calls to read data from, and write data to, general purpose file descriptors. If that's ambiguous, then to be more exact, as a simple example, a C language program handling file descriptors at a general level, would tend to close a file descriptor using a close function call, rather than using the pclose function; pclose would be used if the program had been written specifically to handle pipes.

You seem to have also expressed some concern about mv ( or other command in a situation such as this, or somewhat similar situation ) not reading data from a pipe, that had already been written to the pipe. When I tested your SIGPIPE concerns by writing programs written with any reasonable basic understanding of how to write and read data, to and from, general purpose file descriptors, but NO function calls specific to pipes, no SIGPIPE occurred. That was true even when in the second program in the "pipe chain", I deliberately did not read data from the pipe, that had already been written to the pipe, by the first command.