Trying to understand how Pipe works in this situation
 

I'm using ffmpeg to pull in an input rtmp stream and then push that out to a destination. In case the main input fails, I would like my ffmpeg process to switch to a backup input.

Unfortunately, ffmpeg CLI does not seem to have this ability built in.

So I came across a solution which uses pipes to manage this.

The input rtmp is pushed to a pipe file first.

The pipe file is then used as an input to restream to the final destination.
I;m trying to understand if this pipe file is an actual file or just a temporary construct.

Also, if it is an actual file, will it's size keep increasing as the stream continues.

Also, what are mkfifo and the exec 7 commands actually doing?

Thanks

For starters, see Named pipe in Wikipedia. Here is an introduction to named pipes that may be easier to understand.

The <> is a redirection operator present in Bash, meaning both input and output get redirected to/from $PIPE_FILE on file descriptor 7.

As for exec used for redirections, see this chapter in the Advanced Bash-Scripting Guide.

I should add that using redirections in this way is rarely needed in Bash, because it has process substitution. But when writing a POSIX-conform shell script they often come very handy.

No, it's not a file. It has an entry in a directory somewhere and this takes you to a normal inode on disk, so it has an owner, permissions, access dates and so on. But there are no data blocks. Instead, when a process accesses the "file", the kernel directs the data between the newly opened channel in that program and one in the program at the other end of the pipe.

Thank you for the links going to read up now!

So technically this would show when I do an ls?
And does it exist once the source starts "inputting" into the pipe, or only after the "sink" starts reading out of the pipe?

Yes, it behaves just like any other file. For example, if you have a traditional sysvinit startup, there will be a fifo called initctl in the /dev directory and you can list it with ls. init listens on that fifo and programs like shutdown use it to give instructions to init.
The inode exists from the moment you create the fifo. I don't know at what point the kernel creates the channel. It can certainly exist without the listening process reading from it, because it is possible up to a point to write into a blocked pipe. But as soon as the pipe is full, the write operation halts until the other program reads what was written. My guess is that there is a kernel buffer which actually receives and passes on the text. Maybe a kernel maven can tell us that.

Thanks for the info. My concern is that in my case, the input stream would be coming into the server and therefore the pipe regardless of whether I am reading out of the pipe. So when I finally do read out of the pipe, I don't want a situation where the pipe feeds "older" chunks of the stream to the next stage.

You mean "newer"? No, this is not possible. The pipe is not a black hole, no data get lost.

you can imagine that pipe as a "real" pipe (or tube). From one side you put data into it and on the other side you will take that out.
the pipe [may] exists without any data pushed/pulled.
fifo means First In First Out, so the order of the data cannot be changed inside the pipe.
https://www.softprayog.in/programmin...fifos-in-linux

I've been trying to picture this and I think I know now how pipes originated. They're just a small wrinkle on normal disk I/O.

The kernel maintains huge buffers whenever it has enough free memory for them. When it has to read from a disk, it reads much more than was requested and stores the rest in a buffer. That way it can satisfy subsequent read commands from the process without having to do any further mechanical disk access. And similarly output from a process goes into a buffer and gets written to disk at a convenient time, when the cpu is less busy. A sync function synchronises the disk with the buffers when required.

When a process wants to do I/O, the kernel assigns a file descriptor (just a number) and links it to the appropriate internal buffer. Now to create a pipe, all the kernel has to do is assign two file descriptors to such a buffer, one in read mode and one in write mode. That way the buffer needn't be linked to the disk surface in any way. Instead one program simply reads out of the buffer what the other program has written into it.