How to write a terminal program that I can pipe the input into it?

peonerovv · 03-24-2022, 04:53 AM

Hi,

I want to write a simple terminal program (I'm on linux, if that's important), that will be able to take a piped input. What I mean is the same thing grep does.

Ex:
echo "I\nlike\nlinux" | grep linux
Output:
linux

I want to be able to fetch an input into my program with the pipe (|) symbol and then give arguments that will manipulate what the program does to the input. C will work just fine for it.

I'm sorry if the description is vague, I'm not an experienced programmer. I just want to have general idea of what library I'm supposed to use or how to implement it myself.
Ask questions if something is unclear.

pan64 · 03-24-2022, 04:55 AM

If I understand well that is called stdin. So it means you need to write a program which works on its stdin. I think every language can do that.

TenTenths · 03-24-2022, 05:12 AM

There are three things to be aware of here, assuming you are doing:

Code:

ProgramA | ProgramB

ProgramA writes its output to stdout and possible stderr.
When you pipe | in to ProgramB generally only stdout is passed through to ProgramB (You CAN redirect stderr so that it also gets piped, infact there's a number of ways of treating these streams)
ProgramB reads this as stdin

ProgramB can then write its output to stdout and possibly stderr.

There's a load of interesting and well explained information on this here: https://www.howtogeek.com/435903/wha...derr-on-linux/

As @pan64 says, every language I've come across has a way of reading stdin

peonerovv · 03-24-2022, 07:59 AM

Thanks! I didn't know about this part of the *nix system

sundialsvcs · 03-24-2022, 09:15 AM

Every process in Unix® or Linux® has three standard file-handles: STDIN, STDOUT, and STDERR.

When you use "pipe," the STDOUT of the first process is "piped to" the STDIN of the second.

You can also use the nomenclature 2> to redirect STDERR.

A "pipe" is actually an asynchronous queue, which appears to be an "ordinary file." Kind of like a flexible hose. If the "reader" of a pipe finds nothing to read, it will be "blocked" until the "writer" writes something, at which time it will "wake up" and read it. Likewise, a "writer" may be "blocked" if it is writing things faster than a "reader" can consume them. It's all quite magical, and the real magic is that it "simply works."

danielbmartin · 03-24-2022, 11:51 AM

Caveat: I am a recreational programmer. The code given here is a working example but is NOT claimed to be the best way to do it.

'/usr/share/dict/words' is a list of English words which is built in with many Linux systems.

I use a simple bash shell to store and run my programs.

The purpose of this post is to give you real-world tested code which may serve as a starting point for your learning curve. Good luck! Have fun!!

Code:

 
#!/bin/bash   Daniel B. Martin   Mar22

#  To execute this program, launch a terminal session and enter:
#  bash /home/daniel/Desktop/LQfiles/dbm2320.bin

# This program inspired by ...
# https://www.linuxquestions.org/questions/programming-9/
#   how-to-write-a-terminal-program-that-i-can-pipe-the-input-into-it-4175709904/

# File identification
     Path=${0%%.*}
  OutFile=$Path"out.txt"
 WordList='/usr/share/dict/words'

echo; echo 'Find five-letter words which have "e" in positions 1 and 4.'
grep "^e..e.$" $WordList >$OutFile
cat $OutFile

echo; echo 'Find seven-letter words which begin with a "q" and end with a vowel.'
grep "^q.....[aeéiou]$" $WordList >$OutFile
cat $OutFile

echo; echo 'Find five-letter words which begin and end with a vowel but have no other vowels.'
grep "^[aeéiou][^aeéiou][^aeéiou][^aeéiou][aeéiou]$" $WordList >$OutFile
cat $OutFile

echo "Normal end of job."; echo; exit

Daniel B. Martin

.

boughtonp · 03-24-2022, 12:42 PM

Quote:

Originally Posted by danielbmartin

The code given here is a working example but is NOT claimed to be the best way to do it.

Well you got one of three things right - it's not the best way to do it (ShellCheck will highlight quoting bugs).

However it's neither working (putting your name and date after #!/bin/bash breaks it), nor is it an example of what was asked for, i.e: how to read from stdin.

peonerovv:
The Bash reference manual and Greg Wooledge's Bash FAQ are two very useful sources of information for figuring out what to put in your script, and ShellCheck is a great one for figuring out why a shell script isn't doing what you expect.

You don't have to use Bash (or any shell) to work with pipelines/stdin, but the above links should help to provide an understanding that you can apply to C (or other languages).

danielbmartin · 03-24-2022, 01:11 PM

Quote:

Originally Posted by boughtonp

... However it's neither working (putting your name and date after #!/bin/bash breaks it) ...

The solution offered works fine on my computer. This is the on-screen output:

Code:

Find five-letter words which have "e" in positions 1 and 4.
eager
eased
easel
eases
eaten
eater
eaves
ebbed
edged
edger
edges
egged
egret
eider
elder
elves
embed
ember
emcee
ended
enter
erred
ester
ether
excel
expel

Find seven-letter words which begin with a "q" and end with a vowel.
quibble
quickie
quinine

Find five-letter words which begin and end with a vowel but have no other vowels.
addle
alpha
altho
amble
ample
angle
ankle
apple
extra
ochre
ultra
uncle
Normal end of job.

Daniel B. Martin

.

astrogeek · 03-24-2022, 01:41 PM

Welcome back danielbmartin, you have been absent recently!

Your shebang line is not a shebang at all because it is preceded by an empty line. If you remove the empty line it does indeed break the script because the name is interpreted as a filename argument to bash and cannot be found. This fixes it:

Code:

#!/bin/bash   
#Daniel B. Martin   Mar22

I also suspect you have conflated "pipe" and "redirection" (and I know you know better!). Your example makes use of output redirection but does not make use of pipes as an input path, which is what the original question asks.

Please try again, your examples and challenges are always interesting!

pan64 · 03-24-2022, 02:33 PM

Quote:

Originally Posted by danielbmartin

The solution offered works fine on my computer. This is the on-screen output:

You made 3 mistakes.
1. the shebang must be the very first thing in the file, so #! should be the first two chars of the file. Otherwise it will be taken as a simple comment line
2. the shebang must not contain anything else, but the shebang itself. Additional info/text/whatever is not allowed.
3. Your solution works because of these two issues (shebang appeared on the second line, therefore the extra text is ignored), and additionally you started your script with bash <filename> which makes the shebang completely pointless/useless.

Altogether these 3 issues make your script running, but otherwise it is not related to the original issue at all.
(and also, welcome back here)

danielbmartin · 03-24-2022, 03:37 PM

Quote:

Originally Posted by astrogeek

Welcome back danielbmartin, you have been absent recently!

Thank you for remembering me. I have not truly been absent. Rather, I've relegated myself to being a lurker for one simple reason: I haven't understood most recent questions so didn't attempt to offer solutions.

Quote:

Your shebang line is not a shebang at all because it is preceded by an empty line.

In fact, my shebang was the first line in the script. The appearance of a blank line was merely a botched drag-and-drop copy mistake.

Daniel B. Martin

.

danielbmartin · 03-24-2022, 03:45 PM

Quote:

Originally Posted by pan64

2. the shebang must not contain anything else, but the shebang itself. Additional info/text/whatever is not allowed.

The first line in my script is...

Code:

#!/bin/bash   Daniel B. Martin   Mar22

I guess my computer doesn't know about the "must not contain anything else" rule because it is not enforced. In fact, all my scripts have a first line in the pattern "shebang, my name, and the date in mmmyy format.

Daniel B. Martin

.

pan64 · 03-24-2022, 04:01 PM

Quote:

Originally Posted by danielbmartin

The first line in my script is...

Code:

#!/bin/bash   Daniel B. Martin   Mar22

I guess my computer doesn't know about the "must not contain anything else" rule because it is not enforced. In fact, all my scripts have a first line in the pattern "shebang, my name, and the date in mmmyy format.

Daniel B. Martin

.

That is fine. your computer definitely knows about it. It will be taken as a simple comment (and will be ignored) every time it is not found at the very beginning of the file. It is not enforced, it is your own choice (to use shebang or not).
And if that line works without error message that means it is not taken as shebang, just a comment. I guess you can check it easily yourself.

Code:

> cat a.sh
#!/bin/bash anything
echo ok

Code:

> ./a.sh
/bin/bash: anything: No such file or directory
> bash a.sh
ok

and also you can try the same thing when there is an additional line/space inserted before that shebang.

GazL · 03-24-2022, 04:15 PM

Quote:

Originally Posted by danielbmartin

The first line in my script is...

Code:

#!/bin/bash   Daniel B. Martin   Mar22

.

Yeah, don't do that, it's not safe.

Code:

$ cat ./wibble 
#!/bin/bash Daniel B Martin

for arg in "%@"
do
echo $arg
done
$ ./wibble 
/bin/bash: Daniel B Martin: No such file or directory
$ echo "echo eeeek" > "Daniel B Martin"
$ ./wibble 
eeeek
$

danielbmartin · 03-24-2022, 05:17 PM

Quote:

Originally Posted by GazL

Yeah, don't do that, it's not safe.

I'm confessing ignorance here...

What is "not safe?" What harm does it do? Does it make my code run slower? Does it make my computer vulnerable to malware?

Daniel B. Martin

.