[SOLVED] read a file using pipelines

gigiux · 03-12-2018, 10:30 AM

Hello,

I have written an example code for reading the content of a file maintaining the end of line format using hack from the internet. I have called the shell file "pipeTesting" and the text file to display "textExample". "pipeTesting" works if I call the file as an argument of the shell script.

However, there are cases when files are retrieved via pipelines; if I provide the text to pipeTesting using a cat command, there are no arguments at all since echo $@ does not print anything. To note that I had to use -p /dev/stdin to create a case for pipeline usage and one for argument usage.

Is there a way to display the content of the file maintaining the end of the line in case of a pipeline?

thank you.

The code looks something like this:

#!/bin/bash

if [ -p /dev/stdin ]; then
echo $@
else
while read
do
echo $REPLY
done < $1
fi
exit 0

Its application is:

$ cat textExample.txt
Much I marvelled this ungainly fowl to hear discourse so plainly,
Though its answer little meaning- little relevancy bore;
For we cannot help agreeing that no living human being
Ever yet was blessed with seeing bird above his chamber door-
Bird or beast upon the sculptured bust above his chamber door,
With such name as "Nevermore."
$ pipeTester textExample.txt
Much I marvelled this ungainly fowl to hear discourse so plainly,
Though its answer little meaning- little relevancy bore;
For we cannot help agreeing that no living human being
Ever yet was blessed with seeing bird above his chamber door-
Bird or beast upon the sculptured bust above his chamber door,
With such name as "Nevermore."
$ cat textExample.txt | pipeTester
_

BW-userx · 03-12-2018, 12:36 PM

I can tell you that you're mixing methods of input to your script.

Code:

while read
do
echo $REPLY
done < $1

output

Code:

./endlinescript accounts
Aremou Anziz-Jean-Claude-Agnila 219 1795120 1795120@bdeb.qc.ca
Bayo Ousmane 219 1795284 1795284@bdeb.qc.ca
Belattaf Nadir 219 1795280 1795280@bdeb.qc.ca
Ben-Aissia Walid 219 1795246 1795246@bdeb.qc.ca
Dekea MonhÃ©ssÃ©an-Guy-Roland 219 1795299 1795299@bdeb.qc.ca
Hasbi Safae 219 1695666 1695666@bdeb.qc.ca
Kabiena-Kuluila Roland-Kalomba 219 1695268 1695268@bdeb.qc.ca
Kastaly Youness 219 1695572 1695572@bdeb.qc.ca
Kengni-Souffo Georges 219 1795303 1795303@bdeb.qc.ca
Lebeau Ã‰ric 219 1198153 1198153@bdeb.qc.ca
Levasseur-Grenon Olivier 219 1795193 1795193@bdeb.qc.ca
Louis Patricia 219 1795285 1795285@bdeb.qc.ca
Major-Labelle Adam 219 1795126 1795126@bdeb.qc.ca
Merabet Walid 219 1795121 1795121@bdeb.qc.ca
Mokam-Fogue Sandrine-Lore 219 1795288 1795288@bdeb.qc.ca
Mougoue-Tchoumbou Michel-Richard 219 1795286 1795286@bdeb.qc.ca
Ouali Mariem 219 1795230 1795230@bdeb.qc.ca
Robert Kevin 219 1795264 1795264@bdeb.qc.ca
Roy Julie 219 1795156 1795156@bdeb.qc.ca
Severe Joanny 219 1654522 1654522@bdeb.qc.ca
Turcotte Michael 219 1795259 1795259@bdeb.qc.ca

There is no need to pipe the filename to the script. This method is already reading in off of stdin where $1 is the file to read.

I bet this is where you got you file name an example.
https://stackoverflow.com/questions/...-into-a-script

I do not completely understand what you're trying to accomplish here, when their is no need to pipe in the contents of the file into a bash script in order to read same said contents. the above code does that without the pipe.

as far as end of line?

Code:

while read F ;
do
printf "$F\n"
done < $1

gets same results

gigiux · 03-12-2018, 01:10 PM

I have never visited that page. I think there is a difference between writing scripts for arguments (that is passing arguments to the script the normal way) and one for pipeline usage since the arguments passed to the function are different. The problem is not to use one way or another, but to fix the script so that the end user can use both approaches. If I use the argument approach I properly get $1 etc, but if I use the same script in a pipeline, the order does not withstand anymore. For example, I changed the script into:

#!/bin/bash

if [ -p /dev/stdin ]; then
echo "input from pipeline: first argument is $1"
else
echo "input not from pipeline: first argument is $1"
while read
do
echo $REPLY
done < $1
fi
exit 0

and the output was:

$ pipeTesting textExample.txt
input not from pipeline: first argument is textExample.txt
Much I marvelled this ungainly fowl to hear discourse so plainly,
Though its answer little meaning- little relevancy bore;
For we cannot help agreeing that no living human being
Ever yet was blessed with seeing bird above his chamber door-
Bird or beast upon the sculptured bust above his chamber door,
$ cat textExample.txt | pipeTesting
input from pipeline: first argument is
_

PS: _ stands for empty line

BW-userx · 03-12-2018, 01:53 PM

I got that idea from you stating " using hack from the internet." and now I see your line of thought more clearly that you're wanting to write a script that can be used either way, and you want to preserve end of line on the output of the contents of the file being read?

taking from that link I gave you then modifiying it I get this which seems to be working either way

Code:

#!/bin/bash
# Check to see if a pipe exists on stdin.
if [ -p /dev/stdin ]; then
        echo "Data was piped to this script!"
        # If we want to read the input line by line
        while IFS= read line; do
                echo "Line: ${line}"
        done
        # Or if we want to simply grab all the data, we can simply use cat instead
        # cat
else
        echo "No input was found on stdin, skipping!"
        # Checking to ensure a filename was specified and that it exists
        if [ -f "$1" ]; then
        {
            while read ;
            do
                echo $REPLY
            done <$1
        }
        fi
fi

piping

Code:

cat accounts | ./endlinescript

using $1

Code:

./endlinescript accounts

where the two files are in the same directory.
the basic idea behind it is, if it is one thing loop it through that way, if it is a different way loop it through the other way.

MadeInGermany · 03-12-2018, 07:12 PM

I would rather test for args, and either call a function

Code:

#!/bin/bash

readloop(){
while read
do
  echo "$REPLY"
done
}

if [ -n "$1" ]
then
  readloop <"$1"
else
  readloop
fi

exit 0

or use exec to redirect stdin to the arg

Code:

#!/bin/bash

if [ -n "$1" ]
then
  exec <"$1"
fi

while read
do
  echo "$REPLY"
done

exit 0

gigiux · 03-13-2018, 05:47 AM

PERFECT! exactly what I was looking for. Thank you