LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-12-2018, 10:30 AM   #1
gigiux
LQ Newbie
 
Registered: Feb 2018
Posts: 15

Rep: Reputation: Disabled
read a file using pipelines


Hello,

I have written an example code for reading the content of a file maintaining the end of line format using hack from the internet. I have called the shell file "pipeTesting" and the text file to display "textExample". "pipeTesting" works if I call the file as an argument of the shell script.

However, there are cases when files are retrieved via pipelines; if I provide the text to pipeTesting using a cat command, there are no arguments at all since echo $@ does not print anything. To note that I had to use -p /dev/stdin to create a case for pipeline usage and one for argument usage.

Is there a way to display the content of the file maintaining the end of the line in case of a pipeline?

thank you.

The code looks something like this:

#!/bin/bash

if [ -p /dev/stdin ]; then
echo $@
else
while read
do
echo $REPLY
done < $1
fi
exit 0

Its application is:

$ cat textExample.txt
Much I marvelled this ungainly fowl to hear discourse so plainly,
Though its answer little meaning- little relevancy bore;
For we cannot help agreeing that no living human being
Ever yet was blessed with seeing bird above his chamber door-
Bird or beast upon the sculptured bust above his chamber door,
With such name as "Nevermore."
$ pipeTester textExample.txt
Much I marvelled this ungainly fowl to hear discourse so plainly,
Though its answer little meaning- little relevancy bore;
For we cannot help agreeing that no living human being
Ever yet was blessed with seeing bird above his chamber door-
Bird or beast upon the sculptured bust above his chamber door,
With such name as "Nevermore."
$ cat textExample.txt | pipeTester
_
 
Old 03-12-2018, 12:36 PM   #2
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: FreeBSD/Slackware-14.2+/ArcoLinux
Posts: 9,153

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
I can tell you that you're mixing methods of input to your script.
Code:
while read
do
echo $REPLY
done < $1
output
Code:
./endlinescript accounts
Aremou Anziz-Jean-Claude-Agnila 219 1795120 1795120@bdeb.qc.ca
Bayo Ousmane 219 1795284 1795284@bdeb.qc.ca
Belattaf Nadir 219 1795280 1795280@bdeb.qc.ca
Ben-Aissia Walid 219 1795246 1795246@bdeb.qc.ca
Dekea Monhésséan-Guy-Roland 219 1795299 1795299@bdeb.qc.ca
Hasbi Safae 219 1695666 1695666@bdeb.qc.ca
Kabiena-Kuluila Roland-Kalomba 219 1695268 1695268@bdeb.qc.ca
Kastaly Youness 219 1695572 1695572@bdeb.qc.ca
Kengni-Souffo Georges 219 1795303 1795303@bdeb.qc.ca
Lebeau ‰ric 219 1198153 1198153@bdeb.qc.ca
Levasseur-Grenon Olivier 219 1795193 1795193@bdeb.qc.ca
Louis Patricia 219 1795285 1795285@bdeb.qc.ca
Major-Labelle Adam 219 1795126 1795126@bdeb.qc.ca
Merabet Walid 219 1795121 1795121@bdeb.qc.ca
Mokam-Fogue Sandrine-Lore 219 1795288 1795288@bdeb.qc.ca
Mougoue-Tchoumbou Michel-Richard 219 1795286 1795286@bdeb.qc.ca
Ouali Mariem 219 1795230 1795230@bdeb.qc.ca
Robert Kevin 219 1795264 1795264@bdeb.qc.ca
Roy Julie 219 1795156 1795156@bdeb.qc.ca
Severe Joanny 219 1654522 1654522@bdeb.qc.ca
Turcotte Michael 219 1795259 1795259@bdeb.qc.ca
There is no need to pipe the filename to the script. This method is already reading in off of stdin where $1 is the file to read.

I bet this is where you got you file name an example.
https://stackoverflow.com/questions/...-into-a-script


I do not completely understand what you're trying to accomplish here, when their is no need to pipe in the contents of the file into a bash script in order to read same said contents. the above code does that without the pipe.

as far as end of line?

Code:
while read F ;
do
printf "$F\n"
done < $1
gets same results

Last edited by BW-userx; 03-12-2018 at 12:41 PM.
 
Old 03-12-2018, 01:10 PM   #3
gigiux
LQ Newbie
 
Registered: Feb 2018
Posts: 15

Original Poster
Rep: Reputation: Disabled
I have never visited that page. I think there is a difference between writing scripts for arguments (that is passing arguments to the script the normal way) and one for pipeline usage since the arguments passed to the function are different. The problem is not to use one way or another, but to fix the script so that the end user can use both approaches. If I use the argument approach I properly get $1 etc, but if I use the same script in a pipeline, the order does not withstand anymore. For example, I changed the script into:

#!/bin/bash

if [ -p /dev/stdin ]; then
echo "input from pipeline: first argument is $1"
else
echo "input not from pipeline: first argument is $1"
while read
do
echo $REPLY
done < $1
fi
exit 0


and the output was:

$ pipeTesting textExample.txt
input not from pipeline: first argument is textExample.txt
Much I marvelled this ungainly fowl to hear discourse so plainly,
Though its answer little meaning- little relevancy bore;
For we cannot help agreeing that no living human being
Ever yet was blessed with seeing bird above his chamber door-
Bird or beast upon the sculptured bust above his chamber door,
$ cat textExample.txt | pipeTesting
input from pipeline: first argument is
_


PS: _ stands for empty line
 
Old 03-12-2018, 01:53 PM   #4
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: FreeBSD/Slackware-14.2+/ArcoLinux
Posts: 9,153

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
I got that idea from you stating " using hack from the internet." and now I see your line of thought more clearly that you're wanting to write a script that can be used either way, and you want to preserve end of line on the output of the contents of the file being read?

taking from that link I gave you then modifiying it I get this which seems to be working either way
Code:
#!/bin/bash
# Check to see if a pipe exists on stdin.
if [ -p /dev/stdin ]; then
        echo "Data was piped to this script!"
        # If we want to read the input line by line
        while IFS= read line; do
                echo "Line: ${line}"
        done
        # Or if we want to simply grab all the data, we can simply use cat instead
        # cat
else
        echo "No input was found on stdin, skipping!"
        # Checking to ensure a filename was specified and that it exists
        if [ -f "$1" ]; then
        {
            while read ;
            do
                echo $REPLY
            done <$1
        }
        fi
fi
piping
Code:
cat accounts | ./endlinescript
using $1
Code:
./endlinescript accounts
where the two files are in the same directory.
the basic idea behind it is, if it is one thing loop it through that way, if it is a different way loop it through the other way.

Last edited by BW-userx; 03-12-2018 at 02:01 PM.
 
1 members found this post helpful.
Old 03-12-2018, 07:12 PM   #5
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 1,203

Rep: Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549
I would rather test for args, and either call a function
Code:
#!/bin/bash

readloop(){
while read
do
  echo "$REPLY"
done
}

if [ -n "$1" ]
then
  readloop <"$1"
else
  readloop
fi

exit 0
or use exec to redirect stdin to the arg
Code:
#!/bin/bash

if [ -n "$1" ]
then
  exec <"$1"
fi

while read
do
  echo "$REPLY"
done

exit 0

Last edited by MadeInGermany; 03-13-2018 at 08:10 AM. Reason: syntax: fi
 
Old 03-13-2018, 05:47 AM   #6
gigiux
LQ Newbie
 
Registered: Feb 2018
Posts: 15

Original Poster
Rep: Reputation: Disabled
PERFECT! exactly what I was looking for. Thank you
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
trouble using stdin/pipelines/python rich123 Linux - Newbie 5 10-14-2012 07:51 PM
Q: are pipelines processed in parallel/serial? jamesj629 Linux - General 9 01-19-2008 02:33 AM
Bi-directional pipelines ggeeoo Linux - Newbie 2 09-03-2006 03:02 PM
pixel pipelines? tnelson42345 Linux - Hardware 1 11-12-2005 12:25 PM
6800LE unlocking pipelines and shaders Sapphire Linux - Software 1 12-29-2004 02:16 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:13 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration