shell script to perform more than one command

stevo520 · 06-21-2012, 01:24 PM

I am trying to write a script that will append lines to an existing file, eliminate duplicate lines, and save to a new output file by running "sed -f [scriptname] [filename]" once the script is created correctly.

My script looks like this so far and successfully adds the lines, but creates no output file yet, as I still need it to remove duplicates before adding what is need to create the new output file.

$a\texthere\
moretexthere

I have tried using the pipe operator after my second line of text but then it errors out telling me that "moretexthere" is a bad command? Is the pipe operator the correct thing to use in a script?

unSpawn · 06-21-2012, 01:48 PM

Quote:

Originally Posted by stevo520

My script looks like this so far

I don't see any shell script? Please post it.

Reuti · 06-21-2012, 02:14 PM

He means the sed script, and the redirection of the output has to be given on the command line, not in the sed script. Although a “|” could be used there, a redirection of the output by “>” would do already.

Another way could be to use the w command in sed to write to a file directly, but this seems not to work for the a command, as it never goes to the pattern space.

NB: You don’t need the first \ after the a command.

stevo520 · 06-21-2012, 02:41 PM

The script works when I run it like this, sed -f scriptname filename

$a\texthere\
moretexthere

It adds the lines of text but only displays them in the existing file. My problem is adding a second command "uniq -u" to remove duplicate lines of text, and a third command to save it to a new file. When I use the pipe operator it reads one of my lines of text as a command which it of course cannot interpret? So if this successfully adds my text, what is the correct way to move on the next 2 commands "uniq -u" and state the output file?

David the H. · 06-22-2012, 01:54 PM

Most command line tools such as sed do not write changes back to the input file, they only print to stdout (the screen or the next command in a pipe chain). Generally you have to output the changes to a temporary file, then replace the original with it.

gnu sed does have an "-i" option which will make it write changes back to the original file, but most other tools, like uniq, don't.

In this case, you'll probably want to do something like this:

Code:

sed -f scriptname filename | uniq -u > tempfile
mv -f tempfile filename

Finally, it usually helps if you post, in sufficient detail, everything that's important to your task, the whole script, the commands used to execute it, the actual output you get, if any, and, most importantly, an example of the input data and the desired output format, so that others can replicate your results.

And when you do, please use ***[code][/code] tags*** around your code and data, to preserve formatting and to improve readability. Please do not use quote tags, colors, or other fancy formatting.

stevo520 · 06-22-2012, 04:26 PM

Sorry, but my noobness has me struggling. My intention is to have the following functions occur by running the script, which will be executed by typing ***sed -f scriptname filename*** at the command line:

a. append 2 lines of text to an existing text file
b. find duplicates
c. Use uniq to remove the duplicate lines and place the corrected list in a new file

I want everything to happen from the programming in the script itself? Am I totally out of whack, or at least a little on track?

David the H. · 06-24-2012, 09:43 AM

Do you mean that you want to run uniq inside the sed script?

Can't be done. It's a sed script because it contains commands that control sed. If you want a script that runs other shell commands as well, then you need to write a shell script that contains the entire operation.

Code:

#!/bin/bash
# appends two lines to the end of the given file,
# removes duplicate lines,
# and outputs the result to a new file
# usage is "scriptname <inputfile> <outputfile>"

# first test if input file exists
if [[ -r $1 ]]; then
	infile="$1"
else
	echo "input file isn't readable"
	exit 1
fi

# test if an output filename was supplied
if [[ -n $2 ]]; then
	outfile="$2"
else
	echo "please supply an output filename"
	exit 1
fi

# run sed and uniq on the input file, and print to the output file
sed '$a\texthere\
moretexthere' | uniq -u >"$outfile"

exit 0

Note that it might be possible to write a sed script that emulates the function of uniq, but it probably wouldn't be easy. It's not really designed for that kind of job.

Also be aware that uniq only works properly on input that's already sorted. You may want to insert a sort command in the script as well (or instead, see "sort -u").

Here are a few useful bash scripting references:
http://mywiki.wooledge.org/BashGuide
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashPitfalls
http://wiki.bash-hackers.org/scripting/newbie_traps
http://www.linuxcommand.org/index.php
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html
http://wiki.bash-hackers.org/start
http://ss64.com/bash/