LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-21-2012, 01:24 PM   #1
stevo520
LQ Newbie
 
Registered: Jun 2012
Posts: 9

Rep: Reputation: Disabled
Cool shell script to perform more than one command


I am trying to write a script that will append lines to an existing file, eliminate duplicate lines, and save to a new output file by running "sed -f [scriptname] [filename]" once the script is created correctly.

My script looks like this so far and successfully adds the lines, but creates no output file yet, as I still need it to remove duplicates before adding what is need to create the new output file.

$a\texthere\
moretexthere


I have tried using the pipe operator after my second line of text but then it errors out telling me that "moretexthere" is a bad command? Is the pipe operator the correct thing to use in a script?
 
Old 06-21-2012, 01:48 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,354
Blog Entries: 55

Rep: Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541
Quote:
Originally Posted by stevo520 View Post
My script looks like this so far
I don't see any shell script? Please post it.
 
Old 06-21-2012, 02:14 PM   #3
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 13.1
Posts: 1,328

Rep: Reputation: 254Reputation: 254Reputation: 254
He means the sed script, and the redirection of the output has to be given on the command line, not in the sed script. Although a “|” could be used there, a redirection of the output by “>” would do already.

Another way could be to use the w command in sed to write to a file directly, but this seems not to work for the a command, as it never goes to the pattern space.

NB: You don’t need the first \ after the a command.
 
Old 06-21-2012, 02:41 PM   #4
stevo520
LQ Newbie
 
Registered: Jun 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
The script works when I run it like this, sed -f scriptname filename

$a\texthere\
moretexthere


It adds the lines of text but only displays them in the existing file. My problem is adding a second command "uniq -u" to remove duplicate lines of text, and a third command to save it to a new file. When I use the pipe operator it reads one of my lines of text as a command which it of course cannot interpret? So if this successfully adds my text, what is the correct way to move on the next 2 commands "uniq -u" and state the output file?

Last edited by stevo520; 06-21-2012 at 02:43 PM.
 
Old 06-22-2012, 01:54 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960
Most command line tools such as sed do not write changes back to the input file, they only print to stdout (the screen or the next command in a pipe chain). Generally you have to output the changes to a temporary file, then replace the original with it.

gnu sed does have an "-i" option which will make it write changes back to the original file, but most other tools, like uniq, don't.

In this case, you'll probably want to do something like this:

Code:
sed -f scriptname filename | uniq -u > tempfile
mv -f tempfile filename

Finally, it usually helps if you post, in sufficient detail, everything that's important to your task, the whole script, the commands used to execute it, the actual output you get, if any, and, most importantly, an example of the input data and the desired output format, so that others can replicate your results.

And when you do, please use ***[code][/code] tags*** around your code and data, to preserve formatting and to improve readability. Please do not use quote tags, colors, or other fancy formatting.

Last edited by David the H.; 06-22-2012 at 01:56 PM.
 
Old 06-22-2012, 04:26 PM   #6
stevo520
LQ Newbie
 
Registered: Jun 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
Apologies

Sorry, but my noobness has me struggling. My intention is to have the following functions occur by running the script, which will be executed by typing ***sed -f scriptname filename*** at the command line:

a. append 2 lines of text to an existing text file
b. find duplicates
c. Use uniq to remove the duplicate lines and place the corrected list in a new file

I want everything to happen from the programming in the script itself? Am I totally out of whack, or at least a little on track?
 
Old 06-24-2012, 09:43 AM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960
Do you mean that you want to run uniq inside the sed script?

Can't be done. It's a sed script because it contains commands that control sed. If you want a script that runs other shell commands as well, then you need to write a shell script that contains the entire operation.

Code:
#!/bin/bash
# appends two lines to the end of the given file,
# removes duplicate lines,
# and outputs the result to a new file
# usage is "scriptname <inputfile> <outputfile>"

# first test if input file exists
if [[ -r $1 ]]; then
	infile="$1"
else
	echo "input file isn't readable"
	exit 1
fi

# test if an output filename was supplied
if [[ -n $2 ]]; then
	outfile="$2"
else
	echo "please supply an output filename"
	exit 1
fi

# run sed and uniq on the input file, and print to the output file
sed '$a\texthere\
moretexthere' | uniq -u >"$outfile"

exit 0
Note that it might be possible to write a sed script that emulates the function of uniq, but it probably wouldn't be easy. It's not really designed for that kind of job.

Also be aware that uniq only works properly on input that's already sorted. You may want to insert a sort command in the script as well (or instead, see "sort -u").

Here are a few useful bash scripting references:
http://mywiki.wooledge.org/BashGuide
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashPitfalls
http://wiki.bash-hackers.org/scripting/newbie_traps
http://www.linuxcommand.org/index.php
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html
http://wiki.bash-hackers.org/start
http://ss64.com/bash/

Last edited by David the H.; 06-24-2012 at 09:48 AM. Reason: fixed code mistake
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to pass command line arguments from one shell script to another shell script VijayaRaghavanLakshman Linux - Newbie 5 01-20-2012 09:12 PM
Using shell command output as input in shell script - how to do? EnderX Linux - Newbie 2 06-30-2010 12:46 PM
command with shell script sunlinux Programming 2 12-28-2006 11:36 PM
Shell Script help or proper command jmikeneedham Linux - General 2 04-30-2006 01:25 PM
Command in shell script Grassie Coetzee Linux - Software 1 03-13-2005 11:42 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration