LinuxQuestions.org - [SOLVED] sed question

- Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)

- - sed question (https://www.linuxquestions.org/questions/linux-general-1/sed-question-901607/)

I am learning to write bash scripts, and i created a little project for myself to give me practice in what i've already learned. I am trying to format a text file with sed in a script set to run everyday at a given time. the first 10 or so lines are fluff and can be deleted, so i use read to loop through each line until i find the text that signifies the end of lines to be deleted; also, as i loop through each line i increase a variable by one as shown below:

Code:

while read line; do

        if ! $line="some text" then;

                i=$i++          

        fi

done < /home/myname/Documents/Testfile.txt

looking through some sed examples on the internet i found the following example for deleting the first 10 lines of text:

Code:

cat /home/myname/Documents/Testfile.txt | sed '1,10d'

the number of lines down from the top i need to delete won't always be a fixed number, which is why i am using the loop to find the line of text i want to delete up to, my problem is that i don't understand how to use my variable $i in the above example. I tried this:

Code:

cat /home/myname/Documents/Testfile.txt | sed '1,$id'

but it didn't work. Any suggestions, or hints would be greatly appreciated. Thanks.

You need to encapsulate your variable so the shell will understand where the variable name starts and stops. By having the d just after the name the shell is thinking the variable is $id instead of just $i.

Instead of "$id" use "${i}d". The brackets make it clear where the variable name starts and stops.

single quotes take everything literally;

sed '1,$id' is not the same as sed "1,$id"

Code:

id=10

echo $id

10

echo '$id'

$id

so replace your single quotes with double quotes and it will work fine.

As MensaWater pointed out though, "$i" is the variable, and "d" is the sed command for deletion. The string "$id" would be read as a single variable name and cause an error. So you need to make sure to use the full variable form to differentiate them.

( Or else just stick a space between them; sed won't mind. )

Also, useless use of cat. sed can take the file name as an argument.

Code:

sed "1,${i}d" /home/myname/Documents/Testfile.txt

Thank you all for the responses. I've caught this same mistake in other scripts I've conjured up, so I'm a little embarrassed I didn't catch it this time around...10th time is the charm maybe. anyway, for anybody who finds this post helpful, my final version of the the code i used to delete $i (number) of lines of a text file is:

Code:

while read line; do

        if ! $line="some text" then;

                i=$i++          

        fi

done < /home/myname/Documents/Testfile.txt



sed 1,${i}d /home/myname/Documents/Testfile.txt

Please note that i left off all quotes and it still worked...i will have to go back and review how quoting works in bash scripts to see if one way works better than the next.

Read these three links for a better understanding of how the shell handles arguments and whitespace:

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Based on this, you should really be quoting the entire expression, to ensure that the shell passes the whole thing to sed as a single unit:

Code:

sed "1,${i}d" /home/myname/Documents/Testfile.txt

Not only does it protect variables and other shell-reserved characters, it also helps to visually differentiate the expression on the line, making it more readable.

By the way, it's possible to what you want entirely inside sed, too.

Code:

sed '0,/some text/ {/some text/! d}' /home/myname/Documents/Testfile.txt

To break it down:

Code:

0,/some text/

This matches from the beginning of the file to the first line that contains the text pattern. /../ defines a regex match. 0 is a gnu sed extension that lets you process the range, even if the match is on the first line. 1 wouldn't work correctly in such cases.

Code:

{...}

Run the grouped commands on the range matched above.

Code:

/some text/! d

! reverses the outcome of the regex match, so that only lines that don't match get deleted. This way you can exclude the line that contains the text from being deleted.

Here are a few useful sed references.
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt

I've found the sed faq to be very useful for figuring out how to do things like this.