LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 09-07-2011, 11:56 AM   #1
Brian_Hayward
LQ Newbie
 
Registered: May 2009
Posts: 15

Rep: Reputation: 0
sed question


I am learning to write bash scripts, and i created a little project for myself to give me practice in what i've already learned. I am trying to format a text file with sed in a script set to run everyday at a given time. the first 10 or so lines are fluff and can be deleted, so i use read to loop through each line until i find the text that signifies the end of lines to be deleted; also, as i loop through each line i increase a variable by one as shown below:

Code:
while read line; do
	if ! $line="some text" then;
		i=$i++   	
	fi
done < /home/myname/Documents/Testfile.txt
looking through some sed examples on the internet i found the following example for deleting the first 10 lines of text:

Code:
cat /home/myname/Documents/Testfile.txt | sed '1,10d'
the number of lines down from the top i need to delete won't always be a fixed number, which is why i am using the loop to find the line of text i want to delete up to, my problem is that i don't understand how to use my variable $i in the above example. I tried this:

Code:
cat /home/myname/Documents/Testfile.txt | sed '1,$id'
but it didn't work. Any suggestions, or hints would be greatly appreciated. Thanks.
 
Old 09-07-2011, 12:01 PM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,028
Blog Entries: 5

Rep: Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790Reputation: 790
You need to encapsulate your variable so the shell will understand where the variable name starts and stops. By having the d just after the name the shell is thinking the variable is $id instead of just $i.

Instead of "$id" use "${i}d". The brackets make it clear where the variable name starts and stops.
 
1 members found this post helpful.
Old 09-07-2011, 05:54 PM   #3
trey85stang
Senior Member
 
Registered: Sep 2003
Posts: 1,090

Rep: Reputation: 41
single quotes take everything literally;

sed '1,$id' is not the same as sed "1,$id"

Code:
id=10
echo $id
10
echo '$id'
$id
so replace your single quotes with double quotes and it will work fine.
 
Old 09-07-2011, 07:11 PM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
As MensaWater pointed out though, "$i" is the variable, and "d" is the sed command for deletion. The string "$id" would be read as a single variable name and cause an error. So you need to make sure to use the full variable form to differentiate them.

( Or else just stick a space between them; sed won't mind. )

Also, useless use of cat. sed can take the file name as an argument.
Code:
sed "1,${i}d" /home/myname/Documents/Testfile.txt
 
1 members found this post helpful.
Old 09-08-2011, 10:35 AM   #5
Brian_Hayward
LQ Newbie
 
Registered: May 2009
Posts: 15

Original Poster
Rep: Reputation: 0
Thank you all for the responses. I've caught this same mistake in other scripts I've conjured up, so I'm a little embarrassed I didn't catch it this time around...10th time is the charm maybe. anyway, for anybody who finds this post helpful, my final version of the the code i used to delete $i (number) of lines of a text file is:

Code:
while read line; do
	if ! $line="some text" then;
		i=$i++   	
	fi
done < /home/myname/Documents/Testfile.txt

sed 1,${i}d /home/myname/Documents/Testfile.txt
Please note that i left off all quotes and it still worked...i will have to go back and review how quoting works in bash scripts to see if one way works better than the next.
 
Old 09-08-2011, 12:26 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Read these three links for a better understanding of how the shell handles arguments and whitespace:

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Based on this, you should really be quoting the entire expression, to ensure that the shell passes the whole thing to sed as a single unit:
Code:
sed "1,${i}d" /home/myname/Documents/Testfile.txt
Not only does it protect variables and other shell-reserved characters, it also helps to visually differentiate the expression on the line, making it more readable.


By the way, it's possible to what you want entirely inside sed, too.
Code:
sed '0,/some text/ {/some text/! d}' /home/myname/Documents/Testfile.txt
To break it down:

Code:
0,/some text/
This matches from the beginning of the file to the first line that contains the text pattern. /../ defines a regex match. 0 is a gnu sed extension that lets you process the range, even if the match is on the first line. 1 wouldn't work correctly in such cases.

Code:
{...}
Run the grouped commands on the range matched above.

Code:
/some text/! d
! reverses the outcome of the regex match, so that only lines that don't match get deleted. This way you can exclude the line that contains the text from being deleted.

Here are a few useful sed references.
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt

I've found the sed faq to be very useful for figuring out how to do things like this.

Last edited by David the H.; 09-09-2011 at 11:47 AM. Reason: just noticed a mistake
 
  


Reply

Tags
bash scripting, sed


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sed question wondergirl Programming 21 02-22-2008 03:42 AM
SED question qipman Solaris / OpenSolaris 4 12-26-2007 05:29 AM
[sed] "Advanced" sed question(s) G00fy Programming 2 03-20-2006 01:34 AM
SED question zivota Linux - General 8 07-26-2005 07:07 PM
sed question sphynx Programming 6 04-28-2004 07:25 PM


All times are GMT -5. The time now is 12:00 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration