LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-21-2008, 02:54 AM   #1
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Rep: Reputation: 0
Sed question


Hmm...have been struggling with this for awhile...

I have 2 files : file A and file B. File A contains a list of servers in a certain format, and file B contains servernames that need to be removed from File A.

File A :
======

servera yaddayaddablabla
serverb yahdhdhydhhd
serverc dhhdkkdkkdkd
serverd ddkdkkdd

........ you get the idea


File B :
=======

serverc
serverd

Question : how do I use sed to remove / delete lines from file A using the contents of FileB? Servers in File B, that have entries in File A, those entries should be deleted.

I have this code but it doesnt seem to do the work for some erason..ended up with the same file! Appreciate if you can help.

--------------------

#!/bin/sh
for i in `cat fileB`;
do
sed '/^${i}/d' fileA>newfile
mv newfile fileA
echo ${i}
done
 
Old 02-21-2008, 03:14 AM   #2
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,507

Rep: Reputation: 360Reputation: 360Reputation: 360Reputation: 360
Instead of
Code:
sed '/^${i}/d' fileA>newfile
try
Code:
sed '/^'${i}'/d' fileA>newfile
 
Old 02-21-2008, 04:20 AM   #3
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by jlliagre View Post
Instead of
Code:
sed '/^${i}/d' fileA>newfile
try
Code:
sed '/^'${i}'/d' fileA>newfile
Done your suggestion but looks like the file is still not edited

Last edited by wondergirl; 02-21-2008 at 04:22 AM.
 
Old 02-21-2008, 06:33 AM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,696
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Code:
# join -v 1 file file1
servera yaddayaddablabla
serverb yahdhdhydhhd
 
Old 02-21-2008, 07:04 AM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
To remove lines beginning with the word "opt" from file "list"

i="opt"

sed /^$i/d list

OR

sed "/^$i/d" list

In this example, quoting is not required, since there is no ambiguous meaning. If quotes are used, they must be double-quotes to allow bash to expand $i.

What was the purpose of ${i}? The curly brackets don't seem necessary.
 
Old 02-21-2008, 07:29 AM   #6
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by pixellany View Post
To remove lines beginning with the word "opt" from file "list"

i="opt"

sed /^$i/d list

OR

sed "/^$i/d" list

In this example, quoting is not required, since there is no ambiguous meaning. If quotes are used, they must be double-quotes to allow bash to expand $i.

What was the purpose of ${i}? The curly brackets don't seem necessary.
I'm just used to putting the curly brackets :-P I read somewhere that its good practice..

I did what you mentioned...it worked if I declared i="opt" like, but not if I put all the patterns I want to match, in one file, like my initial example above....
 
Old 02-21-2008, 07:45 AM   #7
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
It seems your basic loop is going to be very inefficient. For every value of "i" it makes one substitution and then writes a new file. If the files are large, it will be slow. Perhaps the solution using "join" is better.

Note:

instead of:
sed /keyword/d file > newfile
mv newfile file

how about:
sed -i /keyword/d file
 
Old 02-21-2008, 08:11 AM   #8
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by pixellany View Post
It seems your basic loop is going to be very inefficient. For every value of "i" it makes one substitution and then writes a new file. If the files are large, it will be slow. Perhaps the solution using "join" is better.

Note:

instead of:
sed /keyword/d file > newfile
mv newfile file

how about:
sed -i /keyword/d file
Hmmm...it seems like it doesnt recognize the -i value..? (I'm on Solaris 5.8 machine).

I have to read a bit about the join command because I'm not sure what that one will do...

Thanks for your comments!
 
Old 02-21-2008, 08:19 AM   #9
slakmagik
Senior Member
 
Registered: Feb 2003
Distribution: Slackware
Posts: 4,113

Rep: Reputation: Disabled
random thoughts

Quote:
Originally Posted by wondergirl View Post
I'm just used to putting the curly brackets :-P I read somewhere that its good practice..
It's really not, IMO - just extra clutter. Though I suppose you could make an argument for consistency. I only use the full ${var} form when (a) it's required, such as for arrays and parameter expansion or (b) when it's practically required, such as with ambiguous strings - ${foo}bar because 'foobar' doesn't exist.

Quote:
Originally Posted by pixellany View Post
Note:

instead of:
sed /keyword/d file > newfile
mv newfile file

how about:
sed -i /keyword/d file
pixellany - I recommend it but always note that it's not standard/portable.

Quote:
Originally Posted by ghostdog74 View Post
Code:
# join -v 1 file file1
servera yaddayaddablabla
serverb yahdhdhydhhd
ghostdog74's probably got the best/simplest approach here, as long as the files are sorted.
 
Old 02-21-2008, 08:22 AM   #10
slakmagik
Senior Member
 
Registered: Feb 2003
Distribution: Slackware
Posts: 4,113

Rep: Reputation: Disabled
Took me a long time to marshal all those quotes and I missed your reply - you already noted #2 ('-i' is a GNU extension to sed) and are looking into #3 (join). Sorry. I didn't *mean* to be redundant.
 
Old 02-21-2008, 08:38 AM   #11
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by digiot View Post
Took me a long time to marshal all those quotes and I missed your reply - you already noted #2 ('-i' is a GNU extension to sed) and are looking into #3 (join). Sorry. I didn't *mean* to be redundant.
That is OK

What do you mean I can do join as long as its sorted? Trying to read about join but would appreciate if you can explain a bit more.
 
Old 02-21-2008, 09:07 AM   #12
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Original Poster
Rep: Reputation: 0
Hmmmm this is funny, when I ran this, to check whether it can get the values from fileB and process it :

#!/bin/sh
for i in `cat fileB`;
do
sed '/^'$i'/d' fileA>newfile
echo $i
exit
done


I ended up with a newfile that was minus the line that matched first $i pattern! So why doesnt it work when it keeps going to the end of the loop???? I dotn understand why it work with only the first $i. Grrrrrrrrr...

Last edited by wondergirl; 02-21-2008 at 09:08 AM.
 
Old 02-21-2008, 10:13 AM   #13
cicorino
Member
 
Registered: May 2006
Location: Italy
Distribution: Slackware, Slackware64
Posts: 31

Rep: Reputation: 16
well if you insert an 'exit' before 'done'...
[edit: and you rewrite newfile at each iteration]

Last edited by cicorino; 02-21-2008 at 10:20 AM.
 
Old 02-21-2008, 10:15 AM   #14
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,507

Rep: Reputation: 360Reputation: 360Reputation: 360Reputation: 360
Are you sure you are really running this very script ?

Code:
#!/bin/sh
for i in `cat fileB`
do
  sed '/^'${i}'/d' fileA>newfile
  mv newfile fileA
  echo ${i}
done
There is no reason for it not to remove from fileA all the lines starting with strings from fileB.

Here is something that demonstrates it works:
Code:
#!/bin/sh
cat >fileA <<%
server1 foo
server2 bar
server3 foo
server4 bar
%
cat >fileB <<%
server1
server4
%
echo ---
echo before fileA=
cat fileA
for i in `cat fileB`
do
  sed '/^'${i}'/d' fileA>newfile
  mv newfile fileA
done
echo ---
echo after fileA=
cat fileA
output:

Code:
---
before fileA=
server1 foo
server2 bar
server3 foo
server4 bar
---
after fileA=
server2 bar
server3 foo
 
Old 02-21-2008, 05:12 PM   #15
wondergirl
LQ Newbie
 
Registered: Feb 2008
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by jlliagre View Post
Are you sure you are really running this very script ?

Code:
#!/bin/sh
for i in `cat fileB`
do
  sed '/^'${i}'/d' fileA>newfile
  mv newfile fileA
  echo ${i}
done
There is no reason for it not to remove from fileA all the lines starting with strings from fileB.

Here is something that demonstrates it works:
Code:
#!/bin/sh
cat >fileA <<%
server1 foo
server2 bar
server3 foo
server4 bar
%
cat >fileB <<%
server1
server4
%
echo ---
echo before fileA=
cat fileA
for i in `cat fileB`
do
  sed '/^'${i}'/d' fileA>newfile
  mv newfile fileA
done
echo ---
echo after fileA=
cat fileA
output:

Code:
---
before fileA=
server1 foo
server2 bar
server3 foo
server4 bar
---
after fileA=
server2 bar
server3 foo
Yes I am sure! I feel like I'm going crazy. It should work! The thing is, if I cut off the file and left off only 3 patterns as a test, it worked..

is there some sort of limitation with sed with the numbers of itireation or something?????? I'm beating my head against the wall for this!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
sed question krock923 Programming 2 04-30-2006 09:36 PM
[sed] "Advanced" sed question(s) G00fy Programming 2 03-20-2006 12:34 AM
sed question provkitir Programming 4 05-03-2005 09:41 AM
sed question from Ch 6.12 na5m Linux From Scratch 1 02-02-2005 03:36 PM
sed question 360 Programming 2 03-15-2002 09:52 AM


All times are GMT -5. The time now is 11:53 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration