LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 09-16-2010, 08:58 AM   #1
cyatomato
LQ Newbie
 
Registered: Sep 2010
Location: France
Distribution: Ubuntu karmic
Posts: 5

Rep: Reputation: 0
How to search file by pattern and then delete corresponding lines in shell


I have such a file(test.txt):
abc 123 456
abc 256 145
axd 125 225
wsd 124 2258
wsd 12589 124
sdez 1245 1235
...........................
I want to search this file by pattern /abc/ and return the values of matched lines. Then update this file by deleting matched lines.

I use script like below:
#!/bin/bash
pattern_line=$(grep abc test.txt)
echo $pattern_lines
sed '/abc/d' test.txt >another.txt

Here, I want to make change because I search the file twice. If the file is very big, it will be slow. So can I only search the file one time and do those two things together?? And I use 'sed' here. I have to create a new file. I just want to update the file, not get a new one.

Expect your help!
thank you very much!

Last edited by cyatomato; 09-16-2010 at 08:59 AM.
 
Old 09-16-2010, 09:01 AM   #2
kurumi
Member
 
Registered: Apr 2010
Posts: 223

Rep: Reputation: 45
Code:
ruby -i.bak -ne 'print unless /^abc/' file
 
Old 09-16-2010, 09:24 AM   #3
DevonB
LQ Newbie
 
Registered: Dec 2009
Posts: 27

Rep: Reputation: 1
Do a sed inline and delete on match

sed -i 's/<text>//' <filename>

-i will do it inline
s/ does a search for the term with the next / replacing what you want replaced. So // replaces with nothing.

To delete the whole line alternatively you can do

sed -i '/<text>/d' <filename>

This will delete any matches from the file.

Devon

Last edited by DevonB; 09-16-2010 at 09:26 AM.
 
1 members found this post helpful.
Old 09-16-2010, 10:28 AM   #4
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,604

Rep: Reputation: 446Reputation: 446Reputation: 446Reputation: 446Reputation: 446
Hi,

from your script example I see that you want to display only the deleted lines on the screen. This can be done with
Code:
sed -n '/abc/ p; /abc/ ! w another.txt
' test.txt
mv another.txt test.txt
I assume that you do not have done something funny like redirecting stdout or so.

The newline after 'w another.txt' is important!
 
Old 09-16-2010, 10:47 AM   #5
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
@crts: Best solution I've seen so far, just one little comment:
Quote:
The newline after 'w another.txt' is important!
Why?
Code:
$ cat test.txt 
abc 123 456
abc 256 145
axd 125 225
wsd 124 2258
wsd 12589 124
sdez 1245 1235

$ cat blaat.sh 
#!/bin/bash
sed -n '/abc/ p; /abc/ ! w another.txt' test.txt
# eof

$ ./blaat.sh 
abc 123 456
abc 256 145

$ cat another.txt 
axd 125 225
wsd 124 2258
wsd 12589 124
sdez 1245 1235
Works on my side.....

It (w file) is a normal sed option:
Quote:
w file
Write the line to file if a replacement was done. A maximum of 10 different files can be opened.
 
1 members found this post helpful.
Old 09-16-2010, 11:26 AM   #6
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,604

Rep: Reputation: 446Reputation: 446Reputation: 446Reputation: 446Reputation: 446
Hi druuna,

you are right, it does work in this case. However, if you were to execute other sed commands after 'w another.txt' like
Code:
sed -n '/abc/ p; /abc/ ! w another.txt; some_command' test.txt
then the file to write to would be interpreted as 'another.txt; some_command'
So for the purpose of general usability I have made it a habit to insert a 'newline' after every 'w' and 'r' command. If anyone else reads the thread and thinks he can use the command with some 'tweaks' there will be no confusion when he executes additional commands after 'w'. Or if for some reason one wants to change the order like
Code:
sed -n '/abc/ ! w another.txt; /abc/ p' test.txt # won't work; sed will try to write to 'another.txt;...'
sed -n '/abc/ ! w another.txt
/abc/ p' test.txt         # will work as expected
[EDIT]
I just took a look at the man page and this behavior is not obvious from the man page. At least not the one I have.

Last edited by crts; 09-16-2010 at 11:31 AM.
 
1 members found this post helpful.
Old 09-16-2010, 12:50 PM   #7
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
@crts: You could rewrite sed -n '/abc/ ! w another.txt; /abc/ p' test.txt to sed -ne '/abc/ ! w another.txt' -e '/abc/ p' test.txt and it works again

It can be a puzzle at times and I can understand that you (and others with you) decide to take the multiple line approach, maybe it is even the better solution in some cases.
 
Old 09-16-2010, 05:42 PM   #8
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,604

Rep: Reputation: 446Reputation: 446Reputation: 446Reputation: 446Reputation: 446
Quote:
Originally Posted by druuna View Post
@crts: You could rewrite sed -n '/abc/ ! w another.txt; /abc/ p' test.txt to sed -ne '/abc/ ! w another.txt' -e '/abc/ p' test.txt and it works again
I do agree that it definitely looks nicer when a one-liner does not span over two lines. There is no argument about that. However, I do see a downside when posting advice in the forum. Someone not familiar with the fact, that everything after 'w' will be interpreted as part of the filename might still fall into that 'trap' when trying to modify the command.
So if posting in the forum I think I will stick with the 'newline' approach. It sort of emphasizes that this command needs 'special' attention if used.
Other than that I do not see any 'serious' reason to avoid the multiple '-e' solution.

As for my personal preference, I simply got used to the newline approach and I do not mind spanning it over two lines. Especially if the sed script is a bit longer I prefer putting every command on a single line for better readability.
 
Old 09-17-2010, 08:08 AM   #9
cyatomato
LQ Newbie
 
Registered: Sep 2010
Location: France
Distribution: Ubuntu karmic
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by crts View Post
Hi druuna,

you are right, it does work in this case. However, if you were to execute other sed commands after 'w another.txt' like
Code:
sed -n '/abc/ p; /abc/ ! w another.txt; some_command' test.txt
then the file to write to would be interpreted as 'another.txt; some_command'
So for the purpose of general usability I have made it a habit to insert a 'newline' after every 'w' and 'r' command. If anyone else reads the thread and thinks he can use the command with some 'tweaks' there will be no confusion when he executes additional commands after 'w'. Or if for some reason one wants to change the order like
Code:
sed -n '/abc/ ! w another.txt; /abc/ p' test.txt # won't work; sed will try to write to 'another.txt;...'
sed -n '/abc/ ! w another.txt
/abc/ p' test.txt         # will work as expected
[EDIT]
I just took a look at the man page and this behavior is not obvious from the man page. At least not the one I have.
thanks for crts.
It is indeedly a good way to do this.
thanks again!!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell script to delete file if a pattern is found and restart servers sun81 Linux - Newbie 2 07-22-2010 02:24 PM
delete multiple lines from file using shell script mech123 Linux - Newbie 4 06-09-2010 04:04 AM
How to use sed to delete all lines before the first match of a pattern? C_Blade Linux - Newbie 9 05-01-2010 04:18 AM
sed: delete lines after last occurrence of a pattern in a file zugvogel Programming 4 11-17-2009 01:49 AM
Need a bash shell script which will delete lines from file scjohnie Linux - Newbie 1 09-13-2008 08:51 PM


All times are GMT -5. The time now is 03:05 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration