LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-23-2010, 11:50 AM   #1
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
How to process multi-line regular expressions?


How to make tools like sed operate on the whole file, instead of line-by-line?
 
Old 02-23-2010, 05:23 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,374

Rep: Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383
You mean
Code:
sed -e 's:\/home\/contcym\/ems_184:\/usr\/local\/ems:' filename >newfile
sed -i -e 's:\/home\/contcym\/ems_184:\/usr\/local\/ems:' filename
1st creates a new file, 2nd does an in-place copy. There are other ways, see http://www.grymoire.com/Unix/Sed.html
 
Old 02-23-2010, 05:37 PM   #3
neonsignal
Senior Member
 
Registered: Jan 2005
Location: Melbourne, Australia
Distribution: Debian Stretch (Fluxbox WM)
Posts: 1,389
Blog Entries: 52

Rep: Reputation: 355Reputation: 355Reputation: 355Reputation: 355
You can set up a multiline pattern space in sed to do the match on.
 
Old 02-24-2010, 05:20 PM   #4
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Original Poster
Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
@neonsignal:

How to do it in the command line, without a sed script file?

@chrism01:

I mean process the whole file in one go, instead of working on each line separately.
 
Old 02-24-2010, 05:23 PM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,374

Rep: Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383
That's what my examples do....
 
Old 02-24-2010, 05:26 PM   #6
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Original Poster
Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
Quote:
Originally Posted by chrism01 View Post
That's what my examples do....
But they look just like normal sed usage.
 
Old 02-24-2010, 05:41 PM   #7
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,374

Rep: Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383
Have you tried them or read the link?
 
Old 02-24-2010, 07:12 PM   #8
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
by default, SED works one line at a time. Any method of working the whole file, including the ideas posted here, will involve removing the line breaks so that the whole file is one big line.

But--please describe the specific situation---there may be a better way.
 
Old 02-24-2010, 07:19 PM   #9
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
Maybe you could provide more information about the type of replacement you need to do.
Generally, if a pattern might take place across two or more lines, you build up the lines in the Hold space with the H command, or use `N' to build up the lines in the pattern space. Once you have a group of lines in the Hold space you need, you can get them (or exchange them) to the pattern space. At this point a substitute command can use `\n' in the RHS pattern. Bear in mind, that you now have to consider patterns that include newlines. You may need to use braces to group commands to run after building up the lines in the pattern space if you need to deal with more than one type of pattern.

Your problem may not be that the pattern to replace is across several lines or the entire file, but that you want to process lines that are in a range of lines, based on a pattern.

Last edited by jschiwal; 02-24-2010 at 07:23 PM.
 
Old 02-24-2010, 11:24 PM   #10
neonsignal
Senior Member
 
Registered: Jan 2005
Location: Melbourne, Australia
Distribution: Debian Stretch (Fluxbox WM)
Posts: 1,389
Blog Entries: 52

Rep: Reputation: 355Reputation: 355Reputation: 355Reputation: 355
You can use awk and perl to slurp up whole files by changing the record separator, though this doesn't scale well (since the whole file ends up in the buffer).

For example:
Code:
awk '{gsub(/line1\nline2/,"replacement"); print}' RS='\f' test.txt
and

Code:
perl -pe 's/line1\nline2/replacement/g' -0777 test.txt
For the awk example I've used the formfeed '\f' on the assumption that it isn't used in the file. Using a null '\0' will work with some versions of awk.

Last edited by neonsignal; 02-24-2010 at 11:29 PM.
 
Old 02-25-2010, 07:06 AM   #11
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Original Poster
Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
My main application was to replace all newlines "\n" with spaces " ".

But I still think it would be useful for some other things.

I haven't finished reading that sed tutorial yet, I have almost gotten to the "N" command.
 
Old 02-25-2010, 10:03 AM   #12
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Original Poster
Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
I wonder itf there are any easier sed tutorials, because with that one I don't really unsderstand much after the "s" command.

I found this page:

http://ilfilosofo.com/blog/2008/04/2...h-and-replace/

It works, but I don't have the slightest clue how.
 
Old 02-25-2010, 12:37 PM   #13
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
grymoire is the best SED tutorial I have ever seen--the only issue is that it does not have the Gnu extensions.

Why not tell us which points you don't understand?
Disclosure: It took me a LONG time to learn SED. But then, I am old......
 
Old 02-25-2010, 12:42 PM   #14
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
Quote:
I found this page:

http://ilfilosofo.com/blog/2008/04/2...h-and-replace/

It works, but I don't have the slightest clue how.
It simply appends each line to the hold buffer until it gets to the last line. Now the whole file is in the hold buffer and it can then be moved back to the working buffer and processed as one line.

I feel your pain---it's pretty easy to write a SED script that will take several bottles of Excedrin to decipher....
 
Old 02-25-2010, 12:43 PM   #15
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Original Poster
Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
I found a lot of other sed tutorials, I will just start reading.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Regular expressions bhuwan Programming 5 02-25-2006 11:07 PM
regular expressions. stomach Linux - Software 1 02-10-2006 06:41 AM
Regular Expressions markjuggles Programming 2 05-05-2005 11:39 AM
Regular Expressions overbored Linux - Software 3 06-24-2004 02:34 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:36 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration