LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-04-2021, 06:47 AM   #1
Faki
Member
 
Registered: Oct 2021
Posts: 147

Rep: Reputation: Disabled
Patterns in awk and sed


Have noticed that a pattern used with awk such as

Code:
charcl='^[[:space:]]*([#;!]+|@c|\/\/)[[:space:]]*'
begrec="${charcl}"'Mode: rec$'
has to be modified for use with `sed` by escaping the `()`.

Code:
charcl='^[[:space:]]*\([#;!]+\|@c\|\/\/\)[[:space:]]*' 
begrec="${charcl}"'Mode: rec$'
Is this correct? I would like to set the same pattern in `charcl` for use in both `awk` and `sed` commands.

Last edited by Faki; 11-04-2021 at 07:43 AM.
 
Old 11-04-2021, 07:18 AM   #2
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 2,385

Rep: Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843Reputation: 1843

Sed (and Grep) defaults to "basic" regex (BRE), but can be switched to "extended" regex (ERE):

This is described in the manual, see:
https://www.gnu.org/software/sed/manual/html_node/Command_002dLine-Options.html#index-_002d_002dregexp_002dextended
and
https://www.gnu.org/software/sed/manual/html_node/BRE-vs-ERE.html

Note that "extended" is relative - ERE is still limited compared to other regex engines (especially Perl/Python/Java).

Also, whilst Awk's regex is mostly the same as ERE there are some differences (e.g. Awk doesn't use "\b" for word boundaries; it uses "\y").


Last edited by boughtonp; 11-04-2021 at 07:34 AM.
 
Old 11-04-2021, 07:30 AM   #3
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 18,364

Rep: Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232
sed and awk use different regex engines. And unfortunately that depends on their versions too.
 
Old 11-04-2021, 07:47 AM   #4
Faki
Member
 
Registered: Oct 2021
Posts: 147

Original Poster
Rep: Reputation: Disabled
I can use BRE on both `awk` and `sed` with the by the backslashed versions \+, \{, \|, \(, and \).

Code:
charcl='^[[:space:]]*\([#;!]\+\|@c\|\/\/\)[[:space:]]*'

Last edited by Faki; 11-04-2021 at 08:03 AM.
 
Old 11-04-2021, 07:58 AM   #5
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 18,364

Rep: Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232Reputation: 6232
awk is a strange case, it has a lot of different implementations, with different syntax and feature set.
Probably better to use gawk. And you need to check the documentation about the syntax of regex-grouping.
 
Old 11-04-2021, 08:15 AM   #6
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 6,250
Blog Entries: 3

Rep: Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227Reputation: 3227
Perl has the best pattern matching and can be used in ways which are similar to AWK.

Code:
perl -n -e '/pattern/ && print;' some.file.txt

perl -p -e 's/old/new/;' some.file.txt

perl -a -e 'print $F[0],"\n";' some.file.txt
See "man perlrun" for the details.

I used Perl for many years before even knowing much about AWK and longer before I tried AWK in any serious manner, so if your pattern matching requirements are a bit more than AWK can handle, try Perl.
 
Old 11-04-2021, 10:52 PM   #7
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 18,024

Rep: Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658Reputation: 2658
If you really want to know regexes & the differences between the various tools (& 'pcre'), this is a great book http://regex.info/book.html amazingly readable for a topic that (on the surface) is very dry.

Last edited by chrism01; 11-09-2021 at 10:30 PM. Reason: dang - removed trailing semi-colon
 
1 members found this post helpful.
Old 11-04-2021, 11:32 PM   #8
Mechanikx
Member
 
Registered: Jul 2018
Distribution: Slackware
Posts: 346

Rep: Reputation: 254Reputation: 254Reputation: 254
Quote:
Originally Posted by chrism01 View Post
If you really want to know regexes & the differences between the various tools (& 'pcre'), this is a great book http://regex.info/book.html; amazingly readable for a topic that (on the surface) is very dry.
The link won't open correctly because there's a trailing semicolon.
 
Old 11-05-2021, 02:52 AM   #9
shruggy
Senior Member
 
Registered: Mar 2020
Posts: 3,578

Rep: Reputation: Disabled
Quote:
Originally Posted by Faki View Post
I can use BRE on both `awk` and `sed` with the by the backslashed versions \+, \{, \|, \(, and \).
No, you cannot. awk doesn't understand BRE. But you can use ERE for both by specifying sed -E or sed -r.
 
  


Reply

Tags
bash


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed inside awk or awk inside awk maddyfreaks Linux - Newbie 4 06-29-2016 01:10 PM
[SOLVED] Once again... awk.. awk... awk shivaa Linux - Newbie 13 12-31-2012 04:56 AM
[SOLVED] Search multiple patterns & print matching patterns instead of whole line Trd300 Linux - Newbie 29 03-05-2012 07:41 PM
How do I replace the text between patterns located on separate lines? (sed, awk, etc) Quon Programming 5 02-12-2012 06:27 AM
Remembering patterns and printing only those patterns using sed bernie82 Programming 5 05-26-2005 05:18 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration