LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-22-2017, 04:32 PM   #1
sysmicuser
Member
 
Registered: Mar 2010
Posts: 437

Rep: Reputation: 0
sed regex assistance


Hi Linux Gurus,

I need your help, i have set of files spread over few directories
example
a/b/*simplifies*
p/q/*simplifies*
x/y/*simplifies*

Now in each of *simplifies* property file there is a property which has a value like

database.password=password
user_name_password=somepassword
username.password=anotherpassword.

I want to create a script but on low level need help with sed regex which will get me the following outcome

database.password=[ENCRYPTION_ENABLED]password[ENCRYPTION_ENABLED]
user_name_password=[ENCRYPTION_ENABLED]somepassword[ENCRYPTION_ENABLED]
username.password=[ENCRYPTION_ENABLED]anotherpassword[ENCRYPTION_ENABLED]

I believe the key is *password=*is to be replaced with
*password={somestring}*{somestring}

Would greatly appreciate your help.

Many Thanks
 
Old 03-22-2017, 06:05 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,783

Rep: Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574
i) are you changing file names or file contents ?.
ii) show real data (several lines at least) and desired result - munged in need, but what the actual data looks like.
iii) what have you tried, and why are you having trouble.
 
Old 03-27-2017, 07:32 AM   #3
sysmicuser
Member
 
Registered: Mar 2010
Posts: 437

Original Poster
Rep: Reputation: 0
I am trying to change file content not names. Since its software related I am unable to show some real data.

I tried few things but unable to achieve the outcome I am looking in my original post.

Any assistance would be sincerely appreciated.
 
Old 03-27-2017, 07:47 AM   #4
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,354
Blog Entries: 13

Rep: Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411
Hi sysmicuser,

Once again it will be helpful to advise LQ members what you have tried and what you have in mind for how to approach this.

You say "script". Do you have any experience writing scripts? And in which language?

Please describe in more detail the few things you have tried and a representation of the outcome. You say you cannot share more details about the files. A good thing to do here then is to share representative examples which are very close, or describe better.

The approach you've describe in your first post is somewhat helpful, however if these are passwords, then they may all be unique, they also may be non-printable binary, or they may be hashed information, once again making them all unique. Therefore people can recommend possible solutions like finding all strings containing "password=" and then adding your new bracketed string after it. And put a condition that you also can add your bracketed string at the end of the same line you modify. This all likely can be accomplished with a combination of a script, and using any, or all of sed, awk, or tr. The issue here is whether or not the representative descriptions shown are accurate enough and cover all search specs for strings you wish to modify.
 
Old 03-27-2017, 12:26 PM   #5
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 1,768

Rep: Reputation: 797Reputation: 797Reputation: 797Reputation: 797Reputation: 797Reputation: 797Reputation: 797
A substitute command for sed
Code:
s/^\([-_.[:alpha:]]*password=\)\(.*\)/\1[ENCRYPTION_ENABLED]\2[ENCRYPTION_ENABLED]/
It substitutes the string password= prefixed with only letters or - _ . characters.
It uses backreferences, i.e. \1 restores what matched in the 1st \( \) and so on.

Last edited by MadeInGermany; 03-27-2017 at 12:27 PM.
 
1 members found this post helpful.
Old 03-30-2017, 08:23 AM   #6
sysmicuser
Member
 
Registered: Mar 2010
Posts: 437

Original Poster
Rep: Reputation: 0
Unhappy

Looks like I will never be able to master regex in my entire life time

It is still not working as expected.
Code:
[root@vm1 ~]# cat /tmp/functional.override.propertyfile
a.b.c.d.password=tuzapassword
p.q.r.s.password=mazapassword
x,z-password=tuzapassword
l_n-_password=mazapassword
[root@vm1 ~]# sed -i s/^\([-_.[:alpha:]]*password=\)\(.*\)/\1[ENCRYPTION_ENABLED]\2[ENCRYPTION_ENABLED]/ /tmp/functional.override.propertyfile
[root@vm1 ~]# cat /tmp/functional.override.propertyfile
a.b.c.d.password=tuzapassword
p.q.r.s.password=mazapassword
x,z-password=tuzapassword
l_n-_password=mazapassword
[root@vm1 ~]#
When output should have been
Code:
a.b.c.d.password=[ENCRYPTION_ENABLED]tuzapassword[ENCRYPTION_ENABLED]
p.q.r.s.password=[ENCRYPTION_ENABLED]mazapassword[ENCRYPTION_ENABLED]
x,z-password=[ENCRYPTION_ENABLED]tuzapassword[ENCRYPTION_ENABLED]
l_n-_password=[ENCRYPTION_ENABLED]mazapassword[ENCRYPTION_ENABLED]
Any idea why it is not working ?
 
Old 03-30-2017, 08:29 AM   #7
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,676
Blog Entries: 3

Rep: Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908
Did you properly enclose the sed expression in quotes?

Code:
sed 's/a/b/'
Also, it's best not to muck around as root. It's a good way to turn small mistakes into big accidents. So it is recommended to make a copy of the data and practice as an unprivileged user, then when you have the details worked out, then try it as root.
 
1 members found this post helpful.
Old 03-30-2017, 11:05 AM   #8
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 1,768

Rep: Reputation: 797Reputation: 797Reputation: 797Reputation: 797Reputation: 797Reputation: 797Reputation: 797
And add a comma to the allowed characters before the passwd=
So the allowed character set becomes [-_.,[:alpha:]]
 
1 members found this post helpful.
Old 03-30-2017, 06:40 PM   #9
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,783

Rep: Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574
Personally I prefer a minimalist approach - define only what is needed.
Code:
sed -r 's#[^=]+$#[ENCRYPTION_ENABLED]&[ENCRYPTION_ENABLED]#' your.file
(# used as separated in case / is in data - use something else if it clashes)
 
1 members found this post helpful.
Old 03-31-2017, 07:45 AM   #10
sysmicuser
Member
 
Registered: Mar 2010
Posts: 437

Original Poster
Rep: Reputation: 0
Many thanks to MadeInGermany, Turbocapitalist and syg00 !! Problem solved.

If I want to master this type of skill, kindly recommend any good book, tutorial that from your experience are excellent learning resources.
 
Old 03-31-2017, 08:04 AM   #11
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,783

Rep: Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574
regex appeals to my perverse nature. Dunno how to teach that to anyone else ...

I learn by doing - learning how good people here did things, and trying out "what-if" thought games. For a long time I had this (or similar) pinned on my cubicle wall to keep me humble.
 
Old 03-31-2017, 08:08 AM   #12
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,783

Rep: Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574
Also be aware that "regex ain't regex" - there are several forms, not all compatible with each other. And if you also have to deal with M$oft, that is a whole different ball-game - not that that's a surprise to anyone.
 
Old 03-31-2017, 08:32 AM   #13
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,676
Blog Entries: 3

Rep: Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908Reputation: 2908
Quote:
Originally Posted by sysmicuser View Post
Many thanks to MadeInGermany, Turbocapitalist and syg00 !! Problem solved.

If I want to master this type of skill, kindly recommend any good book, tutorial that from your experience are excellent learning resources.
As mentioned, there are several variants of regular expressions. The main two you'll find are POSIX and Perl (aka PCRE or Perl-compatible Regular Expressions). You'll find those everywhere. There are two manual pages, which are ok as a reference but maybe not the place to start for learning:

Code:
 
man 7 regex
man perlre
For the POSIX ones, try grymoire regular expresssions or regular-expressions.info.

Then you can move up to the more powerful and flexible Perl regular expressions. For those you pretty much have your choice of languages, if you aren't going to try perl first. Which language is your preference?
 
Old 03-31-2017, 09:45 AM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,821

Rep: Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084
I agree with syg00 that there is no better way to learn regex stuff than just doing it again and again. Another thing I like to do is solve using option A (let us use sed as here) and then see if I can do the same with awk,perl,ruby,python or for something simpler, try and get the string you are looking for with grep (use standard and then if need be, use -E or -P to refine in different ways)

oh ... and the perverse thing .... ditto ... I just like the challenge
 
Old 04-02-2017, 11:54 PM   #15
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,873

Rep: Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600
As above, there are several versions of regex: I can HIGHLY recommend http://regex.info/book.html
Quote:
The Third Edition contains language-specific chapters on Perl (82 pages), Java (40 pages), .NET (34 pages), and PHP (46 pages). The first six chapters on general concepts take 282 pages. The table of contents, preface, index, etc. take up the remaining 50 pages.
The website contains more detail eg List of Contents etc.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Need help with Sed regex Almaz Linux - Newbie 8 07-27-2014 02:39 AM
[SOLVED] sed with RegEx danielbmartin Programming 4 08-06-2012 09:40 PM
[SOLVED] sed regex schneidz Programming 1 02-28-2011 06:46 PM
[SOLVED] In need of some SED assistance EricTRA Linux - Newbie 7 07-22-2010 01:46 PM
regex with sed to process file, need help on regex dwynter Linux - Newbie 5 08-31-2007 05:10 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:55 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration