Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
03-22-2017, 04:32 PM
|
#1
|
Member
Registered: Mar 2010
Posts: 458
Rep:
|
sed regex assistance
Hi Linux Gurus,
I need your help, i have set of files spread over few directories
example
a/b/*simplifies*
p/q/*simplifies*
x/y/*simplifies*
Now in each of *simplifies* property file there is a property which has a value like
database.password=password
user_name_password=somepassword
username.password=anotherpassword.
I want to create a script but on low level need help with sed regex which will get me the following outcome
database.password=[ENCRYPTION_ENABLED]password[ENCRYPTION_ENABLED]
user_name_password=[ENCRYPTION_ENABLED]somepassword[ENCRYPTION_ENABLED]
username.password=[ENCRYPTION_ENABLED]anotherpassword[ENCRYPTION_ENABLED]
I believe the key is *password=*is to be replaced with
*password={somestring}*{somestring}
Would greatly appreciate your help.
Many Thanks
|
|
|
03-22-2017, 06:05 PM
|
#2
|
LQ Veteran
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,439
|
i) are you changing file names or file contents ?.
ii) show real data (several lines at least) and desired result - munged in need, but what the actual data looks like.
iii) what have you tried, and why are you having trouble.
|
|
|
03-27-2017, 07:32 AM
|
#3
|
Member
Registered: Mar 2010
Posts: 458
Original Poster
Rep:
|
I am trying to change file content not names. Since its software related I am unable to show some real data.
I tried few things but unable to achieve the outcome I am looking in my original post.
Any assistance would be sincerely appreciated.
|
|
|
03-27-2017, 07:47 AM
|
#4
|
Moderator
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,968
|
Hi sysmicuser,
Once again it will be helpful to advise LQ members what you have tried and what you have in mind for how to approach this.
You say "script". Do you have any experience writing scripts? And in which language?
Please describe in more detail the few things you have tried and a representation of the outcome. You say you cannot share more details about the files. A good thing to do here then is to share representative examples which are very close, or describe better.
The approach you've describe in your first post is somewhat helpful, however if these are passwords, then they may all be unique, they also may be non-printable binary, or they may be hashed information, once again making them all unique. Therefore people can recommend possible solutions like finding all strings containing "password=" and then adding your new bracketed string after it. And put a condition that you also can add your bracketed string at the end of the same line you modify. This all likely can be accomplished with a combination of a script, and using any, or all of sed, awk, or tr. The issue here is whether or not the representative descriptions shown are accurate enough and cover all search specs for strings you wish to modify.
|
|
|
03-27-2017, 12:26 PM
|
#5
|
Senior Member
Registered: Dec 2011
Location: Simplicity
Distribution: Mint/MATE
Posts: 3,052
|
A substitute command for sed
Code:
s/^\([-_.[:alpha:]]*password=\)\(.*\)/\1[ENCRYPTION_ENABLED]\2[ENCRYPTION_ENABLED]/
It substitutes the string password= prefixed with only letters or - _ . characters.
It uses backreferences, i.e. \1 restores what matched in the 1st \( \) and so on.
Last edited by MadeInGermany; 03-27-2017 at 12:27 PM.
|
|
1 members found this post helpful.
|
03-30-2017, 08:23 AM
|
#6
|
Member
Registered: Mar 2010
Posts: 458
Original Poster
Rep:
|
Looks like I will never be able to master regex in my entire life time
It is still not working as expected.
Code:
[root@vm1 ~]# cat /tmp/functional.override.propertyfile
a.b.c.d.password=tuzapassword
p.q.r.s.password=mazapassword
x,z-password=tuzapassword
l_n-_password=mazapassword
[root@vm1 ~]# sed -i s/^\([-_.[:alpha:]]*password=\)\(.*\)/\1[ENCRYPTION_ENABLED]\2[ENCRYPTION_ENABLED]/ /tmp/functional.override.propertyfile
[root@vm1 ~]# cat /tmp/functional.override.propertyfile
a.b.c.d.password=tuzapassword
p.q.r.s.password=mazapassword
x,z-password=tuzapassword
l_n-_password=mazapassword
[root@vm1 ~]#
When output should have been
Code:
a.b.c.d.password=[ENCRYPTION_ENABLED]tuzapassword[ENCRYPTION_ENABLED]
p.q.r.s.password=[ENCRYPTION_ENABLED]mazapassword[ENCRYPTION_ENABLED]
x,z-password=[ENCRYPTION_ENABLED]tuzapassword[ENCRYPTION_ENABLED]
l_n-_password=[ENCRYPTION_ENABLED]mazapassword[ENCRYPTION_ENABLED]
Any idea why it is not working ?
|
|
|
03-30-2017, 08:29 AM
|
#7
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,756
|
Did you properly enclose the sed expression in quotes?
Also, it's best not to muck around as root. It's a good way to turn small mistakes into big accidents. So it is recommended to make a copy of the data and practice as an unprivileged user, then when you have the details worked out, then try it as root.
|
|
1 members found this post helpful.
|
03-30-2017, 11:05 AM
|
#8
|
Senior Member
Registered: Dec 2011
Location: Simplicity
Distribution: Mint/MATE
Posts: 3,052
|
And add a comma to the allowed characters before the passwd=
So the allowed character set becomes [-_.,[:alpha:]]
|
|
1 members found this post helpful.
|
03-30-2017, 06:40 PM
|
#9
|
LQ Veteran
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,439
|
Personally I prefer a minimalist approach - define only what is needed.
Code:
sed -r 's#[^=]+$#[ENCRYPTION_ENABLED]&[ENCRYPTION_ENABLED]#' your.file
(# used as separated in case / is in data - use something else if it clashes)
|
|
1 members found this post helpful.
|
03-31-2017, 07:45 AM
|
#10
|
Member
Registered: Mar 2010
Posts: 458
Original Poster
Rep:
|
Many thanks to MadeInGermany, Turbocapitalist and syg00 !! Problem solved.
If I want to master this type of skill, kindly recommend any good book, tutorial that from your experience are excellent learning resources.
|
|
|
03-31-2017, 08:04 AM
|
#11
|
LQ Veteran
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,439
|
regex appeals to my perverse nature. Dunno how to teach that to anyone else ...
I learn by doing - learning how good people here did things, and trying out "what-if" thought games. For a long time I had this (or similar) pinned on my cubicle wall to keep me humble.
|
|
|
03-31-2017, 08:08 AM
|
#12
|
LQ Veteran
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,439
|
Also be aware that "regex ain't regex" - there are several forms, not all compatible with each other. And if you also have to deal with M$oft, that is a whole different ball-game - not that that's a surprise to anyone.
|
|
|
03-31-2017, 08:32 AM
|
#13
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,756
|
Quote:
Originally Posted by sysmicuser
Many thanks to MadeInGermany, Turbocapitalist and syg00 !! Problem solved.
If I want to master this type of skill, kindly recommend any good book, tutorial that from your experience are excellent learning resources.
|
As mentioned, there are several variants of regular expressions. The main two you'll find are POSIX and Perl (aka PCRE or Perl-compatible Regular Expressions). You'll find those everywhere. There are two manual pages, which are ok as a reference but maybe not the place to start for learning:
Code:
man 7 regex
man perlre
For the POSIX ones, try grymoire regular expresssions or regular-expressions.info.
Then you can move up to the more powerful and flexible Perl regular expressions. For those you pretty much have your choice of languages, if you aren't going to try perl first. Which language is your preference?
|
|
|
03-31-2017, 09:45 AM
|
#14
|
LQ Guru
Registered: Sep 2009
Location: Perth
Distribution: Arch
Posts: 10,039
|
I agree with syg00 that there is no better way to learn regex stuff than just doing it again and again. Another thing I like to do is solve using option A (let us use sed as here) and then see if I can do the same with awk,perl,ruby,python or for something simpler, try and get the string you are looking for with grep (use standard and then if need be, use -E or -P to refine in different ways)
oh ... and the perverse thing .... ditto ... I just like the challenge 
|
|
|
04-02-2017, 11:54 PM
|
#15
|
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.x
Posts: 18,444
|
As above, there are several versions of regex: I can HIGHLY recommend http://regex.info/book.html
Quote:
The Third Edition contains language-specific chapters on Perl (82 pages), Java (40 pages), .NET (34 pages), and PHP (46 pages). The first six chapters on general concepts take 282 pages. The table of contents, preface, index, etc. take up the remaining 50 pages.
|
The website contains more detail eg List of Contents etc.
|
|
|
All times are GMT -5. The time now is 09:21 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|