LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 07-13-2010, 11:45 AM   #1
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Rep: Reputation: 15
Search and replace line in multiple files


Hello, I need some help searching through multiple files, finding a line and replacing that line. The line I am searching for is:

password key ******* 1222554

ultimately I want to be able to delete the numbers after the asterisks . my thoughts are to create a script that will search for the line password key ******* and delete it then replace it with password key *******

my files are located in /opt and they are all txt files.

Please let me know the best scripting method to get this accomplished.

Thank you,
 
Old 07-13-2010, 11:59 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
It is not clear from your example if the file contains real asterisks or you used them to mask real passwords. Anyway you can accomplish this task with a simple sed command. It might be something like this:
Code:
sed -i -r 's/^(password key .+) [0-9]+$/\1/' file
Please, pay attention to the fact that the -i option edit the file in place, actually changing its content. Better if you test it on a safe copy of the original files.

Then you can repeat the command over a bunch of files either by embedding the sed line in a while loop or by passing all the file names as multiple arguments to the sed command itself. In the first case you can do something like:
Code:
while read file
do
  sed -i -r 's/^(password key .+) [0-9]+$/\1/' "$file"
done < <(find /opt <your search criteria here>)
where the loop is fed by the result of a find command through process substitution. Hope this helps.
 
Old 07-13-2010, 12:30 PM   #3
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Original Poster
Rep: Reputation: 15
colucix thanks for your help but that didnt work. The password does contain real asterisks. should i be using a -e instead of -r? i tried the -e but there are problems with that.

Thank you,
 
Old 07-13-2010, 12:59 PM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Nope. The -r option means "make use of extended regexp" and let the + sign be interpreted as "match one or more occurrences of the previous pattern". In the suggested command line I used anchors to match the beginning ^ and the end $ of the line. Maybe there are extra characters in your lines that prevent the exact matching. What if you try without them?
Code:
sed -r 's/(password key .+) [0-9]+/\1/' file
Please, show us the results (or error messages) of your commands together with the sed version you're running, e.g.
Code:
$ cat testfile
password key ******* 1222554
password key ******* 5362578
$ sed -r 's/(password key .+) [0-9]+/\1/' testfile
password key *******
password key *******
$ sed --version
GNU sed version 4.1.5
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
 
Old 07-13-2010, 01:04 PM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
If you use "*" in a regex, it has to be escaped. (Otherwise, it means "any number of the preceding expression or character")

Untested (some pseudo-code):

Code:
for fil in ls -R; do
    if it is a file; then
         sed -i -r 's/(passwordkey\*+)[0-9]+/\1/g' $fil
    fi
done
\*+ means a group of at least one asterisks
 
Old 07-13-2010, 01:15 PM   #6
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Original Poster
Rep: Reputation: 15
cat me.txt

password key ******** 225541A222221111F8

$ sed -r 's/(password key .+) [0-9]+/\1/' me.txt

password key ********A222221111F8

$ sed --version
GNU sed version 4.1.5
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
 
Old 07-13-2010, 01:24 PM   #7
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
That is working as expected.....

Earlier you said you wanted to "delete the numbers after the asterisks".....
Do you want to:
---delete everything after the asterisks?
---delete a specific pattern after the asterisks?
---delete everything after the asterisks but before another pattern?
---other?
 
Old 07-13-2010, 01:30 PM   #8
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Original Poster
Rep: Reputation: 15
i would like to delete everything after the asterisks on that line. I have multiple lines under there but do not wish to delete any of those.

example

blah blah blah
text
password key ******** 225541A222221111F8
blah blah blah

i want this to be:

blah blah blah
text
password key ******** 225541A222221111F8
blah blah blah
 
Old 07-13-2010, 01:36 PM   #9
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Delete everything after the asterisks:

sed -r 's/(password key \*+).*/\1/' me.txt

".*" means any number of any character---if you kept it that way, it would by itself go to the end of the line, and the second ".*" would have no effect.
 
Old 07-13-2010, 01:44 PM   #10
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Original Poster
Rep: Reputation: 15
pixellany, great thank you so much, this is what i needed.
 
Old 07-13-2010, 01:51 PM   #11
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Original Poster
Rep: Reputation: 15
OK, another question. I have over 100 files that i need to run through. Can you also help me create a while loop for this?
 
Old 07-13-2010, 01:59 PM   #12
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
We already gave some examples of how to do the loops.....

The easiest case---if the files are all in one directory, and there are no sub-directories:

Code:
for fil in *; do
    <sed stuff here>
done
Try something like this first---to get familiar with the concept:

Code:
for fil in *; do
    echo $fil
done
The above loops are setup to be run in the directory where the files are. To run from anywhere, add a full pathname in front of the *.
 
Old 07-13-2010, 02:01 PM   #13
dnoy
Member
 
Registered: Nov 2007
Posts: 69

Original Poster
Rep: Reputation: 15
this seems to search multiple files:

sed -i -r 's/(password key \*+).*/\1/' *.txt

looks like i dont need a while loop
 
Old 07-13-2010, 02:11 PM   #14
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
"If it works, it's OK"
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Search values within multiple files line to line Chrizzieej Programming 5 09-26-2008 05:11 PM
Search and replace across multiple files zouriel Programming 11 12-07-2007 05:36 PM
Search and Replace with multiple-line strings ChristianNerds.com Programming 4 08-21-2005 03:32 PM
Search and Replace over multiple files The_Nerd Linux - Software 8 06-20-2004 07:59 AM
trying to search and replace text file for single & multiple line breaks separately brokenfeet Programming 7 08-29-2003 02:56 PM


All times are GMT -5. The time now is 10:02 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration