Perl replace text in file

ShaqDiesel · 08-22-2006, 07:26 PM

I know how to do substitutions in perl, but how do I have the changed text actually REPLACE the text in the file I am reading from, sed style? So far I have been doing the search and replace, but not really replacing it because I am writing it out to another file.

macemoneta · 08-22-2006, 07:30 PM

This should do it. It's a common perl "one-liner".

Code:

#!/bin/sh

#
# greplace
#
# Globally replace one string with another in a set of files
#

/usr/bin/perl -p -i -e "s/$1/$2/g" $3

ShaqDiesel · 08-22-2006, 07:54 PM

Can you explain this please, this doesn't look like perl(is this a separate file?). Thanks.

macemoneta · 08-22-2006, 08:04 PM

The above script executes perl as a one-liner, using shell to parse the substitution parameters. The perl part is:

/usr/bin/perl -p -i -e "s/$1/$2/g" $3

Where:

Quote:

-p
causes Perl to assume the following loop around your program, which makes it iterate over filename arguments somewhat like sed:

LINE:
while (<>) {
... # your program goes here
} continue {
print or die "-p destination: $!\n";
}

If a file named by an argument cannot be opened for some reason, Perl warns you about it, and moves on to the next file. Note that the lines are printed automatically. An error occurring during printing is treated as fatal. To suppress printing use the -n switch. A -p overrides a -n switch.

BEGIN and END blocks may be used to capture control before or after the implicit loop, just as in awk.

-i[extension]
specifies that files processed by the <> construct are to be edited in-place. It does this by renaming the input file, opening the output file by the original name, and selecting that output file as the default for print() statements. The extension, if supplied, is used to modify the name of the old file to make a backup copy, following these rules:

If no extension is supplied, no backup is made and the current file is overwritten.

If the extension doesn't contain a *, then it is appended to the end of the current filename as a suffix. If the extension does contain one or more * characters, then each * is replaced with the current filename. In Perl terms, you could think of this as:

($backup = $extension) =~ s/\*/$file_name/g;

-e commandline
may be used to enter one line of program. If -e is given, Perl will not look for a filename in the argument list. Multiple -e commands may be given to build up a multi-line script. Make sure to use semicolons where you would in a normal program.

"s/$1/$2/g"
performs the string substitution

$3
is the filename glob

Additional command-line documentation is here.

Ygrex · 08-23-2006, 02:59 AM

heh, i'd write script for vim

because i don't know perl

ShaqDiesel · 08-24-2006, 01:08 PM

Thank you for taking the time to explain, but I must be doing something wrong. Here's what I did:
1.Deleted first line of my perl program(line that says where to look for perl)
2.added your #!/bin/sh, followed w//usr/bin/perl -p -i -e "s/$1/$2/g" $3.
3.wrote while(<>) with the body being my original prog that reads from a file, does the substitution, and writes it to a new file.
4.added your continue statement and block

When I run it the program just hangs. Any ideas? Thanks.

macemoneta · 08-24-2006, 01:34 PM

All you need is this:

/usr/bin/perl -p -i -e "s/$1/$2/g" $3

For example, let's say you want to change all occurrences of "rabbit" to "blueberry" in all ".html" files in the current directory. You would:

Code:

/usr/bin/perl -p -i -e "s/rabbit/blueberry/g" *.html

Nibbl3r · 02-10-2009, 09:22 AM

what if i dont want to have such a script directly on the command line?
Will it work if i just embed it into my script? I tried to put it on the top but that gives me out errors! can somebody give me some additional information how this works in a perl script ?

Telemachos · 02-10-2009, 10:29 AM

(1) It is a terrible idea to use the -i flag on the command line (or its in-script equivalent $^I) without specifying a backup suffix. If you specify a backup suffix (like this, -i.bak), then for essentially no effort you get (1) in-place editing and (2) your original backed up using the suffix you specify. (The suffix can be whatever you like.) If you just say -i, you get no backups. Given how easy it is to make a mistake, this is a bad idea.

(2) Here's how to get in-place editing with a backup in a Perl script:

Code:

#!/usr/bin/env perl
use strict;
use warnings;

$^I = '.bak'  # Call for in-place editing; make backups with a .bak suffix

while (<>) {
  s/foo/bar/
  print;
}

Obviously, the details of the substitutions can get far more complex and you don't have to use a while loop the way I did, but the key is that you turn on in-place editing with the $^I variable, and you print out the lines you edit after you make whatever substitutions you want. (You can also turn on in-place editing by having -i in the shebang line #!/usr/bin/perl -i, but I prefer to use the $^I variable. I find it easier to spot right away.)

Wow - I just realized that the original post here was two years old.

raimizou · 04-23-2009, 12:25 PM

Thanks for replying to this old post

Your answer actually helped me.

I am trying to modify a text file named old.txt on which I want to apply a large number of substitution commands (s/foo/bar/g).

Unfortunately, I am a complete newbie in Perl. Basing on your post I have defined the following script

#!/usr/bin/perl -w
use strict;
while (<>)
{
s/foo1/bar1/sg;
s/foo2/bar2/sg;
}

I tried to run the script with perl my_scrip.pl new.txt but the script does not seem to do want I want it to do.. (In fact it does nothing...)

Any help would be greatly appreciated. Thanks

Respectfully,

A Perl newbie

Sergei Steshenko · 04-23-2009, 02:40 PM

Quote:

Originally Posted by raimizou

Thanks for replying to this old post

Your answer actually helped me.

I am trying to modify a text file named old.txt on which I want to apply a large number of substitution commands (s/foo/bar/g).

Unfortunately, I am a complete newbie in Perl. Basing on your post I have defined the following script

#!/usr/bin/perl -w
use strict;
while (<>)
{
s/foo1/bar1/sg;
s/foo2/bar2/sg;
}

I tried to run the script with perl my_scrip.pl new.txt but the script does not seem to do want I want it to do.. (In fact it does nothing...)

Any help would be greatly appreciated. Thanks

Respectfully,

A Perl newbie

You start from a wrong end.

First write a script which:

1) opens input file for reading;
2) opens output file for writing;
3) copies line by line input file into output file in a loop.
4) closes the files - for clarity/cleanliness, though not strictly necessary in your case.

After that works, i.e. after you understand how to read from files and write to files, modify 3) in order to perform the needed replacement in the line the loop body has just read from input file and about to be written into output file.

raimizou · 04-24-2009, 05:27 AM

Thanks for you help Serguei. I proceeded as you advised me to do. However, I still have some problems. Here is my script:

Code:

!/usr/bin/perl -w

# Print the value of the command line arguments
$numArgs = $#ARGV + 1;
print "You provided $numArgs arguments\n";
print "Input file is $ARGV[0]\n";
print "Output file is $ARGV[1]\n\n";

# Open input file in read mode
open INPUTFILE, "<", $ARGV[0] or die $!;
# Open output file in write mode
open OUTPUTFILE, ">", $ARGV[1] or die $!;

# Read the input file line by line
while (<INPUTFILE>) {

  $_ =~ s/ foo / bar /g;
  print OUTPUTFILE $_; 
}


close INPUTFILE;
close OUTPUTFILE;

I apply it on a simple text file that contains the following text:

Code:

 foo foo foo
 foo

The result is :

Code:

 bar foo bar
 bar

I would like to have :

Code:

 bar bar bar
 bar

ghostdog74 · 04-24-2009, 08:03 AM

ok i am not sure why you want to have spaces when you want to change foo to bar. this should be good enough right?

Code:

$_ =~ s/foo/bar/g;

Sergei Steshenko · 04-24-2009, 12:54 PM

Quote:

Originally Posted by raimizou

Thanks for you help Serguei. I proceeded as you advised me to do. However, I still have some problems. Here is my script:

Code:

!/usr/bin/perl -w

# Print the value of the command line arguments
$numArgs = $#ARGV + 1;
print "You provided $numArgs arguments\n";
print "Input file is $ARGV[0]\n";
print "Output file is $ARGV[1]\n\n";

# Open input file in read mode
open INPUTFILE, "<", $ARGV[0] or die $!;
# Open output file in write mode
open OUTPUTFILE, ">", $ARGV[1] or die $!;

# Read the input file line by line
while (<INPUTFILE>) {

  $_ =~ s/ foo / bar /g;
  print OUTPUTFILE $_; 
}


close INPUTFILE;
close OUTPUTFILE;

I apply it on a simple text file that contains the following text:

Code:

 foo foo foo
 foo

The result is :

Code:

 bar foo bar
 bar

I would like to have :

Code:

 bar bar bar
 bar

ghostdog74 has already correctly suggested the fix, and since '$_' is used, one can even write

Code:

s/foo/bar/g;

I suggest to reread

perldoc perlretut

, specifically the part about character meanings (what is taken literally and what not) in regular expressions.

You are still not using 'use strict;' and in such a manner inviting a lot of trouble.

raimizou · 04-29-2009, 04:40 AM

Thanks all for your help. I now have a functional script (it works at least on the simple "foo bar" example that I posted before), but the performance of this script is quite bad (while loops...). Do you have any idea on how to improve the code pasted below ? Thanks again for your valuable help.

Gilles

Code:

#!/usr/bin/perl -w
use strict;


#-----------------#
#     PREAMBLE    #
#-----------------#


# Configuration variables
my $correctLaTeX = $ARGV[2];
my $verbose = 0; # 0 = false

# Print the value of the command line arguments
if ($verbose){
  my $numArgs = $#ARGV + 1;
  print "You provided $numArgs arguments\n";
  print "Input file is $ARGV[0]\n";
  print "Output file is $ARGV[1]\n";
  print "Modification files is $ARGV[2]\n\n"
}

#-----------------#
# MAIN OPERATIONS #
#-----------------#

# Open input file in read mode
open INPUTFILE, "<", $ARGV[0] or die $!;
# Open output file in write mode
open OUTPUTFILE, ">", $ARGV[1] or die $!;

#$modif = "s/ foo / bar /g";

# Read the input file line by line :
while (my $input_line = <INPUTFILE>) {
  # remove the end of line character
  # Open the list of corrections in read mode
  open CORRECTIONFILE, "<", $correctLaTeX or die $!;	
  # Read the modification file line by line :
  while (my $modif = <CORRECTIONFILE>){
    # Remove the comments	  
    $modif =~ s/#.*$// ;
    if ($modif =~ /^[ 	]*$/) {
      # Nothing to do (empty modification)
    } else {
      if ($verbose){
        print("$input_line") ;
      }
      my $counter == 0
      # Apply the modification (up to twenty times)
      while ( eval("\$input_line =~ $modif") and $counter < 20 ){
	$counter += 1;
      };
      if ($verbose){
        print("$input_line") ;
      }
    }
  }
  # Write the modified line to the output file
  print OUTPUTFILE $input_line;   
  close CORRECTIONFILE;
}

# Close the input and output files
close INPUTFILE;
close OUTPUTFILE;