LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
 
Search this Thread
Old 01-15-2011, 09:05 AM   #1
okcomputer44
Member
 
Registered: Jun 2008
Location: /home/laz
Distribution: CentOS/Debian
Posts: 244

Rep: Reputation: 51
MySQL server 100% CPU usage in a specific time


Hello all,

I'm having a problem with a mysql server. The CPU usage goes up to 100% when the mysql server does something in the background. This is keeping the server for 8-10 minutes in dead position. MySQL reaches the 150 limit and the apache can't serve the sites either.

This happens at 3.40PM and 3.40AM too. I looked the crontab and I didn't find anything that could causing this.

The log says nothing either. Just qmail server send/receive regular mails in that time.

This is a plesk server with CentOS 5. Only the php and the httpd server was updated in the last 3 years. (The MySQL is 5.0.22-2.2.el5_1.1 version) Basically every packages and the kernel too are the old except those two rpm. The plesk old too and about 10 complete updates left in the admin window.

Of course this is an important server for that Company and it serves about 15 domains and of course no local administrator who maintains the server whatsoever.

Sometimes the usage goes to high in a random time once/twice a day but the 3.40AM/PM are happens every day.

An infected site causing this or was it compromised and somewhere is a hidden MySQL or php script running at that particular time?
Or is it just because of the old MySQL server with the old rpms? (bug)

Anyway if someone has an idea what to look for that would be much appreciated.

Is there a crontab in the MySQL server or task scheduler? I don't know much about MySQL.

Thanks guys!

Laz.
 
Old 01-16-2011, 02:42 AM   #2
paulsm4
Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Hi -

1. Looking at crontab was definitely a good idea.
Unfortunately, the culprit might be hiding someplace besides "crontab -l" .
Check all of:
a) /etc/cron.*
b) /var/spool/anacron*
c) etc

2. Try going into mysql and running "show status" and/or "show engine innodb status" around 3:40 - perhaps that might show the culprit.

3. Also look at the mysql log around 3:40 (e.g. /var/log/mysqld.log)

4. Google for "mysql 5.0 performance tuning", and see what tools/techniques might be available for troubleshooting your particular scenario.

My guess is that there is indeed some kind of automated backup being triggered somewhere in your server environment.

My second guess would be some script or task fires off a *very* expensive query every 12 hours. Either way, the tips above should help you track down the culprit.

Good luck! And please post back what you find
 
Old 01-17-2011, 02:08 AM   #3
okcomputer44
Member
 
Registered: Jun 2008
Location: /home/laz
Distribution: CentOS/Debian
Posts: 244

Original Poster
Rep: Reputation: 51
Thanks mate for the hints!

I already found crontab entry that was not listed.
The plesk castrates the whole CentOS. The logs, crontabs definitely placed are in a different files than placed by CentOS.
The main log file bigger than 1.2Gb the MC can't even search in it at all.

I'll have a look in the mysql too. But how come that the coders know nothing about the backup either?

I'll post tomorrow what I found about the culprit. (I didn't know this word yet but describe this situation well)

Thanks again!
 
Old 01-18-2011, 05:41 PM   #4
okcomputer44
Member
 
Registered: Jun 2008
Location: /home/laz
Distribution: CentOS/Debian
Posts: 244

Original Poster
Rep: Reputation: 51
Hi paulsm4,

I found the culprit

The plesk's DrWeb caused it. A full local virus scan started at that time including the email scan.
The DrWeb licence has been run out already.

Users held in MySQL server and that caused to reach the limits I guess.
I found in the syslog exactly at 3.40 smtp_lock_mysql error then the mysql can't serves any request cause reached the maximum connection limit.

I mentioned the random time freezing problem well I just got false information about that. That was a different server even with a different service.
Anyway I switched off the DrWeb and the problem just dissapeared.

I explained to the boss the server should be updated regularly and the DrWeb license has to be renewed otherwise much worse could happen with it.
Well thanks again for the hints!

Laz
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
pdflush daemon consuming 100% CPU usage for longer time makrandelex Linux - Server 1 09-15-2009 09:43 PM
cpu usage 100% ahhkyyda Linux - Enterprise 2 09-03-2009 07:26 AM
X server 100% cpu usage on AMD Athlon 1.1GHz johngreenwood Slackware 9 03-15-2007 03:16 PM
cpu usage stuck to 100% all the time mohama Linux - Software 0 03-05-2006 01:12 PM
100 % CPU usage ???? rolandi Linux - Newbie 1 10-07-2003 11:52 PM


All times are GMT -5. The time now is 08:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration