Hello all,

I'm having a problem with a mysql server. The CPU usage goes up to 100% when the mysql server does something in the background. This is keeping the server for 8-10 minutes in dead position. MySQL reaches the 150 limit and the apache can't serve the sites either.

This happens at 3.40PM and 3.40AM too. I looked the crontab and I didn't find anything that could causing this.

The log says nothing either. Just qmail server send/receive regular mails in that time.

This is a plesk server with CentOS 5. Only the php and the httpd server was updated in the last 3 years. (The MySQL is 5.0.22-2.2.el5_1.1 version) Basically every packages and the kernel too are the old except those two rpm. The plesk old too and about 10 complete updates left in the admin window.

Of course this is an important server for that Company and it serves about 15 domains and of course no local administrator who maintains the server whatsoever.

Sometimes the usage goes to high in a random time once/twice a day but the 3.40AM/PM are happens every day.

An infected site causing this or was it compromised and somewhere is a hidden MySQL or php script running at that particular time?
Or is it just because of the old MySQL server with the old rpms? (bug)

Anyway if someone has an idea what to look for that would be much appreciated.

Is there a crontab in the MySQL server or task scheduler? I don't know much about MySQL.

Thanks guys!

Laz.

Hi -

1. Looking at crontab was definitely a good idea.
Unfortunately, the culprit might be hiding someplace besides "crontab -l" .
Check all of:
a) /etc/cron.*
b) /var/spool/anacron*
c) etc

2. Try going into mysql and running "show status" and/or "show engine innodb status" around 3:40 - perhaps that might show the culprit.

3. Also look at the mysql log around 3:40 (e.g. /var/log/mysqld.log)

4. Google for "mysql 5.0 performance tuning", and see what tools/techniques might be available for troubleshooting your particular scenario.

My guess is that there is indeed some kind of automated backup being triggered somewhere in your server environment.

My second guess would be some script or task fires off a *very* expensive query every 12 hours. Either way, the tips above should help you track down the culprit.

Good luck! And please post back what you find

Thanks mate for the hints!

I already found crontab entry that was not listed.
The plesk castrates the whole CentOS. The logs, crontabs definitely placed are in a different files than placed by CentOS.
The main log file bigger than 1.2Gb the MC can't even search in it at all.

I'll have a look in the mysql too. But how come that the coders know nothing about the backup either?

I'll post tomorrow what I found about the culprit. (I didn't know this word yet but describe this situation well)

Thanks again!

Hi paulsm4,

I found the culprit

The plesk's DrWeb caused it. A full local virus scan started at that time including the email scan.
The DrWeb licence has been run out already.

Users held in MySQL server and that caused to reach the limits I guess.
I found in the syslog exactly at 3.40 smtp_lock_mysql error then the mysql can't serves any request cause reached the maximum connection limit.

I mentioned the random time freezing problem well I just got false information about that. That was a different server even with a different service.
Anyway I switched off the DrWeb and the problem just dissapeared.

I explained to the boss the server should be updated regularly and the DrWeb license has to be renewed otherwise much worse could happen with it.
Well thanks again for the hints!

Laz