LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 02-11-2007, 04:17 AM   #1
guruyaya
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Rep: Reputation: 0
Proformance problem on CentOs 4.3


Hello. I'm having a 2 CentOs 4.3 boxes that tend to get overloaded every once in a while (get to 100 when the problem occur, when they normally go around the 1.0-2.0). Some tests I've made, pointed out that the main problem is that writing to disk creates overload. A simple cp of a 4.4G directory could demostrate the problem:

[02:59:43][ *root* @ machine:/usr/local]$ w | head -n1
02:59:48 up 16 days, 18:38, 11 users, load average: 0.65, 0.96, 1.05
[02:59:48][ *root* @ machine:/usr/local]$ cp -rf apache_prod/ tmp &
[1] 16694
[03:01:03][ *root* @ machine:/usr/local]$ w | head -n1
03:01:08 up 16 days, 18:39, 11 users, load average: 2.91, 1.48, 1.22
[03:01:09][ *root* @ machine:/usr/local]$ w | head -n1
03:01:58 up 16 days, 18:40, 11 users, load average: 6.01, 2.45, 1.55

In a matter of 2 min, the load climbed up to 6, and it would have gone ferther if I would't have stopped it.

As I said, this is the problem happens on 2 machine, with rather different hardware profile: one has SATA disk, using LVM, the other is using and IDE disk, no LVM. Swap is not overused on both machines, and just watch the hdparm test:
[03:10:52][ *root* @ machine:/usr/local]$ hdparm -Tt /dev/sda
/dev/sda:
Timing cached reads: 1726 MB in 2.00 seconds = 862.70 MB/sec
Timing buffered disk reads: 174 MB in 3.03 seconds = 57.42 MB/sec
(The other machine is quite the same).

Both machine host several sites using apache 2.0.52 and PHP 5.2.0, and mysql 5.0.24a-standard. Nothing I find problematic.

Can anyone assist?
Thanks in advance
Yair
 
Old 02-11-2007, 10:34 PM   #2
kstan
Member
 
Registered: Sep 2004
Location: Malaysia, Johor
Distribution: Dual boot MacOS X/Ubuntu 9.10
Posts: 851

Rep: Reputation: 31
Hi guruyaya,
open another command prompt then start top, check 'wa' status. If % high maintain long time means your harddisk already reach bottle neck. Please check is there any application always read/write into your harddisk(Probably beagle, updatedb or etc).

Recommended solution is separate folder which always read/write (/usr, /var, or etc) into difference hard disk.

Good luck,
Ks
 
Old 02-11-2007, 10:40 PM   #3
Cliffster
LQ Newbie
 
Registered: Feb 2007
Posts: 26

Rep: Reputation: 15
Two words: File Server. Two other good words are RAID Array.

CPU and IO bus bottleneck is one good reason file servers were created to smooth out access on very large files accessed often or many many small files to lessen disk thrashing. The computer is, i suspect, not happy about read/writing 4 gigs on a not very fast HD but most of the time, as you state, its doing its best at it.

For comparison, i can get my cpu load to 35 by doing ls -R from /
So dont worry about it unless you want to get into mission critical redundant failsafe stuff and spend a ton of money.

Last edited by Cliffster; 02-11-2007 at 10:43 PM.
 
Old 02-12-2007, 12:00 PM   #4
guruyaya
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
I'm afraid this is a production machine, that hosts several sites. I'm afraid that I cannot "not worry" about it.

I have, however, understood that the right way is to cut on disk read/write. As I moved the apache logs to the home folder (that makes a lot of read write, and is meant to do that), it got a bit better.
To my surprise, that machine that had an improvement was the one with the LVM, using one physical disk. The other machine, with 2 disks (Home folder is set on another physical disk). This machine hosts more sites, but it is, for some reason, more sensitive. It is holding steady on 0.97-1.5, while I would like to see it on less then one most of the time.
As it seems, I need to see if there's something writing constantly, on this machine to the disk, without my knowledge.

Thanks
Yair
 
Old 02-13-2007, 02:32 AM   #5
kstan
Member
 
Registered: Sep 2004
Location: Malaysia, Johor
Distribution: Dual boot MacOS X/Ubuntu 9.10
Posts: 851

Rep: Reputation: 31
u can write a script to email you every 15 minute server load daily. Then you know whether your server is over load or not. 15 minute x 4 x 24= 1440 records. Finally you can plot a graph with openoffice to see the trend.

The easier way is using MRTG if it exist in your server.
 
Old 02-14-2007, 01:37 AM   #6
guruyaya
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by kstan
u can write a script to email you every 15 minute server load daily. Then you know whether your server is over load or not. 15 minute x 4 x 24= 1440 records. Finally you can plot a graph with openoffice to see the trend.

The easier way is using MRTG if it exist in your server.
Acctually, I allready did that, but every 2 min. It helps noticing a big problem (like when I'm very high on load avg), but can't say it helps solving it, at least for now.
 
Old 02-14-2007, 02:35 AM   #7
kstan
Member
 
Registered: Sep 2004
Location: Malaysia, Johor
Distribution: Dual boot MacOS X/Ubuntu 9.10
Posts: 851

Rep: Reputation: 31
So the graph indicate your server overloaded? Do you check the I/O graph too? Probably your can configure syslog to write log file into 1 dedicated machine too. Using Raid 5 is a good ideal too (It increase your read/write speed and also give you redundancy) since it is mission critical server.
Good luck,

Ks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
https problem with CentOS prasoum Linux - Software 2 02-07-2007 09:37 AM
Problem with yum in CentOS 4.3 combilli Linux - Software 2 10-31-2006 04:39 AM
Problem installing Centos 4.3 wrexn Linux - Software 7 07-21-2006 03:13 PM
High proformance sound card dstrbd1 Linux - Hardware 2 02-01-2006 07:52 PM
update centos 4 rc1 to centos 4 trou yum? maxut cAos 2 03-04-2005 03:36 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:37 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration