Hi

I have a Cent OS 5.2 (Database & Apache) Server running on vmware and sitting on a SAN.

We are getting high iowait (Please see below)

CPU %user %nice %system %iowait %steal %idle
08:40:02 AM all 4.20 1.95 10.61 78.23 0.00 5.02
08:50:01 AM all 6.23 2.05 9.67 37.82 0.00 44.23
09:00:04 AM all 9.18 2.07 11.37 59.49 0.00 17.88
09:10:01 AM all 11.93 1.91 10.64 74.21 0.00 1.31
09:20:01 AM all 3.94 1.91 9.84 81.47 0.00 2.84
09:30:01 AM all 3.85 2.07 11.03 80.34 0.00 2.71
09:40:01 AM all 3.77 1.68 9.24 82.46 0.00 2.84
09:50:01 AM all 6.97 1.46 9.35 81.31 0.00 0.92
10:00:03 AM all 3.58 2.24 10.14 66.39 0.00 17.65
10:10:01 AM all 3.61 1.79 9.51 84.12 0.00 0.96
10:20:01 AM all 3.34 1.86 9.39 82.98 0.00 2.44
10:30:01 AM all 3.38 1.52 8.79 85.01 0.00 1.29
10:40:01 AM all 3.40 2.08 9.76 82.50 0.00 2.26
10:50:02 AM all 3.45 1.43 8.12 85.73 0.00 1.28
11:00:02 AM all 3.35 1.32 7.76 86.43 0.00 1.15
11:10:01 AM all 3.27 1.97 9.16 78.55 0.00 7.04
Average: all 3.73 1.83 8.87 23.11 0.00 62.45

We did not get this problem when the server was on a physical server. Users complain that the web site is running slow.

I suspect it could be a bottleneck in the database or web code?? Is there a way i can test the iowait for the server?

Thanks in advance

Oh, yes. You could try running your app and looking at the iowait numbers. Oh wait, that's just what you've done

You are getting high iowait numbers and the question is why the change from 'real' to 'virtual' has done this? If I were to take a couple of guesses, what about name resolution and access to the disk subsystem. So, what I think you ought to be looking at, in the first instance, is whether name look ups or the disk subsystem have suddenly become slow.

In particular if one of those used to be local and is now virtual (on the same phys machine, but having to go through the virtualisation layer) or distant (on different hardware), that could make sense.

IO delays could be caused by busy SAN. Check on that first. To improve IO on virtualized server consider running para-virtualized guest OS. RHEL5 and CentOS5 both do that. In addition mount a volume from the SAN and have virtualized OS use that directly as opposed to reading / writing to a image file. With these 2 steps the IO performance should be right on par with the physical system. I remember seeing some redhat benchmarks where system setup this way archive somewhat 95% of non virtualized IO performance.

Here is how to install:

https://www.redhat.com/docs/en-US/Re...t_from_a_shell

I just completed installing one system like this today and it is fast! I am running the system on XEN hypervisor, but VMWare should be able to do the same.

As mentioned, having all the I/O from the guest intercepted is death to (I/O) performance.
As to iowait, all it tells you is that the CPU(s) is/are idle, but there is I/O that hasn't completed. Doesn't tell you how much I/O, nor who is waiting.
You can get an idea of the latter by looking for tasks in state "D". I use the following