what is a bad IO?

qwertyjjj · 02-08-2010, 03:17 PM

I have some slowness on my squid server at present.
I have tried to optimise the configuration and also put in caching but occasionally when the network speed is anything above 500kbits.sec not that high (the network is suspposed to be 100Mbit) I seem to get a delay when requesting webpages.

My vmstat reads the following. As you can see there is no swapping and the memory isn't at 100% so I don't think it's a CPU problem. Could it be IO? I used to have caching turned off but still had the same issue so maybe it's something else?

Code:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----                                                                                                 -
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      4 126336 164196 469784    0    0     0   180 1025  159  0  0 100  0                                                                                                   0
 0  0      4 126336 164196 469784    0    0     0     0 1015  149  0  0 100  0                                                                                                   0
 0  0      4 126336 164200 469780    0    0     0    36 1056  198  0  0 100  0                                                                                                   0
 0  0      4 126336 164204 469776    0    0     4     0 1080  245  0  1 100  0                                                                                                   0
 0  0      4 126336 164208 469808    0    0     4     0 1070  200  0  0 99  0  0
 0  0      4 126336 164224 469792    0    0    12   112 1127  263  0  1 98  1  0
 0  0      4 126336 164224 469836    0    0     0     0 1031  165  0  0 100  0                                                                                                   0
 0  0      4 126336 164228 469832    0    0     0    36 1101  210  0  1 100  0                                                                                                   0
 0  0      4 126336 164228 469836    0    0     0     0 1144  204  0  1 100  0                                                                                                   0
 0  0      4 126276 164232 469832    0    0     4     0 1299  389  1  2 97  1  0
 0  0      4 126216 164248 469948    0    0    48     0 1261  407  0  1 97  1  0
 0  0      4 126216 164252 469944    0    0     0     0 1277  306  0  1 99  0  0
 0  0      4 125968 164252 470048    0    0     0     0 1231  357  0  2 98  0  0
 0  0      4 125968 164264 470136    0    0    20    52 1175  331  0  1 97  2  0
 0  0      4 125908 164264 470184    0    0    16     0 1177  311  0  1 98  0  0
 0  0      4 125664 164268 470380    0    0   140   308 1337  562  0  2 97  1  0
 0  0      4 125664 164268 470412    0    0    52     0 1137  270  0  1 99  0  0
 0  0      4 125480 164276 470504    0    0    32     0 1200  275  1  1 99  0  0
 0  0      4 125356 164304 470524    0    0    24    52 1206  353  0  2 97  2  0
 0  0      4 125172 164352 470676    0    0    36     0 1168  356  0  1 98  1  0
 0  0      4 125112 164364 470664    0    0    56   384 1263  468  1  2 97  2  0
 0  0      4 124980 164372 470756    0    0     8     0 1207  288  0  1 99  0  0
 0  0      4 124980 164372 470780    0    0     0     0 1177  254  0  1 99  0  0
 0  0      4 124488 164380 471072    0    0   116    56 1358  542  0  2 96  2  0
 0  0      4 124304 164380 471092    0    0     0     0 1327  425  0  1 99  0  0
 0  0      4 124056 164392 471280    0    0     8   488 1315  380  0  2 98  1  0

rweaver · 02-08-2010, 03:23 PM

That's not tremendously readable, try enclosing it in [ code ] tags instead of [ quote ] tags.

Also if you want to know about disk io and such, you might be better looking at iostat or various disk benchmarks and comparing your utilization to the maximum throughput of your system as presently setup vs actual usage.

qwertyjjj · 02-08-2010, 03:34 PM

Quote:

Originally Posted by rweaver

That's not tremendously readable, try enclosing it in [ code ] tags instead of [ quote ] tags.

Also if you want to know about disk io and such, you might be better looking at iostat or various disk benchmarks and comparing your utilization to the maximum throughput of your system as presently setup vs actual usage.

Done.
It's strange because video streaming works with no latency yet loading simple html webpages is very delayed.
I thought it could be DNS but all dig and nslookups work fairly quickly.
It can't be the cache because it was like this when I didn't have caching turned on. Turning caching has actually improved it slightly.

sda2 is where the cache is stored

Code:

[root@server ~]# iostat -d -x 5 3
Linux 2.6.18-164.11.1.el5 (serverxxx.xxx.xxx.xxx)       08/02/10

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.23     7.04  1.38  2.07    40.34    71.08    32.26     0.09   25.92   1.12   0.39
sda1              0.00     0.02  0.00  0.00     0.07     0.04    63.99     0.00   34.29  10.35   0.00
sda2              0.03     0.59  0.01  0.17     0.31     4.36    26.06     0.01   53.86   0.99   0.02
sda3              0.13     2.03  1.14  1.28    25.70    26.51    21.57     0.03   14.35   1.10   0.27
sda4              0.00     0.00  0.00  0.00     0.00     0.00     2.00     0.00   14.33  14.33   0.00
sda5              0.06     4.40  0.23  0.62    14.25    40.17    63.88     0.05   52.91   1.81   0.15
sda6              0.00     0.00  0.00  0.00     0.01     0.00    29.92     0.00    4.50   4.20   0.00

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     7.40  0.00  5.40     0.00   101.20    18.74     0.01    1.07   0.11   0.06
sda1              0.00     0.00  0.00  0.20     0.00     0.40     2.00     0.00    3.00   3.00   0.06
sda2              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda3              0.00     5.20  0.00  4.80     0.00    80.00    16.67     0.01    1.08   0.12   0.06
sda4              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda5              0.00     2.20  0.00  0.40     0.00    20.80    52.00     0.00    0.00   0.00   0.00
sda6              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     1.00  0.00  0.80     0.00    14.37    18.00     0.00    0.00   0.00   0.00
sda1              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda2              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda3              0.00     1.00  0.00  0.80     0.00    14.37    18.00     0.00    0.00   0.00   0.00
sda4              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda5              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sda6              0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

rweaver · 02-09-2010, 09:10 AM

Are you using any features like ident lookups? What file system are you currently using on it? Have you turned on noatime on the file system?

qwertyjjj · 02-09-2010, 09:13 AM

Quote:

Originally Posted by rweaver

Are you using any features like ident lookups? What file system are you currently using on it? Have you turned on noatime on the file system?

I don't think I am using ident lookups.
The filesystem on sda2 is reiserfs - I think I turned on noatime when I monted it but not sure.
I don't think it's the cache causing the problems but could be...the numbers for io look ok don't they?

Code:

[root@ ~]# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw,data=ordered 0 0
/dev /dev tmpfs rw 0 0
/proc /proc proc rw 0 0
/sys /sys sysfs rw 0 0
/proc/bus/usb /proc/bus/usb usbfs rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/sda3 /var ext3 rw,data=ordered 0 0
/dev/sda1 /boot ext3 rw,data=ordered 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
/etc/auto.misc /misc autofs rw,fd=7,pgrp=2586,timeout=300,minproto=5,maxproto=5,indirect 0 0
-hosts /net autofs rw,fd=13,pgrp=2586,timeout=300,minproto=5,maxproto=5,indirect 0 0
nodev /dev/oprofile oprofilefs rw 0 0
/dev/sda2 /var/spool/squid reiserfs rw 0 0