LinuxQuestions.org - RHEL x86_64 - Hugepages enabled - sar -B shows huge pgpout/s values

Hi All,

We have RHEL linux and run Oracle 11GR2 2 node RAC. since we have huge SGA's and since the system was showing heavy paging activity as reported by sar -B, we enabled Hugepages on the system.

Below are the system details :

Questions :
1) The pgpout/s values as shown by sar -B.. Are these normal? The system is not loaded in anyways.
2) Oralce and sys admins say they do not find any issues but my Oracle OEM says that huge paging activity at the system level.

Could you give insights to the pgpout/s values what it means and what are some acceptable values ?

Linux version 2.6.18-164.el5 Red Hat 4.1.2-46

ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x1b356c04 884736 oracle 660 15739125760 59
0x6da025cc 589825 oragrid 660 4096 0
0x27724b64 917506 oracle 660 5773459456 60
0x22f805c0 42139651 oracle 660 12081692672 52

more /proc/meminfo

MemTotal: 264251616 kB
MemFree: 82659408 kB
Buffers: 1217488 kB
Cached: 81433240 kB
SwapCached: 1068 kB
Active: 15349868 kB
Inactive: 69833704 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 264251616 kB
LowFree: 82659408 kB
SwapTotal: 16777208 kB
SwapFree: 16634724 kB
Dirty: 2356 kB
Writeback: 4 kB
AnonPages: 2558472 kB
Mapped: 312996 kB
Slab: 1655112 kB
PageTables: 108220 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 101710952 kB
Committed_AS: 7288268 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 407340 kB
VmallocChunk: 34359330159 kB
HugePages_Total: 46086
HugePages_Free: 39339
HugePages_Rsvd: 9272
Hugepagesize: 2048 kB

vmstat -s
264251616 total memory
181582080 used memory
15352904 active memory
69817688 inactive memory
82669536 free memory
1217488 buffer memory
81417088 swap cache
16777208 total swap
142484 used swap
16634724 free swap
20815220 non-nice user cpu ticks
5233 nice user cpu ticks
6882729 system cpu ticks
3298163810 idle cpu ticks
9451579 IO-wait cpu ticks
84619 IRQ cpu ticks
1248075 softirq cpu ticks
0 stolen cpu ticks
4377738657 pages paged in
2655880655 pages paged out
9413 pages swapped in
4196084 pages swapped out
1484230378 interrupts
333553087 CPU context switches
1314825817 boot time
4805609 forks

sar -B 1 1000

12:37:37 PM pgpgin/s pgpgout/s fault/s majflt/s
12:37:38 PM 12964.00 102745.00 53.00 0.00
12:37:39 PM 12988.00 105402.00 57.00 0.00
12:37:40 PM 4344.00 112829.00 359.00 0.00
12:37:41 PM 64.00 97006.00 47507.00 1.00
12:37:42 PM 116.00 109109.00 41096.00 1.00
12:37:43 PM 902.97 112683.17 45910.89 0.00
12:37:44 PM 128.00 106569.00 47398.00 0.00
12:37:45 PM 159696.00 31502.00 3299.00 0.00
12:37:46 PM 249728.00 2137.00 15772.00 0.00
12:37:47 PM 238178.22 46.53 1487.13 0.00
12:37:48 PM 177309.09 8834.34 157.58 0.00
12:37:49 PM 208.00 86833.00 117.00 0.00
12:37:50 PM 112.00 104642.00 178.00 0.00
12:37:51 PM 48.00 101545.00 408.00 0.00
12:37:52 PM 112.00 111187.00 34.00 0.00
12:37:53 PM 79.21 100012.87 80.20 0.00
12:37:54 PM 80.00 88642.00 56.00 0.00
12:37:55 PM 112.00 81041.00 74.00 0.00
12:37:56 PM 112.00 81410.00 182.00 0.00
12:37:57 PM 114592.00 35041.00 32.00 0.00
12:37:58 PM 228928.00 594.00 42.00 0.00
12:37:59 PM 183664.00 33.00 3783.00 0.00
12:38:00 PM 249040.00 2.00 49.00 0.00
12:38:01 PM 40633.66 37117.82 6441.58 0.00
12:38:02 PM 64.00 104930.00 45.00 0.00
12:38:03 PM 96.97 109782.83 3248.48 0.00
12:38:04 PM 79.21 106207.92 100.99 0.00
12:38:05 PM 176.00 111982.00 38.00 0.00
12:38:06 PM 80.00 107298.00 3091.00 0.00
12:38:07 PM 193.94 104260.61 271.72 0.00
12:38:08 PM 174.26 89439.60 3199.01 0.00
12:38:09 PM 184240.00 25825.00 38.00 0.00
12:38:10 PM 199160.40 204.95 91.09 0.00
12:38:11 PM 151806.06 8639.39 54.55 0.00
12:38:12 PM 160.00 70211.00 53.00 0.00
12:38:13 PM 112.00 97193.00 41.00 0.00
12:38:14 PM 174.26 100849.50 7226.73 0.00
12:38:15 PM 64.65 105996.97 4763.64 0.00
12:38:16 PM 272.00 109232.00 13576.00 0.00
12:38:17 PM 169726.73 26393.07 1061.39 0.00
12:38:18 PM 200048.00 106.00 73.00 0.00
12:38:19 PM 155152.00 15241.00 5304.00 0.00
12:38:20 PM 64.65 109024.24 52.53 0.00
12:38:21 PM 95.05 106523.76 77.23