well it's not the swap that matters, but the amount of REAL ram you have, and how it's being used. when you're system is behaving like you describe, run "free" to see the memory usage.
[root@GBR607SL001 root]# free
total used free shared buffers cached
Mem: 511936 504148 7788 0 13156 243692
-/+ buffers/cache: 247300 264636
Swap: 1052216 177212 875004
here i have 512mb of total memory (ram) 500mb used in some form, the rest unused totally. of that 500mb, 240mb is real data, 260 is older cached data. i also have a gig of swap, 170mb of which is being used, 875mb is totally unused. see how yours breaks down.