I have a VMware ESXi 3.5 hypervisor host with a bunch of guest VMs on it. I am looking to add more VMs but would like to reclaim some RAM from some linux guests, without having to install vmtools to obtain the balloon driver, (which steals memory from running VMs when the host needs it, possibly for higher priority VMs).

Anyway on CentOS 5.5 x86 how can I tell how much ram a running machine actually needs?

I've spotted one particular machine I'll use as an example. This one has 1GB of RAM allocated to it, however it is doing very little, it does not have a gui, web server, database, bind, or dhcpd. It's only running sshd and openvpn to route about 1mbps of traffic.

So I run the free -t -m command with the below results
Do you think the 49 listed as used on the buffers/cache line is indicative of how much memory is actually needed?

Looking at the ESXi monitoring charts the "Memory Usage (Average)" fluctuates from about 40~60MB

On the surface this sounds like I can safely cut the ram allocated to the VM down to 64MB. However, I'm wondering how much of the buffers and cache can be dropped before performance is impacted? On other workstation systems and higher load servers I normally find that having about 50% ram used for cache provides good performance. However that's assuming that the server does frequent file io, this server does not have much file io either. Over the last hour it has not done any disk reads, every 5mins it does about 20KB/s disk writes.

Also, importantly, how low can I go and have CentOS 5.5 still boot?

CentOS manual notes 256MB RAM as miniumum system requirements, I wouldn't go much below that.

In the mainframe world (where they've been doing virtualization for over 30 years) this is handled by reducing the guest until it (just) starts to swap. Then add a small bit of leeway.
If your guest boots and doesn't page swap at 64 Meg, leave it at that. KISS.

Linux page cache as a performance metric is (almost) useless as there's no way to tell the "age" of cache pages (from userspace). Keep an eye on I/O wait and swap.

well I cut it back to 256mb rebooted, it looked fine, so I cut it back to 128mb and it looked fine, so I left it like that during a days work and it performed fine, all the while memory stats were much like

# free -t -m
total used free shared buffers cached
Mem: 122 103 19 0 17 65
-/+ buffers/cache: 20 102
Swap: 2055 0 2055
Total: 2177 103 2074

except if I ran yum list updates while running screen and top in an ssh session then memory usage -buffers/cache would surge up to about 75MB so I think I'll leave it at 128MB and leave it at that. Interestingly though, the quiescent used - buffers/cache memory has better than halved going from 49 to 20MB. Do you think the smaller kernel memory management tables are responsible for most of that? I wonder if cutting the swap space down would make a similar reduction. Perhaps I should try it on a testing server and see how low can you go.

After this initial round of changes on 2 out of the 6 VMs I've regained nearly 2GB of ram allocation, so I'm quite happy with that, enough for another few servers.

Nope - that will be (almost) inconsequential.Do not reduce swap space - especially if reducing RAM. Better to increase it to cover unforeseen spikes.
Screwing around on test boxes is good - but only if you can reasonable reproduce production environment(s).