Well I think that it all depends and how you're gonna use, yeah, I know, you said 'vmware box' but that doesn't mean anything, are you gonna use VMs as web servers? software development? my point is that with your current hardware your capable to run a vmware box, however, if you are plainning to use Windows Vista in a VM, you'll need more ram.
Windows Vista needs an important amount of ram, the bare minimun is 512 MB, but with that amount of ram you're gonna be using disk cache a lot. I think that your problem was that, Windows XP isn't a problem, but Windows Vista is for a VM, if you want to use it seriously, then will better to increase the amount of ram to 2 GB at least, but of course you can leave it as it is now and have in mind that the disk is going to work a lot swapping and caching data.