Im not sure if this is how it works or if there is a memory bug but I have a MC server that I allocated 2GB of memory to (its running Bukkit) and I noticed that when I start it up it goes from 200MB (system idle memory) then to 600MB which is fine, and the memory increases as people join...HOWEVER when people leave the memory count is still high and doesnt go back down but the weird thing is that when they join back it doesnt go higher either.

The behavior your describe is normal for Java applications.

What are your -Xms and -Xmx values? They set the minimum and maximum allocation pools for heap memory.

In general it is recommended for production environments (i.e. environments that are used by the public) to have the same values for -Xms and -Xmx. Java will fluxuate the heap memory between those two memory sizes. If the heap minimum memory size (-Xms) is below -Xmx and it needs to allocate more from the system then it will take extra time to allocate that memory for the heap. Because of that it is recommended to give the heap a fixed size by setting -Xms and -Xmx the same value.

For instance on the minecraft website they recommend starting the JVM with the following options for the dedicated server...
Just remember that your java application is actually a JVM (Java Virtual Machine). You can think of it like any other virtual machine where it emulates its own instruction set architecture and manages its own memory (in this case through garbage collection).

I'd also like to point out that Java has more types of memory than just the heap. Look up on google for types of java memory. That being said since there are other types of memory in Java than just the heap it is possible for your JVM to go above the maximum pool size set by -Xmx. You can control the sizes other types of Java memory by passing additional options but I don't need to get into that. For minecraft, the default sizes for other types of memory are fine and usually only adjusted to tune garbage collection.

If you would like a true representation of how your JVM is managing memory then you must connect jconsole to your JVM. The OS won't report true memory usage within the JVM otherwise.

Alright sweet thanks, I installed OpenJDK thinking Sun Java was a problem however I think OpenJDK is eating up my ram more than Sun Java so I will switch back.

Hi

Are you really sure it's really a problem? If you've allocated 2 Gb on a dedicated server, shouldn't it use it? If it did free that memory to the OS, it would just be slower when it needs memory later. If you don't want it to use 2 Gb, simply allocate less. As sag47 said, if you really want to see how much memory is being used you need to use jconsole. If you use the computer for other things, you should just allocate less memory. If the server is not used, the memory will be swapped to disk if it's needed for other things.