Linux memory management is a lot different than that of other systems. Linux tries to grab as much free memory as it can for uses as internal buffers and cache. This memory can easily be reallocated to applications that need it. The only time you really have a problem is when there's not enough RAM and the OS has to swap pages to and from disk. A little swapping is OK, but heavy swapping causes a condition known as thrashing, which dramatically lowers performance.
Memory is allocated to a specific process based on how much memory the kernel thinks it will need, based on information in the executable file. Programs can also dynamically request more memory when they execute, in which case the kernel will try to honor the request, unless you have set the ulimit to limit memory usage. Note that the kernel can "overcommit" system resources and this will cause problems if every process tries to use all the resources it requests. If you're really interested in how this actually works, check the code in the mm directory of your kernel source.