" where do they go when you compile them?"
There is no one place. Each tarball puts them where the developer thinks best.
"I can usually run the program I compiled by typing it's name into the Run box in KDE, but that doesn't show me where they are located."
You can find out where a program is located with:
"Also, how do you get rid of programs you no longer need?"
If you installed it using rpm then use rpm -e (see man rpm).
If you installed it with a tarball then cd to the install directory and issue:
" How do I get rid of the old one without breaking anything?"
When compiling tarballs it is usually best to get rid of the old one before you install the new one, although often the new one will overwrite the old one.
With rpm you can install with rpm -U
In the case of Mozilla where the old package is a rpm and the new package is a tarball then you should remove the rpm first before compiling the tarball.
You can change the tarball into a rpm by using checkinstall, then rpm -U
Be prepared. Create a LifeBoat CD.