Applications that you install as root, such as rpm packages, and tarballs will install under the /usr hierarchy. Executables will go in /usr/bin or /usr/local/bin. Libraries also installed go in /usr/lib or /usr/local/lib. The /usr directory contains the same directories as the root directory (e.g. bin, lib, sbin ). There is another location at /usr/local which again contains these directories. According to the Linux Hierarchy Standard, this is where programs you install yourself should go. Also, this directory is untouched when upgrading linux.
However, the programs will install where the people who wrote the packages decide.