Hello, krock923,
I think you only need to delete the directory where you installed the program.
At least this is what you do if you install a software after using gunzip or tar (like OpenOffice.org, Mozilla, Thunderbird, Firefox, Ediptpadlite, to name those I've installed -- the tar.gz files contain an installer, or a setup file, which, I should think, makes the make-configure job for you, so uninstalling should be just as straightforward.
There's nothing like registry keys in Linux. Deb packages take apt-get remove to be uninstalled because apt-get installs them across directories, and apt-get remove is supposed (though I'm not sure of this) to remove all the occurrences of the files -- however after uninstalling apache2 forr instance I still can find apache2 directories here and there....
Cheers,