Some source packages include an unistall option, most do not. The way that I handle the problem of uninstalling source packages is to use checkinstall.
Using checkinstall you can install a source package as a rpm, debian, or
slackware package. In your case you would probably want to install it as a rpm. Then you can uninstall the package with rpm -e packagename.
If you have already installed a package as a source package then you can install the package a second time using checkinstall; then uninstall it using rpm.