It will depend on how you installed the package.
If you installed the package in rpm format use rpm -e command.
If you used yum to install the package, yum will uninstall it for you.
If you installed it from source, then go the directory from where you run make install command and from the location run
Do it as root