Originally Posted by bigrigdriver
The kernel itself is installed in /usr/src.
The kernel source would usually be found there, along with object files left over from a compile. You're going to also have a bunch of modules under /lib/modules. Probably some stuff under /usr/share/doc as well.
If you installed the kernel via your distro's package management system, your can probably cleanly remove it using the same. I did that with Debian a few times.