There are a bunch of reasons to compile a new kernel. I did it the first time for security reasons, because I didn't want to allow modular code in my kernel (being able to inject code into the kernel space is definitely a security risk if your paranoid like I am...).
Other reason's I can think of off the top of my head..... To add functionality you need, to reduce size (get ride of stuff you don't need), to upgrade to a version not currently offered by your particular distribution yet....
That is all I can really think of now.
There really are no downsides, providing you follow the golden rules.
1. Always keep a known good kernel image around.
2. Always keep an option on your bootloader to boot that previously stated known good kernel image.
Last thing you want is to have to dig out a boot disk to get your system up and running.
If your going to bother with compiling a new kernel you should look into the grsecurity patches. grsecurity adds some great options to help you lock down your system.