I think, yes you have to recompile the whole kernel... but it's very "fast" (in fact if you have a 2.4 kernel with a pentium 75MHZ and 32Mo RAM, it's very long but the command to type are short
).
You just have to type (if you have thought to recopy the .config file):
make clean && make dep && make bzImage (for a 2.4 kernel)