Don't both recompiling
What you're seeing is your distribution is loading up a lot of things in your kernel
Ask a Slackware person (Never used Slack) how their init scripts work (the scripts that start the comp) and how to remove drivers that you don't need.
Original kernels tend to be packed with things, but most features only add a few kb to the kernel's size.
Unless you want to go through the trouble of doing this (and at first it is a little annoying) by all means
Otherwise just remove the modules that you don't need (and ask how to remove services that you don't need, those are slowing down your computer more then your kernel)
By removing services you'll accomplish a lot more then you can by recompiling the kernel.