The Kernel it the heart of an Operating System. It schedules processes to run, does memory management, disk I/O, etc.. etc... and the kernel has all drivers/modules to handle other devices. In fact, the kernel just sits there, waiting for programs to do request something. The task of the kernel drivers is to make any hard drive look like an ordinary hard drive, any scanner/webcam/whatever is a "video 4 linux" device. It's the compatibility layer between your application and the hardware.
Because this component is the central heart of your system, any improvement will speed up your entire system too.
The linux kernel is in active development. The 2.6.0 kernel, being released next year, has major improvements, even features Microsoft hopes to release with their OS in 4 years from now!
The kernel gets tuned/improved with every upgrade, and drivers get improved. My current 2.4.22 kernel does handle some things (via audio) a lot better then the 2.4.20 kernel; read the changelog and you see the kernel is constantly improved. The Linux kernel code can be downloaded from the web, and you can re-build a kernel, also called "compiling". This allows you to create a custom kernel, optimized for your system (cpu) and hardware, and you could remove all kind of features your systems doesn't have.
The kernel itself is stored in /boot, and the drivers/modules are stored in /lib/modules/<kernel version>/ If you boot a new kernel, your previously installed drivers are not be installed here
Just be carefull with playing with your kernel. it's the heart of your system, so you don't want it to be broken because you've played around too much