Well the Linux kernel is not much of a use without at least some core utilities for navigating in the filesystem, manipulating the configuration files, simple editors and so on.
Most control of what packages you are going to install:
http://www.linuxfromscratch.org/
However a Debian System with minimal install should be rather slick:
http://www.debian.org/
Copying the /boot of an existing system and installing a boot loader should give you only the kernel. However you most likely won't be able to use such a system.