Well, the first comment was rhetorical really.
I get what you want - been there, done that.
If you build your own kernel, and only include things you need, it will get smaller.
As will the initrd.
After all, you only need to include one scheduler instead of all of them, only the filesystems you actually have, only support for the processor you actually have - and so on.
Whether it is noticable later ... rather not, but it may be - slightly.
You can build a kernel optimized for your processor - but all the software you are using still is pre-packaged and not that optimized.
You will also gain hardly anything - speedwise - by removing not needed parts like unneeded display drivers (ATI, Nvidia).
The same driver as before is used - the others are just there, unused.
If you dont build them it does not hurt - you just don't have them lying around - but they would not be used anyway.
It does not get any quicker from that.
And - IMO - it is quite a hassle to do that (your own kernel) in Ubuntu.
Or Debian, or any derivate therefrom - like Mint, which I liked much better.
I would have stayed with Mint - if there had not been this very thing.
...and some more...
I came from Gentoo, which was nice but too much work. For me.
Now I'm using Arch.
Still some work, not as much preconfigured and ready to use as Ubuntu or its kind, but very easily adaptable.
Which was more of a pain in Debian/Ubuntu/Mint...
do try it!
(the kernel thing, not Arch...)
But be careful to NOT remove the standard kernel and its failsafe companion.
Just build your own and add it to the bootloader, thus always being on the safe side should it not work the first time around.
Then try and watch if you notice a difference.
And then evaluate if that was:
- worth the time you spent on it
...talk about _adding_ lightness