You are pretty much describing Linux from Scratch
, it is an excellent reference to guide you through building your own customized distribution.
*edit* I'm an idiot I didn't see that part your wrote about LFS.
However, I find that LFS makes a fine distribution. I build everything from source, and as long as I keep up with security advisories everything is fine. I used it for a while as a web and mail server, and have recently converted it to a desktop system. So in my opinion it is an excellent dist, and not just for learning, I use it to write and test most of my software.