A bash on the head?
Which is not to say it cannot be done.
I just have no idea how modified the kernel can get without throwing off which package dependencies. Too radical and I suspect you'll find some packages stop working and need to be reinstalled... and that assuming the dependency runs off the environment and not the record of the installed kernel package.
The package management is seriously sexy though. With all compatible HW, there really is no need. (Suddenly that intel chipset don't seem so second rate...)