You're sure you chose the correct architecture for both the kernel and the OS in general?
Meaning that in /etc/portage/make.conf (or /etc/make.conf - just be sure that you have only 1 such file) you have something like (64bit Intel CPU):
CFLAGS="-march=native -O2 -pipe"
Have a look at your /var/log/Xorg.0.log: what's that...
[ 728.853] (EE) Failed to load module "modesetting" (module does not exist, 0)
..."modesetting" module? Is it specified in the /etc/X11/xorg.conf file? If yes, then it's most probably wrong.
Please post your xorg.conf.
Please post more informations, otherwise it will be impossible to help you.