Can't say i know of a substitute for X,but the following guide i recommend for minimal Debian installs:
So to do what you need based on the guide i'd do:
aptitude install xorg xdm fluxbox
This should give you an install of X,a basic login manager and Fluxbox.
Have a read see what you think.
Edit:Forgot to say,using a login/display manager is not mandatory.
You can start the gui with the startx
command,or it's possible to configure the system to boot directly into the gui.