Some of the really small distros such as
Tiny Core Linux make use of
xvesa instead of a full blown xorg server (which not only saves install size, but dramatically reduces the RAM required). They are also selective with which packages are installed, and will use 'shells' such as
busybox to reduce the number of utilities required.