Personally, i would download the sources and start compiling... however i wouldnt reccomend it. Perhaps try the rpms on the red hat cd´s?
I dont think there is a nice installer for red hat if xwindows isnt installed (someone please correct me if im wrong..) so you will need to track down the rpm for xwindows.
the generic command for installing rpms is:
rpm -ivh [rpm name here]
try man rpm for more details.
You should be able to find all the rpms on the cds, or you could try googling them and download.
However for a server i dont think its a good idea to run Xwindows... I may be wrong here but it just seems like a waist of cpu and memory to run X on a server. perhaps you should figure out what you need to do work out how to do it from bash? Just a suggestion.