If you wanna use Xfree's own binaries,
http://xfree86.org/MIRRORS.txt ...take your pick.
Pick the right dir for your glibc-<version> and d/l to an empty folder ,then just run the install script
#sh Xinstall.sh.
Here's the minimum required:
Xbin.tgz
Xinstall.sh
Xmod.tgz
Xxserv.tgz
Xdoc.tgz
Xfenc.tgz
Xlib.tgz
Xetc.tgz
Xfnts.tgz
Xman.tgz
Xvar.tgz
extract
...add anything you need/like.
You'll be prompted to overwrite old config files, just use the defaults, which is usually no.
If you wanna play it safe, backup your /usr/X11R6 dir.