I'm pretty sure system-config-display is a GUI program and you can't run it in text mode. Hence the system hang. I still say either install a newer distro and update the programs you need either by installing the appropriate packages or compile them from source if source is available to you. It seems clear to me the version of the distro you are attempting to install does not have the proper drivers to manage your video chip. That's why the "vesa" line is in the xorg.conf file, it's trying to use that because nothing else will and even that won't work. Or maybe you could see if the nvidia support pages have linux drivers you can download and install for that chip.

Oops. I missed the part where you said it locks up the system.
Apparently system-config-display is having problems, so it's
resorting to the vesa driver. However it is also having problems
with that.

Try changing it to:

Once you have an xorg.conf file that works, you should be able
to run system-config-display from either the command line or
the gui. Just be sure to keep a copy of the first version of
xorg.conf that you get to work. Then you should be able to
try the nv driver and/or the vesa driver. Basically my first
choice would be nv, second is vesa, and third is vga. Also
try using a low resolution like 800x600 or even 640x480. That's
just to get it working. After that you can experiment to find
out what driver and resolution works best.

I don't think the "vga" substitution in /etc/X11/xorg.conf is gonna help any.