i installed red hat 8 with grub as my boot loader,,
the system could not install and was giving error 11 : memory insufficient..
so changed to lilo ...linux got installed but after the kernel loaded
the screen goes blank and system hangs badly..

See if you can get to a virtual terminal by pressing control-alt F3 (try F1 through F6 before giving up). Log in as root and configure and test your X server.

# Configuring X
Linux Hardware Compatibility HOWTO - video cards
The Linux XFree86 HOWTO
Configuring XFree86 for a Non-Specific Linux Distribution
Common X configuring tools:
Redhat 7.3 down - the setup utility leads to several config tools
Redhat 7.3 up - redhat-config-xfree86
You may have one or more of these tools:
XF86Setup
XFree86 -configure
Xconfigurator
xf86cfg
xf86config
xconf
ATI Linux drivers
If you need the nVidia driver, you want the Linux IA32 driver

# Red Hat links
Red Hat Linux Manuals
Get your mp3 support here
Maximum RPM
rpmfind
Easier software management: apt4rpm - Red Carpet
Red Hat 8.0 Tips & Tricks

# Red Hat 7.3 down configuration commands
setup leads to several configuration tools

# Red Hat 7.3 up configuration commands
Configure soundcard:
redhat-config-soundcard
Configure X server:
redhat-config-xfree86
Configure network:
redhat-config-network
Manage software:
redhat-config-packages

# Handling NTFS
New Technology FileSystem (NTFS) HOWTOs
Linux NTFS project

I tried going to the virtual terminals as well..but the problem is the sysytem hangs ..it does not take any inputs either..i cant restart my computer manually also..i have to switch off the mains.
is it because ..my first partition contains windows and second linux?

dunno if it helps but my computer wont boot into gnome using red hat 9.

the problem seems to be that red hat uses 3d acceleration driver by default and i cant figure out how to configure it to use the non-3d-accelerated driver (yes im a noob). for some reason, you can test all this stuff in the red hat x configurator and it all seems to work fine but when you reboot, the screen just goes blank. mandrake lets you test and configure and select the non-3d-accelerated driver and the big difference is that it actually works.

you can also avert launching X by pressing "o" when grub comes up to change the kernel args by then entering a space and the "3" and then you can see if redhat-config-xfree86 will let you do anything (mine just goes blank) so i havent really solved the issue yet on my box.

Most likely your problem is with the nvida module.

I had the same problem when installing RH8 and here is how I solved it, it might help you, or someone might have a better way.

I downloaded the nvidia driver from http://www.nvidia.com/content/drivers/drivers.asp

Since I couldn't start a terminal because I foolishly installed redhat with graphical login as default, I used an old Mandrake installation disk in rescue mode (My redhat 8 disk has a broken rescue mode). I mounted the Redhat boot partition as /mnt/disk and ran emacs to edit /etc/inittab

chroot /mnt/disk /usr/bin/emacs /etc/inittab

changed the default runlevel by editing the line
id:5:initdefault:
to
id:3:initdefault:

saved the file in emacs by ctr-x ctrl-s and exited by ctrl-x ctrl-c

Maybe I could have used telinit instead.

After rebooting, installed the nVidia driver by typing
bash NVIDIA-Linux-x86-1.0-4496-pkg2.run

I was prompted to accept a licence agreement and then given the option to download the latest version from ftp, I said NO here because it is not necessary.

The next step was to edit /etc/X11/XFree86Config. The driver installed a sample file named /usr/share/doc/NVIDA/XF86Config.sample.

After backing up the original XF86Config I took this sample file, changed the FontPath to "unix/:7100" and added the option "dpms" to the monitor section, and saved it as /etc/X11/XF86Config.

Typed "startx"... and it worked.

Good Luck