#2 on the FAQ.
Change inittab's default runlevel from 5 to 3 - goes for most distros, so probably RH. Or just drop out of X - why does it have to be at boot?
And you may be able to pass a commandline option through LILO to the kernel, for all I know. Still, changing the default runlevel's the usual way.