Change your default runlevel, by editing /etc/inittab. The relevant line is the one that says the following:
You will likely need to change the 5 to a 3. However, it may not be 3 that you need to use - check the comments at the top of the file, usually they tell you which runlevels are used for what. If in doubt, post said comments here.