In case anyone reads this, it turns out that I had installed the wrong drivers for my video card at install. I mistakenly installed the default Debian selection, which is 'vesa'.
If you are about to install Debian and the xserver-xfree86 package; if you have an nVidia card, choose 'nv' in the driver list, if you have an ATI based card, choose 'ati'.