After 3 hours of struggling/learning I came across a howto that provided this code:
aptitude install module-assistant
I was watching it run and saw that it was removing alot of stuff, but I didn't want to stop it out of fear something might go wrong.
Long story short, I had to re-install synaptic and my theme had changed, there's icons and programs missing from my menus.
Is there an easy way to get them back?
By the way: If there's any other newbies trying to get the drivers installed but are running into the "cannot locate kernel source tree" or whatever the message was... I had to get out of X (ctrl/alt/bkspace)
And then install the drivers I downloaded from nVidia. It was then able to build the kernel source itself.