Is it possible to whitelist URLs or IPs so when a user tries to connect to a webpage that is not in the allowed list they either get an error, or (prefferably) get redirected to an allowed web page that says "that site isn't allowed". But then when they go an allowed IP, everything works as usual.

I'd prefer to block all ports other than 80/443 and whatever SSH is

Here's my guess on how it should be done:

iptables {block all ports}
iptables {unblock ports 80, 443, and 22}
iptables {block all IPs}
iptables {unblock 111.111.111.*, 111.111.222.*, and 111.111.333.*}

If you know the syntax, please let me know, as this would be cool to have done. Users are getting way out of hand with our windows machines and messing everything up, so I'm trying to set up a linux box that will be as un-mess-upable as possible.

The quickest and simplest thing would be to install something like IpCop (www.ipcop.org). Self-installing firewall. Look in their forums for the 3rd party application to filter internet sites. I've used it in the past, and it was rather easy to install. Then you select what types of sites you want to block (porn, hate, sports, etc) and restart the engine. Poof. Now when someone goes to porn.com or whatever, they get redirected to an access denied webpage.