In your /etc/resolve.conf file you need to put in the nameservers.
Syntax is as follows:
You need at least one, you can have as many as you'd like. I recommend at least two. You should be able to get them from the router status information as you probably want to use your ISP's nameservers.
BTW... if you using DHCP to get your address with the linux box it should do this automatically. Unless your router isn't giving out the information for some reason.