Yes that is right. You will need to put an entry in the hosts file of each machine on your lan:
linux - /etc/hosts
win9x - C:\windows\hosts
win2k/XP - C:\windows\system32\drivers\etc\hosts
With the windows files make sure that there is no extenstion. In win9x they had a sample file with a .sam extenstion. The hosts file should have no extension.
The entry should be something like:
192.168.1.100
www.yourdomain.com
To give it multiple names:
192.168.1.100
www.yourdomain.com yourdomain.com server