You don't need a router. If you only need the machines to talk to each other, you only need s switch or hub. I recommend a switch because they're not that expensive these days and they'll provide better performance. You technically don't even need the swtich for just two computers, but you mentioned adding more later.
You'll need to assign ip addresses somehow. With only two computers, you could just assign them manually. Pick a subnet in one of the rfc1918 subnets. (
http://www.ietf.org/rfc/rfc1918.txt?number=1918) and just configure a different ip address for each machine. If you expect your network to grow large you may consider setting up a dhcp and dns server to automatically assign ip addresses and map hostnames to them.