The easy way is to assign static IP address to each machine. They can be any IP address in the range of your sub-net not used by the DHCP server in the router.
You need to know how to connect to your router and how to configure it. Usually form a web browser type 192.168.x.1 where x is the number of your sub-net. Once connected, look for the range of addresses the DHCP server can give out. Usually you can control this number. On my router, I set it for 4 addresses only. I have 3 systems using DHCP, so that leaves one spare address.
The other way is to determine the MAC addresses of the cards in each system. That is easy in linux, from the command line issue as root the command 'ifconfig eth0' ( without the quotes, assuming it is eth0 ). That will dump some information on the screen, the MAC address is in there. Here is what it will look like.
eth1 Link encap:Ethernet HWaddr 00:1D:7D:20:F4:67
inet addr:192.168.1.26 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:628129 errors:0 dropped:0 overruns:0 frame:0
TX packets:589113 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:301913696 (287.9 MiB) TX bytes:94797228 (90.4 MiB)
Interrupt:24 Base address:0x6000
The field HWaddr is the MAC address. Most routers allow you to specify a MAC address, and what IP address to issue to that card. Then you know what IP it will always get.
I think if you look in the networking section in windbloze you can find the MAC address also.
If you need more help with connecting to the router, post the make and model. Most of them have documentation online we can search to find out the details....