Hi,
You do not need a router for what you want to do but it is the simplest option. Without a router you can connect your two machines via a crossover cable. You can probably buy them but its easy to make them with the right tool. You are effectively making a piece of cat 5 bi-directional. Here is some instructions:
http://www.perfectdrivers.com/howto/crossover.html. You would then connect the machines together with it NIC to NIC, load the ip stack on each machine (making sure they are on the same subnet) and ping each other. Once you can ping each other you would need to load some kind of xclient on to your XP box (so it can display Unix xwindows) then from within this xclient ssh to Linux, login, set the display to your XP IP address and you should be away.
Once you have set the display from the XP xclient type 'xclock' and see if the clock appears, if it does your seeing this from your Linux box.
If you choose to buy a router the nice thing about this is you could have more than one client connect and you could do it over wireless so no need for cables from XP to the router.
Hope this helps,
Colin