After wondering about this myself I have found two solutions which seem very easy. The first of these is to use software that all you need to do is tell it what your modem and IP settings are and its up and running (once you configure your windows boxes) For this you need to have a clean hard drive (or you can let it clean it for you.) Its found at www.clarkconnect.org
its based on linux and free. However thats the easy route, and also the boring no fun route (unless you have dialup in which case you'd have to figure out how to use it (being a newbie myself I failed)). Another way is to follow a tutorial thingy that tells you step by step how to set it up, windows boxes included. The best one I've found is at http://ultra.litpixel.com:82/ip_masq...q_content.html
also, if your interested in file sharing, you can use the tutorial at http://ultra.litpixel.com:82/samba/samba_content.html
for instructions for using samba. This site has some other good tutorials for you to check out too www.everythinglinux.com
(its not been updated in quite some time however as the author of the page has moved on to other similar sites). Hope this helps.