First you need a working network connection between your Linux box and your computer, at least enough to issue the 'ping' command to the IP address of the target computer from the command line (in Windows, that will be the DOS prompt; in Linux, the console). To do that you will need to set up TCP/IP networking between the boxes. Since you have a home network, then that's probably already setup for those computers. I suggest using LinuxConf (K->Configuration->Other->LinuxConf) to do the same for the Linux box.
You get your Linux box connected to the internet through Mandrake Control Center->Network & Internet->DrakConnect.
Then you can set up the Linux box to act as a firewall to share your internet connection by using DrakGw (Drake Gateway) in the same part of MCC.
On the Windows side you will probably need to set the network to use a gateway to access the internet. I'm not familiar with Win XP, but in earlier versions of Windows, the gatway setting was in network properties of the TCP/IP protocol bound to the adapter that connects to the internet. For example, in Win ME with a modem connection, you would open Network Properties, select the TCP/IP->dialup adapter, click on Properties, and explore the tabs to find the gateway setting.
Hopefully, that should be enough to do what you want.