I have a small home network. How do i get my fedora core 4 computer to read my network?

Gonna need a bit more info like what sort of network you are running (wired/wireless?) what type of machines are connected to it, what if any proxies you have running etc. etc. what you gave above is no-where near enough info

Wired Networking with 2 windows on the network

Linksys router providing connection

I know the DNS,Gateway,Submask for my network..but i dont know where to put them in

Ok so for the wirless part unless your wireless card is nautrally detected (i dont see much of this) you're going to need to look into Ndiswrapper, which is a wrapper to run windows drivers on linux.

for the actual network you can look into sharing files via samba, or NFS, seeing the other machiens should be as simple as mounting the windows machine as an ip:/drive via NFS.

Samba also has the ability to act as a domain controller etc. and is also a good way for windows machines to share with Linux via a workgroup or domain.

There's a good start for you

run system-config-network from a terminal to access settings your your network interfaces