On the Linux side, I have never had to set up a CVS service. I believe CVS is a client application. I do have a CVS login (cvsuser), and all my clients log in using ssh: My clients are on the same system where the CVS repository exists and external systems to that.
export CVSROOT=:ext:cvsuser@some_server_name:/home/cvsuser/SRC_TREE
export CVS_RSH=ssh
These go into every clients .bashrc file.
I still have to get help with setting up ssh trust, so I don't have to give the password everytime, but at least I can use CVS.
Now as to the client on Windows, you'll need to go look for something that will talk to CVS. You can try
http://www.tortoisecvs.org/. But test it out on archiving and then updating Microsoft documents, like Word, Excel, and so on.