I need to know if it is possible to have my server check an index for a specific file name(s) before it processes a request from the user. The user(s) may often need to download the same file(s) but often find them from different sources. The download link may be different from user to user (http://domain1.com/filenameABC.zip vs http://domain343.com/filenameABC.zip).

The thing is that each user has a dedicated allotement of storage space and the default for the setup is to have the files stored in the users profile.

Problem is, it is very likely that most users will be downloading the same 3-6GB files and I would like to save bandwidth as well as storage.

I'm trying to figure out a way to make this work and don't know where to start looking. Some of the main sources of downloading are going to be OS updates for windows, linux and Apple as well as CD/DVD ISO's often from torrents.

Can anyone suggest a place for me to start looking to make this consolidation possible or know what can be done?

Thanks for any insight on this!

Maybe a proxy?

http://www.squid-cache.org/

As for storing files, if they are on the same partition you can hard link them which effectively just creates pointers to the same set of data.
The users should probably not have write access to the file of course.

If they're not on the same partition, symbolic links work too.

Are you looking for examples of creating linked (symbolic or hard) files in linux?