yes gzipping is helpful in preserving more space but still it doesn't solve the main question, i need some sort of script to organize an html database of files scrambled on multiple disks.

I can confirm (through monitoring my own site) that both IE6 and 5 do also support compressed http. My guess is that the fact that the URI ends in “.gz” threw off IE6. It probably has a rule for downloading by extension (application/gzip) rather than listening to the Content-Type reported by the server (text/html). In most situations using web compression this problem does not exist since the URIs themselves look “normal” to the browser.

Okay, here’s how I understand it. You split up the “content” to multiple media, but leave the “website part” on each media. What I mean to say is that the html, css, and perhaps a few jpg, gif, or png files take up an insignificant amount of space, so you copy them to all of the disks. The other content (videos, pdfs, etc.) do take up significant space, and these are then allocated to different media.

So theriddle’s initial suggestion (to have a dummy page for the nonexistent entries which informs the user the correct disk to load) will not work, since it is specific to html. In particular, you could survive with a “dummy pdf file” and a “dummy mpeg” and so on, but this is not such an elegant solution.

Another solution (again) involves setting up a webserver. But compression is not the purpose of the server. What you can do is write your own 404-page script. The script would have access to a list of files and their locations (this list would also be insignificant in size and could be copied to each disk).

For example, suppose I have a list of files on each disk in a file by the disk number in a directory called “disks” located in the cgi-bin for the server. So it might be something like this:
Then a preliminary 404 error script might look like:
Of course there is currently no sanitation, but it should work in 99% of the cases. You can improve as you see fit.

Another option, compared to osor's suggestion, is to set up an HTML redirect, for example

osor, if i clicked on a link to a file and the server determined the disk containing the linked file, when i insert the correct disk will the file be loaded automatically or do i have to traverse through the disk tree?

Unless I am missing something, an HTML redirect would apply to HTML files, whereas the bulk of the OP’s files are non-html. You could always do a server-side redirect, but if you are going through the trouble of setting up a server, a 404 ErrorDocument seems more appropriate.

When you loaded the correct disk, and just refresh (e.g., F5 on many browsers), the correct file will be accessed. You could get even fancier and have some javascript in the errorpage that basically refreshes for you after you tell it you have loaded the proper disk.

theriddle, i guess i got what you're trying to say. you mean that for every link to a file not located on the working disk i link instead to an html page telling that the file isn't on this disk and what disk to find it on. if that's it, then it's great i know and it'll work for the website i want to archive if i archived it alone. but also i want to make some more sophisticated a solution, i want to put all my data on disks, in a scrambled way according to their sizes so as to maintain more space, then make a large html index on a separate cd with all the web pages/websites i downloaded -this separation will allow me to centralize control to a major index of all my content which will be easier to manage also will allow me to add remove content without further editing of all the library. then when i activate a link in the master index, i get a page or a notification just like your idea, and when i insert the disk the file is automatically loaded -this is the new bit i guess!.

ok osor if what you say is possible, could you redirect me to resources where such solution or usage is explained in more details. also tell me which server(s) is the best choise for both linux/windows and may be freebsd.

Well apache runs on pretty much anything, and for redirecting error pages, you use the ErrorDocument directive in the conf file. Apache is probably a little heavy for your needs, so you might try lighttpd, which also runs on many platforms (although there you probably have to find 3rd-party precompiled binaries for windows). For this, you would use something like server.error-handler-404.

Also, you will need some sort of interpreter if you want to use CGI for your error document. The example I gave above was not very portable, since it used bash and some features specific to GNU grep. For POSIX systems, you would want to use plain sh and no platform-specific extensions. I am unsure of how to proceed in windows, since I don’t think it has by default /bin/sh (but I could be wrong). You might try bundling microperl with the webserver, which shouldn’t take up too much space. This is probably a more natural language for your CGI anyway, but you don’t want to use and CPAN packages, just plain Perl.

Anyway, the actual implementation of the scripts is up to you, but I gave an example of a “first draft”. If you have a central disk which houses the manifests for each of the other disks, you will have to have a two-step script. For example, when a file is not found, the first script will instruct you to load the central disk and press a submit button. This would load a different CGI script residing on the central disk, which would inform you what disk you want to find the file on. Upon inserting the correct disk, you would then press another button, which would take you to where you intended to go in the first place.

wow this is a little unusual to me as my experience in web technologies is not enough -i just know a little html without which i couldn't edit the website. so i need some introductory documentation, where could i find that? apache documentation or will i need third party documentation/books. online documents are always welcome.

If you are new to this, I would recommend lighttpd instead of apache (much simpler to configure, and fewer places to mess up). There is documentation on its website. CGI is just a way to use scripts in place of html files, so when a certain request is made to the webserver, dynamic rather than static content may be delivered. If you are familiar with shell scripting, a properly-configured CGI script will be launched with a few environment variables set (such as $REQUEST_URI). It is up to you to use the variables to produce the output you desire (in this case, you grep the manifest files for the requested uri, and print out which one has it). There are CGI tutorials all over the web, but the only variable you will really need to deal with is $REQUEST_URI.

Perl is a common language used for CGI, and microperl is a (severely) stripped build of the interpreter (which you can have for various platforms). Since you will need very little of Perl’s functionality, while having two or three interpreters bundled on each disk, I think it is a good choice to use. If you need help with a script, just post.

Finally, I think you’ll find that a good search engine is the best source of documentation.

thank you all.