Since I know bash/shell and not Python, bash would be my personal choice.
In my script, I'd (this is very rough) use the `find` command to locate every directory within the parent tree, including the top directory. Then, inside every one of the found directories, I'd use `find` again (with a -maxdepth of 1) to locate all directories & html files present; then I'd create the index.html file, and insert (probably just using `echo`) a <href=link> or whatever, to each directory and/or html file present within my current directory.
Again, this is a pretty rough draft, but I think the idea is a decent one though pretty simplified in my description.
Once you begin to come up with some code and posting it for us to see where you're at, we can help you further with problem areas that you run into.
NOTE: You probably want to set up a testing directory tree, on which to test your script or program as it develops. I would actually have a testing tree, which I would copy to a second location on which to test my program; after each test run, I'd delete the copy of the testing tree and replace it fresh with a new copy, after having examined the results of the program run.