I think the .config file that resides in your kernel source directory is all you will need if you plan to recompile with your current settings. (Can anyone confirm this?) It's not absolutely necessary to keep it - by the time you're ready to recompile, you might be ready to configure it all yourself (though everyone makes a few mistakes the first time
So what I'd recommend is backing up the .config file from your old kernel source tree, and then removing the whole directory. That oughta clear up more than 100MB (mine's about 190MB for kernel 2.4.19).