I have to respectfully disagree.
The only way to guarantee that the system will be correctly cleaned is to preserve the build tree, and do "make uninstall" before doing a new installation.
Bear in mind that if you use different configure options (for example, --prefix but there can be many more that can potentially leave crap in your system) files from previous installations will be left behind, and probably in a default path. This will later lead to problems of the kind "I have installed it 1000 times and it complains that it can't find a library, but I see it, and it is there". This problems are extremely hard to diagnose and solve, I've seen this like thousands of times, a tipical case is when users install the nvidia or ati drivers using their package manager, and then (for I-don't-know-what-reason) they reinstall them by hand using the nvidia or amd installer. This leads to cases where the kernel modules and the glx api will not match, and the price is a two weeks vacation fishing libraries all around your fs and wandering why the heck linux is that bad and crappy.
Besides that, when you update a package the newer versions will not necessarily use the same file set. Some files may no longer exist, some others might live in other place. In this case, leftovers are usually not dangerous, though still some problems can arise (and I am not considering relevant the disk space waste).
So, when you are going to update a package to a new version or you are going to use a different set of configure options, first, make sure that you uninstall properly before.