Your solution is in using some tool like "make". Most libraries are distributed with a "configure" script inside them, you should follow the instructions (usually in the README file) before continuing.
Basically, after having run "configure", you'll end up with a "Makefile" file. Then, you can run the command "make".
This is the most important point: make will compile only things that are changed! And then relink only parts that must be relinked. So, no useless recompilation will happen.
For more information, you should refer to manpages or to documentation or books, as this is a very important, and long, topic.