What you are describing, is what 'makefiles' and the utility 'make' are for - keeping track of separate code directories or libraries, and their dependencies.
If you want to use gnu make, take a look at the documentation here:
Or pick up any book or online tutorial describing how to use make, and it will show you what to do. Basically you specify a series of 'targets' each directory could be one target, then you need to specify which other targets need to be built first, the dependencies... Depending on how many code files you have, it can get very complicated. Good luck!