Hello All

I have been developing this C++ based engineering application for many years now. The code is contained within 100's of source files. I was smart to have separate .h and .cpp files.

I don't understand make and other similar tools, so I have been using various #define #ifndef and #include "<definition_file>.cpp" directives so that it is built in one g++ command.

The complexity of the program is starting to overwhelm me. When ever I add new files or something, I have to spend many hours de-bugging all my directives to ensure that the right *.h and *.cpp files are included in the correct order.

I would like to strip out all the #include "<definition_file>.cpp" directives, so that each *.cpp file is compiled just with the needed *.h files, then linked later. Yet this would require a multiple file compilation process and supporting tools. So I have been looking into the GNU suite of build tools.

I toyed with some tutorials on the automake/autoconf tools. I am disappointed/discouraged that I am getting compilation errors from the resulting makeFile's. Being completely new to makeFiles I don't know where to start debugging this new language.

Also, the fact that I have to define the Makefile.am for every source directory, will make testing very awkward. I have gotten into the habit of testing every source file in some test.cpp file that only includes the necessary *.h files to test that specific source file. I don't want to be making multiple Makefile.am files for every little test program.

I am hoping that experienced developers might be able to give me some advice on managing the development of complex projects. If you know of some alternative tools for compiling multiple files, let me know, I am looking to try anything. Any advice will be appreciated. Thanks,

pilotmm

Take a look at SCons. I've found it quite easy to use.

I'm not very good with makefiles either

I've posted a makefile in http://www.linuxquestions.org/questi...project-380437.

I think that you need to change the linkcc to be able to use it with g++ and obviously the name of the executable.

For a bigger project, it's probably advisable to split the project over different subdirectories in which case this one will no longer work.

I recommend learning to use make. The online manual for GNU make is very useful. I find the section on automatic prerequisites very handy.

You can use "targets" to build your different configurations. For example, you could have a "test" target that goes through your sub-directories and builds only your test programs.

The downside to using make is that it takes quite a while to get to the point where you can write them on your own without referring to examples and documentation. The upside is that make is well documented, and there are many examples available on the internet to show you how to solve almost any problem.

In this case, I do think that SCons {http://www.scons.org/} might well be what you're looking for.

Let's face it: the problem here is figuring out what your Makefile needs to consist of. With hundreds of source-files, that becomes very difficult for a human to do, but it is a straightforward problem for the computer. The goal of "smart build" utilities, such as this one, is not simply "to figure out what needs to be in a Makefile," but to largely eliminate the need to build (and to separately maintain) one. As projects grow bigger and bigger, this becomes more and more of a problem.

Your requirement is not new. Look around.

I looked at the manual and tutorial for SCons. It looks very powerful, but it looks just as hard to learn as make. Also, using SCons seems to require some knowledge of how make works. You have to understand targets:

You have to understand standard make variables:
The vast majority of open source projects rely on make, so I still stand by my suggestion.

Learning to use make will help a person to understand how a typical project gets built (e.g. the Linux kernel). It will also help a user debug build problems in open source projects. If a person is going to spend much time working with open source software, that person will have to acquire some knowledge of how make works.

Hello All

I appreciate all the advice. There seems to be a debate on whether make or SCons is the best route. I am going to experiment with both. I would like to go the GNU route because it would be better accepted by the community. However, as my original post states, I was thrown some errors on a simple tutorial. Hence I am discouraged it will simplify things. I will report back with my experiences as things develop. Maybe it will be useful for others. Once again thanks all for your comments.

Take Care
Pilotmm

Hello All

So I have tried scon. Seems to work Ok, there are some settings you have to set for it to find source files in other directories. I was disappointed that I still had to specify the source files for the dependencies. Thus I still had to search through my source code to determine what *.cpp files should be compiled. This is what I was trying to avoid.

I don't think make would be any easier to use if I have to manually specify dependencies. Given the errors I received before, I am not going to go down that route any further.

I did manage to find a solution to my problem. I created a little utility program that traverses my source code looking for #include <file>.h directives. When a new one is found it will print #include <file>.cpp and then traverse through that file to find more #include directives. I pipe this output to a separate file that I include last in my program. This guarantees that I have everything declared before anything else is defined. I would like to know your opinions, would this utility be useful? It might be added to scon or something else, so source code is automatically found for the dependencies. Let me know and I will post the source.

One complication arose with my template class header files. I had to keep the #include *.cpp directives in the header files of the template class. This is the C++ standard, and the reason for this is obvious, but mentioning it for others facing similar issues.

When multiple file compilation is applied, I received many "duplicate definition" errors with templates classes that are used in more than one object file. The problem was fixed by adding the inline keyword for all my template function specialization. Again notes for other frustrated programmers.

I wonder, are all template functions compiled inline? Also, is there a away to avoid multiple definition errors without forcing specialized templates to be inline? This is not a problem, but some of these functions are large and I would like to keep bloat down.

Using this utility, I managed to simplify the compilation problem without getting any errors about things not being defined. At the same time I did not have to rely on a multiple file compilation process. Could someone please explain why the multiple file compilation process used? What are the benefits? The only thing that I can see is when fixing/debuging you only need to compile the erroneous *.cpp file and link. Speeding up the compilation process. Yet, compiling is so fast on my computer I don't see any really value, given the head aches of keeping track of the dependencies. I am curious why do people use make and scon and all those builders? Just some questions/curiosities.

Thanks for all your help so far. I would still appreciate more comments. I always want to be a better programmer so any advice is always appreciated.

Take Care
pilotmm

As for why people use make and other such utilities to only compile what's required... I think you can probably appreciate that even small projects may one day grow into larger ones and if the methods of managing the build are bad to start with it only causes headaches later on. Would you be happy to compile every file if your project grew twice the size? Also, if you ever plan to distribute the project as source (which we all hope you will... ) then not everyone has a lightening fast computer should they wish to modify your code. People are also used to tar, make, make install.

You can also use gcc for this:
http://gcc.gnu.org/onlinedocs/gcc-4....cessor-Options

See How can I avoid linker errors with my template functions?

Hi MMat, ntubski

MMat, I understand what you are saying. It is my intention at some point to release my code. Naturally I want to have it in a familiar form for other users/developers. At this point I don't have the time to package it nicely for make and such. I have deadlines just to get working code.

ntubski, thanks a lot for addressing my questions. I overlooked C++ FAQ-Lite when I was trying to solve that problem, instead came across bad advice on other forums. That makes me happy, I can have a seperate *.cpp implementation file that tells the compiler what template functions to compile. Hence templates will be compiled as regular functions, just as intended.

ntubski, once again thanks for passing on what obscure gcc options is required to determine the dependencies. I was always suspicious that gcc would have that, but I am always overwhelmed when I go researching the gcc manual.

I am now encouraged by the prospects of make and scon. There is hope that I can apply conventional build tools for my application. It is something I will do eventually. For now I have some deadlines, and I have a hacked method of re-solving dependencies that seems to work and let me go back to programming. So instead I am going to go back to that and hopefully get the program running in good time.

Since not all reasons are always obvious, let me explain why template definitions need to be included when compiling. A template by itself does not compile into code. The template parameters (Typically referred to as T) need to be defined for this to happen. Typically they are only defined at instantiation (ie when the function is called). That is why terms like <long> are appended to functions names and such (Note that template parameter definition can be inferred by the arguments to the various templates functions themselves). Thus, the template definition is only compiled when the compiler comes across a call to the template in other code. If the definition is not available at that point, then the code will not be able to compile.

Thanks to advice from ntubski, you can get around this problem. The problem is that the template parameter needs to be defined for the template definition to be compiled into code. You can define the template parameters in function/class declarations (ie: `int func<int>();'). When the compiler sees this, it will compile the template definition with the whatever you declared for the template parameters (`int' for the example above). This allows you to separate the template declaration with the definition. You just need a third file that tells the compiler what versions of the template are needed.

Note a third file is not absolutely needed, the declarations could be in the template definition file. However, this can be problematic if multiple programs are using different versions of the same template. Under these circumstances, if the declarations are in the definition file, the other programs will be forced to compile versions of the template that are not needed by that program.

All in all I hope that clears up the template issue.

Forgive me for the verbose posts. I like to explain everything for future frustrated and confused people, who have problems just like me.

Take Care
pilotmm