Clustering Filesystems
 

So I have 20 developers with 20 PC's all doing compile / splint / link / debug.

We build 6 product variants from the same code so all in doing one full build I'm running about 4000 processes.

I have distcc'd and ccache'd to the max and my limitation now is splint and link steps.

And I want somehow to convert those 20 PC's into 20 times faster builds...

(Splint == secure lint see www.splint.org if you don't know it.)

Question 1:

How can I make those splints & links faster, much faster?


Question 2:

Which clustering file system would you recommend?

I'm not sure if the word "clustering filesystem" is quite the right word.

I know that there is a "distcc" tool (see here) which purports to be able to distribute compilation workload among several machines. I know that Gentoo offers it, so I presume it works.

I'm already using distcc (and ccache).

Distcc works very nicely for speeding the compiles and probably cannot be much improved on.

This leaves me with the two remaining, undistributed items in the build taking the vast majority of the time....

1) Splint (a static C checker)
2) The link step. Which distcc does nothing about.

For these two items the only option (I can see) is a clustering file system.