Help answer threads with 0 replies.
Go Back > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Linux - Kernel This forum is for all discussion relating to the Linux kernel.


  Search this Thread
Old 06-15-2006, 12:37 AM   #1
Registered: Aug 2001
Location: ChristChurch New Zealand
Distribution: Ubuntu
Posts: 279

Rep: Reputation: 52
Clustering Filesystems

So I have 20 developers with 20 PC's all doing compile / splint / link / debug.

We build 6 product variants from the same code so all in doing one full build I'm running about 4000 processes.

I have distcc'd and ccache'd to the max and my limitation now is splint and link steps.

And I want somehow to convert those 20 PC's into 20 times faster builds...

(Splint == secure lint see if you don't know it.)

Question 1:

How can I make those splints & links faster, much faster?

Question 2:

Which clustering file system would you recommend?
Old 06-15-2006, 10:53 PM   #2
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,720

Rep: Reputation: 1294Reputation: 1294Reputation: 1294Reputation: 1294Reputation: 1294Reputation: 1294Reputation: 1294Reputation: 1294Reputation: 1294
I'm not sure if the word "clustering filesystem" is quite the right word.

I know that there is a "distcc" tool (see here) which purports to be able to distribute compilation workload among several machines. I know that Gentoo offers it, so I presume it works.
Old 06-15-2006, 11:27 PM   #3
Registered: Aug 2001
Location: ChristChurch New Zealand
Distribution: Ubuntu
Posts: 279

Original Poster
Rep: Reputation: 52
I'm already using distcc (and ccache).

Distcc works very nicely for speeding the compiles and probably cannot be much improved on.

This leaves me with the two remaining, undistributed items in the build taking the vast majority of the time....

1) Splint (a static C checker)
2) The link step. Which distcc does nothing about.

For these two items the only option (I can see) is a clustering file system.


clustering, filesystems, linking

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
clustering Ammad Linux - General 1 08-11-2005 08:05 AM
clustering mimithebrain Linux - General 2 06-13-2005 09:02 AM
windows filesystems vs. linux filesystems irfanhab General 8 05-25-2004 07:21 AM
Clustering Stephanie General 5 01-29-2004 12:04 AM
New to clustering... sb73542 Linux - Networking 2 07-31-2003 11:05 AM

All times are GMT -5. The time now is 04:09 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration