LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices

Reply
 
Search this Thread
Old 06-15-2006, 01:37 AM   #1
cyent
Member
 
Registered: Aug 2001
Location: ChristChurch New Zealand
Distribution: Ubuntu
Posts: 257

Rep: Reputation: 45
Clustering Filesystems


So I have 20 developers with 20 PC's all doing compile / splint / link / debug.

We build 6 product variants from the same code so all in doing one full build I'm running about 4000 processes.

I have distcc'd and ccache'd to the max and my limitation now is splint and link steps.

And I want somehow to convert those 20 PC's into 20 times faster builds...

(Splint == secure lint see www.splint.org if you don't know it.)

Question 1:

How can I make those splints & links faster, much faster?


Question 2:

Which clustering file system would you recommend?
 
Old 06-15-2006, 11:53 PM   #2
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,423

Rep: Reputation: 1158Reputation: 1158Reputation: 1158Reputation: 1158Reputation: 1158Reputation: 1158Reputation: 1158Reputation: 1158Reputation: 1158
I'm not sure if the word "clustering filesystem" is quite the right word.

I know that there is a "distcc" tool (see here) which purports to be able to distribute compilation workload among several machines. I know that Gentoo offers it, so I presume it works.
 
Old 06-16-2006, 12:27 AM   #3
cyent
Member
 
Registered: Aug 2001
Location: ChristChurch New Zealand
Distribution: Ubuntu
Posts: 257

Original Poster
Rep: Reputation: 45
I'm already using distcc (and ccache).

Distcc works very nicely for speeding the compiles and probably cannot be much improved on.

This leaves me with the two remaining, undistributed items in the build taking the vast majority of the time....

1) Splint (a static C checker)
2) The link step. Which distcc does nothing about.

For these two items the only option (I can see) is a clustering file system.
 
  


Reply

Tags
clustering, filesystems, linking


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
clustering Ammad Linux - General 1 08-11-2005 09:05 AM
clustering mimithebrain Linux - General 2 06-13-2005 10:02 AM
windows filesystems vs. linux filesystems irfanhab General 8 05-25-2004 08:21 AM
Clustering Stephanie General 5 01-29-2004 01:04 AM
New to clustering... sb73542 Linux - Networking 2 07-31-2003 12:05 PM


All times are GMT -5. The time now is 09:20 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration