LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 06-15-2006, 12:37 AM   #1
cyent
Member
 
Registered: Aug 2001
Location: ChristChurch New Zealand
Distribution: Ubuntu
Posts: 398

Rep: Reputation: 87
Clustering Filesystems


So I have 20 developers with 20 PC's all doing compile / splint / link / debug.

We build 6 product variants from the same code so all in doing one full build I'm running about 4000 processes.

I have distcc'd and ccache'd to the max and my limitation now is splint and link steps.

And I want somehow to convert those 20 PC's into 20 times faster builds...

(Splint == secure lint see www.splint.org if you don't know it.)

Question 1:

How can I make those splints & links faster, much faster?


Question 2:

Which clustering file system would you recommend?
 
Old 06-15-2006, 10:53 PM   #2
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940
I'm not sure if the word "clustering filesystem" is quite the right word.

I know that there is a "distcc" tool (see here) which purports to be able to distribute compilation workload among several machines. I know that Gentoo offers it, so I presume it works.
 
Old 06-15-2006, 11:27 PM   #3
cyent
Member
 
Registered: Aug 2001
Location: ChristChurch New Zealand
Distribution: Ubuntu
Posts: 398

Original Poster
Rep: Reputation: 87
I'm already using distcc (and ccache).

Distcc works very nicely for speeding the compiles and probably cannot be much improved on.

This leaves me with the two remaining, undistributed items in the build taking the vast majority of the time....

1) Splint (a static C checker)
2) The link step. Which distcc does nothing about.

For these two items the only option (I can see) is a clustering file system.
 
  


Reply

Tags
clustering, filesystems, linking



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
clustering Ammad Linux - General 1 08-11-2005 08:05 AM
clustering mimithebrain Linux - General 2 06-13-2005 09:02 AM
windows filesystems vs. linux filesystems irfanhab General 8 05-25-2004 07:21 AM
Clustering Stephanie General 5 01-29-2004 12:04 AM
New to clustering... sb73542 Linux - Networking 2 07-31-2003 11:05 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 08:12 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration