LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Gentoo
User Name
Password
Gentoo This forum is for the discussion of Gentoo Linux.

Notices

Reply
 
Search this Thread
Old 07-13-2012, 10:43 AM   #1
fakie_flip
Senior Member
 
Registered: Feb 2005
Location: san antonio, texas
Distribution: Fedora 64 bit RAID0 + LUKS, CentOS (server), Backtrack, Gentoo Hardened
Posts: 1,441

Rep: Reputation: 80
distcc failing to compile


I've been making several attempts to get distcc working on my older Dell Latitude C800. It only has a 1ghz Pentium 3 Coppermine CPU and 512mb of ram. So I think it could really benefit from the efficiency of Gentoo, smaller and faster binaries, etc., but the compiling usually is taking too long. Especially when I'll need to rebuild the entire system or run revdamp-rebuild.

I have a desktop with a 2.8ghz quad core and 4GB ram, so I am attempting to use it to do the compiling. Last time I had Gentoo on the desktop, it compiled extremely fast, but I don't think the extra performance of Gentoo was so noticeable on it because it's already a fairly high end computer.

So on both machines, I've set:

Code:
export DISTCC_LOG=/var/log/distccd.log
export DISTCC_VERBOSE=1
Only the laptop produced a log.

/var/log/distccd.log:
http://vpaste.net/PWrNQ

Laptop LAN IP: 192.168.1.117
Desktop LAN IP: 192.168.1.127

On the laptop, I've

Code:
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24"
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.1.117"
in the /etc/conf.d/distccd

I have done distcc-config --set-host "192.168.1.127" and the /etc/distcc/hosts shows that ip in there, that it had been set.

In the make.conf, I set

Code:
FEATURES="distcc"
MAKEOPTS="-J8" # 2X cpus, quad core, so 8
After setting all of that up on the laptop, I then did

Code:
/etc/init.d/distccd restart
==================

On the desktop, I've

Code:
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24"
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.1.127"
in the /etc/conf.d/distcc

Code:
export DISTCC_LOG=/var/log/distccd.log
export DISTCC_VERBOSE=1
but no log was created.

set in the /etc/make.conf
Code:
FEATURES="distcc"
MAKEOPTS="-j8"
Code:
/etc/init.d/distcc start
This last attempt, I emerged epdfview on the laptop. I ran distccmon-gui on both machines. It showed no activity. I checked the processes on the desktop. I saw nothing that indicated compiling and the cpu usage was very low.

==================

Both computers have the same distcc version.
On the laptop, I had to upgrade the gcc version to

Code:
beastlinux ~ # gcc -v 2>&1 | tail -1
gcc version 4.6.2 (Gentoo Hardened 4.6.2 p1.4, pie-0.5.0) 
beastlinux ~ #
I then rebuilt libtool as described by the Gentoo Docs for upgrading gcc.

CHOST in make.conf on both machines is the same
Code:
CHOST="i686-pc-linux-gnu"
To avoid cross-compiling, I burned a 32-bit Sabayon/Gentoo disk and booted it up on the desktop. I had first tried doing all of this in virtual box. and the networking was all set up correctly. I could nmap port 3632 from either machine to the other and see that it was open and running distcc. I couldn't seem to figure out what was going wrong, so this last attempt was booting it up as a live cd, and then setting up distcc. If I can get distcc working first, then I'll consider using a 32-bit chroot or cross-compiling.


EDIT:
It seems be working now. According to logs, messages like these:

Code:
distcc[4938] (dcc_r_file_timed) 3148 bytes received in 0.000477s, rate 6445kB/s
distcc[4938] 4681009 bytes from main.cpp compiled on 192.168.1.127 in 9.5097s, rate 481kB/s
distcc[4938] (dcc_unlock) release lock fd4
distcc[4938] compile main.cpp on 192.168.1.127 completed ok
distcc[4938] elapsed compilation time 14.536347s
--
distcc[4868] (dcc_r_file_timed) 38936 bytes received in 0.001141s, rate 33325kB/s
distcc[4868] 4625570 bytes from notebooks/notebookmenuitem.cpp compiled on 192.168.1.127 in 9.8320s, rate 459kB/s
distcc[4868] (dcc_unlock) release lock fd4
distcc[4868] compile notebooks/notebookmenuitem.cpp on 192.168.1.127 completed ok
distcc[4868] elapsed compilation time 43.951518s
--
distcc[4907] (dcc_compile_remote) client finished sending request to server
distcc[4907] (dcc_note_state) note state 5, file "(NULL)", host "192.168.1.127"
distcc[4907] (dcc_select_for_read) select for read on fd6 for 300s
--
distcc[4907] (dcc_r_file_timed) 39656 bytes received in 0.001285s, rate 30137kB/s
distcc[4907] 5958039 bytes from dbus/remotecontrol.cpp compiled on 192.168.1.127 in 11.3617s, rate 512kB/s
distcc[4907] (dcc_unlock) release lock fd4
distcc[4907] compile dbus/remotecontrol.cpp on 192.168.1.127 completed ok
distcc[4907] elapsed compilation time 31.423067s
Still, the laptop is doing most of the work. It's cpu is pegged at nearly 100% while the quad core has a bump in cpu usage here and there.

The distcc monitor apps distccmon-text, distccmon-gui, distccmon-gnome do not work. None of them have displayed a single message about the distcc helping with the compiling.

Last edited by fakie_flip; 07-13-2012 at 02:21 PM.
 
Old 07-15-2012, 11:03 AM   #2
segmentation_fault
Member
 
Registered: Sep 2008
Location: Ioannina, Greece
Distribution: Gentoo
Posts: 332

Rep: Reputation: 54
The distccmon-* doesn't show anything when running emerge. When you compile a kernel you will see some output there. Also, not all packages use distcc. If the package is small, it will be compiled locally. Also the -j8 may be overkill for the laptop. If the package doesn't utilize distcc, the laptop will end up trying to compile with 8 threads, which will result in a huge load avg. I personally use (# of cores on the local host) + (# of distcc hosts) + 2. But I don't have a pc with more than 2 cores, so I can't be sure.
I think you can choose a different apcroach. You can compile the packages on the desktop and then export the distfiles to the laptop. But I don't have any experience on that field.
 
Old 07-15-2012, 11:16 PM   #3
fakie_flip
Senior Member
 
Registered: Feb 2005
Location: san antonio, texas
Distribution: Fedora 64 bit RAID0 + LUKS, CentOS (server), Backtrack, Gentoo Hardened
Posts: 1,441

Original Poster
Rep: Reputation: 80
Quote:
Originally Posted by segmentation_fault View Post
The distccmon-* doesn't show anything when running emerge. When you compile a kernel you will see some output there.
Really? http://i.imgur.com/zD6i1.jpg

Quote:
Originally Posted by segmentation_fault View Post
Also the -j8 may be overkill for the laptop.
As you can see -j8 has the virtual machine that helping the compile overloaded. Just look at that cpu usage.. And from the previous picture, the laptop is over loaded with compiling processes in distcc. Actually, the laptop is not set to do ANY compiling when distcc is in use. Yes I know what distcc does and doesn't compile. It's not always enabled nor is the -j8. I comment it out when not using distcc.
http://img6.imagebanana.com/img/5so0...llRunningO.png

Quote:
Originally Posted by segmentation_fault View Post
I personally use (# of cores on the local host) + (# of distcc hosts) + 2.
Quote:
(Modify MAKEOPTS to include -jN, where N is twice the number of CPUs +1 available) MAKEOPTS="-jN"
Quoted from http://www.gentoo.org/doc/en/distcc.xml It actually makes mention of using that formula twice.

Quote:
distcc is intended to be used with GNU Make's -j option, which runs several compiler processes concurrently. distcc spreads the jobs across both local and remote CPUs. Because distcc is able to distrib- ute most of the work across the network a higher concurrency level can be used than for local builds. The -j value should normally be set to about twice the total number of available CPUs, to allow for some tasks being blocked waiting for disk or network IO. distcc can also work with other build control tools such as SCons.
Quoted from the distcc man page. http://www.manpagez.com/man/1/distcc/
I just asked why my distcc was not working, not all that. I ended up solving it on my own (you can see in the screenshots, it is working), so I'll mark this thread as solved.

Last edited by fakie_flip; 07-15-2012 at 11:18 PM.
 
Old 07-16-2012, 09:01 AM   #4
segmentation_fault
Member
 
Registered: Sep 2008
Location: Ioannina, Greece
Distribution: Gentoo
Posts: 332

Rep: Reputation: 54
Well, I have never seen any output on distccmon, but I don't run it always. Maybe it's something I misconfigured.
Anyway, it's good you figured it out.
 
Old 07-17-2012, 11:56 AM   #5
fakie_flip
Senior Member
 
Registered: Feb 2005
Location: san antonio, texas
Distribution: Fedora 64 bit RAID0 + LUKS, CentOS (server), Backtrack, Gentoo Hardened
Posts: 1,441

Original Poster
Rep: Reputation: 80
Here's how you get output on distccmon-gui or distccmon-text.

Create /etc/env.d/99local

Add these three lines to it.

DISTCC_VERBOSE=1
DISTCC_LOG="/var/log/distccd.log"
DISTCC_DIR="/var/tmp/portage/.distcc"

env-update && source /etc/profile

Btw, I read about creating this file under /etc/env.d/99local on the Gentoo Handbook, so it's supported, not a hack.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
distcc to compile Gentoo on the laptop fakie_flip Gentoo 1 06-14-2012 03:03 AM
How do I cross-compile/distcc for FreeBSD/i386 on Linux/amd64? thosch97 Linux - Software 0 04-27-2012 09:42 AM
distcc ERROR: compile dir.c on <server ip> failed DragonM15 Linux - Software 1 01-08-2009 03:24 PM
Compile 2.6.9 keeps failing. musicman_ace Suse/Novell 6 11-27-2004 04:16 AM
using distcc to compile pengStudent Programming 3 03-10-2004 08:33 PM


All times are GMT -5. The time now is 09:04 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration