LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 01-30-2019, 07:55 PM   #1
1337_powerslacker
Member
 
Registered: Nov 2009
Distribution: Slackware64-current,Ubuntu,openSuSE,Manjaro
Posts: 829
Blog Entries: 9

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
[HOWTO] Set up icecream


As some of you may have noticed, Pat V has added icecream to -current. What others may not know is what purpose for which icecream was created. In a nutshell, icecream is a centralized model for distributed compilation of packages; similar to distcc, but uses a central "server" to coordinate compiling.

Why would someone want to use this?
  1. A lot of users nowadays have lots of computers lying around their house with no purpose; just taking up space, but their owners are unwilling to throw away what still works.
  2. There are some packages which users may want to compile themselves, but one machine won't cut it (takes too long). Sharing the load among multiple machines cuts down on the load that one machine had to do in the past.
  3. We are a community-based distro; true to its roots, Slackware's strength lies in the willingness of its users to improve on deficiencies, such as compiling their own packages (Perhaps taking the load off of core members who need a break)?

How to set it up

Software

I would recommend a fresh install of -current, and exclude E,KDE,X,XAP,& XFCE as these won't be needed for the client machines.

Post-install, set your machine hostname to whatever your fancy is; icecream ignores this and uses the domain name (next prompt), so make sure that all of the machines you intend to use for icecream are in the same domain! Next, select static IP and assign it an IP of the form 192.168.x.x, where the first x is usually 0, and the second x can be any number between 1 and 254. (0 and 255 are used for special purposes).

On first boot, the client machines should have the rc.iceccd startup script made executable. Select the machine you intend to use as a scheduler machine and make the rc.icecc-scheduler script executable. Only one machine can have this script executable! This is usually the main machine. I usually don't need to set up an additional user account, as these machines are on a private network and totally inaccessible from the Internet;and, per the icecream wiki, root access is necessary for it to run. No additional setup for icecream is necessary.

From the icecream README.md:
Quote:
It is possible to run the scheduler and the daemon on one machine and only the daemon on another, thus forming a compile cluster with two nodes.
Quote:
On the scheduler machine, in your user's .bashrc file, add this line:
Code:
export CC=/usr/bin/icecc
This will tell the make command to use the icecc program (which is a wrapper program for gcc), because icecc and gcc reside in the same directory.

Execute
Code:
. ~/.bashrc
if you don't want to reboot. This will force bash to re-read its config file and incorporate the change you made.
EDIT: Via volkerdi:

Quote:
There's actually no need to define CC because if /etc/profile.d/icecream.sh sees either iceccd or icecc-scheduler running on the machine, it will add /usr/libexec/icecc/bin to the beginning of the $PATH, and this directory contains symlinks from all the usual compiler names to /usr/bin/icecc.

Really, all there is to do is make /etc/rc.d/rc.icecc-scheduler executable on one machine, and /etc/rc.d/rc.iceccd executable on all the machines that will be part of the compile cluster. And then set -j high enough.
Hardware

Obviously, several computers are involved. Just how many is actually immaterial to icecream, but can accommodate what spare computers you have lying around.

I highly recommend connecting all computers through a gigabit switch,as they are inexpensive. Make sure that the computer running the scheduler can see all of the client computers via ping.

I have all my machines hooked up to a keyboard and monitor for local control, but if you like to ssh into your machines "remotely", feel free to do so.

That's it! You can use any number of packages, but I used a kernel compile to demonstrate icecc's power. The number of jobs used will obviously be dependent on the total number of cores in your cluster. I have a total of 22 (1 octa-core machine, 3 quad-core machines and 1 dual-core). I add one extra job per machine, so I use a command like this:

Code:
make -j 27
Quote:
The load on my main machine dropped significantly, from ~115 degrees F to ~80 degrees, and compile time dropped appreciably as well.
After having also enabled the daemon on my scheduler machine, the temp did increase to near my previous temp (about 7-10 degrees cooler) for a much lesser period of time.

This is obviously a basic tutorial, and I don't claim to be an expert in icecream, so feel free to send constructive critique my way and/or suggestions for improvement.

Thanks for reading!

Last edited by 1337_powerslacker; 02-06-2019 at 12:38 PM. Reason: Clarification (per volkerdi & chris.willing)
 
Old 01-30-2019, 09:01 PM   #2
chris.willing
Member
 
Registered: Jun 2014
Location: Brisbane, Australia
Distribution: Slackware,LFS
Posts: 711

Rep: Reputation: 464Reputation: 464Reputation: 464Reputation: 464Reputation: 464
Quote:
Originally Posted by 1337_powerslacker View Post
...
so make sure that all of the machines you intend to use for icecream have the same domain name!
...
This instruction doesn't make sense to me. Could you clarify please?

chris
 
Old 01-30-2019, 09:12 PM   #3
1337_powerslacker
Member
 
Registered: Nov 2009
Distribution: Slackware64-current,Ubuntu,openSuSE,Manjaro
Posts: 829

Original Poster
Blog Entries: 9

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
Quote:
Originally Posted by chris.willing View Post
This instruction doesn't make sense to me. Could you clarify please?

chris
The hostname is actually made up of two parts, which is formally called the Fully Qualified Domain Name, or FQDN. The first part of the hostname is whatever you want to call your machine. As an example, my sig shows my icecream setup. The second part of the hostname is the domain name, which forms a specific group of networked computers. This is what icecream uses to address machines which also have iceccd running. I use the simple 'org' as a domain name. Not fancy, but it works.

Here's something you can read if you're still confused.

Last edited by 1337_powerslacker; 01-30-2019 at 09:14 PM.
 
Old 01-30-2019, 10:42 PM   #4
volkerdi
Slackware Maintainer
 
Registered: Dec 2002
Location: Minnesota
Distribution: Slackware! :-)
Posts: 1,966

Rep: Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428Reputation: 6428
Quote:
Originally Posted by 1337_powerslacker View Post
On the scheduler machine, in your user's .bashrc file, add this line:
Code:
export CC=/usr/bin/icecc
This will tell the make command to use the icecc program (which is a wrapper program for gcc), because icecc and gcc reside in the same directory.
There's actually no need to define CC because if /etc/profile.d/icecream.sh sees either iceccd or icecc-scheduler running on the machine, it will add /usr/libexec/icecc/bin to the beginning of the $PATH, and this directory contains symlinks from all the usual compiler names to /usr/bin/icecc.

Really, all there is to do is make /etc/rc.d/rc.icecc-scheduler executable on one machine, and /etc/rc.d/rc.iceccd executable on all the machines that will be part of the compile cluster. And then set -j high enough. Per ppr:kut, it doesn't hurt to set the job number too high with this, because the scheduler won't start too many jobs.

Just in case you might find it useful, here's a script I use to return the number of jobs available from the scheduler:

Code:
#!/bin/sh

SCHEDULER="zap.slackware.lan"

cat << EOF | nc $SCHEDULER 8766 | grep jobs= | cut -f 3 -d = | cut -f 2 -d / | cut -f 1 -d ' ' | paste -sd+ - | bc
listcs
exit
EOF
 
5 members found this post helpful.
Old 01-30-2019, 10:54 PM   #5
1337_powerslacker
Member
 
Registered: Nov 2009
Distribution: Slackware64-current,Ubuntu,openSuSE,Manjaro
Posts: 829

Original Poster
Blog Entries: 9

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
Quote:
Originally Posted by volkerdi View Post
There's actually no need to define CC because if /etc/profile.d/icecream.sh sees either iceccd or icecc-scheduler running on the machine, it will add /usr/libexec/icecc/bin to the beginning of the $PATH, and this directory contains symlinks from all the usual compiler names to /usr/bin/icecc.

Really, all there is to do is make /etc/rc.d/rc.icecc-scheduler executable on one machine, and /etc/rc.d/rc.iceccd executable on all the machines that will be part of the compile cluster. And then set -j high enough. Per ppr:kut, it doesn't hurt to set the job number too high with this, because the scheduler won't start too many jobs.

Just in case you might find it useful, here's a script I use to return the number of jobs available from the scheduler:

Code:
#!/bin/sh

SCHEDULER="zap.slackware.lan"

cat << EOF | nc $SCHEDULER 8766 | grep jobs= | cut -f 3 -d = | cut -f 2 -d / | cut -f 1 -d ' ' | paste -sd+ - | bc
listcs
exit
EOF
Thanks much, Pat! I stand corrected! And yes, the script was very helpful
 
Old 01-30-2019, 10:55 PM   #6
chris.willing
Member
 
Registered: Jun 2014
Location: Brisbane, Australia
Distribution: Slackware,LFS
Posts: 711

Rep: Reputation: 464Reputation: 464Reputation: 464Reputation: 464Reputation: 464
Quote:
Originally Posted by 1337_powerslacker View Post
The hostname is actually made up of two parts, which is formally called the Fully Qualified Domain Name, or FQDN. The first part of the hostname is whatever you want to call your machine. As an example, my sig shows my icecream setup. The second part of the hostname is the domain name, which forms a specific group of networked computers. This is what icecream uses to address machines which also have iceccd running. I use the simple 'org' as a domain name. Not fancy, but it works.
Sorry, I misinterpreted your statement as saying that all machines must have the same FQDN. Saying that all machines must be in the same domain would have been clearer (to me).

chris
 
1 members found this post helpful.
Old 01-30-2019, 11:57 PM   #7
1337_powerslacker
Member
 
Registered: Nov 2009
Distribution: Slackware64-current,Ubuntu,openSuSE,Manjaro
Posts: 829

Original Poster
Blog Entries: 9

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
Quote:
Originally Posted by chris.willing View Post
Sorry, I misinterpreted your statement as saying that all machines must have the same FQDN. Saying that all machines must be in the same domain would have been clearer (to me).

chris
Thanks for the post. I edited my OP to make that clearer.
 
Old 01-31-2019, 04:04 AM   #8
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,582

Rep: Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158
I have iptables based firewalls and have these rules. (I found the icecream man page a little confusing on this.)
Code:
## Icecream distributed compiler
#   Daemon computers use TCP on port 10245
$IPTABLES -A INPUT -i $EXTIF -p tcp -s $LOCAL_LAN --dport 10245 -j ACCEPT
#   Scheduler computer uses UDP broadcast on port 8765 to locate daemon computers
$IPTABLES -A INPUT -i $EXTIF -p udp -s $LOCAL_LAN --sport 8765 -j ACCEPT
$IPTABLES -A INPUT -i $EXTIF -p udp -s $LOCAL_LAN --dport 8765 -j ACCEPT
 
1 members found this post helpful.
Old 02-03-2020, 12:16 PM   #9
1337_powerslacker
Member
 
Registered: Nov 2009
Distribution: Slackware64-current,Ubuntu,openSuSE,Manjaro
Posts: 829

Original Poster
Blog Entries: 9

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
I *love* icecream, and not just the kind you eat I now have a total of 10 Mac Pros (9 - 1,1s & 1 - 3,1), and 4 older PCs from the mid-2000s, running Slackware Linux, and kernel compilation is fast. As mentioned before, I don't run this cluster often enough for it to make a significant impact on my electric bill. Awesome performance!
 
1 members found this post helpful.
Old 02-05-2020, 04:23 PM   #10
karlmag
Member
 
Registered: Apr 2014
Distribution: Slackware
Posts: 100

Rep: Reputation: Disabled
Quote:
Originally Posted by 1337_powerslacker View Post
Select the machine you intend to use as a scheduler machine and make the rc.icecc-scheduler script executable. Only one machine can have this script executable!
Uhm, isn't that incorrect?

According to the README.md for icecream (https://github.com/icecc/icecream/blob/master/README.md)


Quote:
- At least one machine that runs the scheduler ("./icecc-scheduler -d")
It also states that if there are multiple schedulers (at least version 1.1), they will decide among themselves who's the boss. Also running mixed scheduler versions are not recommended, but I doubt that being a huge problem if one runs a fairly up-to-date -current on all machines.

That said, I can't say I've ever used icecream that much and also never tested that particular setup.

Thanks
--
KarlMag
 
Old 02-06-2020, 12:14 AM   #11
1337_powerslacker
Member
 
Registered: Nov 2009
Distribution: Slackware64-current,Ubuntu,openSuSE,Manjaro
Posts: 829

Original Poster
Blog Entries: 9

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
Quote:
Originally Posted by karlmag View Post
Uhm, isn't that incorrect?

According to the README.md for icecream (https://github.com/icecc/icecream/blob/master/README.md)




It also states that if there are multiple schedulers (at least version 1.1), they will decide among themselves who's the boss. Also running mixed scheduler versions are not recommended, but I doubt that being a huge problem if one runs a fairly up-to-date -current on all machines.

That said, I can't say I've ever used icecream that much and also never tested that particular setup.

Thanks
--
KarlMag
Thanks for pointing that out. I'm rather a newbie to icecream myself, and this "HOWTO" was intended to encourage wider use of the package. I'm not surprised I made a few errors. Thanks again!
 
Old 02-08-2020, 10:27 AM   #12
bamunds
Member
 
Registered: Sep 2013
Location: Mounds View MN
Distribution: Slackware64-14.2 XDM_Themes/WMaker
Posts: 735

Rep: Reputation: 229Reputation: 229Reputation: 229
I hope this is added to the Slackware Documentation project. It is an excellent writeup. Cheers
 
1 members found this post helpful.
Old 02-08-2020, 10:47 AM   #13
Alien Bob
Slackware Contributor
 
Registered: Sep 2005
Location: Eindhoven, The Netherlands
Distribution: Slackware
Posts: 8,436

Rep: Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631
Quote:
Originally Posted by bamunds View Post
I hope this is added to the Slackware Documentation project. It is an excellent writeup. Cheers
Well it won't appear there all by itself... someone will have to make the effort and write a page.
 
1 members found this post helpful.
Old 02-09-2020, 08:26 AM   #14
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,582

Rep: Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158
My initial effort for comment and improvement.
Quote:
Icecream quick use guide

What is icecream?
Icecream is a distributed compile system for C and C++.

Icecream is created by SUSE and is based on ideas and code by distcc. Like distcc it takes compile jobs from your build and distributes it to remote machines allowing a parallel build on several machines.

Why would I use it?
A lot of users nowadays have multiple computers within their local network. These can be harnessed together into a compile cluster.
As an icecream compile client sends the compile environment the first time a job is sent to a particular remote daemon, the environment of the remote daemon does not have to match the one of the client.
e.g. An old 32-bit client can harness the processing power of more powerful remote 64-bit hardware to speed compilation.

Requirements
Icecream was introduced into Slackware-current on Aug 14 00:10:54 UTC 2018, so all machines to be harnessed must have a suitable install.

Setup
The iceccd daemon must be started on all computers in the cluster.
The iceccd daemon can be started at boot by making /etc/rc.d/rc.iceccd executable. This has the advantage that the /etc/profile.d/icecream.[c]sh scripts will automatically prepend /usr/libexec/icecc/bin to the beginning of the PATH environment variable.
If the iceccd daemon is started manually, as root, by 'sh /etc/rc.d/rc.iceccd start' on the icecream compile client, then use the command 'export PATH=/usr/libexec/icecc/bin:$PATH'.

The icecc-scheduler must be started on the compile client. Multiple client clients are allowed.
The icecc-scheduler can be started at boot by making /etc/rc.d/rc.icecc-scheduler executable.
The icecc-scheduler can be started manually, as root, by 'sh /etc/rc.d/rc.icecc-scheduler start'.

If you have firewalls on the computers in your local network, you will need to open ports for communication.
Example iptables rules
## Icecream distributed compiler
# Scheduler computer uses TCP on port 8765
$IPTABLES -A INPUT -i $EXTIF -p tcp -s $LOCAL_LAN --dport 8765 -j ACCEPT
# Daemon computers use TCP on port 10245
$IPTABLES -A INPUT -i $EXTIF -p tcp -s $LOCAL_LAN --dport 10245 -j ACCEPT
# Scheduler computer uses UDP broadcast on port 8765 to locate daemon computers
$IPTABLES -A INPUT -i $EXTIF -p udp -s $LOCAL_LAN --sport 8765 -j ACCEPT
$IPTABLES -A INPUT -i $EXTIF -p udp -s $LOCAL_LAN --dport 8765 -j ACCEPT

Use
Find the 'make' command in your SlackBuild script and add ' -j X' where X is the number of jobs to be used.
X can be manually calculated as the number of cores + 1 for each computer in the compile cluster.

Our BDFL has also provided this little gem.
Code:
#!/bin/sh

SCHEDULER="zap.slackware.lan"

cat << EOF | nc $SCHEDULER 8766 | grep jobs= | cut -f 3 -d = | cut -f 2 -d / | cut -f 1 -d ' ' | paste -sd+ - | bc
listcs
exit
EOF
Comments
It has been said that it does not hurt to set the job number too high, because the scheduler won't start too many jobs.
The compile client is solely responsible for preparing the compile configuration and the linking of the object code, which may be a bottleneck.
Using a compile cluster will provide variable amounts of speed-up depending on the nature of the project being compiled.
Projects with a heavy reliance on gcc (e.g. wine) will show a high amount of speed-up.
Projects that make heavy use of Fortran (e.g. R) will show less speed-up.
The logs for the iceccd and icecc-scheduler can be useful for troubleshooting and are kept in the /var/log/icecream/ directory.
If you like Der Blinkenlights, open 'htop' in a terminal on a remote computer during a compile.

Last edited by allend; 02-09-2020 at 08:28 AM.
 
1 members found this post helpful.
Old 02-09-2020, 09:59 AM   #15
Alien Bob
Slackware Contributor
 
Registered: Sep 2005
Location: Eindhoven, The Netherlands
Distribution: Slackware
Posts: 8,436

Rep: Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631Reputation: 7631
A nice run-through of how icecream works and how to prepare your networked computers.
What I miss is how icecream should be used. Does it work transparently? Do my build scripts have to be adapted?
Could you give a specific code example which defines MAKEFLAGS and/or NUMJOBS in the environment based on the available compute power on your icecream nodes?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Icecream distrobuted cross compiler djsmiley2k Linux - Software 4 01-06-2021 04:57 AM
Icecream Slackbuild fail mlpa Slackware 1 09-14-2012 09:36 AM
Linux Multicasting HOWTO.HOWTO Join Multicast Groups ar24458 Linux - Networking 0 04-02-2007 01:55 AM
Howto update KDE and howto switch off kdm/gdm Canaris Linux - Software 1 06-15-2003 08:40 PM
Howto set up the ati Radeon64DDR for performance in 3D environments? Debianewb Linux - Hardware 7 05-05-2003 07:05 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 06:39 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration