LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 07-06-2009, 08:41 PM   #1
mattydee
Member
 
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479

Rep: Reputation: 48
make -j options


It's been said that the rule of thumb for this is the number of cores +1. How about for a P4 with hyper threading? -j3?

What are your personal preferences for the -j option?

On my P4 with hyperthreading, I got the following results compiling Filezilla using the default configure script:

Code:
make:
real    7m22.850s
user    6m53.716s
sys     0m25.261s

make -j3: 
real    6m14.278s
user    11m24.595s
sys     0m35.921s
 
Old 07-06-2009, 08:55 PM   #2
Cheesesteak
Member
 
Registered: Jun 2008
Distribution: Slackware
Posts: 102

Rep: Reputation: 24
I've also seen recently that the core+1 rule should be tossed, just set -j to however many cores/threads your CPU supports.

How about some results for your box using -j2 ?
 
Old 07-07-2009, 12:51 AM   #3
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: McCalla, AL, USA
Distribution: Gentoo (all servers at work are openSUSE)
Posts: 6,938

Rep: Reputation: 128Reputation: 128
make -j10 here
 
Old 07-07-2009, 01:30 AM   #4
mattydee
Member
 
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by Cheesesteak View Post
I've also seen recently that the core+1 rule should be tossed, just set -j to however many cores/threads your CPU supports.

How about some results for your box using -j2 ?
make -j2
Code:
real    6m38.670s
user    11m56.031s
sys     0m34.069s
@Bruce:
Why so high?
 
Old 07-07-2009, 02:34 AM   #5
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 423

Rep: Reputation: 45
Quote:
Originally Posted by Bruce Hill View Post
make -j10 here
What machine do you use?

BTW, I saw many 'make -j6' in official SlackBuilds.
 
Old 07-07-2009, 03:27 AM   #6
ppr:kut
Slackware Contributor
 
Registered: Aug 2006
Location: Netherlands
Distribution: Slackware
Posts: 519

Rep: Reputation: 315Reputation: 315Reputation: 315Reputation: 315
In my experience the results of the "-j" parameter depend on the build-system used.
Using "-j3" on a *Single core* machine gives slightly better results than "-j1", if cmake is used.
However, it gives slightly worse results than "-j1" in about every other build system I tested.

Note that I haven't tested this on multicore machines yet.
 
Old 07-07-2009, 05:18 AM   #7
Linux.tar.gz
Senior Member
 
Registered: Dec 2003
Location: Paris
Distribution: Slackware forever.
Posts: 2,437

Rep: Reputation: 95
I use -j(number of core/cpu * 2)
 
Old 07-07-2009, 12:16 PM   #8
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: McCalla, AL, USA
Distribution: Gentoo (all servers at work are openSUSE)
Posts: 6,938

Rep: Reputation: 128Reputation: 128
From "man make":
Code:
       -j [jobs], --jobs[=jobs]
            Specifies  the number of jobs (commands) to run simultaneously.  If there is more than one -j option, the last one is effective.  If the -j option is given without
            an argument, make will not limit the number of jobs that can run simultaneously.
It alludes me who said to, or where I read to, use "-j10".

The main thing I compile is kernels, and using -j10 both of
the CPUs run equally, and my compile times dropped significantly.

Here's the last kernel I built and timed:
Code:
mingdao@jeremiah:~/kernel/linux-2.6.30$ time make -j10

  BUILD   arch/x86/boot/bzImage
Root device is (8, 7)
Setup is 11804 bytes (padded to 12288 bytes).
System is 2288 kB
CRC 2765ea1f
Kernel: arch/x86/boot/bzImage is ready  (#5)

real    0m55.014s
user    0m39.142s
sys     0m7.226s
Which is relative -- I don't build many modules in my kernels,
and don't think that was the first build of 2.6.30.

And that system:
Code:
mingdao@jeremiah:~$ cat /proc/cpuinfo                                                                                                                               
processor       : 0                                                                                                                                                 
vendor_id       : GenuineIntel                                                                                                                                      
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     T8300  @ 2.40GHz
stepping        : 6
cpu MHz         : 800.000
cache size      : 3072 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida tpr_shadow vnmi flexpriority
bogomips        : 4788.32
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     T8300  @ 2.40GHz
stepping        : 6
cpu MHz         : 800.000
cache size      : 3072 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida tpr_shadow vnmi flexpriority
bogomips        : 4787.74
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:
At this point it's a habit, and a command that sits in my
kernel rebuild guide. Perhaps when 2.6.31 comes out, I'll
build it on this workstation and see what "-j4" outputs.
 
Old 07-08-2009, 11:41 AM   #9
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: McCalla, AL, USA
Distribution: Gentoo (all servers at work are openSUSE)
Posts: 6,938

Rep: Reputation: 128Reputation: 128
I'm up for learning, and short of time for researching...
Code:
mingdao@jeremiah:~/kernel/linux-2.6.30.1$ time make -j4

real    6m10.493s
user    9m44.883s
sys     1m13.913s

mingdao@jeremiah:~/kernel/linux-2.6.30.1$ time make -j10

real    6m6.737s
user    9m53.467s
sys     1m14.438s
Not a significant difference. The second was run after
removing the original source, then untarring again.
 
Old 07-08-2009, 12:02 PM   #10
Dump_Weed
LQ Newbie
 
Registered: Jul 2009
Posts: 12

Rep: Reputation: 0
If you have a spare system kicking around you can always use distcc which then allows you to build your software across multipule systems. It is also good if you are trying to build on a system with limied resources, you can push the build off to your core2duo etc.

http://distcc.samba.org/

I used a live CD a while back to boot up my games pc(3.2 duel core) and used the spare cpu for building stuff on a little system (533mhz, 256mb ram).

Works well, takes a little playing with.

Cheers
Dump_Weed
 
Old 07-08-2009, 12:29 PM   #11
ppr:kut
Slackware Contributor
 
Registered: Aug 2006
Location: Netherlands
Distribution: Slackware
Posts: 519

Rep: Reputation: 315Reputation: 315Reputation: 315Reputation: 315
Or you can use icecream (http://en.opensuse.org/Icecream)

It's similar to distcc, but I can't tell which is better. Never used distcc.
 
Old 07-08-2009, 02:38 PM   #12
Dump_Weed
LQ Newbie
 
Registered: Jul 2009
Posts: 12

Rep: Reputation: 0
icecream is a new one to me, ill have a look when i get a moment. Thanks
 
Old 07-08-2009, 09:37 PM   #13
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: McCalla, AL, USA
Distribution: Gentoo (all servers at work are openSUSE)
Posts: 6,938

Rep: Reputation: 128Reputation: 128
Thanks for the replies. I knew about distcc. However, on this LAN
we have one server which never gets new software or compiles; one
laptop (the Core2 Duo) and two PCs that are all 64-bit; and one test
box that is 32-bit which is converted to a Chinese system and does
not need any new software. Just the new builds for new versions of
Slackware, which are not painful over ssh. Usually requires very
little change to the old SlackBuilds.

I do want to research the -j option, and get the opinion of some
kernel developers. I just rebuilt 2.6.30.1 with -j2, which was a
bit slower. Maybe I'll build the new X.Org 1.6.2 and check flags.
 
Old 07-23-2009, 06:03 AM   #14
Bruce Hill
HCL Maintainer
 
Registered: Jun 2003
Location: McCalla, AL, USA
Distribution: Gentoo (all servers at work are openSUSE)
Posts: 6,938

Rep: Reputation: 128Reputation: 128
What do you think?

Linux Kernel Modules Installation HOWTO - Chapter 3. Compiler Speed-up
 
Old 07-23-2009, 06:50 AM   #15
sahko
Senior Member
 
Registered: Sep 2008
Distribution: Slackware
Posts: 1,041

Rep: Reputation: Disabled
Quote:
Originally Posted by Bruce Hill View Post
Sounds archaic.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
make-kpkg options dravenloft Debian 0 05-16-2006 11:16 AM
make architecture options mcirsta Linux - Newbie 2 04-30-2005 08:09 PM
make <systype> [options] spideywebsling Linux - Newbie 10 07-03-2004 05:05 PM
'make mrproper' - make bzImage options. liguorir Linux - Software 4 06-26-2003 07:48 PM
including options in make watashiwaotaku7 Linux - General 0 01-22-2003 08:55 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 05:17 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration