LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   make -j options (https://www.linuxquestions.org/questions/slackware-14/make-j-options-738187/)

mattydee 07-06-2009 07:41 PM

make -j options
 
It's been said that the rule of thumb for this is the number of cores +1. How about for a P4 with hyper threading? -j3?

What are your personal preferences for the -j option?

On my P4 with hyperthreading, I got the following results compiling Filezilla using the default configure script:

Code:

make:
real    7m22.850s
user    6m53.716s
sys    0m25.261s

make -j3:
real    6m14.278s
user    11m24.595s
sys    0m35.921s


Cheesesteak 07-06-2009 07:55 PM

I've also seen recently that the core+1 rule should be tossed, just set -j to however many cores/threads your CPU supports.

How about some results for your box using -j2 ?

Bruce Hill 07-06-2009 11:51 PM

make -j10 here

mattydee 07-07-2009 12:30 AM

Quote:

Originally Posted by Cheesesteak (Post 3599077)
I've also seen recently that the core+1 rule should be tossed, just set -j to however many cores/threads your CPU supports.

How about some results for your box using -j2 ?

make -j2
Code:

real    6m38.670s
user    11m56.031s
sys    0m34.069s

@Bruce:
Why so high?

grissiom 07-07-2009 01:34 AM

Quote:

Originally Posted by Bruce Hill (Post 3599255)
make -j10 here

What machine do you use?;)

BTW, I saw many 'make -j6' in official SlackBuilds.

ppr:kut 07-07-2009 02:27 AM

In my experience the results of the "-j" parameter depend on the build-system used.
Using "-j3" on a *Single core* machine gives slightly better results than "-j1", if cmake is used.
However, it gives slightly worse results than "-j1" in about every other build system I tested.

Note that I haven't tested this on multicore machines yet.

Linux.tar.gz 07-07-2009 04:18 AM

I use -j(number of core/cpu * 2)

Bruce Hill 07-07-2009 11:16 AM

From "man make":
Code:

      -j [jobs], --jobs[=jobs]
            Specifies  the number of jobs (commands) to run simultaneously.  If there is more than one -j option, the last one is effective.  If the -j option is given without
            an argument, make will not limit the number of jobs that can run simultaneously.

It alludes me who said to, or where I read to, use "-j10".

The main thing I compile is kernels, and using -j10 both of
the CPUs run equally, and my compile times dropped significantly.

Here's the last kernel I built and timed:
Code:

mingdao@jeremiah:~/kernel/linux-2.6.30$ time make -j10

  BUILD  arch/x86/boot/bzImage
Root device is (8, 7)
Setup is 11804 bytes (padded to 12288 bytes).
System is 2288 kB
CRC 2765ea1f
Kernel: arch/x86/boot/bzImage is ready  (#5)

real    0m55.014s
user    0m39.142s
sys    0m7.226s

Which is relative -- I don't build many modules in my kernels,
and don't think that was the first build of 2.6.30.

And that system:
Code:

mingdao@jeremiah:~$ cat /proc/cpuinfo                                                                                                                             
processor      : 0                                                                                                                                               
vendor_id      : GenuineIntel                                                                                                                                     
cpu family      : 6
model          : 23
model name      : Intel(R) Core(TM)2 Duo CPU    T8300  @ 2.40GHz
stepping        : 6
cpu MHz        : 800.000
cache size      : 3072 KB
physical id    : 0
siblings        : 2
core id        : 0
cpu cores      : 2
apicid          : 0
initial apicid  : 0
fpu            : yes
fpu_exception  : yes
cpuid level    : 10
wp              : yes
flags          : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida tpr_shadow vnmi flexpriority
bogomips        : 4788.32
clflush size    : 64
cache_alignment : 64
address sizes  : 36 bits physical, 48 bits virtual
power management:

processor      : 1
vendor_id      : GenuineIntel
cpu family      : 6
model          : 23
model name      : Intel(R) Core(TM)2 Duo CPU    T8300  @ 2.40GHz
stepping        : 6
cpu MHz        : 800.000
cache size      : 3072 KB
physical id    : 0
siblings        : 2
core id        : 1
cpu cores      : 2
apicid          : 1
initial apicid  : 1
fpu            : yes
fpu_exception  : yes
cpuid level    : 10
wp              : yes
flags          : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida tpr_shadow vnmi flexpriority
bogomips        : 4787.74
clflush size    : 64
cache_alignment : 64
address sizes  : 36 bits physical, 48 bits virtual
power management:

At this point it's a habit, and a command that sits in my
kernel rebuild guide. Perhaps when 2.6.31 comes out, I'll
build it on this workstation and see what "-j4" outputs.

Bruce Hill 07-08-2009 10:41 AM

I'm up for learning, and short of time for researching...
Code:

mingdao@jeremiah:~/kernel/linux-2.6.30.1$ time make -j4

real    6m10.493s
user    9m44.883s
sys    1m13.913s

mingdao@jeremiah:~/kernel/linux-2.6.30.1$ time make -j10

real    6m6.737s
user    9m53.467s
sys    1m14.438s

Not a significant difference. The second was run after
removing the original source, then untarring again.

Dump_Weed 07-08-2009 11:02 AM

If you have a spare system kicking around you can always use distcc which then allows you to build your software across multipule systems. It is also good if you are trying to build on a system with limied resources, you can push the build off to your core2duo etc.

http://distcc.samba.org/

I used a live CD a while back to boot up my games pc(3.2 duel core) and used the spare cpu for building stuff on a little system (533mhz, 256mb ram).

Works well, takes a little playing with.

Cheers
Dump_Weed

ppr:kut 07-08-2009 11:29 AM

Or you can use icecream (http://en.opensuse.org/Icecream)

It's similar to distcc, but I can't tell which is better. Never used distcc.

Dump_Weed 07-08-2009 01:38 PM

icecream is a new one to me, ill have a look when i get a moment. Thanks :)

Bruce Hill 07-08-2009 08:37 PM

Thanks for the replies. I knew about distcc. However, on this LAN
we have one server which never gets new software or compiles; one
laptop (the Core2 Duo) and two PCs that are all 64-bit; and one test
box that is 32-bit which is converted to a Chinese system and does
not need any new software. Just the new builds for new versions of
Slackware, which are not painful over ssh. Usually requires very
little change to the old SlackBuilds.

I do want to research the -j option, and get the opinion of some
kernel developers. I just rebuilt 2.6.30.1 with -j2, which was a
bit slower. Maybe I'll build the new X.Org 1.6.2 and check flags.

Bruce Hill 07-23-2009 05:03 AM

What do you think?

Linux Kernel Modules Installation HOWTO - Chapter 3. Compiler Speed-up

sahko 07-23-2009 05:50 AM

Quote:

Originally Posted by Bruce Hill (Post 3617519)

Sounds archaic.


All times are GMT -5. The time now is 07:48 AM.