Compile binaries in GUEST for HOST machine?

emon_lq · 02-10-2016, 05:15 AM

Hi everyone

I am very new to Gentoo, this is my first post ever in a Gentoo forum

I have been playing around with Gentoo_x64 in VirtualBox (in Debian_x64) for the last couple of days. Here is my scenario....

I have a dual boot desktop (Win10 & Debian_x64)
CPU: AMD Athlon(tm) II X2 250 (has support for AMD-V)
RAM: 8 GB
MB: MSI 880GMA E35(FX)

I have read the following wiki

https://wiki.gentoo.org/wiki/Binary_package_guide

https://wiki.gentoo.org/wiki/GCC_optimization

https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs

I have the following line in my /etc/portage/make.conf

------
CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
CHOST="x86_64-pc-linux-gnu"
------

I have been building binaries by including the following with every emerge
--buildpkg --buildpkg-exclude "virtual/* sys-kernel/*-sources"

Now I plan to upgrade my Host-OS to Gentoo; will the binary pkgs work if I transfer '/user/portage' from the Guest-Machine to the Host-Machine??

I ask this cause I ran the following commands on both the Host & Guest machines and did a diff; found significant differences, especially in the flags. (diff at the end of post)

cat /proc/cpuinfo
gcc -c -Q -march=native --help=targetgcc -c -Q -march=native –help=target

I understand that VirtualBox is a “level-2” hypervisor and not all features of the Host CPU will be exposed to the Guest OS and thus some difference should be expected.

But I don't know how much it will affect the binaries, if at all ?

Off the top of my head I can predict that since the Guest “CPU” is a subset (reduced version) of the actual(Physical) CPU, things should be fine :-P but that's just my 2 cent.

Thanks everyone
Emon

diff Guest.txt Host.txt (/proc/cpuinfo....)

Quote:

7,8c7,8
< microcode : 0x10000bf
< cpu MHz : 3001.474
---
> microcode : 0x10000c7
> cpu MHz : 800.000
20,22c20,21
< flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx fxsr_opt rdtscp lm 3dnowext 3dnow rep_good nopl extd_apicid pni lahf_lm cmp_legacy cr8_legacy 3dnowprefetch vmmcall
< bugs : tlb_mmatch apic_c1e fxsave_leak sysret_ss_attrs
< bogomips : 6002.94
---
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate npt lbrv svm_lock nrip_save vmmcall
> bogomips : 6000.19
27c26
< power management:
---
> power management: ts ttp tm stc 100mhzsteps hwpstate

diff Guest.txt Host.txt (for gcc)

Quote:

12c12
< -mabm [disabled]
---
> -mabm [enabled]
23c23
< -march= k8-sse3
---
> -march= amdfam10
41c41
< -mcx16 [disabled]
---
> -mcx16 [enabled]
68c68
< -mlzcnt [disabled]
---
> -mlzcnt [enabled]
85c85
< -mpopcnt [disabled]
---
> -mpopcnt [enabled]
89c89
< -mprfchw [disabled]
---
> -mprfchw [enabled]
109c109
< -msse4a [disabled]
---
> -msse4a [enabled]
121c121
< -mtune= k8
---
> -mtune= amdfam10

mimosinnet · 02-10-2016, 06:54 AM

I have been building from a:

Quote:

Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm dtherm tpr_shadow vnmi flexpriority

to a:

Quote:

Intel(R) Core(TM) i5 CPU M 430 @ 2.27GHz
flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida arat dtherm tpr_shadow vnmi flexpriority ept vpid

I have not had any issues when using the binaries in a desktop environment. I cannot see what would the effects of VirtualBox in the binaries.

Cheers!

emon_lq · 02-10-2016, 09:26 AM

Quote:

Originally Posted by mimosinnet

I have not had any issues when using the binaries in a desktop environment. I cannot see what would the effects of VirtualBox in the binaries.

Humm..., this is very encouraging

But what I was wondering is that, Core-i5 has all the feature of Core-2-Duo *PLUS* some more!
So will the binaries compiled in core-2-Duo be able to take advantage all the features in Core-i5?

Excuse me if the above question sounded foolish, I am new to compiling from code :P

Thanks
Emon

mimosinnet · 02-11-2016, 09:57 AM

Quote:

Originally Posted by emon_lq

But what I was wondering is that, Core-i5 has all the feature of Core-2-Duo *PLUS* some more!
So will the binaries compiled in core-2-Duo be able to take advantage all the features in Core-i5?

I am not an expert, and I have often search for similar questions ;-)

As far as I can understand, the binaries in core-2-Duo will not be able to take advantage of the features in Core-i5. Nevertheless, as they share the same architecture, the binaries produced from the Core-2-Duo will run on the Core-i5.

Cheers!

emon_lq · 02-15-2016, 12:02 PM

Quote:

Originally Posted by mimosinnet

I am not an expert, and I have often search for similar questions ;-)

I see

Quote:

Originally Posted by mimosinnet

As far as I can understand, the binaries in core-2-Duo will not be able to take advantage of the features in Core-i5. Nevertheless, as they share the same architecture, the binaries produced from the Core-2-Duo will run on the Core-i5.

Well, I guess that makes sense...

Thanks man
Emon

emon_lq · 05-31-2016, 01:43 AM

Quote:

Originally Posted by mimosinnet

I am not an expert, and I have often search for similar questions ;-)

As far as I can understand, the binaries in core-2-Duo will not be able to take advantage of the features in Core-i5. Nevertheless, as they share the same architecture, the binaries produced from the Core-2-Duo will run on the Core-i5.

Cheers!

Hi
I know that it's been a while, but I finally figured it out....
The answer was given in one of the links that I provided!
I just can't believe how I missed it the first time around :-P

http://https://wiki.gentoo.org/wiki/...get_machine.3F

Quote:

Some readers might wonder if compiling outside the target machine with a strictly inferior CPU or GCC sub-architecture will result in inferior optimization results (compared to a native compilation). The answer is simple: No. Regardless of the actual hardware on which the compilation takes place and the CHOST for which GCC was built, as long as the same arguments are used (except for -march=native) and the same version of GCC is used (although minor version might be different), the resulting optimizations are strictly the same.

I have started to compile binaries with the following option in /etc/make.conf

-march=amdfam10

Hope this helps

Emon

i92guboj · 06-03-2016, 12:22 PM

Edited: re-reading it I noticed this doesn't mean what it was meant to mean. So, rewrote it.

Object code, indeed, depends only upon the compiler version used and the flags used, as said, except for -march=native, logically.

That's also why, by default, when you compile gcc, it bootstraps itself in a three stage process, so, imagine you have gcc-version-A installed, and want to upgrade to gcc-version-B. When you launch the compilation of gcc-version-B, you use gcc-version-A to compile gcc-version-B-stage1. stage1 will be able to product the same exact code that stage2 and 3 will be able to.

So, why not stop at this point: well, gcc-version-B stage1 can produce the same object code, but the object code of the binaries of gcc-version-B-stage1 is not as optimized as it could be, which means compilations will take a bit longer to end when you use this compiler.

So, now gcc uses gcc-version-B-stage1 to produce gcc-version-B-stage2, which will produce version-B object code and will be as efficient as any other version-B produced binary. So, we could indeed stop at this point.

The 3rd stage is used just as a check. It's compared against stage2 and they should be exact copies, if not, then there's some odd bug in the compiled that needs attention.

emon_lq · 06-04-2016, 12:03 PM

Humm..., I guess that makes good sense, at least from a software/compiler point of view.
After all it's normal to expect later version (of GCC) to perform better than previous version.

As for myself, I was more concerned form a Hardware point of view.
Will code compiled on a relatively "inferior" hardware perform optimally on a "superior" hardware?
Hence I was more impressed by the use of "-march="

All the best
Emon

i92guboj · 06-04-2016, 04:30 PM

Yeah, both question are tied. It was just a way to emphasize that.

mimosinnet · 06-05-2016, 10:07 AM

This has been my experience on compiling packages for a higher CPU:
https://forums.gentoo.org/viewtopic-t-915820.html

In summary, although it can build binaries, it fails at the test phase when it tries to use instructions from the higher CPU in the lower CPU.

Cheers!

Luridis · 12-26-2016, 12:48 AM

As primarily a Linux From Scratch user, I fiddle with compiler flags quite a bit. Note that I prefer "reliably fast" to "insane w/ segfaults".

I do most of this on a VM.

My best results come from setting the architecture to be broad and the tune to be narrow. Basically it builds code to run on a lot of similar CPUs, but lines things up to run best on a specific microarchitecture.

For the base LFS system, which is basically the Gnu toolchain, kernel and sysadmin CLI tools I use: -march=core2 -mtune=generic -O2 Which provides max stability in the toolchain.

After the minimal first boot I use: -march=core2 -mtune=native -O2. In this case, the code will run on any core2 or newer, but runs best on a Haswell, which is where it was built. You could try this for building on your VM for the host. Heck, when I do core2/generic I use that base for both my IvyBridge laptop and my Haswell desktop.

mimosinnet · 12-29-2016, 03:48 AM

Quote:

Originally Posted by Luridis

For the base LFS system, which is basically the Gnu toolchain, kernel and sysadmin CLI tools I use: -march=core2 -mtune=generic -O2 Which provides max stability in the toolchain.

After the minimal first boot I use: -march=core2 -mtune=native -O2. In this case, the code will run on any core2 or newer, but runs best on a Haswell, which is where it was built. You could try this for building on your VM for the host. Heck, when I do core2/generic I use that base for both my IvyBridge laptop and my Haswell desktop.

Waw! This is an excellent tip! Thanks a lot!