LFS Build error in 6.14. maybe bad lib loading

daspork · 01-20-2017, 02:23 AM

So I have been working on a tool to automate a LFS build:
https://github.com/chris-statzer/swell I have ran into a little snag with the
binutils in chapter 6. I am doing this on a ubuntu 16.04 host which seems to be
fine as the temp system build is flawless except for some minor test failures
that are already noted in the book.

When compiling/linking the libgmp package (6.14) I get the following error:

Code:

libtool: link: ar cq .libs/libmpn.a .libs/fib_table.o .libs/mp_bases.o .libs/invert_limb_table.o .libs/add.o .libs/add_1.o .libs/add_n.o .libs/sub.o .libs/sub_1.o .libs/sub_n.o .libs/cnd_add_n.o .libs/cnd_sub_n.o .libs/cnd_swap.o .libs/neg.o .libs/com.o .libs/mul_1.o .libs/addmul_1.o .libs/submul_1.o .libs/add_err1_n.o .libs/add_err2_n.o .libs/add_err3_n.o .libs/sub_err1_n.o .libs/sub_err2_n.o .libs/sub_err3_n.o .libs/lshift.o .libs/rshift.o .libs/dive_1.o .libs/diveby3.o .libs/divis.o .libs/divrem.o .libs/divrem_1.o .libs/divrem_2.o .libs/fib2_ui.o .libs/mod_1.o .libs/mod_34lsub1.o .libs/mode1o.o .libs/pre_mod_1.o .libs/dump.o .libs/mod_1_1.o .libs/mod_1_2.o .libs/mod_1_3.o .libs/mod_1_4.o .libs/lshiftc.o .libs/mul.o .libs/mul_fft.o .libs/mul_n.o .libs/sqr.o .libs/mul_basecase.o .libs/sqr_basecase.o .libs/nussbaumer_mul.o .libs/mulmid_basecase.o .libs/toom42_mulmid.o .libs/mulmid_n.o .libs/mulmid.o .libs/random.o .libs/random2.o .libs/pow_1.o .libs/rootrem.o .libs/sqrtrem.o .libs/sizeinbase.o .libs/get_str.o .libs/set_str.o .libs/scan0.o .libs/scan1.o .libs/popcount.o .libs/hamdist.o .libs/cmp.o .libs/zero_p.o .libs/perfsqr.o .libs/perfpow.o .libs/gcd_1.o .libs/gcd.o .libs/gcdext_1.o .libs/gcdext.o .libs/gcd_subdiv_step.o .libs/gcdext_lehmer.o .libs/div_q.o .libs/tdiv_qr.o .libs/jacbase.o .libs/jacobi_2.o .libs/jacobi.o .libs/get_d.o .libs/matrix22_mul.o .libs/matrix22_mul1_inverse_vector.o .libs/hgcd_matrix.o .libs/hgcd2.o .libs/hgcd_step.o .libs/hgcd_reduce.o .libs/hgcd.o .libs/hgcd_appr.o .libs/hgcd2_jacobi.o .libs/hgcd_jacobi.o .libs/mullo_n.o .libs/mullo_basecase.o .libs/sqrlo.o .libs/sqrlo_basecase.o .libs/toom22_mul.o .libs/toom32_mul.o .libs/toom42_mul.o .libs/toom52_mul.o .libs/toom62_mul.o .libs/toom33_mul.o .libs/toom43_mul.o .libs/toom53_mul.o .libs/toom54_mul.o .libs/toom63_mul.o .libs/toom44_mul.o .libs/toom6h_mul.o .libs/toom6_sqr.o .libs/toom8h_mul.o .libs/toom8_sqr.o .libs/toom_couple_handling.o .libs/toom2_sqr.o .libs/toom3_sqr.o .libs/toom4_sqr.o .libs/toom_eval_dgr3_pm1.o .libs/toom_eval_dgr3_pm2.o .libs/toom_eval_pm1.o .libs/toom_eval_pm2.o .libs/toom_eval_pm2exp.o .libs/toom_eval_pm2rexp.o .libs/toom_interpolate_5pts.o .libs/toom_interpolate_6pts.o .libs/toom_interpolate_7pts.o .libs/toom_interpolate_8pts.o .libs/toom_interpolate_12pts.o .libs/toom_interpolate_16pts.o .libs/invertappr.o .libs/invert.o .libs/binvert.o .libs/mulmod_bnm1.o .libs/sqrmod_bnm1.o .libs/div_qr_1.o .libs/div_qr_1n_pi1.o .libs/div_qr_2.o .libs/div_qr_2n_pi1.o .libs/div_qr_2u_pi1.o .libs/sbpi1_div_q.o .libs/sbpi1_div_qr.o .libs/sbpi1_divappr_q.o .libs/dcpi1_div_q.o .libs/dcpi1_div_qr.o .libs/dcpi1_divappr_q.o .libs/mu_div_qr.o .libs/mu_divappr_q.o .libs/mu_div_q.o .libs/bdiv_q_1.o .libs/sbpi1_bdiv_q.o .libs/sbpi1_bdiv_qr.o .libs/dcpi1_bdiv_q.o .libs/dcpi1_bdiv_qr.o .libs/mu_bdiv_q.o .libs/mu_bdiv_qr.o .libs/bdiv_q.o .libs/bdiv_qr.o .libs/broot.o .libs/brootinv.o .libs/bsqrt.o .libs/bsqrtinv.o .libs/divexact.o .libs/bdiv_dbm1c.o .libs/redc_1.o .libs/redc_2.o .libs/redc_n.o .libs/powm.o .libs/powlo.o .libs/sec_powm.o .libs/sec_mul.o .libs/sec_sqr.o .libs/sec_div_qr.o .libs/sec_div_r.o .libs/sec_pi1_div_qr.o .libs/sec_pi1_div_r.o .libs/sec_add_1.o .libs/sec_sub_1.o .libs/sec_invert.o .libs/trialdiv.o .libs/remove.o .libs/and_n.o .libs/andn_n.o .libs/nand_n.o .libs/ior_n.o .libs/iorn_n.o .libs/nior_n.o .libs/xor_n.o .libs/xnor_n.o .libs/copyi.o .libs/copyd.o .libs/zero.o .libs/sec_tabselect.o .libs/comb_tables.o .libs/invert_limb.o .libs/sqr_diag_addlsh1.o .libs/mul_2.o .libs/addmul_2.o .libs/addlsh1_n.o .libs/sublsh1_n.o .libs/rsblsh1_n.o .libs/rsh1add_n.o .libs/rsh1sub_n.o .libs/addlsh2_n.o .libs/sublsh2_n.o .libs/rsblsh2_n.o .libs/addlsh_n.o .libs/rsblsh_n.o .libs/add_n_sub_n.o .libs/addaddmul_1msb0.o 
ar: error while loading shared libraries: libbfd-2.27.so: cannot open shared object file: No such file or directory
make[2]: *** [Makefile:477: libmpn.la] Error 127
make[2]: Leaving directory '/tools/swell/build/base-gmp/mpn'
make[1]: *** [Makefile:956: all-recursive] Error 1
make[1]: Leaving directory '/tools/swell/build/base-gmp'
make: *** [Makefile:775: all] Error 2

It seems the binutils installed in 6.13 is broken in some way, or my
configuration. Trying to run the utils from the cli inside the chroot I get the
following:

Code:

root:/tools/swell# ar
ar: error while loading shared libraries: libbfd-2.27.so: cannot open shared object file: No such file or directory

Output of ldd on the binary:

Code:

root:/tools/swell# ldd /usr/bin/ar
	linux-vdso.so.1 (0x00007ffcf81f2000)
	libbfd-2.27.so => not found
	libz.so.1 => /tools/lib/libz.so.1 (0x00007f14e26d1000)
	libdl.so.2 => /tools/lib/libdl.so.2 (0x00007f14e24cd000)
	libc.so.6 => /tools/lib/libc.so.6 (0x00007f14e212f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f14e28ed000)
root:/tools/swell#

But if i supply a LD_LIBRARY_PATH var it runs fine:

Code:

root:/tools/swell# LD_LIBRARY_PATH="/usr/lib" ar -V
GNU ar (GNU Binutils) 2.27
Copyright (C) 2016 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.

root:/tools/swell# LD_LIBRARY_PATH="/usr/lib" ldd /usr/bin/ar
	linux-vdso.so.1 (0x00007ffe9f0fc000)
	libbfd-2.27.so => /usr/lib/libbfd-2.27.so (0x00007fd878c44000)
	libz.so.1 => /tools/lib/libz.so.1 (0x00007fd878a28000)
	libdl.so.2 => /tools/lib/libdl.so.2 (0x00007fd878824000)
	libc.so.6 => /tools/lib/libc.so.6 (0x00007fd878486000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fd878f7b000)

Output of ldconfig -V

Code:

root:/tools/swell# ldconfig -v
ldconfig: Can't stat /libx32: No such file or directory
ldconfig: Can't stat /usr/lib64: No such file or directory
ldconfig: Can't stat /usr/libx32: No such file or directory
/lib:
	libresolv.so.2 -> libresolv-2.24.so
	libnss_dns.so.2 -> libnss_dns-2.24.so
	libSegFault.so -> libSegFault.so
	libz.so.1 -> libz.so.1.2.10
	libnss_nis.so.2 -> libnss_nis-2.24.so
	libmemusage.so -> libmemusage.so
	librt.so.1 -> librt-2.24.so
	libnsl.so.1 -> libnsl-2.24.so
	libthread_db.so.1 -> libthread_db-1.0.so
	ld-linux-x86-64.so.2 -> ld-2.24.so
	libnss_db.so.2 -> libnss_db-2.24.so
	libm.so.6 -> libm-2.24.so
	libanl.so.1 -> libanl-2.24.so
	libnss_hesiod.so.2 -> libnss_hesiod-2.24.so
	libpthread.so.0 -> libpthread-2.24.so
	libc.so.6 -> libc-2.24.so
	libnss_files.so.2 -> libnss_files-2.24.so
	libmvec.so.1 -> libmvec-2.24.so
	libBrokenLocale.so.1 -> libBrokenLocale-2.24.so
	libnss_compat.so.2 -> libnss_compat-2.24.so
	libcidn.so.1 -> libcidn-2.24.so
	libcrypt.so.1 -> libcrypt-2.24.so
	libnss_nisplus.so.2 -> libnss_nisplus-2.24.so
	libutil.so.1 -> libutil-2.24.so
	libpcprofile.so -> libpcprofile.so
	libdl.so.2 -> libdl-2.24.so
/lib64:
	ld-linux-x86-64.so.2 -> ld-lsb-x86-64.so.3
/usr/lib:
	libbfd-2.27.so -> libbfd.so
	libmagic.so.1 -> libmagic.so.1.0.0
	libopcodes-2.27.so -> libopcodes.so

could this be an error of mine in 6.10. Adjusting the Toolchain? I have checked
and triple checked the commands and everything is fine. I would also point out
that I have in fact stepped through the build and ran all the suggested tests
and EVERYTHING returned correctly. My best guess is here:

https://github.com/chris-statzer/swe...toolchain.yaml

I have ZERO idea what thr gcc -dumpspecs command there does to be honest, but
givent that the error seems to be with the library loader/linker it seems most
likely to be there?

Any guidence would be awesome and I would be happy to provide anything else needed.

glibc config.log
https://gist.github.com/chris-statze...f53db5b62aa304

binutils config.log
https://gist.github.com/chris-statze...232c17f8e2e542

libgmp config.log
https://gist.github.com/chris-statze...7aadab1cf876f6

hazel · 01-21-2017, 11:32 AM

Try looking at the actual output file created by the gcc adjustment edit. You will find it at /tools/lib/gcc/x86_64-pc-linux-gnu/6.2.0/specs.

The first part of the edit removes all references to "/tools", so grep "/tools" specs should yield nothing. I think you are right that this is where the loader is picked up from. grep "/lib" specs will pick up the relevant lines.

The second part tells gcc where to find its start files. It should be looking in the /usr/lib tree, not in /tools/lib. Look for the line following startfile_prefix_spec.

The third part tells gcc where to look for headers: /usr/include. Look for the line following *cpp:

daspork · 01-22-2017, 01:37 AM

Well it looks like all is fine in the specs file:
https://gist.github.com/chris-statze...8a57f3ac17deb1

and the test after the toolchain adjust looks right

Code:

I have no name!:/tools/swell# echo 'int main(){}' > dummy.c
I have no name!:/tools/swell# cc dummy.c -v -Wl,--verbose &> dummy.log
I have no name!:/tools/swell# readelf -l a.out | grep ': /lib'
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
I have no name!:/tools/swell#

really at a loss here honestly...

hazel · 01-22-2017, 03:06 AM

I notice that your root user still has no name. This means that you have not carried out the adjustments described in Chapter 6.6. These include putting links in /usr/lib to certain gcc libraries in the /tools tree. Go through this section again and then try.

"ln -sv /tools/lib/libstdc++.so{,.6} /usr/lib
...................
/usr/lib/libstdc++{,.6}

This is needed by several tests in Glibc's test suite, as well as for C++ support in GMP."

daspork · 01-23-2017, 01:27 AM

The bash username is unrelated. The script executes a fresh unhashed bash with a fresh copy of the env each command so this is just the toplevel instance that the initial build script was run with. I just finished building it all by hand and its perfect so while I agree it's 100% in my scripting, I'm not sure its the environment as I have run many tests to make sure how I execute the commands is in a good env. I think I might actually have a problem with config.cache. I don't think the script as-is is deleting the source folders between the passes so I'm gonna do more work on that. I also saved snapshots of the filesystem through my handmade build so I'll be able to check what's there vs. what should be there every step. Thanks for the attention and help.

Luridis · 02-07-2017, 02:54 AM

/lib64/ld-linux-x86-64.so.2 probably exists on your host too. If you're in chroot and having library problems, being able to tell which a program has linked to becomes important. So, login to your host system and type:

ldd /bin/ls

You'll see something like:

Code:

linux-vdso.so.1 (Hex-Number)
libcap.so.2 => /lib/libcap.so.2 (Hex-Number)
libc.so.6 => /lib/libc.so.6 (Hex-Number)
/lib64/ld-linux-x86-64.so.2 (Hex-Number)

Pipe that output into a file named "/sources/host-link-chain". Then enter the chroot and do it again, piping the output to "/sources/chroot-link-chain". Finally, cat them one on top of the other and see if they're different.

If the chains DO NOT look different, or you see the "tools" directory past a certain point (post-strip-chapter6), something is wrong. However, if the chains look identical, the hex addresses at the end should still be different between the host and chroot system, once libc is built of course. If everything is identical after building Glibc in the chroot, something has gone very wrong and it's time to start back at the completion of the tools directory.

strc · 05-01-2017, 07:43 PM

My apologies for ressurecting an old post. I was just wondering if you tracked down your problem. I am having a similar issue (not finding libraries built in Stage 2 post adjusting the toolchain).

In my case, the libraries are not in the listed cache but are in the default search directory (/usr/lib). If I manually update the cache but calling ldconfig, the library is linked used as expected.

The installers for gmp etc install using ldconfig -n which by extension implies -N (do not update cache). However, as I said, the installed libraries are properly located in (for me) a default search directory /usr/lib. To my knowledge, the only reason that this directory would not be verified would be if an executable were built with "no default libraries".

Anyway, if you happened to figure out the source of your problem, I would be delighted to know.