ld is skipping over the libm.so.6 I have set in LD_LIBRARY_PATH

davidzory · 08-13-2022, 12:48 AM

I am working on creating an AppImage for postgres.

I've found all the dynamic libraries it uses and placed them in a custom folder and run postgres prefixed with LD_LIBRARY_PATH=/data2/root/usr/lib/aarch64-linux-gnu:/data2/root/lib/aarch64-linux-gnu

It works great for some libraries, for example:

Quote:

51: find library=liblber-2.4.so.2 [0]; searching
51: search path=/data2/root/usr/lib/aarch64-linux-gnu:/data2/root/lib/aarch64-linux-gnu (LD_LIBRARY_PATH)
51: trying file=/data2/root/usr/lib/aarch64-linux-gnu/liblber-2.4.so.2

I can see ld is searching for libraries and using the ones in my custom folder /data2/root/...

however it's not working with libm.so.6 and ends up using the system library for this instead, which doesn't work on older distros (buster):

Quote:

/lib/aarch64-linux-gnu/libm.so.6: error: version lookup error: version `GLIBC_2.29' not found (required by /data2/root/usr/lib/postgresql/12/bin/postgres) (continued)
/data2/root/usr/lib/postgresql/12/bin/postgres: /lib/aarch64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /data2/root/usr/lib/postgresql/12/bin/postgres)

When I run the command with LD_DEBUG=libs I get this:

Quote:

50: find library=libm.so.6 [0]; searching
50: search path=/data2/root/usr/lib/aarch64-linux-gnu:/data2/root/lib/aarch64-linux-gnu (LD_LIBRARY_PATH)
50: trying file=/data2/root/usr/lib/aarch64-linux-gnu/libm.so.6
50: trying file=/data2/root/lib/aarch64-linux-gnu/libm.so.6
50: search cache=/etc/ld.so.cache
50: trying file=/lib/aarch64-linux-gnu/libm.so.6

which is quite strange because you can see ld is looking at my provided copy of libm.so, /data2/root/lib/aarch64-linux-gnu/libm.so.6, but its skipping it even though (when I run this on bullseye which is where I sourced libm.so from) you can see it both exists and is identical to the system one:

Quote:

root@95ebc03dd154:/# md5sum /data2/root/lib/aarch64-linux-gnu/libm.so.6
d2aa3efcdac8c0c00393ec248dfbf264 /data2/root/lib/aarch64-linux-gnu/libm.so.6

root@95ebc03dd154:/# md5sum /lib/aarch64-linux-gnu/libm.so.6
d2aa3efcdac8c0c00393ec248dfbf264 /lib/aarch64-linux-gnu/libm.so.6

So my question here is, why is it the case that ld is skipping over the libm.so I provide and falling back to the system one? Even if I move libm.so from bullseye to buster directly and overwrite /lib/aarch64-linux-gnu/libm.so.6 it appears that ld will even start ignoring /lib/aarch64-linux-gnu/libm.so.6 and say the module isn't found.

business_kid · 08-13-2022, 03:23 AM

Check it with file; there may be a 32/64 bit mismatch.

davidzory · 08-14-2022, 01:05 AM

Quote:

Originally Posted by business_kid

Check it with file; there may be a 32/64 bit mismatch.

Thanks for the suggestion! I ended up figuring it out! /lib/aarch64-linux-gnu/libm.so.6 for some reason had very open permissions (777) and for whatever reason I also had to chmod my copy at /data2/root/lib/aarch64-linux-gnu/libm.so.6 to be 777 too and then ld picked it up. I'm not sure why the original copy from the system is 777 though or why it has to be so for ld to see it.

business_kid · 08-14-2022, 03:40 AM

On my system, libm.so.6 is a SYMLINK, with permossions 0777.

libm.so.6 --> libm-2.33.so, with permissions 0755.

NevemTeve · 08-14-2022, 05:13 AM

While not explicitly stated, it sounds like some sort of cross-compilation.

Guttorm · 08-15-2022, 08:08 AM

Hi

One way to troubleshoot is to run the postgres binary with the ldd prefix. I use mysql. If I type "ldd /usr/bin/mysql" i get a list of .so files it will use - and use LD_LIBRARY_PATH if you have set it. And maybe you get some better error message?

The list of .so files of a binary file is not complete - The .so files depends on other .so files. For example I see this:

Code:

ldd /usr/bin/mysql

        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007feb1aff9000)

ldd /lib/x86_64-linux-gnu/libm.so.6

        linux-vdso.so.1 (0x00007ffe273fe000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc471d89000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc4720a0000)

ldd /lib/x86_64-linux-gnu/libc.so.6

        /lib64/ld-linux-x86-64.so.2 (0x00007f25917c9000)
        linux-vdso.so.1 (0x00007ffc289f9000)

You probably need to copy a lot of .so files for this to work.

NevemTeve · 08-15-2022, 10:20 AM

I think ldd module is recursive, unlike readelf -d module | grep '(NEEDED)'

business_kid · 08-15-2022, 11:33 AM

Quote:

Originally Posted by NevemTeve

I think ldd module is recursive, unlike readelf -d module | grep '(NEEDED)'

It's recursive in $LD_LIBRARY_PATH. so '/usr/lib64' in $LD_LIBRARY_PATH includes all subdirs.