[SOLVED] Glibc backward compatibility

After installing second glibc from source in a separate directory, compiled with debug symbols, I encountered some problems. I found information on the web that they are most likely compatibility issues.

Apparently, the loader that is statically linked in the executables that come with the distribution is incompatible with the shared objects from the new glibc that was installed. On the other hand, I read from a few sources that new minor versions of glibc should be backward-compatibile. Meaning, that executables linked against the old version of the loader should link the new versions of the shared objects fine.

Here is an example of what I observe. Initially, the ls command works just fine. Then:

root# LD_TRACE_LOADED_OBJECTS=1 /bin/ls
        linux-vdso.so.1 =>  (0x00007fff2ce3b000)
        libselinux.so.1 => /lib/libselinux.so.1 (0x00007fb8a33a3000)
        librt.so.1 => /lib/librt.so.1 (0x00007fb8a319b000)
        libacl.so.1 => /lib/libacl.so.1 (0x00007fb8a2f94000)
        libc.so.6 => /lib/libc.so.6 (0x00007fb8a2c32000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007fb8a2a2e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb8a35c1000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007fb8a2812000)
        libattr.so.1 => /lib/libattr.so.1 (0x00007fb8a260e000)
root# export LD_LIBRARY_PATH=/usr/local/lib
root# LD_TRACE_LOADED_OBJECTS=1 /bin/ls
        linux-vdso.so.1 =>  (0x00007fff620e2000)
        libselinux.so.1 => /lib/libselinux.so.1 (0x00007f826134e000)
        librt.so.1 => /usr/local/lib/librt.so.1 (0x00007f8261146000)
        libacl.so.1 => /lib/libacl.so.1 (0x00007f8260f3f000)
        libc.so.6 => /usr/local/lib/libc.so.6 (0x00007f8260b93000)
        libdl.so.2 => /usr/local/lib/libdl.so.2 (0x00007f826098f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f826156c000)
        libpthread.so.0 => /usr/local/lib/libpthread.so.0 (0x00007f8260772000)
        libattr.so.1 => /lib/libattr.so.1 (0x00007f826056e000)
root# ls
ls: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument

Both versions of the library are built for the x86-64 architecture. The new one is built with debug symbols as I said. The old version is 2.11.3, the new one is 2.17. Debian distribution.

At least, by looking at http://upstream-tracker.org/versions/glibc.html, I don't notice breaking changes.

Is there backward compatibility for glibc in the sense that I described above? If yes, what can break the compatibility between the loader and the shared objects, besides major version upgrades? For example, can the compilation and linker options affect the compatibility? If there is no or very limited backward compatibility in practice, does that mean that upgrading the primary glibc on your machine is virtually impossible?

PS: I know how to fix the problem per se, or at least I hope that I do. I just want to get clarity on the subject in general.

Thanks

One thing is sure : You cannot have two times libc in the *path.
Better delete / uninstall libc-2.17 immediately.

It is possible to have an extra libc for experimental reasons.
But : A hideaway location is a must. Like /usr/local/glibc/(lib)
The current setup may at some time make the whole OS unusable.


If you want debugging symbols : Install the package libc6-dbg
http://packages.debian.org/squeeze/libc6-dbg


About the conflicts :
(error while loading shared libraries: __vdso_time:
invalid mode for dlopen(): Invalid argument)
→ → You are using the linker from version 2.11.3 for the "2.17" commands :
root# export LD_LIBRARY_PATH=/usr/local/lib
root# LD_TRACE_LOADED_OBJECTS=1 /bin/ls


Another thing : Do not use root.
The above commands are not "administrative commands".
*** Root is for administrative tasks only !


-

Another thing : Do not use root. The above commands are not "administrative commands".

Agreed. I was doing some other stuff, but sudo should be used in such cases instead (when you need to switch for some make install command.)

If you want debugging symbols : Install the package libc6-dbg

True. I learnt about the package in the process myself. But we may have to do some printf debugging or try something with the sources if we suspect a bug at some point. Just decided to go all the way, so that we can always rely that we can do things from the source if necessary. To put in other words, the way I see it right now, upgrading the primary glibc in /lib from 2.11.3 to 2.17 will also require rebuilding every piece of software. Having a secondary location will still most likely require rebuilding the shared objects of libgcc, libstdc++, etc. from gcc. So my question is, is there some official source of information or policy concerning the compatibility of the statically linked loader part and the shared objects of glibc between minor versions? Do those remain fixed for a given installation?

Other than that, I will use libc6-dbg for the moment. Thanks.

One thing is sure : You cannot have two times libc in the *path. Better delete / uninstall libc-2.17 immediately.

Well, at least in this case, it was not in the loader path. From the loader traces you can see that there is no mix and match going on if the new library is not placed in LD_LIBRARY_PATH. Probably bad cofiguration to begin with.

glibc is amazingly backward compatible. But glibc is not only libc.so.6 It also ld-linux.so and libnss* libraries.

You can compile your program against very old glibc and run same binary against very recent glibc.
But you can't take libc.so.6 from one version and ld-linux from another and hope that they will play well.

OK, you can hope, but no more than that.

I think that if you run your program as:

LD_LIBRARY_PATH=/usr/local/lib /usr/local/lib//lib64/ld-linux-x86-64.so.2 /bin/ls

it should work OK.

Anyway, why do you need another libc, what do you trying to achieve?

Ahaaa. I was left with the impression that the system is designed such that the loader can be changed only by patching the executable. But actually, this applies only to the statically linked loader, which is used by default. Apparently, one can always override this loader from the command line, as you did. I did not know that.

Thanks for the response. I will try it on Monday, as I don't have access to the machine at the moment. I may then have a couple of clarifying questions, but I will dig for the answers on my own before I state them.

Anyway, why do you need another libc, what do you trying to achieve?

The libc6-dbg package will be enough for a while. The overall aim is developing for Linux, and as a final measure, we want to be confident that we can upgrade the versions of the various components from source if needed. For example, the kernel, glibc, the compiler run-time, etc. I am sure this can be useful in many subtle ways occasionally, but also raises the question of compatibility. Especially after it was obviated to me from this encounter.

Thanks again.

Originally Posted by simeon_z Ahaaa. I was left with the impression that the system is designed such that the loader can be changed only by patching the executable. But actually, this applies only to the statically linked loader, which is used by default. Apparently, one can always override this loader from the command line, as you did. I did not know that. Thanks for the response. I will try it on Monday, as I don't have access to the machine at the moment. I may then have a couple of clarifying questions, but I will dig for the answers on my own before I state them. The libc6-dbg package will be enough for a while. The overall aim is developing for Linux, and as a final measure, we want to be confident that we can upgrade the versions of the various components from source if needed. For example, the kernel, glibc, the compiler run-time, etc. I am sure this can be useful in many subtle ways occasionally, but also raises the question of compatibility. Especially after it was obviated to me from this encounter. Thanks again.

I am not sure that I gave you correct command to try. Anyway, even if previous one doesn't work this one should:

/usr/local/lib/lib64/ld-linux-x86-64.so.2 --library-path  /usr/local/lib/ /bin/ls

About your goal - isn't it easier just install in VM a number of different distribution and try your program there?

Thanks again.

About your goal - isn't it easier just install in VM a number of different distribution and try your program there?

Yes, but sometimes a bug (or a feature) may force us to go beyond what the distribution offers. We can stay more current by installing from source. I am not trying to be imaginative and this is not a goal in itself, but it was already necessary with the compiler toolchain. Thus, we can even do intrusive debugging by modifying the run-time and bridging code, experiment with different build options, debug info formats, etc. Just hypothetically.

PS: I am afraid there was a small rush on the job. The test is a simple job, but the snapshot of the virtual machine with the alternative installation will not boot. It may be related to a software failure from last week or it may be that the glibc finally started to kick in after reboot. I will make the test tomorrow by other means.

With very slight modification (to include the original library path for non-glibc dependencies), both methods worked.

LD_LIBRARY_PATH=/usr/local/stow/glibc-2.17/lib:/lib /usr/local/stow/glibc-2.17/lib/ld-linux-x86-64.so.2 /bin/ls

/usr/local/stow/glibc-2.17/lib/ld-linux-x86-64.so.2 --library-path /usr/local/stow/glibc-2.17/lib:/lib /bin/ls

Thanks again. Very useful information.