32-bit GLX (nvidia) not working after update, unless running with strace
 

TLDR: using binutils-2.42 to compile glibc-multilib on slackware isn't compatible with the 32-bit nvidia binary libGLX_nvidia. If binutils-2.41 is used to compile glibc-multilib then everything is ok. Also copying ld-2.39.so from a slackware-current 32-bit system breaks libGLX_nvidia in the same way (but libGLX_mesa is ok).

I did upgrade-all on slackware64-current and restarted the PC since it also updated glibc.
Now 32-bit GLX things segfault unless I run them with strace.

I have recompiled the nvidia-kernel and nvidia-driver 535.154.05_multilib packages.
Also upgraded mesa-compat32 by using convertpkg on the mesa-24.0.0-i586 package.

I would have tried downgrading glibc back to 2.38_multilib but there doesn't seem to be anywhere to download older versions from.

Anything else I can try?

The backtrace from /usr/bin/32/glxinfo is:

Hrm. I got that, too. But the 32-bit vkcube works, as does a 32-bit WINE game. Does anything else that's 32-bit break for you?

ETA: Steam segfaults for me. I tried to reset it, and gets to about this line and crashes: "steam.sh[5893]: Can't find 'steam-runtime-check-requirements'"

glxgears is also broken for me. LDD isn't showing anything missing, though.

It is only 32 bit glx things that segfault for me. I just tested vkcube and that works ok.
It was actually when running steam I first noticed the problem :) I got steam working last night by running it as:

I've reinstalled (completely) nvidia drivers (instead of just using -K). (Patched to work with the 6.6.15 kernel.), 535.154.05. I tried the old kernel, too, thinking it might be the patch causing it.
I've upgraded mesa for 32-bit (thinking there was a mismatch), ditto for SDL2.
I've tried a completely new user.
Steam is apparently working for Chrisretusn: https://www.linuxquestions.org/quest...ml#post6481208

I wonder if it's something to do with nvidia 535. Chrisretus seems to be using nvidia driver 470.
I will have to try building nvidia 545 or 550 and see if they work.

Nope. I just built nvidia-driver-550.40.07_multilib but still get the segfault with glx* programs.

So could possibly be a multilib issue? (Something needs to sync)

I guess it is something to do with multilib since only the 32bit versions are broken. Maybe some more of the packages need recompiling against glibc-2.39. It seems to be /usr/lib/libGLX.so.0 where the problem is, and that is the libglvnd-compat32-1.7.0 package.

edit: Just recompiled libglvnd but that doesn't fix it.

What's weird is that it hasn't been changed since September, though. (And why does the 64-bit one work?)

Removed

I think that glvnd just calls whatever the real libGLX is on the system, and it looks like when trying to load the nvidia libGLX it is running into the problem.
You can tell libglvnd to use a different vendor GLX using the __GLX_VENDOR_LIBRARY_NAME env var.

This works:
(tested with 32-bit mesa 24 and 23.3.2)

This doesn't:
But this does:
So it looks like a problem with either the nvidia driver or libglvnd, but only for 32-bit libs.

edit:
I just tried changing the symlink /usr/lib/libGLX_nvidia.so.0 to point to libGLX_mesa.so.0.0.0 and then glxinfo works using the "nvidia" vendor. So we know it is finding the nvidia GLX library OK, it's just crashing when trying to load it.

From a reddit thread.

This isn't that problem...it's any 32-bit GLX/OpenGL program not running unless a debugger is also running.

Wild-assed guess, but:

Was added on January 29th. Could we need it on the 32-bit end of things?

Otherwise, I'm waiting for the official multilib refresh. (Should be Real Soon Now) I know Chrisretusn is more current than we are, and there could be something we need an update on that has been updated since Jan. 5. It shouldn't matter, I know, but weird library errors sometimes happen when multilib gets stale.

That package only contains header files, no binaries or libraries so won't need a compat32 version.

Everything else 32-bit seems to be fine, including vkcube, so I don't think it is a problem with multilib. Since libglvnd works OK with the Mesa 32-bit libs, it looks more like the nvidia libGLX isn't compatible with glibc-2.39 and so when libglvnd tries to dlopen it, it segfaults.
Can't see any mention of it on the nvidia forums though.

If you want to open a thread on the forums there, I'm game. Looks like Arch is on 2.39, but I don't know if any problems there have been reported? (Kind of waiting for multilib to sync, just to rule out any weirdness with stale libraries there.)

I have started a thread on the nvidia forums now at https://forums.developer.nvidia.com/...egfault/281769
So will see if any others have the same problem.

OK...will post there if I'm still having problems after multilib refreshes.

Just thought I would report in. I just tested this right now and I am having the same issue.

/usr/bin/32/glxgears
steam
dump file (portion, if helpful)

I used Alien Bob's mirror_slackware_current.sh script to create an up to date 32bit iso, so my multilib packages are perfectly in sync with -current. I will start investigating and report back here if I find something.

Added my two cents to the nvidia thread. So strange that Arch isn't having the same problem. Normally if they encounter a strange bug, so do we. (Since slackware-current and arch are often on the same versions of things.)

Yes, I did check the Arch forums too but don't see any mention of it. Since it affects steam I would have expected a lot of people to have the problem.
Arch also seems to have the same version of libglvnd without any extra patches.
Arch does use different configure options when building 32bit glibc though (I think)

Arch from https://gitlab.archlinux.org/archlin...ref_type=heads:
Slackware from http://www.slackware.com/~alien/mult...lib.SlackBuild:
I might be reading the build scripts wrongly though :)

So I'm wondering if it's something 64-bit has that 32-bit doesn't, namely something added to Slackware that needs to be picked up for multilib. Here's a complete list of everything that was added since Jan. 5, the last time multilib was refreshed:

l/libnvme-1.7.1-x86_64-1.txz
d/nv-codec-headers-12.1.14.0-x86_64-1.txz
l/libass-0.17.1-x86_64-1.txz
l/libplacebo-6.338.2-x86_64-1.txz
l/python-editables-0.5-x86_64-1
l/python-hatchling-1.21.0-x86_64-1.txz
l/python-pathspec-0.12.1-x86_64-1.txz
l/python-pluggy-1.4.0-x86_64-1.txz:
l/python-sphinx_rtd_theme-2.0.0-x86_64-1.txz
l/python-trove-classifiers-2024.1.8-x86_64-1.txz
l/python-html5lib-1.1-x86_64-1.txz
l/python-webencodings-0.5.1-x86_64-1.txz

I'm not completely convinced it's an nvidia bug, since Arch isn't having the same problem. (Nor is 64-bit Slackware.)

If it was a missing library then it just wouldn't work at all, even when run with gdb or strace.
The reason I think it's something to do with the way the nvidia libGLX_nvidia is compiled is because it works OK when forcing libGLX_mesa instead.

ldd /usr/lib/libGLX_nvidia.so.0 shows that no libraries are missing for me. Maybe we could try rebuilding some of the libraries listed there though, like libxcb, libXau, libXdmcp.

edit: it looks like they are all built as part of a bigger x11 build script, so can't just build them individually

I just recompiled glibc 2.39 multilib and added the configure options that are present in the Arch build script (except --enable-systemtap) but I still get the same segfault with glx.

In the past, though, whenever there's been an unexplained issue with a library and multilib, it's usually something missing from multilib. My logic was that 64-bit slackware works, so it's got something the multilib set doesn't have, since a whole lot was added to it. Aside from that and what's been tried here, I'm not sure. There's not much movement on the nvidia forum thread.

I know what you mean about waiting for multilib to catchup, I have noticed that before too in the past :)
This problem only started for me after installing updates yesterday, and those updates were:

Also multilib glibc 2.39 - but since I have been playing with different versions that isn't showing up as installed at the same time any more.

I was sure it was glibc 2.39 which caused the problem.. but I have now recompiled glibc 2.38 and I am still getting exactly the same segfault. So now i'm more confused :)

Edit: just had another thought about the kernel headers. Maybe since I have the 6.6.15 kernel headers installed now, my glibc build used them and so it's not the same as the old 2.38 that was built before. I'm not sure if they would make a difference to the resulting glibc but I guess trying with older kernel headers is something I could test tomorrow.

I don't think it's 6.6.15. I tried 6.6.14 with nividia drivers compiled for 6.6.14, and I was getting the same segfault. (My theory was it was the patch causing issues.) 6.6.14 was working fine yesterday before updating.

I wish I had an AMD card I could toss in and see what happens.

going through logs. I found this:

I am not so sure it's the nvidia patch either. As a test I recompiled the kernel with the patch that Bathory provided here. After compiling the kernel I was able to build both the nvidia-driver and nvidia-kernel without patching, but still got the error unless I ran steam and 32bit glxgears with strace.

I have now installed the linux kernel headers 6.6.8 and used them to rebuild glibc-2.38 and libglvnd but still getting the segfault.
Also other libraries listed by ldd for glxinfo and libGLX_nvidia.so were last modified before February. So that should mean I have all the versions back to how they were before it went wrong, but it's still not working.

The previous stock glibc-2.38 was built 2023-10-13 (Pat) or 2023-10-14 (Eric). It was built against the latest kernel headers at that time. It was kernel-headers-6.1.57 (2023-10-11). There shouldn't be much or any difference between the 6.6.8 and 6.6.15 headers, but in principle, there could be some difference between 6.1.57 and 6.6.15 headers. Maybe you could try to rebuild glibc-2.38 against the 6.1.57 headers?

Ah ok, I will try with older headers later then :)
The reason I used 6.6.8 is because that's the kernel I currently have installed, and it was after updating to 6.6.15 that people needed an extra patch to build the nvidia module.

Or you could use Alien Bob's old multilib glibc-2.38. Google found a mirror not upgraded yet: https://www.slackjeff.com.br/multilib/current/. If you check the packages against the gpg signatures, it should be safe.

Thanks, I had tried searching for old versions but google just kept sending me to Eric's site, and archive.org only had 2.37. Eric still has the signatures up on his site so I can use them to verify.

Finding those old packages made things much easier :)
I have just reinstalled Eric's original versions (verified the md5sums) and now 32bit glxinfo and steam are working fine again.

Just before, I did recompile glibc using kernel headers 5.15.145 from Slackware-15.0 since it was easy to install them but that glibc also caused a segfault.

So there seems to be something else different about those 2.38 packages. GCC hasn't been updated since August 2023 so shouldn't be that. Maybe binutils makes a difference? But Arch is also using binutils-2.42.

The working versions for me are:
glibc-profile-2.38_multilib-x86_64-3alien.txz
glibc-i18n-2.38_multilib-x86_64-3alien.txz
aaa_glibc-solibs-2.38_multilib-x86_64-3alien.txz
glibc-2.38_multilib-x86_64-3alien.txz

I have just rebuilt binutils-2.41 but without the binutils-readelf-other-sym-info.patch.gz patch which seems to have been added for the 2.42 build.
I then rebuilt glibc-2.38 again using my binutils-2.41 and kernel-headers 5.15. This glibc 2.38 is also working with 32bit GLX.

So now I'm not sure if the problem is binutil-2.42, the binutils-readelf-other-sym-info.patch.gz patch, or does it just work because I built it while I already had Eric's original glibc-2.38 installed? So many possibilities :)

I guess the next thing to try is update everything except binutils and then recompile glibc-2.39 using binutils-2.41.

https://slackware.uk/cumulative/ Does this help out any?

So if glibc 2.39 is the issue, could it be something with our build? I'm still stuck on arch having binutils 2.42 and glibc 2.39 and no problems reported... That would indicate an issue with us, correct? (And why Christesrun isn't having problems.)

It doesn't look like glibc-2.39 is really the issue, since when I compiled my own 2.38 that also didn't work - so it looks more like the tools used to build glibc are causing the problem.
If Christesrun is using an old version of the nvidia driver it might not be using libglvnd and programs are just linked directly to the nvidia libGLX. The crash is happening when libglvnd tries to load the nvidia libGLX.

I have just updated my system again now apart from binutils (it looks like a lot of compat32 updates are there now too). After the update 32 bit glx is broken again. So now I will rebuild glibc-2.39 using binutils-2.41 and see what happens.

edit: glibc-2.39 built with binutils-2.41 + kernel-headers 6.6.15 works. So next is to try binutils-2.42 without the readelf patch.

https://gitlab.archlinux.org/archlin...ls/-/tree/main Could be something to it. They don't use the readelf patch.

I tried building binutils-2.42 with all but the readelf patch, and then glibc using kernel-headers 6.6.15 but that gave a bad glibc.
Since they only used the gold-warn-unsupported patch in arch I tried building binutils-2.42 with only that patch but that also resulted in a bad glibc.
I now got a good glibc-2.39 by using binutils-2.41 with all but the readelf patch.

So far I have:
I wonder if I now try rebuilding binutils 2.42 using my good glibc it will result in a binutils which can build another working glibc.

I have uploaded my working glibc-2.39 which was compiled with binutils-2.41 and headers 6.6.15 to http://135.125.183.217/glibc/ in case anyone wanted to test if it works for them. At your own risk of course :)

Hrm. I wonder why we need the readelf patch? (Is Pat lurking? Some insight would be interesting.)

The only reason I built 2.41 without the readelf patch is because it wouldn't apply to 2.41 :) but when I built 2.42 without it, that also didn't fix the glibc problem.

edit: just tried again building binutils-2.42 with all except readelf patch, and rebuilding glibc starting with a working glibc this time and the result is a broken glibc again. So it looks like binutils-2.42 is causing the problem somehow.

There was a binutils-readelf-other-sym-info.patch already in slackware-15.0 (binutils-2.37). It's from fedora:
I guess some builds run readelf and expect a certain output.

Would you do this test? Install a bad glibc-2.39 produced by binutils-2.42. And then copy only /lib/libdl-2.33.so from a good glibc-2.39 produced by binutils-2.41. Does the good libdl fix GLX?

I will do, I am just building another glibc-2.39 using binutils-2.42 with arch configure options which I expect to end up bad so I can use that for your test.

While looking at the Arch build script I saw the comment
So now I am also wondering if gcc needs to be rebuilt after building a new binutils, and use that to create the new glibc. Since gcc hasn't been rebuilt since the binutils upgrade on Slackware that might be why using binutils-2.41 is working for me since the current gcc was built with binutils-2.41.

Never mind. Forget my test. I just noticed that Alien Bob's /lib/libdl-2.38.so and /lib/libdl-2.39.so have exactly the same md5sum even though they are from different glibc versions, built using different binutils versions!

But maybe you see changes in md5sum between a working and broken libc-2.39.so ?

I don't think there's ever been a gcc rebuild before/after glibc? There was a giant mass rebuild a couple years ago, though.

The new broken glibc is built now and crashes with:
The md5sums are the same for my bad installed one, and my working one
I did your test anyway by copying the "good" libdl over the bad one, and glxinfo still crashed in exactly the same way - which is a good thing otherwise I would have been more confused :)

I think I should try installing my binutils-2.41 glibc again, then build binutils-2.42 with all the Slackware patches, and then rebuild gcc _before_ building glibc again.
I haven't compiled gcc since about 1999 on a Pentium 166Mhz, hopefully it doesn't take as long as it did back then :)

How about /lib/ld-2.39.so? Is its md5sum the same for the good and bad one? Or /lib/libc-2.39.so?

No, the md5sum for those a different. The sizes are the same though
It is currently building gcc-multilib so I can't test overwriting a broken ld-2.39 with a good one at the moment. But maybe the GCC->glibc build will also fail and then I can :)

edit: the libc sizes are also the same, but md5sums different
edit2: I did objdump -D on them and diff and there are a lot of changes to the instructions, so the difference isn't just some version string