lighttpd Segmentation fault on x86_64 in -current on __strcmp_ssse3

Hello,
I've upgraded today my -current (several month old) on, to be exact, Xeon E5335, to today's current and got
Segmentation fault in lighttpd (hand-compiled). I've made

make clean ;  ./configure --prefix=/usr/local/ --with-openssl --with-mysql ; make install

, then I've downloaded/compiled latest lighttpd, but still I have the same.

gdb says:

(gdb) run -D -f /usr/local/etc/lighttpd/lighttpd.conf
Starting program: /usr/local/sbin/lighttpd -D -f /usr/local/etc/lighttpd/lighttpd.conf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[Detaching after fork from child process 7355]
2018-09-17 20:26:28: (configfile.c.59) Warning: please add "mod_openssl" to server.modules list in lighttpd.conf.  A future release of lighttpd 1.4.x *will not* automatically load mod_openssl and lighttpd *will not* use SSL/TLS where your lighttpd.conf contains ssl.* directives

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7066056 in __strcmp_ssse3 () from /lib64/libc.so.6
(gdb)

Also, I've got the same problem with motion software (and again, I recompiled the version that worked, then took the lastest code), that looks like

(gdb) run -c /opt/motion/etc/motion/runconf/10082.conf -n
Starting program: /opt/motion/bin/motion -c /opt/motion/etc/motion/runconf/10082.conf -n
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[4641520:motion] [NTC] [ALL] conf_load: Processing thread 0 - config file /opt/motion/etc/motion/runconf/10082.conf
[4641520:motion] [ALR] [ALL] conf_cmdparse: Deprecated config option "ffmpeg_timelapse" since after version 4.0.1:
[4641520:motion] [ALR] [ALL] conf_cmdparse: "ffmpeg_timelapse" replaced with "timelapse_interval" option.
[4641520:motion] [ALR] [ALL] conf_cmdparse: Deprecated config option "ffmpeg_timelapse_mode" since after version 4.0.1:
[4641520:motion] [ALR] [ALL] conf_cmdparse: "ffmpeg_timelapse_mode" replaced with "timelapse_mode" option.
[4641520:motion] [NTC] [ALL] motion_startup: Motion 4.1.1 Started
[4641520:motion] [NTC] [ALL] motion_startup: Logging to file (/var/log/cam10082.log)

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff4ec6056 in __strcmp_ssse3 () from /lib64/libc.so.6
(gdb)

Just to be sure, I show you

# ldd motion | grep libc\.so
	libc.so.6 => /lib64/libc.so.6 (0x00007f916d665000)
# ls -l /lib64/libc.so.6
lrwxrwxrwx 1 root root 12 Sep 17 02:58 /lib64/libc.so.6 -> libc-2.27.so*
# ls -l /lib64/libc-2.27.so
-rwxr-xr-x 1 root root 2225736 Jun 13 00:52 /lib64/libc-2.27.so*

... and more! I've made a fresh install of 64-current and facing the same problem there!

What's happening? He-e-e-elp please!

hi, just to understand I have a few questions:

- can you please paste the output of the command

ls -la /var/lib/pkgtools/packages/glibc*

- if you execute the commands

slackpkg update
slackpkg install slackware64

which packages does slackpkg propose to install?

-

Originally Posted by dreadbit I've made a fresh install of 64-current and facing the same problem there!

you mean that you made a fresh full installation on the same hardware?

- where do you have installed motion from? a precompiled binary?

Originally Posted by ponce can you please paste the output of the command ls -la /var/lib/pkgtools/packages/glibc*

Thanks for reply.

# ls -la /var/lib/pkgtools/packages/glibc*
-rw-r--r-- 1 root root  24998 Sep 17 02:58 /var/lib/pkgtools/packages/glibc-2.27-x86_64-4
-rw-r--r-- 1 root root 275967 Sep 17 02:58 /var/lib/pkgtools/packages/glibc-i18n-2.27-x86_64-4
-rw-r--r-- 1 root root   1043 Sep 17 02:59 /var/lib/pkgtools/packages/glibc-profile-2.27-x86_64-4
-rw-r--r-- 1 root root   8552 Sep 17 02:22 /var/lib/pkgtools/packages/glibc-solibs-2.27-x86_64-4
-rw-r--r-- 1 root root  71354 Sep 17 02:59 /var/lib/pkgtools/packages/glibc-zoneinfo-2018e-noarch-3

can you please have a look also at the other questions?

Originally Posted by ponce can you please have a look also at the other questions?

Oh, sorry!

Originally Posted by ponce which packages does slackpkg propose to install?

slackpkg install slackware64

on original, upgraded machine proposes:

calligra*, dash, dovecot, easytag, efivar, fig2dev, fluidsync, gcc-brid, gexiv2, gnucobol, haveged, icecream, id3lib, igt-gpu-tools, intel-vaapi-driver, jmtpfs, json-c, json-glib, kde-i10n-*, lbzip, libidn2, libmilter, libmypaint, libsodium, libunwind, libva-utils, lmdb, lzlib, Mako, man-db, meson, mlocate, mypaint-brushes, ninja, npth, openssl10 & solibs, opusfile, parallel, patchelf, prce2, plzip, postfix, pyrasing, python-appdirs, python-certifi, python-chardets, python-docutils, python-inda, python-notify2, python-packaging, python-pip, python-requests, python-sane, python-urllib2, re2c, rust, rxvt-unicode, sc-im, SDL2_gfx, SDL2_image, SDL2_mixer, SDL2_net, SDL2_ttf, tdb, texlive, ttf-tlwg, urw-core35-fonts-otf-(smth), utf8proc, vulkan-sdk, wireless_tools, x86-video(chips,glint,i740,mga,nv,r128,savage,siliconmotion,sis,tdfx,trident,vboxvideo), xorgproto,xorriso, zstd)[/code]

The other computer is a full-fresh-install without Emacs,T,KDEI,Y, on the different hardware - but x86_64, too, some Core2 desktop.

Originally Posted by ponce - where do you have installed motion from? a precompiled binary?

As lighttpd, it is self-compiled. First I've run the existing version (4.0, which worked before upgrade) compiled with

make clean
autoreconf -fiv
./configure --prefix=/opt/motion
make

, then I recompiled that code on upgraded system, then I've downloaded 4.1.1 and recompiled with the same command

have you tried to look at the backtrace in gdb? when you have done this (please test on the full installation)

(gdb) run -D -f /usr/local/etc/lighttpd/lighttpd.conf
Starting program: /usr/local/sbin/lighttpd -D -f /usr/local/etc/lighttpd/lighttpd.conf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[Detaching after fork from child process 7355]
2018-09-17 20:26:28: (configfile.c.59) Warning: please add "mod_openssl" to server.modules list in lighttpd.conf.  A future release of lighttpd 1.4.x *will not* automatically load mod_openssl and lighttpd *will not* use SSL/TLS where your lighttpd.conf contains ssl.* directives

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7066056 in __strcmp_ssse3 () from /lib64/libc.so.6
(gdb)

after you should be able to issue these commands

set pagination off
bt full

then please paste here the resulting backtrace.

another test that you could do is installing the two applications via the SlackBuilds on SBo (read all the READMEs from there carefully) using their default configuration files (you can modify them later) and see if with those you got the same behaviour

https://slackbuilds.org/repository/1...work/lighttpd/
https://slackbuilds.org/repository/14.2/system/motion/

Motion:

Fresh -current install, motion 4.0.1 from SBo, works. Also, when I change 4.0.1 to 4.1.1 in SlackBuild files, 4.1.1 works too, and that is a good news.

Then, the upgraded computer.
4.0.1 from SBo works!
4.1.1 build with the same manner - changing release from 4.0.1 to 4.1.1 in Slackbuild file

Starting program: /usr/bin/motion n -c /opt/motion/etc/motion/runconf/10094.conf -n
(skipped)
[4645616:motion] [NTC] [ALL] motion_startup: Motion 4.1.1 Started
[4645616:motion] [NTC] [ALL] motion_startup: Logging to file (/var/log/cam10094.log)

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff4ec6056 in __strcmp_ssse3 () from /lib64/libc.so.6

(gdb) set pagination off
(gdb) bt full
#0  0x00007ffff4ec6056 in __strcmp_ssse3 () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007ffff68d972d in lh_insert () from /lib64/libcrypto.so.1
No symbol table info available.
#2  0x00007ffff68269cb in OBJ_NAME_add () from /lib64/libcrypto.so.1
No symbol table info available.
#3  0x00007ffff28be79e in ?? () from /lib64/libssl.so.1.1
No symbol table info available.
#4  0x00007ffff72aa517 in __pthread_once_slow () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007ffff279c129 in CRYPTO_THREAD_run_once () from /lib64/libcrypto.so.1.1
No symbol table info available.
#6  0x00007ffff28be97b in OPENSSL_init_ssl () from /lib64/libssl.so.1.1
No symbol table info available.
#7  0x00007ffff7f94d5b in ?? () from /usr/lib64/libmariadb.so.3
No symbol table info available.
#8  0x00007ffff7f7ebd6 in mysql_real_escape_string () from /usr/lib64/libmariadb.so.3
No symbol table info available.
#9  0x00007ffff72aa517 in __pthread_once_slow () from /lib64/libpthread.so.0
No symbol table info available.
#10 0x0000000000406078 in main ()
No symbol table info available.
(gdb)

So, a clear thing that something was broken during upgrades.

Lighttpd - for investigating tomorrow. Looks like that's ok on the fresh install from SBo (+ —with-mysql even in SlackBuild).

Thanks for help.

some considerations (in sparse order):

- I would try executing binaries using default configurations, to isolate problems tied to custom configurations: I'm saying this because I saw, just above, that you passed motion a custom configuration file.

- to have (maybe) more useful info in the backtrace you should execute it on binaries that had not been stripped from the debug symbols: it's ok if you build it by hand but in case of motion.SlackBuild you should remove this block from it before running gdb on the resulting installed binary

find $PKG -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF \
  | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true

this admitting that the issue is in motion and not in one of its dependencies: in that case you should rebuild with debug symbols also its dependencies (glibc, openssl, mariadb, etc.)

- you can try fixing the broken things reinstalling the packages you have already installed: regarding the fact that you choose to do not a full installation but a custom one (and, who knows, the issue you have might be also originated by that choice) I suppose that you can manage dependencies yourself and I won't get in that matter.
if that were to happen to me I would just issue a

slackpkg reinstall slackware64
slackpkg install slackware64

(the first one to reinstall the installed packages and the second one to install the missing packages not yet installed)

Originally Posted by ponce some considerations (in sparse order):

Thanks for your help once again.
Strange to report, but the problem... ahm.. faded away.
I started with slackbuilds and they worked.
There is absolutely no magic on slackbulds, they are much like make ; make install DESTROOT= ; makepkg.
Then I loaded ./configure in the slackbuilds with all the stuff I needed.
They worked again.
Then I rebuild motion & lighttpd from sources by hand on both machines with the same configure options, and they worked once more.

So the problem is magically resolved. And I cannot reproduce it.

The only.. I remember that was a warning like "libcrypto used by libmysql is different from a linked one" or like that. May be the problem was in the order of "--with-openssl —-with-mysql"... But, well, I have the only libcrypto in /usr/lib64 and... well, something with the same name in /lib , but it should be there, right?