Rebuild FFmpeg fails with "nvidia-legacy340" driver installed

joenew · 06-06-2022, 11:11 AM

I've installed nvidia-legacy340 both kernel and driver, from SBo:

https://slackbuilds.org/repository/1...acy340-driver/

After this I noticed some issues with OpenCL:

Code:

/usr/lib64/libOpenCL.so.1.0.0

is overwritten by nvidia driver package, the default was from:

Code:

ocl-icd-2.3.1-x86_64-1

Runtime issues and warnings seem solved by reinstall this ocl-icd package for orig file restoring.
Anyway I was wondering if could be better to rebuild ffmpeg against the nvidia OpenCL version of that lib.

So I reinstalled nvidia driver, so that nvidia libOpenCL.so is in place and I've tried to build FFmpeg from source using Pat SlackBuild:

https://mirrors.slackware.com/slackw...urce/l/ffmpeg/

Anybody tried?
It returns this issue to me:

Code:

MAN     doc/libavformat.3
MAN     doc/libavdevice.3
MAN     doc/libavfilter.3
LD      libswscale/libswscale.so.5
LD      libpostproc/libpostproc.so.55
LD      libswresample/libswresample.so.3
LD      libavresample/libavresample.so.4
LD      libavcodec/libavcodec.so.58
LD      libavformat/libavformat.so.58
LD      libavfilter/libavfilter.so.7
LD      libavdevice/libavdevice.so.58
LD      ffmpeg_g
LD      ffplay_g
LD      ffprobe_g
/usr/bin/ld: libavutil/libavutil.so: undefined reference to `clGetExtensionFunctionAddressForPlatform'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clEnqueueFillBuffer'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clCreateImage'
collect2: error: ld returned 1 exit status
make: *** [Makefile:124: ffprobe_g] Errore 1
make: *** Attesa per i processi non terminati....
/usr/bin/ld: libavutil/libavutil.so: undefined reference to `clGetExtensionFunctionAddressForPlatform'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clEnqueueFillBuffer'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clCreateImage'
collect2: error: ld returned 1 exit status
make: *** [Makefile:124: ffplay_g] Errore 1
/usr/bin/ld: libavutil/libavutil.so: undefined reference to `clGetExtensionFunctionAddressForPlatform'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clEnqueueFillBuffer'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clCreateImage'
collect2: error: ld returned 1 exit status
make: *** [Makefile:124: ffmpeg_g] Errore 1
LD      ffmpeg_g
/usr/bin/ld: libavutil/libavutil.so: undefined reference to `clGetExtensionFunctionAddressForPlatform'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clEnqueueFillBuffer'
/usr/bin/ld: libavfilter/libavfilter.so: undefined reference to `clCreateImage'
collect2: error: ld returned 1 exit status
make: *** [Makefile:124: ffmpeg_g] Errore 1

It is possible to solve or only chance is to restore the OpenCL lib from official slack?

RadicalDreamer · 06-06-2022, 12:48 PM

Try disabling opencl and then build ffmpeg. Change

Code:

opencl=""     ; [ "${OPENCL:-yes}" != "no" ]      && opencl="--enable-opencl"

to

Code:

opencl=""     ; [ "${OPENCL:-yes}" != "yes" ]      && opencl="--enable-opencl"

joenew · 06-06-2022, 01:36 PM

I'll try and report here...

Anyway is this a right way to solve?
I'm trying to link FFmpeg to Nvidia libOpenCL.so, not to exclude OpenCL from FFmpeg.

joenew · 06-06-2022, 02:18 PM

Just for report, disabling "opencl" make the above error disappeared:

Code:

MAN     doc/libavformat.3
MAN     doc/libavdevice.3
MAN     doc/libavfilter.3
LD      libswscale/libswscale.so.5
LD      libpostproc/libpostproc.so.55
LD      libswresample/libswresample.so.3
LD      libavresample/libavresample.so.4
LD      libavcodec/libavcodec.so.58
LD      libavformat/libavformat.so.58
LD      libavfilter/libavfilter.so.7
LD      libavdevice/libavdevice.so.58
LD      ffmpeg_g
LD      ffprobe_g
LD      ffplay_g
STRIP   ffmpeg
STRIP   ffplay
STRIP   ffprobe

But the question remains: how to build FFmpeg linking it properly to Nvidia libOpenCL.so.* ?
(if it is possible)

ppr:kut · 06-06-2022, 02:59 PM

libOpenCL is a vendor neutral dispatching library, not an actual OpenCL implementation. Applications are built against that vendor neutral library so it doesn't matter which hardware is available at runtime. Consequently it should also not be replaced, nor overwritten. In that regard it works similar to how libvulkan and libglvnd work for Vulkan and OpenGL respectively.

LuckyCyborg · 06-06-2022, 03:39 PM

Quote:

Originally Posted by ppr:kut

libOpenCL is a vendor neutral dispatching library, not an actual OpenCL implementation. Applications are built against that vendor neutral library so it doesn't matter which hardware is available at runtime. Consequently it should also not be replaced, nor overwritten. In that regard it works similar to how libvulkan and libglvnd work for Vulkan and OpenGL respectively.

Tell this to the guy who manages those NVIDIA SlackBuilds from SBo.

Looks like he has ZERO respect for things named like libOpenCL, libvulkan and libglvnd.

No offense intended, BUT his packages are so invasive (and I would say, destructive), that makes the puppies cries.

joenew · 06-06-2022, 03:47 PM

So, Nvidia provides libOpenCL.so within their driver distribution (legacy340 in my case), while the one provided with vanilla slackware installation is "something different".

1. In what they differ exactly?
2. Why FFmpeg can link fine to the slackware libOpenCL.so, but returns build errors with the one provided by Nvidia?

EDIT:
I mailed SBo nvidia-legacy340 maintainer asking for infos about this issue, but no response for now.
I don't know if he are still monitoring and testing his slackbuild.
Hope my ignorant questions above could be useful to find a proper solution as well as help myself to better understand how it works under the hood.

LuckyCyborg · 06-06-2022, 03:57 PM

Quote:

Originally Posted by joenew

So, Nvidia provides libOpenCL.so within their driver distribution (legacy340 in my case), while the one provided with vanilla slackware installation is "something different".

1. In what they differ exactly?
2. Why FFmpeg can link fine to the slackware libOpenCL.so, but returns build errors with the one provided by Nvidia?

This "something different" means that they offer different API.

That's why you get build errors. They aren't compatible implementations - probably the versions differ.

Better to build your ffmpeg without OpenCL.

ppr:kut · 06-07-2022, 01:34 AM

libOpenCL in -current provides support for the latest OpenCL version, matching the headers. libOpenCL then internally checks which version the vendor supports. If you replace the system libOpenCL with a different one, especially a much older one like for the nvidia legacy drivers, the OpenCL version between lib and headers no longer matches.

But even if you update the headers as well, you have then other applications in Slackware that are linked against a libOpenCL that supports the newer OpenCL APIs that the older libOpenCL wouldn't support. It *should* be fine if the .so version matches, but there might be other things going wrong.

joenew · 06-07-2022, 04:23 AM

Maybe I understood inner workings a bit more... the dispatcher idea in particular. From "man libopencl", If I well understood, what in fact nvidia driver looking for are opencl-libs by scanning /etc/OpenCL/vendors/nvidia.icd. Here we have a record which defines libnvidia-opencl.so.1:

Code:

$ cat /etc/OpenCL/vendors/nvidia.icd 
libnvidia-opencl.so.1

Both nvidia.icd and following files are from nvidia driver package:

Code:

$ find /usr/lib* -name libnvidia-opencl* -exec file "{}" \; 2>/dev/null
/usr/lib/libnvidia-opencl.so.1: symbolic link to libnvidia-opencl.so.340.108
/usr/lib/libnvidia-opencl.so.340.108: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
/usr/lib64/libnvidia-opencl.so.1: symbolic link to libnvidia-opencl.so.340.108
/usr/lib64/libnvidia-opencl.so.340.108: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped

So the driver provides its own old dispatcher libOpenCL.so, but the system has a newer dispatcher from OpenCL ICD loader (thnks to ocl-icd package). The behavior during FFmpeg build suggests that gcc or the linker or some other piece of the builder can't find libnvidia-opencl.so.1 through the old libOpenCL.so (provided with the driver), while it doesn't return any error with the new OpenCL dispatcher "slackware stock".
I would conclude that OpenCL from nvidia-legacy not only is unneeded, but it also breaks new OpenCL dispatcher, and it breaks opencl mechanism not just for nvidia but for any other modern program too. It causes useless opencl managing for the whole system.

So, libOpencl.so.1.0.0 has to be remove from the nvidia driver package, this is verified. A confirm is appreciated.
Anyway we could use some "acid test" to check whether nvidia OpenCL feature is really working properly using the new dispatcher provided by ocl-icd.
Could you suggest how to check it?
For instance, I could build FFmpeg with opencl enabled and ocl-icd dispatcher in place. It should compile and link without any issue. But how can I test then if ffmpeg can really use that feature?

ponce · 06-07-2022, 05:20 AM

Quote:

Originally Posted by joenew

Anyway we could use some "acid test" to check whether nvidia OpenCL feature is really working properly using the new dispatcher provided by ocl-icd.
Could you suggest how to check it?

to test if you have an opencl device available and working you can use a small C program called clinfo, you can find the SlackBuild here
https://ponce.cc/slackware/testing/clinfo/

Quote:

Originally Posted by joenew

For instance, I could build FFmpeg with opencl enabled and ocl-icd dispatcher in place. It should compile and link without any issue. But how can I test then if ffmpeg can really use that feature?

at this link there should be some hints ffmpeg-specific: note that the command

Code:

ffmpeg -h filters | grep opencl

in the newer ffmpeg versions has become

Code:

ffmpeg -filters | grep opencl

joenew · 06-07-2022, 08:39 AM

Tried to edit original slackbuild so that it doesn't generate libOpenCL.so anymore. A patch is attached to this message.

After being sure to:

- removed nvidia-legacy340-driver (after launching nvidia-switch --remove)
- slackpkg reinstall ocl-icd
- install again the new nvidia package as changed through the patch
- rebuilt ffmpeg using Pat slackbuild with some additions:

Code:

#!/bin/sh
X264=yes \
ASS=yes \
AOM=yes \
DAV1D=yes \
./ffmpeg.SlackBuild

It builds properly without any issue, opencl support should be enabled by default.
After following Ponce instructions (thank you!) I share the output below:

Code:

$ ffmpeg -filters|grep opencl
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (GCC)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/doc/ffmpeg-4.4.1/html --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-gpl --enable-version3 --enable-avresample --arch=x86_64 --disable-encoder=aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libcdio --enable-frei0r --enable-openal --enable-libopus --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libmp3lame --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libpulse --enable-libsmbclient --enable-libxml2 --enable-librsvg --enable-libdrm --enable-libaom --enable-libdav1d
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
 ... avgblur_opencl    V->V       Apply average blur filter
 ... boxblur_opencl    V->V       Apply boxblur filter to input video
 ... colorkey_opencl   V->V       Turns a certain color into transparency. Operates on RGB colors.
 ... convolution_opencl V->V       Apply convolution mask to input video
 ... deshake_opencl    V->V       Feature-point based video stabilization filter
 ... dilation_opencl   V->V       Apply dilation effect
 ... erosion_opencl    V->V       Apply erosion effect
 ... nlmeans_opencl    V->V       Non-local means denoiser through OpenCL
 ... overlay_opencl    VV->V      Overlay one video on top of another
 ... pad_opencl        V->V       Pad the input video.
 ... prewitt_opencl    V->V       Apply prewitt operator
 ... program_opencl    |->V       Filter video using an OpenCL program
 ... roberts_opencl    V->V       Apply roberts operator
 ... sobel_opencl      V->V       Apply sobel operator
 ... tonemap_opencl    V->V       Perform HDR to SDR conversion with tonemapping.
 ... transpose_opencl  V->V       Transpose input video
 ... unsharp_opencl    V->V       Apply unsharp mask to input video
 ... xfade_opencl      VV->V      Cross fade one video with another video.
 ... openclsrc         |->V       Generate video using an OpenCL program

Many matching lines containing opencl related infos. From the banner line at the top "--enable-opencl" is present too.
Seems opencl support is built in FFmpeg and the system noticed it.
Could you confirm from this output?
Or some other check should be executed to ensure it?

joenew · 06-08-2022, 01:45 PM

I have to chnange what I reported above.
No, removing libOpenCL.so from Nvidia installed files isn't enough.
May be I had reinstalled ocl-icd after driver installation. So it worked properly.

But if you have a vanilla slackware and:
- black list nouveau
- install nvidia kernel (patched thanks to AUR diff files)
- install nvidia driver, even with my diff applied to SlackBulid script as I attached in previous message (it will not install libOpenCL.so file)

Then If you attempt to build FFmpeg, it doesn't work and returns the same usual Error as reported at the top of this topic.
Somehow the nvidia driver script overwrites some other ocl-icd file, maybe during post-install when various symlinks are created.

Anyway by reinstalling ocl-icd the issue seems solved, at least partially and allows to build ffmepg without errors.
But this isn't a good way to fix, it would be better to rewrite slackbuilds completely.