Hardware accelerated video playback

pchristy · 10-20-2021, 07:44 AM

Has anybody got any experience of activating hardware accelerated video playback on 64-bit Pis?

I'm running a Pi 400 with slarm64. I have VLC (supplied by sndwvs) installed along with MPV (+smplayer) and ffmpeg (ffplay) installed, both of which I have compiled.

Playing 1080 h264 files on any of these results in 70-90% CPU useage on ALL FOUR cores! Playing 4K stutters badly.

ffmpeg contains the h264_v4l2m2m codec (and the hevc version, as well). Calling fflplay thus:

Code:

ffplay -codec:v h264_v4l2m2m Videos/test.mp4

produces a still frame - though the audio is running!

/BOOT/config.txt contains:

Code:

dtoverlay=vc4-fkms-v3d

Switching from fkms to kms makes the whole system slow and stuttery, and I have read (somewhere) that hardware acceleration only works with fkms. (Any comments?)

According to everything I've read, hardware accelerated video playback should be possible. Threads elsewhere seems to indicate that 64-bit Raspbian is also having the same issues - ie: hardware acceleration only seems to work with specific players (that I've never heard of!) and are only available as 32-bit.

So my question is: Has anyone managed to get hardware accelerated video playback on a 64-bit system, and if so, how did you manage it?

Cheers,

--
Pete

Dunc. · 10-25-2021, 10:57 AM

Things to check would be the amount of gpu memory > 256M should be enough. When you do lsmod you should see

Code:

bcm2835_isp            32768  0
bcm2835_codec          49152  0
bcm2835_v4l2           45056  0

Also try v4l2-ctl --list-devices

Code:

bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12
        /dev/media1

bcm2835-isp (platform:bcm2835-isp):
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16
        /dev/media0

I don't get accelerated video playback but does try on /dev/video10 and just sits there doing nothing.

Dunc.

pchristy · 10-26-2021, 04:28 AM

Hi Dunc, and thanks for the reply! Sorry for the delay in response, but I have family staying for a few days, and grand-children can be very demanding...

I've been following a guide I found here: https://www.dedoimedo.com/computers/...eleration.html

I've added the suggested lines to config.txt, and checking for the 3D drivers suggests they are loaded and working:

Code:

cat /proc/device-tree/soc/firmwarekms@7e600000/status
okay
cat /proc/device-tree/v3dbus/v3d@7ec04000/status
okay

Moving on to Firefox, and setting "layers.acceleration.force-enabled" to "true" does NOT result in compositing being set to "OpenGL". Of the video players, only VLC offers OpenGL as an option, but the CPU usage is the same as MPV/SMPlayer without OpenGL. This makes me wonder if the issue is with OpenGL.

Following your suggestions:

Code:

lsmod | grep bcm2835
bcm2835_isp            32768  0
bcm2835_codec          49152  0
bcm2835_v4l2           45056  0
v4l2_mem2mem           40960  1 bcm2835_codec
bcm2835_mmal_vchiq     40960  3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp
videobuf2_dma_contig    24576  2 bcm2835_codec,bcm2835_isp
videobuf2_vmalloc      20480  1 bcm2835_v4l2
videobuf2_v4l2         32768  4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
videobuf2_common       65536  8 bcm2835_codec,videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem,videobuf2_memops,bcm2835_isp
snd_bcm2835            28672  0
videodev              270336  6 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp
snd_pcm               131072  4 snd_bcm2835,snd_compress,snd_soc_core,snd_pcm_dmaengine
mc                     61440  7 videodev,bcm2835_codec,videobuf2_v4l2,dvb_core,videobuf2_common,v4l2_mem2mem,bcm2835_isp
i2c_bcm2835            16384  0
vc_sm_cma              36864  2 bcm2835_mmal_vchiq,bcm2835_isp
snd                    98304  5 snd_bcm2835,snd_timer,snd_compress,snd_soc_core,snd_pcm
spi_bcm2835            24576  0

Code:

v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12
        /dev/video18
        /dev/media1

bcm2835-isp (platform:bcm2835-isp):
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16
        /dev/media0

Not quite sure where this leaves me! As far as I can see, everything that should be present is, but it still doesn't work!

Any further suggestions very welcome!

--
Pete

Dunc. · 10-27-2021, 07:30 AM

I have tinkered a little and the only thing I did significant was a reboot. It now works doing:
ffplay -codec:v h264_v4l2m2m Big_Buck_Bunny_1080_10s_10MB.mp4

Code:

:~/Downloads$ ffplay -codec:v h264_v4l2m2m  Big_Buck_Bunny_1080_10s_10MB.mp4
ffplay version 4.4 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/doc/ffmpeg-4.4/html --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-gpl --enable-version3 --enable-avresample --arch=aarch64 --disable-encoder=aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gnutls --enable-libbluray --enable-libcaca --enable-libcdio --enable-frei0r --enable-openal --enable-libopus --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libmp3lame --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libpulse --enable-libsmbclient --enable-libxml2 --enable-librsvg --enable-libdrm
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Big_Buck_Bunny_1080_10s_10MB.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    composer        : Sacha Goedegebure
    encoder         : Lavf57.63.100
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
  Duration: 00:00:10.00, start: 0.000000, bitrate: 7986 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 7983 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
[h264_v4l2m2m @ 0x7f68004b20] Using device /dev/video10
[h264_v4l2m2m @ 0x7f68004b20] driver 'bcm2835-codec' on card 'bcm2835-codec-decode' in mplane mode
[h264_v4l2m2m @ 0x7f68004b20] requesting formats: output=H264 capture=YU12
  16.37 M-V: -0.018 fd=  75 aq=    0KB vq=    0KB sq=    0B f=0/0

Mplayer however stutters its way through. An error is shown.

Code:

Failed to open VDPAU backend libvdpau_vc4.so: cannot open shared object file: No such file or directory

I shall try to hunt for the missing library.

pchristy · 11-01-2021, 11:30 AM

Sorry for the delay in replying! I had the HDD go down in my workshop computer, and it took me ages to salvage the data off it!

Anyhoo: Still not getting anywhere with this. The latest upgrade to libvpx broke ffmpeg, mpv and vlc, so I took the opportunity to rebuild ffmpeg and mpv using the latest sources. (vlc was kindly sent me by sndwvs, but I don't have his source package, and my attempts to build it descend into dependency hell!)

If I try and play a test video (1080p/25fps shot on my video camera) using hardware acceleration, it goes through the motions:

Code:

ffplay -codec:v h264_v4l2m2m Videos/test.1080-25.mp4  
ffplay version 4.4.1 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 11.2.0 (GCC)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/doc/ffmpeg-4.4.1/html --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-gpl --enable-version3 --enable-avresample --arch=aarch64 --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libcdio --enable-frei0r --enable-openal --enable-libopus --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libmp3lame --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libpulse --enable-libsmbclient --enable-libxml2 --enable-librsvg --enable-libdrm
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Videos/test.1080-25.mp4':f=0/0   
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf53.32.100
  Duration: 00:02:25.47, start: 0.000000, bitrate: 5908 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 5645 kb/s, 50 fps, 50 tbr, 50 tbn, 100 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, stereo, fltp, 256 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
[h264_v4l2m2m @ 0x7f80031900] Using device /dev/video10
[h264_v4l2m2m @ 0x7f80031900] driver 'bcm2835-codec' on card 'bcm2835-codec-decode' in mplane mode
[h264_v4l2m2m @ 0x7f80031900] requesting formats: output=H264 capture=YU12
  15.88 A-V: -1.740 fd=   1 aq=   32KB vq= 9047KB sq=    0B f=0/0

The counter at the end runs as if it is playing, but the image is frozen. The video plays OK without the -codec option (software decoding), but with a very high cpu useage on all 4 cores.

And I still can't play 4K at normal speed.

Clearly I'm missing something, but what?

--
Pete