[SOLVED] Slarm64 on the Pinebook 1080 - help with kernel?

shelldweller · 04-02-2021, 06:27 AM

Well, so far 5.11.y is an interesting release. It works fine on my Raspberry Pi 4, but is unstable on the Rock64 (acts fine at first, but eventually the board locks up and stops responding entirely), and acts REALLY strange on the Pinebook.

For the Pinebook, even the two images give different behavior. The base image will boot all the way to a command prompt, and you can build up a system from there, but eventually random features do not work as expected, like shutting down or rebooting.

The xfce image is even weirder: on first boot it hangs right after displaying the ssh keys random art, right before the "devpts /dev/pts" line. It never gets through even the first boot completely. This only happens on the xfce image, which is beyond strange.

I even tried using an older image and upgrading the kernel on that one to 5.11.y. It acts like the base image above: random features go away such as rebooting.

This is happening on both 5.11.10 and 5.11.11 so far. I have tried multiple builds on multiple machines, and the behavior has been repeatable. I have not had the time to generate and look at logs yet. That might happen over the weekend (depending on my workload, I am trying to meet multiple deadlines in the next 2-3 weeks...). I just thought I would let you know that so far it is super buggy. Does it have all the right patches and configurations for the Pinebook's quirks? Just checking.....

I still need to do some long-term debugging on 5.4.y and 5.10.y. Both eventually give some sort of instability to the X server, either crashing or just plain locking up, after they have been up and running for a while. I still am having the best experience on 5.6.19, for what it is worth....

thanks much, as always.

UPDATE: I was able to grab some log files from a fresh xfce 5.11.11 image's first boot. I put them all here.

sndwvs · 04-02-2021, 02:30 PM

Quote:

Originally Posted by shelldweller

Well, so far 5.11.y is an interesting release. It works fine on my Raspberry Pi 4, but is unstable on the Rock64 (acts fine at first, but eventually the board locks up and stops responding entirely), and acts REALLY strange on the Pinebook.

For the Pinebook, even the two images give different behavior. The base image will boot all the way to a command prompt, and you can build up a system from there, but eventually random features do not work as expected, like shutting down or rebooting.

The xfce image is even weirder: on first boot it hangs right after displaying the ssh keys random art, right before the "devpts /dev/pts" line. It never gets through even the first boot completely. This only happens on the xfce image, which is beyond strange.

I even tried using an older image and upgrading the kernel on that one to 5.11.y. It acts like the base image above: random features go away such as rebooting.

This is happening on both 5.11.10 and 5.11.11 so far. I have tried multiple builds on multiple machines, and the behavior has been repeatable. I have not had the time to generate and look at logs yet. That might happen over the weekend (depending on my workload, I am trying to meet multiple deadlines in the next 2-3 weeks...). I just thought I would let you know that so far it is super buggy. Does it have all the right patches and configurations for the Pinebook's quirks? Just checking.....

I still need to do some long-term debugging on 5.4.y and 5.10.y. Both eventually give some sort of instability to the X server, either crashing or just plain locking up, after they have been up and running for a while. I still am having the best experience on 5.6.19, for what it is worth....

thanks much, as always.

UPDATE: I was able to grab some log files from a fresh xfce 5.11.11 image's first boot. I put them all here.

shelldweller, thanks for the logs provided.

for allwinner the patches are the same as 5.10.y.
confused by this error: drivers/clocksource/arm_arch_timer.c:364 sun50i_a64_read_cntpct_el0+0x30/0x40. parsing

Rock64 which version of the bootloader, stable work with DDR blobs.

update:
I checked 5.11.11 on Rock64 there are no freezes.

shelldweller · 04-03-2021, 09:03 AM

Quote:

Originally Posted by sndwvs

Rock64 which version of the bootloader, stable work with DDR blobs.

update:
I checked 5.11.11 on Rock64 there are no freezes.

Ah,that would explain it then. I had only updated the kernel files on my LAN server, I did not update the bootloader at the same time. I will try that when I get the chance next. It might not be for another few weeks before I can really test it thoroughly. If I run into anything interesting, I will post in the appropriate forum.

Regarding the Pinebook stuff, it makes sense that it is all connected to cpufreq somehow. I went back to an old 5.5.19 image just to see what was so great about it, and one thing I noticed is that there was no cpufreq capabilities at all. It would just spit out an error message saying that no governor had been loaded. So, no wonder I was not having those issues back then. The cpufreq driver probably did not work at all then, and now it works but is buggy.

In any case, thanks for that Armbian link. I had checked there a while ago, but had not seen the newest posts. Appreciated.

shelldweller · 04-18-2021, 01:31 PM

Some interesting results on a few recent builds:

5.4.108 has an issue with fbturbo. If I start with a base image and build up to Xfce using only upstream packages, then I have no problems. However, if I start with an xfce image, then the screen goes blank before it gets to the login screen. If I remove (or rename, actually) the file /etc/X11/xorg.conf.d/XX-fbturbo.conf, then the blank screen problem goes away, and I can log in like normal. Xorg seems to work fine after that point.

I recall we moved to fbturbo when I was noticing a sluggish desktop. This issue wound up being the cpu-fequency governor (needing to be set to "performance"), and it had nothing to do with Xorg per se. Given the fact that it is crashing Xorg now, I think we might as well remove it from the build. I recall seeing something on the Armbian forum (or somewhere, cannot find it right now) about fbturbo causing crashes.

Here are the logs for the xfce-build. I did not collect logs for the base-image, because that one worked without issues.

https://shelldweller.beauxbead.com/s...turbo-5.4.108/

5.11.15 still has major CPU issues. The xfce image still does not boot past the generation of the ssh keys random art images. Here are the logs for the first half-boot from that image:

https://shelldweller.beauxbead.com/s.../xfce-5.11.15/

The base image gets much farther. It gets through the first two boots okay, and I can get to a login prompt like normal. I can even install a full system and reboot. However, startx just hangs. There is nothing beyond the console screen at all. Here are the logs for that one, in full:

https://shelldweller.beauxbead.com/s.../xfce-5.11.15/

So, on the plus side, the 5.4.y series is seeming very stable, especially if we remove fbturbo from the xfce build. On the downside, 5.11.y is still acting very strange. Just curious, should we try switching back to 5.10.y as the next branch? That one seemed much better on the Pinebook at least.

I am in the middle of a fresh legacy build, but I do not anticipate it acting any differently than the 5.4.108 I built a week or so ago, especially if fbturbo is still being installed; but you never know, that is what testing is for....

sndwvs · 04-18-2021, 02:33 PM

removed the x / xf86-video-fbturbo driver.
bug

Code:

[    6.638775] WARNING: CPU: 0 PID: 498 at drivers/clocksource/arm_arch_timer.c:364 sun50i_a64_read_cntpct_el0+0x30/0x40
[    6.638881] WARNING: CPU: 2 PID: 21 at drivers/clocksource/arm_arch_timer.c:364 sun50i_a64_read_cntpct_el0+0x30/0x40
[    6.641400] Modules linked in:

fix patches for kernel 5.4.y 5.10.y and 5.11.y are the same in fix-a64-timejump.patch
perhaps there is a conflict or lack of some kernel configuration. and update 5.4.109 -> 5.4.113

shelldweller · 05-02-2021, 07:59 AM

I have detected an OpenGL memory leak, but I have no idea how to fix it. I was doing some disk benchmarking on a 5.4.115 kernel image, and eventually Xorg crashed. The only log with any relevant error message that I could find was Xorg.0.log. I put that and the rest up here:

https://shelldweller.beauxbead.com/slack/logs/

In particular, there is this section of Xorg.0.log:

Code:

[ 22324.995] (WW) glamor: Failed to allocate 103x22 FBO due to GL_OUT_OF_MEMORY.
[ 22325.569] (WW) glamor: Expect reduced performance.
[ 95363.157] (EE) 
[ 95363.222] (EE) Backtrace:
[ 95363.360] (EE) 0: /usr/libexec/Xorg (OsLookupColor+0x188) [0x587f78]
[ 95363.360] (EE) unw_get_proc_info failed: no unwind info found [-10]
[ 95363.360] (EE) 
[ 95363.565] (EE) Segmentation fault at address 0x50
[ 95363.565] (EE) 
Fatal server error:
[ 95363.565] (EE) Caught signal 11 (Segmentation fault). Server aborting
[ 95363.566] (EE) 
[ 95363.640] (EE) 
Please consult the The X.Org Foundation support 
         at http://wiki.x.org
 for help. 
[ 95363.640] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[ 95363.640] (EE) 
[ 95364.196] (EE) Server terminated with error (1). Closing log file.

I tried searching around for solutions. It seems like this is usually hardware-specific though. Any ideas on how to avoid this in the future?

Much thanks, as always.

sndwvs · 05-02-2021, 08:06 AM

try to return fbturbo driver for now
or

Code:

echo "install lima /bin/false" > /etc/modprobe.d/blacklist-lima.conf

shelldweller · 05-02-2021, 09:26 AM

Quote:

Originally Posted by sndwvs

Code:

echo "install lima /bin/false" > /etc/modprobe.d/blacklist-lima.conf

I am trying the blacklist method for now, since it is the simplest to implement. So far, so good. First thing I noticed is that glxgears runs a lot smoother and faster this way, it is no longer bound to 60Hz. I have the benchmark running again. It takes around 20 hours, so we shall see if xorg holds out this time.

thank you.

sndwvs · 05-02-2021, 10:26 AM

restored the configuration of the modules when creating the image, when it was already there.

sndwvs · 05-05-2021, 12:33 AM

update kernel 5.12.1
slarm64-current-aarch64-base-pinebook-5.12.1-build-20210504.img.zst
slarm64-current-aarch64-base-pinebook-5.12.1-build-20210504.img.zst.sha256
slarm64-current-aarch64-xfce-pinebook-5.12.1-build-20210504.img.zst
slarm64-current-aarch64-xfce-pinebook-5.12.1-build-20210504.img.zst.sha256

shelldweller · 05-15-2021, 04:59 PM

Some observations on recent kernels:

5.12.y has the same sorts of issues that 5.11.y did. The xfce images do not get past the ssh key random art parts, and never boot completely to the login prompt. The server image seems okay at first, and then when you build it up, startx does nothing, and rebooting does not work either. No error messages show on the screen, it just hangs there without any output at all. I will try to generate some logs, but so far 5.12.y is not too exciting.

5.4.118 is great so far, and I have images up for download here. The only major fix I have to do after an install is to run

Code:

xset -dpms

in an Xorg session to keep the screen from going blank and staying blank. Otherwise, even the console blanking seems to have been fixed. And I have not crashed Xorg in a while, but I have not stressed it really hard either.

5.4.119 does not build. I am getting this during the kernel package creation stage:

Code:

lib/firmware/mellanox/
lib/firmware/mellanox/mlxsw_spectrum-13.1420.122.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1530.152.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1620.192.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1702.6.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1703.4.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1910.622.mfa2
/sbin/makepkg: line 416:  9693 Done                    find ./
      9694                       | LC_COLLATE=C sort
      9695                       | sed '2,$s,^\./,,'
      9696 Broken pipe             | tar --no-recursion $ACLS $XATTRS $MTIME -T - -cvf -
      9697 Killed                  | $COMPRESSOR > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
ERROR:  xz -9 --threads=2 -c returned error code 137 -- makepkg failed.

I have tried a few times with a clean directory. I keep getting the same error. This is my usual build machine, I have never seen this before. Any clues?

much thanks...

sndwvs · 05-15-2021, 05:22 PM

Let's see the version of pkgtools

Code:

ls /var/log/packages | grep pkgtools

You can also try instead

Code:

xset -dpms

Having created a file in /etc/X11/xorg.conf.d/10-monitor.conf

Code:

Section "Monitor"
	Identifier		"Monitor0"
	Option			"DPMS" "false"
EndSection
Section "ServerFlags"
	Option			"BlankTime" "0"
	Option			"StandbyTime" "0"
	Option			"SuspendTime" "0"
	Option			"OffTime" "0"
EndSection

shelldweller · 05-16-2021, 02:30 PM

Quote:

Originally Posted by shelldweller

5.4.119 does not build. I am getting this during the kernel package creation stage:

Code:

lib/firmware/mellanox/
lib/firmware/mellanox/mlxsw_spectrum-13.1420.122.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1530.152.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1620.192.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1702.6.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1703.4.mfa2
lib/firmware/mellanox/mlxsw_spectrum-13.1910.622.mfa2
/sbin/makepkg: line 416:  9693 Done                    find ./
      9694                       | LC_COLLATE=C sort
      9695                       | sed '2,$s,^\./,,'
      9696 Broken pipe             | tar --no-recursion $ACLS $XATTRS $MTIME -T - -cvf -
      9697 Killed                  | $COMPRESSOR > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
ERROR:  xz -9 --threads=2 -c returned error code 137 -- makepkg failed.

I have tried a few times with a clean directory. I keep getting the same error. This is my usual build machine, I have never seen this before. Any clues?

Never mind, it was an issue with my swap file. I created a new swap file, and the problem went away. Thanks for the consideration though.

And thanks for setting the DPMS default. That should make the pinebook images work as intended after a new install.

sndwvs · 09-20-2021, 09:57 AM

update kernel 5.14.6

slarm64-current-aarch64-server-pinebook-5.14.6-build-20210919.img.zst
slarm64-current-aarch64-server-pinebook-5.14.6-build-20210919.img.zst.sha256
slarm64-current-aarch64-xfce-pinebook-5.14.6-build-20210919.img.zst
slarm64-current-aarch64-xfce-pinebook-5.14.6-build-20210919.img.zst.sha256

shelldweller · 09-24-2021, 02:02 PM

Quote:

Originally Posted by sndwvs

update kernel 5.14.6

slarm64-current-aarch64-server-pinebook-5.14.6-build-20210919.img.zst
slarm64-current-aarch64-server-pinebook-5.14.6-build-20210919.img.zst.sha256
slarm64-current-aarch64-xfce-pinebook-5.14.6-build-20210919.img.zst
slarm64-current-aarch64-xfce-pinebook-5.14.6-build-20210919.img.zst.sha256

I tested these out, as well as some that I had built myself with the same kernel. The Pinebook-A64 has not worked with any next-kernel since 5.11.y. The symptoms have been more or less the same ever since then, and I have spent some time trying to debug what is going on with little success. Here is what I know:

The server images seem to work okay at first. Things go awry if/when you install and switch to NetworkManager at any point. From that point forward, booting and rebooting do not work. It seems to hang when "Stopping network interfaces...". Actually, the server images work a bit better than they used to, so if anyone needed a server-only image for the pinebook, the next-kernel might be an option. I just started with a server image and installed everything that is in an Xfce image except NetworkManager, and it seems to work okay in dhcp mode with a usb ethernet dongle at least.

Xfce images do not even get through the first boot sequence. It hangs after generating the ssh random art images, and right before the "devpts on /dev/pts type devpts" line and before the daemons are launched. This seems to be when it is activating network daemons.

Even with dhcpcd enabled, networking does not work on the Xfce image, although boot/reboot work. I chrooted into an Xfce image before booting it and changed the netconfig to dhcpcd, and it boots but there is still no networking. Everything else seems to work, including Bluetooth, which is interesting.

So, something is going on with NetworkManager being used with any kernel beyond 5.11.y, even when dhcpd is being used via netconfig. I checked out the latest images from Armbian (5.13.?) and Manjaro (5.14.? -> 5.15.0-rc) and they do not seem to have the networking problem, although I am detecting some strange audio problems on those images that are not present in the slarm64 images. I looked at a diff of the kernel config files for each distro, and there are way more differences than I expected, not just a few simple things to toggle and try. So, I am not sure where to begin. It seems like Pinebook-A64 needs a different set of patches and configs to work with the next kernels, but I am not sure what to tweak or where to source the new versions from.

I tried collecting some fresh logs:

pinebook-xfce-logs

pinebook-server-logs

In the mean time, the legacy kernel 5.4.y works fine, so I have just been using that. I recall 5.10.y working pretty well, so if/when we ever want to bump the legacy kernel up to 5.10.y, that might work out if necessary. Although, I recently tried building a 5.10.y image using some patches and configs borrowed from Armbian and some code repo archives, but it did not go all that well. The black screen problem came back, and I did not investigate beyond that. I probably overlooked something essential, because I know 5.10.y worked at one point in time.

I am happy to test any next-kernel images, however the legacy images are the only ones I have been building and uploading for some time now, since most people would want to use NetworkManager with a gui interface for the Pinebook I believe.

Thanks, as always.