[SOLVED] new Kubuntu 20.04 install locks up every day overnight

Red Squirrel · 04-27-2020, 10:01 PM

I'm hoping someone can help me figure this out, it's super frustrating.

Basically as title, just did a clean install of Kubuntu 20.04 and it works ok when I go to use it, but when I go to bed, next morning it will be completely locked up and I have to do a hard reset. Every single night. the times seem to differ it might do it as early as 9pm while I'm at work on night shift, or do it as late as 9am the next day if I'm still in bed. Basically if it's idle for any significant amount of time, it completely dies.

I've tried letting dmesg -w on a console to see if it spits anything out before it dies, but nothing so far. I'm open to suggestions on other things to try so I can at least get some kind of log or something.

Today at work I also decided to keep a SSH session open, as well as a X session of a console just to see if it's X/wayland that's locking up or the whole system. I lost the SSH connection so that confirms it's the whole system.

Hardware is as follows:

CPU: AMD Ryzen 3 1300X quad core
Motherboard: Asus Prime B350-plus
GPU: AMD Radeon 580
8GB of system ram

This hardware except for the GPU was used as a mining rig before with Ubuntu Server so I know it works fine unless there's some weird issue that's specific to Kubuntu 20.04.

This is probably not really enough info to troubleshoot this but hoping someone can point me in right direction of something I can try at least so I can get more info on what the system is doing at the time of the crash.

mrmazda · 04-28-2020, 03:16 AM

If you have persistent journal enabled, you can access the previous boot's journal after a hard reset to look for clues. It's not unusual to find things in the journal that never show up in dmesg.

Red Squirrel · 04-28-2020, 03:17 AM

Where would I look to find that?

mrmazda · 04-28-2020, 04:10 AM

Code:

apropos journal

Red Squirrel · 04-28-2020, 05:33 AM

Sorry that does not tell me much. it's redirecting me to a command called jfs_logdump but it wants a file.

I need to know what file I'm suppose to look at to see the journal you're talking about. Ex: something that would be in /var/log. That way I can do a tail -f on it and leave it up on the screen so I can see what happened right before it locked up.

JZL240I-U · 04-28-2020, 07:36 AM

How dead is dead? Poweroff? No mouse, no keyboard, fan(s) running or not... Please elaborate a little.

mrmazda · 04-28-2020, 09:48 AM

Quote:

Originally Posted by Red Squirrel

Sorry that does not tell me much. it's redirecting me to a command called jfs_logdump but it wants a file.

Code:

# apropos journal
journalctl (1)       - Query the systemd journal
journald.conf (5)    - Journal service configuration files
journald.conf.d (5)  - Journal service configuration files
named-journalprint (8) - print zone journal in human-readable form
sdjournal (1)        - Provide an interface to capture systemd journal entries.
systemd-cat (1)      - Connect a pipeline or program's output with the journal
systemd-journald (8) - Journal service
systemd-journald-audit.socket (8) - Journal service
systemd-journald-dev-log.socket (8) - Journal service
systemd-journald.service (8) - Journal service
systemd-journald.socket (8) - Journal service
systemd.journal-fields (7) - Special journal fields
# man journalctl
JOURNALCTL(1)                     journalctl                     JOURNALCTL(1)

NAME
       journalctl - Query the systemd journal

SYNOPSIS
       journalctl [OPTIONS...] [MATCHES...]

DESCRIPTION
       journalctl may be used to query the contents of the systemd(1) journal
       as written by systemd-journald.service(8).
...
# man journald.conf
JOURNALD.CONF(5)                 journald.conf                JOURNALD.CONF(5)

NAME
       journald.conf, journald.conf.d - Journal service configuration files

SYNOPSIS
       /etc/systemd/journald.conf

       /etc/systemd/journald.conf.d/*.conf

       /run/systemd/journald.conf.d/*.conf

       /usr/lib/systemd/journald.conf.d/*.conf
...

Quote:

I need to know what file I'm suppose to look at to see the journal you're talking about. Ex: something that would be in /var/log. That way I can do a tail -f on it and leave it up on the screen so I can see what happened right before it locked up.

With journalctl, you don't look at a file directly. The journalctl command opens the binary formatted journal according to the parameters you give it, typically for viewing it automatically using the less command. journdald.conf* describes how one officially configures the journal to be persistent, but I find simply

Code:

sudo mkdir /var/log/journal

is sufficient to enable it. Once persistence is enabled, try this after a forced reboot, for a start:

Code:

sudo journalctl -b -1 | grep -i failed

See also the -f option.

Red Squirrel · 04-28-2020, 04:24 PM

Quote:

Originally Posted by JZL240I-U

How dead is dead? Poweroff? No mouse, no keyboard, fan(s) running or not... Please elaborate a little.

Complete system lockup. Whatever was on the screen stays, but no mouse movement, no network, nothing. (ex: can't ping it from another machine)

I found info from 2017 that Linux does not seem to support Ryzen that well though... so maybe that's my issue. Lot of people getting similar issues including crashes. For me it does not crash it just completely locks up and only when it's idle at night.

I did find something about disabling "C States" in the bios, so I went ahead and did that.

I played around with that journal command but not finding anything significant. Basically moments before the last lockup it was just SSH activity from me logging in and out a few times. At one point I just lost my connection and that's when it locked up again.

there is some failures in journal around the time of a reboot but don't think it's anything that would cause this:

Code:

journalctl -b -1 | grep -i failed
Apr 28 07:17:23 falcon kernel: Initramfs unpacking failed: Decoding failed
Apr 28 07:17:25 falcon systemd[1]: Starting GRUB failed boot detection...
Apr 28 07:17:25 falcon systemd[1]: Finished GRUB failed boot detection.
Apr 28 07:17:25 falcon thermald[939]: I/O warning : failed to load external entity "/etc/thermald/thermal-conf.xml"
Apr 28 07:17:25 falcon thermald[939]: [ERR]THD engine start failed
Apr 28 07:17:26 falcon systemd[1]: vboxweb.service: Failed with result 'protocol'.
Apr 28 07:17:26 falcon systemd[1]: Failed to start VirtualBox Web Service.
Apr 28 07:17:26 falcon udisksd[941]: failed to load module mdraid: libbd_mdraid.so.2: cannot open shared object file: No such file or directory
Apr 28 07:17:26 falcon udisksd[941]: Failed to load the 'mdraid' libblockdev plugin
Apr 28 07:17:30 falcon kernel: vboxdrv: module verification failed: signature and/or required key missing - tainting kernel
Apr 28 07:17:45 falcon systemd[1227]: pulseaudio.service: Failed with result 'exit-code'.
Apr 28 07:17:45 falcon org.kde.kglobalaccel[1642]: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Apr 28 07:17:45 falcon dbus-daemon[1251]: [session uid=1000 pid=1251] Activated service 'org.kde.kglobalaccel' failed: Process org.kde.kglobalaccel received signal 6
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:46 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'rtkit-daemon.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:48 falcon systemd[1227]: obex.service: Failed with result 'exit-code'.
Apr 28 07:17:48 falcon dbus-daemon[902]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.nm-dispatcher.service': Refusing activation, D-Bus is shutting down.
Apr 28 07:17:48 falcon ifdown[1781]: run-parts: failed to stat component /etc/network/if-post-down.d/avahi-daemon: No such file or directory

This is interesting though... keep in mind this is around the time I rebooted this morning, it never locked up yet.

Code:

Apr 28 07:17:23 falcon kernel: RAS: Correctable Errors collector initialized.
Apr 28 07:17:23 falcon kernel: EXT4-fs (sda2): re-mounted. Opts: errors=remount-ro
Apr 28 07:17:25 falcon kernel: EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: errors=remount-ro
Apr 28 07:17:25 falcon systemd[1]: Condition check resulted in Process error reports when automatic reporting is enabled (file watch) being skipped.
Apr 28 07:17:25 falcon thermald[939]: [WARN]error: could not parse file /etc/thermald/thermal-conf.xml
Apr 28 07:17:30 falcon systemd-resolved[861]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Apr 28 07:17:30 falcon systemd-resolved[861]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Apr 28 07:17:32 falcon org.kde.KScreen[1435]: kscreen.xcb.helper: Event Error:  147
Apr 28 07:17:39 falcon systemd-resolved[861]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Apr 28 07:17:45 falcon sddm[1176]: Authentication error: "Process crashed"
Apr 28 07:17:45 falcon sddm[1176]: Authentication error: "Process crashed"
Apr 28 07:17:45 falcon org.kde.kglobalaccel[1336]: The X11 connection broke (error 1). Did the X11 server die?
Apr 28 07:17:45 falcon org.kde.KScreen[1435]: The X11 connection broke (error 1). Did the X11 server die?
Apr 28 07:17:45 falcon org.kde.ActivityManager[1394]: The X11 connection broke (error 1). Did the X11 server die?
Apr 28 07:17:45 falcon pulseaudio[1234]: ICE default IO error handler doing an exit(), pid = 1234, errno = 11

mrmazda · 04-28-2020, 10:40 PM

Does sources.list have updates and security repos enabled? Have you updated every day since the initial upgrade? Are any KDE or SDDM packages from 18.04 remaining installed? Do any orphans remain installed? Have you tried reconfiguring or disabling the screen locker? 20.04 is very new, and KDE is anything but *buntu's flagship, probably not beta tested so well as Gnome or some of the other *buntu sub-distro DEs.

Those sddm and connection broke errors seem to me to indicate either sddm and/or KDE are having a problem with X11. I suggest tonight logging out of KDE at bedtime to see if you can login in the AM. If fail, next night log out of KDE, log into IceWM, then goto bed and see what happens in the AM. IceWM is my goto for checking for whether problems are related to the DE (here, KDE/Plasma), the login manager (SDDM), or X itself. It consumes very little RAM or disk space.

Red Squirrel · 04-29-2020, 02:21 AM

So far so good tonight... still early to tell though, and I did open xclock remotely from work as a good way to monitor it, so I don't know if doing that counted as an interaction in the same way as if I was there physically. So it may have reset the "timer".

Will try the stuff you said if it does it again. I did update yesterday but there was only a few minor updates. Have not tried yet as I don't want to change anything in middle of the test.

TBH I did not realize how new this was until after I installed it, I just needed to upgrade since I changed my GPU and old system would not detect new one properly and I was due to update to a new OS anyway. I have a bunch of other less serious issues so hopefully these get solved over time too.

also, is this running X11 or Wayland? I had the impression it was Wayland but I do see lot of references to X11. Is Wayland something I need to explicitly enable? Wonder if that's something worth trying too.

mrmazda · 04-29-2020, 02:48 AM

Quote:

Originally Posted by Red Squirrel

also, is this running X11 or Wayland? I had the impression it was Wayland but I do see lot of references to X11. Is Wayland something I need to explicitly enable? Wonder if that's something worth trying too.

I've yet to try Wayland except by accident, so don't know. What does output from

Code:

inxi -Gxxx
inxi -V | head -n1

report when run from Konsole?

Red Squirrel · 04-30-2020, 02:33 PM

This is output:

Code:

ryan@falcon:~$ inxi -Gxxx
Graphics:  Device-1: Advanced Micro Devices [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] 
           vendor: Micro-Star MSI driver: amdgpu v: kernel bus ID: 07:00.0 chip ID: 1002:67df 
           Display: x11 server: X.Org 1.20.8 driver: amdgpu FAILED: ati unloaded: fbdev,modesetting,vesa compositor: kwin_x11 
           resolution: 3840x2160~60Hz 
           OpenGL: renderer: Radeon RX 580 Series (POLARIS10 DRM 3.35.0 5.4.0-26-generic LLVM 9.0.1) v: 4.6 Mesa 20.0.4 
           direct render: Yes 
ryan@falcon:~$ 
ryan@falcon:~$ 
ryan@falcon:~$ inxi -V | head -n1
inxi 3.0.38-00 (2020-03-14)
ryan@falcon:~$

So looks like it's X11 after all.

And so far so good with the lockups... going to give this a few more days before I consider it solved but I'm thinking the C state setting may have been the fix.

mrmazda · 04-30-2020, 02:53 PM

Quote:

Originally Posted by Red Squirrel

So looks like it's X11 after all.

Don't be too sure about that. Following success with amdgpu by "FAILED: ati" I think was a bug. -G got extensive fixes and improvements with the release of inxi v3.1.0 last week. Give it a try from the author if you can't get it with apt. It's just a script.

Red Squirrel · 05-07-2020, 04:33 AM

Never got a chance to do that script but does not really matter at this point since I think the C State setting was indeed the issue.

Been several days now without a lockup, so looks like it's solved!