First ever unexplained hardlock with Slackware in almost 20 years.

keithpeter · 10-15-2018, 03:27 PM

I recompiled the huge kernel using the .config file from slackware-14.2/source/k/config-x86_64/config-huge-4.4.14.x64 and then used make menuconfig to add the configurations shown below (from the resulting .config)

Code:

#
# Debug Lockups and Hangs
#
CONFIG_LOCKUP_DETECTOR=y
CONFIG_HARDLOCKUP_DETECTOR=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=1
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=1
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
CONFIG_BOOTPARAM_HUNG_TASK_PANIC=y
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=1
# CONFIG_PANIC_ON_OOPS is not set
CONFIG_PANIC_ON_OOPS_VALUE=0
CONFIG_PANIC_TIMEOUT=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHED_INFO=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_SCHED_STACK_END_CHECK is not set
# CONFIG_DEBUG_TIMEKEEPING is not set
# CONFIG_TIMER_STATS is not set

Newbie question: Should I have recompiled the kernel modules as well? They are still the ones installed by slackpkg.

I copied the resulting arch/x86/boot/bzImage file to /boot/vmlinuz-test (as well as System.map) and added the test kernel to lilo.conf

Rebooted, started X, plugged my external hard drive in, unmounted the hard drive and then moved the Thunar window round and triggered the freeze. Nothing happened, no reboots or anything. Left it for a good 10 minutes. Should I be looking for some kind of core dump?

Previously to recompiling, I booted into command line (no X) mounted the same external hard drive using the mount command and a mount point in my home drive, changed to user puggled about on the drive using mc and then umounted as root. Nothing happened, no freezes.

elcore · 10-16-2018, 03:05 AM

Quote:

Originally Posted by keithpeter

Rebooted, started X, plugged my external hard drive in, unmounted the hard drive and then moved the Thunar window round and triggered the freeze.

Well I can only tell what I suspect is going on there.

According to your log on previous page, xhci module has crashed. I don't have the (usb-3.0) hardware to confirm or deny that, also I don't normally mount ntfs.
Furthermore, ntfs-3g write support is known to be buggy, and it depends on FUSE which has caused some unrelated issues for me in the past.

I've read lockup-watchdogs.txt from kernel docs, it says kernel will panic or notify depending on how it's configured. So there must be something in dmesg.
If you tail dmesg and connect the drive, output should tell you something's wrong, also if you grep '/var/log/messages' it may reveal something about the drive.
Like, it may not be present in smartd database, and it's possible that eudev lacks information on the drive.
And since the crash is triggered by moving the window, it could be something completely unrelated, like a buggy GPU driver.

It may be the case where you lack proper firmware for the drive controller, since certain vendors only ship windows drivers for these things.
I've witnessed a case where the drive had to be disassembled and connected with sata cable, because the controller would not work with standard firmware that is shipped with the kernel.
Another thing to watch out for, is that Slackware kernel contains a lot of unstable "staging" drivers to support hardware which would otherwise just fail.

Anyway, If the exact same scenario works fine on older kernel, than it's either regression in new kernel, or the old userspace tool (ntfs-3g, udev, udisks, gvfs) depends on old kernel module.
If it's the latter, then I'd suggest a separate system for your use-case, where you can continue to use the old kernel until the matter is resolved upstream.
Personally, I depend on many outdated programs, so I build separate systems with no network stack. They're exploitable, have known vulnerabilities, and shouldn't be networked.
Where networking is needed I build updated, minimal kernels, these systems will not mount anything other than root fs, and won't load certain modules even if hardware is there.
Some might even call them useless systems, but I see some value in that even if nobody else can.

keithpeter · 10-16-2018, 04:35 AM

Quote:

Originally Posted by elcore

According to your log on previous page, xhci module has crashed. I don't have the (usb-3.0) hardware to confirm or deny that, also I don't normally mount ntfs.
Furthermore, ntfs-3g write support is known to be buggy, and it depends on FUSE which has caused some unrelated issues for me in the past.

Thanks for taking the time to write this detailed and informative reply. I've noticed that the freeze does not occur with vfat formatted small USB sticks. I think I might explore formatting an old 3.5 inch hard drive as ext4 and using that as a back up device via a caddy and see if I can still provoke a freeze. If not, then I use ext4 backups until Slackware 15 comes out. This PC is the one that sits in the corner with lots of files on and with an internet connection.

Below the logs when plugging in the ntfs formatted external drive which is USB3 it turns out (blanked the serial number of the drive out for general paranoia)...

Code:

dmesg -w

[  487.172267] usb 4-1: new SuperSpeed USB device number 2 using xhci_hcd
[  487.184157] usb 4-1: New USB device found, idVendor=1058, idProduct=10b8
[  487.184169] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=5
[  487.184173] usb 4-1: Product: Elements 10B8
[  487.184176] usb 4-1: Manufacturer: Western Digital
[  487.184178] usb 4-1: SerialNumber: 
[  487.528088] usb-storage 4-1:1.0: USB Mass Storage device detected
[  487.529019] scsi host8: usb-storage 4-1:1.0
[  487.529169] usbcore: registered new interface driver usb-storage
[  487.536570] usbcore: registered new interface driver uas
[  488.529632] scsi 8:0:0:0: Direct-Access     WD       Elements 10B8    1012 PQ: 0 ANSI: 6
[  488.530933] sd 8:0:0:0: [sdb] Spinning up disk...
[  489.532106] .....ready
[  493.539417] sd 8:0:0:0: [sdb] 976707584 512-byte logical blocks: (500 GB/466 GiB)
[  493.539696] sd 8:0:0:0: [sdb] Write Protect is off
[  493.539702] sd 8:0:0:0: [sdb] Mode Sense: 47 00 10 08
[  493.539967] sd 8:0:0:0: [sdb] No Caching mode page found
[  493.539973] sd 8:0:0:0: [sdb] Assuming drive cache: write through
[  493.637331]  sdb: sdb1
[  493.638697] sd 8:0:0:0: [sdb] Attached SCSI disk

and

Code:

root@espresso:~# tail -f /var/log/{messages,syslog}
==> /var/log/messages <==
Oct 16 10:19:09 espresso kernel: [  487.172267] usb 4-1: new SuperSpeed USB device number 2 using xhci_hcd
Oct 16 10:19:09 espresso kernel: [  487.184157] usb 4-1: New USB device found, idVendor=1058, idProduct=10b8
Oct 16 10:19:09 espresso kernel: [  487.184169] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=5
Oct 16 10:19:09 espresso kernel: [  487.184173] usb 4-1: Product: Elements 10B8
Oct 16 10:19:09 espresso kernel: [  487.184176] usb 4-1: Manufacturer: Western Digital
Oct 16 10:19:09 espresso kernel: [  487.184178] usb 4-1: SerialNumber: 
Oct 16 10:19:09 espresso mtp-probe: checking bus 4, device 2: "/sys/devices/pci0000:00/0000:00:14.0/usb4/4-1" 
Oct 16 10:19:09 espresso mtp-probe: bus: 4, device: 2 was not an MTP device 
Oct 16 10:19:09 espresso kernel: [  487.528088] usb-storage 4-1:1.0: USB Mass Storage device detected
Oct 16 10:19:09 espresso kernel: [  487.529019] scsi host8: usb-storage 4-1:1.0
Oct 16 10:19:09 espresso kernel: [  487.529169] usbcore: registered new interface driver usb-storage
Oct 16 10:19:09 espresso kernel: [  487.536570] usbcore: registered new interface driver uas
Oct 16 10:19:10 espresso kernel: [  488.529632] scsi 8:0:0:0: Direct-Access     WD       Elements 10B8    1012 PQ: 0 ANSI: 6
Oct 16 10:19:10 espresso kernel: [  488.530933] sd 8:0:0:0: [sdb] Spinning up disk...

==> /var/log/syslog <==
Oct 16 10:19:15 espresso kernel: [  489.532106] .....ready

==> /var/log/messages <==
Oct 16 10:19:15 espresso kernel: [  493.539417] sd 8:0:0:0: [sdb] 976707584 512-byte logical blocks: (500 GB/466 GiB)
Oct 16 10:19:15 espresso kernel: [  493.539696] sd 8:0:0:0: [sdb] Write Protect is off

==> /var/log/syslog <==
Oct 16 10:19:15 espresso kernel: [  493.539967] sd 8:0:0:0: [sdb] No Caching mode page found
Oct 16 10:19:15 espresso kernel: [  493.539973] sd 8:0:0:0: [sdb] Assuming drive cache: write through

==> /var/log/messages <==
Oct 16 10:19:15 espresso kernel: [  493.637331]  sdb: sdb1
Oct 16 10:19:15 espresso kernel: [  493.638697] sd 8:0:0:0: [sdb] Attached SCSI disk

elcore · 10-17-2018, 01:50 AM

This log looks fine to me, the kernel usually notes there's a 'quirk' if something's off with the disk.
I'd check with cfdisk if partition table is non-standard or corrupt, enforce a mbr with msdos partition, other than that I don't know.

Daedra · 10-17-2018, 08:10 PM

Hi all,

Just an update, I had to add some more music to my backup drive today and wanted to see if I could recreate the hard lock. Using my USB to SATA adapter I plugged in my 320gb 2.5 hard drive I use just for music backups. I then added the music, and then from caja I "safely remove drive" the hard drive. Everything went ok, but I remounted the drive and tried one more time to see If I could get the lock up and it happened. As soon as I tried to "safely remove drive" the second time the system hard locked. Not sure if this is caused by mate, the kernel, my drive, or the adapter.

keithpeter · 10-18-2018, 05:16 PM

Quote:

Originally Posted by Daedra

As soon as I tried to "safely remove drive" the second time the system hard locked.

That is what I see if I move a window while the (xfce4) notification that says 'data is being written to the drive...' is still up.

I have compiled the 4.18.14 kernel from kernel.org and have had no problems since.

Agba in post #26 found a bug report that looks relevant. My drive is USB3 and the front socket on my PC appears to be USB2.