[SOLVED] Random failing boot Debian 12

Paobee · 02-23-2024, 10:53 AM

My problem here is that this failing boot happens randomly, not "every" boot. It happened after some time after I changed the etc/fstab file, adding an entry to automount a second partition of my SSD just for data (/media/paobee/Dati). When the boot fails it goes in emergency mode, in a kind of loop. If I force the power off and reboot after 1 or 2 attempts the system boot properly. Thanks for any help.

Message on the emergency mode (copied from screen):

Quote:

Cannot open access to console, the root account is locked.
See sulogin(8) man page for more details.

Press Enter to continue.

Reloading system manager configuration
Starting default.target
[ 65.649045] /dev/nvme1n1p1: Can't open blockdev
[FAILED] Failed to mount media-paobee-Dati.mount - /media/paobee/Dati.
[DEPEND] Dependency failed for local-fs.target - Local File Systems.
You are in emenrgency mode. After logging in, type "journalctl -xb" to view
system logs, "Systemctl reboot" to reboot, "systemctl default" or "exit" to boot into default mode.

Cannot open access to console, the root account is locked.
See sulogin(8) man page for more details.

Press Enter to continue.

My fstab file now:

Code:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
#proc /proc proc nodev,noexec,nosuid 0  0
/dev/nvme0n1p2 / ext4 noatime,errors=remount-ro 0 1
/dev/nvme0n1p3 none swap sw 0 0
/dev/nvme1n1p1 /media/paobee/Dati ext4 noatime,defaults 0 0
#/dev/nvme0n1p1 /boot/efi vfat umask=0077 0 1
UUID=8820-1B74  /boot/efi       vfat    defaults      0       1

Output of my lsblk command:

Code:

paobee@debian-laptop:~$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0         7:0    0     4K  1 loop /snap/bare/5
loop1         7:1    0 105,4M  1 loop /snap/core/16574
loop2         7:2    0  55,7M  1 loop /snap/core18/2812
loop3         7:3    0 102,2M  1 loop /snap/easy-installer/34
loop4         7:4    0 218,4M  1 loop /snap/gnome-3-34-1804/93
loop5         7:5    0  91,7M  1 loop /snap/gtk-common-themes/1535
nvme0n1     259:0    0 238,5G  0 disk 
├─nvme0n1p1 259:1    0   300M  0 part /boot/efi
├─nvme0n1p2 259:2    0 229,4G  0 part /
└─nvme0n1p3 259:3    0   8,8G  0 part [SWAP]
nvme1n1     259:4    0 238,5G  0 disk 
└─nvme1n1p1 259:5    0 238,5G  0 part /media/paobee/Dati

Thank you for any help

michaelk · 02-23-2024, 06:25 PM

nvme0n1 = first drive discovered on controller 0
nvme1n1 - first drive discovered on controller 1

It appears you have two different devices with the second maybe experiencing a hardware problem. Since it just a data drive you could either comment out the line or add the noauto option in your /etc/fstab file so the system does not try to mount the drive a boot time.

Try running smartctl (install the smartmontools package if necessary) to see if the drive itself has errors or maybe just worn out.

syg00 · 02-23-2024, 09:52 PM

I'd be more inclined to blame the /media mount - don't do that, use a real mountpoint that is known to exist when needed. Back in the day /mnt used to be quite handy, but the fun police keep bashing its use these days.

Paobee · 02-24-2024, 04:32 AM

Quote:

Originally Posted by michaelk

It appears you have two different devices with the second maybe experiencing a hardware problem. Since it just a data drive you could either comment out the line or add the noauto option in your /etc/fstab file so the system does not try to mount the drive a boot time.
Try running smartctl (install the smartmontools package if necessary) to see if the drive itself has errors or maybe just worn out.

Thanks for your answer michaelk. The laptop is new and so are the 2 Kioxia SSD, so I really hope you wrong about that

But yes I installed & used already smartctl for that SSD, here's the result. For what I can see there are no special errors, but who knows, can you give a look?

Code:

sudo smartctl -a /dev/nvme1n1
[sudo] password di paobee: 
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.5.0-0.deb12.4-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       KBG50ZNV256G KIOXIA
Serial Number:                      83UPHDZQQH4U
Firmware Version:                   HP02AN00
PCI Vendor/Subsystem ID:            0x1e0f
IEEE OUI Identifier:                0x8ce38e
Total NVM Capacity:                 256.060.514.304 [256 GB]
Unallocated NVM Capacity:           0
Controller ID:                      0
NVMe Version:                       1.4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          256.060.514.304 [256 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            8ce38e 040465ed2d
Local Time is:                      Fri Feb 23 18:34:54 2024 CET
Firmware Updates (0x14):            2 Slots, no Reset required
Optional Admin Commands (0x001f):   Security Format Frmw_DL NS_Mngmt Self_Test
Optional NVM Commands (0x00df):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp Verify
Log Page Attributes (0x0e):         Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     79 Celsius
Critical Comp. Temp. Threshold:     84 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     4.00W       -        -    0  0  0  0        1       1
 1 +     3.00W       -        -    1  1  1  1        1       1
 2 +     1.90W       -        -    2  2  2  2        1       1
 3 -   0.0500W       -        -    3  3  3  3      800    1000
 4 -   0.0030W       -        -    4  4  4  4     3000   32000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         1

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        27 Celsius
Available Spare:                    100%
Available Spare Threshold:          5%
Percentage Used:                    0%
Data Units Read:                    74.864 [38,3 GB]
Data Units Written:                 122.622 [62,7 GB]
Host Read Commands:                 2.054.784
Host Write Commands:                351.860
Controller Busy Time:               3
Power Cycles:                       140
Power On Hours:                     279
Unsafe Shutdowns:                   37
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               27 Celsius

Error Information (NVMe Log 0x01, 16 of 256 entries)
No Errors Logged

Thank you

Paobee · 02-24-2024, 04:39 AM

Quote:

Originally Posted by syg00

I'd be more inclined to blame the /media mount - don't do that, use a real mountpoint that is known to exist when needed. Back in the day /mnt used to be quite handy, but the fun police keep bashing its use these days.

Thank you so much syg00, I haven't thought about that, it could explain the random behavior maybe, if the failing hardware (SSD is new) could be discarded... I'm not so great at mountpoints, would you please suggest a safer alternative to /media mountpoint (it was suggested by the system if I recall, but not sure). Better to create a ssd directory, and where? Thank you, sorry for the maybe stupid question _/\_

michaelk · 02-24-2024, 07:51 AM

Check the journal as suggested above.
You might try switching from device IDs to UUIDs in the fstab file like the efi partition.

Paobee · 02-25-2024, 02:13 AM

Thanks again Michaelk. UPDATE:
1. I didn't change (yet) the SSD mountpoint - was waiting for some further suggestion by syg00.
2. I followed the suggestion by Michaelk and replaced the filesystem name with its own UUID, taken from blkid. First boot ok.
3. I then had ANOTHER boot issue - this happening random too (getting crazy about that, not easy to understand for me):
the boot goes on without error messages, then at the Debian splash screen - just before the user pw screen - the boot freeze... only ctrl+alt+canc forced the reboot. Usually the reboot works.

This morning no problem and normal boot. I'll try more reboot now.
I went with the journalctl command to look for the failed boot. Even if maybe not present (!) into the boot list output ( journalctl --list-boots ), I think I found it on a longer log, using also the timestamp reference. I can't understand a real error message there tho, (except some usual error message that the system autofix at every boot). I think I found the moment the screen froze, I attach below the code just before and after the freeze, where I just left a blank space (line) corresponding to a pause where I waited for something to happen, at the 19:46:52 timestamp. I can't understand the issue, I just can guess the system was running the bluetooth services:

Code:

feb 24 19:46:51 debian-laptop systemd[1]: modprobe@loop.service: Deactivated successfully.
feb 24 19:46:51 debian-laptop systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
feb 24 19:46:51 debian-laptop systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
feb 24 19:46:51 debian-laptop systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
feb 24 19:46:51 debian-laptop systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
feb 24 19:46:51 debian-laptop systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
feb 24 19:46:51 debian-laptop snapd-apparmor[1041]: main.go:124: Loading profiles [/var/lib/snapd/apparmor/profiles/snap-confine.core.16574 /var/lib/snapd/apparmor/profiles/snap-update-ns.core /var/li>
feb 24 19:46:51 debian-laptop audit[1205]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap-update-ns.core" pid=1205 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1204]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="/snap/core/16574/usr/lib/snapd/snap-confine" pid=1204 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1204]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="/snap/core/16574/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=12>
feb 24 19:46:51 debian-laptop audit[1207]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.core.hook.configure" pid=1207 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1209]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.easy-installer.hook.configure" pid=1209 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1206]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap-update-ns.easy-installer" pid=1206 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1208]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.easy-installer.easy-installer" pid=1208 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop systemd[1]: Finished snapd.apparmor.service - Load AppArmor profiles managed internally by snapd.
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Waiting for firmware download to complete
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Firmware loaded in 1626177 usecs
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Waiting for device to boot
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Device booted in 15164 usecs
feb 24 19:46:52 debian-laptop kernel: bluetooth hci0: firmware: direct-loading firmware intel/ibt-19-0-4.ddc
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Applying Intel DDC parameters completed
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Firmware revision 0.4 build 15 week 45 2022
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: HCI LE Coded PHY feature bit is set, but its usage is not supported.
feb 24 19:46:51 debian-laptop systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
feb 24 19:46:51 debian-laptop systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
feb 24 19:46:51 debian-laptop systemd[1]: modprobe@loop.service: Deactivated successfully.
feb 24 19:46:51 debian-laptop systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
feb 24 19:46:51 debian-laptop systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
feb 24 19:46:51 debian-laptop systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
feb 24 19:46:51 debian-laptop systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
feb 24 19:46:51 debian-laptop systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
feb 24 19:46:51 debian-laptop snapd-apparmor[1041]: main.go:124: Loading profiles [/var/lib/snapd/apparmor/profiles/snap-confine.core.16574 /var/lib/snapd/apparmor/profiles/snap-update-ns.core /var/li>
feb 24 19:46:51 debian-laptop audit[1205]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap-update-ns.core" pid=1205 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1204]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="/snap/core/16574/usr/lib/snapd/snap-confine" pid=1204 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1204]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="/snap/core/16574/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=12>
feb 24 19:46:51 debian-laptop audit[1207]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.core.hook.configure" pid=1207 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1209]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.easy-installer.hook.configure" pid=1209 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1206]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap-update-ns.easy-installer" pid=1206 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop audit[1208]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.easy-installer.easy-installer" pid=1208 comm="apparmor_parser"
feb 24 19:46:51 debian-laptop systemd[1]: Finished snapd.apparmor.service - Load AppArmor profiles managed internally by snapd.
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Waiting for firmware download to complete
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Firmware loaded in 1626177 usecs
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Waiting for device to boot
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Device booted in 15164 usecs
feb 24 19:46:52 debian-laptop kernel: bluetooth hci0: firmware: direct-loading firmware intel/ibt-19-0-4.ddc
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-19-0-4.ddc
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Applying Intel DDC parameters completed
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: Firmware revision 0.4 build 15 week 45 2022
feb 24 19:46:52 debian-laptop kernel: Bluetooth: hci0: HCI LE Coded PHY feature bit is set, but its usage is not supported.

feb 24 19:47:20 debian-laptop systemd[1]: systemd-fsckd.service: Deactivated successfully.
feb 24 19:48:17 debian-laptop systemd[1]: Received SIGINT.
feb 24 19:48:17 debian-laptop systemd[1]: Activating special unit reboot.target...
feb 24 19:48:17 debian-laptop systemd[1]: Unnecessary job was removed for dev-nvme0n1p3.device - /dev/nvme0n1p3.
feb 24 19:48:17 debian-laptop systemd[1]: Removed slice system-getty.slice - Slice /system/getty.
feb 24 19:48:17 debian-laptop systemd[1]: Removed slice system-modprobe.slice - Slice /system/modprobe.
feb 24 19:48:17 debian-laptop systemd[1]: Stopping systemd-random-seed.service - Load/Save Random Seed...
feb 24 19:48:17 debian-laptop systemd[1]: Starting plymouth-reboot.service - Show Plymouth Reboot Screen...
feb 24 19:48:17 debian-laptop systemd[1]: Stopped target time-set.target - System Time Set.
feb 24 19:48:17 debian-laptop systemd[1]: Stopped target nss-user-lookup.target - User and Group Name Lookups.
feb 24 19:48:17 debian-laptop systemd[1]: Stopping networking.service - Raise network interfaces...
feb 24 19:48:17 debian-laptop systemd[1]: Stopped target remote-fs.target - Remote File Systems.
feb 24 19:48:17 debian-laptop systemd[1]: systemd-ask-password-plymouth.path: Deactivated successfully.
feb 24 19:48:17 debian-laptop systemd[1]: Stopped systemd-ask-password-plymouth.path - Forward Password Requests to Plymouth Directory Watch.
feb 24 19:48:17 debian-laptop systemd[1]: Stopped target slices.target - Slice Units.
feb 24 19:48:17 debian-laptop systemd[1]: Removed slice user.slice - User and Session Slice.
feb 24 19:48:17 debian-laptop systemd[1]: Stopped target cryptsetup.target - Local Encrypted Volumes.
feb 24 19:48:17 debian-laptop systemd[1]: systemd-ask-password-wall.path: Deactivated successfully.

Thanks in advance to expert users if they can understand and help me out. This random failing boot is becoming really weird to me.
I mean, considered the laptop is new, the Debian 12 installation went well (Calamares simplified install) the correct upgrade to 12.5 should be ok. The only thing, I allowed the backports repository and the system is hence running kernel 6.5.0 and not the 6.1.0, default for the stable.
Thank you again for your patience.

Paobee · 02-26-2024, 10:37 AM

UPDATE
Quick update. I kept having issues with failed boots like the one up here. I couldn't really understand from the journal logs which was the problem, I don't have the actual expertise to get that, probably. So I thought that maybe the more recent kernel could be behind all that. Also because the apparent cause of the issue seemed to change at every boot.
I kinda *downgraded from the 6.5.0 kernel (coming from backports repository) to the 6.1.0-17* (I think is the second-to-last for the 12.5) stable version, keeping the previous 6.1.0-15 as possible backup. I also purged the 6.5.0 and disabled the backports repositories. Ran apt update && upgrade and change the default kernel for Grub2.
Now everything seems to be wonderfully fixed, and no boot issues or SSD mounting problems.
I'm now performing several reboots without any failure.
So thanks to anyone who answered and gave their help, and I hope this could be helpful to someone in the same condition.
Cheers