[SOLVED] Kernel panic after having been unable to execute any command

Wario. · 01-28-2019, 03:12 PM

Dear Slackware community, I was trying learning more about the system...and the opportunity came by itself

Slackware 14.2 64 bit here, just properly configured (docs instructions - https://docs.slackware.com/slackware:beginners_guide).

Some minutes ago I run an upgrade: it ended and asked me for a #lilo update considering the new kernel image; I accepted but obtained "command not found". Quite strange...but then I remembered I have grub (oh

...but again "command not found". I opened another terminal, but it was unable to be called.

At that point, the system was unable to launch any command.
I did the mistake of (hard) rebooting...and have now a kernel panic (attached picture).

Does someone have a piece of advice about how can I fix the system?
Thanks and cheers, Wario

Ps: meanwhile I am searching for this "kernel panic" situation on the web, but a point was to highlight out how the system broke after the most recent upgrade. 99% my fault but...never say never.

giomat · 01-28-2019, 03:30 PM

You first need to chroot into your system using an usb or dvd installation disk. The one you used to install is good, just boot the computer from that and get a shell prompt, then follow instructions here slackware_admin:how_to_chroot_from_media.
If you are using lilo as bootloader, once you are in the chroot, execute "/sbin/lilo" (you probably didn't prepend /sbin/, that could be why it told you that the command does not exists). if you switched to a generic kernel during configuration make sure you have a new initrd generated as well before running lilo.
If you on the other hand use grub i think the command is something similar (like update-grub or grub-update).

individual · 01-28-2019, 04:24 PM

Quote:

Originally Posted by Wario.

Dear Slackware community, I was trying learning more about the system...and the opportunity came by itself

Slackware 14.2 64 bit here, just properly configured (docs instructions - https://docs.slackware.com/slackware:beginners_guide).

Some minutes ago I run an upgrade: it ended and asked me for a #lilo update considering the new kernel image; I accepted but obtained "command not found". Quite strange...but then I remembered I have grub (oh

...but again "command not found". I opened another terminal, but it was unable to be called.

At that point, the system was unable to launch any command.
I did the mistake of (hard) rebooting...and have now a kernel panic (attached picture).

Does someone have a piece of advice about how can I fix the system?
Thanks and cheers, Wario

Ps: meanwhile I am searching for this "kernel panic" situation on the web, but a point was to highlight out how the system broke after the most recent upgrade. 99% my fault but...never say never.

Don't worry, I did the same thing when I first started using Slackware. As giomat said, use your Slackware install media to chroot into your system.

EDIT: Sorry, I forgot to add this since you likely didn't do it before restarting. You'll need to generate a new initrd.

You will probably need to do the following since you have grub:

Code:

# mount /dev/sda# [use whatever root is on] /mnt
# mount /dev/sda# [use whatever boot is on] /mnt/boot
# mount /dev/sda# [use this if you have efi] /mnt/boot/efi
# mount --rbind /sys /mnt/sys
# mount --make-rslave /mnt/sys
# mount --rbind /dev /mnt/dev
# mount --make-rslave /mnt/dev
# mount --rbind /proc /mnt/proc
# mount --make-rslave /mnt/proc
# chroot /mnt /bin/sh
[chrooted]# /usr/share/mkinitrd/mkinitrd_command_generator.sh | sh
[chrooted]# grub-mkconfig -o /boot/grub/grub.cfg
[chrooted]# exit
# umount -l /mnt
# reboot

Richard Cranium · 01-28-2019, 11:05 PM

Ooh. I didn't know about the --make-rslave option to the mount command. Thanks @individual!

EDIT: If you use grub2, you might be interested in this.

enorbet · 01-29-2019, 06:22 AM

...and this is exactly why automated system updates are risky business. IMHO It's just easier to do them manually.

Wario. · 02-01-2019, 11:52 AM

Hello, thanks for the suggestions.
I'm trying with my Lubuntu64 live, but unfortunately, I obtain:

Code:

chroot: failed to run command '/bin/sh/': No such file or directory

(but of course, the file exists). Maybe this post might help:

https://unix.stackexchange.com/quest...e-or-directory

But executing:

Code:

cp -a /usr rootfs/ 
cp -a /lib rootfs/
cp -a /lib64 rootfs/

does not solve the problem. Can it maybe be possible, that during the updates I messed up with the right architecture libraries?

Thanks for your help.

hazel · 02-01-2019, 12:04 PM

How did you mount it? If it's user-mountable, it will mount with noexec, meaning that executables like sh cannot be run. In that case you will need to mount it as root.

Gordie · 02-01-2019, 12:05 PM

The original commands after you updated should have been run with root access.

Try, as was suggested [post #2], to use the Slackware Install Media to chroot into your system and then run the commands

Wario. · 02-01-2019, 01:00 PM

My apologies for having been imprecise. All the listed commands are now intended under the Slackware64 boot disk (14.2, default kernel).
The partition to restore is located in /dev/sda3. I use grub2 (no efi, classical bios) but to run lilo would be good anyway.

First attempt:

Code:

# mount /dev/sda3 /mnt
# mount -o bind /dev /mnt/dev
# mount -o bind /proc /mnt/proc
# mount -o bind /sys /mnt/sys
# chroot /mnt /bin/sh
chroot: can't execute '/bin/sh': No such file or directory

Second attempt (after rebooting, just for being completely sure):

Code:

# mount /dev/sda3 /mnt
# mount --rbind /sys /mnt/sys
# mount --make-rslave /mnt/sys
# mount --rbind /dev /mnt/dev
# mount --make-rslave /mnt/dev
# mount --rbind /proc /mnt/proc
# mount --make-rslave /mnt/proc
# chroot /mnt /bin/sh
chroot: can't execute '/bin/sh': No such file or directory

Finally, in both cases, after the error message, I continued with:

Code:

# cp -a /usr rootfs/
# cp -a /lib rootfs/
# cp -a /lib64 rootfs/
# chroot /mnt /bin/sh
chroot: can't execute '/bin/sh': No such file or directory

I hope my error-debugging to be better now

Richard Cranium · 02-01-2019, 02:46 PM

You don't have a /boot partition?

After you mount /dev/sda3 on /mnt, what do you see when you run

Code:

ls /mnt

?

bassmadrigal · 02-02-2019, 12:07 AM

I've never added /bin/sh after my chroot command. Is that expecting to run the host's /bin/sh or the chroot's /bin/sh?

Either way, I've been doing this for a long time and have never needed that portion of the command. After running the commands in your first code block, I simply run:

Code:

chroot /mnt

and it works fine.

Petri Kaukasoina · 02-02-2019, 02:39 AM

The original problem was that no command executed. chroot runs commands in the non-working installation (chroot /mnt runs /bin/sh under the chroot, too). Why would commands execute now?

Maybe a 64-bit system was upgraded with 32-bit patches or vice versa. Or a 14.2-current mismatch.

I suggest that after mounting the partition at /mnt, try 'ls -lrt /mnt/var/log/packages/' to see what the latest installed packages were: do the file names contain portions -x86_64- and _slack14.2 ? Like bash-4.3.048-x86_64-1_slack14.2.

What is the output of this command: ls /mnt/var/log/packages/glibc-solibs*

(Should be /mnt/var/log/packages/glibc-solibs-2.23-x86_64-4_slack14.2 or /mnt/var/log/packages/glibc-solibs-2.23-x86_64-1 if you have a 64-bit 14.2 installation.)

By the way, the photo you attached reports kernel 4.14.12. It's odd as Slackware 14.2 has a 4.4 kernel. (And current uses a 4.19 kernel). Maybe you were booting another OS, not Slackware? (The -current kernel was at 4.14.12 in January, 2018.)

Wario. · 02-02-2019, 04:23 AM

Dear @Petri, I was asking myself the same question.

On my hdd I only have Slackware (sda1 win7, sda2 is for backup data), but I installed it a long time ago from the liveDVD with the MATE Desktop environment. Could it explain the different kernel versions?

Here is the glibc output:

Code:

# ls /mnt/var/log/packages/glibc-solibs*
/mnt/var/log/packages/glibc-solibs-2.23-x86_64-4_slack14.2
/mnt/var/log/packages/glibc-solibs-2.26-x86_64-3-upgraded-2019-01-28,22:33:01

While the output for

Code:

# ls -lrt /mnt/var/log/packages | less

is in the attached picture (edit: now I know less to be superfluous).

Honestly, it looks probably a bit strange: my previous system update was one year ago (Jan 2018) - right, while this summer I installed some other packages like ripperX and Skype - as correctly shown. But I am surprised to see so few lines about the fatal update (Jan 28)...

Let me point out again how just typing

Code:

$ /bin/bash

did not work on the original system, too! ("no such file or directory" error).
Trying anyway to follow the suggestion about chroot, just typing it (after the mount operations previously listed):

Code:

# chroot /mnt
chroot: can't execute '/bin/sh': No such file or directory

Finally, yes, I have the "/boot" partition under "/mnt" and I realized to have forgotten during the mount operation.
Do you mind repeating again how should I mount it?

EDIT: more information about the way in which I updated: in "/etc/slackpkg/mirrors" the only non-commented line is under the section Slackware64-14.2 (so not under Slackware64-current - mistake here?) and the mirror that I used was "http://ftp.gwdg.de/pub/linux/slackware/slackware64-14.2" which is surely different from what I used one year ago (now I am temporarily in Germany). Then I literally run the commands explained in the doc-section.

Thanks and have a nice weekend!

Petri Kaukasoina · 02-02-2019, 04:55 AM

You didn't have a 14.2 installation, but a -current installation from Jan, 2018, which was much newer that 14.2 (from 2016). But now you started a slackpkg upgrade process for 14.2 (wrong mirror URL in /etc/slackpkg/mirrors) which tried to downgrade everything, like glibc from 2.26 to 2.23. But your -current binaries were linked against glibc-2.26, so nothing worked with glibc-2.23. Slackpkg first upgrades slackpkg, pkgtools, glibs-solibs before other packages, so it stopped at glibc-solibs, because no command worked after that. I guess you can fix it by installing glibc-solibs from current. Get glibc-solibs-2.29-x86_64-2.txz from your favourite mirror and copy it to some medium and installpkg --root /mnt /some_medium/glibc-solibs-2.29-x86_64-2.txz (the slackware partition and /some_medium should be mounted at this moment, of course).

Wario. · 02-02-2019, 06:06 AM

Wow, thanks for the comment, now I completely understand my mistake.
I installed glibc as suggested, having now the following problem:

Code:

# mount /dev/sda3 /mnt
# mount -o bind /dev /mnt/dev
# mount -o bind /proc /mnt/proc
# mount -o bind /sys /mnt/sys
# chroot /mnt
# /usr/share/mkinitrd/mkinitrd_command_generator.sh | sh
Modules for kernel 4.4.14 aren't installed.

Do I need to install another specific package?