Kali does not boot - Firmware mess-up after an upgrade - after a harddisk migration.
DebianThis forum is for the discussion of Debian Linux.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Kali does not boot - Firmware mess-up after an upgrade - after a harddisk migration.
Hi all,
So I've been happily using my linux distro for one year. I've used kali in the past in 2013[Edited], and because I felt comfortable with it I completely switched to that as my main OS that I use daily. And I've been on it for 1 year.
First 8-9 months I've used it on an external SSD drive. Just for a test ride to figure out all it's capabilities and all I can do to not look back to my old os. It went well so I decided to buy an m.2 drive and I used clonezilla to migrate the disk from my SSD to m.2
That's when I messed up the first time. I cloned the entire disk rather than only cloning the user files and the system. I cloned EFI and Swap as well somehow .
Then I found a solution to be able to boot that was involving changing the UUID's of partitions on the bootloader. So I did that. And it worked. I logged in my full fledged os. Which worked a couple of months more.
I'm using the Grub Bootloader just because I still have a windows hdd in my setup that I use to connect to my company for work.
Then one day I just sudo apt upgrade. Somehow the UUID's got messed up again. I tried the same trick but it didn't work. First it complained about the UUID's then when I solved it it said that my firmware is bad. Then I've tried the following.
*I've tried to restore the broken firmware by downloading and replacing the firmware that it's complaining, but each time it want's me to correct a new file in the firmware.
*I've tried to update upgrade, didn't work.
*I've tried to update the firmware forcefully however it didn't work either.
*I've tried to format the hdd completely and reinstall from scratch it didn't work.
*I've tried to format the hdd by completely clean it with clean all with diskpart in windows. didn't work
*I've changed kali installer versions didn't work.
*I've also tried to install Debian, which worked perfectly. however I couldn't install nvidia drivers on it again because of the messed up firmware.
So my question is...
Why is the firmware cannot reset with a format? Where does it exactly live? And how do I purge it and fix my hdd to be able to accomodate Kali linux once again?
Sorry for the huge backstory, I just wanted to tell how I got to this stage, that might give some people where and how I might have screwed up.
I've used kali in the past about 20 years ago, and because I felt comfortable with it I completely switched to that as my main OS that I use daily. And I've been on it for 1 year.
That would be a neat trick as Kali first released in 2013
Quote:
Originally Posted by Atimen
Why is the firmware cannot reset with a format? Where does it exactly live?
Firmware generally lives in an imbedded area within a device / controller. As you've given no useful information about what "firmware" or any examples of error messages you're receiving there's not much anyone can do.
Try providing more detailed information, logs, messages, even screenshots rather than vague "it's firmware"
So I've been happily using my linux distro for one year. I've used kali in the past about 20 years ago, and because I felt comfortable with it I completely switched to that as my main OS that I use daily. And I've been on it for 1 year.
The fact of the matter is, however, that Kali is a Linux distribution specifically geared towards professional penetration testers and security specialists, and given its unique nature, it is NOT a recommended distribution if you’re unfamiliar with Linux or are looking for a general-purpose Linux desktop distribution for development, web design, gaming, etc.
Quote:
Originally Posted by Atimen
Then one day I just sudo apt upgrade. Somehow the UUID's got messed up again. I tried the same trick but it didn't work.
apt upgrade cannot and will not change uuids.
Quote:
Originally Posted by Atimen
First it complained about the UUID's then when I solved it it said that my firmware is bad. Then I've tried the following.
*I've tried to restore the broken firmware by downloading and replacing the firmware that it's complaining, but each time it want's me to correct a new file in the firmware.
Exactly what kind of firmware is it, what did you try?
Quote:
Originally Posted by Atimen
*I've tried to update upgrade, didn't work.
*I've tried to update the firmware forcefully however it didn't work either.
*I've tried to format the hdd completely and reinstall from scratch it didn't work.
*I've tried to format the hdd by completely clean it with clean all with diskpart in windows. didn't work
*I've changed kali installer versions didn't work.
That would be a neat trick as Kali first released in 2013
I might have mixed up the years a bit, sorry
Quote:
How is it possible? what kind of firmware could survive a full OS reinstall?
So yeah! that was the exact question that surprised me. Each time even I do a full wipe, when do an OS reinstall, I continue where I left off my journey, the bootloader complains about the exact part of firmware that I left fixing.
I first thought it was somehow protected from rewrite and it would be protected by all diskpart commands somehow. Then I discarded the possibility of it being there, so maybe rather I am breaking the installation files in the kali img flash. To test that I just created another kali img and still it kept coming up with the same complaint.
Returning back to my initial thought where somehow the firmware survived, and I created a secondary partition in the disk so I can start slicing it to necessary partitions, and somehow it fixed the issue, but when I did a sudo apt upgrade, it broke the firmware again.
Quote:
apt upgrade cannot and will not change uuids.
Yep apt upgrade doesn't change my UUID list of targets in /etc/fstab. However with each apt upgrade, I'm breaking something about UUID's probably linking some stuff back to those targets. Feels like I'm fixing some target that is not there, so it searches for a firmware in the wrong path. I might be completely off. But I can definitely say that these started manifesting after I did a clone disk and changed the UUID's manually in /etc/fstab myself. Which is probably something that I shouldn't have done.
Quote:
Exactly what kind of firmware is it, what did you try?
I'm exactly not sure what the firmware is in this context. Is it a hdd firmware? is it an OS firmware, or individual device firmware. However I'll post all the details now about the error and where the complained files are at,
However if you could guide me to how I can give more info about the:
-Firmware path info and where system thinks it lives. I can send lshw & lsblk output and will send them in this thread.
-Boot log so that I can pass the file here.
Quote:
Din't work does not mean anything or may mean anything. Without details telling us "didn't work" is pointless.
Sorry I was vague there. I meant it exactly resumed from where I left of fixing the firmware files with wget one by one once it complains about them. I also checked if I was maybe messing up with the live disk's firmware but I confirmed that I was operating on my nvme0n1
I'm really specific about the choice that I made, Kali somehow with it's account management defaults, zsh, all the tools that I want to explore, attracts me to kali.
It feels like I'm in the right place. I also work on some projects regarding machine learning, and pipeline, and on the side and I also would like to explore some gpu tools in Kali which I haven't tried out previously, which is why I need a complete passthrough on gpu. That I think diminishes the virtual machine possibility on Kali. At least free solutions. I could have used mint, but I wanted to at least challange myself that Kali could run as a main OS if needed. And my workplace also uses Mint so it was to actually not feel like I'm working
Thank you for all your patience all, I also receive tons of tickets in a day without any tangible info and I've posted one without those info regardless. I'll get back with more info as much as I can do tonight.
Also I was expecting that I'm missing a really basic step where maybe I need to delete the firmware by hand or force install it and it would work, However that might not be the case so sorry again. I'll supply more data to go on.
So when I tried to boot again, this time I think I've addressed all the problematic firmware files so it only prompts me this, which never prevented me booting the system. But now it hangs there.
Then I tried to get into recovery mode to get some details of my system, but this time, I wasn't able to, and there seems to be 2 different versions there as shown.
I tried both recovery modes there and I'm putting the screen capture of both down below.
It doesn't let me enter the command line. So I couldn't get the info that we may need.
I don't have any essential data and I have my backups so I can start over if that is necessary.
Back when my chromebook came with Kali installed. They had 2 type of upgrade. A real big one that grabs all the tools for pen testing.
A smaller one if not wanting to pull in everything.
Thanks rokytnji! I'll give it a go and see if it fixes things. I didn't install the extended tools, I was planning to do so after booting. Meanwhile I tried to install it on windows with wsl2 and rdm and I will also try to do the GPU passthrough so that I can utilize it. But I will definitely try to fix the main os as it's the most convenient way of it working.
I'm really specific about the choice that I made, Kali somehow with it's account management defaults, zsh, all the tools that I want to explore, attracts me to kali.
I'm sorry, it is definitely wrong. Kali doesn't aim to attract users with zsh or account management. It's like a hammer that you use because it's yellow. [almost] All these features are available on [almost] any other distro.
But don't worry, it's your decision, you made your life unnecessarily difficult.
What you posted is quite strange. There is a message "root account locked" which should be solved. You have corrupted journaling. Also it looks like some kind of bluetooth related firmware was mentioned.
None of these issues related to boot or grub, the boot had been started and kernel was found. Probably your disk is damaged or your filesystem is corrupted or it is just a simple disk full.
The main thing I see is what you wrote, you totally messed up your system and not only once.
You wrote you installed debian and kali several times without success. It is again something which cannot be solved with grub.
Would be nice to give us exact details what did you try and make, otherwise nobody can help you to step forward.
Personally I would go with debian and solve that nvidia and/or firmware related issue, and also I would definitely try to do a clean install without dirty hacks.
So I guess somehow some things survived clean all in diskpart. Maybe it's cached in some reserved space somewhere, however I followed up a page on how to clean all EFI stuff to wipe all. And it worked, at least now with my new installed kali I can now boot to recovery mode with GUI. When I usually installed Kali, it errors on nouveau driver and Nvidia driver installation. So I fixed that. And for my case I also had a bluetooth issue where I had to do some stuff after booting to make it work, but I think they are red herrings. So currently I have a kali that works as expected from every aspect, however it's not booting with the normal mode, It doesn't say anything while not booting.
When I boot in recovery mode, everything is usable and everything works fine. I have the GPU working, I tried a couple of projects and all is working fine with the GPU. I have all sound and internet connections, the users are set correctly, I have my account admin privileges. So I can't figure why it's not booting now. I've included in the post various kinds of info and I'm attaching a file for journalctl.txt for booting troubleshooting.
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 39 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: GenuineIntel
BIOS Vendor ID: Intel(R) Corporation
Model name: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
BIOS Model name: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz To Be Filled By O.E.M. CPU @ 3.6GHz
BIOS CPU family: 207
CPU family: 6
Model: 158
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
Stepping: 13
CPU(s) scaling MHz: 26%
CPU max MHz: 5000.0000
CPU min MHz: 800.0000
BogoMIPS: 7200.00
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities
Virtualization: VT-x
L1d cache: 256 KiB (8 instances)
L1i cache: 256 KiB (8 instances)
L2 cache: 2 MiB (8 instances)
L3 cache: 16 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Vulnerable: No microcode
Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
Vulnerability Retbleed: Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds: Vulnerable: No microcode
Vulnerability Tsx async abort: Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
mount
Code:
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=32838196k,nr_inodes=8209549,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=6576240k,mode=755,inode64)
/dev/nvme0n1p5 on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2152)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
/dev/nvme0n1p6 on /home type ext4 (rw,relatime)
/dev/nvme0n1p3 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=6576236k,nr_inodes=1644059,mode=700,uid=1000,gid=1000,inode64)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
/dev/nvme0n1p2 on /media/reaper/Storage type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)
Ok I've just went through and fixed all the warnings and errors. My system works perfectly in Recovery mode, I found some mounting issues and fixed them as well. The only thing that is left it says SGX is disabled by bios. And It's always said that. Never failed a boot.
So I start to feel like it might be bios. It persist and survive the format, I dont have anything complaining in os side.
My fast boots and secure boots are disabled. I dont use legacy mode for EFI, I've selected other os rather than windows os. What might I be missing?
I'm sorry, it is definitely wrong. Kali doesn't aim to attract users with zsh or account management. It's like a hammer that you use because it's yellow. [almost] All these features are available on [almost] any other distro.
But don't worry, it's your decision, you made your life unnecessarily difficult.
+1
However... I would go further, to state that this endeavour and thread is a waste of everyone's time, including the OP's.
There seems to be two problems:
- A UUID mess up, primarily caused by a lack of understanding, when cloning the disk/ssd/block device. Cloning Linux installations is not so straightforward, as e.g. a Windows or FreeBSD installation due to the grub bootloader and UUIDs, which are usually how block devices are mounted in /etc/fstab. It requires more work and understanding.
- Messages about (missing?) firmware, appearing during boot messages.
The first one is not really worth anyone's time in unraveling, as it's your own self-inflicted problem. Just install the SSD in the machine and clean install Debian on the SSD. Install the bootloader to the primary drive and ensure you run os-prober to chainload the Windows installation.
The second problem could be the usual "possible missing firmware". Next time it appears, take note of the missing firmware file names and search the web for info. If you install the latest Debian release, all of the firmware is now included - if there's still messages about missing firmware, you will need to note down the file names and do some research.
As it stands you've made a few lengthy posts about the "firmware", yet no one is any the wiser in terms of what kind of firmware you're talking about... even though you said you were downloading missing files and installing one by one - what files and from where?
However... I would go further, to state that this endeavour and thread is a waste of everyone's time, including the OP's.
Thank you for your feedback. I'm not expecting investment of everyone's time, just some people that could help or throw in some ideas. Actually the thread really helped me. I solved a lot of things once people pointed out what I should look for to troubleshoot.
Quote:
- A UUID mess up, primarily caused by a lack of understanding, when cloning the disk/ssd/block device. Cloning Linux installations is not so straightforward, as e.g. a Windows or FreeBSD installation due to the grub bootloader and UUIDs, which are usually how block devices are mounted in /etc/fstab. It requires more work and understanding.
This is now fixed. Everything is mounted in the right place, I've done that before and I've fixed it again.
Quote:
- Messages about (missing?) firmware, appearing during boot messages.
That I also solved as well. The firmware that was missing was coming from a broken firmware. I still don't know how it survided the whole zeroed disk, however when I installed kali, I dove in and removed it with random data to make sure it's purged correctly, and another reinstall from scratch just fixed it.
Quote:
The first one is not really worth anyone's time in unraveling, as it's your own self-inflicted problem. Just install the SSD in the machine and clean install Debian on the SSD. Install the bootloader to the primary drive and ensure you run os-prober to chainload the Windows installation.
That I've already tried and was successful with Debian 12. I wanted to do the same with kali but failed to do so.
Quote:
The second problem could be the usual "possible missing firmware". Next time it appears, take note of the missing firmware file names and search the web for info. If you install the latest Debian release, all of the firmware is now included - if there's still messages about missing firmware, you will need to note down the file names and do some research.
That's exactly what I did to solve it, By the time just after my prior post I fixed the files one by one to boot up. Now I don't have a firmware error, that I know of.
Quote:
As it stands you've made a few lengthy posts about the "firmware", yet no one is any the wiser in terms of what kind of firmware you're talking about... even though you said you were downloading missing files and installing one by one - what files and from where?
Since the firmware is solved I didn't put what the firmware was. Actually I didn't even know what I was fixing. But the complained files lived here, /lib/firmware.
Completely renewed now. I'm just suspecting something is not right in my bios to boot in regular mode rather than rescue mode. Or maybe it can be read and write priviliges, which I checked from the user groups and user privileges as well as root account access.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
Kali does not boot - Firmware mess-up after an upgrade - after a harddisk migration.
.
