Ubuntu Bionic hangs during boot

achim_59 · 12-21-2019, 04:01 PM

This actually applies to my neighbour's laptop, but he's not very technically minded and I try to help him when he has problems. To make it easier I'll just refer to him as D.

He recently brought his laptop over, saying he couldn't log on. When I fired up the machine it started the usual way, but then it just hung on the splash screen. He left the machine with me and I've been trying to figure out what's wrong. It isn't anything that he installed, since he always calls me before installing anything new.

The following link suggests that it has something to do with the snapd service:
https://askubuntu.com/questions/1051...dist-upgrade-o

However, D's machine doesn't just have a long delay, it hangs completely, so I think it is a different problem. Besides, that askubuntu post talks of kernel 4.15.0-24 and D's machine is booting with 4.15.0-72.

Another askubuntu post says that the problem is with Wayland (replacement for X) and suggests simply disabling it:
https://askubuntu.com/questions/1149...5-after-update

I tried that, but the problem persisted.

Yet another site (https://itsfoss.com/fix-ubuntu-freezing/) suggests the problem lies with the NVIDIA graphics drivers. I followed the steps described, and sure enough the boot started with a non-graphics splash screen and eventually went further (like in the old days when you got the log output as part of the boot sequence). Sadly, this process also hangs eventually. The last few messages are as follows:

Code:

[FAILED] Failed to start Snappy daemon.
See 'systemctl status snapd.service for details.
         Starting failure handling of the snapd snap...
         Starting Wait until snapd is fully seeded...
[  OK  ] Stopped Snappy daemon.
         Starting Snappy daemon...
[  OK  ] Started Failure handling of the snapd snap.
[  OK  ] Started Snappy daemon.
[  OK  ] Started Wait until snapd is fully seeded.
[***   ] A start job is running fo Hold unt...finishes up (14min 30s / no limit)

Since I started typing this stuff a couple of cleanup messages were added to that list, but the clock is still running and the boot up is still not finished.

That's where I'm at now. There is a possibility that the boot problems can be fixed as described here:
https://linuxconfig.org/ubuntu-boot-repair
I guess I'll try that, but judging from what I've seen so far, an automated boot repair probably won't do the trick.

What I'd like to try is to get the latest update in the hope that that will fix the problem. It should also be possible with apt-get to re-install the snapd package explicitly. Trouble is, there's no networking running when I start in recovery mode. There is an option to try starting it, but from what I've read, it doesn't work properly. Certainly the one time I tried that option it didn't work. I notice on the display of D's laptop, that the network manager script dispatcher service has been started, but whether that means I'd be able to run apt-get is another matter. Besides, the snapd seeding process is still not finished after 30+ minutes.

I'm really grasping at straws at this point. can anybody provide me with a hint as to what else I could try?

If I manage to solve the issue, then I'll describe what I did so that others with the same problem can benefit from my experimentation. I'll also mark this thread as solved if I get that far.

All suggestions welcome.

yancek · 12-21-2019, 06:55 PM

If you get boot repair and run it, I would suggest that you use the Create BootInfo Summary optio which outputs a link and you can review the output to see if there is something obvious. You can also post the link here so that members can review it and hopefully, make a suggestion.

mrmazda · 12-21-2019, 11:04 PM

Has Snappy run the / filesystem out of freespace?

achim_59 · 12-22-2019, 03:11 AM

@yanceck - I'm planning to give it a try today some time. I'll post the results.

@mmazda - No, snappy definitely hasn't used up all the free space. There are 490 GB of free spce on D's machine.

yancek · 12-22-2019, 06:53 AM

If you have an Ubuntu 'live' system (or any major derivative) and get boot repair using the ppa as suggested (more current version that way), make sure not to make any changes but just do the Create BootInfo Summary and review it, and/or post the link here if necessary. Hopefully, that will provide some information which will help. Too many possibilities in a situation like this.

achim_59 · 12-22-2019, 11:30 AM

@yancek - Your admonition not to change anything came a tick too late. I'd already run both the report and the repair before I read it. (missed it by about 20 minutes)

No matter, I don't think anything significant was changed. I've uploaded the before and after versions (RESULTS_01 is before RESULTS_02 is after). I made a comparison using kdiff3 and didn't see anything that would cause problems (I changed the input language from German to English whilst I was working which resulted in some minor text differences, but the meaning is the same).

The only significant thing I noticed is that the mount point for /dev/sda1 has changed, but when I try to boot D's machine I get the same grub menu as before. What then happens is also precisely the same as before. I also noted that the mount point for /dev/sda1 in RESULTS_02 is somewhere under /media instead of /mnt. So I ran the whole procedure again after establishing that boot repair had not solved the problem. This time I just ran the report and the mount point for /dev/sda1 was once again /mnt/boot-sav/sda1. I've uploaded that as RESULTS_03.

I really think that the problem is not the boot loader itself, but rather something to do with snapd. However, the behaviour, as mentioned before, is not the same as described in those other posts on askubutu. The system just plain hangs. So I'd really like to know how I can replace that package (or maybe even just do an update to see if that fixes things). But in order to do that I would need to initiate networking whilst logged in as root in recovery mode and I'm not at all sure how to go about that.

I'm about ready to give up for today. If nothing else works, I'll reinstall from the dvd I burned and retain D's data files. Unfortunately at the very least we'll lose the software that I helped him install for his card reader, printer, and scanner. I'll also probably have to reinstall the MATE desktop, too, since that's the one he likes. If I can avoid all that, it would be great.

For reasons I'd rather not go into, I've only got until the 24th to get this done. It won't be the end of the earth if I don't manage it, but D will be disappointed.

Any tips that anyone can give are very very welcome.

mrmazda · 12-22-2019, 03:14 PM

You should be able to get a useful idea when the hang occurs by appending 3 to the kernel cmdline in Grub menu. If you reach a shell prompt, it means your hang is X related, and you should be able to fix whatever is going wrong more simply than via a rescue boot. If it doesn't help, instead of 3, try 1 or S, which won't give you networking, but will provide a running OS.

Sometimes Plymouth causes strange trouble. Try disabling Plymouth in Grub same way as above by including noplymouth or plymouth.enable=0 instead of 3, 1 or S. Which is appropriate for Ubuntu I don't know because I never have Plymouth installed except on Mageia, where it's mandatory.

In case you're not familiar with appending to kernel cmdline, visit
https://en.opensuse.org/SDB:Nomodese...tion_Obstacles where instructions how are included. It might even be worth trying nomodeset.

scasey · 12-22-2019, 03:22 PM

Quote:

Originally Posted by mrmazda

Has Snappy run the / filesystem out of freespace?

Quote:

Originally Posted by achim_59

@mmazda - No, snappy definitely hasn't used up all the free space. There are 490 GB of free spce on D's machine.

Please note that mrmazda was asking about the root partition, which is mounted at /
If you can get to a command line (see mrmazda'a last post) check that with df -h

mrmazda · 12-22-2019, 04:03 PM

Quote:

Originally Posted by scasey

If you can get to a command line (see mrmazda'a last post) check that with df -h

With Snappy, is BTRFS in use instead of EXT4? If so, the command would be:

Code:

sudo btrfs fi df /

or:

Code:

sudo btrfs fi usage /

achim_59 · 12-23-2019, 12:07 AM

@mrmazda: The only way I can get to a console is by using recovery mode. Also, I already tried the "nomodeset" option. See my original post... that was the 3rd thing I looked into ((https://itsfoss.com/fix-ubuntu-freezing/). That is how I got the information about snapd seeding, which just never seems to complete. I'll see if I can retrieve the relevant logs from the login attempts. That might shed some light on the matter, though it's a bit difficult to retrieve stuff when you can't really log in properly.

Quote:

With Snappy, is BTRFS in use instead of EXT4? If so, the command would be:

I confess that I haven't checked, but to the best of my knowledge, Ubuntu standard installations use ext4. I will look into it though.

@scasey @mrmazda:

Quote:

Please note that mrmazda was asking about the root partition, which is mounted at /

So was I! I use

Code:

df

pretty regularly on my own machine.

The more I think about it, the more I suspect that snapd is the problem. None of the grub or graphics solutions have worked (yet). If I could just get the networking to function in recovery mode, then I might be able to fix it.

Thanks for the advice so far, none-the-less.

syg00 · 12-23-2019, 12:32 AM

snapd in this context is not the snapshot product, but the snapcraft.io snap (similar concept to appimage).

That seeding message sounds like a low entropy issue - try tapping some keys and moving the mouse around (fast) at the same time. That should help if it is entropy related.

yancek · 12-23-2019, 06:27 AM

Nothing wrong with the Grub or boot files but I did notice that there was no entry in /etc/fstab for the / filesystem partition in any of the 3 results.txt files. You might try putting a proper entry there.

achim_59 · 12-28-2019, 03:08 PM

OK, Christmas is over and I can get back to trying to solve this issue.

Quote:

Originally Posted by mrmazda

You should be able to get a useful idea when the hang occurs by appending 3 to the kernel cmdline in Grub menu. If you reach a shell prompt, it means your hang is X related, and you should be able to fix whatever is going wrong more simply than via a rescue boot. If it doesn't help, instead of 3, try 1 or S, which won't give you networking, but will provide a running OS.

I tried your suggestion of appending "3" and I did get a console. However, I couldn't really do anything, because the entire filesystem was read only. I read up on the kernel command line parameters and the "ro" parameter is the reason for this. I'm assuming that, under normal circumstances, the read only file system is set to read/write at a later point in time? Anyway, I remounted it in read/write mode on the command line.

Code:

mount -o remount,rw /

I then tried an "apt-get update" and promptly got a stack of errors along the lines of:

Code:

Fehl:2 http://de.archive.ubuntu.com/ubuntu bionic InRelease
  "de.archive.ubuntu.com" konnte nicht aufgelöst werden

That's German for: "de.archive.ubuntu.com" could not be resolved. Apparently there are also several entries in "sources.list" which are multiply configured, but I can probably sort that out without too much trouble. What I really need to do now is to figure out how to get the networking on D's machine to running from the console.

Quote:

Sometimes Plymouth causes strange trouble. Try disabling Plymouth in Grub same way as above by including noplymouth or plymouth.enable=0 instead of 3, 1 or S. Which is appropriate for Ubuntu I don't know because I never have Plymouth installed except on Mageia, where it's mandatory.

Disabling Plymouth is something I have not tried... yet. From what I've read, in Ubuntu you just remove the "quiet" and "splash" parameters from the kernel command line.

I would like to post the output of some of the commands I've tried, but I cannot mount a USB drive from the console either. I'll have to try running the live CD again, because that allowed me to mount a USB drive previously. The snapd.seeded.service entries in journalctl do not really indicate a problem there after all, but then I'm really no guru when it comes to such matters. I'll try to get the output from journalctl onto the USB stick and post them here... maybe tomorrow.

As always any tips are more than welcome.

mrmazda · 12-28-2019, 03:55 PM

Quote:

Originally Posted by achim_59

I then tried an "apt-get update" and promptly got a stack of errors along the lines of:

Code:

Fehl:2 http://de.archive.ubuntu.com/ubuntu bionic InRelease
  "de.archive.ubuntu.com" konnte nicht aufgelöst werden

That's German for: "de.archive.ubuntu.com" could not be resolved. Apparently there are also several entries in "sources.list" which are multiply configured, but I can probably sort that out without too much trouble. What I really need to do now is to figure out how to get the networking on D's machine to running from the console.

While it's possible sources.list needs some tweaking, I find it more likely networking is defective. It's not unusual on systems that normally use a GUI tool for network "management", that networking is absent or defective until either the GUI works, or some intervention is applied. For that particular message I have found the only problem, after finding 'ip a' output as expected, to be that /etc/resolv.conf is a symlink to an empty file, and that putting a valid nameserver entry in it solves the name resolution problem.

There should be a cmdline utility available to take the place of the GUI manager, such as nmcli.

achim_59 · 12-29-2019, 09:31 AM

Just to be sure I had tried absolutely everything, I disabled plymouth (i.e. remove the "quiet" and "splash" parameters from the kernel command line). This meant I got a scrolling list of messages... That takes me back a bit! :-) Anyhow, whilst I couldn't catch all of it, there wee messages indicating that the networking wasn't started. A bit further on, however, I get the following:

Code:

[  OK  ] Started Network Manager.
         Starting Network Manager Wait Online...
[  OK  ] Reached target Network.
         Starting Permit User Sessions...
[  OK  ] Started Unattended Upgrades Shutdown.
         Starting OpenVPN service...
[  OK  ] Started Permit User Sessions.
[  OK  ] Started OpenVPN service.
         Starting Hold until boot process finishes up...
         Starting GNOME Display Manager...
[  OK  ] Started Hold until boot process finishes up.
         Starting Set console scheme...
         Starting Network Manager Script Dispatcher Service...
[  OK  ] Started Set console scheme.
[  OK  ] Created slice system-getty.slice
[  OK  ] Started GNOME Display Manager.
[  OK  ] Started Network Manager Script Dispatcher Service.
[  OK  ] Started Network Manager Wait Online.
[  OK  ] Reached target Network is Online.
         Starting LSB: disk temperature monitoring daemon...
[  OK  ] Started crash report submission daemon.
         Starting Daily apt download activities...
         Starting Message of the Day...
         Starting Tool to automatically collect and submit kernel crash signatures...
[  OK  ] Started LSB: disk temperature monitoring daemon.
[  OK  ] Started Tool to automatically collect and submit kernel crash signatures.
[FAILED] Failed to start Message of the Day.
See 'systemctl status motd-news.service' for details.
[FAILED] Failed to start Daily apt download activities.
See 'systemctl status apt-daily.service' for details.
         Starting Daily apt upgrade and clean activities...
[FAILED] Failed to start Daily apt upgrade and clean activities.
See 'systemctl status apt-daily-upgrade.service' for details.
[FAILED] Failed to start Manage, Install and Generate Color Profiles.
See 'systemctl status colord.service' for details.
[FAILED] Failed to start Snappy daemon.
See 'systemctl status snapd.service' for details.
         Starting Failure handling of the snapd snap...
         Starting Wait until snapd is fully seeded...
[  OK  ] Started Failure handling of the snapd snap.
[  OK  ] Stopped Snappy daemon.
         Starting Snappy daemon...
[  OK  ] Started Snappy daemon.
[  OK  ] Started Wait until snapd is fully seeded.
[  OK  ] Reached target Multi-User System.
[  OK  ] Reached target Graphical Interface.
[  OK  ] Started Stop ureadahead data collection 45s after completed startup.

That's the last page of output from the startup sequence... And that's where it's been hanging for the past 30 minutes whilst I've been typing the results into this post by hand!! Well, there's no other way to get the information over onto my machine.

There was no keyboard response for CTL-C, CTL-D, or any of the function keys.... I'd kind of hoped I could toggle my way to a GUI interface, but no luck. I then tried CTL-ALT-DEL (the so called three finger salute) and the boot sequence stopped. The machine then did a reboot and I'm now staring at the Grub menu again. It all scrolled by so fast, that I didn't catch anything helpful... looked like a lot of OK messages, nothing more.

I could try the live CD again and maybe the journalctl entries from the past few days might provide something useful (provided I can get them onto a usb drive). At the moment I am not feeling particularly motivated, though. I seem to be running around in circles.

I think I'll go and talk to my better half for a while and get my mind off this problem for a bit.