Shutdown hangs at "Stopping remaining crypto disks (busy)"

fornax · 07-26-2015, 08:48 AM

Hi, my question is basically the subject.
It is an up to date Debian testing amd64. It worked flawlessly until my last dist-upgrade some weeks ago. I'm using lvm+luks to encrypt all drives.

When shutting down, (w/o error messages), the last lines are:

Code:

[ ok ] Deactivating swap..done.
[ ok ] Unmounting local filesytems..done.
[....] Stopping remaining crypto disks...sdc5_crypt (busy)...

The systems hangs at this point, I need to kill it by long pressing the power button.

Can anyone give me some pointers how to debug this problem? So far, I could not find a log file which would give me useful information what's happening here.

Thank you!

unSpawn · 07-26-2015, 09:04 AM

Quote:

Originally Posted by fornax

The systems hangs at this point, I need to kill it by long pressing the power button.

While file systems are getting better all of the time and while I do understand your predicament you realize that still is an extremely ill advised thing to do, right?

Quote:

Originally Posted by fornax

Can anyone give me some pointers how to debug this problem?

There may be a process keeping files open during shutdown. The problem is you're way too far down the shutdown progress to have any terminal access available to diagnose things. I'd drop to (the equivalent of) runlevel 1 and see what processes are still running ('ps ax -eopid,args' should do) and then check open files with 'lsof -Pwlnp <pid>;' or just 'lsof -Pwln -a +D<mountpoint>;'.

sgosnell · 07-26-2015, 09:24 AM

You may have a terminal open, with a drive displayed. If you cd to a drive in a terminal, Linux won't allow you to unmount that drive, or do many other things with it. Something is certainly going on with the drive, but it's not possible to tell exactly from long distance.

fornax · 07-26-2015, 03:22 PM

Thanks for the answers so far. Of course I know that killing the system is a bad idea, but it really
hangs here (I've waited for several hours). So it unfortunately leaves me no choice.

I tried checking processes in single user but found nothing suspicious. I can post the output if you would like to have a look.

After more debugging, I found some other interesting issues.
To begin with, the upgrade (maybe since systemd, yay) renamed my disks.
Before the update, I had /dev/sdb1 -> sdb1_crypt and /dev/sdc5 -> sdc5_crypt.
Now, the disks got renamed to /dev/sda1 and /dev/sdb5.
Both still worked as they should with using UUIDs.

I adapted /etc/crypttab and /etc/blkid (new file? never saw that one before), shutdown
was successfull ONCE.

After that, I tried updating the initramfs, which gave me this warning:

Code:

cryptsetup: WARNING: invalid line in /etc/crypttab for sdc5_crypt -

The resulting initramfs did not boot and had no cryptsetup in the initramfs shell.
Had to fix it with a recovery shell from USB.

Now the names are correct, the problem stays the same except it is sdb5_crypt now.

kiyop · 08-01-2015, 04:16 AM

Is there any relevant info on the release notes especially some URL such as the following?
https://www.debian.org/releases/jess...mation.en.html

Head_on_a_Stick · 08-01-2015, 07:02 AM

Use UUIDs to identify the drives rather than block devices.

Does the system shut down cleanly under SysVinit?
https://wiki.debian.org/FAQsFromDebi...t_on_Jessie.3F

sgosnell · 08-01-2015, 07:44 AM

If things work as they should using UUIDs, then why not use them and get on with life? You can't rely on drives being mounted as the same sd? every time. You could also give the drives labels, and mount using those labels, which is sometimes more reliable than UUID.

fornax · 08-02-2015, 03:47 PM

Okay, seems like my comments about the naming caused more uncertainties than they helped.
To bring it to the point: I was using UUIDs all the time, booting & mounting worked and still work, but *no* effect on the described problem.
The *one* time the system was shutting down was at one point after adapting the name (not the UUID!) in the crypttab. I could neither reproduce this, nor did it shut
down after rebooting with the "fixed" cryptab.
The whole naming thing was just something that I found quite interesting to cause problems (like the initrd/grub upgrade) at all. I thought it might be related. Turns out, it was not.

I tried sysvinit as well, same behavior.

kiyop · 08-09-2015, 09:31 PM

Of course, you have read https://www.debian.org/releases/jess...orted-features , haven't you?

How about using "poweroff" command instead of "shutdown -h now" or "halt" command?

fornax · 08-10-2015, 08:45 AM

Tried poweroff, same behavior. I think the power down bug which is referred here happens at a later stage.

BeniBela2 · 11-19-2015, 05:44 AM

Has anyone found a solution to this?

After a dist-upgrade I have the same problem.

I do not even have a running crypto disk (I mount my crypto disk like once a month).

Quote:

Code:

$ cat /etc/crypttab
# <target name>	<source device>		<key file>	<options>
# /dev/mapper/crypt-home /dev/sda7 none luks,noauto
cswap /dev/sda5 /dev/urandom swap
/dev/mapper/crypt-home UUID=89fd998a-dfba-479b-b80a-f99ae6b2f35d none luks,noauto

mzsade · 11-19-2015, 07:54 AM

Did your dist-upgrade also upgrade your kernel? If so, and if you still also have the old kernel, boot into it. Dist-upgrade to a new kernel caused the same problem with my hardware on Sid. Fortunately, i also had the original kernel 3.16 to boot into from which i flushed kernel 4.2.5. I have found that upgrading to a new kernel works without errors on my laptop only when i compile it from source while upgrading it from the repos or backports buggers up my system..more than once.

sgosnell · 11-20-2015, 05:45 PM

I always keep at least one previous known-good kernel installed, sometimes two. You never know when things will go south unexpectedly.

BeniBela2 · 11-24-2015, 12:51 PM

So, now I have tried 3.16, 3.14 and 3.12

It does not work with any

I do not think it is a kernel issue

Any other ideas?

BeniBela2 · 12-02-2015, 02:49 PM

Found this: https://bugs.debian.org/cgi-bin/bugr...cgi?bug=792552