TL;DR:
System was installed with Ubuntu-MATE 16.04 with lvm on luks on HDD with /boot on USB drive.
USB drive died. Trying to reinstall /boot to USB drive.
After:
sudo -i
cryptsetup open UUID=f2d86e4b-9625-48f3-80f9-f2f5192ece7c foobaz_crypt
mount -v -t ext4 /dev/mapper/foobaz-root /mnt
mount -v -t ext2 /dev/sdc1 /mnt/boot
mount -v -t ext4 /dev/mapper/foobaz-home /mnt/home
mount -v -o rbind /dev /mnt/dev
mount -v -o bind /proc /mnt/proc
mount -v -t sysfs -o bind /sys /mnt/sys
chroot /mnt
grub-install -v --boot-directory=boot /dev/sdc
Add to /etc/default/grub:
GRUB_CMDLINE_LINUX="cryptdevice=UUID=f2d86e4b-9625-48f3-80f9-f2f5192ece7c:foobaz_crypt root=/dev/mapper/foobaz-root"
GRUB_ENABLE_CRYPTODISK=y
grub-mkdevicemap -v
update-initramfs -k all -c
grub-mkconfig -o /boot/grub/grub.cfg
Reboot.
I only get the grub command line. ls only shows (hd0) (hd0,msdos1), ls (hd0,msdos1) shows files on USB drive.
I just discovered the original install had some different versions.
Original install:
LVM VERSION:
LVM version: 2.02.133(2) (2015-10-30)
Library version: 1.02.110 (2015-10-30)
Driver version: 4.37.0 <---
DEVICE MAPPER VERSION:
Library version: 1.02.110 (2015-10-30)
Driver version: 4.37.0
KERNEL VERSION:
Linux foobaz 4.15.0-33-generic #36~16.04.1-Ubuntu SMP Wed Aug 15 17:21:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
DM TARGETS VERSIONS:
crypt v1.18.1 <---
striped v1.6.0
linear v1.4.0
error v1.5.0
Versions I'm using now:
LVM VERSION:
LVM version: 2.02.133(2) (2015-10-30)
Library version: 1.02.110 (2015-10-30)
Driver version: 4.35.0 <---
DEVICE MAPPER VERSION:
Library version: 1.02.110 (2015-10-30)
Driver version: 4.35.0
KERNEL VERSION:
Linux ubuntu-mate 4.8.0-36-generic #36~16.04.1-Ubuntu SMP Sun Feb 5 09:39:57 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
DM TARGETS VERSIONS:
crypt v1.14.1 <---
mirror v1.14.0
striped v1.6.0
linear v1.3.0
error v1.5.0
This is really killing me. I am pretty sure I am making some stupid typo or missing some configuration key.
Your assistance will be greatly appreciated!
I do not have interweb at home so I won't be able to reply right away but I will reply A.S.A.P.
Thank you!
Background:
This system was successfully installed on my Thinkpad T440P notebook with one HDD fully encrypted with LVM on LUKS and the /boot partition on a USB drive. BIOS is set to legacy boot. As best as I recall I used these same steps for the original install. My notes on the original install have these same steps.
The USB drive died and I am trying to reinstall the /boot partition on another USB drive.
I can boot my notebook with Linux ubuntu-mate 16.04.1 live. This is the same distro I used for the original install. The original install was from the optical drive but now I am using ubuntu-mate from a usb drive. I can decrypt the HDD with no problems and I can mount, read, and write all the logical volumes with no problem.
I have done these steps (many) many times, with various changes, without success. When I shutdown the live session and boot from the USB drive I only get the GRUB command-line, no menu. When I enter 'ls' I only see (hd0) and (hd0,msdos1). The command ls (hd0,msdos1) gives the contents of the USB drive.
I believe I have not added the correct lines in /etc/default/grub. I have read (many) many "tutorials" and they are not entirely consistent. I have not found a good guide describing the options for the GRUB_CMDLINE_LINUX= key in /etc/default/grub.
I believe doing all the mounts before chroot is correct.
I am pretty sure the mounts for /sys, /proc, and /dev, are correct. Some instructions say use bind, some say use rbind, some don't specify either, and some say mount /dev/pts separately (one "guide" even used a script to mount each entry in /dev/pts one by one).
I have tried several different ways but the permutations are killing me.
Something that seems like a problem is when I run any of these commands, they never return a UUID (this is after 'cryptsetup open ... etc.):
'grub-probe -v --target=cryptodisk_uuid --device /dev/sda'
'grub-probe -v --target=cryptodisk_uuid --device /dev/sda1'
'grub-probe -v --target=cryptodisk_uuid --device /dev/mapper/foobaz_crypt'
I know lsblk only shows /dev/sda1 as a luks device but I am trying anything at this point (complete output for /dev/sda1 is in pastebin).
If you want even more information I put the report from Boot-Repair here:
https://paste.ubuntu.com/p/QXFrkhd8JG/
Here is a summary of what I have done to reinstall the boot partition with various notes and parameters:
Immediately after boot from Ubuntu-MATE 16.04 live (from USB drive):
ubuntu-mate@ubuntu-mate:~$ uname -a
Linux ubuntu-mate 4.8.0-36-generic #36~16.04.1-Ubuntu SMP Sun Feb 5 09:39:57 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
ubuntu-mate@ubuntu-mate:~$ lsblk -pf
NAME FSTYPE LABEL UUID MOUNTPOINT
/dev/sdb iso9660 Ubuntu-MATE 16.04.2 LTS amd64 2017-02-15-21-17-48-00 /cdrom
├─/dev/sdb2 vfat Ubuntu-MATE 16.04.2 LTS amd64 E561-C446
└─/dev/sdb1 iso9660 Ubuntu-MATE 16.04.2 LTS amd64 2017-02-15-21-17-48-00
/dev/sr0
/dev/loop0 squashfs /rofs
/dev/sda
└─/dev/sda1 crypto_LUKS f2d86e4b-9625-48f3-80f9-f2f5192ece7c
NOTE: /dev/sdb is the USB drive with Ubuntu-MATE 16.04 live.
/dev/sda is LUKS encrypted HDD and contains the logical volumes with root filesystem.
ubuntu-mate@ubuntu-mate:~$ sudo dpkg -l | grep cryptsetup
ii cryptsetup 2:1.6.6-5ubuntu2 amd64 disk encryption support - startup scripts
ii cryptsetup-bin 2:1.6.6-5ubuntu2 amd64 disk encryption support - command line tools
ii libcryptsetup4:amd64 2:1.6.6-5ubuntu2 amd64 disk encryption support - shared library
ubuntu-mate@ubuntu-mate:~$ sudo dpkg -l | grep initramfs
ii busybox-initramfs 1:1.22.0-15ubuntu1 amd64 Standalone shell setup for initramfs
ii initramfs-tools 0.122ubuntu8.8 all generic modular initramfs generator (automation)
ii initramfs-tools-bin 0.122ubuntu8.8 amd64 binaries used by initramfs-tools
ii initramfs-tools-core 0.122ubuntu8.8 all generic modular initramfs generator (core tools)
ii libklibc 2.0.4-8ubuntu1.16.04.3 amd64 minimal libc subset for use with initramfs
ubuntu-mate@ubuntu-mate:~$ sudo dpkg -l | grep lvm
ii libllvm3.8:amd64 1:3.8-2ubuntu4 amd64 Modular compiler and toolchain technologies, runtime library
ii liblvm2app2.2:amd64 2.02.133-1ubuntu10 amd64 LVM2 application library
ii liblvm2cmd2.02:amd64 2.02.133-1ubuntu10 amd64 LVM2 command library
ii lvm2 2.02.133-1ubuntu10 amd64 Linux Logical Volume Manager
Tools are present.
ubuntu-mate@ubuntu-mate:~$ sudo -i
root@ubuntu-mate:/# ls
bin boot cdrom dev etc home initrd.img lib lib64 media mnt opt proc rofs root run sbin snap srv sys tmp usr var vmlinuz
root@ubuntu-mate:/# ls -a /mnt/
. ..
So /mnt is completely empty.
root@ubuntu-mate:/dev# cd mapper/
root@ubuntu-mate:/dev/mapper# ls -a
. .. control
/dev/mapper is completely empty.
root@ubuntu-mate:/# cryptsetup -v open UUID=f2d86e4b-9625-48f3-80f9-f2f5192ece7c foobaz_crypt
Enter passphrase for /dev/disk/by-uuid/f2d86e4b-9625-48f3-80f9-f2f5192ece7c:
Key slot 0 unlocked.
Command successful.
NOTE: I used UUID= just to verify the uuid works. I try to use the UUID throughout this process.
root@ubuntu-mate:~# lsblk -pf
NAME FSTYPE LABEL UUID MOUNTPOINT
/dev/sdd
└─/dev/sdd1 vfat USB_FB74 FB74-7EC2 /media/ubuntu-mate/USB_FB74
/dev/sdb iso9660 Ubuntu-MATE 16.04.2 LTS amd64 2017-02-15-21-17-48-00 /cdrom
├─/dev/sdb2 vfat Ubuntu-MATE 16.04.2 LTS amd64 E561-C446
└─/dev/sdb1 iso9660 Ubuntu-MATE 16.04.2 LTS amd64 2017-02-15-21-17-48-00
/dev/sr0
/dev/loop0 squashfs /rofs
/dev/sdc
└─/dev/sdc1 ext2 foobaz_boot 066a4e20-6b78-44f9-a384-a4fe7badc503
/dev/sda
└─/dev/sda1 crypto_LUKS f2d86e4b-9625-48f3-80f9-f2f5192ece7c
└─/dev/mapper/foobaz_crypt LVM2_member 0eNVpf-LXoP-RaSS-U8dW-460J-B6w2-JVmwYV
├─/dev/mapper/foobaz-swap swap 3c0f7f81-b1d8-4a9f-9be8-e6313d813b9f
├─/dev/mapper/foobaz-root ext4 939a2249-f1f4-46a8-bde5-b877042b3c4b
└─/dev/mapper/foobaz-home ext4 2a978294-1e7c-430c-81de-dda6021d3a9f
NOTE: foobaz_crypt is the physical volume.
foobaz is the volume group.
/dev/sdc is the USB drive that I want to make my boot partition and install GRUB. Sometimes this appears as /dev/sdd and I use that label.
/dev/sdd is a USB drive I use to save commands and notes. I will remove /dev/sdd before I do any system or GRUB updates or install GRUB.
*****
root@ubuntu-mate:~# vgscan -v
Wiping cache of LVM-capable devices
Wiping internal VG cache
Reading all physical volumes. This may take a while...
Using volume group(s) on command line.
Found volume group "foobaz" using metadata type lvm2
root@ubuntu-mate:~# vgchange -vay
Using volume group(s) on command line.
3 logical volume(s) in volume group "foobaz" already active
3 existing logical volume(s) in volume group "foobaz" monitored
Activating logical volume "swap".
activation/volume_list configuration setting not defined: Checking only host tags for foobaz/swap
Activating logical volume "root".
activation/volume_list configuration setting not defined: Checking only host tags for foobaz/root
Activating logical volume "home".
activation/volume_list configuration setting not defined: Checking only host tags for foobaz/home
Activated 3 logical volumes in volume group foobaz
3 logical volume(s) in volume group "foobaz" now active
NOTE: I did vgscan and vgchange for two attempts but they didn't help and I think Ubuntu-MATE 16.04 does this automatically.
*****
NOTE: Some guides make a separate directory for the mounts, e.g., /mnt/target but I don't think this is necessary.
root@ubuntu-mate:~# mount -v -t ext4 /dev/mapper/foobaz-root /mnt
mount: /dev/mapper/foobaz-root mounted on /mnt.
root@ubuntu-mate:~# mount -v -t ext4 /dev/mapper/foobaz-home /mnt/home
mount: /dev/mapper/foobaz-home mounted on /mnt/home.
root@ubuntu-mate:~# mount -v -t ext2 /dev/sdc1 /mnt/boot
mount: /dev/sdc1 mounted on /mnt/boot.
root@ubuntu-mate:~# mount -v -o rbind /dev /mnt/dev
mount: /dev bound on /mnt/dev.
root@ubuntu-mate:~# mount -v -o bind /proc /mnt/proc
mount: /proc bound on /mnt/proc.
root@ubuntu-mate:~# mount -v -t sysfs -o bind /sys /mnt/sys
mount: /sys bound on /mnt/sys.
NOTE: Some guides say use bind. Some guides say mount /dev/pts in an additional step. When I use mount -v -o rbind /dev /mnt/dev and look in /mnt/dev/pts the stuff is there (files? inodes?).
*****
root@ubuntu-mate:~# mount -v -o rbind /run/lvm /mnt/run/lvm
mount: /run/lvm bound on /mnt/run/lvm.
root@ubuntu-mate:~# mount -v -o rbind /run/lock/lvm /mnt/run/lock/lvm
mount: /run/lock/lvm bound on /mnt/run/lock/lvm.
NOTE: One guide says to mount /run/lvm and /run/lock/lvm for "older" systems.
lshw shows: product: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz, so my system is not that old but I tried this one time anyway.
*****
root@ubuntu-mate:/# cat /mnt/etc/lvm/lvm.conf | grep 'use_lvmetad ='
use_lvmetad = 0
NOTE: When use_lvmetad = 1 , grub-mkconfig gives this error: /run/lvm/lvmetad.socket: connect failed: No such file or directory.
I tried to enable lvm2-lvmetad but it didn't work.
So:
root@ubuntu-mate:~# chroot /mnt
Add these lines to /etc/default/grub:
GRUB_CMDLINE_LINUX="cryptdevice=UUID=f2d86e4b-9625-48f3-80f9-f2f5192ece7c:foobaz_crypt root=/dev/mapper/foobaz-root"
GRUB_ENABLE_CRYPTODISK=y
NOTE: foobaz_crypt is the physical volume.
foobaz is the volume group.
foobaz-root is the logical volume that contains the root filesystem.
Put this in /etc/initramfs-tools/conf.d/cryptroot:
CRYPTROOT=target=foobaz_crypt,source=UUID=f2d86e4b-9625-48f3-80f9-f2f5192ece7c
My crypttab:
# <target name> <source device> <key file> <options>
foobaz_crypt UUID=f2d86e4b-9625-48f3-80f9-f2f5192ece7c none luks,loud,lvm=foobaz
NOTE: foobaz is the volume group that contains foobaz-root, foobaz-home, and foobaz-swap.
The man page for crypttab does not list lvm as an option! I believe lvm= should be the volume group.
The lvm option is not mentioned at all in the crypttab manpage! I have only seen it mentioned
(very briefly) in /usr/share/doc/cryptsetup/README.initramfs.gz
I put in loud hoping to get a message if there were any errors.
root@ubuntu-mate:~# grub-mkdevicemap -v
NOTE: The first time I ran grub-install, I got this error:
grub-install: info: cannot open `/boot/grub/device.map': No such file or directory.
Contents of /boot/grub/device.map:
(hd0) /dev/disk/by-id/lvm-pv-uuid-0eNVpf-LXoP-RaSS-U8dW-460J-B6w2-JVmwYV
(hd1) /dev/disk/by-id/ata-ST2000LM003_HN-M201RAD_S377J9BGB01532
(hd2) /dev/disk/by-id/usb-SanDisk_SDDR-B531_Reader_0811185658222650-0:0
(hd3) /dev/disk/by-id/usb-Generic_STORAGE_DEVICE_000000000272-0:0
NOTE: This map does not seem to change after the first time I ran grub-mkdevicemap, so I only ran grub-mkdevicemap twice.
(hd3) is the USB drive where I want to install the /boot partition. After I reboot, grub shows only this drive, as (hd0) and (hd0,msdos1).
My fstab:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/foobaz-root / ext4 errors=remount-ro 0 1
# /boot was on /dev/sdb1 during first installation
# /boot was on /dev/sdc1 during reinstallation UUID=066a4e20-6b78-44f9-a384-a4fe7badc503
#
UUID=066a4e20-6b78-44f9-a384-a4fe7badc503 /boot ext2 defaults 0 2
/dev/mapper/foobaz-home /home ext4 defaults 0 2
/dev/mapper/foobaz-swap none swap sw 0
Devices after chroot:
root@ubuntu-mate:/# lsblk -pf
NAME FSTYPE LABEL UUID MOUNTPOINT
/dev/sdd
└─/dev/sdd1
/dev/sdb
├─/dev/sdb2
└─/dev/sdb1
/dev/sr0
/dev/loop0
/dev/sdc
└─/dev/sdc1 /boot
/dev/sda
└─/dev/sda1
└─/dev/mapper/foobaz_crypt
├─/dev/mapper/foobaz-swap
├─/dev/mapper/foobaz-root /
└─/dev/mapper/foobaz-home /home
root@ubuntu-mate:~# shutdown now 'RUN FOR THE HILLS!'
NOTE: I don't umount everything before shutdown. I thought Linux does this automatically.
