A disk with a partition which was an md underlying device in another system was fitted. After booting, the foreign md was listed in /proc/mdstat but the foreign VG was not listed by vgs.

The vgchange -ay run during boot was expected to have discovered the VG.

Running vgchange -ay changed nothing. Trying to make the md a PV, both implicitly using vgcreate and explicitly using pvcreate failed. Here's the last run:
What to try next?

The gory details

The Debian 8 Jessie 64-bit server was running normally.

It had been installed with EFI boot disabled in the BIOS but the BIOS did not detect the HDDs until EFI BIOS was enabled (we have this situation on at least two Intel motherboard servers).

There were two 1 TB HDDs, formatted GPT. The first partitions were used for md0 which provided /boot. The second were used for md1 which provided the LVM PV.

One of the HDDs generated a few SMART errors. It was removed but the manufacturer's test software found no defects. It was refitted (that was probably a mistake. When one of a mirrored pair starts going bad but not in ways that make md remove it from the array, it may cause file system corruption. Your views?). The SMART errors increased. It was removed. The server would not boot, reporting it could not find the LV for the root file system. Puzzlement.

The defective HDD was refitted. The sever booted. The md devices were using only partitions from the defective HDD. Perhaps md had detected discrepancies and dropped the wrong underlying devices from the array.

The partitions from the good HDD were manually added back to the array. When synchronisation finished, the partitions from the defective HDD were removed from the array.

The replacement 1 TB HDD arrived, was fitted, partitioned and added to the md arrays. Trying to install GRUB generated multiple error messages including "/usr/sbin/grub-probe: error: disk `mduuid/af598d16317f889bd7975cb97d51ad19' not found". Using GNU parted and running "set 1 bios_grub on" on both sda and sdb allowed GRUB to install normally. IDK what "set 1 bios_grub on" does (!) but it made the server bootable without the defective HDD.

The defective HDD was removed and RMAed.

A temporary replacement 3 TB HDD was fitted and its partitions added to the md devices.

The replacement HDD arrived and was fitted but the server would not boot, complaining again about not being able to find the root LV. Puzzlement.

In an effort to clean up this mess, the OS was installed to the replacement HDD as the only HDD connected, this time in EFI mode.

The next step is to recover the data LVs from the old disks. The temporary 3 TB HDD was fitted and ... we are back to the beginning of this story.

Did you try vgscan?

Is the partition table on the 3 TB drive dos or gpt? Usually past 2 TB you need gpt but many of these drives come in with dos partition table. (Note "table" here - we're talking about the overall layout of the disk NOT a specific partition's "type" on it - the partition type for the partition.)

Is your LV's PV the metadisk device?

Thanks MensaWater

I did not run vgscan and did so now. It did not find the "foreign" VG.

All partition tables are GPT.

IDK what a "metadisk device" is and searching the 'net did not help. Insofar as each VG (the one from the old system and the one from the new) has a single PV which is an md device and the VG provides the LV for the / file system, then the answer is probably "yes".

Some progress made. In the busybox/ash shell examined /proc/mdstat and found the md devices inactive. Used mdadm to stop them and then to start them (no forcing);
After running lvchange -ay for each LV and exiting the busybox/ash shell, the server booted normally.

The question is now why did the kernel and/or initrd mark the md devices inactive?

1 members found this post helpful.

Fixed by:
The fix was tested on two servers; after the fix both booted normally with one HDD connected.

One of the two servers tested had kernel parameter bootdegraded=true, one did not.

1 members found this post helpful.