Resizing Raid 6 Array

Driekus · 10-06-2019, 05:27 PM

New to the forum and really hoping someone can help. My linux knowledge is average and I am no expert.

I have an encrypted Raid 6 array with 6 drives. I have added the 7 drive using the grow command with Webmin.

I have been roughly following the below link
http://jotschi.de/2014/02/09/lvm-mda...debian-wheezy/

I was able to resize the crypt layer using the cryptsetup command.

Code:

# Resize the crypt layer and check the result
  cryptsetup resize cryptotest
  fdisk -l /dev/mapper/cryptotest

  # Resize physical volume and check the result
  pvresize /dev/mapper/cryptotest
  pvdisplay

  # Resize logical volume and check result
  lvextend -l +100%FREE /dev/mapper/testvg-testlv
  lvdisplay

  # Resize filesystem
  e2fsck -f /dev/mapper/testvg-testlv
  resize2fs /dev/mapper/testvg-testlv
  mount /dev/mapper/testvg-testlv test

The problem I have is that when I run pvresize I get the following error.

Code:

root@peter-X10SL7-F:/media/Raid# pvresize /dev/mapper/Raid
  Failed to find physical volume "/dev/mapper/Raid".
  0 physical volume(s) resized / 0 physical volume(s) not resized

Indeed pvdisplay and lvdisplay come up blank when I enter those commands.
pvs does show the volume but only when I do the "pvs -a"

Code:

root@peter-X10SL7-F:/media/Raid# pvs -a
  PV                     VG Fmt Attr PSize PFree
  /dev/mapper/Raid              ---     0     0 
  /dev/mapper/cryptswap1        ---     0     0 
  /dev/md0                      ---     0     0 
  /dev/sda1                     ---     0     0 
  /dev/sda5                     ---     0     0 
  /dev/sdb1                     ---     0     0

I have included output from mdadm

Code:

root@peter-X10SL7-F:/media/Raid# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Thu May 14 16:11:40 2015
        Raid Level : raid6
        Array Size : 19534425600 (18629.48 GiB 20003.25 GB)
     Used Dev Size : 3906885120 (3725.90 GiB 4000.65 GB)
      Raid Devices : 7
     Total Devices : 7
       Persistence : Superblock is persistent

       Update Time : Sun Oct  6 16:11:21 2019
             State : clean 
    Active Devices : 7
   Working Devices : 7
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : mint:0
              UUID : 12c9466f:10054afa:7a70f710:a476c3cf
            Events : 13332

    Number   Major   Minor   RaidDevice State
       0       8       81        0      active sync   /dev/sdf1
       6       8       33        1      active sync   /dev/sdc1
       2       8       49        2      active sync   /dev/sdd1
       3       8       97        3      active sync   /dev/sdg1
       4       8       65        4      active sync   /dev/sde1
       5       8      129        5      active sync   /dev/sdi1
       7       8      112        6      active sync   /dev/sdh

Code:

root@peter-X10SL7-F:/media/Raid# fdisk -l /dev/mapper/Raid
Disk /dev/mapper/Raid: 18.2 TiB, 20003249717248 bytes, 39068847104 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 2621440 bytes

Any help on what to do next would be appreciated.

berndbausch · 10-08-2019, 05:08 AM

From the pvs man page:

Quote:

--all
Include information in the output about devices that have not been initialized with pvcreate(8).

Your -a option is an alias of --all.

/dev/mapper/Raid exists, but it's not a physical volume. pvs -a lists all storage devices on your system, but none of them is a PV.

If I understand you right, you created a RAID device, encrypted it, then created a physical volume on the encrypted device.
You then grew the RAID device, resized the encryption layer, and boom the physical volume structures have disappeared. Perhaps one of the steps didn't work and you didn't see the error messages? Is anything in the kernel message buffer or the system log?

Perhaps we can see something if you share the precise commands that you issued.

By the way, I would probably create the PV on the RAID, create logical volumes and encrypt the LVs rather than creating a PV on an encrypted device. However, I don't see why your approach shouldn't work.

Driekus · 10-08-2019, 10:52 PM

Quote:

Originally Posted by berndbausch

From the pvs man page:

Your -a option is an alias of --all.

/dev/mapper/Raid exists, but it's not a physical volume. pvs -a lists all storage devices on your system, but none of them is a PV.

If I understand you right, you created a RAID device, encrypted it, then created a physical volume on the encrypted device.
You then grew the RAID device, resized the encryption layer, and boom the physical volume structures have disappeared. Perhaps one of the steps didn't work and you didn't see the error messages? Is anything in the kernel message buffer or the system log?

Perhaps we can see something if you share the precise commands that you issued.

By the way, I would probably create the PV on the RAID, create logical volumes and encrypt the LVs rather than creating a PV on an encrypted device. However, I don't see why your approach shouldn't work.

The grow command that I used was in the Webmin interface. Likely the command is similar to below.

Code:

mdadm --add /dev/md2 /dev/loop2

  mdadm --add /dev/md2 /dev/loop3
  mdadm --grow --raid-devices=4 /dev/md2

The only other command I successfully executed is

Code:

  cryptsetup resize cryptotest
  fdisk -l /dev/mapper/cryptotest

Would the pvcreate, vgcreate and lvcreate erase the existing data on the disk? The Raid was setup 4 years ago so I cannot remember exactly all the commands that were done. I am preparing a backup drive for the data before I risk expanding the array.

berndbausch · 10-09-2019, 01:02 AM

Quote:

Originally Posted by Driekus

Would the pvcreate, vgcreate and lvcreate erase the existing data on the disk? The Raid was setup 4 years ago so I cannot remember exactly all the commands that were done. I am preparing a backup drive for the data before I risk expanding the array.

pvcreate creates an LVM header, which vgcreate and lvcreate then modify. The header is probably not large, but it obviously overwrites whatever was there before.

Assuming the encrypted RAID device was a physical volume before, I wonder if the LVM header is still there but at a different location. I am sure there are tools that scan a disk for LVM data and perhaps repair a situation like yours.