mdadm s/w raid module ramains device file in privileged container

shellwedance · 08-02-2022, 07:16 PM

Hello all.
I am testing software raid in container environment.
Issue is that garbage files of mdadm remain in host when I stopped raid device in container.

Below is creating s/w raid with two NVMe-oF devices in privileged container.

Code:

## host kernel version is 5.3.153
$ uname -r
5.4.153-1.20211202.el7.x86_64

$ docker run --privileged -i -t --name test-volume1 -v /dev:/dev -v /sys:/sys centos_7.9.2009_x86_64 /bin/bash

## In privileged container
$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0    0   11G  0 disk 
nvme1n1 259:1    0   11G  0 disk 
...


$ /sbin/mdadm --create /dev/md/testraidvol --assume-clean --failfast --bitmap=internal --level=1 --raid-devices=2 /dev/nvme0n1 /dev/nvme1n1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/testraidvol started.

## After creating s/w raid, disk status is like below.
$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
nvme0n1 259:0    0   11G  0 disk  
`-md127   9:126  0   11G  0 raid1 
nvme1n1 259:1    0   11G  0 disk  
`-md127   9:126  0   11G  0 raid1 
...

$ ls /dev/md*
/dev/md127

/dev/md:
testraidvol

Below is deleting s/w raid in privileged container.

Code:

## Stopping s/w raid
$ /sbin/mdadm --stop /dev/md/testraidvol
mdadm: stopped /dev/md/testraidvol


$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0    0   11G  0 disk 
nvme1n1 259:1    0   11G  0 disk 
...

## Raid disk does not exist in lsblk, but device files still remain..
$ ls /dev/md*
/dev/md127

/dev/md:
testraidvol

We can check mdadm garbage device files still remain in /dev path.
After host reboot, the remaining device files are gone. But we don't want to reboot every time when we delete raid device.
If same process with above is done in host itself, mdadm garbage files do not exist...

I need your help. Please ask me freely!

syg00 · 08-02-2022, 08:54 PM

Have a look at --remove. To save having to specify the device each time, maybe try "detached".

shellwedance · 08-03-2022, 12:46 PM

Quote:

Originally Posted by syg00

Have a look at --remove. To save having to specify the device each time, maybe try "detached".

I appreciate to your suggestion.
But unfortunately, it seems that --remove and "detached" do not work...

If I do --remove instead of --stop, the command does nothing like below.

Code:

## In privileged container,

$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
nvme0n1 259:0    0   11G  0 disk  
└─md126   9:126  0   11G  0 raid1 
nvme1n1 259:1    0   11G  0 disk  
└─md126   9:126  0   11G  0 raid1 
...
  
$ ls /dev/md*
/dev/md126  /dev/md127

/dev/md:
testvol

$ /sbin/mdadm --remove /dev/md126 detached

$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
nvme0n1 259:0    0   11G  0 disk  
└─md126   9:126  0   11G  0 raid1 
nvme1n1 259:1    0   11G  0 disk  
└─md126   9:126  0   11G  0 raid1 
...

$ ls /dev/md*
/dev/md126  /dev/md127

/dev/md:
testvol

If I do --remove after --stop, it cannot find raid device like below.

Code:

## In privileged container,

$ /sbin/mdadm --stop /dev/md126
mdadm: stopped /dev/md126

$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0    0   11G  0 disk 
nvme1n1 259:1    0   11G  0 disk 
...

$ ls /dev/md*
/dev/md126  /dev/md127

/dev/md:
testvol

$ /sbin/mdadm --remove /dev/md126 detached
mdadm: Cannot get array info for /dev/md126

May I ask which functionality exactly mdadm --remove with "detached" do?