I am trying to setup a new RAID5 in my newly initialized DEBIAN 9.4. The motherboard is a GigaByte GA Z170N. I do not use the RAID options in the UEFI setup.

Until recent I had a RAID5 running with 4 x 2TB HDDs. Now I want to upgrade to 4x 8 TB HDDs. I made some faulty attemps initializing the new RAID5. The number of paramamters for MDADM is really enormeous.

My last strategy was:

Removing a possibly exitisting former version of the RAID

mdadm --remove /dev/md0

"Cleaning" the HDDs with

wipefs -af /dev/sda
wipefs -af /dev/sdb
wipefs -af /dev/sdc
wipefs -af /dev/sdd


Creating the new RAID5:

mdadm --create --verbose /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd --level=5 --raid-devices=4

At first glance everything looks fine.

Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md0 : active raid5 sdd[4] sdc[2] sdb[1] sda[0]
23441685504 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]
[>....................] recovery = 0.2% (21901724/7813895168) finish=644
.6min speed=201447K/sec
bitmap: 0/59 pages [0KB], 65536KB chunk

unused devices: <none>

But after restarting the server I get the following:

Every 2,0s: cat /proc/mdstat uw-srv-1: Sun Apr 22 12:45:44 2018

Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [ra
id10]
md127 : active (auto-read-only) raid5 sdc[2] sdb[1] sda[0] sdd[4]
23441685504 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]
bitmap: 0/59 pages [0KB], 65536KB chunk

unused devices: <none>



Anybody out there who can tell me what's wrong with it?

Regards
Nomos

If you are starting fresh and wiping the disks in the array then it is much easier to start over.

Be sure that the file sytem that was on the array is unmounted. Then clear the beginning. This is a bit much but works if you are erasing everything on it:

Check availability:

Then assemble the array, using md0 if there are no othe arrays on the machine:

Clear the newly created array and format it:

Grab the UUID and use that for the /etc/fstab entry.

Then you can save the output of the following to /etc/mdadm/mdadm.conf

Be sure to confirm all that with the manual pages on your particular system.

1 members found this post helpful.

Moin, Turbocapitalist

Thank you for your help und your input. I did as you suggested.

1. DDing all 4 HDDs


root@uw-srv-1:~# dd if=/dev/zero bs=1M count=100 of=/dev/sda
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,0337322 s, 3,1 GB/s
root@uw-srv-1:~# dd if=/dev/zero bs=1M count=100 of=/dev/sdb
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,0331339 s, 3,2 GB/s
root@uw-srv-1:~# dd if=/dev/zero bs=1M count=100 of=/dev/sdc
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,0334686 s, 3,1 GB/s
root@uw-srv-1:~# dd if=/dev/zero bs=1M count=100 of=/dev/sdd
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,0335101 s, 3,1 GB/s[/COLOR]

2. Zeroing the superblock:

root@uw-srv-1:~# mdadm --zero-superblock /dev/sda
mdadm: Couldn't open /dev/sda for write - not zeroing
root@uw-srv-1:~# mdadm --zero-superblock /dev/sdb
mdadm: Couldn't open /dev/sdb for write - not zeroing
root@uw-srv-1:~# mdadm --zero-superblock /dev/sdc
mdadm: Couldn't open /dev/sdc for write - not zeroing
root@uw-srv-1:~# mdadm --zero-superblock /dev/sdd
mdadm: Couldn't open /dev/sdd for write - not zeroing


3. Check availability:

NAME SIZE FSTYPE TYPE MOUNTPOINT
sda 7,3T disk
└─md127 21,9T raid5
└─md127p1 21,9T md
sdb 7,3T disk
└─md127 21,9T raid5
└─md127p1 21,9T md
sdc 7,3T disk
└─md127 21,9T raid5
└─md127p1 21,9T md
sdd 7,3T disk
└─md127 21,9T raid5
└─md127p1 21,9T md
nvme0n1 238,5G disk
├─nvme0n1p1 512M vfat part /boot/efi
├─nvme0n1p2 69,9G ext4 part /
├─nvme0n1p3 9,3G ext4 part /var
├─nvme0n1p4 31,9G swap part [SWAP]
├─nvme0n1p5 1,9G ext4 part /tmp
└─nvme0n1p6 125G ext4 part /home


4. Cause of this result I did this:

root@uw-srv-1:~# mdadm --stop /dev/md127
mdadm: stopped /dev/md127

5. Again:

root@uw-srv-1:~# mdadm --zero-superblock /dev/sda
mdadm: Unrecognised md component device - /dev/sda
root@uw-srv-1:~# mdadm --zero-superblock /dev/sdb
mdadm: Unrecognised md component device - /dev/sdb
root@uw-srv-1:~# mdadm --zero-superblock /dev/sdc
mdadm: Unrecognised md component device - /dev/sdc
root@uw-srv-1:~# mdadm --zero-superblock /dev/sdd
mdadm: Unrecognised md component device - /dev/sdd

6. Check availability - #2:

root@uw-srv-1:~# lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
NAME SIZE FSTYPE TYPE MOUNTPOINT
sda 7,3T disk
sdb 7,3T disk
sdc 7,3T disk
sdd 7,3T disk
nvme0n1 238,5G disk
├─nvme0n1p1 512M vfat part /boot/efi
├─nvme0n1p2 69,9G ext4 part /
├─nvme0n1p3 9,3G ext4 part /var
├─nvme0n1p4 31,9G swap part [SWAP]
├─nvme0n1p5 1,9G ext4 part /tmp
└─nvme0n1p6 125G ext4 part /home

7. Creating the RAID

root@uw-srv-1:~# mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 512K
mdadm: size set to 7813895168K
mdadm: automatically enabling write-intent bitmap on large array
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

8. DDing the newly crated array:

root@uw-srv-1:~# dd if=/dev/zero bs=1M count=100 of=/dev/md0
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,293413 s, 357 MB/s

9. Creating the filesystem:

root@uw-srv-1:~# mkfs -t ext4 /dev/md0
mke2fs 1.43.4 (31-Jan-2017)
Ein Dateisystems mit 5860421376 (4k) Blöcken und 366276608 Inodes wird erzeugt.
UUID des Dateisystems: 1fcfcb01-9038-41fc-a920-c5ab77c66551
Superblock-Sicherungskopien gespeichert in den Blöcken:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544, 1934917632,
2560000000, 3855122432, 5804752896

beim Anfordern von Speicher für die Gruppentabellen: erledigt[means: successfully done]
Inode-Tabellen werden geschrieben: erledigt [means: successfully done]
Das Journal (262144 Blöcke) wird angelegt: erledigt [means: successfully done]
Die Superblöcke und die Informationen über die Dateisystemnutzung werden
geschrieben: erledigt [means: successfully done]

10. Retrieving the metadata of /dev/md0

root@uw-srv-1:~# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sun Apr 22 15:39:59 2018
Raid Level : raid5
Array Size : 23441685504 (22355.73 GiB 24004.29 GB)
Used Dev Size : 7813895168 (7451.91 GiB 8001.43 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Sun Apr 22 16:04:13 2018
State : clean, degraded, recovering
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Rebuild Status : 3% complete

Name : uw-srv-1:0 (local to host uw-srv-1)
UUID : a509d083:6fc32c6e:3046d464:c79b38bc
Events : 285

Number Major Minor RaidDevice State
0 8 0 0 active sync /dev/sda
1 8 16 1 active sync /dev/sdb
2 8 32 2 active sync /dev/sdc
4 8 48 3 spare rebuilding /dev/sdd

11. Editing the mdadm.conf

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays
# ARRAY /dev/md/0 metadata=1.2 UUID=4831bbbc:1cfa82b4:a7d95521:f9e6947b name=uw-srv-1:0
ARRAY /dev/md0 metadata=1.2 UUID=a509d083:6fc32c6e:3046d464:c79b38bc name=uw-srv-1:0


12. Restarting the system

And again: the dev/md0 has vanished.

Every 2,0s: cat /proc/mdstat uw-srv-1: Sun Apr 22 16:39:26 2018

Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [ra
id10]
md127 : active (auto-read-only) raid5 sda[0] sdd[4] sdc[2] sdb[1]
23441685504 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]
bitmap: 0/59 pages [0KB], 65536KB chunk

unused devices: <none>

What the hell is it? Do you have any more ideas?

Regards

PS: I regret the format of this post. But I couldn`t find how to find other charaacter sets and os on

Re-check steps 8 through 11. It might be device md127 instead of md0 it seems.

If you want to keep fixed width text, you can put it between [code] [/code] tags.

1 members found this post helpful.

Moin, Turbocapitalist:

Another source in the WWW reports the following:

And there seems to be some change to do with /dev/md/* and /dev/md* which I don't yet understand. You need to update initramfs so it contains your mdadm.conf settings during boot.


After doing so I get the following results:

And also:

And look at this:

The remaining questions are:

1. How to change the state from (auto-read-only) to read/write?


2. Is my entry in mdadm.con correct?

Regards
Nomos

I think you'd get more out of a query: