Reshaping Raid6 does not star

switcher51 · 09-28-2015, 06:36 AM

I was using a raid array of 4 drives (3 TB) and tried to add 2 more drives (3 TB).
I partitioned the disks:

Disk /dev/sde: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): F5574F3B-19B1-420F-AAD5-F559F518C3F1
Partition table holds up to 128 entries
First usable sector is 2048, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 911 sectors (455.5 KiB)

Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB EF02
2 4096 8450047 4.0 GiB 0700 Microsoft basic data
3 8450048 29415423 10.0 GiB EF00 EFI System
4 29415424 50380799 10.0 GiB FD00 Linux RAID
5 50380800 71360511 10.0 GiB FD00 Linux RAID
6 71360512 5860532223 2.7 TiB FD00 Linux RAID

and

Disk /dev/sdf: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): A12B3C9A-162F-4973-B078-5DF0C97148F7
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB EF02 primary
2 4096 8450047 4.0 GiB 0700 primary
3 8450048 29415423 10.0 GiB EF00 EFI System
4 29415424 50380799 10.0 GiB FD00 Linux RAID
5 50380800 71360511 10.0 GiB FD00 Linux RAID
6 71360512 5860532223 2.7 TiB FD00 Linux RAID

Then I added them to raid array:

mdadm /dev/md127 -a /dev/sde6 /dev/sde6
mdadm /dev/md127 -a /dev/sde6 /dev/sdf6

I got two spare divices.
So I tried to grow the array:

mdadm --grow --raid-devices=6 /dev/md127

That is the result:

mdadm -D /dev/md127

/dev/md127:
Version : 1.0
Creation Time : Sat Jan 10 19:49:58 2015
Raid Level : raid6
Array Size : 5789171200 (5520.98 GiB 5928.11 GB)
Used Dev Size : 2894585600 (2760.49 GiB 2964.06 GB)
Raid Devices : 6
Total Devices : 6
Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Sun Sep 27 21:13:40 2015
State : clean, reshaping
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 128K

Reshape Status : 0% complete
Delta Devices : 2, (4->6)

Name : any:lvm
UUID : b1b8b834:53750cf0:63e16d72:f4e5c91f
Events : 18400

Number Major Minor RaidDevice State
0 8 6 0 active sync /dev/sda6
1 8 22 1 active sync /dev/sdb6
2 8 38 2 active sync /dev/sdc6
3 8 54 3 active sync /dev/sdd6
5 8 86 4 active sync /dev/sdf6
4 8 70 5 active sync /dev/sde6

cat /proc/mdstat[INDENT]
Personalities : [raid6] [raid5] [raid4]
md127 : active raid6 sdf6[5] sde6[4] sda6[0] sdb6[1] sdc6[2] sdd6[3]
5789171200 blocks super 1.0 level 6, 128k chunk, algorithm 2 [6/6] [UUUUUU]
[>....................] reshape = 0.0% (0/2894585600) finish=103002019460.0min speed=0K/sec
bitmap: 2/22 pages [8KB], 65536KB chunk

System Log a day after the change:

2015-09-27T11:32:03.758966+02:00 xix kernel: [131453.189195] md: bind<sdf6>
2015-09-27T11:32:03.852942+02:00 xix kernel: [131453.282659] md127: unknown partition table
2015-09-27T11:32:03.872956+02:00 xix kernel: [131453.302932] RAID conf printout:
2015-09-27T11:32:03.872967+02:00 xix kernel: [131453.302939] --- level:6 rd:4 wd:4
2015-09-27T11:32:03.872968+02:00 xix kernel: [131453.302943] disk 0, o:1, dev:sda6
2015-09-27T11:32:03.872969+02:00 xix kernel: [131453.302946] disk 1, o:1, dev:sdb6
2015-09-27T11:32:03.872969+02:00 xix kernel: [131453.302948] disk 2, o:1, dev:sdc6
2015-09-27T11:32:03.872970+02:00 xix kernel: [131453.302951] disk 3, o:1, dev:sdd6

My interpretation:

The array changed from 4 to 6 devices
The Size of the array did noct change: it remains 5789171200 KB
Reshaping did not really start: 0% or 0/2894585600 after 48 hours, there is no activity!
The raid array is still working, the system is running

My Questions:

How show I proceed best?
Is there a lock preventing reshape to work?
I did not reboot after partitioniing the disks - may this be part of the problem!?
- Shall I shutdown and reboot the system?
- Shall I unmout the file systems?
- Shall I stop the raid array?
- Shall I set sde6 and sdf6 to failed?
- Sahll I recreate the raid array?
Are my daily backups on a separate device save? Or may they be corrupt?

Of cause I can provide mor information if needed.

Many thanks in advance for help!

Volker

smallpond · 09-28-2015, 03:06 PM

According to the RAID wiki "check to see if you are using a write-intent bitmap. If you are you must remove the bitmap first, grow the array, then re-add the bitmap."

switcher51 · 09-29-2015, 04:07 PM

Thanks for that hint.
I recreated the raid array and checked it.
That took more than 12 hours but was successfull.