LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 12-22-2009, 02:44 PM   #1
moon300
LQ Newbie
 
Registered: Dec 2009
Posts: 5

Rep: Reputation: 0
Add disk to raid1 array in imsm container with mdadm (not losing data). 82801ER ICH5R


Hi,

imsm = Intel Matrix Storage Manager (called fakeraid by some)
82801ER = Intel chipset sata raid controller
Main question:
I have a imsm container created with mdadm. Inside it is 1 disk (sdb). I also added sdb to a raid1 array with 1 missing disk.
How do I add a new disk (sdc) to the *same* imsm container?
I've tried "mdadm --add /dev/md/imsm /dev/sdc", but this just seems to create a *new* imsm container.

My goal is to recreate imsm raid1 array from 2 new disks after 1 of the original 2 disks failed. The puzzle is to see if this is possible without data loss (so without using the raid bios, because that seems to destroy all data).

Why not just use linux software raid? I want to be able to use the disks in a dualboot configuration from the 'other' OS as well.
Much more details below.
I've searched a lot about this, but could find much information about imsm containers and how to practically use them.
Hope someone with mdadm knowledge and containers is able to help.

Greetings,
moon300

More details:
-------------
I'm trying to build an raid1 setup with two 1.5TB disks connected to ICH5R without losing data (just to see if it can be done - all data has been backed up).
Imagine a raid1 on ICH5R where 1 disk has failed, and you want to replace the failed disk. Then the array should rebuild without data loss. How can this be done?

history:
- 2 x 300GB SATA harddisks (A1 and A2) on Intel ICH5R in raid1 (under windows only)
1 disk for OS(es) on IDE (osdisk)
- Disk A2 failed
- Disabled Raid in BIOS (not the 'RAID' BIOS, but the normal BIOS)
- Bought 2 x 1.5TB SATA (B and C) harddisks
- Took failed disk A2 out, Put disk C in
- Installed Linux on osdisk (and left windows on it as well)
- Copied everything from disk A1 to C
- Took disk A1 out
- Put disk B in, then started the quest described below

- sdc = original 'with data', 1st partition mounted as /dwin, 2nd as /dlin
sdb = new = empty
- Create layout on new disk
copy the partition information to the new disk.
sfdisk -d /dev/sdc | sfdisk /dev/sdb
check with fdisk -l
(it has 2 partitions)
- Change partition type on new disk to raid-auto-detect-linux (fd)
fdisk /dev/sdb
t 1 fd
t 2 fd
w
- Make sure no raid info left from earlier:
mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb2
(if there are no remains already you should see "mdadm: Unrecognised md component device - /dev/sdb1")
- Create the container
- mdadm --create /dev/md/imsm --metadata=imsm --raid-disks=2 /dev/sdb1 missing
missing not allowed
Whatever I try: I cannot create the container with a missing disk
- mdadm --create /dev/md/imsm -e imsm --level=container -n 1 --force /dev/sdb
- CREATION OF CONTAINER SUCCEEDED
- mdadm --create /dev/md/myraid --level raid1 -n 2 /dev/sdb missing
- l /dev/md*
imsm -> ../md127 = the container
myraid -> ../md126 = the raid array with 1 missing
/dev/md126
/dev/md126p1
/dev/md126p2
/dev/md127
- cat /proc/mdstat
Personalities : [raid1]
md126 : active raid1 sdb[0]
1465136128 blocks super external:/md127/0 [2/1] [U_]
m1d127 : inactive sdb[0](S)
209 blocks super external:imsm
- fdisk /dev/md126
Device Boot Start End Blocks Id System
/dev/md126p1 1 89238 716800000 fd Linux raid autodetect
/dev/md126p2 89238 182401 748336001 fd Linux raid autodetect
- Create filesystem on new md126
- mkfs.ntfs /dev/md126p1 (slow)
- mkfs.xfs /dev/md126p2
- Mount both partitions and copy all data from sdc to new md126 (is build from 'sdb' and 'missing')
- mount /dev/md126p1 /mnt/win/
- mount /dev/md126p2 /mnt/lin/
- rsync -ax /dwin /mnt/win/
- rsync -ax /dlin /mnt/lin
- Add the sdc disk to the container (first unmount)
- umount /dwin
- umount /dlin
- fdisk /dev/sdc
t 1 fd
t 2 fd
w
- mdadm --add /dev/md/imsm /dev/sdc
mdadm: added /dev/sdc
- Check
- cat /proc/mdstat
Personalities : [raid1]
md126 : active raid1 sdb[0]
1465136128 blocks super external:/md127/0 [2/1] [U_]
md127 : inactive sdc[1](S) sdb[0](S)
2466 blocks super external:imsm
- Try to add sdc to array
- mdadm --add /dev/md/myraid /dev/sdc
mdadm: Cannot add disks to a 'member' array, perform this operation on the parent container
- mdadm --add /dev/md/myraid /dev/md/imsm
mdadm: Cannot add disks to a 'member' array, perform this operation on the parent container

Looking in the raid bios, I see:
Raid volumes:
ID name level strip size status bootable
0 myraid raid1(mirror) n/a 1397.2GB Rebuild yes
|_samsung HD154UI ...3411 Port0 1397.2GB Normal

Other-RAID disks:
Port Drive Model
1 samsung HD154UI ...3292 1397.2GB Unknown no

So - as expected: sdc is not in the same container.
 
Old 12-23-2009, 01:54 PM   #2
mostlyharmless
Senior Member
 
Registered: Jan 2008
Distribution: Slackware -current (multilib) with kernel 3.15.5
Posts: 1,500
Blog Entries: 12

Rep: Reputation: 155Reputation: 155
Quote:
- Create the container
- mdadm --create /dev/md/imsm --metadata=imsm --raid-disks=2 /dev/sdb1 missing
missing not allowed
Whatever I try: I cannot create the container with a missing disk
- mdadm --create /dev/md/imsm -e imsm --level=container -n 1 --force /dev/sdb
- CREATION OF CONTAINER SUCCEEDED
- mdadm --create /dev/md/myraid --level raid1 -n 2 /dev/sdb missing
I have no experience using mdadm 3.0 in it's new guise (I use dmraid for my "fakeraid"), but it seems as though you can't use "missing" when making a container, as the above error message indicates. When you forced it, that didn't solve the problem. See this bug report:
http://www.issociate.de/board/post/5...ug_report.html
 
Old 12-24-2009, 02:29 AM   #3
moon300
LQ Newbie
 
Registered: Dec 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi mostly,

thanks for your reply and the link to the bug report. Would dmraid be capable of what I'm trying to achieve? In that case I would be happy to use it - perhaps you have any pointers on that?

I've also been thinking to use the BIOS to re-create the volume, but it warns you will lose all data. Although I have a backup: there must be a way to replace a failed disk without having to copy everything 2x I hope. The quest continues... =]

Greetings,
moon300
 
Old 12-24-2009, 12:02 PM   #4
mostlyharmless
Senior Member
 
Registered: Jan 2008
Distribution: Slackware -current (multilib) with kernel 3.15.5
Posts: 1,500
Blog Entries: 12

Rep: Reputation: 155Reputation: 155
I don't think dmraid does anything like that; it just sets up what's already there.

I would think that in a Raid 1 setup that the bios would let you do what you want.

However, one strategy would be to rebuild the array on disk C using disk a1, then removing a1 and rebuilding the array with disk B. But, you say, disk a1 is only 300 G and disks b,c are 1.5 tb. You might be able to extend the container size in the bios after the fact, or use the space as a separate array.

Another strategy would be to make a "bare metal" backup of a1, installing disks b and c and then restoring your backup to the new array.

Hope that's helpful.
 
Old 01-05-2010, 10:16 AM   #5
moon300
LQ Newbie
 
Registered: Dec 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi mostlyharmless,

I didn't try to rebuild the array on disk C using disk a1 because of the size difference. The 'restore from backup' option was the last resort I was going to use, but after I created the container and the array (and it was done syncing), no data was lost.
So I managed to (re)create the container with the array in it, without data loss.
See for details next post.

Greetings,
moon300
 
Old 01-05-2010, 10:25 AM   #6
moon300
LQ Newbie
 
Registered: Dec 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Have been busy with celebrating new year (best wishes to all of you), so only post my progress now.
I managed to build the imsm container without data loss.
I did not manage to create the raidarray with a missing member and then add the member.
Finally, I just created the raidarray with both disks immediately, hence
the raidarray resynced.
Million dollar question: how could I have known which disk would be synced to which? If sdc would have been 'source material' for sdb, data loss would have happened?
(was it because sdb was member number 0? And if so: how could I have made sdc member number 0, in case sdc container the good material)

Anyways.. all looks good to the raid bios now, and also in the Intel Matrix Storage Console (running in windows).
But in windows the disks are not given a drive letter, in Disk Management I see:
Disk 0 Basic 1397.26GB Online | 683.59GB NTFS Healthy (Unknown Partition) | 713.67GB Healthy (Unknown Partition)
(see screenshots)

Anyone an idea why windows does not give the 2 partitions driver letters? Was my raidarray created with wrong information with mdadm?

Greetings,
moon300


More details:
- mdadm --stop /dev/md126
- mdadm --stop /dev/md127
Did *not* do: mdadm --zero-superblock /dev/sdb
- mdadm --create /dev/md/imsm -e imsm -n 2 /dev/sdb /dev/sdc
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdc appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 01:00:00 1970
mdadm: imsm unable to enumerate platform support
array may not be compatible with hardware/firmware
Continue creating array? y
mdadm: container /dev/md/imsm prepared.
- cat /proc/mdstat
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
418 blocks super external:imsm
- l /dev/md/imsm : lrwxrwxrwx. 1 root root 8 2009-12-24 10:35 /dev/md/imsm -> ../md127
- mdadm --stop /dev/md127
mdadm: stopped /dev/md127
- mdadm --create /dev/md/imsm_my -e imsm -n 2 /dev/sdb /dev/sdc
... mdadm: container /dev/md/imsm_my prepared.
- l /dev/md/imsm_my
lrwxrwxrwx. 1 root root 8 2009-12-24 10:39 /dev/md/imsm_my -> ../md127
- cat /proc/mdstat
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
418 blocks super external:imsm
unused devices: <none>
- mdadm --detail /dev/md127
/dev/md127:
Version : imsm
Raid Level : container
Total Devices : 2
Working Devices : 2
UUID : ffffffff:ffffffff:ffffffff:ffffffff
Member Arrays :
Number Major Minor RaidDevice
0 8 16 - /dev/sdb
1 8 32 - /dev/sdc
- fdisk /dev/md/imsm_my : Unable to read /dev/md/imsm_my
- mdadm -C /dev/md/myraid --level raid1 -n 2 /dev/sdb missing
mdadm: /dev/sdb appears to be part of a raid array:
level=container devices=0 ctime=Thu Jan 1 01:00:00 1970
mdadm: imsm unable to enumerate platform support
array may not be compatible with hardware/firmware
Continue creating array? y
mdadm: Creating array inside imsm container /dev/md/imsm_my
mdadm: array /dev/md/myraid started.
- cat /proc/mdstat
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
md126 : active raid1 sdb[0] 1465136128 blocks super external:/md127/0 [2/1] [U_]
super external:imsm
- mdadm --add /dev/md/myraid /dev/sdc: mdadm: Cannot add disks to a 'member' array, perform this operation on the parent container
- mdadm --add /dev/md126 /dev/sdc: mdadm: Cannot add disks to a 'member' array, perform this operation on the parent container
- mdadm --add /dev/md/imsm_my /dev/sdc: mdadm: Cannot open /dev/sdc: Device or resource busy
- mdadm --stop /dev/md/imsm_my: mdadm: stopped /dev/md/imsm_my
- cat /proc/mdstat
Personalities : [raid1]
unused devices: <none>
- mdadm --assemble --scan
mdadm: Container /dev/md/imsm0 has been assembled with 2 drives
mdadm: Started /dev/md/myraid_0 with 1 devices
- cat /proc/mdstat
Personalities : [raid1]
md126 : active raid1 sdb[0]
1465136128 blocks super external:/md127/0 [2/1] [U_]
md127 : inactive sdb[1](S) sdc[0](S)
2466 blocks super external:imsm
unused devices: <none>
- HOW TO ADD SDC TO MD126?
- http://osdir.com/ml/linux-raid/2009-10/msg00189.html
When disks have conflicting container memberships (same container ids
but incompatible member arrays) --update=uuid can be used to move
offenders to a new container id by changing 'orig_family_num'.
Note that this only supports random updates of the uuid as the actual
uuid is synthesized. We also need to communicate the new
'orig_family_num' value to all disks involved in the update. A new
field 'update_private' is added to struct mdinfo to allow this
information to be transmitted.
-
- mdadm --stop [all /dev/md...]
- mdadm --assemble --scan : created container with only sdb in it! So lets clean /dev/sdc
- did NOT do this: mdadm --zero-superblock /dev/sdc
- Instead decided to create the array without a missing device

- mdadm --create /dev/md/imsm -e imsm -n 2 /dev/sdb /dev/sdc
- mdadm -C /dev/md/myraid --level raid1 -n 2 /dev/sdb /dev/sdc
- cat /proc/mdstat
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
418 blocks super external:imsm
md126 : active raid1 sdc[1] sdb[0]
1465136128 blocks super external:/md127/0 [2/2] [UU]
[>....................] resync = 0.4% (6194368/1465136327) finish=270.5min speed=89880K/sec
- Remark,
so apparently its resyncing, but how would I have know which disk would be synced to which?
- mdadm -D /dev/md127
/dev/md127:
Version : imsm
Raid Level : container
Total Devices : 2
Working Devices : 2
UUID : 54182b74:bf818d68:c604e234:ffc99ad5
Member Arrays :
Number Major Minor RaidDevice
0 8 16 - /dev/sdb
1 8 32 - /dev/sdc
- mdadm --examine /dev/md127 (or /dev/md/imsm_my
/dev/md127:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.1.00
Orig Family : cd03a1df
Family : cd03a1df
Generation : 00000004
UUID : 54182b74:bf818d68:c604e234:ffc99ad5
Checksum : 13fa2f31 correct
MPB Sectors : 2
Disks : 2
RAID Devices : 1
Disk00 Serial : ...3411
State : active
Id : 00000000
Usable Size : 2930272654 (1397.26 GiB 1500.30 GB)
[myraid]:
UUID : 7379363f:25f2221b:52d6eca7:e49d8542
RAID Level : 1
Members : 2
This Slot : 0
Array Size : 2930272256 (1397.26 GiB 1500.30 GB)
Per Dev Size : 2930272654 (1397.26 GiB 1500.30 GB)
Sector Offset : 0
Num Stripes : 2142152810
Chunk Size : 64 KiB
Reserved : 0
Migrate State : migrating: initializing
Map State : normal <-- uninitialized
Dirty State : clean

Disk01 Serial : ...3292
State : active
Id : 00000000
Usable Size : 2930272654 (1397.26 GiB 1500.30 GB)

- mdadm --examine /dev/sdb /dev/sdc
/dev/sdb:
Magic : Intel Raid ISM Cfg Sig. Intel Raid ISM Cfg Sig.
Version : 1.1.00 1.1.00
Orig Family : cd03a1df cd03a1df
Family : cd03a1df cd03a1df
Generation : 00000004 00000004
UUID : 54182b74:bf818d68:c604e234:ffc99ad5 54182b74:bf818d68:c604e234:ffc99ad5
Checksum : 13fa2f31 correct 13fa2f31 correct
MPB Sectors : 2 same...
Disks : 2
RAID Devices : 1
Disk00 Serial : ...3411
State : active
Id : 00000000
Usable Size : 2930272654 (1397.26 GiB 1500.30 GB)
[myraid]:
UUID : 7379363f:25f2221b:52d6eca7:e49d8542 same...
RAID Level : 1
Members : 2
This Slot : 0
Array Size : 2930272256 (1397.26 GiB 1500.30 GB)
Per Dev Size : 2930272654 (1397.26 GiB 1500.30 GB)
Sector Offset : 0
Num Stripes : 2142152810
Chunk Size : 64 KiB
Reserved : 0
Migrate State : migrating: initializing
Map State : normal <-- uninitialized
Dirty State : clean

Disk01 Serial : ...3292 Disk00 Serial : ...3411
State : active State : active
Id : 00000000 Id : 00000000
Usable Size : 2930272654 (1397.26 GiB 1500.30 GB) Usable Size : 2930272654 (1397.26 GiB 1500.30 GB)
- date; cat /proc/mdstat
Thu Dec 24 12:03:25 CET 2009
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
418 blocks super external:imsm
md126 : active raid1 sdc[1] sdb[0]
1465136128 blocks super external:/md127/0 [2/2] [UU]
[>....................] resync = 4.2% (61862720/1465136327) finish=250.3min speed=93405K/sec
unused devices: <none>
Thu Dec 24 12:47:43 CET 2009 [====>................] resync = 20.6% (303198976/1465136327) finish=210.9min speed=91817K/sec
Thu Dec 24 13:46:50 CET 2009 [========>............] resync = 42.6% (625098880/1465136327) finish=152.4min speed=91856K/sec
Thu Dec 24 15:46:53 CET 2009
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
418 blocks super external:imsm
md126 : active raid1 sdc[1] sdb[0]
1465136128 blocks super external:/md127/0 [2/2] [UU]
[================>....] resync = 82.3% (1206595712/1465136327) finish=61.5min speed=70058K/sec
- date; cat /proc/mdstat
Fri Dec 25 00:14:12 CET 2009
Personalities : [raid1]
md127 : inactive sdc[1](S) sdb[0](S)
418 blocks super external:imsm
md126 : active raid1 sdc[1] sdb[0]
1465136128 blocks super external:/md127/0 [2/2] [UU]
- Rebooting, looking to the bootup screen, I see:
Raid volumes:
ID name level strip size status
0 myraid raid1(mirror) n/a 1397.2GB Normal
|_samsung HD154UI ...3411 Port0 1397.2GB Normal
|_samsung HD154UI ...3292 Port0 1397.2GB Normal
Non-RAID Disks:
None defined
- However windows does not give the partitions drive letters (see screenshot)
Attached Images
File Type: png 2010-01-02_matrixStorageManager.png (24.2 KB, 4 views)
File Type: png 2010-01-02_diskManagement.png (55.0 KB, 5 views)
 
Old 01-06-2010, 02:00 PM   #7
mostlyharmless
Senior Member
 
Registered: Jan 2008
Distribution: Slackware -current (multilib) with kernel 3.15.5
Posts: 1,500
Blog Entries: 12

Rep: Reputation: 155Reputation: 155
Quote:
Million dollar question: how could I have known which disk would be synced to which? If sdc would have been 'source material' for sdb, data loss would have happened?
I don't know: perhaps one disk is the default, perhaps the newest information is the source if the two disks differ...

Quote:
Anyone an idea why windows does not give the 2 partitions driver letters?
Usually Windows does that, but it is not automatic with a new file system made elsewhere. It's got nothing to do with mdadm. Just like with a network drive in Windows, you can assign a letter to the filesystem. Just go into Disk Management; I think* if you right click on the filesystem and get Properties it'll give you an option to assign a drive letter.

*=not sure if that's where you need to click, haven't done that in a while but I'm positive that there's an option to do it.
 
Old 01-06-2010, 03:39 PM   #8
moon300
LQ Newbie
 
Registered: Dec 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by mostlyharmless View Post
I think* if you right click on the filesystem and get Properties it'll give you an option to assign a drive letter.
*=not sure if that's where you need to click, haven't done that in a while but I'm positive that there's an option to do it.
Yep I know, normally that's how you can then change the driveletter. But I can't set a new driveletter. Will make screenshot when in windows some day again.

Greetings!
moon300
 
  


Reply

Tags
ich5r, mdadm, raid1


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
raid1 mdadm repair degraded array with used good hard drive catbird Linux - Hardware 7 07-09-2009 12:31 AM
gutsy + mdadm + raid1 => hardy = 1TB of lost (corrupt?) data! LeeSimpson Ubuntu 1 05-21-2008 05:47 AM
gutsy + mdadm + raid1 => hardy = 1TB of lost (corrupt?) data! LeeSimpson Ubuntu 0 05-21-2008 01:55 AM
Degraded raid1 array problem on slack 12.0.0 / mdadm slack-12.0.0 Slackware 5 10-12-2007 06:36 AM
add a new disk to raid1 array retrev Linux - General 1 04-08-2007 01:57 AM


All times are GMT -5. The time now is 04:32 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration