LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
Reload this Page MDADM fails to hot remove disk
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-22-2008, 03:58 PM   #1
alirezan1
Member
 
Registered: Nov 2004
Location: Vancouver
Distribution: Ubunty, CentOS ,Mandriva, Gentoo, RedHat, Fedora, Knoppix
Posts: 150

Rep: Reputation: 15
MDADM fails to hot remove disk

Hi there,

I created a RAID0 array with 2 disks:

root@revolution1:/root> cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [multipath] [faulty]
md0 : active raid0 sdb1[0] sda2[1]
93771200 blocks 64k chunks


Everything works great and I marked one of my disks as failed:

root@revolution1:/root> mdadm --manage /dev/md0 --fail /dev/sda2
mdadm: set /dev/sda2 faulty in /dev/md0


So, I take it as MDADM successfully marked my device as failed, right? /proc/mdstat shows:


root@revolution1:/root> cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [multipath] [faulty]
md0 : active raid0 sdb1[0] sda2[1]
93771200 blocks 64k chunks

unused devices: <none>

and

root@revolution1:/root> mdadm -E /dev/sda2
/dev/sda2:
Magic : a92b4efc
Version : 00.90.00
UUID : e3128c16:ee63fcad:a4135057:27c78241
Creation Time : Sat Jan 1 00:49:32 2000
Raid Level : raid0
Used Dev Size : 0
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0

Update Time : Sat Jan 1 01:00:48 2000
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 337 - expected b7584bde
Events : 0.3

Chunk Size : 64K

Number Major Minor RaidDevice State
this 1 8 2 1 active sync /dev/sda2

0 0 8 17 0 active sync /dev/sdb1
1 1 8 2 1 active sync /dev/sda2



When I want to hot-remove my disk, I get the error message that the device is busy.

By the way, I have already formatted my md0 and it's not mounted:

root@revolution1:/root> mdadm --manage /dev/md0 --remove /dev/sda2
mdadm: hot remove failed for /dev/sda2: Device or resource busy

root@revolution1:/root> mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw)
tmpfs on /dev/shm type tmpfs (rw)
rpc_pipefs on /drbd/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)


Why is that? Can anybody please help me?

Thanks
 
Old 07-22-2008, 07:17 PM   #2
kenoshi
Member
 
Registered: Sep 2007
Location: SF Bay Area, CA
Distribution: CentOS, SLES 10+, RHEL 3+, Debian Sarge
Posts: 159

Rep: Reputation: 32
You need to --manage --fail before you remove it.

Hope this helps.
 
Old 07-22-2008, 08:17 PM   #3
kenoshi
Member
 
Registered: Sep 2007
Location: SF Bay Area, CA
Distribution: CentOS, SLES 10+, RHEL 3+, Debian Sarge
Posts: 159

Rep: Reputation: 32
Haha sorry, I should of read the whole thing more carefully.

You have a RAID-0 array...the command:

mdadm --manage /dev/md0 --fail /dev/sda2

Does not work on RAID-0 devices.

When you cat /proc/mdstat, the failed drive would have been marked like this if it worked:

md0 : active raid0 sdb1[0] sda2[1](F)

And not:

md0 : active raid0 sdb1[0] sda2[1]

Reason why this doesn't work is that soon as you fail a drive in a RAID-0 array, you would have corrupted the array. This was put in as a safety feature to prevent accidental failing a member disk, which results in a corrupted RAID-0 array (trust me, human error is a terrible thing).

That's why your --manage --remove command doesn't work...it makes no sense to "hot remove" a drive in a RAID-0 array, because once a member drive becomes faulty, you have to replace and build a new array anyway as all data is lost...you can only hot swap a disk in a RAID1,3,4,5,6 and nested arrays.

I suppose you can try forcing it, but I doubt it'll work.

Hope his helps.
Last edited by kenoshi; 07-22-2008 at 08:20 PM.
 
Old 09-28-2008, 09:26 AM   #4
vikas027
Senior Member
 
Registered: May 2007
Location: Sydney
Distribution: RHEL, CentOS, Ubuntu, Debian, OS X
Posts: 1,305

Rep: Reputation: 107Reputation: 107
Question I Am Also Getting The Same Error !!
Hi All,

I am using RHEL 5.0 and new to RAID Levels.

I am unable to remove a disk by setting it "faulty".

These are the commands, and files which I am using.


Code:
[root@vikas_vicky /]# cat /proc/mdstat
Personalities : [linear]
md0 : active linear hda6[2] hdc5[1] hda5[0]
      3256896 blocks 64k rounding

unused devices: <none>

Code:
[root@vikas_vicky /]# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Mon Sep 29 00:41:01 2008
     Raid Level : linear
     Array Size : 3256896 (3.11 GiB 3.34 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Sep 29 00:41:01 2008
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

       Rounding : 64K

           UUID : 31f20cd4:0122f710:a0624d77:41e2ee92
         Events : 0.1

    Number   Major   Minor   RaidDevice State
       0       3        5        0      active sync   /dev/hda5
       1      22        5        1      active sync   /dev/hdc5
       2       3        6        2      active sync   /dev/hda6
[root@vikas_vicky /]#

Code:
[root@vikas_vicky /]# mdadm --manage /dev/md0 --fail /dev/hda6
mdadm: set /dev/hda6 faulty in /dev/md0
Code:
[root@vikas_vicky /]# umount /dev/md0
umount: /dev/md0: not mounted

Code:
[root@vikas_vicky /]#  mdadm --manage /dev/md0 --remove /dev/hda6
mdadm: hot remove failed for /dev/hda6: Device or resource busy
Code:
[root@vikas_vicky /]# mount
/dev/hda1 on / type ext3 (rw,acl,usrquota,grpquota)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@vikas_vicky /]#

PLEASE HELP !!

Regards,
VIKAS
 
Old 09-29-2008, 07:02 PM   #5
alirezan1
Member
 
Registered: Nov 2004
Location: Vancouver
Distribution: Ubunty, CentOS ,Mandriva, Gentoo, RedHat, Fedora, Knoppix
Posts: 150

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by vikas027 View Post
Hi All,

I am using RHEL 5.0 and new to RAID Levels.

I am unable to remove a disk by setting it "faulty".

These are the commands, and files which I am using.


Code:
[root@vikas_vicky /]# cat /proc/mdstat
Personalities : [linear]
md0 : active linear hda6[2] hdc5[1] hda5[0]
      3256896 blocks 64k rounding

unused devices: <none>

Code:
[root@vikas_vicky /]# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Mon Sep 29 00:41:01 2008
     Raid Level : linear
     Array Size : 3256896 (3.11 GiB 3.34 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Sep 29 00:41:01 2008
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

       Rounding : 64K

           UUID : 31f20cd4:0122f710:a0624d77:41e2ee92
         Events : 0.1

    Number   Major   Minor   RaidDevice State
       0       3        5        0      active sync   /dev/hda5
       1      22        5        1      active sync   /dev/hdc5
       2       3        6        2      active sync   /dev/hda6
[root@vikas_vicky /]#

Code:
[root@vikas_vicky /]# mdadm --manage /dev/md0 --fail /dev/hda6
mdadm: set /dev/hda6 faulty in /dev/md0
Code:
[root@vikas_vicky /]# umount /dev/md0
umount: /dev/md0: not mounted

Code:
[root@vikas_vicky /]#  mdadm --manage /dev/md0 --remove /dev/hda6
mdadm: hot remove failed for /dev/hda6: Device or resource busy
Code:
[root@vikas_vicky /]# mount
/dev/hda1 on / type ext3 (rw,acl,usrquota,grpquota)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@vikas_vicky /]#

PLEASE HELP !!

Regards,
VIKAS
I think your problem is that
1)you're trying to remove the failed disk way to quickly and MDADM has not released it yet (happened to me alot). In this case, just add it back, mark it again as failed, wait for 5-6 seconds, then remove it.

2) Depending on your redundancy level, you can't remove the failed disk or your data will be corrupted. When you mark the device as FAILED, print /proc/mdstat output and post it here so we can take a look at it.

Good luck
 
Old 09-29-2008, 07:51 PM   #6
alirezan1
Member
 
Registered: Nov 2004
Location: Vancouver
Distribution: Ubunty, CentOS ,Mandriva, Gentoo, RedHat, Fedora, Knoppix
Posts: 150

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by vikas027 View Post
Hi All,

I am using RHEL 5.0 and new to RAID Levels.

I am unable to remove a disk by setting it "faulty".

These are the commands, and files which I am using.


Code:
[root@vikas_vicky /]# cat /proc/mdstat
Personalities : [linear]
md0 : active linear hda6[2] hdc5[1] hda5[0]
      3256896 blocks 64k rounding

unused devices: <none>

Code:
[root@vikas_vicky /]# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Mon Sep 29 00:41:01 2008
     Raid Level : linear
     Array Size : 3256896 (3.11 GiB 3.34 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Sep 29 00:41:01 2008
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

       Rounding : 64K

           UUID : 31f20cd4:0122f710:a0624d77:41e2ee92
         Events : 0.1

    Number   Major   Minor   RaidDevice State
       0       3        5        0      active sync   /dev/hda5
       1      22        5        1      active sync   /dev/hdc5
       2       3        6        2      active sync   /dev/hda6
[root@vikas_vicky /]#

Code:
[root@vikas_vicky /]# mdadm --manage /dev/md0 --fail /dev/hda6
mdadm: set /dev/hda6 faulty in /dev/md0
Code:
[root@vikas_vicky /]# umount /dev/md0
umount: /dev/md0: not mounted

Code:
[root@vikas_vicky /]#  mdadm --manage /dev/md0 --remove /dev/hda6
mdadm: hot remove failed for /dev/hda6: Device or resource busy
Code:
[root@vikas_vicky /]# mount
/dev/hda1 on / type ext3 (rw,acl,usrquota,grpquota)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@vikas_vicky /]#

PLEASE HELP !!

Regards,
VIKAS

I just noticed you have a "linear" RAID device. You can't remove partitions from a Linear RAID (RAID-0). Because when you have a linear RAID, you have practically merged two disks and created a larger disk. You can't simply throw away part of your "new" hard disk. You will lose data and MDADM wants to make sure you don't lose data => you can't remove/fail.

RAID-0 doesn't offer any redundancy and failure protection either.

Ali
 
Old 09-30-2008, 03:35 AM   #7
vikas027
Senior Member
 
Registered: May 2007
Location: Sydney
Distribution: RHEL, CentOS, Ubuntu, Debian, OS X
Posts: 1,305

Rep: Reputation: 107Reputation: 107
Smile
Quote:
Originally Posted by alirezan1 View Post
I just noticed you have a "linear" RAID device. You can't remove partitions from a Linear RAID (RAID-0). Because when you have a linear RAID, you have practically merged two disks and created a larger disk. You can't simply throw away part of your "new" hard disk. You will lose data and MDADM wants to make sure you don't lose data => you can't remove/fail.

RAID-0 doesn't offer any redundancy and failure protection either.

Ali

Thank you so much for the useful information alirezan1, thanks a TON !!
I am new to RAIDS you see, But surely this community and its users are awesome.
 
Old 03-21-2012, 04:11 AM   #8
Chinnu16
LQ Newbie
 
Registered: Mar 2012
Posts: 1

Rep: Reputation: Disabled
Here is the command to remove the RAID0

mdadm --manage /dev/md0 --stop
 
Old 03-21-2012, 09:34 AM   #9
vikas027
Senior Member
 
Registered: May 2007
Location: Sydney
Distribution: RHEL, CentOS, Ubuntu, Debian, OS X
Posts: 1,305

Rep: Reputation: 107Reputation: 107
Quote:
Originally Posted by Chinnu16 View Post
Here is the command to remove the RAID0

mdadm --manage /dev/md0 --stop
Hi Chinnu16,

Thanks for the command but please dont wake up threads which are years old. Now, I am pretty well versed with RAID.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
help with mdadm disk failure ufmale Linux - Server 3 05-29-2008 09:59 AM
rhel5 raid device cannot remove, no /etc/mdadm.conf file? hocheetiong Linux - Newbie 1 11-20-2007 06:08 AM
RAID-1 with mdadm. Disk fails sometime. jostmart Linux - Server 5 08-15-2007 05:13 AM
mdadm fails to assemble my RAID device tomhildebrand Fedora 6 06-28-2007 12:08 AM
Trying to remove current array and create a new one using mdadm ghostz00 Linux - Hardware 0 12-09-2006 09:47 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 02:55 AM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration