Hello Everyone

Im working on a RHEL 5.4 x64 system.

I have 2 internal drives on a RAID configuration to support my / partition and swap.

On the other hand i have and old storage IBM EXP400 on a raid 5 with 1449 GB holding my /home in a VG.

Due to my /home partition was almost reaching full capacity i decided to add an extra storage, this time with a newer storage model , IBM DS4700 with a raid 5 with 2047 GB.

The DS4700 is a FC dual controller, so i had to add a dual FC hba emulex to the system as well as the RDAC driver to make multipath available.

At the time i try to configure my newly device (/dev/sdc) im not able to mount it or format it , either as a lvm or as a common ext2 or 3 type

Commands output:

df -h

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 26G 19G 5.3G 78% /
/dev/mapper/VolGroup00-LogVol00
1.4T 1.2T 124G 91% /home
tmpfs 2.0G 0 2.0G 0% /dev/shm

------------------------------------------------------------------------

fdisk -l

Disk /dev/sda: 36.4 GB, 36401315840 bytes
255 heads, 63 sectors/track, 4425 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 3393 27254241 83 Linux
/dev/sda2 3394 4425 8289540 82 Linux swap / Solaris

Disk /dev/sdb: 1499.9 GB, 1499998453760 bytes
255 heads, 63 sectors/track, 182364 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 182364 1464838798+ 8e Linux LVM

Disk /dev/sdc: 2047.8 GB, 2047868928000 bytes
255 heads, 63 sectors/track, 248972 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 248972 1999867558+ 83 Linux

Disk /dev/dm-0: 2047.8 GB, 2047868928000 bytes
255 heads, 63 sectors/track, 248972 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/dm-0p1 1 248972 1999867558+ 83 Linux

-------------------------------------------------------------------------

file -s

/dev/sda: x86 boot sector; partition 1: ID=0x83, active, starthead 1, startsector 63, 54508482 sectors; partition 2: ID=0x82, starthead 254, startsector 54508545, 16579080 sectors, code offset 0x48
/dev/sda1: Linux rev 1.0 ext3 filesystem data (needs journal recovery) (large files)
/dev/sda2: Linux/i386 swap file (new style) 1 (4K pages) size 2072384 pages Label SWAP-sda2
/dev/sdb: x86 boot sector; partition 1: ID=0x8e, active, starthead 1, startsector 63, 2929677597 sectors
/dev/sdb1: LVM2 (Linux Logical Volume Manager) , UUID: DAwwpeS9MAdTDVBfVWNmDTaMiXCs7pm
/dev/sdc: x86 boot sector; partition 1: ID=0x83, starthead 1, startsector 63, 3999735117 sectors, extended partition table (last)\011
/dev/sdc1: x86 boot sector; partition 1: ID=0x83, starthead 2, startsector 63, 3999735054 sectors, extended partition table (last)\011

----------------------------------------------------------------------------------------------------------------------------
mkfs.ext3 /dev/sdc1

mke2fs 1.39 (29-May-2006)
/dev/sdc1 is apparently in use by the system; will not make a filesystem here!

----------------------------------------------------------------------------------------------------------------------------

mount |grep sd

/dev/sda1 on / type ext3 (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)

(sdc is not mounted at all)

----------------------------------------------------------------------------------------------------------------------------
cat /proc/partitions

major minor #blocks name

8 0 35548160 sda
8 1 27254241 sda1
8 2 8289540 sda2
8 16 1464842240 sdb
8 17 1464838798 sdb1
8 32 1999872000 sdc
8 33 1999867558 sdc1
253 0 1999872000 dm-0
253 1 1464827904 dm-1


Any thoughts on why the system keeps saying the partition is in use? i have tried fuser -ck /dev/sdc1 and still im unable to mount it, also i partioned once sdc as a lvm and tried to create a pv with sdc and has a similar message to format one: Can't open /dev/sdc1 exclusively. Mounted filesystem?

Doesn't the device have firmware controlling disk access ? And software to manage the disks ?

The Storage controller does have firmware and software to manage the disks, This is the way the array and mapping was done to the server, the problem seems to be now at the OS level which sees the new device as it is used when it is a 100% new device

Should sdc1 be listed as a boot sector ? Neither sda or sdb listing look like that

I agree that sda and sdb dpesmt looks like sdc1, the problem is that i didnt set those values, so How can i possibly change them? im stuck since i dont know if this is a problem with my storage or with my fc card config, or The OS, i have done this before with qlogic adapters, and devices works just fine, by reading the RDAC driver documentation it says Emulex hbas should work as well so, no way to go by now, i could say i ve tried almost anything and im just able to see my device is present but not configurable at all

Hi, seems you got the same problem I am solving right now ...

The problem here is (I think) that OS Multipath is "stealing" the (already multipathed device created by RDAC) and trying to make another failover layer/device ...

If you take a look at /dev/mpath/* and you see some device[s] that means that this is the case, right now Im in the proces of analyzing why the heck it is doint it.

Simple solution is disabling the devices created by OS multipath, the the MPP device is free again and you can work with it normaly ...

# pvcreate /dev/sdb
Can't open /dev/sdb exclusively. Mounted filesystem?

# cat /proc/partitions
major minor #blocks name

8 0 143247360 sda
8 1 104391 sda1
8 2 143139150 sda2
8 16 292444160 sdb
253 0 20840448 dm-0
253 1 5111808 dm-1
253 2 5111808 dm-2
253 3 5111808 dm-3
253 4 4096000 dm-4
253 5 292444160 dm-5

#ls /dev/mpath/*
/dev/mpath/3600a0bxxxxxxxxxxxxxxx6fb50

# dmsetup status
RootVG-swap: 0 8192000 linear
RootVG-root: 0 10223616 linear
RootVG-root: 10223616 31457280 linear
mpath8: 0 584888320 multipath 2 0 0 0 1 1 A 0 1 0 8:16 A 0
RootVG-tmp: 0 10223616 linear
RootVG-usr: 0 10223616 linear
RootVG-var: 0 10223616 linear

# dmsetup ls
RootVG-swap (253, 4)
RootVG-root (253, 0)
mpath8 (253, 5)
RootVG-tmp (253, 3)
RootVG-usr (253, 2)
RootVG-var (253, 1)

# dmsetup remove mpath8
# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created

... voila

but how to make this persistent Im not sure yet .. will post after I get to it ...

Well this was quite easy ... one has to blacklist the mpath in its configuration file (simple remove from init doesnt do the job since it is started by device-mapper anyway). So ...

/etc/multipathd.conf:

blacklist {
devnode "*"
}

Since MPIO and RDAC cant cooexist I blacklisted all devices ...

Then mppUpdate, Reboot .. and thats that

1 members found this post helpful.

You are right, this solve the whole thing, just changing that simple file, i tried lots of stuff and even think on change to qlogic, Thanks

I solved this problem in Centos by running

i_m_mahmoud:
Solved for me
--------------------------------------------------------------------------------

I solved this problem in Centos by running

Quote:

/sbin/dmsetup remove_all
==============================
it is ok. thanks.

Hi friends

I got the same problem cannot open /dev/sdd1 exclusively. mounted filesystem ?

no need of take headk

just do the following steps

1:-find all the physical volumes

pvs -o +uuid

2:-
pvremove -ffy /dev/sdd

Hi I was reading this and found a solution elsewhere that I think may help. I need to make this post in order for the forum to allow me to post a URL.

---------- Post added 05-31-13 at 10:34 AM ----------

here is another possibility if you are using multipath: blacklisting the drive in multipath.conf

http://www.centos.org/docs/5/html/5....procedure.html

Thanks to All
solves all my problems.

I was unable to format / delete an lvm partition

This solved my problem, thanks