Greetings Fellow Knights Of The Penguin Clan....!


I am having issues with device mapper seeing a 275GB Raid5 Lun from my SAN storage.
I'm using IBM 2145 San Volume Controller.



I am able to see a 40 GB Raid 10 device though..
########################################################################
fdisk -l |grep 'dm-*'
Disk /dev/dm-6: 42.9 GB, 42949672960 bytes
########################################################################



These are the error messages that i get..
########################################################################
kernel: device-mapper: table: 253:11: multipath: error getting device
kernel: device-mapper: ioctl: error adding target to table
########################################################################




Also i am suspecting lvm taking over and changing the luns to scsi devices.. because i am seeing about 19 40GB devices extra (/sdr...etc)
########################################################################
Device Boot Start End Blocks Id System

Disk /dev/sdr: 42.9 GB, 42949672960 bytes
64 heads, 32 sectors/track, 40960 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
########################################################################




I used to use SDD driver to boot from SAN..then we had the configuration in /etc/rc.sysinit for the sdd driver to boot before LVM.
########################################################################
# Starting SDD
/etc/init.d/sdd start

# LVM2 initialization
if [ -x /sbin/lvm.static -o -x /sbin/multipath.static -o -x /sbin/dmraid ]; then
if ! LC_ALL=C fgrep -q "device-mapper" /proc/devices 2>/dev/null ;

########################################################################



Should i do this here too? but in RH5 i see an entry before LVM initialization to initialize device mapper...
########################################################################
# Device mapper & related initialization
if ! LC_ALL=C fgrep -q "device-mapper" /proc/devices 2>/dev/null ; then
modprobe dm-mod >/dev/null 2>&1
########################################################################



I'm really lost here...any help would be appreciated.....





My multipath configuration (/etc/multipath.conf)
########################################################################
defaults {
polling_interval 30
failback immediate
no_path_retry 5
rr_min_io 100
path_checker tur
user_friendly_names yes
}


device {
vendor "IBM"
product "2145"
path_grouping_policy group_by_prio
prio_callout "/sbin/mpath_prio_alua /dev/%n"
}
#######################################################################


Here is a link to a book on device mapper..
------------------------------------------------------------------------
http://www.redhat.com/docs/manuals/e..._Multipath.pdf
------------------------------------------------------------------------

Thanks Alot !

Young Sky Walker..

Hi

your information is a bit rare. Please post the output of

and

Are there really 19 40GB LUN's expected to be there?

Thanks

A few thoughts, hopefully one will help.

Device mapper should run before LVM, because you want LVM to use the multipathed device.

At work we use a Clariion, and do (and should) see multipath errors on boot, since it's an active/passive device: multipath sees the connections, but can't communicate through them, so it fails them.
Once it's running, if a connections fails the Clarrion activates a previously passive connection. The multipather sees it can communicate through a connection that was previously failed, and starts communicating through it.
Check to see if your device is active/passive. Some errors may be expected.

I've never done boot-from-san, but I have used FC and ISCSI with multipath and linux, using LVM. For RHEL/CentOS 4/5 you don't need to muck about with anything in the init scripts, everything you need is in the standard tools, or in /etc/multipath.conf.

Without a reboot, sometimes you do need to run vgchange -ay for LVM to see the changes to your LUNs, or to recognise them when they are new.

Hope one of those helps.

HI Mesiol,

No I am not expecting 19 40GB Luns, but i have 5 X 40GB RAID10 luns though,


dmsetup -v status
#################################################################################################### #########################
# dmsetup -v status
Name: mpath2
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 7
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c1

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:64 A 0 65:0 A 0 E 0 2 0 8:16 A 0 8:208 A 0

Name: mpath1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 6
Number of targets: 1
UUID: mpath-360050768018f032908000000000005bf

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:0 A 0 8:192 A 0 E 0 2 0 8:48 A 0 8:240 A 0

Name: vg00-tmp
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 4
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7OCSOpUMyQDWxuh7RC3sGQH5Mak9tZwFZ

0 2097152 linear

Name: vg00-home
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 1
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7BiHzA6meVlazlrYuHHZsBlLBxCIWwswX

0 4194304 linear

Name: vg00-usr
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 3
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar74i1YnBH4xte9kgK6QfyRG9gILienHkT0

0 20971520 linear

Name: vg00-swap
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 5
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7Qv5k4DooHm9QNcF7pwpzTviEmFgV8YZQ

0 75497472 linear

Name: vg00-var
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 2
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7fBcgKpK1iqkpuPp94Zf5hO9YsUYoYHqz

0 10485760 linear

Name: vg00-root
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 0
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar754YzXHRk55atcB7Z6ncGRgTyxJ11Ca2m

0 14680064 linear

Name: mpath5
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 10
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c2

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:176 A 0 65:112 A 0 E 0 2 0 8:128 A 0 65:64 A 0

Name: mpath4
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 9
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c0

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:112 A 0 65:48 A 0 E 0 2 0 8:160 A 0 65:96 A 0

Name: mpath3
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 8
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c3

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:32 A 0 8:224 A 0 E 0 2 0 8:80 A 0 65:16 A 0


#################################################################################################### ########################

multipath -ll
#################################################################################################### ########################
mpath2 (360050768018f032908000000000005c1) dm-7 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:1:1 sde 8:64 [active][ready]
\_ 1:0:1:1 sdq 65:0 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:0:1 sdb 8:16 [active][ready]
\_ 1:0:0:1 sdn 8:208 [active][ready]
mpath1 (360050768018f032908000000000005bf) dm-6 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:0:0 sda 8:0 [active][ready]
\_ 1:0:0:0 sdm 8:192 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:1:0 sdd 8:48 [active][ready]
\_ 1:0:1:0 sdp 8:240 [active][ready]
mpath5 (360050768018f032908000000000005c2) dm-10 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:3:2 sdl 8:176 [active][ready]
\_ 1:0:3:2 sdx 65:112 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:2:2 sdi 8:128 [active][ready]
\_ 1:0:2:2 sdu 65:64 [active][ready]
mpath4 (360050768018f032908000000000005c0) dm-9 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:2:1 sdh 8:112 [active][ready]
\_ 1:0:2:1 sdt 65:48 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:3:1 sdk 8:160 [active][ready]
\_ 1:0:3:1 sdw 65:96 [active][ready]
mpath3 (360050768018f032908000000000005c3) dm-8 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:0:2 sdc 8:32 [active][ready]
\_ 1:0:0:2 sdo 8:224 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:1:2 sdf 8:80 [active][ready]
\_ 1:0:1:2 sdr 65:16 [active][ready]
#################################################################################################### ########################

Thank You Fellow Knight..

Hi kschmitt,

Could you advise what setting i need to make in /etc/multipath.conf, also could you advise on what filter or types that i should use in /etc/lvm/lvm.conf

Currently i am using

filter = [ "r/disk/", "r/sd.*/", "a/.*/" ]

but failover is not working great with this..or does this setting matter for the failover..??


Thanks alot fellow Knight..

Well, I'd leave lvm.conf alone. Use the standard tools with LVM, they are enough.

As far as multipath.conf, I keep mine pretty sparse, except for the EMC specific stuff (which I've clipped out, since it's irrelevant, since you don't have an EMC array). All mine really is, is a regular-expression based blacklist.

#multipath.conf

devnode_blacklist {
devnode "^sda[0-9]*"
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z][[0-9]*]"
devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}

Greetings,

Thanks for the quick replies... i resolved the issue.


1. The extra storage that i couldn't see was an issue on the SAN the admin assigned the storage to the wrong node..!
2. Putting in a different filter and type in /etc/lvm/lvm.conf resolved other issues with failover

filter = [ "a/mapper/", "r/.*/" ]

types = [ "device-mapper", 1]



Thank You Fellow Knights..