LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices



Reply
 
Search this Thread
Old 02-26-2010, 09:11 AM   #1
sampappachan_nyc
Member
 
Registered: Jan 2009
Location: New York
Distribution: Red Hat, CentOS,Fedora, Ubuntu, SUSE, linux mint
Posts: 55

Rep: Reputation: 0
Lightbulb Device Mapper not recognizing RAID storage luns on RH5


Greetings Fellow Knights Of The Penguin Clan....!


I am having issues with device mapper seeing a 275GB Raid5 Lun from my SAN storage.
I'm using IBM 2145 San Volume Controller.



I am able to see a 40 GB Raid 10 device though..
########################################################################
fdisk -l |grep 'dm-*'
Disk /dev/dm-6: 42.9 GB, 42949672960 bytes
########################################################################



These are the error messages that i get..
########################################################################
kernel: device-mapper: table: 253:11: multipath: error getting device
kernel: device-mapper: ioctl: error adding target to table
########################################################################




Also i am suspecting lvm taking over and changing the luns to scsi devices.. because i am seeing about 19 40GB devices extra (/sdr...etc)
########################################################################
Device Boot Start End Blocks Id System

Disk /dev/sdr: 42.9 GB, 42949672960 bytes
64 heads, 32 sectors/track, 40960 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
########################################################################




I used to use SDD driver to boot from SAN..then we had the configuration in /etc/rc.sysinit for the sdd driver to boot before LVM.
########################################################################
# Starting SDD
/etc/init.d/sdd start

# LVM2 initialization
if [ -x /sbin/lvm.static -o -x /sbin/multipath.static -o -x /sbin/dmraid ]; then
if ! LC_ALL=C fgrep -q "device-mapper" /proc/devices 2>/dev/null ;

########################################################################



Should i do this here too? but in RH5 i see an entry before LVM initialization to initialize device mapper...
########################################################################
# Device mapper & related initialization
if ! LC_ALL=C fgrep -q "device-mapper" /proc/devices 2>/dev/null ; then
modprobe dm-mod >/dev/null 2>&1
########################################################################



I'm really lost here...any help would be appreciated.....





My multipath configuration (/etc/multipath.conf)
########################################################################
defaults {
polling_interval 30
failback immediate
no_path_retry 5
rr_min_io 100
path_checker tur
user_friendly_names yes
}


device {
vendor "IBM"
product "2145"
path_grouping_policy group_by_prio
prio_callout "/sbin/mpath_prio_alua /dev/%n"
}
#######################################################################


Here is a link to a book on device mapper..
------------------------------------------------------------------------
http://www.redhat.com/docs/manuals/e..._Multipath.pdf
------------------------------------------------------------------------

Thanks Alot !

Young Sky Walker..
 
Old 02-27-2010, 02:37 AM   #2
mesiol
Member
 
Registered: Nov 2008
Location: Lower Saxony, Germany
Distribution: CentOS, RHEL, Solaris 10, AIX, HP-UX
Posts: 731

Rep: Reputation: 137Reputation: 137
Hi

your information is a bit rare. Please post the output of

Code:
dmsetup -v status
and

Code:
multipath -ll
Are there really 19 40GB LUN's expected to be there?

Thanks
 
Old 02-27-2010, 05:18 PM   #3
kschmitt
Member
 
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96

Rep: Reputation: 23
A few thoughts, hopefully one will help.

Device mapper should run before LVM, because you want LVM to use the multipathed device.

At work we use a Clariion, and do (and should) see multipath errors on boot, since it's an active/passive device: multipath sees the connections, but can't communicate through them, so it fails them.
Once it's running, if a connections fails the Clarrion activates a previously passive connection. The multipather sees it can communicate through a connection that was previously failed, and starts communicating through it.
Check to see if your device is active/passive. Some errors may be expected.

I've never done boot-from-san, but I have used FC and ISCSI with multipath and linux, using LVM. For RHEL/CentOS 4/5 you don't need to muck about with anything in the init scripts, everything you need is in the standard tools, or in /etc/multipath.conf.

Without a reboot, sometimes you do need to run vgchange -ay for LVM to see the changes to your LUNs, or to recognise them when they are new.

Hope one of those helps.
 
Old 03-01-2010, 11:27 AM   #4
sampappachan_nyc
Member
 
Registered: Jan 2009
Location: New York
Distribution: Red Hat, CentOS,Fedora, Ubuntu, SUSE, linux mint
Posts: 55

Original Poster
Rep: Reputation: 0
HI Mesiol,

No I am not expecting 19 40GB Luns, but i have 5 X 40GB RAID10 luns though,


dmsetup -v status
#################################################################################################### #########################
# dmsetup -v status
Name: mpath2
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 7
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c1

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:64 A 0 65:0 A 0 E 0 2 0 8:16 A 0 8:208 A 0

Name: mpath1
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 6
Number of targets: 1
UUID: mpath-360050768018f032908000000000005bf

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:0 A 0 8:192 A 0 E 0 2 0 8:48 A 0 8:240 A 0

Name: vg00-tmp
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 4
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7OCSOpUMyQDWxuh7RC3sGQH5Mak9tZwFZ

0 2097152 linear

Name: vg00-home
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 1
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7BiHzA6meVlazlrYuHHZsBlLBxCIWwswX

0 4194304 linear

Name: vg00-usr
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 3
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar74i1YnBH4xte9kgK6QfyRG9gILienHkT0

0 20971520 linear

Name: vg00-swap
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 5
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7Qv5k4DooHm9QNcF7pwpzTviEmFgV8YZQ

0 75497472 linear

Name: vg00-var
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 2
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar7fBcgKpK1iqkpuPp94Zf5hO9YsUYoYHqz

0 10485760 linear

Name: vg00-root
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 0
Number of targets: 1
UUID: LVM-V4xC3iTbrBVHLnlfhdN95szvRVwNaar754YzXHRk55atcB7Z6ncGRgTyxJ11Ca2m

0 14680064 linear

Name: mpath5
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 10
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c2

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:176 A 0 65:112 A 0 E 0 2 0 8:128 A 0 65:64 A 0

Name: mpath4
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 9
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c0

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:112 A 0 65:48 A 0 E 0 2 0 8:160 A 0 65:96 A 0

Name: mpath3
State: ACTIVE
Read Ahead: 256
Tables present: LIVE
Open count: 0
Event number: 2
Major, minor: 253, 8
Number of targets: 1
UUID: mpath-360050768018f032908000000000005c3

0 83886080 multipath 2 0 0 0 2 1 A 0 2 0 8:32 A 0 8:224 A 0 E 0 2 0 8:80 A 0 65:16 A 0


#################################################################################################### ########################

multipath -ll
#################################################################################################### ########################
mpath2 (360050768018f032908000000000005c1) dm-7 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:1:1 sde 8:64 [active][ready]
\_ 1:0:1:1 sdq 65:0 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:0:1 sdb 8:16 [active][ready]
\_ 1:0:0:1 sdn 8:208 [active][ready]
mpath1 (360050768018f032908000000000005bf) dm-6 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:0:0 sda 8:0 [active][ready]
\_ 1:0:0:0 sdm 8:192 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:1:0 sdd 8:48 [active][ready]
\_ 1:0:1:0 sdp 8:240 [active][ready]
mpath5 (360050768018f032908000000000005c2) dm-10 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:3:2 sdl 8:176 [active][ready]
\_ 1:0:3:2 sdx 65:112 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:2:2 sdi 8:128 [active][ready]
\_ 1:0:2:2 sdu 65:64 [active][ready]
mpath4 (360050768018f032908000000000005c0) dm-9 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:2:1 sdh 8:112 [active][ready]
\_ 1:0:2:1 sdt 65:48 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:3:1 sdk 8:160 [active][ready]
\_ 1:0:3:1 sdw 65:96 [active][ready]
mpath3 (360050768018f032908000000000005c3) dm-8 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 0:0:0:2 sdc 8:32 [active][ready]
\_ 1:0:0:2 sdo 8:224 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:1:2 sdf 8:80 [active][ready]
\_ 1:0:1:2 sdr 65:16 [active][ready]
#################################################################################################### ########################

Thank You Fellow Knight..
 
Old 03-01-2010, 11:36 AM   #5
sampappachan_nyc
Member
 
Registered: Jan 2009
Location: New York
Distribution: Red Hat, CentOS,Fedora, Ubuntu, SUSE, linux mint
Posts: 55

Original Poster
Rep: Reputation: 0
Hi kschmitt,

Could you advise what setting i need to make in /etc/multipath.conf, also could you advise on what filter or types that i should use in /etc/lvm/lvm.conf

Currently i am using

filter = [ "r/disk/", "r/sd.*/", "a/.*/" ]

but failover is not working great with this..or does this setting matter for the failover..??


Thanks alot fellow Knight..
 
Old 03-04-2010, 12:13 PM   #6
kschmitt
Member
 
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96

Rep: Reputation: 23
Well, I'd leave lvm.conf alone. Use the standard tools with LVM, they are enough.

As far as multipath.conf, I keep mine pretty sparse, except for the EMC specific stuff (which I've clipped out, since it's irrelevant, since you don't have an EMC array). All mine really is, is a regular-expression based blacklist.

#multipath.conf

devnode_blacklist {
devnode "^sda[0-9]*"
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z][[0-9]*]"
devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}

Last edited by kschmitt; 03-04-2010 at 12:14 PM. Reason: re-wording
 
Old 03-05-2010, 10:10 AM   #7
sampappachan_nyc
Member
 
Registered: Jan 2009
Location: New York
Distribution: Red Hat, CentOS,Fedora, Ubuntu, SUSE, linux mint
Posts: 55

Original Poster
Rep: Reputation: 0
Greetings,

Thanks for the quick replies... i resolved the issue.


1. The extra storage that i couldn't see was an issue on the SAN the admin assigned the storage to the wrong node..!
2. Putting in a different filter and type in /etc/lvm/lvm.conf resolved other issues with failover

filter = [ "a/mapper/", "r/.*/" ]

types = [ "device-mapper", 1]



Thank You Fellow Knights..
 
  


Reply

Tags
device, devicemapper, error, kernel, multipath, table


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
mkrescue Fatal: device-mapper: only linear boot device supported Molly Slackware 5 01-06-2009 05:35 PM
device mapper with Intel Embedded RAID rmir Linux - Software 2 07-24-2008 10:28 AM
device mapper / multipath creating extra device, won't let me talk to the one i want chakkerz Linux - Server 1 03-16-2008 06:52 PM
SATA Raid controller failure - luns unmountable kfh Linux - Hardware 1 11-15-2007 10:37 AM
device-mapper: dm-linear: Device lookup failed tajo01 Linux - Kernel 1 05-18-2006 09:22 PM


All times are GMT -5. The time now is 11:50 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration