LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 10-13-2017, 08:21 AM   #1
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Rep: Reputation: 0
URGENT problem with Multipath partitions not present on reboot - CentOS 7


Hi,

I have configured multipath, well attempted to on CentOS7, all was fine until I rebooted the system and all of my partitions had disappeared? I have this server in production so rebooting is difficult. OK here is my config:

# cd /dev/mapper
# ll
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpatha -> ../dm-3
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpatha1 -> ../dm-7
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 mpatha1p1 -> ../dm-12
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 mpatha1p1p1 -> ../dm-13
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathb -> ../dm-4
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathb1 -> ../dm-8
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 mpathb1p1 -> ../dm-15
lrwxrwxrwx. 1 root root 8 Oct 12 10:09 mpathb2 -> ../dm-10
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathc -> ../dm-2
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathc1 -> ../dm-6
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathc1p1 -> ../dm-17
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathc1p1p1 -> ../dm-18
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathd -> ../dm-5
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathd1 -> ../dm-9
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathd1p1 -> ../dm-20
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathd1p1p1 -> ../dm-21
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 vg_file01_vdp-lv_file01 -> ../dm-16
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 vg_iscsi_storage-lv_iscsi_storage -> ../dm-19
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 vg_iscsi_vdp-lv_vdp -> ../dm-22
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 vg_iscsi_vms-lv_iscsi_vm -> ../dm-14

OK so I rightly or wrongly read that to get multipath's you have to create the partitions on the /dev/mapper/mpath*1 so that is what I did, as you can see from the output above I had a few issues creating these (perhaps because this was the wrong thing to do?) So what I read just said to run the command again and you will just end up with another partition, for example:

lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathc1p1p1 -> ../dm-18

While the system is up after using kpartx to hotadd the filesystems all works OK.

Now looking at the default config for multipath.conf "dm-" is blacklisted, so I have done the following changes to the multipath.conf:

blacklist_exceptions {
device {
vendor "HP"
product "P2000 G3 FC"
}
devnode "dm-"
}

## Use user friendly names, instead of using WWIDs as names.
defaults {
user_friendly_names yes
find_multipaths yes
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
failback "5"
rr_weight priorities
no_path_retry "5"
}

#Multipaths
multipaths {
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff00012e03163ea655901000000
alias mpathb
}
#----------------- END VG=DESCRIPTION ----------------------
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff000dadd07d959635901000000
alias mpatha
}
#----------------- END VG=DESCRIPTION ----------------------
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff00014ebf5c757635901000000
alias mpathc
}
#----------------- END VG=DESCRIPTION ----------------------
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff00012e031efe9655901000000
alias mpathd
}
#----------------- START VG=DESCRIPTION ----------------------
}

devices {
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
hardware_handler "0"
failback 15
rr_weight priorities
no_path_retry queue
}
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
}
}

blacklist {
devnode "^(sda|ram|raw|loop|fd|md|sr|scd|st|cl-home|cl-swap|cl-root)[0-9]*"
}

Now in the multipath.conf, there is a section for configuring multipaths, however the only relevant entry that I could see that I could possibly make here in the example is alias:

#multipaths {
# multipath {
# wwid 3600508b4000156d700012000000b0000
# alias yellow
Do i need to change the alias to one of my multipaths for example, such as /dev/mapper/mpathc1p1p1 for example? But this doesn't really make a lot of sense to me.

As far as I can tell/understand there is not an entry in the system to state that these paths shown here, should exist during the bootup process:

# pvs && vgs && lvs
PV VG Fmt Attr PSize PFree
/dev/mapper/mpatha1p1p1 vg_iscsi_vms lvm2 a-- <2.73t 0
/dev/mapper/mpathb1p1 vg_file01_vdp lvm2 a-- <7.26t 0
/dev/mapper/mpathc1p1p1 vg_iscsi_storage lvm2 a-- <9.09t 0
/dev/mapper/mpathd1p1p1 vg_iscsi_vdp lvm2 a-- 12.73t 0
/dev/sda2 cl lvm2 a-- <135.70g 0

VG #PV #LV #SN Attr VSize VFree
vg_file01_vdp 1 1 0 wz--n- <7.26t 0
vg_iscsi_storage 1 1 0 wz--n- <9.09t 0
vg_iscsi_vdp 1 1 0 wz--n- 12.73t 0
vg_iscsi_vms 1 1 0 wz--n- <2.73t 0

LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
lv_file01 vg_file01_vdp -wi-ao---- <7.26t
lv_iscsi_storage vg_iscsi_storage -wi-ao---- <9.09t
lv_vdp vg_iscsi_vdp -wi-ao---- 12.73t
lv_iscsi_vm vg_iscsi_vms -wi-ao---- <2.73t

This turned my attention to the lvm.conf file where the RedHat docco states that if you are using multipath you should add the following entry to the lvm.conf under devices:

filter = [ "r/block/", "r/disk/", "r/sd.*/", "a/.*/" ]

I did this and updated the initramfs and it caused the system not to boot. Once I removed this from the initramfs the system booted, I believe this is reference to if you are booting from a multipath SAN :(

if I show the blkid:

# blkid
/dev/sda1: UUID="887de158-d44f-4365-aee5-f16930f5a947" TYPE="xfs"
/dev/sda2: UUID="595V2O-D2xc-blZc-iMsc-aKDf-gVwk-wMkD1I" TYPE="LVM2_member"
/dev/mapper/cl-root: UUID="cf0bb0fa-de76-4e4e-b2df-a0acaf1a380b" TYPE="xfs"
/dev/mapper/cl-swap: UUID="eabf4b79-78d9-41c0-a99c-fb03ca7ba919" TYPE="swap"
/dev/mapper/cl-home: UUID="88a919f3-ab0e-4176-9a2f-d26a1f23393f" TYPE="xfs"
/dev/mapper/mpatha1p1p1: UUID="eSRWwg-A0qQ-CKZv-XWCp-sDib-JZUP-22LwR9" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="8ef397db-c6d8-4dc7-9470-d3c079afc010"
/dev/mapper/mpathb1p1: UUID="lp8I4T-4bcf-P2TE-2iyv-p1wJ-csik-4vjX0r" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="0e906c88-2d19-40f9-af91-ac3ab36b9f9e"
/dev/mapper/mpathc1p1p1: UUID="8LIbJ3-0r9U-ZuRq-F2kj-MysM-nryN-yM1JUN" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="634786a1-578d-4827-9ab0-ec7c797b2532"
/dev/mapper/mpathd1p1p1: UUID="SpXMnR-3fNu-LjQz-a32y-sEO6-jLkH-M9XvNw" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="ec6dfc62-55c3-41bd-bb1c-54a954d3e7bc"
/dev/sdb1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="82a53134-64e9-4918-8e8e-de166145f50f"
/dev/sdc1: PTTYPE="gpt" PARTUUID="5b90fa45-698b-467d-96f2-e5110478f193"
/dev/sdd1: PTTYPE="gpt" PARTUUID="5b90fa45-698b-467d-96f2-e5110478f193"
/dev/sde1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="82a53134-64e9-4918-8e8e-de166145f50f"
/dev/sdf1: PTTYPE="gpt" PARTUUID="4b647143-af50-4e82-ae05-7ad9d694b6cb"
/dev/sdg1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="b542f6ec-fbdf-437e-9d2f-baa496e0d1e4"
/dev/sdg2: PARTLABEL="primary" PARTUUID="5ca89dab-e827-4301-ab66-10929162b772"
/dev/sdh1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="b542f6ec-fbdf-437e-9d2f-baa496e0d1e4"
/dev/sdh2: PARTLABEL="primary" PARTUUID="5ca89dab-e827-4301-ab66-10929162b772"
/dev/sdi1: PTTYPE="gpt" PARTUUID="4b647143-af50-4e82-ae05-7ad9d694b6cb"
/dev/mapper/mpathc: PTTYPE="gpt"
/dev/mapper/mpatha: PTTYPE="gpt"
/dev/mapper/mpathb: PTTYPE="gpt"
/dev/mapper/mpathd: PTTYPE="gpt"
/dev/mapper/mpathc1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="82a53134-64e9-4918-8e8e-de166145f50f"
/dev/mapper/mpatha1: PTTYPE="gpt" PARTUUID="4b647143-af50-4e82-ae05-7ad9d694b6cb"
/dev/mapper/mpathb1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="b542f6ec-fbdf-437e-9d2f-baa496e0d1e4"
/dev/mapper/mpathd1: PTTYPE="gpt" PARTUUID="5b90fa45-698b-467d-96f2-e5110478f193"
/dev/mapper/mpathb2: PARTLABEL="primary" PARTUUID="5ca89dab-e827-4301-ab66-10929162b772"
/dev/mapper/mpatha1p1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="7e1262c8-a1db-4cab-be14-06b2d9a5bab3"
/dev/mapper/vg_iscsi_vms-lv_iscsi_vm: PTTYPE="gpt"
/dev/mapper/vg_file01_vdp-lv_file01: PTTYPE="gpt"
/dev/mapper/mpathc1p1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="94adcc42-771d-4a26-bd23-a8ab944148e1"
/dev/mapper/vg_iscsi_storage-lv_iscsi_storage: PTTYPE="gpt"
/dev/mapper/mpathd1p1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="ece1e107-6ae7-42db-9c22-a7153d50b6c8"
/dev/mapper/vg_iscsi_vdp-lv_vdp: PTTYPE="gpt"

It shows entries from /dev/sdb1 through to /dev/sdi1 and also my paths, which looks messy due to using kpartx to bring them online. I then turned my attention to trying fstab but did not want to cause system instability so tried using mount first and as I suspected nothing mounts correctly as it is already mounted. For reference here is the output from lvdisplay:

# lvdisplay
--- Logical volume ---
LV Path /dev/vg_iscsi_vdp/lv_vdp
LV Name lv_vdp
VG Name vg_iscsi_vdp
LV UUID SDT2sZ-RMzK-7NRk-0f1Y-OQma-5hQC-kCYVUp
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-24 09:42:05 +0100
LV Status available
# open 1
LV Size 12.73 TiB
Current LE 3337858
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:22

--- Logical volume ---
LV Path /dev/vg_iscsi_vms/lv_iscsi_vm
LV Name lv_iscsi_vm
VG Name vg_iscsi_vms
LV UUID OtB1Q1-3jiK-UVQF-dqSU-uJ6z-Zsy1-O4cx46
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-19 12:10:09 +0100
LV Status available
# open 1
LV Size <2.73 TiB
Current LE 714539
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:14

--- Logical volume ---
LV Path /dev/vg_iscsi_storage/lv_iscsi_storage
LV Name lv_iscsi_storage
VG Name vg_iscsi_storage
LV UUID cOjc8O-04DC-Exbw-3acC-2J3Q-cFW8-aV6HkE
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-21 12:55:08 +0100
LV Status available
# open 1
LV Size <9.09 TiB
Current LE 2382255
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:19

--- Logical volume ---
LV Path /dev/vg_file01_vdp/lv_file01
LV Name lv_file01
VG Name vg_file01_vdp
LV UUID QpPn6z-1zlp-H3so-XNF3-BwtL-JxID-XikQeI
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-19 11:04:10 +0100
LV Status available
# open 1
LV Size <7.26 TiB
Current LE 1903103
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:16

multipath -v2 command displays nothing.

I have updated my initramfs with:

dracut --force --add multipath --include /etc/multipath

What am I missing, everything I have read doesnt state that I have to do any more than I have done but yet when I reboot none of my partitions are there?????????

Any ideas please?
 
Old 10-13-2017, 03:30 PM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,830
Blog Entries: 15

Rep: Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668
I don't think you need the black list exceptions section you added.

On RHEL7/CentOS7 they have some predefined stuff you can see with "multipath -t".
On checking my CentOS7 server I see the following in the predefined black list exceptions (among many others):
Code:
        device {
                vendor "HP"
                product "P2000 G3 FC|P2000G3 FC/iSCSI|P2000 G3 SAS|P2000 G3 iSCSI"
                path_grouping_policy "group_by_prio"
                path_checker "tur"
                features "0"
                hardware_handler "0"
                prio "alua"
                failback immediate
                rr_weight "uniform"
                no_path_retry 18
                rr_min_io 100
        }
The multipath.conf on RHEL7/CentOS7 is a lot simpler than on earlier RHEL6/CentOS6. By default it only needs:
Code:
defaults {
        user_friendly_names yes
        find_multipaths yes
}
The latter line tells it to ignore things that don't have multipath so you don't need to exclude things like PERC/LSI/Avago internal RAID controllers that present virtual disks as single paths any longer. (My read is one wouldn't have to exclude USB disks either as I've had to do in past but I haven't tested that on RHEL7/CentOS7.

You are defining your aliases as "mpath" names but that is what "user_friendly_names" does by default (though they can change between boots). I'm wondering if your use of mpath names is confusing it. What if you try something different like:
Code:
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff00012e03163ea655901000000
alias hp_pathb
}
#----------------- END VG=DESCRIPTION ----------------------
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff000dadd07d959635901000000
alias hp_patha
}
#----------------- END VG=DESCRIPTION ----------------------
#----------------- START VG=DESCRIPTION ----------------------
multipath {
wwid 3600c0ff00014ebf5c757635901000000
alias hp_pathc
etc...

Note that you do not need to create the links normally. multipathd would typically detect the device given with the wwid then create the alias name in /dev/mapper automaticlly with link to the /dev/dm-* device. Note the /dev/dm-* can change between boots but so long as you use the /dev/mapper devices as the PVs that shouldn't matter.

Don't forget you have to restart multipathd any time you change /etc/multipathd.conf.

Also make sure you haven't got a filter in lvm.conf that is excluding the names you've assigned. LVM should find the names on the /dev/sd* the /dev/dm* and the /dev/mapper/* devices so running "pvs" ought to tell you what it thinks you have.
 
Old 10-15-2017, 03:42 AM   #3
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
URGENT problem with Multipath partitions not present on reboot - CentOS 7

Hi MensaWater,

I really appreciate your response on this as this has be pretty baffled right now. Perhaps it would be pertinent to add to this that when leaving the multipath.conf as default I had exactly the same problem. In fact I did not make the alias changes to the multipath.conf until Friday morning before I posted this but I have not restarted the multipathd. How ever I had added things such as blacklisting devnode sda. But additionally added the following blacklisting Friday morning:

}

blacklist {
devnode "^(sda|ram|raw|loop|fd|md|sr|scd|st|cl-home|cl-swap|cl-root)[0-9]*"
}


Rather than restarting the multipathd I have been using:

multipathd -k

Then issuing the'reconfigure' command, of which I can see does not issue all the changes to the OS as my alias's that I have created are not present in /dev/mapper/

So are you saying there is no need for:

blacklist_exceptions {
device {
vendor "HP"
product "P2000 G3 FC"
}
devnode "dm-"
}

The reason that I added particularly 'devnode "dm-"' is due to the fact that when I enter multipathd -k and issue the 'show blacklist' command I could see that the dm-*'s were blacklisted by default. I have made this recent change to attempt to resolve the fact that none of my paths:

PV VG Fmt Attr PSize PFree
/dev/mapper/mpatha1p1p1 vg_iscsi_vms lvm2 a-- <2.73t 0
/dev/mapper/mpathb1p1 vg_file01_vdp lvm2 a-- <7.26t 0
/dev/mapper/mpathc1p1p1 vg_iscsi_storage lvm2 a-- <9.09t 0
/dev/mapper/mpathd1p1p1 vg_iscsi_vdp lvm2 a-- 12.73t 0

Are present when I reboot the server.

Currently these LV's are presented to an VMware DR environment via iSCSI and I have terabytes of data in here with no space to copy the .vmdk's off to do any testing, of which I guess is always the case :(

Will restarting the multipathd cause disruption to the path to the disks? As I have a couple of Virtual Machine's running on here?

Are you aware of any logging I can look at, my boot.log has no entries and /var/log/messages doesn't have any thing useful either. I will try and post the output multipath -v3 but last time I tried it was beyond the allowed limit of characters, but I will get in what I can.
 
Old 10-15-2017, 03:47 AM   #4
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
URGENT problem with Multipath partitions not present on reboot - CentOS 7

I have attached the output of multipath -v3 if some one would be kind enough to take a look please?
Attached Files
File Type: txt multipath-v3_Output.txt (34.4 KB, 35 views)
 
Old 10-16-2017, 03:49 AM   #5
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
Hi

Should I be adding the entries to /etc/fstab?
 
Old 10-23-2017, 09:14 AM   #6
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
Hi,

So looking at the journalctl, I can see the disks get found during the bootup process:

Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: sdb: sdb1
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 rel port 01
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdc: sdc1
Oct 12 10:09:47 HOSTNAME kernel: sr 2:0:0:0: [sr0] scsi3-mmc drive: 24x/24x writer dvd-ram cd/rw xa/form2 cdda tray
Oct 12 10:09:47 HOSTNAME kernel: cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 12 10:09:47 HOSTNAME kernel: sr 2:0:0:0: Attached scsi CD-ROM sr0
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 rel port 05
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdd: sdd1
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 rel port 01
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: sde: sde1
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 rel port 06
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdf: sdf1
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 rel port 02
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdg: sdg1 sdg2
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 rel port 06
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdh: sdh1 sdh2
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 rel port 02
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 state A preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: random: crng init done
Oct 12 10:09:47 HOSTNAME kernel: sdi: sdi1
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Attached SCSI disk

Then multipathd joins the party:

Oct 12 10:09:50 HOSTNAME systemd[1]: Started Device-Mapper Multipath Device Controller.
Oct 12 10:09:50 HOSTNAME multipathd[697]: path checkers start up

Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:16 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdb [8:16]: path added to devmap mpathc
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:128 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi [8:128]: path added to devmap mpatha
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:112 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh [8:112]: path added to devmap mpathb
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde : delaying path addition until mpathc is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg : delaying path addition until mpathb is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd [8:48]: path added to devmap mpathd
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf : delaying path addition until mpatha is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc : delaying path addition until mpathd is fully initialized
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: Found new BMC (man_id: 0x00000b, prod_id: 0x2000, dev_id: 0x11)
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: IPMI kcs interface initialized
Oct 12 10:09:52 HOSTNAME kernel: IPMI SSIF Interface driver
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME systemd[1]: Created slice system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting LVM2 PV scan on device 8:2...
Oct 12 10:09:52 HOSTNAME lvm[870]: WARNING: lvmetad is being updated, retrying (setup) for 10 more seconds.
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:16 1 round-robin 0 1 1 8:64 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:128 1 round-robin 0 1 1 8:80 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:32 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:96 1 round-robin 0 1 1 8:112 1]

So, what ever ‘spurious uevent, path already in pathvec’ means appears to be the issue as the partitions never load, any ideas?
 
Old 10-24-2017, 06:56 AM   #7
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
Yet another day of talking to myself about this issue

I appreciate any response on this. I checked the journalctl and found the following:

#journalctl --boot

Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 rel port 01
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: sdc: sdc1
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: sr 2:0:0:0: [sr0] scsi3-mmc drive: 24x/24x writer dvd-ram cd/rw xa/form2 cdda tray
Oct 24 09:10:51 HOSTNAME kernel: cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 24 09:10:51 HOSTNAME kernel: sr 2:0:0:0: Attached scsi CD-ROM sr0
Oct 24 09:10:51 HOSTNAME kernel: sdb: sdb1
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdc: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[346]: inotify_add_watch(6, /dev/sdc1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME systemd-udevd[344]: inotify_add_watch(6, /dev/sdb1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: device-mapper: multipath service-time: version 0.3.0 loaded
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:32 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathd: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdc [8:32]: path added to devmap mpathd
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdb: add path (uevent)
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:16 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathc: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdb [8:16]: path added to devmap mpathc
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 rel port 05
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdd: sdd1
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 rel port 01
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdd: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[348]: inotify_add_watch(6, /dev/sdd1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:32 1 8:48 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdd [8:48]: path added to devmap mpathd
Oct 24 09:10:51 HOSTNAME kernel: sde: sde1
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: random: crng init done
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 rel port 06
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: Attached
Oct 24 09:10:51 HOSTNAME multipathd[389]: sde: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[344]: inotify_add_watch(6, /dev/sde1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:16 1 8:64 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: sde [8:64]: path added to devmap mpathc
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdf: sdf1
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 rel port 02
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdg: sdg1 sdg2
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdg: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[348]: inotify_add_watch(6, /dev/sdg1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 rel port 06
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 24 09:10:51 HOSTNAME systemd-udevd[349]: inotify_add_watch(6, /dev/sdg2, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME systemd-udevd[344]: inotify_add_watch(6, /dev/sdf1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:96 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdg [8:96]: path added to devmap mpathb
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdf: add path (uevent)
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdh: sdh1 sdh2
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:80 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpatha: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdf [8:80]: path added to devmap mpatha
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdh: add path (uevent)
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 rel port 02
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 state A preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: Attached
Oct 24 09:10:51 HOSTNAME systemd-udevd[346]: inotify_add_watch(6, /dev/sdh1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME systemd-udevd[352]: inotify_add_watch(6, /dev/sdh2, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdh : delaying path addition until mpathb is fully initialized
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: performing delayed actions
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:96 1 8:112 1]
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: sdi: sdi1
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdi: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[345]: inotify_add_watch(6, /dev/sdi1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:80 1 8:128 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdi [8:128]: path added to devmap mpatha

#journalctl -u multipathd

Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:16 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdb [8:16]: path added to devmap mpathc
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:128 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi [8:128]: path added to devmap mpatha
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:112 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh [8:112]: path added to devmap mpathb
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde : delaying path addition until mpathc is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg : delaying path addition until mpathb is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd [8:48]: path added to devmap mpathd
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf : delaying path addition until mpatha is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc : delaying path addition until mpathd is fully initialized
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: Found new BMC (man_id: 0x00000b, prod_id: 0x2000, dev_id: 0x11)
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: IPMI kcs interface initialized
Oct 12 10:09:52 HOSTNAME kernel: IPMI SSIF Interface driver
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME systemd[1]: Created slice system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting LVM2 PV scan on device 8:2...
Oct 12 10:09:52 HOSTNAME lvm[870]: WARNING: lvmetad is being updated, retrying (setup) for 10 more seconds.
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:16 1 round-robin 0 1 1 8:64 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:128 1 round-robin 0 1 1 8:80 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:32 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:96 1 round-robin 0 1 1 8:112 1]

During the boot process the multipaths are being picked up on sd* and then seeing:

'spurious uevent, path already in pathvec'

It lead me to think that there was some thing wrong with the config, so I have stumbled across a RH KB which has the same eror as such and talks about the blacklisting not being correct:

https://access.redhat.com/solutions/2360781

I tried blacklisting sd* and also the sdb through to sdi which meant none of my /dev/mapper/mpath* showed up which was worse I even tried disabling smartd which is why the output above may look a little different. I have enabled this since my testing.

So I had to fall back to my work around. I am now completely lost as to what the problem is. I dont think my lvm.conf is wrong because as soon as I hot add the partitions the VGs and LVs appear, I am out of ideas at this point
 
Old 10-24-2017, 07:54 AM   #8
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,830
Blog Entries: 15

Rep: Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668
Hi, I went on vacation after my last response.

Yes I was suggesting you don't need the blacklist exception you added for HP as one for HP appears to already be in the default as shown by "multipath -t".

Yes you should restart multipathd each time.
No restarting multipathd does not stop running devices that are in use.
It is possible renaming any already in use would cause an issue - I've never tried that but don't think it would. The purposes of multipathd is to detect events so if the device already exists and is in use there should be no event. I've done the restart on systems with dozens of multipath devices when adding new ones multiple times.

On checking the "show blacklist" you mention I also see it has dm-:
Code:
- blacklist:
        (default rule)     ^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*
        (default rule)     ^(td|hd|vd)[a-z]
        (default rule)     ^dcssblk[0-9]*
As I mentioned before the "dm-" devices can change on boot so the above rule makes sense to me. You should NOT use the "dm-" names in LVM but instead use the /dev/mapper/<device> names (e.g. mpath or the ones I suggested before).

The errors you see in journal suggest to me that my surmise of using literal "mpath" names in your multipath.conf confusing it is likely the case. Try using the alternate names I put in my original post or just removing the definitions altogether. As I noted the "friendly names" created by default ARE "mpath" names so you don't have to specify those and it appears having specified them may be causing your issue. The reason to define your own is if you don't want them to change on boots or want different naming such as what I suggested.
 
Old 10-25-2017, 03:35 AM   #9
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
Hi MensaWater,

I hope you had a great vacation Thank you for your response.

Thanks for the information on restarting multipathd that does really help

OK, I have since removed the alias's altogether from multipath.conf as they were not really necessary. However when they are in place as I am sure you already know /dev/mapper/* changes from having the default name such as /dev/mapper/mpatha and /dev/mapper/mpatha1 to the alias in my case as suggested /dev/mapper/hp-mpatha. This had no effect on the partitions being available at boot hence I removed them and now I just have the standard config for this as can be seen in /etc/multipath/bindings:

mpatha 3600c0ff000dadd07d959635901000000
mpathb 3600c0ff00012e03163ea655901000000
mpathc 3600c0ff00014ebf5c757635901000000
mpathd 3600c0ff00012e031efe9655901000000

I get your point about the blacklist exceptions for the device and devnode "dm-" I have removed this and restarted multipathd, journalctl shows:

Oct 25 08:01:20 HOSTNAME systemd[1]: Stopping Device-Mapper Multipath Device Controller...
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpatha: stop event checker thread (139940414228224)
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpathb: stop event checker thread (139940414195456)
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpathc: stop event checker thread (139940414162688)
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpathd: stop event checker thread (139940414129920)
Oct 25 08:01:20 HOSTNAME multipathd[882]: --------shut down-------
Oct 25 08:01:20 HOSTNAME systemd[1]: Starting Device-Mapper Multipath Device Controller...
Oct 25 08:01:21 HOSTNAME systemd[1]: Started Device-Mapper Multipath Device Controller.
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:80 1 8:128 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:96 1 8:112 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:16 1 8:64 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:32 1 8:48 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpatha: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathb: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathc: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathd: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: path checkers start up
Oct 25 08:03:25 HOSTNAME kernel: sdb: sdb1
Oct 25 08:03:26 HOSTNAME kernel: sdc: sdc1
Oct 25 08:03:26 HOSTNAME kernel: sdd: sdd1
Oct 25 08:03:26 HOSTNAME kernel: sde: sde1
Oct 25 08:03:26 HOSTNAME kernel: sdf: sdf1
Oct 25 08:03:26 HOSTNAME kernel: sdg: sdg1 sdg2
Oct 25 08:03:26 HOSTNAME kernel: sdh: sdh1 sdh2
Oct 25 08:03:26 HOSTNAME kernel: sdi: sdi1

Which tells me that my partitions are being picked up on /dev/sd* instead of /dev/mapper/mpath* using multipath -v3 confirms this:

===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod
3600508b1001c029881c94e4b91967594 0:1:0:0 sda 8:0 1 undef undef HP ,
3600c0ff000dadd07d959635901000000 4:0:0:0 sdf 8:80 10 undef undef HP ,
3600c0ff00012e03163ea655901000000 4:0:1:0 sdg 8:96 50 undef undef HP ,
3600c0ff00012e03163ea655901000000 4:0:2:0 sdh 8:112 10 undef undef HP ,
3600c0ff000dadd07d959635901000000 4:0:3:0 sdi 8:128 50 undef undef HP ,
3600c0ff00014ebf5c757635901000000 1:0:0:0 sdb 8:16 50 undef undef HP ,
3600c0ff00012e031efe9655901000000 1:0:1:0 sdc 8:32 50 undef undef HP ,
3600c0ff00012e031efe9655901000000 1:0:2:0 sdd 8:48 10 undef undef HP ,
3600c0ff00014ebf5c757635901000000 1:0:3:0 sde 8:64 10 undef undef HP ,

It's almost like multipathd cannot see the partitions that I created on /dev/mapper/mpath*

FYI: I used parted to create these partitions as they are very large with gpt disks.

The rest of my multipath.conf that sits in /etc/ and not in /etc/multipath/ is as follows:

## Use user friendly names, instead of using WWIDs as names.
defaults {
user_friendly_names yes
find_multipaths yes

}

devices {
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
hardware_handler "0"
failback 15
rr_weight priorities
no_path_retry queue
}
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
}
}

blacklist {
devnode "^(sda|ram|raw|loop|fd|md|sr|scd|st|cl-home|cl-swap|cl-root)[0-9]*"
}

I have also tried setting this back to the default configuration with just the blacklist. I have tried changing the 'path checker' option to all of the available options:

path_checker - Specifies the default method used to determine the state of the paths. Possible values include:

readsector0: Read the first sector of the device.
tur: Issue a TEST UNIT READY to the device.
emc_clariion: Query the EMC Clariion specific EVPD page 0xC0 to determine the path.
hp_sw: Check the path state for HP storage arrays with Active/Standby firmware.
rdac: Check the path status for LSI/Engenio RDAC storage controller.
directio: Read the first sector with direct I/O.

Then restarted multipathd with the same results. Now here is the bit I do not get, there is no mention of the paths in /dev/mapper/ unless I run the command:

#dmsetup ls

mpatha1p1p1 (253:13)
cl-swap (253:10)
cl-root (253:9)
3600c0ff00012e031efe9655901000000p1 (253:2)
mpatha1p1 (253:12)
3600c0ff000dadd07d959635901000000p1 (253:8)
mpathc1p1p1 (253:18)
vg_file01_vdp-lv_file01 (253:16)
3600c0ff00012e03163ea655901000000 (253:4)
mpathd1p1p1 (253:21)
vg_iscsi_vms-lv_iscsi_vm (253:14)
3600c0ff000dadd07d959635901000000 (253:5)
mpathd1p1 (253:20)
3600c0ff00014ebf5c757635901000000 (253:1)
vg_iscsi_vdp-lv_vdp (253:22)
vg_iscsi_storage-lv_iscsi_storage (253:19)
3600c0ff00012e03163ea655901000000p2 (253:7)
3600c0ff00012e03163ea655901000000p1 (253:6)
mpathc1p1 (253:17)
3600c0ff00014ebf5c757635901000000p1 (253:3)
cl-home (253:11)
3600c0ff00012e031efe9655901000000 (253:0)
mpathb1p1 (253:15)

Which is the only place I can find a mention of my partitions and also looks incorrect to me.

So I guess the question really is, why doesn't multipath see my partitions on /dev/mapper/mpath*?

I am happy to reconfigure the iSCSI to work around the problem and subsequently use /dev/sd* but even this would defeat the object of multipath because if I used say /dev/sdb and this path failed the connection would be dropped via iSCSI

NOTE: I have NOT made any changes to /etc/lvm/lvm.conf as I keep reading conflicting information and the last time I did so (following RHELs guide) the system failed to boot so I had to recover the initramfs to get back to a working state, which turns out is not much fun

Is it possible my lack of entry to the lvm.conf is my issue?

In regard to what you said about using “dm-“ names in LVM this is not something I have done as can be seen here:

# pvs && vgs && lvs
PV VG Fmt Attr PSize PFree
/dev/mapper/mpatha1p1p1 vg_iscsi_vms lvm2 a-- <2.73t 0
/dev/mapper/mpathb1p1 vg_file01_vdp lvm2 a-- <7.26t 0
/dev/mapper/mpathc1p1p1 vg_iscsi_storage lvm2 a-- <9.09t 0
/dev/mapper/mpathd1p1p1 vg_iscsi_vdp lvm2 a-- 12.73t 0
/dev/sda2 cl lvm2 a-- <135.70g 0
VG #PV #LV #SN Attr VSize VFree
cl 1 3 0 wz--n- <135.70g 0
vg_file01_vdp 1 1 0 wz--n- <7.26t 0
vg_iscsi_storage 1 1 0 wz--n- <9.09t 0
vg_iscsi_vdp 1 1 0 wz--n- 12.73t 0
vg_iscsi_vms 1 1 0 wz--n- <2.73t 0

I hope this information helps

Last edited by h1tchiker; 10-25-2017 at 03:38 AM.
 
Old 10-25-2017, 08:06 AM   #10
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,830
Blog Entries: 15

Rep: Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668
What is the output of:
multipath -l -v2
 
Old 10-25-2017, 09:07 AM   #11
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
# multipath -l -v2
3600c0ff00012e03163ea655901000000 dm-4 HP ,P2000 G3 FC
size=7.3T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 4:0:1:0 sdg 8:96 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 4:0:2:0 sdh 8:112 active undef unknown
3600c0ff000dadd07d959635901000000 dm-5 HP ,P2000 G3 FC
size=2.7T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 4:0:3:0 sdi 8:128 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 4:0:0:0 sdf 8:80 active undef unknown
3600c0ff00014ebf5c757635901000000 dm-1 HP ,P2000 G3 FC
size=9.1T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 1:0:0:0 sdb 8:16 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 1:0:3:0 sde 8:64 active undef unknown
3600c0ff00012e031efe9655901000000 dm-0 HP ,P2000 G3 FC
size=13T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 1:0:1:0 sdc 8:32 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 1:0:2:0 sdd 8:48 active undef unknown
 
Old 10-25-2017, 10:39 AM   #12
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,830
Blog Entries: 15

Rep: Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668
The first line shown for each of your multipath devices is similar to this first one:
3600c0ff00012e03163ea655901000000 dm-4 HP ,P2000 G3 FC

That shows you the WWID of the multipath device and its dm- name. If using your own names or user friendly names the alternate name would be at start of that line and the WWID would be in parentheses similar to:
mpatha (3600c0ff00012e03163ea655901000000) dm-4 HP ,P2000 G3 FC
or:
hp_patha (3600c0ff00012e03163ea655901000000) dm-4 HP ,P2000 G3 FC

Did you remove the default section for some reason:
Code:
defaults {
        user_friendly_names yes
        find_multipaths yes
}

Last edited by MensaWater; 10-25-2017 at 01:17 PM.
 
Old 10-25-2017, 12:20 PM   #13
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,830
Blog Entries: 15

Rep: Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668
You mention partitions in several of your posts.

1) Are you using separate partitions on the same disk? If not and using LVM you don't have to put a partition on it at all. You can use the entire disk as your PV. At one point it was suggested to create a single partition and use that but last year while migrating from one disk array to another I couldn't find anything suggesting that is actually required. Here we only have one multipath setup that still uses partitions as we use Oracle ASM on those partitions. Everything else (multiple servers/multiple SAN disks) we use entire disk without partitioning.

2) I'd found some oddities dealing with partitioning in cluster and multipath. Notes I wrote at the time:
a) When creating partitions create them on one of the underlying sd devices NOT the multipath device.
e.g. For the WWID 3600c0ff000dadd07d959635901000000 you listed you have component disks "sdi" and "sdf". You'd run
fdisk or parted on sdi or sdf in /dev NOT the name in /dev/mapper or the dm- device.

b) Run "partprobe"

c) Run "blockdev --flushbufs" on each of those sd component disks.
e.g. "blockdev --flushbufs /dev/sdi" and "blockdev --flushbufs /dev/sdf"

d) Run "fdisk -luc" on each of those devices to verify they show as expected.

After I did the above I saw the names multipath device and its partition in /dev/mapper linked backed to separate dm- devices:
e.g. lrwxrwxrwx 1 root root 7 Feb 24 14:19 mpatha -> ../dm-9 < Entire multipath disk
lrwxrwxrwx 1 root root 8 Feb 24 15:36 mpathap1 -> ../dm-13 < Partition 1 of that same multipath disk.

Last edited by MensaWater; 10-25-2017 at 02:31 PM.
 
Old 10-26-2017, 01:59 AM   #14
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
Hi MensaWater,

Yes in my stupidity I commented these out, here is the output:

# multipath -l -v2
mpathd (3600c0ff00012e031efe9655901000000) dm-0 HP ,P2000 G3 FC
size=13T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 1:0:1:0 sdc 8:32 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 1:0:2:0 sdd 8:48 active undef unknown
mpathc (3600c0ff00014ebf5c757635901000000) dm-1 HP ,P2000 G3 FC
size=9.1T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 1:0:0:0 sdb 8:16 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 1:0:3:0 sde 8:64 active undef unknown
mpathb (3600c0ff00012e03163ea655901000000) dm-4 HP ,P2000 G3 FC
size=7.3T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 4:0:1:0 sdg 8:96 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 4:0:2:0 sdh 8:112 active undef unknown
mpatha (3600c0ff000dadd07d959635901000000) dm-5 HP ,P2000 G3 FC
size=2.7T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 4:0:3:0 sdi 8:128 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
`- 4:0:0:0 sdf 8:80 active undef unknown
 
Old 10-26-2017, 03:11 AM   #15
h1tchiker
LQ Newbie
 
Registered: Apr 2007
Posts: 11

Original Poster
Rep: Reputation: 0
Hi MensaWater,

In answer to question 1. - No I am not, I have two SANS connected via FC and these are setup with 2 disk partitions/arrays in each using different channels, I have an FC pair per disk array in the SAN. So I am using the entire disk presented in each array.

I think I get what you are saying. I did also run fdisk on /dev/sd* and then used parted on the /dev/mapper/mpath* to get the partition sizes that I wanted (I was following a Tin'TerWeb guide some time ago), I believe I created the partition using fdisk and then did something like:

dd if=/dev/zero of=/dev/mapper/mpathc bs=512 count=1 conv=notrunc

parted /dev/sdf

parted /dev/mapper/mpathc mklabel gpt

parted -a optimal -- /dev/mapper/mpathc1 mkpart primary xfs 0% 100%

To achieve my partition sizes.

So i have followed what you have said:

partprobe

blockdev --flushbufs /dev/sdb
blockdev --flushbufs /dev/sdc
blockdev --flushbufs /dev/sdd
blockdev --flushbufs /dev/sde
blockdev --flushbufs /dev/sdf
blockdev --flushbufs /dev/sdg
blockdev --flushbufs /dev/sdh
blockdev --flushbufs /dev/sdi

# fdisk -luc /dev/sdb
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdb: 9991.9 GB, 9991908556800 bytes, 19515446400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: 543C1570-18EF-45CF-A6AC-DBD48B4A758F


# Start End Size Type Name
1 2048 19515445247 9.1T Microsoft basic primary

# fdisk -luc /dev/sdc
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdc: 14000.0 GB, 13999999942656 bytes, 27343749888 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: E072C4CB-3058-4B47-9C6D-0E0755C52449


# Start End Size Type Name
1 2048 27343749854 12.8T Linux filesyste

# fdisk -luc /dev/sdd
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdd: 14000.0 GB, 13999999942656 bytes, 27343749888 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: E072C4CB-3058-4B47-9C6D-0E0755C52449


# Start End Size Type Name
1 2048 27343749854 12.8T Linux filesyste

# fdisk -luc /dev/sde
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sde: 9991.9 GB, 9991908556800 bytes, 19515446400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: 543C1570-18EF-45CF-A6AC-DBD48B4A758F


# Start End Size Type Name
1 2048 19515445247 9.1T Microsoft basic primary

# fdisk -luc /dev/sdf
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdf: 2997.6 GB, 2997551759360 bytes, 5854593280 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: 5625E25E-E1A4-449B-911A-86BE1A596676


# Start End Size Type Name
1 2048 5853517823 2.7T Linux filesyste

# fdisk -luc /dev/sdg
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdg: 7982.2 GB, 7982198882304 bytes, 15590232192 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: D4AB3606-5403-42B5-B904-F4C0EB50EF9C


# Start End Size Type Name
1 2048 15590232063 7.3T Microsoft basic primary
2 34 2047 1007K Microsoft basic primary

# fdisk -luc /dev/sdh
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdh: 7982.2 GB, 7982198882304 bytes, 15590232192 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: D4AB3606-5403-42B5-B904-F4C0EB50EF9C


# Start End Size Type Name
1 2048 15590232063 7.3T Microsoft basic primary
2 34 2047 1007K Microsoft basic primary

# fdisk -luc /dev/sdi
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdi: 2997.6 GB, 2997551759360 bytes, 5854593280 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disk label type: gpt
Disk identifier: 5625E25E-E1A4-449B-911A-86BE1A596676


# Start End Size Type Name
1 2048 5853517823 2.7T Linux filesyste

However my /dev/mapper has always looks like this:

# ll /dev/mapper
total 0
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 cl-home -> ../dm-11
lrwxrwxrwx. 1 root root 7 Oct 26 08:48 cl-root -> ../dm-9
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 cl-swap -> ../dm-10
crw-------. 1 root root 10, 236 Oct 24 09:10 control
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpatha -> ../dm-5
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpatha1 -> ../dm-8
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpatha1p1 -> ../dm-12
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpatha1p1p1 -> ../dm-13
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathb -> ../dm-4
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathb1 -> ../dm-6
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpathb1p1 -> ../dm-15
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathb2 -> ../dm-7
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathc -> ../dm-1
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathc1 -> ../dm-3
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpathc1p1 -> ../dm-17
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpathc1p1p1 -> ../dm-18
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathd -> ../dm-0
lrwxrwxrwx. 1 root root 7 Oct 26 08:55 mpathd1 -> ../dm-2
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpathd1p1 -> ../dm-20
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 mpathd1p1p1 -> ../dm-21
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 vg_file01_vdp-lv_file01 -> ../dm-16
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 vg_iscsi_storage-lv_iscsi_storage -> ../dm-19
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 vg_iscsi_vdp-lv_vdp -> ../dm-22
lrwxrwxrwx. 1 root root 8 Oct 26 08:48 vg_iscsi_vms-lv_iscsi_vm -> ../dm-14

I restarted multipathd but all looks the same as before.

Thanks again for your input
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] CentOS 6.6 multipath filling /var/log/messages usao Linux - Server 3 04-14-2017 08:50 AM
how to make device mapper multipath pseudo name consistence through reboot robertngo Linux - Server 1 06-29-2009 05:24 AM
wifi0 and ath0 device not present after reboot Gamock Mandriva 1 03-27-2008 11:12 PM
CentOS - libsysfs.so.1 & /sbin/multipath Boby Red Hat 0 01-04-2006 10:51 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:14 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration