Multipath issue
Hi,
I have HS-23 blade server configured with 2 Emulex FC cards. I have installed RHEL 6.5. I have IBM v7000 storage connected to brocade switch 2 Numbers(2 ports of canister A to switch1, the remaining two to switch2, similarly 2 ports of canister B to switch1 & the other two to switch 2) Upon zoning and presenting the LUN to the server it was showing in four path -- sdb,sdc,sdd,sde and mpath. device sda is hard disk configured in Raid 1 on LSI. I have device mapper package installed and did the following #mpathconf --enable #service multipathd start Output of the command #fdisk -l Disk /dev/sda: 299.0 GB, 298999349248 bytes 255 heads, 63 sectors/track, 36351 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 36352 291991551+ ee GPT Partition 1 does not start on physical sector boundary. Disk /dev/sdb: 536.9 GB, 536870912000 bytes 255 heads, 63 sectors/track, 65270 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc: 536.9 GB, 536870912000 bytes 255 heads, 63 sectors/track, 65270 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdd: 536.9 GB, 536870912000 bytes 255 heads, 63 sectors/track, 65270 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sde: 536.9 GB, 536870912000 bytes 255 heads, 63 sectors/track, 65270 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/mapper/mpathb: 536.9 GB, 536870912000 bytes 255 heads, 63 sectors/track, 65270 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 #multipath -ll mpathb (36005076d028100d3280000000000001f) dm-1 IBM,2145 size=500G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 2:0:1:0 sdc 8:32 active ready running | `- 1:0:0:0 sdd 8:48 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 2:0:0:0 sdb 8:16 active ready running `- 1:0:1:0 sde 8:64 active ready running multipath -v3 Apr 03 18:46:21 | ram0: device node name blacklisted Apr 03 18:46:21 | ram1: device node name blacklisted Apr 03 18:46:21 | ram2: device node name blacklisted Apr 03 18:46:21 | ram3: device node name blacklisted Apr 03 18:46:21 | ram4: device node name blacklisted Apr 03 18:46:21 | ram5: device node name blacklisted Apr 03 18:46:21 | ram6: device node name blacklisted Apr 03 18:46:21 | ram7: device node name blacklisted Apr 03 18:46:21 | ram8: device node name blacklisted Apr 03 18:46:21 | ram9: device node name blacklisted Apr 03 18:46:21 | ram10: device node name blacklisted Apr 03 18:46:21 | ram11: device node name blacklisted Apr 03 18:46:21 | ram12: device node name blacklisted Apr 03 18:46:21 | ram13: device node name blacklisted Apr 03 18:46:21 | ram14: device node name blacklisted Apr 03 18:46:21 | ram15: device node name blacklisted Apr 03 18:46:21 | loop0: device node name blacklisted Apr 03 18:46:21 | loop1: device node name blacklisted Apr 03 18:46:21 | loop2: device node name blacklisted Apr 03 18:46:21 | loop3: device node name blacklisted Apr 03 18:46:21 | loop4: device node name blacklisted Apr 03 18:46:21 | loop5: device node name blacklisted Apr 03 18:46:21 | loop6: device node name blacklisted Apr 03 18:46:21 | loop7: device node name blacklisted Apr 03 18:46:21 | sda: not found in pathvec Apr 03 18:46:21 | sda: mask = 0x3f Apr 03 18:46:21 | sda: dev_t = 8:0 Apr 03 18:46:21 | sda: size = 583983104 Apr 03 18:46:21 | sda: subsystem = scsi Apr 03 18:46:21 | sda: vendor = LSI Apr 03 18:46:21 | sda: product = Logical Volume Apr 03 18:46:21 | sda: rev = 3000 Apr 03 18:46:21 | sda: h:b:t:l = 0:1:0:0 Apr 03 18:46:21 | sda: serial = 356617175691262642 Apr 03 18:46:21 | sda: get_state Apr 03 18:46:21 | sda: path checker = directio (config file default) Apr 03 18:46:21 | sda: checker timeout = 30000 ms (sysfs setting) Apr 03 18:46:21 | sda: state = running Apr 03 18:46:21 | directio: starting new request Apr 03 18:46:21 | directio: io finished 4096/0 Apr 03 18:46:21 | sda: state = 3 Apr 03 18:46:21 | sda: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (config file default) Apr 03 18:46:21 | sda: uid = 3600508e0000000006c768fd4b28e7005 (callout) Apr 03 18:46:21 | sda: state = running Apr 03 18:46:21 | sda: detect_prio = 1 (config file default) Apr 03 18:46:21 | sda: prio = const (config file default) Apr 03 18:46:21 | sda: const prio = 1 Apr 03 18:46:21 | sdb: not found in pathvec Apr 03 18:46:21 | sdb: mask = 0x3f Apr 03 18:46:21 | sdb: dev_t = 8:16 Apr 03 18:46:21 | sdb: size = 1048576000 Apr 03 18:46:21 | sdb: subsystem = scsi Apr 03 18:46:21 | sdb: vendor = IBM Apr 03 18:46:21 | sdb: product = 2145 Apr 03 18:46:21 | sdb: rev = 0000 Apr 03 18:46:21 | sdb: h:b:t:l = 2:0:0:0 Apr 03 18:46:21 | sdb: tgt_node_name = 0x5005076802035582 Apr 03 18:46:21 | sdb: serial = 0340a04034caXX00 Apr 03 18:46:21 | sdb: get_state Apr 03 18:46:21 | loading /lib64/multipath/libchecktur.so checker Apr 03 18:46:21 | sdb: path checker = tur (controller setting) Apr 03 18:46:21 | sdb: checker timeout = 30000 ms (sysfs setting) Apr 03 18:46:21 | sdb: state = running Apr 03 18:46:21 | sdb: state = 3 Apr 03 18:46:21 | sdb: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting) Apr 03 18:46:21 | sdb: uid = 36005076d028100d3280000000000001f (callout) Apr 03 18:46:21 | sdb: state = running Apr 03 18:46:21 | sdb: detect_prio = 1 (config file default) Apr 03 18:46:21 | loading /lib64/multipath/libprioalua.so prioritizer Apr 03 18:46:21 | sdb: prio = alua (controller setting) Apr 03 18:46:21 | reported target port group is 1 Apr 03 18:46:21 | aas = 01 [active/non-optimized] Apr 03 18:46:21 | sdb: alua prio = 10 Apr 03 18:46:21 | sdc: not found in pathvec Apr 03 18:46:21 | sdc: mask = 0x3f Apr 03 18:46:21 | sdc: dev_t = 8:32 Apr 03 18:46:21 | sdc: size = 1048576000 Apr 03 18:46:21 | sdc: subsystem = scsi Apr 03 18:46:21 | sdc: vendor = IBM Apr 03 18:46:21 | sdc: product = 2145 Apr 03 18:46:21 | sdc: rev = 0000 Apr 03 18:46:21 | sdc: h:b:t:l = 2:0:1:0 Apr 03 18:46:21 | sdc: tgt_node_name = 0x5005076802035583 Apr 03 18:46:21 | sdc: serial = 0340a04034caXX00 Apr 03 18:46:21 | sdc: get_state Apr 03 18:46:21 | sdc: path checker = tur (controller setting) Apr 03 18:46:21 | sdc: checker timeout = 30000 ms (sysfs setting) Apr 03 18:46:21 | sdc: state = running Apr 03 18:46:21 | sdc: state = 3 Apr 03 18:46:21 | sdc: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting) Apr 03 18:46:21 | sdc: uid = 36005076d028100d3280000000000001f (callout) Apr 03 18:46:21 | sdc: state = running Apr 03 18:46:21 | sdc: detect_prio = 1 (config file default) Apr 03 18:46:21 | sdc: prio = alua (controller setting) Apr 03 18:46:21 | reported target port group is 0 Apr 03 18:46:21 | aas = 00 [active/optimized] Apr 03 18:46:21 | sdc: alua prio = 50 Apr 03 18:46:21 | dm-0: device node name blacklisted Apr 03 18:46:21 | sr0: device node name blacklisted Apr 03 18:46:21 | sdd: not found in pathvec Apr 03 18:46:21 | sdd: mask = 0x3f Apr 03 18:46:21 | sdd: dev_t = 8:48 Apr 03 18:46:21 | sdd: size = 1048576000 Apr 03 18:46:21 | sdd: subsystem = scsi Apr 03 18:46:21 | sdd: vendor = IBM Apr 03 18:46:21 | sdd: product = 2145 Apr 03 18:46:21 | sdd: rev = 0000 Apr 03 18:46:21 | sdd: h:b:t:l = 1:0:0:0 Apr 03 18:46:21 | sdd: tgt_node_name = 0x5005076802035583 Apr 03 18:46:21 | sdd: serial = 0340a04034caXX00 Apr 03 18:46:21 | sdd: get_state Apr 03 18:46:21 | sdd: path checker = tur (controller setting) Apr 03 18:46:21 | sdd: checker timeout = 30000 ms (sysfs setting) Apr 03 18:46:21 | sdd: state = running Apr 03 18:46:21 | sdd: state = 3 Apr 03 18:46:21 | sdd: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting) Apr 03 18:46:21 | sdd: uid = 36005076d028100d3280000000000001f (callout) Apr 03 18:46:21 | sdd: state = running Apr 03 18:46:21 | sdd: detect_prio = 1 (config file default) Apr 03 18:46:21 | sdd: prio = alua (controller setting) Apr 03 18:46:21 | reported target port group is 0 Apr 03 18:46:21 | aas = 00 [active/optimized] Apr 03 18:46:21 | sdd: alua prio = 50 Apr 03 18:46:21 | sde: not found in pathvec Apr 03 18:46:21 | sde: mask = 0x3f Apr 03 18:46:21 | sde: dev_t = 8:64 Apr 03 18:46:21 | sde: size = 1048576000 Apr 03 18:46:21 | sde: subsystem = scsi Apr 03 18:46:21 | sde: vendor = IBM Apr 03 18:46:21 | sde: product = 2145 Apr 03 18:46:21 | sde: rev = 0000 Apr 03 18:46:21 | sde: h:b:t:l = 1:0:1:0 Apr 03 18:46:21 | sde: tgt_node_name = 0x5005076802035582 Apr 03 18:46:21 | sde: serial = 0340a04034caXX00 Apr 03 18:46:21 | sde: get_state Apr 03 18:46:21 | sde: path checker = tur (controller setting) Apr 03 18:46:21 | sde: checker timeout = 30000 ms (sysfs setting) Apr 03 18:46:21 | sde: state = running Apr 03 18:46:21 | sde: state = 3 Apr 03 18:46:21 | sde: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting) Apr 03 18:46:21 | sde: uid = 36005076d028100d3280000000000001f (callout) Apr 03 18:46:21 | sde: state = running Apr 03 18:46:21 | sde: detect_prio = 1 (config file default) Apr 03 18:46:21 | sde: prio = alua (controller setting) Apr 03 18:46:21 | reported target port group is 1 Apr 03 18:46:21 | aas = 01 [active/non-optimized] Apr 03 18:46:21 | sde: alua prio = 10 Apr 03 18:46:21 | dm-1: device node name blacklisted Apr 03 18:46:21 | dm-2: device node name blacklisted Apr 03 18:46:21 | dm-3: device node name blacklisted Apr 03 18:46:21 | dm-4: device node name blacklisted ===== paths list ===== uuid hcil dev dev_t pri dm_st chk_st vend/prod 3600508e0000000006c768fd4b28e7005 0:1:0:0 sda 8:0 1 undef ready LSI,Logic 36005076d028100d3280000000000001f 2:0:0:0 sdb 8:16 10 undef ready IBM,2145 36005076d028100d3280000000000001f 2:0:1:0 sdc 8:32 50 undef ready IBM,2145 36005076d028100d3280000000000001f 1:0:0:0 sdd 8:48 50 undef ready IBM,2145 36005076d028100d3280000000000001f 1:0:1:0 sde 8:64 10 undef ready IBM,2145 Apr 03 18:46:21 | params = 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:32 1 8:4 8 1 round-robin 0 2 1 8:16 1 8:64 1 Apr 03 18:46:21 | status = 2 0 0 0 2 1 A 0 2 0 8:32 A 0 8:48 A 0 E 0 2 0 8:16 A 0 8:64 A 0 Apr 03 18:46:21 | Found matching wwid [3600508e0000000006c768fd4b28e7005] in bin dings file. Setting alias to mpatha Apr 03 18:46:21 | sda: ownership set to mpatha Apr 03 18:46:21 | sda: not found in pathvec Apr 03 18:46:21 | sda: mask = 0xc Apr 03 18:46:21 | sda: get_state Apr 03 18:46:21 | sda: state = running Apr 03 18:46:21 | directio: starting new request Apr 03 18:46:21 | directio: io finished 4096/0 Apr 03 18:46:21 | sda: state = 3 Apr 03 18:46:21 | sda: state = running Apr 03 18:46:21 | sda: const prio = 1 Apr 03 18:46:21 | mpatha: pgfailover = -1 (internal default) Apr 03 18:46:21 | mpatha: pgpolicy = failover (internal default) Apr 03 18:46:21 | mpatha: selector = round-robin 0 (internal default) Apr 03 18:46:21 | mpatha: features = 0 (internal default) Apr 03 18:46:21 | mpatha: hwhandler = 0 (internal default) Apr 03 18:46:21 | mpatha: rr_weight = 1 (internal default) Apr 03 18:46:21 | mpatha: minio = 1 rq (config file default) Apr 03 18:46:21 | mpatha: no_path_retry = NONE (internal default) Apr 03 18:46:21 | pg_timeout = NONE (internal default) Apr 03 18:46:21 | mpatha: retain_attached_hw_handler = 1 (config file default) Apr 03 18:46:21 | mpatha: set ACT_CREATE (map does not exist) Apr 03 18:46:21 | mpatha: domap (0) failure for create/reload map Apr 03 18:46:21 | mpatha: ignoring map #cat /sys/block/sdb/device/vendor IBM #cat /sys/block/sdb/device/model 2145 #cat /etc/multipath/bindings mpatha 3600508e0000000006c768fd4b28e7005 mpathb 36005076d028100d3280000000000001f The above file shows two path.Even after multipath is installed it is still showing four path. In multipath.conf the following is enabled defaults { user_friendly_names yes } blacklist { # devnode ".*" } Do I need to enable anything in the multipath.conf file, was there anything wrong. I used fdisk /dev/sdb to partition it. |
The output of 'multipath -ll' shows the 4 paths to mpathb (i.e. /dev/sdb, /dev/sdc, /dev/sdd, and /dev/sde all point to the LUN).
The output of 'multipath -v3' shows that your internal RAID is being picked by multipath (Apr 03 18:46:21 | sda: ownership set to mpatha), but no mapping gets created, so it doesn't show up in the 'multipath -ll' listing. The 'cat /etc/multipath/bindings' is showing the 2 mappings that multipath tried to create - mpathb, which is the LUN, and mpatha, which is the map that didn't actually get created. You can blacklist the local device. For example, adding the following to the multipath.conf should ignore it: Code:
blacklist { |
Thank for the reply. I will blacklist my local device. I have a dobut after configuring multipath how many LUNs will be shown in fdisk. In my case it shows 4 disks will it show four. As said I should use multipath device and not the individual paths. Can you please clarify how to use it, as I am new to this concept.
|
Quote:
The device files /dev/sdb, /dev/sdc, /dev/sdd, and /dev/sde represent individual connections to the LUN (e.g. 2 dual port HBAs connected on all ports will provide 4 paths to the storage, so 4 /dev/sd* devices). These are not separate LUNs, just different ways to reach the same LUN. When you use the storage, use the /dev/mapper/mpathb device, not the /dev/sd* devices. So partition, create file systems/physical volumes, etc. on the /dev/mapper/mpathb device. What multipath does is monitor those /dev/sd* paths, so if one goes down, the LUN is still reachable through /dev/mapper/mpathb because it will automatically switch to a working path. However, if you use /dev/sdb directly, and that path goes down, then your device is unavailable even though other paths are up because you've chosen one specific way to reach the storage. |
Hi sgrlscz,
Thanks for the clear explanation you have given. Apolgoies for the delayed response.In continuation I did a fresh installation and it showed as follows Disk /dev/mapper/mpathb -LUN Disk /dev/mapper/sda Disk /dev/mapper/sdb - Four paths from SAN Disk /dev/mapper/sdc Disk /dev/mapper/sdd Now my setup has 3 standard partitons /boot, /boot/efi, swap outside and LVM partitions has /data /var partitions which will grow. I have to allocate and increase the size of those two partitions alone(/data and /var in LVM). Now I have mpathb, which can I use as an extended partition and use 8e, convert it as LVM partition, is it possible. Also I have used partprobe command instead of kpartx command to register LUN. Is that correct?Thanks in advance. |
There is no problem with using /dev/mapper/mpathb with LVM. I do it all of the time. As far as partprobe vs. kpartx, on RHEL 6, I've never really needed to use either. On RHEL 5, I used kpartx after partitioning a LUN, but on RHEL 6, it hasn't been necessary.
For LVM, you should modify the filter in the lvm.conf so that it will skip the individual paths to the LUN and pick up the multipath device. If you don't filter out the individual paths, LVM could choose one of those paths, and you would lose the benefits of multipathing. After changing the filter, you should verify it with the pvscan/vgscan tools before any rebooting of the system so that you're sure you haven't filtered out something you need. For instance, if you're running LVM on the local disk, you don't want to filter that out, but you do want to filter out the other /dev/sd* devices. Also, I usually set the preferred_names in the lvm.conf so LVM will favor the /dev/mapper/mpath* name over /dev/dm-* names. With preferred_names set, when you display LVM information, you will get the descriptive names for the devices. It doesn't impact the way LVM works, it just provides a preference for how information gets displayed. |
Hi,
I thank you for the quick response for fixing the issue.I have made the following changes in the lvm.conf files filter = [ "^/dev/mpath/*/", "r/.*/" ] preferred_names = [ "^/dev/mpath/", "^/dev/mapper/mpath", "^/dev/[hs]d" ] Please clarify. Also I have created the Lun as extended partition as follows fdisk /dev/mapper/mpathb n -new e - extended partiion partition -1 n -new l -logical partition w After i created this i tried to change the file system type to LVM by giving 8e but it throws the following error You cannot change a partition into an extended one or vice versa Delete it first. I will be getting 6-8 LUNS of 500GB size and need to extend my LVM partition /data and /var alone, so I went with extended partitions as I cannot create more than 4 primary partitions. But it throws error. I want to make the extended partition LVM. After partition it showed mpathbp5 as extended mpathbp6 as Linux and shows unable to read mpathb5. I have used mpathbp6 to to create a new disk. I know I made mistake, can you please throw some light on this. Thanks in advance. |
Additional LUNs will be new disks, so the partitioning on the current LUN has no impact on any LUNs that may be added down the line. Partition tables are per device, so there is no reason to use an extended partition.
If you're going to be using the entire LUN in the LVM volume group, you can use the whole device or create a single primary partition that spans the whole disk. You don't seem to have much understanding of LVM. Since you're using RHEL 6, you should take a look at https://access.redhat.com/site/docum...dministration/ or any of the dozens of tutorials on LVM. |
I will go through the document as said, but for now can you plese clarify this when i used
fdisk /dev/mapper/mpatha and primary partition 1 it allowed again presented a lun and used primary partition it allowed, but upon presenting the third LUN and create as primary it said no free sectors available. I already have 2 primary partitions /boot |
Please supply the results of 'multipath -ll' and 'fdisk -l'.
Also, what are the exact commands you are running? |
Hi,
I thank you very much for helping me to this extent. After going through the documentation I have understood LVM concepts to some extent. Initially I have used extended partition on mpathb. Now I have configured new LUNS as below. Initial output of fdisk ________________________ [root@mail ~]# fdisk -l WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 299.0 GB, 298999349248 bytes 255 heads, 63 sectors/track, 36351 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 36352 291991551+ ee GPT Partition 1 does not start on physical sector boundary. Disk /dev/mapper/vg_mail-RootLV: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdb1 1 133674 1073736373+ 5 Extended /dev/sdb5 1 133674 1073736342 83 Linux Disk /dev/sdc: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdc1 1 133674 1073736373+ 5 Extended /dev/sdc5 1 133674 1073736342 83 Linux Disk /dev/sdd: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdd1 1 133674 1073736373+ 5 Extended /dev/sdd5 1 133674 1073736342 83 Linux Disk /dev/sde: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sde1 1 133674 1073736373+ 5 Extended /dev/sde5 1 133674 1073736342 83 Linux Disk /dev/mapper/mpathb: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/mapper/mpathbp1 1 133674 1073736373+ 5 Extended /dev/mapper/mpathbp5 1 133674 1073736342 83 Linux Unable to read /dev/mapper/mpathbp1 Multipath Output _________________ [root@mail ~]# multipath -ll mpathe (36005076d028100d32800000000000039) dm-9 IBM,2145 size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 1:0:1:3 sdk 8:160 active ready running | `- 2:0:1:3 sdq 65:0 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 1:0:0:3 sdh 8:112 active ready running `- 2:0:0:3 sdn 8:208 active ready running mpathd (36005076d028100d3280000000000003b) dm-8 IBM,2145 size=3.0G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 1:0:1:1 sdi 8:128 active ready running | `- 2:0:1:1 sdo 8:224 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 1:0:0:1 sdf 8:80 active ready running `- 2:0:0:1 sdl 8:176 active ready running mpathc (36005076d028100d3280000000000003a) dm-7 IBM,2145 size=2.0G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 1:0:0:2 sdg 8:96 active ready running | `- 2:0:0:2 sdm 8:192 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 1:0:1:2 sdj 8:144 active ready running `- 2:0:1:2 sdp 8:240 active ready running mpathb (36005076d028100d32800000000000035) dm-1 IBM,2145 size=1.0T features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 2:0:0:0 sdb 8:16 active ready running | `- 1:0:0:0 sdd 8:48 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 2:0:1:0 sdc 8:32 active ready running `- 1:0:1:0 sde 8:64 active ready running Added New LUNs as below ________________________ [root@mail ~]# fdisk /dev/mapper/mpathc WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): d No partition is defined yet! Command (m for help): m Command action a toggle a bootable flag b edit bsd disklabel c toggle the dos compatibility flag d delete a partition l list known partition types m print this menu n add a new partition o create a new empty DOS partition table p print the partition table q quit without saving changes s create a new empty Sun disklabel t change a partition's system id u change display/entry units v verify the partition table w write table to disk and exit x extra functionality (experts only) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-261, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261): Using default value 261 Command (m for help): t Selected partition 1 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 22: Invalid argument. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks. [root@mail ~]# partprobe /dev/mapper/mpathc [root@mail ~]# fdisk /dev/mapper/mpathd Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x1cc4dd73. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-391, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-391, default 391): Using default value 391 Command (m for help): t Selected partition 1 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 22: Invalid argument. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks. [root@mail ~]# partprobe /dev/mapper/mpathd [root@mail ~]# ls -la /dev/mapper total 0 drwxr-xr-x 2 root root 300 Apr 16 01:36 . drwxr-xr-x 17 root root 4820 Apr 16 01:36 .. crw-rw---- 1 root root 10, 58 Apr 15 16:21 control lrwxrwxrwx 1 root root 7 Apr 16 01:19 mpathb -> ../dm-1 lrwxrwxrwx 1 root root 7 Apr 16 01:19 mpathbp1 -> ../dm-2 lrwxrwxrwx 1 root root 7 Apr 16 01:19 mpathbp5 -> ../dm-3 lrwxrwxrwx 1 root root 7 Apr 16 01:35 mpathc -> ../dm-7 lrwxrwxrwx 1 root root 8 Apr 16 01:35 mpathcp1 -> ../dm-10 lrwxrwxrwx 1 root root 7 Apr 16 01:36 mpathd -> ../dm-8 lrwxrwxrwx 1 root root 8 Apr 16 01:36 mpathdp1 -> ../dm-11 lrwxrwxrwx 1 root root 7 Apr 16 01:19 mpathe -> ../dm-9 lrwxrwxrwx 1 root root 7 Apr 15 16:21 vg_mail-DataLV -> ../dm-6 lrwxrwxrwx 1 root root 7 Apr 15 16:21 vg_mail-ImapLV -> ../dm-5 lrwxrwxrwx 1 root root 7 Apr 15 16:21 vg_mail-RootLV -> ../dm-0 lrwxrwxrwx 1 root root 7 Apr 15 16:21 vg_mail-VarLV -> ../dm-4 [root@mail ~]# fdisk /dev/mapper/mpathe Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x9258920a. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-130, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): Using default value 130 Command (m for help): t Selected partition 1 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 22: Invalid argument. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks. [root@mail ~]# partprobe /dev/mapper/mpathe Final fdisk Output ___________________ [root@mail ~]# fdisk -l WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 299.0 GB, 298999349248 bytes 255 heads, 63 sectors/track, 36351 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 36352 291991551+ ee GPT Partition 1 does not start on physical sector boundary. Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdb1 1 133674 1073736373+ 5 Extended /dev/sdb5 1 133674 1073736342 83 Linux Disk /dev/sdc: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x1cc4dd73 Device Boot Start End Blocks Id System /dev/sdc1 1 391 3140676 8e Linux LVM Disk /dev/sde: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x9258920a Device Boot Start End Blocks Id System /dev/sde1 1 130 1044193+ 8e Linux LVM Disk /dev/sdd: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x206d6f8b Device Boot Start End Blocks Id System /dev/sdd1 1 261 2096451 8e Linux LVM Disk /dev/sdg: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x1cc4dd73 Device Boot Start End Blocks Id System /dev/sdg1 1 391 3140676 8e Linux LVM Disk /dev/sdh: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x206d6f8b Device Boot Start End Blocks Id System /dev/sdh1 1 261 2096451 8e Linux LVM Disk /dev/sdf: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdf1 1 133674 1073736373+ 5 Extended /dev/sdf5 1 133674 1073736342 83 Linux Disk /dev/sdi: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x9258920a Device Boot Start End Blocks Id System /dev/sdi1 1 130 1044193+ 8e Linux LVM Disk /dev/sdj: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdj1 1 133674 1073736373+ 5 Extended /dev/sdj5 1 133674 1073736342 83 Linux Disk /dev/sdl: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x206d6f8b Device Boot Start End Blocks Id System /dev/sdl1 1 261 2096451 8e Linux LVM Disk /dev/sdk: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x1cc4dd73 Device Boot Start End Blocks Id System /dev/sdk1 1 391 3140676 8e Linux LVM Disk /dev/sdm: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x9258920a Device Boot Start End Blocks Id System /dev/sdm1 1 130 1044193+ 8e Linux LVM Disk /dev/sdn: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/sdn1 1 133674 1073736373+ 5 Extended /dev/sdn5 1 133674 1073736342 83 Linux Disk /dev/sdo: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x1cc4dd73 Device Boot Start End Blocks Id System /dev/sdo1 1 391 3140676 8e Linux LVM Disk /dev/sdp: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x206d6f8b Device Boot Start End Blocks Id System /dev/sdp1 1 261 2096451 8e Linux LVM Disk /dev/sdq: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x9258920a Device Boot Start End Blocks Id System /dev/sdq1 1 130 1044193+ 8e Linux LVM Disk /dev/mapper/vg_mail-RootLV: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/mapper/mpathb: 1099.5 GB, 1099511627776 bytes 255 heads, 63 sectors/track, 133674 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x2435608a Device Boot Start End Blocks Id System /dev/mapper/mpathbp1 1 133674 1073736373+ 5 Extended /dev/mapper/mpathbp5 1 133674 1073736342 83 Linux Disk /dev/mapper/mpathd: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x1cc4dd73 Device Boot Start End Blocks Id System /dev/mapper/mpathdp1 1 391 3140676 8e Linux LVM Disk /dev/mapper/mpathe: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x9258920a Device Boot Start End Blocks Id System /dev/mapper/mpathep1 1 130 1044193+ 8e Linux LVM Disk /dev/mapper/mpathc: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x206d6f8b Device Boot Start End Blocks Id System /dev/mapper/mpathcp1 1 261 2096451 8e Linux LVM Unable to read /dev/mapper/mpathbp1 I hope what i did may be correct for LUN mpathc,mpathd and mpathe. Can you please throw some light on this. #pvcreate /dev/mapper/mpathcp1 /dev/mapper/mpathdp1 /dev/mapper/mpathep1 for pvcreation Also I was given a suggestion that i can directly add as follows instead of using fdisk as LVM doesnt look partition table at all. #pvcreate /dev/mapper/mpathc /dev/mapper/mpathd /dev/mapper/mpathe Thanks in advance |
Quote:
After you do the pvcreate, you need to use either vgcreate (new volume group) or vgextend (add to existing volume group) to add the LUNs to a volume group. Then you'd do either a lvcreate (new) or lvextend (grow existing). Finally, a mkfs (new) or resize2fs (grow - assuming an ext filesystem). Quote:
I use partitions because it makes it easier for me and my coworkers to identify disks in use. Where I work, we're always adding new storage to servers, and it makes it easier for the SAs to identify it if we can just look at the partition tables of the disks to see how the devices are being used. If a device doesn't have a partition table, it's new, and that's what we need to configure. |
Hi sgrlscz,
At the outset I thank and appreciate you very much for clarifying and providing support to the issues. I am marking the thread as resolved. Thank you once again for all. |
All times are GMT -5. The time now is 07:04 PM. |