I have RHEL 5 64-bit loaded on a blade. The server is boot from SAN - using IBM SVC.

On Windows hosts, with the help of IBM's SDDDSM driver, we can expand vDisks on the SVC side and just initiate a rescan of disks in the Disk Management on the server and it will pick it up - does not require a reboot.

I cannot seem to find out how to do this in RHEL 5. The multipath is loaded and working properly from what I can tell. I know how to add a new vDisk and add it to the exceptions in /etc/multipath.conf and then do a service multipathd reload. But I cannot get the system to detect the new size of the disk at run-time. The server will pick up the new size at a reboot but that's not what we're striving for!

Here's the output of multipath -ll:
mpath1 (360050768019001b9b00000000000009a) dm-7 IBM,2145
[size=10G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=200][active]
\_ 0:0:2:1 sdc 8:32 [active][ready]
\_ 0:0:3:1 sdd 8:48 [active][ready]
\_ 1:0:2:1 sdk 8:160 [active][ready]
\_ 1:0:3:1 sdl 8:176 [active][ready]
\_ round-robin 0 [prio=40][enabled]
\_ 0:0:0:1 sda 8:0 [active][ready]
\_ 0:0:1:1 sdb 8:16 [active][ready]
\_ 1:0:0:1 sdi 8:128 [active][ready]
\_ 1:0:1:1 sdj 8:144 [active][ready]
mpath0 (360050768019001b9b000000000000099) dm-0 IBM,2145
[size=85G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=200][active]
\_ 0:0:4:0 sde 8:64 [active][ready]
\_ 0:0:5:0 sdf 8:80 [active][ready]
\_ 1:0:4:0 sdm 8:192 [active][ready]
\_ 1:0:5:0 sdn 8:208 [active][ready]
\_ round-robin 0 [prio=40][enabled]
\_ 0:0:6:0 sdg 8:96 [active][ready]
\_ 0:0:7:0 sdh 8:112 [active][ready]
\_ 1:0:6:0 sdo 8:224 [active][ready]
\_ 1:0:7:0 sdp 8:240 [active][ready]

The mpath0 is the first vDisk that the system boots to. I've increased the size on mpath1 to 15GB but I can't seem to get it to see the new size.

We want a system where we can add storage to a server if needed and expand the LVM volume groups as needed without interruption of service. Any ideas?

Thanks!

Chad

Hi

You can try this:
just replace x with the host nr.

and you should try on a testsystem first!!!!!!!

I forgot to say that:

[root@test ~]# echo "- - -" > /sys/class/scsi_host/host0/scan
[root@test ~]# echo "- - -" > /sys/class/scsi_host/host1/scan

AND

[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:0\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:1\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:2\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:3\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:4\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:5\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:6\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/0\:0\:7\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:0\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:1\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:2\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:3\:1/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:4\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:5\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:6\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/1\:0\:7\:0/rescan
[root@test ~]# echo 1 > /sys/bus/scsi/devices/4\:0\:0\:0/rescan

Do not work. The multipath daemon cannot be restarted since the server is boot from san meaning that / is being run by the multipath daemon. We can do reloads of the daemon but that doesn't pick up the new storage.

Any other ideas?

Chad

The rescans do seem to work for the scsi devices to see the new storage. For example, /dev/sdi is now showing the extra storage when I do a fdisk -l command. It looks like the issue is that the multipath daemon is not picking up the new storage and putting it in its multipath device.

Chad

So, You resize the LUN? well, we use PowerPath (EMC) instead of native multipathing and we always just define new LUNs and then add them to the volume group.

Yes that's right. We expand the virtual disk on the SAN (IBM's SVC in this case) and try to get the host to recognize the new size so that we can define new partitions on the expanded space and then add them to an existing volume group. Method works great if the system is a VM because there's no guest OS multipathing going on there.

Is powerpath a multipath tool from EMC for Linux? I wonder if there's something else we can use on Red Hat instead of the native multipathing. I'm going to open a ticket with Red Hat to see what they have to say.

ok. let me know what they tell you. but, why do you prefer resizing the LUN instead of defining another one?

cheers

Found the solution. We prefer to resize the lun on the SVC so that we don't have cluttered vDisks defined. Its better just to expand the vDisk - in our opinion

How to add a virtual disk to a Red Hat physical server.

1. Create the virtual disk in the SVC and map it to the host.
2. Log in as root and open a bash terminal.
3. Re-scan for the new multipath device:
a. echo “1” > /sys/class/fc_host/host0/issue_lip
b. echo “1” > /sys/class/fc_host/host1/issue_lip
c. echo “- - -“ > /sys/class/scsi_host/host0/scan
d. echo “- - -“ > /sys/class/scsi_host/host1/scan
4. Find the wwid for the new virtual disk:
a. multipath -v3
b. You should see the 8 identical wwid’s listed near the bottom of this output. You are looking for something like 360050768019001b9b0000000000000aa. It will not be identical to this as all wwid’s are different but it will generally match the ID that is listed in the SVC. It will generally be identical to SVC’s wwid for this virtual disk with a 3 added to the front. RHEL will have it “blacklisted” because it has not been entered in the system to be used.
5. Add the wwid of the new virtual disk.
a. Record the wwid and open /etc/multipath.conf.
b. In the “blacklist_exceptions” stanza, enter in “wwid <wwid>”
c. Save the file and exit.
6. Reload the multipath daemon:
a. service multipathd reload
7. Verify the new multipath device:
a. multipath -v3

The device is now ready to be used like any other scsi device in RHEL. However, instead of /dev/sda for example, you will use /dev/mapper/mpath# in your commands.

For increasing storage, do the following:

1. Expand the vDisk on the SVC side.
2. Rescan all devices. Execute this for each device:
a. echo “1” > /sys/class/scsi_disk/<device>/device/rescan
b. echo "1" > /sys/class/fc_host/host#/issue_lip
c. echo "- - -" > /sys/class/scsi_host/host#/scan
3. Look to see if the scsi devices see the new storage using a “fdisk –l" command.
4. Go into the multipath daemon command prompt
a. multipathd -k
b. resize map <multipath-device>
5. Create a new partition using parted on the multipath interface. You will receive an error of ioclt device busy do not worry about these errors.
6. Make it lvm in parted:
a. set <partition-number> lvm on
7. Execute: partprobe
8. Create the physical volume on the partition
a. pvcreate /dev/mapper/<multipath-device>p<partition-number>
9. Use the LVM gui to add it in and expand the logical volume.

Let me know if you have any questions.

Chad