I leave notes here that I find particularly worth remembering myself.

Multipath on Debian Lenny and Intel Modular Server (MFSYS25)

Posted 08-16-2010 at 02:56 AM by bittner
Updated 09-13-2010 at 04:33 AM by bittner (added link to FAI discussion)

The Intel Modular Server is a complete server system designed to simplify a complex fail-safe IT environment. One characteristic is that it provides virtual disks and two storage controllers for redundancy. Systems that use those virtual disks have two physical paths to access their disk(s). (NOTE: This is not a RAID solution against disk defects, but safeguards against a breakdown of a storage controller!)

When you install and boot Linux on such a machine you will see both paths (e.g. /dev/sda and /dev/sdb pointing to one and the same disk), but only one at a time can be accessed. Accessing the other disk will result in a I/O error (logged to /var/log/syslog on Debian Lenny):

Quote:

end_request: I/O error, dev sdb, sector 0

If one storage controller gives up the second one takes over, and the system still has access to the disk -- but: with a different device name! To transparently make use of this feature on a Linux system you can install the Multipath Tools which provide you with a "virtual device" to access the disk via the path that is intact and currently active.

Set up Multipath step-by-step (Debian Lenny)

There are two Debian packages that provide Multipath:

multipath-tools ... to set up multipathing in general, especially for all devices but the root file system
multipath-tools-boot ... to set up multipathing for the root file system too (which requires to boot a modified initramfs)

(In the following instructions you can safely ignore everything that refers to multipath-tools-boot, GRUB and the system disk if you run a system that doesn't have multiple paths to its root file system.)

All you have to do:

Install the packages multipath-tools and multipath-tools-boot
Create the Multipath configuration file /etc/multipath.conf
Replace references to the disk(s) in /etc/fstab by the mapped device(s)
Replace references to the root file system in GRUB's boot configuration file /boot/grub/menu.lst by the mapped device

1. Install packages

Code:

# apt-get install multipath-tools multipath-tools-boot

2. Multipath configuration file /etc/multipath.conf
The configuration I found working with the Intel Modular Server and two virtual disks on Debian Lenny:

Code:

##
## multipath-tools configuration file /etc/multipath.conf
##

defaults {
	user_friendly_names	yes
}

blacklist {
	devnode	"^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
	devnode	"^(hd|xvd)[a-z][[0-9]*]"
	devnode	"^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}

devices {
	device {
		vendor			"Intel"
		product			"Multi-Flex"
		path_grouping_policy	"group_by_prio"
		getuid_callout		"/lib/udev/scsi_id -g -u /dev/%n"
		prio_callout		"/sbin/mpath_prio_alua /dev/%n"
		path_checker		tur
		path_selector		"round-robin 0"
		failback		immediate
		rr_weight		uniform
		rr_min_io		100
		no_path_retry		queue
		features		"1 queue_if_no_path"
	}
}

multipaths {
	multipath {
		wwid	2222c00015552cceb
		alias	system
	}
	multipath {
		wwid	22209000155faaffa
		alias	database
	}
}

The Debian package does not create this file, you have to do it. (There is an annotated version of this configuration file in /usr/share/doc/multipath-tools/examples/ if you want to have all options explained and are too lazy to read the multipath.conf manpage.)

Note the wwid values in the multipaths section: You must fill in the ID value corresponding to the respective drive which is obtained executing the getuid_callout command from the above configuration and the usual device path, e.g.

Code:

# /lib/udev/scsi_id -g -u /dev/sda

Alternatively, you can also check out /dev/disk/by-id/ to figure out to which ID your disk has:

Code:

# ls -gG /dev/disk/by-id/

3. Adapt /etc/fstab

Edit your /etc/fstab (as root) and replace all occurences of /dev/sda by /dev/mapper/system-part (analogously for all other mapped devices), e.g.

Code:

# /etc/fstab: static file system information.

/dev/mapper/system-part1    /              ext3         rw,noatime,errors=remount-ro  0  1
/dev/mapper/system-part2    none           swap         rw                            0  2
/dev/mapper/database-part2  /var/db        ext3         rw,noatime,errors=remount-ro  0  1
/dev/sr0                    /media/cdrom0  udf,iso9660  ro,user,noauto                0  0

4. Adapt /boot/grub/menu.lst

Edit your /boot/grub/menu.lst (as root) and replace all occurences of /dev/sda by /dev/mapper/system-part. The default boot entry should look something like this:

Code:

title	Debian GNU/Linux, kernel 2.6.26-2-amd64
root	(hd0,0)
kernel	/boot/vmlinuz-2.6.26-2-amd64 root=/dev/mapper/system-part1 ro
initrd	/boot/initrd.img-2.6.26-2-amd64

Function Test

Now you can reboot your system. It should boot up as usual. Log in as root and check whether multipath is doing okay:

Code:

# multipath -ll
system (2222c00015552cceb) dm-0 Intel   ,Multi-Flex
[size=4.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=50][active]
 \_ 0:0:0:0 sda 8:0   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 0:0:1:0 sdc 8:32  [active][ready]
database (22209000155faaffa) dm-1 Intel   ,Multi-Flex
[size=800G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=50][active]
 \_ 0:0:1:1 sdd 8:48  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 0:0:0:1 sdb 8:16  [active][ready]

Both disks have 2 paths each, one of them is currently active (status active), the other one is on standby (status enabled) waiting to become active when the other path fails.

References

Intel Modular Server overview (Intel.com)
Threads tagged 'multipath' (LinuxQuestions.org)
Intel Modular Server System MFSYS25 Installation Experiences (ProxMox forum)
multipath-tools homepage (by author Christophe Varoqui)
How safely partition disks in a multipath environment? (discussion on FAI mailing list)

Multipath on Debian Lenny and Intel Modular Server (MFSYS25)

Comments