MD3000i multipath caching problem
 

We have 2 Dell servers and a MD3000i SAN.

The configuration is the following:
- 2 hosts both connectec to 2 switches which in turn are connected to de MD3000i.

multipath -ll gives:
mpatha (36a4badb0004e41e500000e264c284067) dm-0 DELL,MD3000i
size=3.0T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='round-robin 0' prio=12 status=active
| |- 5:0:0:1 sdg 8:96 active ready running
| `- 6:0:0:1 sdh 8:112 active ready running
`-+- policy='round-robin 0' prio=2 status=enabled
|- 4:0:0:1 sdc 8:32 active ghost running
`- 3:0:0:1 sdi 8:128 active ghost running

Both hosts are Fedora 13. We use dm-multipath (multipathd). Qemu-kvm is the hypervisor.
The problem is that when host1 writes something to the SAN host 2
won't see it until the host reestablishes a connection to the LUN (i.e. log of the iscsi targets and log in again). So writing a file on the LUN by host1 is not visible by host2 which has also mounted that LUN.

The consequense of the cachingproblem is that when a VM migrates to the other host it will crash because the virtual disk is different for host 1 and 2. Migrating with Qemu-kvm succeeds when both hosts are just rebooted.

Does anyone have a clue which component is caching? Is it the MD3000i itself? iSCSI-client (iscsiadm)? Multipathd? LVM?

Thanks advance

You are sharing the same LUN between two hosts, with simultaneous access, and no clustering set up. Either use GFS or something like clvm.

If you switch to RHEV, this will be done for you automatically.

Thanks a lot for this great tip dyasny! I'll try it out.

I can understand that writing simultaniously to the same shared lun by host1 and host2 would screw up the files but does it also explains why changes made by host1 are not seen by host2 (which only reads)?

thanks for your help!

Yes, that explains it. If you use a clustered FS, you will be able to see the changes from all the nodes right away

Great! thanks for your quick response.