Xen domUs on remote storage (iSCSI / NFS)

ternarybit · 07-11-2013, 09:05 AM

I'm tinkering with a Debian Xen dom0, curious about the ways I can host domUs. This is not production, I'm just learning.

I'm considering a setup that centralizes all irreplaceable data on a box running FreeNAS, which can export iSCSI or NFS. If I understand correctly, iSCSI would allow me to layer LVM on top, which is the preferred way to run domUs for flexibility and performance. NFS would be used if I did a file-based VM setup, which I consider second choice.

The way I *think* it would work best:

Code:

                        FreeNAS SAN
                        -----------
Raw Disks -> RAID -> zpool -> dataset -> iSCSI target


                        Xen Server
                        ----------

iSCSI initiator -> 1 pv -> 1 vg -> multiple LVs for VMs

Would anyone care to comment on the viability, merits and demerits of such a setup? I'm interested in comments on performance, maintainability, backup and snapshotting, and any other administrative concern in a production environment.

In short: is there a standard or generally-accepted way to deploy Xen domUs to remote storage, all these considered?

ternarybit · 07-14-2013, 09:37 AM

Maybe I've put some off with the whole "Xen domUs on a FreeNAS SAN" notion.

I suppose the heart of my question is "What are the best practices for deploying a Linux server running from a SAN?" From here I can flesh out my Xen question.

Ser Olmy · 07-29-2013, 08:48 PM

What you're describing is certainly possible, but it's not too common to use LVM or similar technology on top of iSCSI on the hypervisor side. The LVM/iSCSI part is usually configured the other way around:

Code:

              Storage system:
Raw disks -> RAID set -> LVM/zpool/other -> iSCSI Target

             Hypervisor node:
iSCSI Initiator -> (whatever) -> VM

You don't really need to set up LVM on the initiator side, since the storage system can increase the size of the iSCSI target directly. Of course, you can still use LVM inside the VM if you so wish.

If you let the VMs use iSCSI targets directly you can easily do live migration of VMs between hypervisor nodes. I don't think that's possible if the iSCSI device is part of an LVM setup on the hypervisor side.