[SOLVED] Network Bonding Citrix Xenserver

al_bye · 04-16-2010, 09:42 AM

From the title this sort of thing belongs on a XEN forum. As Xen is based on LINUX I'm hoping that someone out there could help me. I'm struggling with 802.3ad on a cisco switch (LACP). I'm going to be using this for my storage network on a dedicated cisco switch talking to a teamed pair of intel NIC's ISCSI storage. From the intel side I can set them to Link aggregated one team 2GB this is fine and my config shows active active on the NICs. How would I go about doing this from a LINUX box. In order to remove any bottle necks it needs to be active active from Xen. If I do a pif-forget uuid= on a XEN server I'm in complete control using Linux. I don't know what distro xen is based on I think Red Hat "please don't quote me" any help I'd be grateful.

tkmsr · 04-17-2010, 05:07 AM

I am not that much into Cisco stuff so I could not understand your question completely but I do understand NAS/SAN . Xen in not based on any particular distro.
In fact xen is a hypervisor which can be used like any other application say mplayer works with all linux and then once hypervisor is there in your Linux box you can use xen management tools like xm or libvirt there are a few more which I forgot.

To be able to do networking on Xen you have a xend daemon running depending upon what sort of configuration you are running it takes
care of your network stuff.In some scenarios you may need to use IPTABLES to route etc but mostly you wont.
Now as I understand from your post you are not clear with networking on Xen if this is what you mean.
Check following links

http://wiki.xensource.com/xenwiki/XenNetworking
http://wiki.xensource.com/xenwiki/Ho...r_each_of_them.

if you want to contact Xen forum where more xen users are then you can post your question here
http://news.gmane.org/gmane.comp.emu...r/cutoff=56911
to subscribe
http://dir.gmane.org/gmane.comp.emulators.xen.user

and Citrix Xen server is based on Debian.

al_bye · 04-18-2010, 02:44 PM

Thanks for the links and info. I'll check out the Debian side of things. The weird thing is I can see everything communicating properly across the swtich but I'm unable to ping the CITRIX Xenserver. Either from the Switch or another computer I'm going to to try instead of 802.3ad to mode 4 and give it a go and post back on the results.

tkmsr · 04-18-2010, 10:22 PM

Okay well based on my understanding upto here I will say see if you have disabled ICMP echo on CitrixXen or some thing in your network is blocking ICMP echo.May be the switch you are referring to is configured like that.
Check if from CitrixXen server you are able to ping the gateway,DNS your routers.
As a basic rule if you are trying to ping VMs then IP forwarding should be enabled in your Xen server Dom0.
/etc/sysctl.conf
there is a variable #net.ipv4.ip_forward=1
enable it.
Also may be you can post your question here.
http://forums.citrix.com/support
and on LinuxQuestions.org
in Virtualization forum.
http://www.linuxquestions.org/questi...ualization-90/

al_bye · 04-19-2010, 04:01 AM

Thanks again for the reply. I know ICMP work's as I've split the data and mangement ports and both respond well. I've had a closer look and noticed that the bonded team have a status of suspended. I presume CITRIX Xenserver is having a hard time and just doesn't want to play ball. I've read that others have managed to get it going but I don't know how to display the output logs from a Dom0 view point.

I'll keep digging around.

Thanks

tkmsr · 04-19-2010, 09:02 AM

On my Debian Xen system the logs are in following directory.
/var/log/xen
You may like to check that or use netstat or some thing like that to find out where connections are going on.I had a similar problem not exactly as you are facing I was able to see my lan card up in my Dom0.
But network was unreachable.Doing an ifconfig showed the ip.
After some hit and trials I found that plugging the physical connection to a different lan card solved this issue.May not be the case with you but you may try.Sounds silly but it worked for me when I gave IP to my Dom0 it was by default activating eth2 and not eth0 as I expected it to.
There is a utility known as strace you may try to diagnose it.
There is a tool known as dtrace
http://www.brendangregg.com/dtrace.html
See if this can help you.
You may not need the following but just in case.
http://en.wikipedia.org/wiki/Strace
See if the following tools do help you any how
http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html
Try to do some network activity from your network card which is recognized by dom0.
I use Debian in that the settings are in
/etc/network/interfaces this is how the interfaces file look like

Code:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

iface eth2 inet static
address 192.168.1.18
netmask 255.255.255.0
gateway 192.168.1.1

auto eth2

If you have physical access to the box then
there is a live CD from Citrix Xen for free you can down load and use just an idea.It may help you do to any sort of change debug the problem.
http://wiki.xensource.com/xenwiki/LiveCD
http://forums.citrix.com/message.jspa?messageID=1418723
http://forums.citrix.com/message.jspa?messageID=1382681
https://forums.openfiler.com/viewtopic.php?id=3777
what ever architecture you have just try to use it rather than changing the original system.If you do solve the problem do post here.

The links I mentioned above I found this useful in your context

Code:

Not really but sort of.  802.3ad will attempt to push all the 
traffic from a single session over a single link to avoid reordering. 
If you start multiple sessions over the links from a single system 
it should use both (though again, it will probably start with both 
sessions on a single link).  It's also been my experience that many 
vendors ignore that "all sessions on a single link" thing and will 
spill over.  I have done more than 1Gbit/s between two Windows 
systems with dual Intel NICs on a single session.

Let me know if it works or even if it does not.

al_bye · 04-20-2010, 05:18 AM

Hi thanks for the links.

The only way I've managed to get this to work was to create two seperate ether channels on the single switch. What ever mode I used it would mark the switch ports as inactive (Config mismatch). My understanding is in order to gain throughput from Client to Storage it would require all four ports being in the same channel? Although CITRIX don't officially support 802.3ad I think they allow it for failover. In order for it to all sing happily they need to be in Link Aggregated mode which again my limited linux knowledge would be link mode 4. I know from a config point of view it's spot on as I link switches using etherchannels. But I'll crack on with the list you gave me to do once I've got Xenserver back to an orginal state and try again. Prooving a little more difficult do to limitations set from CITRIX.

al_bye · 04-20-2010, 02:40 PM

I'm sorry to say I've thrown in the towel on this one and gone back to multipathing. I was finding that everything I changed at linux level was being overwritten by CITRIX and their script files. Although some are well document some are hidden and I spent most of my time chasing my tail. Thank very much for your help. Out of interest do you use ISCSI as storage? If so what performance do you see?

tkmsr · 04-20-2010, 10:36 PM

No I am not using iSCSI.I have just planned to do a setup of SAN for some work.I am aware of it but never myself worked on it.Openfiler is the only thing that I am aware.I am setting up a streaming server.

al_bye · 04-22-2010, 02:37 PM

I'm glad to say I've cracked the problem. Maybe a proper solution was I lost site of main goal. I found a configuration problem with the disk controller. After running multiple routines network sniffing and disk analysis. Withing Xenserver all the disk's are controlled in /sys/block/<deviceID>/queue within that there's files which control the KB throughput and read ahead. Match these up with your vendor requirements and now I get 120MB/Sec read and around 100MB/SEC write.