I’ve recently upgraded to the latest version of Citrix Xenserver 7 (codenamed “Dundee”.) 7 is based on CentOS 7 and has a massive amount of changes under the hood. One such change was how they handle PCI Passthrough.
It took some time to figure PCI Passthrough out. 7 uses grub instead of extlinux for the bootloader. It appears to be grub2 but they don’t use the standard update-grub tool, rather you simply edit the config file and do nothing else.
After much searching I found this post which led me in the right direction. In Xenserver 7, for pci passthrough support you must do the following:
- Prepare the VM for PCI passthrough (this part hasn’t changed)
xe vm-param-set other-config:pci=0/0000:B:D.f uuid=<vm uuid>
- Modify /boot/grub/grub.cfg and append the following to the end of the module2 line (if you boot from EFI the file to modify is /boot/efi/EFI/xenserver/grub.cfg)
xen-pciback.hide=(B:D.f)
- Reboot
You will now be able to pass through hardware to your virtual machines in Xenserver 7. Hooray.
I’m having the same problem described here with Xenserver 7 (final release), trying to pass through an LSI HBA card to Win2012R2 VM
lspci output:
08:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev03)
The server is an HP Proliant DL-380 Gen9.
I did:
Wrote: pciback.hide=(08:00.0) to /boot/grub/grub.cfg and rebooted;
Ran: xe vm-param-set other-config:pci=0/0000:08:00.0 uuid=
When I boot up the VM, it stays forever in the blue window screen with cyrcling dots. If I clear the “other-config” param, the VM boots up normaly.
In grub.cfg make sure you append xen-pciback.hide=(B:D.f). Don’t forget the xen- in front of pciback. Also make sure you place that at the end of the module2 line.
Hi Nicholas,
with this configuration of the VM starts only if you have disconnected with the SAS cable from the HBA card
I have a similar setup and can see the LSI card but cannot seem to find the drives. I can see the card on the target vm but… I am very inexperienced with xenserver but am eager to learn, any suggestions would help a lot.
It is worth mentionning that if you are using EFI to boot Xenserver, the grub.cfg file you need modified is here:
/boot/efi/EFI/xenserver/grub.cfg
Thanks for the clarification. I’ll update it.
Hmm. I am not able to get any of my usb controllers to passthrough at all. Let me clarify, they don’t get passed to VMs but device connected to those usb ports don’t work at all. I wonder how you got it to work…It appears to be a bug. Here are some links of what I am talking about.
https://discussions.citrix.com/topic/378774-xenserver-7-usb-passthrough-on-xen-46/
https://bugs.xenserver.org/browse/XSO-435
I’m not sure why it isn’t working for you. It worked fine for me when I passed my storage controller through. A commenter above said that it worked for them if you unplug devices first and then plug them in later after VM boot. Does this problem exist in xenserver 6.5?
Nope. In 6.5 everything works fine. Have you tried passing through anything but storage controllers?
I haven’t. Very curious. Seeing as 7 has such a different architecture under the hood than 6 (moved from Cent5 to Cent7 base, systemd, etc) I guess a few bugs like this are to be expected.
I read that pci passthrough officially only supports storage devices. This used to work on 6.5, i guess it was just not a priority fix for 7…
Is there any comprehensive guide on PCI passthrough on Xen 7 out there? I am coming from Esxi where I had it passthrough the onboard Intel patsburg Sata/SAS controller to a VM (FreeNAS).on my supermicro X9DR3-F. It is much easier to do on esxi GUI. Unfortunately I can no longer use esxi because the new version 6.0 does not support an older nic – Chelsio S320 – on my system, whereas the xenserver happily picks it up. It seems most Xen configuration is CLI driven, I am struggling to understand the concepts. Basically, my issue is how to pass the on-board sata controller to a FreeNAS VM, the other issue I may have but have not really investigated is Xenserver support for Ethernet nic channel bonding. Any help or pointers will be greatly appreciated.
Unfortunately there isn’t a lot of documentation out there about PCI passthrough in Xenserver 7 (that’s mainly why I decided to write this blog post myself.) PCI Passthrough is not a high priority with Xenserver 7 it seems. There is much more documentation with Xenserver 6.5, though.
When I start VM
I got Hvm_Build error for ‘Could not load ELF image’
But when I remove other-config:pci
VM is start…
any suggestion?
after I update XenServer pack
Also getting Hvm_Build error for ‘Could not load ELF image’.
Won’t load VM.
Try following the steps outlined here: http://xenserver.org/blog/entry/pci-pass-through-on-xenserver-7-0.html
It looks like there was a new blog post over at xenserver.org regarding this. Give it a try and let me know if it fixed your issue. http://xenserver.org/blog/entry/pci-pass-through-on-xenserver-7-0.html
The config resets my XenServer….how do you clear de “other-config” param?
You can use the “xe vm-param-clear” command to clear any custom parameters you may have set.
Thanks!
It worked…but I still can´t pass PCI Devices…
Any other ideas?