Following your incredibly detailed and well-written guide, I was able to set up GPU passthrough to a Windows 7 guest using a Radeon HD6670 card and Ubuntu 12.10 Desktop as dom0. Thanks for the guide!
Originally Posted by heiko_s
But I too would prefer a server-based dom0, and so I've been experimenting with XCP 1.6, which is an open source version of XenServer. Using XCP 1.6, I've had good luck with GPU passthrough to a Windows 7 guest using the Catalyst 13.4 driver with a Radeon HD6670 card.
However, at least for me, getting GPU passthough to work with a Linux guest has been much more of a challenge, regardless of whether I use XCP 1.6 or Ubuntu 12.10 as dom0. Using XCP 1.6 as dom0, I've tried Ubuntu 12.04, 12.10 and 13.04 as guests, each with various display drivers. So far, of the nearly innumerable combinations that I tried, the only combination that has worked for me has been Ubuntu 12.04 combined with whatever default version of Catalyst (not the update) that is found in the Ubuntu repository.
So to reiterate, in addition to GPU passthrough to a Windows 7 guest, GPU passthrough to an Ubuntu 12.04 guest is also possible using XCP 1.6 as dom0. Passthrough to other Linux guests may also be possible using proper settings. I would greatly appreciate hearing about any successes others have had with GPU passthough to other Linux guests, whether using vanilla Xen, XCP or even KVM.
Regarding XCP 1.6, although passthrough of the primary GPU to a guest is possible, based on my experiences, I do not recommend doing so. Rather, I recommend devoting the primary video card to the XCP console. But the good news is that even a cheap card is sufficient for use as primary video card for the XCP console. And if the bios settings of your motherboard allow for selecting between PCI and PCIe as the primary boot graphics card (my Asus M5A99FX motherboard allows for this), to conserve your PCIe slots, I recommend devoting a cheap PCI card (not PCIe card) to the XCP console. In may case, an ATI Rage 8 MB PCI Video Graphics Card ($10), which only offers VGA output, works great for me as the primary graphics card devoted to the XCP console. (Other PCI cards, such as those having DVI and/or HDMI output my be more suitable for some applications.)
The XCP console provides an easy (menu-based) means for starting and stopping VMs. From its menus, you can also find lots of system information. However, XCP relies on "xe" rather than "xm" or "xl" for management. And unfortunately, "xe" commands can be a bit cumbersome. So many prefer a GUI based approach for managing more complicated tasks in XCP, with the Windows-based XenCenter currently being the most popular option. (XenCenter can be run from a network-attached laptop.) Although XenCenter isn't open source, it is, at least as of the time of this writing, freely downloadable from Citrix after creating a free Citrix account. (XenOrchestra is an open source web GUI that is currently under development, but not yet ready for production.)
Even if you use XenCenter for VM management, you'll still need to use "xe" commands on limited occasions, such as for setting up PCI passthrough to USB controllers, etc. The PCI passthrough command in "xe" takes the following form:
The <uuid> for a given VM is obtainable from XenCenter and each <pci-id#> can be obtained using the familiar "lspci" command after connecting to XCP, such as via ssh. Here's an example using an "xe" command for passthrough of seven usb controllers to one VM:
xe vm-param-set other-config:pci=0/<pci-id#0>,1/<pci-id#1>,2/<pci-id#2>,n/<pci-id#n> uuid=<uuid>
To add or remove devices from passthough, shut down the VM, edit the command to add or remove the <pci-id#>, rerun the edited command, and then restart the VM. For example, to remove all devices from the VM in the above example, shutdown the VM and run the following command:
xe vm-param-set other-config:pci=0/0000:00:12.0,1/0000:00:12.2,2/0000:00:13.0,3/0000:00:13.2,4/0000:00:14.5,5/0000:00:16.0,6/0000:00:16.2 uuid=0370e4b4-5d5b-10b5-245b-74317cc049aa
I hope that the above is of help to anyone who wants to give XCP a try.
xe vm-param-set other-config:pci= uuid=0370e4b4-5d5b-10b5-245b-74317cc049aa