Put Xen dom0 to sleep with active pci passthrough VMs

Thanks to Xen 4.3, it is now possible to suspend / resume dom0 while domUs are running. Unfortunately, if you have a VM actively using pci passthrough, the whole machine completely locks up about 10 seconds after resuming from S3 sleep.

As Xen is a lot more geared to servers I realize I might be an edge case; However, I would really like to be able to suspend my entire machine to S3 with VMs actively using PCI passthrough (in my case, a video card and USB controller). For quite some time I thought I was out of luck. After learning about the hot swapping capabilities of Xen 4.2+ and its pciback driver, I thought I would take another whack at it.

My solution is to create a custom script which detaches all PCI passthrough devices on the VM before going to sleep. That same script would re-attach those devices to my VM on resume.

My dom0 is currently Linux Mint 16 so I placed the resulting script in the /etc/pm/sleep.d/ directory and named it 20_win8.1 . It works like a charm! I can suspend and resume to my heart’s content without having to worry about if I remembered to shut down my VM first.

My script is below. Be sure to modify it for the BDF of your devices and the name of your VM(s) if you decide to use it.

#!/bin/bash
#Sleep / hibernate script for Xen with active DomUs using PCI Passthrough
#This script is necessary to avoid freezing of dom0 on resume for Xen 4.3
#Modified 08/19/2014

#Name of the VM we're passing PCI things to
VM="win8.1"

#B:D.F of PCI devices passed through to VM
VIDCARD="01:00.0"
VIDCARDAUDIO="01:00.1"
FRONTUSB="00:1d.0"

#xen attach/detach commands. Replace with xm if you're using that toolstack instead
ATTACH="xl pci-attach"
DETACH="xl pci-detach"

case "$1" in
    hibernate|suspend)
        $DETACH $VM $VIDCARD
        $DETACH $VM $VIDCARDAUDIO
        $DETACH $VM $FRONTUSB
        ;;
    thaw|resume)
        $ATTACH $VM $VIDCARD
        $ATTACH $VM $VIDCARDAUDIO
        $ATTACH $VM $FRONTUSB
        ;;
    *)
        ;;
esac
exit $?

4 thoughts on “Put Xen dom0 to sleep with active pci passthrough VMs”

  1. Hi,
    Why don’t you use Qubes OS? Its based on Xen and handles sleep properly even with PCI passthrough enabled (in fact, Qubes will use it for NICs whenever VT-d is available). As a benefit you also get nice GUI features for your VMs, enhanced security and secure copy/paste/filecopy between VMs.

    1. I have read about Qubes OS and it does sound intriguing. Does it have the ability to run Windows DomUs? I was under the impression that it couldn’t do Windows, but if it can, I will definitely give it another look.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.