VGA Passthrough with Threadripper

An unfortunate bug exists for the AMD Threadripper family of GPUs which causes VGA Passthrough not to work properly. Fortunately some very clever people have implemented a workaround to allow proper VGA passthrough until a proper Linux Kernel patch can be accepted and implemented. See here for the whole story.

Right now my Thrdearipper 1950x successfully has GPU passthrough thanks to HyenaCheeseHeads “java hack” applet.  I went this route because I really didn’t want to try and recompile my ProxMox kernel to get passthrough to work. Per the description “It is a small program that runs as any user with read/write access to sysfs (this small guide assumes “root”). The program monitors any PCIe device that is connected to VFIO-PCI when the program starts, if the device disconnects due to the issues described in this post then the program tries to re-connect the device by rewriting the bridge configuration.” Instructions taken from the above Reddit post.

  • Go to https://pastebin.com/iYg3Dngs and hit “Download” (the MD5 sum is supposed to be 91914b021b890d778f4055bcc5f41002)
  • Rename the downloaded file to “ZenBridgeBaconRecovery.java” and put it in a new folder somewhere
  • Go to the folder in a terminal and type “javac ZenBridgeBaconRecovery.java”, this should take a short while and then complete with no errors. You may need to install the Java 8 JDK to get the javac command (use your distribution’s software manager)
  • In the same folder type “sudo java ZenBridgeBaconRecovery”
  • Make sure that the PCIe device that you intend to passthru is listed as monitored with a bridge
  • Now start your VM

In my case (Debian Stretch, ProxMox) I needed to install openjdk-8-jdk-headless

sudo apt install openjdk-8-jdk-headless
javac ZenBridgeBaconRecovery.java

Next I have a little script on startup to spawn this as root in a detached tmux session, so I don’t have to remember to run it (If you try to start your VM before running this, it will hose passthrough on your system until you reboot it.) Be sure to change the script to point to wherever you compiled ZenBridgeBaconRecovery

#!/bin/bash
cd /home/nicholas  #change me to suit your needs
sudo java ZenBridgeBaconRecovery

And here is the command I use to run on startup:

tmux new -d '/home/nicholas/passthrough.sh'

Again, be sure to modify the above to point to the path of wherever you saved the above script.

So far this works pretty well for me. I hate having to run a java process as sudo, but it’s better than recompiling my kernel.


Update 6/27/2018:  I’ve created a systemd service script for the ZenBaconRecovery file to run at boot. Here is my file, placed in
/etc/systemd/system/zenbridge.service:  (change your working directory to match the zenbridgebaconrecovery java file location. Don’t forget to do systemctl daemon-reload.)

[Unit] 
Description=Zen Bridge Bacon Recovery 
After=network.target 

[Service] 
Type=simple 
User=root 
WorkingDirectory=/home/nicholas 
ExecStart=/usr/bin/java ZenBridgeBaconRecovery 
Restart=on-failure # or always, on-abort, etc 

[Install] 
WantedBy=multi-user.target 
~

2 thoughts on “VGA Passthrough with Threadripper”

  1. Hi Nicholas,

    What’s the latest on this setup, do you still use it regularly? We troubleshot for a good month back in Jan/Feb. MSI X399 + 1900x + GTX 770 or GTX 1030.

    The Java hack had me working for a week on my passthrough VM. Eventually, it fell apart, starting with graphics/performance issues. We experimented with different drivers… The stock NVidia drivers make the ZenBridgeBaconRecovery.java console spam dozens of errors every second. Nouveau had me working for most of the week, but it ultimately became unbootable. We’ve tried a multitude of Guest OSes, different video cards, different PCI slot positions, different IOMMU grouping flags in the GRUB config, different drivers, and finally, we went down the route of patching and compiling our own kernel.

    It was nice to have 16 cores to run make with… but, even with the kernel patch, I had similar issues. I’m running 4.15.12 (with tr.patch) now in ProxMox, and the guest OS works with a GTX 770 passed through without the Java hack, but dies and becomes unbootable similar to the last attempt.

    1. I currently use my threadripper daily with the java passthrough method and haven’t had any of the issues you’ve described (so far)

      Asrock X399 Taichi
      Threadripper 1950x
      GTX 1070
      Radeon HD 7450

      In reading your post it sounds like you only have one graphics card, is that correct? I haven’t tried that setup. My system has two video cards – one for Proxmox, and the other for the VM. I’ve configured Proxmox to blacklist both nouveau and nvidia drivers as well as add the card’s PCI ID to vfio.conf so the vfio driver claims that card on bootup (see details here.)

      I currently have the java hack running as a systemd init script on bootup. It works pretty well.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.