KVM PCI device passthru with high performance and low overhead

 KVM supports passing through PCI devices to its guests. A typical libvirt xml statement looks like

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0012' bus='0x00' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'>
        <zpci uid='0x0001' fid='0x00000000'/>
      </address>
    </hostdev>

See KVM Virtual Server Management SC34-2752 for details.

With newer kernel and QEMU versions (e.g. those found in RHEL 10.2, SLES 16.1 and Ubuntu 25.10) it is now possible to improve performance and reduce overhead when doing so. 

You have to specify the parameter iommu.passthrough=1 on the guest kernel command line via zipl or /boot/loader/entries.

This will pin the guest in memory, but gets rid of the expensive RPCIT instructions for IO that are used by the guest to notify the host about memory used for IO to control host paging.

As of today KVM already pins memory in use by the guest for PCI I/O. With todays lazy free of dma memory large parts of guest memory will be pinned over time. So pinning the whole guest during startup only has a small downside in real life.

By checking in the guest

   cat /sys/bus/pci/devices/<device number>/iommu_group/type 

it is possible to check if everything worked. The file should say "identity".