KVM PCI device passthru with high performance and low overhead

 KVM supports passing through PCI devices to its guests. A typical libvirt xml statement looks like

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0012' bus='0x00' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'>
        <zpci uid='0x0001' fid='0x00000000'/>
      </address>
    </hostdev>

See KVM Virtual Server Management SC34-2752 for details.

With newer kernel and QEMU versions (e.g. those found in RHEL 10.2, SLES 16.1 and Ubuntu 25.10) it is now possible to improve performance and reduce overhead when doing so. 

You have to specify the parameter iommu.passthrough=1 on the guest kernel command line via zipl or /boot/loader/entries.

This will pin the guest in memory, but gets rid of the expensive RPCIT instructions for IO that are used by the guest to notify the host about memory used for IO to control host paging.

As KVM on Z does pin the guest in memory anyway as of today when a PCI device is passed through, this has no immediate downside.

By checking in the guest

   cat /sys/bus/pci/devices/<device number>/iommu_group/type 

it is possible to check if everything worked. The file should say "identity".

Updated Publication: "Networking with PCI adapters and functions"

Coinciding with the switch to the new Network Express adapters, we have updated our documentation to include the new card here.

 From the introduction:

"This publication explores what PCI network adapters offer for network connections of Linux® on IBM Z® and IBM® LinuxONE. The publication applies to Network Express and RoCE Express2 or RoCE Express3 adapters."

Note that Network Express adapters now also support promiscuous mode as required by Open vSwitch in a KVM context! 

New Release: Ubuntu 25.04

Canonical released a new version of their Ubuntu server offering Ubuntu Server 25.04!

Highlights include IBM z17 support, in particular:

• Exploitation of the AI facility in the Telum II processor
• New Secure Execution and Crypto features
• Network Express support​
• Power consumption reporting​
  

See the announcement on the mailing list here, and the blog entry at Canonical with all Z-specific highlights here.

IBM z17 announced!!

Our latest offering in the IBM Z family, IBM z17, was announced yesterday. General availability will be June 18.

See the official announcement, the updated Linux support matrix, and the Technical Guide with all the insights you will need.

Migration fails with something like ..pckmo.. ?

 If you run into a situation where migration fails with something like

internal error: QEMU unexpectedly closed the monitor (vm='testguest'):
qemu-kvm: Some features requested in the CPU model are not available in the current configuration: pckmo-aes-256 pckmo-aes-192 pckmo-aes-128 pckmo-etdea-192 pckmo-etdea-128 pckmo-edea msa9_pckmo Consider a different accelerator, QEMU, or kernel version

This indicates that both host are configured differently regarding the CPACF key management Operations

So you can (preferred if the security scheme allows for that) configure both LPARs the same way, de-activate and re-activate the LPAR

or

you can change the CPU model of the guest to no longer have these pckmo functions. Change your guest xml from "host-model" to "host-model with some features disabled".

So shutdown the guest and change the XML from

  <cpu mode='host-model' check='partial'/>

to

  <cpu mode='host-model' check='partial'>
    <feature policy='disable' name='msa3'/>

    <feature policy='require' name='msa3-base'/>
    <feature policy='disable' name='msa9_pckmo'/>
  </cpu>

This guest can now be migrated to a host without key managment functions. The downside is that this guest no longer has access to the key management functions. This makes it harder to use encrypted swap devices with automatic key generation. Therefore, enabling these functions on both LPARs is preferred.

New Feature: Installation Assistant for Linux on IBM Z

Ever struggled to create configuration files for starting Linux on IBM Z and LinuxONE installations? Fear no more, we got you covered now: A new assistant available online will help you create parameter files!
Writing parameter files can be a challenge, with bugs triggering cycles with lengthy turnaround times. Our new installation assistant generates installer parameter files by walking you through a step-by-step process, where you answer simple questions to generate a parameter file. Comes with contextual help in every stage, so you can follow along what is happening!
Currently supports OSA and PCI networking devices, IPv4/v6, and VLAN installations.

Currently supports RHEL 9 and SLES 15 SP5 or later.
Access the assistant at https://ibm.github.io/liz/

New Video: Configuring Crypto Express Adapters for KVM Guests

A new video illustrating the steps to perform on a KVM host and in a virtual server configuration to make AP queues of cryptographic adapters available to a KVM guest can be found here.