QEMU v3.1 released

QEMU v3.1 is out. Besides a number of small enhancements, some items that we would like to highlight from a KVM on Z perspective:

• Huge Pages Support: KVM guests can now utilize 1MB pages. As this removes one layer of address translation for the guest backing, less page-faults need to be processed, and less translation lookaside buffer (TLB) entries are needed to hold translations. This, as well as the TLB improvements in z14, will improve KVM guest performance.
  To use:
  Create config file /etc/modprobe.d/kvmhpage.conf file with the following content to enable huge pages for KVM:
  
     options kvm hpage=1
  
  Furthermore, add the following line to /etc/sysctl.conf to reserve N huge pages:
  
     vm.nr_hugepages = N
  
  Alternatively, append the following statement to the kernel parameter line in case support is compiled into the kernel: kvm.hpage=1 hugepages=N.
  Note that means to add hugepages dynamically after boot exist, but with effects like memory fragmentation, it is preferable to define huge pages as early as possible.
  If successful, the file /proc/sys/vm/nr_hugepages should show N huge pages. See here for further documentation.
  Then, to enable huge pages for a guest, add the following element to the respective domain XML:
  
     <memoryBacking>
       <hugepages/>
     </memoryBacking>
  
  The use of huge pages in the host is orthogonal to the use of huge pages in the guest. Both will improve the performance independently by reducing the number of page faults and the number of page table walks after a TLB miss.
  The biggest performance improvement can be achieved by using huge pages in both, host and guest, e.g. with libhugetlbfs, as this will also make use of the larger 1M TLB entries in the hardware.
  Requires Linux kernel 4.19.
• vfio-ap: The Adjunct Processor (AP) facility is an IBM Z cryptographic facility comprised of three AP instructions and up to 256 cryptographic adapter cards. Each adapter card is partitioned into up to 85 domains, each of which provides cryptographic services. An AP queue is the means by which AP messages are sent to and received from an AP adapter. Each AP queue is connected to a particular domain within a particular adapter. vfio-ap enables assignment of a subset of AP adapters and domains to one or more guests such that each guest has exclusive access to a discrete set of AP queues.
  Here is a small sample script illustrating host setup:
  
     # load vfio-ap device driver
     modprobe vfio-ap
  
     # reserve domain 7 for use by KVM guests
     echo -0x7 > /sys/bus/ap/aqmask
     # to reserve all domains of an adapter, use the following
     # line instead (by uncommenting it), and replace NN with the
     # adapter number:
     # echo -0xNN > /sys/bus/ap/apmask
  
     # create a mediated device (mdev) to provide userspace access
     # to a device in a secure manner   UUID=e926839d-a0b4-4f9c-95d0-c9b34190c4ba
     echo $UUID > /sys/devices/vfio_ap/matrix/mdev_supported_types/ \
                  vfio_ap-passthrough/create
  
     # assign adapter, domain and control domain
     echo 0x3 > /sys/devices/vfio_ap/matrix/${UUID}/assign_adapter
     echo 0x7 > /sys/devices/vfio_ap/matrix/${UUID}/assign_domain
     echo 0x7 > /sys/devices/vfio_ap/matrix/${UUID}/ \
                assign_control_domain
  
  To make use of the AP device in a KVM guest, add the following element to the respective domain XML:
  
     <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-ap'>
       <source>
         <address uuid='e926839d-a0b4-4f9c-95d0-c9b34190c4ba'/>
       </source>
     </hostdev>
  
  Once complete, use the passthrough device in a KVM guest just like a regular crypto adapter.
  Requires Linux kernel 4.20 and libvirt 4.9, and is also available in RHEL 8, Ubuntu 18.04 and SLES 15 SP1.

SLES 12 SP4 released

SLES 12 SP4 is out! See the announcement and their release note with Z-specific changes.
It ships the following code levels:

• Linux kernel 4.12 (SP3: 4.4),
• QEMU v2.11 (SP3: v2.9), and
• libvirt v4.0 (SP3: v3.3).

See previous blog entries on QEMU v2.10 and v2.11 for details on new features that become available by the QEMU package update.
See previous blog entries on Linux kernel 4.8 and 4.11 for details on new features becoming available through the kernel update, e.g. nested virtualization support.
An additional feature in this release is the availability of STHYI information in LPAR environments. Requires qclib v1.3 or later. See this blog post for general information on qclib.

Furthermore, note that these changes provide a full CPU model, which provides protection against live guest migration compatibility troubles. E.g. migrating a guest exploiting the latest features to a KVM instance running on an earlier IBM Z machine lacking said feature would be detected an prevented.
Note: With this feature, live guest migration back to a KVM instance that does not yet support CPU models (e.g. SLES 12 SP3) will not work anymore.

Ubuntu 18.10 released

Ubuntu Server 18.10 is out! Support for IBM Z is available here.
It ships

• Linux kernel 4.18,
• QEMU v2.12, and
• libvirt v4.6.

Knowledge Series: Black Box Guest Analysis Using kvm_stat

Another new entry in our Knowledge Series details how to gain insights into black box KVM guests using kvm_stat.

Knowledge Series: How to use vnc for Guest Installs

A new entry in our Knowledge Series details how to utilize vnc for graphical installs, exemplified using RHEL 7.5.

2018 Linux on IBM Z and LinuxONE Workshop, Poughkeepsie, NY

Meet us at this event, taking place November 5-6, 2018, at IBM Poughkeepsie, NY. See the full announcement here.
Naturally, KVM on IBM Z will be covered by both, presentations and workgroup sessions.

Find the agenda here.

Registration is open here till October 25.

libvirt v4.7.0 released

libvirt v4.7, available for download at the libvirt project website, adds support for vsock for CCW.
For a full usage example and related information, see this article in our Knowledge series.

QEMU v3.0 released

QEMU v3.0 is out. Besides a number of small enhancements, some items that we would like to highlight from a KVM on Z perspective:

• A new CPU model representing IBM z14 Model ZR1 was added:
  14ZR1 (long name: IBM z14 Model ZR1 GA1).
• Re-use your existing infrastructure for LPAR installs by utilizing the newly added support for .INS files in network boot.

Knowledge: Use virt-install for KVM Guest Installations

Tired of fiddling with guest XML files and virtual DJing with ISO images? Then our latest entry in our Knowledge Series might come handy! Plus it also offers access to further Linux distributions on IBM Z!
Read more on how virt-install can help here.

Getting Started: Alpine Linux Instructions added

Instructions for Alpine Linux were added to the Getting Started with KVM on Z series.
See here for the actual page.

KVM at Share St.Louis 2018

Yes, we will be at SHARE in St.Louis this August!
See our sessions in the Linux and VM track as follows:

• KVM on IBM Z (Session #23164): Latest news on our development work with the open source community
• KVM on Z Performance Analysis with Open Source Tooling (Session #23167): We will take a look at several tools and techniques to investigate KVM performance on IBM Z.
  

Knowledge Series: Managing KVM on IBM Z with oVirt

See here for a new entry in our "knowledge" series, providing step-by-step instructions on how to include IBM Z hosts in an oVirt data center.

Getting Started: RHEL 7.5 Instructions added

Instructions for RHEL7.5 were added to the Getting Started with KVM on Z series.
See here for the actual page.

Ubuntu 18.04 released

Ubuntu Server 18.04 LTS is out! Support for IBM Z is available here.
It ships

• Linux kernel 4.15,
• QEMU v2.11, and
• libvirt v4.0.

As obvious from these package versions, support for IBM z14 is readily in place.

Since this is a so-called LTS (Long Term Support) release providing approx. 5 years of support (in contrast to the usual 9 months of non-LTS releases), it is of particular interest to Ubuntu users interested in a stable environment for production deployments.

QEMU v2.12 released

QEMU v2.12 is out. Here are the highlights from a KVM on Z perspective:

• Added support for an interactive bootloader. As always, we strongly recommend to use the existing support in libvirt.
  To enable/disable, add the following element to your guest definition:
  
     <os>
       <bootmenu enable=’yes|no’ timeout=’n’/>
       ...
     </os>
  
  The timeout parameter specifies a timeout in milliseconds after which the default entry is chosen.
  Alternatively, set attribute loadparm to PROMPT to enable the boot menu without timeout in the respective disk's element:
  
     <disk ...>
       <boot order=’1’ loadparm=’PROMPT’/>
       ...
     </disk>
  
  Example:
  To enable the boot menu for 32 seconds for a guest using a libvirt
  domain XML format follows:
  
     <domain type=’kvm’>
       <os>
         <bootmenu enable=’yes’ timeout=’32000’/>
         ...
       </os>
• Exposure of guest crash information: When a guest is started using libvirt and crashes due to disabled wait, wrong interrupts or a program check loop, libvirt will print the information to the guest’s log, typically located at /var/log/libvirt/qemu.
  E.g. a crash due to a disabled wait results in an entry as follows:
  
     s390: psw-mask=’0xXXXXXXXXXXXXXXXX’, psw-addr=’0xXXXXXXXXXX
           XXXXXX’,crash reason: disabled wait
  
  Requires  libvirt v4.2.
• Added support for guests with more than 8TB of memory.

White Paper: Exploiting HiperSockets in a KVM Environment Using IP Routing with Linux on Z

Our performance group has published a new white paper titled "Exploiting HiperSockets in a KVM Environment Using IP Routing with Linux on Z".
Abstract:

"The IBM Z platforms provide the HiperSockets technology feature for high-speed communications. This paper documents how to set up and configure KVM virtual machines to use HiperSockets with IP routing capabilities of the TCP/IP stack.
It provides a Network Performance comparison between various network configurations and illustrates how HiperSockets can achieve greater performance for many workload types, across a wide range of data-flow patterns, compared with using an OSA 10GbE card."

This white paper is available as .pdf and .html.

RHEL 7.5 with support for KVM on Z available

Red Hat Enterprise Linux 7.5 is out. From the release notes, available here:

Availability across multiple architectures
To further support customer choice in computing architecture, Red Hat Enterprise Linux 7.5 is simultaneously available across all supported architectures, including x86, IBM Power, IBM z Systems, and 64-bit Arm.

Support for IBM Z is available through the kernel-alt package, as indicated earlier here, which provides Linux kernel 4.14. QEMU ships v2.10 via package qemu-kvm-ma, and libvirt is updated to v3.9.0 for all platforms.
Thereby, all IBM z14 features as previously listed here are available.
Check these instructions on how to get started. 

SLES 12 SP3 Updates

SLES 12 SP3, released late last year, received a couple of mostly performance and security-related updates in support of IBM z14 and LinuxONE through the maintenance web updates.
In particular:

• Instruction Execution Protection
• SIMD Extensions
• STHYI Support
• Transparent activation of new hardware features: This will enable a set of performance-oriented hardware facilities of z14 that do not require hypervisor changes.
  This includes the following facilities:
• miscellaneous-instruction-extensions facility 2
• entropy-encoding compression facility
• order-preserving-compression facility

RHEL 7.5 Beta supports KVM on Z

The Red Hat Enterprise Linux 7.5 Beta ships with support for KVM on Z through the kernel-alt packages. This will essentially ship Linux kernel 4.14.
Here is the respective section from the release notes:

KVM virtualization is now supported on IBM z Systems. However, this feature is only available in the newly introduced user space based on kernel version 4.14, provided by the kernel-alt packages.

See here for further details.