Monday, December 18, 2017

QEMU v2.11 released

QEMU v2.11 is out. Here are the highlights from a KVM on Z perspective:
  • TOD-Clock Epoch Extension Support: Extends the TOD clock beyond the year 2042.
  • Setting sysctl vm.allocate_pgste is now superfluous.
  • Netboot: The network boot firmware sets the client architecture option (93) in the DHCP request to 0x1f ("s390 Basic"). This allows a DHCP server to deliver the correct boot image for IBM Z guests. This is useful in situations where a single DHCP server has to provide network boot images for multiple architectures, e.g. for the purpose of installing operating systems.
  • Added support for virtio-input-ccw and virtio-gpu-ccw. These newly supported devices lay the foundation for applications that require graphical interfaces, which thereby become usable from remote via VNC or SPICE.
    Here is a sample XML snippet for a guest definition:

        <input type='keyboard' bus='virtio'/>
        <graphics type='vnc' port='-1' autoport='yes' listen=''>
          <listen type='address' address=''/>
          <model type='virtio' heads='1' primary='yes'/>

Wednesday, December 13, 2017

Performance: Network Storage Protocols in a KVM Environment

Our performance group has published a new white paper titled "Network Storage Protocols in a KVM Environment - NFS/SMB/iSCSI Report" (.pdf and .html).

"iSCSI, NFS, and SMB protocols provide access to storage resources via a TCP/IP network. While iSCSI provides block device access, NFS and SMB are file-level access protocols. In cases where the devices are mounted on the KVM host (as in our case) they could be functionally considered as equivalent when utilized for KVM guest device attachment with image files."

Tuesday, December 12, 2017

Friday, December 1, 2017

New "Getting Started" Series

A new series of articles will provide step-by-step instructions on how to setup and run KVM on Z. It will be updated continuously for new Linux distributions.
See here for the entry page.

Monday, September 11, 2017

DataCenter Insider Article

See here for an article (in German) in DataCenter Insider about KVM on z's history, usage concepts, support and z14 features.

Tuesday, September 5, 2017

QEMU v2.10 released

QEMU v2.10 is out. See here for a detailed list of all changes related to IBM Z. Here are some of the highlights from a KVM on z perspective:
  • z14 CPU model
    Provides base support for all of the features introduced with the z14's new processor generation.
    See this previous blog post for details.
  • Emulated 3270 Device
    This implementation has been tested with x3270 only. Your mileage with other clients might vary. For practical matters, the sclp console remains to be the recommended terminal type for KVM guests on Z.
    As this feature is still marked as experimental in QEMU, there is currently no libvirt support available. Therefore to configure a 3270 device on a guest (accessible on all IP addresses of the host at port 23), we must use libvirt’s ability to pass-through arbitrary QEMU command line arguments via domain XML (see here for furher details):
    <domain type='kvm' xmlns:qemu=''>
        <qemu:arg value="-chardev"/>
        <qemu:arg value="socket,id=charconsole1,host=,port=23,\
        <qemu:arg value="-device"/>
        <qemu:arg value="x-terminal3270,chardev=charconsole1,\

    See here for further hints on how to use.
  • Bootparameter Support
    Until now, when a guest had multiple kernels to boot with, one had to change the default kernel in /etc/zipl.conf and re-run zipl to boot with a different kernel next time.
    With this feature, we now support option -L in the chreipl command from the s390-tools package, which allows to switch to a different boot option dynamically, e.g. using
  • $ chreipl ccw -d 0.0.1234 -L 2
    However, if somebody has set a new default kernel that does not boot, this will not help, since the guest will not start anymore in the first place.
    However, with the newly introduced attribute loadparm, one can switch to a different (working) kernel within the host in a guest's XML:
    <boot order='1' loadparm='2'>
    • chreipl changes will no longer be effective after the respective guest has been powered off.
    • The loadparm attribute became available with libvirt v3.5 or higher. 

Thursday, July 27, 2017

KVM on z14 features

While the latest addition to the IBM Z family has been announced, here is a list of features in support of specific features of the new hardware generation in past releases of the Linux kernel, QEMU and libvirt, all activated by default in the z14 CPU model:
  • Instruction Execution Protection
    This feature provides KVM hypervisor support for the Instruction Execution Protection (IEP) facility in the z14. The IEP prevents code execution from memory regions marked as non-executable, improving the security model.
    Other than activating/deactivating this feature in the applicable the CPU models in QEMU (which holds true for most hardware-related features on IBM Z in general), there are no switches associated with this feature.
    Requires Linux kernel 4.11 in the KVM host and guests, as well as QEMU v2.10 (host only).
    In the z14 CPU model, the respective feature is:
      iep       Instruction-execution-protection facility
  • SIMD Extensions
    Following up to the SIMD instructions as introduced with the previous z13 model, the new z14 provides further vector instructions, which can again be used in KVM guests.
    These new vector instructions can be used to improve decimal calculations as well as for implementing high performance variants of certain cryptographic operations.
    Requires Linux kernel 4.11 as well as QEMU v2.10 in the KVM host, and binaries or a respective Java Runtime Environment in guests using the new vector instructions.
    In the z14 CPU model, the respective feature is:
      vxpd      Vector packed decimal facility
          Vector enhancements facility
  • Keyless Guest Support
    This feature supports the so-called Keyless Subset (KSS) facility, a new feature of the z14 hardware. With the KSS facility enabled, a host is not required to perform the (costly) storage key initialization and management for KVM guests, unless a guest issues a storage key instruction.
    Requires Linux kernel 4.12 in the KVM host. As for the guests, note that starting with SLES12SP1, RHEL7.2 and Ubuntu 16.04, Linux on IBM Z does not issue any storage key operations anymore.
    This feature does not have a separate entry in the z14 CPU model.
  • CPUMF Basic Sample Configuration Level Indication
    Basic mode samples as defined in "The Load-Program-Parameter and the CPU-Measurement Facilities" (SA23-2260) do not provide an indication whether the sample was taken in a KVM host or guest. Beginning with z14, the hardware provides an indication of the configuration level (level of SIE, e.g. LPAR or KVM). This item exploits this information to make the perf guest/host decision reliable.
    Requires Linux kernel 4.12 in the KVM host.
    There is no separate entry in the z14 CPU model, since this feature applies to the host only.
  • Semaphore assist
    Improves performance of semaphore locks.
    Requires Linux kernel 4.7 and QEMU v2.10 in the KVM host. Exploitation in Linux kernels in guests is still in progress here, scheduled for 4.14.
    In the z14 CPU model, the respective feature is:
      sema      Semaphore-assist facility
  • Guarded storage
    This feature is specifically aimed at Java Virtual Machines running in KVM guests to run with fewer and shorter pauses for garbage collection.
    Requires Linux kernel 4.12 and QEMU 2.10 in the KVM host, and a Java Runtime Environment with respective support in the guests.
    In the z14 CPU model, the respective feature is:
      gs        Guarded-storage facility
  • MSA Updates
    z14 introduces 3 new Message Security Assists (MSA) for the following functionalities:
        MSA6: SHA3 hashing
        MSA7: A True Random  Number Generator (TRNG)
                    which provides support for the Galois-counter-mode (GCM)
    MSA6 and MSA 7 require Linux kernel 4.7, while MSA8 requires Linux kernel 4.12. All require QEMU v2.10 in the KVM host. These features can be exploited in KVM guests' kernels and userspace applications independently (i.e. a KVM guest's userspace applications can take advantage of these features irrespective of the guest's kernel version).
    In the z14 CPU model, the respective features are:
      msa6      Message-security-assist-extension 6 facility

      msa7      Message-security-assist-extension 7 facility
      msa8      Message-security-assist-extension 8 facility
  • Compression enhancements
    New instructions improve compression capabilities and performance.
    Requires Linux kernel 4.7 in the KVM host.
    In the z14 CPU model, the respective features are:
      opc       Order Preserving Compression facility
      eec       Entropy encoding compression facility
  • Miscellaneous instructions
    Details on these instructions are to be published in the forthcoming z14 Principles of Operation (PoP).
    Requires Linux kernel 4.7 and QEMU 2.10 in the KVM host, and binaries that were compiled for the z14 instruction set using binutils v2.28 and gcc v7.1 in the guests.
    In the z14 CPU model, the respective feature is:
      minste2   Miscellaneous-instruction-extensions facility 2
Note: All versions specified are minimum versions.

Further features will be announced in future blog posts as usual as they find their way into the respective Open Source projects.
Also, don't forget to check this blog entry with further details on z14 in general and Linux on z in particular.

Monday, July 17, 2017

z14 announced

Today, IBM announced the new IBM Z model named z14. See here for the full press release.

We will look at features in support of the new IBM Z model in a separate blog entry soon.

Thursday, July 6, 2017

Migrating from KVM for IBM z to Ubuntu

In case you are contemplating a migration from KVM for IBM z to Ubuntu 16.04 or later, you might find the instructions published on the Ubuntu Wiki here helpful.

Furthermore, Canonical published a nice article here that provides an overview of the many options how to deploy Ubuntu on IBM z Systems, including KVM.

Wednesday, July 5, 2017

libvirt v.3.5.0 released

libvirt v3.5.0 is now available for download at the libvirt project website.
A new z Systems-specific feature is the introduction of parameter loadparm for boot devices.
The loadparm parameter can be used to select a specific boot configuration from the zipl menu when IPL’ing/booting from a boot device, similar to what the chreipl command does in Linux on z.
Being able to select a configuration from multiple boot configurations provides more flexibility and allows to recover from error situations by booting a "last known good configuration".
To use, specify the loadparm attribute for a boot device in a guest's XML as follows:

      <boot order='1' loadparm='3'/>

Note that this feature requires the forthcoming QEMU 2.10 release.

Tuesday, May 30, 2017

Migrating from KVM for IBM z to SLES12

In case you are contemplating a migration from KVM for IBM z to SLES12 SP2 or later, you might find the instructions published by SuSE here helpful.

Wednesday, May 10, 2017

QEMU v2.9 released

QEMU v2.9 is out. Here are the highlights from a KVM on z perspective:
  • Added support for virtio-crypto, providing a virtual crypto device as a front-end to the host's crypto capabilities on z Systems.
  • Added multiqueue support for virtio-ccw devices, improving throughput in presence of multiple processors.
  • Provided network boot enablement. See here for further details.

Tuesday, May 2, 2017

Linux kernel 4.11 released

Linux kernel 4.11 (available here) has been released, improving Kernel Samepage Merging (KSM) support for KVM on z:

KSM is an existing feature of the Linux kernel that allows the kernel to merge pages with identical content, indicated as MADV_MERGEABLE via madvise(), across different processes to save memory.
This z Systems specific extension improves the performance of userspace applications (e.g. QEMU) in case large amounts of empty/zeroed pages are merged.
This feature is disabled by default, enable as follows:

   $ echo 1 > /sys/kernel/mm/ksm/use_zero_pages

Thursday, April 27, 2017

Saturday, April 8, 2017

Webinar: OpenStack for KVM with Ubuntu on IBM z Systems

Abstract: This session is divided into three parts, each with an equal time share of 15-20 minutes. We will start with a recap what Infrastructure as a Service (IaaS) is all about to resolve some minor potential misconceptions. After that, an overview of the most important capabilities of OpenStack are shown. The most common (at least in the past) pain points and mitigations are discussed too. Eventually, after having enough context, we take a look at Ubuntu and OpenStack on z. This includes deployment, operations and supported features. This session will bring you benefit if you're new(-ish) to OpenStack, evaluating IaaS solutions, and care about the user/operators-view.

Speaker : Markus Zoeller, OpenStack Development, IBM Germany

Audience: IT Manager, IT Architects, System Administrators

Date: Wednesday, April 26th, 2017
Time: 11:00 AM EST / New York, 4:00 PM CET / Germany
Duration: 75 minutes

Registration: You can register here. After registering, you will receive a
confirmation email containing information about joining the webinar.

Replay & Archive: All sessions are recorded. For the archive as well as a replay and handout of this session (available by Apr 30, 2017) and all previous webcasts please visit

Contact: Please direct any questions to Stephanie Gherghe at

Tuesday, March 14, 2017

KVM for IBM z Withdrawal

IBM announced that KVM for IBM z will be withdrawn, effective March 31, 2018. From the notice:
"KVM for IBM z Systems Version 1.1.2 will be the last release delivered by IBM. However, IBM will continue to provide the Linux and KVM enablement for z Systems [...]"
Make no mistake: The withdrawal only affects the product KVM for IBM z. It does not mean the end for KVM on z (our open source effort to support KVM on IBM z Systems). To the contrary, development will not only continue unaffected, but the options for users grow, especially with the recent addition of SuSE to the existing support in Ubuntu.
See the FAQ for further details.

SLES now supports KVM on z

SuSE has announced formal support for KVM on z starting with SLES 12 SP2. See the official announcement here for full details.

Monday, February 13, 2017

Knowledge Series: Choosing the Right Configuration

Large server setups are quite typical for KVM on z. One common pitfall is that system and application defaults that work well for a small number of servers may not work very well for a large number of virtual servers or in combination with huge host resources. The following sections present a list of snags that could happen, along with respective suggestions on how to resolve them.

Tuesday, January 24, 2017

QEMU v2.8 & libvirt v3.0 released

QEMU v2.8 and libvirt v3.0 (available for download at the libvirt project website) are out! The highlight from a KVM on z perspective is the introduction of CPU models.
The CPU models are of primary use in live migration scenarios. For example, in a setup with different z Systems machine generations, it is now possible to check up-front whether the target host will support all facilities required by a guest for a successful migration. In case any of the required facilities are missing, the migration is aborted, and the guest will continue to run on the current host.
Furthermore, a guest can be defined with a backlevel CPU model compatible with the target machine (of a previous z Systems generation), so that migrations become possible.

To see what CPU models QEMU (and hence the host) supports, use virsh domcapabilities:

  $ virsh domcapabilities

      <mode name='custom' supported='yes'>
        <model usable='unknown'>z10EC-base</model>
        <model usable='unknown'>z9EC-base</model>
        <model usable='unknown'>z196.2-base</model>
        <model usable='unknown'>z900-base</model>
        <model usable='unknown'>z990</model>
        <model usable='unknown'>z900.2-base</model>
        <model usable='unknown'>host</model>
        <model usable='unknown'>z900.3</model>
        <model usable='unknown'>z114</model>
        <model usable='unknown'>z890-base</model>
        <model usable='unknown'>z13.2-base</model>
        <model usable='unknown'>zEC12.2</model>