Ubuntu 21.10 released

Ubuntu Server 21.10 is out!
It ships

• Linux kernel 5.13 (including, among others, features as described here and here)
  
• QEMU v6.0
• libvirt v7.6

See the release notes here, and the blog entry at Canonical with Z-specific highlights here.

Documentation Update: KVM Virtual Server Management

Intended for KVM virtual server administrators, the "Linux on Z and LinuxONE - KVM Virtual Server Management" book illustrates how to set up, configure, and operate Linux on KVM instances and their virtual devices running on the KVM host and IBM Z hardware.

This major update includes libvirt commands and XML elements for managing the lifecycle of VFIO mediated devices, performance tuning tips, and a simplified method for configuring a virtual server for IBM Secure Execution for Linux.

qeth Devices: Promiscuous Modes, Live Guest Migration, and more

qeth devices, namely OSA-Express and HiperSockets, have a vast array of functionalities that is easy to get lost in. This entry illustrates some of the most commonly sought functionalities, while trying to avoid confusing the reader with too much background information.

TLDR:

• IBM z14: For KVM, always use separate OSA devices on source and target for LGM; For OVS, use a primary bridgeport with OSA, and VNIC characteristics with HiperSockets.
  
• IBM z15: For KVM with OVS, use VNIC characteristics for any qeth device; for KVM and MacVTap, use VNIC characteristics if you want to use the same device on source and target system in LGM scenarios.
  

 

Bridgeport Mode

Initially, the only way to enable promiscuous mode on OSA-Express adapters and HiperSockets was through the so-called bridgeport mode. The concept of the bridgeport mode distinguishes between between ports as follows:

• Primary bridgeport: The primary bridgeport receives all traffic for all destination addresses unknown to the device. Or, in other words: If the device receives data for a destination unknown to it, instead of dropping it, it will be forwarded to the current primary bridgeport instead. Which further implies that as soon as an operating system registers a MAC address with the device, traffic destined for that MAC address becomes "invisible" to the bridgeport.
  Note: Only a single operating system can use the primary bridgeport on an adapter at any time.
  
• Secondary bridgeport: Whenever the operating system that currently has the primary bridgeport gives up on it, one of the secondary bridgeports will become the new primary. An arbitrary number of operating systems can register as a secondary bridgeport.
  

Bridgeport mode is available in Layer 2 mode only. Furthermore, HiperSockets devices need to be defined as external-bridged in IOCDS.
Use attributes in the device's sysfs directory as follows:

• bridge_role: Set the desired role (none, primary, or secondary), and query the current one.
• bridge_state: Query the current state (active or inactive)

Bridgeport mode effectively provides a promiscuous mode. But note that in addition to enabling the primary bridgeport mode, the respective interface has to have the promiscuous mode set, still!
All in all, here is how usage of this feature typically looks like:

  $ echo primary >/sys/devices/qeth/0.0.bd00/bridge_role

  # verify that we got primary bridgeport, not secondary, and are active:
  $ cat /sys/devices/qeth/0.0.bd00/bridge_state 
  active
  $ cat /sys/devices/qeth/0.0.bd00/bridge_role
  primary

  # enable promiscuous mode on the interface
  $ ip link set <interface> promisc on

The downside of this approach is that only a single operating system per device can enable the primary bridgeport mode, which scales only that far. Therefore, something better, with more functionality was introduced to the platform.

VNIC Characteristics

Introduced with IBM z14/LinuxONE II for HiperSockets, and IBM z15/LinuxONE III for OSA, the VNIC characteristics feature provides promiscuous mode for multiple operating systems attached to the same device, and provides additional functionality which can be very handy especially with KVM.
The VNIC characteristics can be controlled through a number of attributes located in an extra subdirectory called vnicc in the device's sysfs directory.

Let us focus on two main functionalities.

Promiscuous Mode

Technically, VNIC does not provide a traditional promiscuous mode (just like bridgeport mode did not in the literal sense), but rather emulates a self-learning switch. However, for users looking for a promiscuous mode that is usable in conjunction with a Linux bridge or an Open vSwitch, the end-result is the same.

To activate, set the attributes as follows:

  echo 1>/sys/devices/qeth/0.0.bd00/vnicc/flooding
  echo 1>/sys/devices/qeth/0.0.bd00/vnicc/mcast_flooding
  echo 1>/sys/devices/qeth/0.0.bd00/vnicc/learning

Again, in addition to enabling the promiscuous mode on the device, the respective interface has to have the promiscuous mode set, still:

  ip link set <interface> promisc on

KVM Live Guest Migration

Providing connectivity to virtual servers running in KVM, administrators have two choices to provide connectivity:

• Via Open vSwitch: Requires a promiscuous mode, see above. Virtual servers migrated between the two Open vSwitches will have uninterrupted connectivity thanks to the devices being configure in promiscuous mode, provided that the networking architecture is set up accordingly. The two Open vSwitches may or may not share the same networking device.
  
• Via MAC Address Takeover: This is only required in case both, the source and the target KVM host share the same device and use MacVTap to connect to it. While the traffic will still run through the same device, some handshaking has to take place to make sure that the MAC address is configured correctly, and traffic forwared to the target KVM host once migration has completed. This has to be authorized - otherwise, an attacker could divert traffic elsewhere.

Luckily, VNIC characteristics offers functionality for MAC address takeover, too. To enable, set the VNIC characteristics as follows:

On the source KVM host:

  echo 1>/sys/devices/qeth/0.0.bd00/vnicc/takeover_learning

On the target KVM host:

  echo 1>/sys/devices/qeth/0.0.bd00/vnicc/takeover_setvmac

Final Words

Note that bridgeport mode and VNIC characteristics are mutually exclusive! Meaning as soon as e.g. a single VNIC characteristics-related attribute is activated, bridgeport-related functionality is not available anymore until that VNIC-characteristic is disabled again.

Furthermore, check your Linux distribution's tools on how to persist the changes outlined above. On many distros, chzdev (comes with the s390-tools package) does the job, but not (yet) on all.

This article only provides a brief overview. Both, promiscuous mode and the VNIC characteristics have a lot more to it than what was covered in this brief overview, which merely aims to provide just enough information to get readers started with the most common usecases. For a deeper understanding, check the respective sections in the Device Drivers, Features, and Commands book.

Webinar: 2021 Linux on IBM Z and LinuxONE Technical Client Workshop

Join us for the 2021 Linux on IBM Z and LinuxONE Virtual Client Workshop!

Abstract

Get the latest news about the Linux exploitation and advantages of the IBM Z and LinuxONE platform in this technical workshop. Presented by our developers and solution architects, the training focuses on the latest news and technical information for Linux on IBM Z, LinuxONE, z/VM, and KVM, such as Red Hat OpenShift Container Platform, Red Hat OpenShift Container Storage, Security, Performance, Networking and Virtualization. You will have the chance to interact directly with IBM developers and solution experts during the event, especially in the interactive workgroup sessions, which will be held on the last day.

This workshop is free of charge.

Agenda Highlights

• What's New on RHOCP on IBM Z & LinuxONE 
• Hybrid Cloud and why RHOCP on IBM Z & LinuxONE can enable highest flexibility
• Introduction of Red Hat OpenShift Container Storage
• Red Hat OpenShift Container Platform on IBM Z & LinuxONE: CPU Consumption Demystified
• Cloud Ready Development, can now profit from multi Architecture capabilities and several features in RHOCP on IBM Z
• FUJITSU Enterprise Postgres: Finally! An OCP-certified Database for Linux on IBM Z and LinuxONE that exploits our hardware capabilities
• Reduce your IT costs with IBM LinuxONEHow IBM Cloud Paks drive business value and lower IT costs
• z/VM Platform Update
• Linux and KVM on IBM Z and LinuxONE - What's New
• kdump - Recommendations for Linux on IBM Z and LinuxONE
• Elasticsearch on IBM Z - Performance Experiences, Hints and Tips
• Crypto Update
• Fully homomorphic encryption Introduction and Update
• Putting SMC-Dv2 to work
• Java on IBM Z - News, Updates, and other Pulp Fiction
• Various workgroup sessions
  
  

Schedules & Registration

Americas, Europe, Middle East & Africa
July 12-16, every day 8:30 - 11:30 AM EST / 14:30 - 17:30 CET
Register here.

Asia Pacific
July 27-29, 2021, every day 8:30 - 11:30 AM CET / 2:30 - 5:30 PM Singapore time
Register here.

SLES 15 SP3 Released

SUSE Linux Enterprise Server 15 SP3 is out! See the official announcement and the release notes. It provides

• QEMU v5.2, supporting virtio-fs on IBM Z
  
• libvirt v7.1
  

For a detailed list of IBM Z and LinuxONE-specific (non-KVM) features see here.

RHEL 8.4 Released

RHEL 8.4 is out! See the official announcement and the release notes.

KVM is supported via Advanced Virtualization, and provides

• QEMU v5.2, supporting virtio-fs on IBM Z
  
• libvirt v7.0
  

Furthermore, RHEL 8.4 now supports graphical installation for guest installs. Just add the highlighted arguments to your virt-install command line for an RHEL 8.4 install on an RHEL 8.4 KVM host:

    virt-install --input keyboard,bus=virtio --input mouse,bus=virtio \
    --graphics vnc --video virtio --disk size=8 --memory 2048 --name rhel84 \
    --cdrom /var/lib/libvirt/images/RHEL-8.4.0-20210503.1-s390x-dvd1.iso

And the installation will enter the fancy graphical installer:

Make sure to have package virt-viewer installed on the host, and X forwarding enabled (option -X for ssh).

This new support also allows graphical installs started in cockpit:

Webinar: Red Hat OpenShift for IBM Z and LinuxONE on RHEL 8.3 KVM

Join us for our webinar on Wednesday, April 21, 11:00 AM - 12:00 PM EST!

Abstract

Red Hat OpenShift is available on RHEL 8.3 KVM starting with Red Hat OpenShift version 4.7 on IBM Z and LinuxONE. We discuss the deployment of a Red Hat OpenShift Cluster on RHEL KVM from a high-level perspective, including supported configurations and requirements, especially the available network and storage options.
Furthermore, we explain the installation steps of Red Hat OpenShift 4.7 on RHEL KVM in detail, including best practices and a short excursion on cluster debugging.

Speakers

• Dr. Wolfgang Voesch, Iteration Manager - OpenShift on IBM Z and LinuxONE
• Holger Wolf, Product Owner - OpenShift on Linux on IBM Z and LinuxONE
  
  

Registration

Register here. You can check the system requirements here.
After registering, you will receive a confirmation email containing information about joining the webinar.

Replay & Archive

All sessions are recorded. For the archive as well as a replay and handout of this session and all previous webinars see here.