R-Car/Virtualization

From eLinux.org
< R-Car
Revision as of 15:02, 15 February 2018 by Laurent.pinchart (talk | contribs) (Add instructions for QEMU manual cross-compilation)
Jump to: navigation, search

Virtualization on Renesas R-Car Platforms

Virtualization with Xen hypervisor

Xen supports R-Car chips out of the box. Example instructions on how to run Xen on R-Car H3/H2 can be found @ XenProject Wiki:


Virtualization with QEMU

QEMU is a generic and open source machine emulator and virtualizer. It emulates CPUs and provides a set of emulated devices to support running unmodified guest operating systems. When running on Linux hosts, it can use the host's hardware virtualization extensions (if supported) through the KVM API to run guests at near-native speed.

Quickstart

  • Install QEMU (e.g. "apt-get install qemu-system-arm")
  • Download a guest image (e.g. "openwrt-arm64-qemu-virt.Image" from OpenWRT)
  • Start QEMU with

    $ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -M virt -nographic -kernel openwrt-arm64-qemu-virt.Image

    and press <ENTER> to enjoy your new ARM64 system!

Installing QEMU

When not using a binary distribution that provides a QEMU package, QEMU can be cross-compiled from sources, manually or with the help of a cross-compilation build system such as BuildRoot or OpenEmbedded.

Manual cross-compilation

QEMU supports a wide variety of options, in particular related to emulation of architectures or various classes of devices. The instructions given here only enable a minimal set of options to reduce the required dependencies. We only enable the aarch64-softmmu target architecture, FDT for ARM Linux guest support, KVM for hardware virtualization acceleration and SDL for display acceleration.

The following packages are needed on the host to cross-compile QEMU.

  • pkg-config
  • Python 2.x

The following packages are needed in the host build environment to cross-compile and on the target to run QEMU.

  • DTC (Device Tree Compiler)
  • GLib 2.x
  • Pixman
  • SDL (1.x or 2.x)
  • zlib

Other options can be enabled as needed and may require extra dependencies.

After obtaining the QEMU sources, the build must be setup. The following environment variables must be set.

CROSS_COMPILE
Cross-compilation toolchain prefix (e.g. aarch64-linux-gnu-).
PKG_CONFIG
Path to the pkg-config script for the cross-compilation environment (typically ${HOST_DIR}/bin/pkg-config).
The script must set the PKG_CONFIG_LIBDIR environment variable and execute the pkg-config binary.
SDL_CONFIG
Path to the sdl-config script (typically ${STAGING_DIR}/usr/bin/sdl-config).

Additionally, if not using the host system Python, the following environment variables must be set.

PYTHON
Path to the host build environment Python 2 binary.
PYTHONPATH
Search file for Python module files for the Python interpreter referenced by $PYTHON.

You can then run the configure script.

$ ./configure                                                  \
    --prefix=/usr --cross-prefix=${CROSS_COMPILE}              \
    --target-list=aarch64-softmmu                              \
    --enable-attr       --enable-fdt       --enable-kvm        \
    --enable-sdl        --enable-system    --enable-tools      \
    --audio-drv-list=                                          \
    --disable-bluez     --disable-brlapi   --disable-bsd-user  \
    --disable-cap-ng    --disable-curl     --disable-curses    \
    --disable-docs      --disable-libiscsi --disable-linux-aio \
    --disable-rbd       --disable-seccomp  --disable-slirp     \
    --disable-sparse    --disable-spice    --disable-strip     \
    --disable-usb-redir --disable-vde      --disable-virtfs    \
    --disable-vnc       --disable-werror   --disable-xen

QEMU can then be compiled and installed and installed to the destination directory $DESTDIR.

$ make
$ make DESTDIR=${DESTDIR} install

Running QEMU

Recent kernels may access ARM64 registers that require a recent version of QEMU. If your kernel crashes with

Code: ........ ........ ........ ........ (d5380740)
Internal error: undefined instruction: 0 [#1] SMP
...
pc : __cpuinfo_store_cpu+0x68/0x1b0

you have to upgrade QEMU, or comment out the following two lines in arch/arm64/kernel/cpuinfo.c:__cpuinfo_store_cpu():

info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1);

Using KVM

When using an arm64 platform as the host system, guest performance can be improved drastically by making use of the Kernel-based Virtual Machine (KVM).

Host Kernel Configuration

Make sure your host kernel has the following options enabled:

CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y

"renesas_defconfig" should be fine.

Firmware Support

To use KVM, the firmware on your board must start Linux in hypervisor (EL2 / HYP) mode.

If your host kernel prints:

CPU: All CPU(s) started at EL2
...
kvm [1]: Hyp mode initialized successfully

during boot up, everything is fine, and you can skip the next section.

If your host kernel prints:

CPU: All CPU(s) started at EL1
...
kvm [1]: HYP mode not available

during bootup, HYP mode is not available, and KVM cannot be used, unless you first replace the firmware of your board with a version that supports HYP mode.

Enabling HYP Support

To build your own ARM Trusted Firmware with hypervisor mode support:

$ git clone https://github.com/renesas-rcar/arm-trusted-firmware.git
$ cd arm-trusted-firmware
$ make CROSS_COMPILE=aarch64-linux-gnu- PLAT=rcar RCAR_DRAM_SPLIT=3 RCAR_BL33_EXECUTION_EL=1 LSI=H3 # or LSI=M3 

Replace the BL2 and BL31 binaries in your firmware package by the generated "build/rcar/release/bl2.srec" and "build/rcar/release/bl31.srec", and follow your normal firmware upgrade procedure.

Running QEMU with KVM

$ qemu-system-aarch64 -enable-kvm -m 1024 -cpu cortex-a57 -M virt -nographic -kernel openwrt-arm64-qemu-virt.Image

and press <ENTER> to enjoy a faster new ARM64 system!

Note that the above command requires running on a Cortex-A57 CPU core. When running on a Cortex-A53 CPU core in a big.LITTLE configuration, the command will fail with:

kvm_init_vcpu failed: Invalid argument

Just retry, or force your luck by offlining all Cortex-A53 cores first:

for i in $(grep -lr arm,cortex-a53 /sys/bus/cpu/devices/cpu*/of_node/compatible); do
	echo 0 > $(dirname $(dirname $i))/online
done

Note: The same is true for running with '-cpu cortex-a53', with a53 and a57 exchanged.

Device Pass-Through

See VFIO


Managing virtual machines with libvirt

See Libvirt.


Building your own guest kernel

Kernel Config

The kernel config file used for the OpenWRT guest image is a good starting point. You can extract it from '/proc/config.gz' on the running system.

RAM Disk

You can extract the initramfs from an OpenWRT guest image using 'binwalk'. Make sure to truncate the initramfs file after the first 256-byte boundary after "TRAILER!!!", else the kernel will crash with

Kernel panic - not syncing: broken padding