Re: [PATCH v2 0/5] hw/arm/virt: Introduce kvm-steal-time

From: Andrew Jones <drjones@redhat.com>
To: qemu-devel@nongnu.org, qemu-arm@nongnu.org
Cc: peter.maydell@linaro.org
Subject: Re: [PATCH v2 0/5] hw/arm/virt: Introduce kvm-steal-time
Date: Mon, 7 Sep 2020 09:04:52 +0200	[thread overview]
Message-ID: <20200907070452.g4y52iqjgvop6rt2@kamzik.brq.redhat.com> (raw)
In-Reply-To: <20200805091640.11134-1-drjones@redhat.com>

FYI, the new KVM cap is on its way into the kernel and Peter has submit
patches to remove kvm32 from qemu. So I'll be refreshing this series to
incorporate both those changes.

Thanks,
drew

On Wed, Aug 05, 2020 at 11:16:35AM +0200, Andrew Jones wrote:
> v2:
>   - Changed the introduction of the feature to 5.2 from 5.1
>     (The 5.2 machine type patch posted by Cornelia was thrown
>      in for completeness, but I suppose that'll get picked up
>      separately.)
>   - Added a patch adding g_assert_not_reached() to many KVM
>     stubs. (This isn't exactly related to the series, but the
>     series does add two more stubs that can now be added in
>     the same way.)
>   - Fixed a patch that wasn't suppose to have a functional
>     change, but did (fdt_add_gic_node() shouldn't generate
>     the node without the PMU) [Peter]
>   - Pass sysmem to virt_cpu_post_init() [Peter]
>   - Introduced a define for the pvtime struct size
>   - Calculate the pvtime memory region size based on max-cpus
>     and host-page-size [Beata]
>   - Renamed kvm_no_steal_time to no_kvm_steal_time [Peter]
>   - Fixed a parameter misordering with object_property_set_bool()
>   - Added a comment explaining why the feature isn't supported
>     for AArch32 guests
>   - Changed a !kvm_steal_time to a kvm_steal_time == ON_OFF_AUTO_OFF
>     as it should be
>   - Picked up one r-b from Peter
> 
> 
> KVM supports the ability to publish the amount of time that VCPUs
> were runnable, but not running due to other host threads running
> instead, to the guest. The guest scheduler may use that information
> when making decisions and the guest may expose it to its userspace
> (Linux publishes this information in /proc/stat). This feature is
> called "steal time" as it represents the amount of time stolen from
> a guest by scheduling out its VCPUs. To enable this feature KVM
> userspace must provide a memory region that will be used to publish
> the information to the guest. The memory region is typical migratable
> region. The GPA of the region is given to KVM through a VCPU device
> ioctl interface. This feature is only available for 64-bit hosts
> running 64-bit guests.
> 
> This series provides the QEMU support of this feature. It will
> be enabled by default for 5.2 machine types and later, but may
> be disabled with a new CPU property "kvm-steal-time".
> 
> While testing migration it was observed that the amount of
> steal time as viewed by the guest was getting reset on each
> migration. Patch 4/6 of a pvtime fix series posted[*] for KVM
> should fix that. Also, we may still want to change the way we
> probe KVM for the feature in this QEMU series to the new KVM
> cap proposed in that KVM series.
> 
> Migration testing:
> 
> * virt-5.1 can migrate as usual, no steal-time enabled
> 
> * virt-5.2 can migrate between hosts with steal-time enabled
>   (the default) and disabled when both hosts support steal-time
> 
> * virt-5.2 with steal-time disabled can migrate to a host that
>   does not support steal-time
> 
> * virt-5.2 with steal-time enabled will cleanly fail when migrating
>   to a host that does not support steal-time
> 
> * virt-5.2 without the kvm-steal-time property specified can
>   boot on a host that does not support the feature - the feature
>   will be disabled. However, if the guest is migrated to a host
>   that does support the feature, then after the guest has reboot
>   the feature will become available with the guest kernel supports
>   it (this is the nature of migrating guests that use '-cpu host').
>   Additionally, once this guest has been migrated to a host that
>   does have the feature, then it will fail (cleanly) to migrate to
>   a host that does not have the feature. If this behavior isn't
>   desired, then the user should explicitly disable steal-time with
>   the kvm-steal-time property if they want to boot a 5.2 guest on
>   a host that doesn't support steal-time.
> 
> [*] https://lists.cs.columbia.edu/pipermail/kvmarm/2020-August/041823.html
> 
> Thanks,
> drew
> 
> 
> Andrew Jones (4):
>   target/arm/kvm: Make uncalled stubs explicitly unreachable
>   hw/arm/virt: Move post cpu realize check into its own function
>   hw/arm/virt: Move kvm pmu setup to virt_cpu_post_init
>   hw/arm/virt: Implement kvm-steal-time
> 
> Cornelia Huck (1):
>   hw: add compat machines for 5.2
> 
>  docs/system/arm/cpu-features.rst |  11 +++
>  hw/arm/virt.c                    | 119 +++++++++++++++++++++++--------
>  hw/core/machine.c                |   3 +
>  hw/i386/pc.c                     |   6 +-
>  hw/i386/pc_piix.c                |  14 +++-
>  hw/i386/pc_q35.c                 |  13 +++-
>  hw/ppc/spapr.c                   |  15 +++-
>  hw/s390x/s390-virtio-ccw.c       |  14 +++-
>  include/hw/arm/virt.h            |   5 ++
>  include/hw/boards.h              |   3 +
>  include/hw/i386/pc.h             |   3 +
>  target/arm/cpu.c                 |  10 +++
>  target/arm/cpu.h                 |   4 ++
>  target/arm/kvm.c                 |  20 ++++++
>  target/arm/kvm32.c               |   5 ++
>  target/arm/kvm64.c               |  77 ++++++++++++++++++--
>  target/arm/kvm_arm.h             |  74 ++++++++++++++-----
>  target/arm/monitor.c             |   2 +-
>  tests/qtest/arm-cpu-features.c   |  25 +++++--
>  19 files changed, 358 insertions(+), 65 deletions(-)
> 
> -- 
> 2.25.4
> 