[Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Zhang, Lei]

Hi Andrew,

I have tested your patch on kernel- 5.2.7 + QEMU (4.0.94 + patch).

This patch series works fine for my tests when use qemu-system-aarch64 directly.
But I can't startup kvm when I use virsh[1].

Command I executed.
# virsh start test1

The error message is [internal error: CPU features not supported by hypervisor for aarch64 architecture.]
Do you have any ideas for this error? 

[1]
https://www.redhat.com/archives/libvir-list/2019-July/msg01524.html

Best Regards,
Lei Zhang

Re: [Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Andrew Jones]

On Tue, Aug 20, 2019 at 06:08:04AM +0000, Zhang, Lei wrote:
> Hi Andrew,
> 
> I have tested your patch on kernel- 5.2.7 + QEMU (4.0.94 + patch).

Thanks for the testing! I guess it's time for me to get back to this
series and spin a v4 (so we can test some more :-)

> 
> This patch series works fine for my tests when use qemu-system-aarch64 directly.
> But I can't startup kvm when I use virsh[1].
> 
> Command I executed.
> # virsh start test1
> 
> The error message is [internal error: CPU features not supported by hypervisor for aarch64 architecture.]
> Do you have any ideas for this error? 

I've CC'ed Andrea.

Thanks,
drew

> 
> [1]
> https://www.redhat.com/archives/libvir-list/2019-July/msg01524.html
> 
> Best Regards,
> Lei Zhang

Re: [Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Andrea Bolognani]

On Tue, 2019-08-20 at 09:53 +0200, Andrew Jones wrote:
> On Tue, Aug 20, 2019 at 06:08:04AM +0000, Zhang, Lei wrote:
> > Hi Andrew,
> > 
> > I have tested your patch on kernel- 5.2.7 + QEMU (4.0.94 + patch).
> 
> Thanks for the testing! I guess it's time for me to get back to this
> series and spin a v4 (so we can test some more :-)
> 
> > This patch series works fine for my tests when use qemu-system-aarch64 directly.
> > But I can't startup kvm when I use virsh[1].
> > 
> > Command I executed.
> > # virsh start test1
> > 
> > The error message is [internal error: CPU features not supported by hypervisor for aarch64 architecture.]
> > Do you have any ideas for this error? 
> 
> I've CC'ed Andrea.

I've specifically patched out that check in my series... Are you
sure you're using the modified libvirt version, and that your guest
is configured to use the modified QEMU binary?

Anyway, once v4 has been posted I'll respin the libvirt series as
well, since in the meantime conflicts have popped up and it no longer
applies cleanly on top of master.

-- 
Andrea Bolognani / Red Hat / Virtualization

[Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Andrew Jones]

Since Linux kernel v5.2-rc1 KVM has support for enabling SVE in guests.
This series provides the QEMU bits for that enablement. First, we
select existing CPU properties representing features we want to
advertise in addition to the SVE vector lengths and prepare
them for a qmp query. Then we introduce the qmp query, applying
it immediately to those selected features. We also document ARM CPU
features at this time. We next add a qtest for the selected CPU
features that uses the qmp query for its tests - and we continue to
add tests as we add CPU features with the following patches. So then,
once we have the support we need for CPU feature querying and testing,
we add our first SVE CPU feature property, 'sve', which just allows
SVE to be completely enabled/disabled. Following that feature property,
we add all 16 vector length properties along with the input validation
they need and tests to prove the validation works. At this point the
SVE features are still only for TCG, so we provide some patches to
prepare for KVM and then a patch that allows the 'max' CPU type to
enable SVE with KVM, but at first without vector length properties.
After a bit more preparation we add the SVE vector length properties
to the KVM-enabled 'max' CPU type along with the additional input
validation and tests that that needs.  Finally we allow the 'host'
CPU type to also enjoy these properties by simply sharing them with it.

v3:
  - Added documentation (docs/arm-cpu-features.rst)
  - Fixed TCG emulation of ZCR
  - Fixed KVM put/get_sve registers to not forget to bswap FFR
  - Now ensure we never end up with no vector lengths selected and sve=on
  - Now also auto-disable uninitialized larger vector lengths that are
    no longer possible due to disabling a dependent smaller length
  - Now only use cpu_isar_feature(aa64_sve, cpu) to track SVE enablement
  - Now sve-max-vq can come wherever it wants in an sve* property sequence
  - No longer abuse sve_max_vq for additional states; instead use a couple
    new helper functions to determine the same states
  - Changed some error messages
  - Dropped some asserts
  - Added more code comments
  - Churned a good amount of code for refactoring and cleanups
  - Picked up some tags from Eric and Richard


Thanks!
drew

Andrew Jones (15):
  target/arm/cpu64: Ensure kvm really supports aarch64=off
  target/arm/cpu: Ensure we can use the pmu with kvm
  target/arm/monitor: Introduce qmp_query_cpu_model_expansion
  tests: arm: Introduce cpu feature tests
  target/arm/helper: zcr: Add build bug next to value range assumption
  target/arm/cpu: Use div-round-up to determine predicate register array
    size
  target/arm: Allow SVE to be disabled via a CPU property
  target/arm/cpu64: max cpu: Introduce sve<vl-bits> properties
  target/arm/kvm64: Fix error returns
  target/arm/kvm64: Move the get/put of fpsimd registers out
  target/arm/kvm64: Add kvm_arch_get/put_sve
  target/arm/kvm64: max cpu: Enable SVE when available
  target/arm/kvm: scratch vcpu: Preserve input kvm_vcpu_init features
  target/arm/cpu64: max cpu: Support sve properties with KVM
  target/arm/kvm: host cpu: Add support for sve<vl-bits> properties

 docs/arm-cpu-features.rst | 301 ++++++++++++++++++++++
 qapi/machine-target.json  |   6 +-
 target/arm/cpu.c          |  38 ++-
 target/arm/cpu.h          |  19 +-
 target/arm/cpu64.c        | 499 +++++++++++++++++++++++++++++++++--
 target/arm/helper.c       |  14 +-
 target/arm/kvm.c          |  32 ++-
 target/arm/kvm32.c        |   6 +-
 target/arm/kvm64.c        | 439 ++++++++++++++++++++++++++-----
 target/arm/kvm_arm.h      |  67 +++++
 target/arm/monitor.c      | 154 +++++++++++
 tests/Makefile.include    |   5 +-
 tests/arm-cpu-features.c  | 529 ++++++++++++++++++++++++++++++++++++++
 13 files changed, 2005 insertions(+), 104 deletions(-)
 create mode 100644 docs/arm-cpu-features.rst
 create mode 100644 tests/arm-cpu-features.c

-- 
2.20.1

Re: [Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Peter Maydell]

On Fri, 2 Aug 2019 at 13:25, Andrew Jones <drjones@redhat.com> wrote:
>
> Since Linux kernel v5.2-rc1 KVM has support for enabling SVE in guests.
> This series provides the QEMU bits for that enablement. First, we
> select existing CPU properties representing features we want to
> advertise in addition to the SVE vector lengths and prepare
> them for a qmp query. Then we introduce the qmp query, applying
> it immediately to those selected features. We also document ARM CPU
> features at this time. We next add a qtest for the selected CPU
> features that uses the qmp query for its tests - and we continue to
> add tests as we add CPU features with the following patches. So then,
> once we have the support we need for CPU feature querying and testing,
> we add our first SVE CPU feature property, 'sve', which just allows
> SVE to be completely enabled/disabled. Following that feature property,
> we add all 16 vector length properties along with the input validation
> they need and tests to prove the validation works. At this point the
> SVE features are still only for TCG, so we provide some patches to
> prepare for KVM and then a patch that allows the 'max' CPU type to
> enable SVE with KVM, but at first without vector length properties.
> After a bit more preparation we add the SVE vector length properties
> to the KVM-enabled 'max' CPU type along with the additional input
> validation and tests that that needs.  Finally we allow the 'host'
> CPU type to also enjoy these properties by simply sharing them with it.

Hi -- I see there have been some review comments on these patches
that mean there'll be a v4. In the meantime, patches 1, 2, 5, 6, 9, 10
seem to me to be independent bugfixes/cleanups that have been reviewed.
Would you like me to take those into target-arm.next to reduce the
size of the patchset for v4, or is that going to make rebasing
painful on your end?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v3 00/15] target/arm/kvm: enable SVE in guests

[Andrew Jones]

On Thu, Aug 15, 2019 at 09:31:35AM +0100, Peter Maydell wrote:
> On Fri, 2 Aug 2019 at 13:25, Andrew Jones <drjones@redhat.com> wrote:
> >
> > Since Linux kernel v5.2-rc1 KVM has support for enabling SVE in guests.
> > This series provides the QEMU bits for that enablement. First, we
> > select existing CPU properties representing features we want to
> > advertise in addition to the SVE vector lengths and prepare
> > them for a qmp query. Then we introduce the qmp query, applying
> > it immediately to those selected features. We also document ARM CPU
> > features at this time. We next add a qtest for the selected CPU
> > features that uses the qmp query for its tests - and we continue to
> > add tests as we add CPU features with the following patches. So then,
> > once we have the support we need for CPU feature querying and testing,
> > we add our first SVE CPU feature property, 'sve', which just allows
> > SVE to be completely enabled/disabled. Following that feature property,
> > we add all 16 vector length properties along with the input validation
> > they need and tests to prove the validation works. At this point the
> > SVE features are still only for TCG, so we provide some patches to
> > prepare for KVM and then a patch that allows the 'max' CPU type to
> > enable SVE with KVM, but at first without vector length properties.
> > After a bit more preparation we add the SVE vector length properties
> > to the KVM-enabled 'max' CPU type along with the additional input
> > validation and tests that that needs.  Finally we allow the 'host'
> > CPU type to also enjoy these properties by simply sharing them with it.
> 
> Hi -- I see there have been some review comments on these patches
> that mean there'll be a v4. In the meantime, patches 1, 2, 5, 6, 9, 10
> seem to me to be independent bugfixes/cleanups that have been reviewed.
> Would you like me to take those into target-arm.next to reduce the
> size of the patchset for v4, or is that going to make rebasing
> painful on your end?
>

Hi Peter,

Please do take the fixups. I think the rebasing should go fine, and indeed
reducing the number of patches in the patchset should reduce some of my
maintenance and also some reviewer strain for v4.

Thanks,
drew