From: Oliver Upton <oupton@google.com> To: kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini <pbonzini@redhat.com>, Marc Zyngier <maz@kernel.org>, James Morse <james.morse@arm.com>, Alexandru Elisei <alexandru.elisei@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, Anup Patel <anup@brainfault.org>, Atish Patra <atishp@atishpatra.org>, Sean Christopherson <seanjc@google.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, Peter Shier <pshier@google.com>, Reiji Watanabe <reijiw@google.com>, Ricardo Koller <ricarkol@google.com>, Raghavendra Rao Ananta <rananta@google.com>, Jing Zhang <jingzhangos@google.com>, Oliver Upton <oupton@google.com> Subject: [PATCH v3 00/19] KVM: arm64: Implement PSCI SYSTEM_SUSPEND Date: Wed, 23 Feb 2022 04:18:25 +0000 [thread overview] Message-ID: <20220223041844.3984439-1-oupton@google.com> (raw) The PSCI v1.0 specification describes a call, SYSTEM_SUSPEND, which allows software to request that the system be placed into the lowest possible power state and await an IMPLEMENTATION DEFINED wakeup event. This call is optional in v1.0 and v1.1. KVM does not currently support it in the v1.0 implementation. This series adds support for the PSCI SYSTEM_SUSPEND call to KVM/arm64. By default, KVM will treat the call as equivalent to CPU_SUSPEND, wherein KVM handles the call as a guest WFI. However, this series also introduces an opt-in for the SYSTEM_SUSPEND call to exit to userspace. VMMs may use the event as a hint to save the VM to resume at a later time, freeing up system resources. Userspace can decide at the time of the exit whether or not to honor the SYSTEM_SUSPEND call. Patch 1 is a small cleanup already present in kvmarm/next, but the series depends on it so it has been included to guarantee the series builds. Patches 2-3 adds an additional check to the CPU_ON PSCI call. As Reiji noted, PSCI implementations can return INVALID_ADDRESS if it is determined that the provided entry address does not exist in the guest address space. Patch 4 is another small cleanup to generically filter SMC64 calls when running an AArch32 EL1, avoiding the need to add a special case for the new PSCI call introduced in this series. Patches 5-6 add support for tracking a vCPU's power state using KVM_MP_STATE_* values. This is significant as the series introduces an additional power state, which cannot be represented by the `vcpu->arch.power_off` boolean. Patch 7 is a nitpick regarding the naming of a KVM_REQ_ handler. Patches 8-9 provide the default implementation of PSCI SYSTEM_SUSPEND by synchronously resetting the calling vCPU and entering WFI. Patches 10-12 introduce a new MP state, KVM_MP_STATE_SUSPENDED, which implements 'sticky' suspension. If userspace puts a vCPU in this state, it will exit to userspace for every recognized wakeup event (pending interrupt). When userspace is satisfied that a VM should resume, it must explicitly unpark the vCPU by marking it runnable again. This is useful for userspace to implement PSCI SYSTEM_SUSPEND if it decides to trap the call. Patch 13 extends upon the implementation of PSCI SYSTEM_SUSPEND, granting userspace the opt-in capability of exiting to userspace on such a call. *NOTE* KVM_SYSTEM_EVENT_SUSPEND breaks away from the semantics of other system events. Userspace is required to manipulate the vCPU to either reset it or reject the call. Other PSCI calls that exit set an SMCCC return value before exiting, but doing so would clobber all of the pending reset state. I wanted to avoid adding additional API to convey the reset context to userspace so it may simply be expressed in the architected state. Patch 14 increments the reported PSCI version to 1.1, as KVM already meets the requirements. Patches 15-18 rework the PSCI selftest to make it amenable to additional test cases Lastly, patch 19 tests that the KVM_SYSTEM_EVENT_SUSPEND exits are working as intended, and that KVM rejects invalid calls to PSCI SYSTEM_SUSPEND. This series applies cleanly to v5.17-rc5. Testing was performed with the included selftest and suspending a QEMU guest (i.e. no system event exits) on an Ampere Altra machine. v2: https://patchwork.kernel.org/project/kvm/cover/20210923191610.3814698-1-oupton@google.com/ v2 -> v3: - rebase to 5.17-rc5 - Reject CPU_ON and SYSTEM_SUSPEND calls that provide an invalid IPA (Reiji) - do *not* defer WFI as a requested event (Marc) - Add support for userspace filtering of wakeup events if SUSPEND exits are enabled (Marc) - Bump the reported PSCI verision to v1.1 (Marc) Oliver Upton (19): KVM: arm64: Drop unused param from kvm_psci_version() KVM: arm64: Create a helper to check if IPA is valid KVM: arm64: Reject invalid addresses for CPU_ON PSCI call KVM: arm64: Clean up SMC64 PSCI filtering for AArch32 guests KVM: arm64: Dedupe vCPU power off helpers KVM: arm64: Track vCPU power state using MP state values KVM: arm64: Rename the KVM_REQ_SLEEP handler KVM: arm64: Add reset helper that accepts caller-provided reset state KVM: arm64: Implement PSCI SYSTEM_SUSPEND KVM: Create helper for setting a system event exit KVM: arm64: Return a value from check_vcpu_requests() KVM: arm64: Add support for userspace to suspend a vCPU KVM: arm64: Add support KVM_SYSTEM_EVENT_SUSPEND to PSCI SYSTEM_SUSPEND KVM: arm64: Raise default PSCI version to v1.1 selftests: KVM: Rename psci_cpu_on_test to psci_test selftests: KVM: Create helper for making SMCCC calls selftests: KVM: Use KVM_SET_MP_STATE to power off vCPU in psci_test selftests: KVM: Refactor psci_test to make it amenable to new tests selftests: KVM: Test SYSTEM_SUSPEND PSCI call Documentation/virt/kvm/api.rst | 62 ++++- arch/arm64/include/asm/kvm_host.h | 27 ++- arch/arm64/include/asm/kvm_mmu.h | 9 + arch/arm64/kvm/arm.c | 88 +++++-- arch/arm64/kvm/psci.c | 129 ++++++++--- arch/arm64/kvm/reset.c | 45 ++-- arch/arm64/kvm/vgic/vgic-kvm-device.c | 2 +- arch/riscv/kvm/vcpu_sbi_v01.c | 4 +- arch/x86/kvm/x86.c | 6 +- include/kvm/arm_psci.h | 9 +- include/linux/kvm_host.h | 7 + include/uapi/linux/kvm.h | 4 + tools/testing/selftests/kvm/.gitignore | 2 +- tools/testing/selftests/kvm/Makefile | 2 +- .../selftests/kvm/aarch64/psci_cpu_on_test.c | 121 ---------- .../testing/selftests/kvm/aarch64/psci_test.c | 218 ++++++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 22 ++ .../selftests/kvm/lib/aarch64/processor.c | 25 ++ tools/testing/selftests/kvm/steal_time.c | 13 +- 19 files changed, 571 insertions(+), 224 deletions(-) delete mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c create mode 100644 tools/testing/selftests/kvm/aarch64/psci_test.c -- 2.35.1.473.g83b2b277ed-goog
WARNING: multiple messages have this Message-ID (diff)
From: Oliver Upton <oupton@google.com> To: kvmarm@lists.cs.columbia.edu Cc: Wanpeng Li <wanpengli@tencent.com>, kvm@vger.kernel.org, Joerg Roedel <joro@8bytes.org>, Atish Patra <atishp@atishpatra.org>, Peter Shier <pshier@google.com>, kvm-riscv@lists.infradead.org, Marc Zyngier <maz@kernel.org>, Paolo Bonzini <pbonzini@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Jim Mattson <jmattson@google.com> Subject: [PATCH v3 00/19] KVM: arm64: Implement PSCI SYSTEM_SUSPEND Date: Wed, 23 Feb 2022 04:18:25 +0000 [thread overview] Message-ID: <20220223041844.3984439-1-oupton@google.com> (raw) The PSCI v1.0 specification describes a call, SYSTEM_SUSPEND, which allows software to request that the system be placed into the lowest possible power state and await an IMPLEMENTATION DEFINED wakeup event. This call is optional in v1.0 and v1.1. KVM does not currently support it in the v1.0 implementation. This series adds support for the PSCI SYSTEM_SUSPEND call to KVM/arm64. By default, KVM will treat the call as equivalent to CPU_SUSPEND, wherein KVM handles the call as a guest WFI. However, this series also introduces an opt-in for the SYSTEM_SUSPEND call to exit to userspace. VMMs may use the event as a hint to save the VM to resume at a later time, freeing up system resources. Userspace can decide at the time of the exit whether or not to honor the SYSTEM_SUSPEND call. Patch 1 is a small cleanup already present in kvmarm/next, but the series depends on it so it has been included to guarantee the series builds. Patches 2-3 adds an additional check to the CPU_ON PSCI call. As Reiji noted, PSCI implementations can return INVALID_ADDRESS if it is determined that the provided entry address does not exist in the guest address space. Patch 4 is another small cleanup to generically filter SMC64 calls when running an AArch32 EL1, avoiding the need to add a special case for the new PSCI call introduced in this series. Patches 5-6 add support for tracking a vCPU's power state using KVM_MP_STATE_* values. This is significant as the series introduces an additional power state, which cannot be represented by the `vcpu->arch.power_off` boolean. Patch 7 is a nitpick regarding the naming of a KVM_REQ_ handler. Patches 8-9 provide the default implementation of PSCI SYSTEM_SUSPEND by synchronously resetting the calling vCPU and entering WFI. Patches 10-12 introduce a new MP state, KVM_MP_STATE_SUSPENDED, which implements 'sticky' suspension. If userspace puts a vCPU in this state, it will exit to userspace for every recognized wakeup event (pending interrupt). When userspace is satisfied that a VM should resume, it must explicitly unpark the vCPU by marking it runnable again. This is useful for userspace to implement PSCI SYSTEM_SUSPEND if it decides to trap the call. Patch 13 extends upon the implementation of PSCI SYSTEM_SUSPEND, granting userspace the opt-in capability of exiting to userspace on such a call. *NOTE* KVM_SYSTEM_EVENT_SUSPEND breaks away from the semantics of other system events. Userspace is required to manipulate the vCPU to either reset it or reject the call. Other PSCI calls that exit set an SMCCC return value before exiting, but doing so would clobber all of the pending reset state. I wanted to avoid adding additional API to convey the reset context to userspace so it may simply be expressed in the architected state. Patch 14 increments the reported PSCI version to 1.1, as KVM already meets the requirements. Patches 15-18 rework the PSCI selftest to make it amenable to additional test cases Lastly, patch 19 tests that the KVM_SYSTEM_EVENT_SUSPEND exits are working as intended, and that KVM rejects invalid calls to PSCI SYSTEM_SUSPEND. This series applies cleanly to v5.17-rc5. Testing was performed with the included selftest and suspending a QEMU guest (i.e. no system event exits) on an Ampere Altra machine. v2: https://patchwork.kernel.org/project/kvm/cover/20210923191610.3814698-1-oupton@google.com/ v2 -> v3: - rebase to 5.17-rc5 - Reject CPU_ON and SYSTEM_SUSPEND calls that provide an invalid IPA (Reiji) - do *not* defer WFI as a requested event (Marc) - Add support for userspace filtering of wakeup events if SUSPEND exits are enabled (Marc) - Bump the reported PSCI verision to v1.1 (Marc) Oliver Upton (19): KVM: arm64: Drop unused param from kvm_psci_version() KVM: arm64: Create a helper to check if IPA is valid KVM: arm64: Reject invalid addresses for CPU_ON PSCI call KVM: arm64: Clean up SMC64 PSCI filtering for AArch32 guests KVM: arm64: Dedupe vCPU power off helpers KVM: arm64: Track vCPU power state using MP state values KVM: arm64: Rename the KVM_REQ_SLEEP handler KVM: arm64: Add reset helper that accepts caller-provided reset state KVM: arm64: Implement PSCI SYSTEM_SUSPEND KVM: Create helper for setting a system event exit KVM: arm64: Return a value from check_vcpu_requests() KVM: arm64: Add support for userspace to suspend a vCPU KVM: arm64: Add support KVM_SYSTEM_EVENT_SUSPEND to PSCI SYSTEM_SUSPEND KVM: arm64: Raise default PSCI version to v1.1 selftests: KVM: Rename psci_cpu_on_test to psci_test selftests: KVM: Create helper for making SMCCC calls selftests: KVM: Use KVM_SET_MP_STATE to power off vCPU in psci_test selftests: KVM: Refactor psci_test to make it amenable to new tests selftests: KVM: Test SYSTEM_SUSPEND PSCI call Documentation/virt/kvm/api.rst | 62 ++++- arch/arm64/include/asm/kvm_host.h | 27 ++- arch/arm64/include/asm/kvm_mmu.h | 9 + arch/arm64/kvm/arm.c | 88 +++++-- arch/arm64/kvm/psci.c | 129 ++++++++--- arch/arm64/kvm/reset.c | 45 ++-- arch/arm64/kvm/vgic/vgic-kvm-device.c | 2 +- arch/riscv/kvm/vcpu_sbi_v01.c | 4 +- arch/x86/kvm/x86.c | 6 +- include/kvm/arm_psci.h | 9 +- include/linux/kvm_host.h | 7 + include/uapi/linux/kvm.h | 4 + tools/testing/selftests/kvm/.gitignore | 2 +- tools/testing/selftests/kvm/Makefile | 2 +- .../selftests/kvm/aarch64/psci_cpu_on_test.c | 121 ---------- .../testing/selftests/kvm/aarch64/psci_test.c | 218 ++++++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 22 ++ .../selftests/kvm/lib/aarch64/processor.c | 25 ++ tools/testing/selftests/kvm/steal_time.c | 13 +- 19 files changed, 571 insertions(+), 224 deletions(-) delete mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c create mode 100644 tools/testing/selftests/kvm/aarch64/psci_test.c -- 2.35.1.473.g83b2b277ed-goog _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
next reply other threads:[~2022-02-23 4:19 UTC|newest] Thread overview: 94+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-02-23 4:18 Oliver Upton [this message] 2022-02-23 4:18 ` [PATCH v3 00/19] KVM: arm64: Implement PSCI SYSTEM_SUSPEND Oliver Upton 2022-02-23 4:18 ` [PATCH v3 01/19] KVM: arm64: Drop unused param from kvm_psci_version() Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 6:14 ` Reiji Watanabe 2022-02-24 6:14 ` Reiji Watanabe 2022-02-23 4:18 ` [PATCH v3 02/19] KVM: arm64: Create a helper to check if IPA is valid Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 6:32 ` Reiji Watanabe 2022-02-24 6:32 ` Reiji Watanabe 2022-02-24 12:06 ` Marc Zyngier 2022-02-24 12:06 ` Marc Zyngier 2022-02-23 4:18 ` [PATCH v3 03/19] KVM: arm64: Reject invalid addresses for CPU_ON PSCI call Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 6:55 ` Reiji Watanabe 2022-02-24 6:55 ` Reiji Watanabe 2022-02-24 12:30 ` Marc Zyngier 2022-02-24 12:30 ` Marc Zyngier 2022-02-24 19:21 ` Oliver Upton 2022-02-24 19:21 ` Oliver Upton 2022-02-25 15:35 ` Marc Zyngier 2022-02-25 15:35 ` Marc Zyngier 2022-02-23 4:18 ` [PATCH v3 04/19] KVM: arm64: Clean up SMC64 PSCI filtering for AArch32 guests Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 05/19] KVM: arm64: Dedupe vCPU power off helpers Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 7:07 ` Reiji Watanabe 2022-02-24 7:07 ` Reiji Watanabe 2022-02-23 4:18 ` [PATCH v3 06/19] KVM: arm64: Track vCPU power state using MP state values Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 13:25 ` Marc Zyngier 2022-02-24 13:25 ` Marc Zyngier 2022-02-24 22:08 ` Oliver Upton 2022-02-24 22:08 ` Oliver Upton 2022-02-25 15:37 ` Marc Zyngier 2022-02-25 15:37 ` Marc Zyngier 2022-02-23 4:18 ` [PATCH v3 07/19] KVM: arm64: Rename the KVM_REQ_SLEEP handler Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 08/19] KVM: arm64: Add reset helper that accepts caller-provided reset state Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 09/19] KVM: arm64: Implement PSCI SYSTEM_SUSPEND Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 14:02 ` Marc Zyngier 2022-02-24 14:02 ` Marc Zyngier 2022-02-24 19:35 ` Oliver Upton 2022-02-24 19:35 ` Oliver Upton 2022-02-25 18:58 ` Marc Zyngier 2022-02-25 18:58 ` Marc Zyngier 2022-03-03 1:01 ` Oliver Upton 2022-03-03 1:01 ` Oliver Upton 2022-03-03 11:37 ` Marc Zyngier 2022-03-03 11:37 ` Marc Zyngier 2022-02-23 4:18 ` [PATCH v3 10/19] KVM: Create helper for setting a system event exit Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 6:37 ` Anup Patel 2022-02-23 6:37 ` Anup Patel 2022-02-24 14:07 ` Marc Zyngier 2022-02-24 14:07 ` Marc Zyngier 2022-02-23 4:18 ` [PATCH v3 11/19] KVM: arm64: Return a value from check_vcpu_requests() Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 12/19] KVM: arm64: Add support for userspace to suspend a vCPU Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 15:12 ` Marc Zyngier 2022-02-24 15:12 ` Marc Zyngier 2022-02-24 19:47 ` Oliver Upton 2022-02-24 19:47 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 13/19] KVM: arm64: Add support KVM_SYSTEM_EVENT_SUSPEND to PSCI SYSTEM_SUSPEND Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-24 15:40 ` Marc Zyngier 2022-02-24 15:40 ` Marc Zyngier 2022-02-24 20:05 ` Oliver Upton 2022-02-24 20:05 ` Oliver Upton 2022-02-26 11:29 ` Marc Zyngier 2022-02-26 11:29 ` Marc Zyngier 2022-02-26 18:28 ` Oliver Upton 2022-02-26 18:28 ` Oliver Upton 2022-03-02 9:52 ` Marc Zyngier 2022-03-02 9:52 ` Marc Zyngier 2022-03-02 9:57 ` Oliver Upton 2022-03-02 9:57 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 14/19] KVM: arm64: Raise default PSCI version to v1.1 Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:26 ` Oliver Upton 2022-02-23 4:26 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 15/19] selftests: KVM: Rename psci_cpu_on_test to psci_test Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 16/19] selftests: KVM: Create helper for making SMCCC calls Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 17/19] selftests: KVM: Use KVM_SET_MP_STATE to power off vCPU in psci_test Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 18/19] selftests: KVM: Refactor psci_test to make it amenable to new tests Oliver Upton 2022-02-23 4:18 ` Oliver Upton 2022-02-23 4:18 ` [PATCH v3 19/19] selftests: KVM: Test SYSTEM_SUSPEND PSCI call Oliver Upton 2022-02-23 4:18 ` Oliver Upton
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220223041844.3984439-1-oupton@google.com \ --to=oupton@google.com \ --cc=alexandru.elisei@arm.com \ --cc=anup@brainfault.org \ --cc=atishp@atishpatra.org \ --cc=james.morse@arm.com \ --cc=jingzhangos@google.com \ --cc=jmattson@google.com \ --cc=joro@8bytes.org \ --cc=kvm-riscv@lists.infradead.org \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=maz@kernel.org \ --cc=pbonzini@redhat.com \ --cc=pshier@google.com \ --cc=rananta@google.com \ --cc=reijiw@google.com \ --cc=ricarkol@google.com \ --cc=seanjc@google.com \ --cc=suzuki.poulose@arm.com \ --cc=vkuznets@redhat.com \ --cc=wanpengli@tencent.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.