From: Marc Zyngier <maz@kernel.org> To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse <james.morse@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, Oliver Upton <oliver.upton@linux.dev>, Zenghui Yu <yuzenghui@huawei.com>, Ricardo Koller <ricarkol@google.com>, Simon Veith <sveith@amazon.de>, Reiji Watanabe <reijiw@google.com>, Colton Lewis <coltonlewis@google.com>, Joey Gouly <joey.gouly@arm.com>, dwmw2@infradead.org Subject: [PATCH v3 00/18] KVM: arm64: Rework timer offsetting for fun and profit Date: Fri, 24 Mar 2023 14:46:46 +0000 [thread overview] Message-ID: <20230324144704.4193635-1-maz@kernel.org> (raw) This series aims at satisfying multiple goals: - allow a VMM to atomically restore a timer offset for a whole VM instead of updating the offset each time a vcpu get its counter written - allow a VMM to save/restore the physical timer context, something that we cannot do at the moment due to the lack of offsetting - provide a framework that is suitable for NV support, where we get both global and per timer, per vcpu offsetting, and manage interrupts in a less braindead way. We fix a couple of issues along the way, both from a stylistic and correctness perspective. This results in a new per VM KVM API that allows a global offset to be set at any point in time, overriding both of the timer counter writebacks. We also take this opportunity to rework the way IRQs are associated with timers, something that was always a bit dodgy. This relies on a new lock, which should disappear once Oliver's lock ordering series is merged (we can reuse the config_lock for this). This has been tested with nVHE, VHE and NV. I do not have access to CNTPOFF-aware HW, but Colton managed to give it a go. Note that the NV patches in this series are here to give a perspective on how this gets used. I've updated the arch_timer selftest to allow an offset to be provided from the command line, and fixed a couple of glaring issues along the way. Note that this is at best 6.4 material. I have a branch stashed at [0] and based on 6.3-rc3, as well as a minimal example of the use of the API at [3] based on kvmtool. Simon: I'd appreciate some feedback as whether this change fits your requirements, given that you brought this up the first place. Thanks, M. * From v2 [2]: - Fixed 32bit handling of the physical counter when the offset is non-zero - Dropped unused -O option from the selftest - Added lockdep_assert_held() to (un)lock_all_vcpus() - Reordered the last two patches - Added Colton's RBs, with thanks - Dropped the initial patch which has been merged - Rebased on 6.3-rc3 * From v1 [1]: - Switched from a dual offset to a single one which gets applied to both virtual and physical counters. Which means that NV doesn't behave oddly anymore by ignoring the virtual offset. - Some cosmetic repainting of the UAPI symbols - Added patches to rework the IRQ mapping to timers - Patch #1 on its way to Paolo - Rebased on 6.3-rc1 [0] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/timer-vm-offsets [1] https://lore.kernel.org/r/20230216142123.2638675-1-maz@kernel.org [2] https://lore.kernel.org/r/20230313124837.2264882-5-maz@kernel.org [3] https://git.kernel.org/pub/scm/linux/kernel/git/maz/kvmtool.git/commit/?h=zero-offset&id=3b1253073ee57c0d92baf7b214362829b487b8d5 Marc Zyngier (18): KVM: arm64: timers: Use a per-vcpu, per-timer accumulator for fractional ns arm64: Add CNTPOFF_EL2 register definition arm64: Add HAS_ECV_CNTPOFF capability KVM: arm64: timers: Use CNTPOFF_EL2 to offset the physical timer KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2 KVM: arm64: Expose {un,}lock_all_vcpus() to the rest of KVM KVM: arm64: timers: Allow userspace to set the global counter offset KVM: arm64: timers: Allow save/restoring of the physical timer KVM: arm64: timers: Rationalise per-vcpu timer init KVM: arm64: timers: Abstract per-timer IRQ access KVM: arm64: timers: Move the timer IRQs into arch_timer_vm_data KVM: arm64: Abstract the number of valid timers per vcpu KVM: arm64: Document KVM_ARM_SET_CNT_OFFSETS and co KVM: arm64: nv: timers: Add a per-timer, per-vcpu offset KVM: arm64: nv: timers: Support hyp timer emulation KVM: arm64: selftests: Add physical timer registers to the sysreg list KVM: arm64: selftests: Deal with spurious timer interrupts KVM: arm64: selftests: Augment existing timer test to handle variable offset Documentation/virt/kvm/api.rst | 38 ++ arch/arm64/include/asm/kvm_host.h | 13 + arch/arm64/include/asm/sysreg.h | 2 + arch/arm64/include/uapi/asm/kvm.h | 11 + arch/arm64/kernel/cpufeature.c | 11 + arch/arm64/kvm/arch_timer.c | 539 ++++++++++++++---- arch/arm64/kvm/arm.c | 53 ++ arch/arm64/kvm/guest.c | 29 +- arch/arm64/kvm/hyp/nvhe/timer-sr.c | 18 +- arch/arm64/kvm/hypercalls.c | 2 +- arch/arm64/kvm/sys_regs.c | 9 + arch/arm64/kvm/trace_arm.h | 6 +- arch/arm64/kvm/vgic/vgic-kvm-device.c | 38 -- arch/arm64/kvm/vgic/vgic.c | 15 + arch/arm64/kvm/vgic/vgic.h | 3 - arch/arm64/tools/cpucaps | 1 + arch/arm64/tools/sysreg | 4 + include/clocksource/arm_arch_timer.h | 1 + include/kvm/arm_arch_timer.h | 36 +- include/kvm/arm_vgic.h | 1 + include/uapi/linux/kvm.h | 3 + .../selftests/kvm/aarch64/arch_timer.c | 56 +- .../selftests/kvm/aarch64/get-reg-list.c | 5 +- 23 files changed, 690 insertions(+), 204 deletions(-) -- 2.34.1
WARNING: multiple messages have this Message-ID (diff)
From: Marc Zyngier <maz@kernel.org> To: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse <james.morse@arm.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, Oliver Upton <oliver.upton@linux.dev>, Zenghui Yu <yuzenghui@huawei.com>, Ricardo Koller <ricarkol@google.com>, Simon Veith <sveith@amazon.de>, Reiji Watanabe <reijiw@google.com>, Colton Lewis <coltonlewis@google.com>, Joey Gouly <joey.gouly@arm.com>, dwmw2@infradead.org Subject: [PATCH v3 00/18] KVM: arm64: Rework timer offsetting for fun and profit Date: Fri, 24 Mar 2023 14:46:46 +0000 [thread overview] Message-ID: <20230324144704.4193635-1-maz@kernel.org> (raw) This series aims at satisfying multiple goals: - allow a VMM to atomically restore a timer offset for a whole VM instead of updating the offset each time a vcpu get its counter written - allow a VMM to save/restore the physical timer context, something that we cannot do at the moment due to the lack of offsetting - provide a framework that is suitable for NV support, where we get both global and per timer, per vcpu offsetting, and manage interrupts in a less braindead way. We fix a couple of issues along the way, both from a stylistic and correctness perspective. This results in a new per VM KVM API that allows a global offset to be set at any point in time, overriding both of the timer counter writebacks. We also take this opportunity to rework the way IRQs are associated with timers, something that was always a bit dodgy. This relies on a new lock, which should disappear once Oliver's lock ordering series is merged (we can reuse the config_lock for this). This has been tested with nVHE, VHE and NV. I do not have access to CNTPOFF-aware HW, but Colton managed to give it a go. Note that the NV patches in this series are here to give a perspective on how this gets used. I've updated the arch_timer selftest to allow an offset to be provided from the command line, and fixed a couple of glaring issues along the way. Note that this is at best 6.4 material. I have a branch stashed at [0] and based on 6.3-rc3, as well as a minimal example of the use of the API at [3] based on kvmtool. Simon: I'd appreciate some feedback as whether this change fits your requirements, given that you brought this up the first place. Thanks, M. * From v2 [2]: - Fixed 32bit handling of the physical counter when the offset is non-zero - Dropped unused -O option from the selftest - Added lockdep_assert_held() to (un)lock_all_vcpus() - Reordered the last two patches - Added Colton's RBs, with thanks - Dropped the initial patch which has been merged - Rebased on 6.3-rc3 * From v1 [1]: - Switched from a dual offset to a single one which gets applied to both virtual and physical counters. Which means that NV doesn't behave oddly anymore by ignoring the virtual offset. - Some cosmetic repainting of the UAPI symbols - Added patches to rework the IRQ mapping to timers - Patch #1 on its way to Paolo - Rebased on 6.3-rc1 [0] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/timer-vm-offsets [1] https://lore.kernel.org/r/20230216142123.2638675-1-maz@kernel.org [2] https://lore.kernel.org/r/20230313124837.2264882-5-maz@kernel.org [3] https://git.kernel.org/pub/scm/linux/kernel/git/maz/kvmtool.git/commit/?h=zero-offset&id=3b1253073ee57c0d92baf7b214362829b487b8d5 Marc Zyngier (18): KVM: arm64: timers: Use a per-vcpu, per-timer accumulator for fractional ns arm64: Add CNTPOFF_EL2 register definition arm64: Add HAS_ECV_CNTPOFF capability KVM: arm64: timers: Use CNTPOFF_EL2 to offset the physical timer KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2 KVM: arm64: Expose {un,}lock_all_vcpus() to the rest of KVM KVM: arm64: timers: Allow userspace to set the global counter offset KVM: arm64: timers: Allow save/restoring of the physical timer KVM: arm64: timers: Rationalise per-vcpu timer init KVM: arm64: timers: Abstract per-timer IRQ access KVM: arm64: timers: Move the timer IRQs into arch_timer_vm_data KVM: arm64: Abstract the number of valid timers per vcpu KVM: arm64: Document KVM_ARM_SET_CNT_OFFSETS and co KVM: arm64: nv: timers: Add a per-timer, per-vcpu offset KVM: arm64: nv: timers: Support hyp timer emulation KVM: arm64: selftests: Add physical timer registers to the sysreg list KVM: arm64: selftests: Deal with spurious timer interrupts KVM: arm64: selftests: Augment existing timer test to handle variable offset Documentation/virt/kvm/api.rst | 38 ++ arch/arm64/include/asm/kvm_host.h | 13 + arch/arm64/include/asm/sysreg.h | 2 + arch/arm64/include/uapi/asm/kvm.h | 11 + arch/arm64/kernel/cpufeature.c | 11 + arch/arm64/kvm/arch_timer.c | 539 ++++++++++++++---- arch/arm64/kvm/arm.c | 53 ++ arch/arm64/kvm/guest.c | 29 +- arch/arm64/kvm/hyp/nvhe/timer-sr.c | 18 +- arch/arm64/kvm/hypercalls.c | 2 +- arch/arm64/kvm/sys_regs.c | 9 + arch/arm64/kvm/trace_arm.h | 6 +- arch/arm64/kvm/vgic/vgic-kvm-device.c | 38 -- arch/arm64/kvm/vgic/vgic.c | 15 + arch/arm64/kvm/vgic/vgic.h | 3 - arch/arm64/tools/cpucaps | 1 + arch/arm64/tools/sysreg | 4 + include/clocksource/arm_arch_timer.h | 1 + include/kvm/arm_arch_timer.h | 36 +- include/kvm/arm_vgic.h | 1 + include/uapi/linux/kvm.h | 3 + .../selftests/kvm/aarch64/arch_timer.c | 56 +- .../selftests/kvm/aarch64/get-reg-list.c | 5 +- 23 files changed, 690 insertions(+), 204 deletions(-) -- 2.34.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next reply other threads:[~2023-03-24 14:47 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-03-24 14:46 Marc Zyngier [this message] 2023-03-24 14:46 ` [PATCH v3 00/18] KVM: arm64: Rework timer offsetting for fun and profit Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 01/18] KVM: arm64: timers: Use a per-vcpu, per-timer accumulator for fractional ns Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 02/18] arm64: Add CNTPOFF_EL2 register definition Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 03/18] arm64: Add HAS_ECV_CNTPOFF capability Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 04/18] KVM: arm64: timers: Use CNTPOFF_EL2 to offset the physical timer Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 05/18] KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2 Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-30 6:42 ` Oliver Upton 2023-03-30 6:42 ` Oliver Upton 2023-03-30 10:09 ` Marc Zyngier 2023-03-30 10:09 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 06/18] KVM: arm64: Expose {un,}lock_all_vcpus() to the rest of KVM Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 07/18] KVM: arm64: timers: Allow userspace to set the global counter offset Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-30 6:26 ` Oliver Upton 2023-03-30 6:26 ` Oliver Upton 2023-03-30 10:15 ` Marc Zyngier 2023-03-30 10:15 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 08/18] KVM: arm64: timers: Allow save/restoring of the physical timer Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 09/18] KVM: arm64: timers: Rationalise per-vcpu timer init Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 10/18] KVM: arm64: timers: Abstract per-timer IRQ access Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 11/18] KVM: arm64: timers: Move the timer IRQs into arch_timer_vm_data Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-30 7:02 ` Oliver Upton 2023-03-30 7:02 ` Oliver Upton 2023-03-30 10:19 ` Marc Zyngier 2023-03-30 10:19 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 12/18] KVM: arm64: Abstract the number of valid timers per vcpu Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:46 ` [PATCH v3 13/18] KVM: arm64: Document KVM_ARM_SET_CNT_OFFSETS and co Marc Zyngier 2023-03-24 14:46 ` Marc Zyngier 2023-03-24 14:47 ` [PATCH v3 14/18] KVM: arm64: nv: timers: Add a per-timer, per-vcpu offset Marc Zyngier 2023-03-24 14:47 ` Marc Zyngier 2023-03-24 14:47 ` [PATCH v3 15/18] KVM: arm64: nv: timers: Support hyp timer emulation Marc Zyngier 2023-03-24 14:47 ` Marc Zyngier 2023-03-24 14:47 ` [PATCH v3 16/18] KVM: arm64: selftests: Add physical timer registers to the sysreg list Marc Zyngier 2023-03-24 14:47 ` Marc Zyngier 2023-03-24 14:47 ` [PATCH v3 17/18] KVM: arm64: selftests: Deal with spurious timer interrupts Marc Zyngier 2023-03-24 14:47 ` Marc Zyngier 2023-03-24 14:47 ` [PATCH v3 18/18] KVM: arm64: selftests: Augment existing timer test to handle variable offset Marc Zyngier 2023-03-24 14:47 ` Marc Zyngier 2023-03-29 5:41 ` [PATCH v3 00/18] KVM: arm64: Rework timer offsetting for fun and profit Veith, Simon 2023-03-29 5:41 ` Veith, Simon 2023-03-30 17:46 ` Marc Zyngier 2023-03-30 17:46 ` Marc Zyngier
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230324144704.4193635-1-maz@kernel.org \ --to=maz@kernel.org \ --cc=coltonlewis@google.com \ --cc=dwmw2@infradead.org \ --cc=james.morse@arm.com \ --cc=joey.gouly@arm.com \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.linux.dev \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=oliver.upton@linux.dev \ --cc=reijiw@google.com \ --cc=ricarkol@google.com \ --cc=suzuki.poulose@arm.com \ --cc=sveith@amazon.de \ --cc=yuzenghui@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.