From: Marc Zyngier <maz@kernel.org>
To: Jamie Iles <jamie@nuviainc.com>
Cc: linux-arm-kernel@lists.infradead.org,
kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org,
Andre Przywara <andre.przywara@arm.com>,
Christoffer Dall <christoffer.dall@arm.com>,
Jintack Lim <jintack@cs.columbia.edu>,
Haibo Xu <haibo.xu@linaro.org>, James Morse <james.morse@arm.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Alexandru Elisei <alexandru.elisei@arm.com>,
kernel-team@android.com
Subject: Re: [PATCH v4 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
Date: Thu, 03 Jun 2021 12:08:39 +0100 [thread overview]
Message-ID: <87y2bryzm0.wl-maz@kernel.org> (raw)
In-Reply-To: <87zgw7z6j6.wl-maz@kernel.org>
On Thu, 03 Jun 2021 09:39:09 +0100,
Marc Zyngier <maz@kernel.org> wrote:
>
> Hi Jamie,
>
> Funny, your email has a "Mail-Followup-To:" field that contains
> everyone but you... Not ideal! ;-)
>
> On Thu, 03 Jun 2021 08:07:22 +0100,
> Jamie Iles <jamie@nuviainc.com> wrote:
> >
> > Hi Marc,
> >
> > On Mon, May 10, 2021 at 05:58:14PM +0100, Marc Zyngier wrote:
> > > Here the bi-annual drop of the KVM/arm64 NV support code.
> > >
> > > Not a lot has changed since [1], except for a discovery mechanism for
> > > the EL2 support, some tidying up in the idreg emulation, dropping RMR
> > > support, and a rebase on top of 5.13-rc1.
> > >
> > > As usual, blame me for any bug, and nobody else.
> > >
> > > It is still massively painful to run on the FVP, but if you have a
> > > Neoverse V1 or N2 system that is collecting dust, I have the right
> > > stuff to keep it busy!
> >
> > I've been testing this series on FVP and get a crash when returning from
> > __kvm_vcpu_run_vhe because the autiasp is failing.
>
> Ah, the joy of testing with older guests. I guess i should upgrade by
> test rig and play with some newer guests at L1.
>
> >
> > The problem is when the L1 boots and during EL2 setup sets hcr_el2 to
> > HCR_HOST_NVHE_FLAGS and so enables HCR_APK|HCR_API. Then the guest
> > enter+exit logic in L0 starts performing the key save restore, but as we
> > didn't go through __hyp_handle_ptrauth, we haven't saved the host keys
> > and invoked vcpu_ptrauth_enable() so restore the host keys back to 0.
> >
> > I wonder if the pointer auth keys should be saved+restored
> > unconditionally for a guest when running nested rather than the lazy
> > faulting that we have today?
>
> I'd like to try and avoid that in order to keep the basic logic as
> simple as possible for the time being, and as close to the tried and
> trusted flow we have today.
>
> > Alternatively we would need to duplicate
> > the lazy logic for hcr_el2 writes. A quick hack of saving the host keys
> > in __kvm_vcpu_run_vhe before sysreg_save_host_state_vhe is enough to
> > allow me to boot an L1 with --nested and then an L2.
> >
> > Do we also need to filter out HCR_APK|HCR_API for hcr_el2 writes when
> > pointer authentication hasn't been exposed to the guest? I haven't yet
> > tried making ptrauth visible to the L1.
>
> I think this is the real thing. We should never propagate trap bits
> for features we don't want to support in guests. The L1 kernel sets
> these bits unconditionally, despite PtrAuth never being advertised,
> which trips the host code.
>
> Could you try the untested hack below?
>
> Thanks,
>
> M.
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index ce682bcce56f..54301d5ce58c 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -82,6 +82,7 @@
> HCR_BSU_IS | HCR_FB | HCR_TAC | \
> HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \
> HCR_FMO | HCR_IMO | HCR_PTW )
> +#define HCR_GUEST_NV_FILTER_FLAGS (HCR_ATA | HCR_API | HCR_APK | HCR_RW)
> #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF)
> #define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA)
> #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC)
> diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
> index 67f8b7d89db6..bf39bf4ef63c 100644
> --- a/arch/arm64/kvm/hyp/vhe/switch.c
> +++ b/arch/arm64/kvm/hyp/vhe/switch.c
> @@ -64,6 +64,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
> */
> u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
>
> + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS;
> +
> /*
> * We already set TVM to handle set/way cache maint
> * ops traps, this somewhat collides with the nested
> @@ -91,7 +93,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
> SYS_VNCR_EL2);
> }
> } else if (nested_virt_in_use(vcpu)) {
> - hcr |= __vcpu_sys_reg(vcpu, HCR_EL2);
> + u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2);
> +
> + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS;
> + hcr |= vhcr_el2;
> }
>
> ___activate_traps(vcpu, hcr);
>
FWIW, I can boot a 5.13-rc4 kernel as a L1 guest without any visible
issue with this patch (and another guest as L2). I've folded this into
the series and pushed the result out (with a rebase on -rc4 for a good
measure).
Thanks again,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-06-03 11:10 UTC|newest]
Thread overview: 77+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-10 16:58 [PATCH v4 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 01/66] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature Marc Zyngier
2021-05-20 13:32 ` Zenghui Yu
2021-05-24 12:38 ` Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 02/66] KVM: arm64: nv: Introduce nested virtualization VCPU feature Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 03/66] KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 04/66] KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 05/66] KVM: arm64: nv: Add EL2 system registers to vcpu context Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 06/66] KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 07/66] KVM: arm64: nv: Handle HCR_EL2.NV system register traps Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 08/66] KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 09/66] KVM: arm64: nv: Support virtual EL2 exceptions Marc Zyngier
2021-05-20 12:55 ` Zenghui Yu
2021-05-24 12:35 ` Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 10/66] KVM: arm64: nv: Inject HVC exceptions to the virtual EL2 Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 11/66] KVM: arm64: nv: Handle trapped ERET from " Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 12/66] KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 13/66] KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg() Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 14/66] KVM: arm64: nv: Handle SPSR_EL2 specially Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 15/66] KVM: arm64: nv: Handle HCR_EL2.E2H specially Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 16/66] KVM: arm64: nv: Save/Restore vEL2 sysregs Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 17/66] KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 18/66] KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2 Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 19/66] KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from " Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 20/66] KVM: arm64: nv: Trap CPACR_EL1 access in " Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 21/66] KVM: arm64: nv: Handle PSCI call via smc from the guest Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 22/66] KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 23/66] KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP, FPEN} settings Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 24/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 25/66] KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 26/66] KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 27/66] KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2 Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 28/66] KVM: arm64: nv: Forward debug traps to the nested guest Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 29/66] KVM: arm64: nv: Configure HCR_EL2 for nested virtualization Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 30/66] KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2 changes Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 31/66] KVM: arm64: nv: Filter out unsupported features from ID regs Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 32/66] KVM: arm64: nv: Hide RAS from nested guests Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 33/66] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 34/66] KVM: arm64: nv: Implement nested Stage-2 page table walk logic Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 35/66] KVM: arm64: nv: Handle shadow stage 2 page faults Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 36/66] KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 37/66] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 38/66] KVM: arm64: nv: Introduce sys_reg_desc.forward_trap Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 39/66] KVM: arm64: nv: Set a handler for the system instruction traps Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 40/66] KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2 Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 41/66] KVM: arm64: nv: Trap and emulate TLBI " Marc Zyngier
2021-07-14 16:40 ` Chase Conklin
2021-11-29 18:12 ` Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 42/66] KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 43/66] KVM: arm64: nv: arch_timer: Support hyp timer emulation Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 44/66] KVM: arm64: nv: Add handling of EL2-specific timer registers Marc Zyngier
2021-05-10 16:58 ` [PATCH v4 45/66] KVM: arm64: nv: Load timer before the GIC Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 46/66] KVM: arm64: nv: Nested GICv3 Support Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 47/66] KVM: arm64: nv: Don't load the GICv4 context on entering a nested guest Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 48/66] KVM: arm64: nv: vgic: Emulate the HW bit in software Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 49/66] KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 50/66] KVM: arm64: nv: Implement maintenance interrupt forwarding Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 51/66] KVM: arm64: nv: Add nested GICv3 tracepoints Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 52/66] KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 53/66] KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 54/66] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 55/66] KVM: arm64: Allow populating S2 SW bits Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 56/66] KVM: arm64: nv: Tag shadow S2 entries with nested level Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 57/66] KVM: arm64: nv: Add include containing the VNCR_EL2 offsets Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 58/66] KVM: arm64: Map VNCR-capable registers to a separate page Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 59/66] KVM: arm64: nv: Move nested vgic state into the sysreg file Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 60/66] KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 61/66] KVM: arm64: nv: Synchronize PSTATE early on exit Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 62/66] KVM: arm64: nv: Sync nested timer state with ARMv8.4 Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 63/66] KVM: arm64: nv: Allocate VNCR page when required Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 64/66] KVM: arm64: nv: Enable ARMv8.4-NV support Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 65/66] KVM: arm64: nv: Fast-track 'InHost' exception returns Marc Zyngier
2021-05-10 16:59 ` [PATCH v4 66/66] KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests Marc Zyngier
2021-06-03 7:07 ` [PATCH v4 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support Jamie Iles
2021-06-03 8:39 ` Marc Zyngier
2021-06-03 11:08 ` Marc Zyngier [this message]
2021-06-07 9:59 ` Jamie Iles
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y2bryzm0.wl-maz@kernel.org \
--to=maz@kernel.org \
--cc=alexandru.elisei@arm.com \
--cc=andre.przywara@arm.com \
--cc=christoffer.dall@arm.com \
--cc=haibo.xu@linaro.org \
--cc=james.morse@arm.com \
--cc=jamie@nuviainc.com \
--cc=jintack@cs.columbia.edu \
--cc=kernel-team@android.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=suzuki.poulose@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).