From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=BAYES_00,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42EEAC47082 for ; Thu, 3 Jun 2021 08:39:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 276E9613DE for ; Thu, 3 Jun 2021 08:39:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229629AbhFCIk7 (ORCPT ); Thu, 3 Jun 2021 04:40:59 -0400 Received: from mail.kernel.org ([198.145.29.99]:53986 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229479AbhFCIk7 (ORCPT ); Thu, 3 Jun 2021 04:40:59 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4DB1E6136E; Thu, 3 Jun 2021 08:39:15 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loisj-005D8B-7S; Thu, 03 Jun 2021 09:39:13 +0100 Date: Thu, 03 Jun 2021 09:39:09 +0100 Message-ID: <87zgw7z6j6.wl-maz@kernel.org> From: Marc Zyngier To: Jamie Iles Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Andre Przywara , Christoffer Dall , Jintack Lim , Haibo Xu , James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com Subject: Re: [PATCH v4 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20210510165920.1913477-1-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: jamie@nuviainc.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, andre.przywara@arm.com, christoffer.dall@arm.com, jintack@cs.columbia.edu, haibo.xu@linaro.org, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi Jamie, Funny, your email has a "Mail-Followup-To:" field that contains everyone but you... Not ideal! ;-) On Thu, 03 Jun 2021 08:07:22 +0100, Jamie Iles wrote: > > Hi Marc, > > On Mon, May 10, 2021 at 05:58:14PM +0100, Marc Zyngier wrote: > > Here the bi-annual drop of the KVM/arm64 NV support code. > > > > Not a lot has changed since [1], except for a discovery mechanism for > > the EL2 support, some tidying up in the idreg emulation, dropping RMR > > support, and a rebase on top of 5.13-rc1. > > > > As usual, blame me for any bug, and nobody else. > > > > It is still massively painful to run on the FVP, but if you have a > > Neoverse V1 or N2 system that is collecting dust, I have the right > > stuff to keep it busy! > > I've been testing this series on FVP and get a crash when returning from > __kvm_vcpu_run_vhe because the autiasp is failing. Ah, the joy of testing with older guests. I guess i should upgrade by test rig and play with some newer guests at L1. > > The problem is when the L1 boots and during EL2 setup sets hcr_el2 to > HCR_HOST_NVHE_FLAGS and so enables HCR_APK|HCR_API. Then the guest > enter+exit logic in L0 starts performing the key save restore, but as we > didn't go through __hyp_handle_ptrauth, we haven't saved the host keys > and invoked vcpu_ptrauth_enable() so restore the host keys back to 0. > > I wonder if the pointer auth keys should be saved+restored > unconditionally for a guest when running nested rather than the lazy > faulting that we have today? I'd like to try and avoid that in order to keep the basic logic as simple as possible for the time being, and as close to the tried and trusted flow we have today. > Alternatively we would need to duplicate > the lazy logic for hcr_el2 writes. A quick hack of saving the host keys > in __kvm_vcpu_run_vhe before sysreg_save_host_state_vhe is enough to > allow me to boot an L1 with --nested and then an L2. > > Do we also need to filter out HCR_APK|HCR_API for hcr_el2 writes when > pointer authentication hasn't been exposed to the guest? I haven't yet > tried making ptrauth visible to the L1. I think this is the real thing. We should never propagate trap bits for features we don't want to support in guests. The L1 kernel sets these bits unconditionally, despite PtrAuth never being advertised, which trips the host code. Could you try the untested hack below? Thanks, M. diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index ce682bcce56f..54301d5ce58c 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -82,6 +82,7 @@ HCR_BSU_IS | HCR_FB | HCR_TAC | \ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO | HCR_PTW ) +#define HCR_GUEST_NV_FILTER_FLAGS (HCR_ATA | HCR_API | HCR_APK | HCR_RW) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) #define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 67f8b7d89db6..bf39bf4ef63c 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -64,6 +64,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu) */ u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2); + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS; + /* * We already set TVM to handle set/way cache maint * ops traps, this somewhat collides with the nested @@ -91,7 +93,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu) SYS_VNCR_EL2); } } else if (nested_virt_in_use(vcpu)) { - hcr |= __vcpu_sys_reg(vcpu, HCR_EL2); + u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2); + + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS; + hcr |= vhcr_el2; } ___activate_traps(vcpu, hcr); -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=BAYES_00,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F221DC47096 for ; Thu, 3 Jun 2021 08:39:23 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 4C136613DA for ; Thu, 3 Jun 2021 08:39:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C136613DA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id CA5BD4A7FD; Thu, 3 Jun 2021 04:39:22 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pwcxrMUuP4TQ; Thu, 3 Jun 2021 04:39:20 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 11C2D4B0D4; Thu, 3 Jun 2021 04:39:20 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id A59344A4A4 for ; Thu, 3 Jun 2021 04:39:18 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t1MxVbL9+5Yj for ; Thu, 3 Jun 2021 04:39:16 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 8B9004A195 for ; Thu, 3 Jun 2021 04:39:16 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4DB1E6136E; Thu, 3 Jun 2021 08:39:15 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loisj-005D8B-7S; Thu, 03 Jun 2021 09:39:13 +0100 Date: Thu, 03 Jun 2021 09:39:09 +0100 Message-ID: <87zgw7z6j6.wl-maz@kernel.org> From: Marc Zyngier To: Jamie Iles Subject: Re: [PATCH v4 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20210510165920.1913477-1-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: jamie@nuviainc.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, andre.przywara@arm.com, christoffer.dall@arm.com, jintack@cs.columbia.edu, haibo.xu@linaro.org, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: kernel-team@android.com, kvm@vger.kernel.org, Andre Przywara , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hi Jamie, Funny, your email has a "Mail-Followup-To:" field that contains everyone but you... Not ideal! ;-) On Thu, 03 Jun 2021 08:07:22 +0100, Jamie Iles wrote: > > Hi Marc, > > On Mon, May 10, 2021 at 05:58:14PM +0100, Marc Zyngier wrote: > > Here the bi-annual drop of the KVM/arm64 NV support code. > > > > Not a lot has changed since [1], except for a discovery mechanism for > > the EL2 support, some tidying up in the idreg emulation, dropping RMR > > support, and a rebase on top of 5.13-rc1. > > > > As usual, blame me for any bug, and nobody else. > > > > It is still massively painful to run on the FVP, but if you have a > > Neoverse V1 or N2 system that is collecting dust, I have the right > > stuff to keep it busy! > > I've been testing this series on FVP and get a crash when returning from > __kvm_vcpu_run_vhe because the autiasp is failing. Ah, the joy of testing with older guests. I guess i should upgrade by test rig and play with some newer guests at L1. > > The problem is when the L1 boots and during EL2 setup sets hcr_el2 to > HCR_HOST_NVHE_FLAGS and so enables HCR_APK|HCR_API. Then the guest > enter+exit logic in L0 starts performing the key save restore, but as we > didn't go through __hyp_handle_ptrauth, we haven't saved the host keys > and invoked vcpu_ptrauth_enable() so restore the host keys back to 0. > > I wonder if the pointer auth keys should be saved+restored > unconditionally for a guest when running nested rather than the lazy > faulting that we have today? I'd like to try and avoid that in order to keep the basic logic as simple as possible for the time being, and as close to the tried and trusted flow we have today. > Alternatively we would need to duplicate > the lazy logic for hcr_el2 writes. A quick hack of saving the host keys > in __kvm_vcpu_run_vhe before sysreg_save_host_state_vhe is enough to > allow me to boot an L1 with --nested and then an L2. > > Do we also need to filter out HCR_APK|HCR_API for hcr_el2 writes when > pointer authentication hasn't been exposed to the guest? I haven't yet > tried making ptrauth visible to the L1. I think this is the real thing. We should never propagate trap bits for features we don't want to support in guests. The L1 kernel sets these bits unconditionally, despite PtrAuth never being advertised, which trips the host code. Could you try the untested hack below? Thanks, M. diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index ce682bcce56f..54301d5ce58c 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -82,6 +82,7 @@ HCR_BSU_IS | HCR_FB | HCR_TAC | \ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO | HCR_PTW ) +#define HCR_GUEST_NV_FILTER_FLAGS (HCR_ATA | HCR_API | HCR_APK | HCR_RW) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) #define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 67f8b7d89db6..bf39bf4ef63c 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -64,6 +64,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu) */ u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2); + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS; + /* * We already set TVM to handle set/way cache maint * ops traps, this somewhat collides with the nested @@ -91,7 +93,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu) SYS_VNCR_EL2); } } else if (nested_virt_in_use(vcpu)) { - hcr |= __vcpu_sys_reg(vcpu, HCR_EL2); + u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2); + + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS; + hcr |= vhcr_el2; } ___activate_traps(vcpu, hcr); -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3466DC47096 for ; Thu, 3 Jun 2021 08:41:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0462361008 for ; Thu, 3 Jun 2021 08:41:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0462361008 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IXxrr7ScEuxRE0zYVc8snBStwgKn6C8WNtIkGVIFfdM=; b=kgAVxIlbnRLrKY 9NmYedRu8cQkKD549SVdRZOFfXyB/3vYZlU0zo8YtCiArydgYcfK9gcB5Y+rH70kgGikFL+QOjMae koU1q6y8h8zfo7QWz7Ny/Z5cocjiv1Fr9qmvymO52tjQIG4dXu4ewmdHuojmYTuqLGvkTCzZzK51j TiHU0cTmDCSf+hZv5oXbzSedpKqCbXoems9F5BLv7HIgxrHVuEmIYf8zmAZ/lRnaI1KTaZ88jdydR 4tC+sRBHwelhnCYQTyb7W+iHtR7vnpPbVLzGd6MDOpEiU7YHOafasmA08jI3mj84JowFnVHQZsRkw hqXFqBkBuKa0YSp00zkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1loiso-007ilL-AN; Thu, 03 Jun 2021 08:39:18 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1loisl-007iks-Je for linux-arm-kernel@lists.infradead.org; Thu, 03 Jun 2021 08:39:17 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4DB1E6136E; Thu, 3 Jun 2021 08:39:15 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1loisj-005D8B-7S; Thu, 03 Jun 2021 09:39:13 +0100 Date: Thu, 03 Jun 2021 09:39:09 +0100 Message-ID: <87zgw7z6j6.wl-maz@kernel.org> From: Marc Zyngier To: Jamie Iles Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Andre Przywara , Christoffer Dall , Jintack Lim , Haibo Xu , James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com Subject: Re: [PATCH v4 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20210510165920.1913477-1-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: jamie@nuviainc.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, andre.przywara@arm.com, christoffer.dall@arm.com, jintack@cs.columbia.edu, haibo.xu@linaro.org, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210603_013915_704742_8A72375A X-CRM114-Status: GOOD ( 38.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Jamie, Funny, your email has a "Mail-Followup-To:" field that contains everyone but you... Not ideal! ;-) On Thu, 03 Jun 2021 08:07:22 +0100, Jamie Iles wrote: > > Hi Marc, > > On Mon, May 10, 2021 at 05:58:14PM +0100, Marc Zyngier wrote: > > Here the bi-annual drop of the KVM/arm64 NV support code. > > > > Not a lot has changed since [1], except for a discovery mechanism for > > the EL2 support, some tidying up in the idreg emulation, dropping RMR > > support, and a rebase on top of 5.13-rc1. > > > > As usual, blame me for any bug, and nobody else. > > > > It is still massively painful to run on the FVP, but if you have a > > Neoverse V1 or N2 system that is collecting dust, I have the right > > stuff to keep it busy! > > I've been testing this series on FVP and get a crash when returning from > __kvm_vcpu_run_vhe because the autiasp is failing. Ah, the joy of testing with older guests. I guess i should upgrade by test rig and play with some newer guests at L1. > > The problem is when the L1 boots and during EL2 setup sets hcr_el2 to > HCR_HOST_NVHE_FLAGS and so enables HCR_APK|HCR_API. Then the guest > enter+exit logic in L0 starts performing the key save restore, but as we > didn't go through __hyp_handle_ptrauth, we haven't saved the host keys > and invoked vcpu_ptrauth_enable() so restore the host keys back to 0. > > I wonder if the pointer auth keys should be saved+restored > unconditionally for a guest when running nested rather than the lazy > faulting that we have today? I'd like to try and avoid that in order to keep the basic logic as simple as possible for the time being, and as close to the tried and trusted flow we have today. > Alternatively we would need to duplicate > the lazy logic for hcr_el2 writes. A quick hack of saving the host keys > in __kvm_vcpu_run_vhe before sysreg_save_host_state_vhe is enough to > allow me to boot an L1 with --nested and then an L2. > > Do we also need to filter out HCR_APK|HCR_API for hcr_el2 writes when > pointer authentication hasn't been exposed to the guest? I haven't yet > tried making ptrauth visible to the L1. I think this is the real thing. We should never propagate trap bits for features we don't want to support in guests. The L1 kernel sets these bits unconditionally, despite PtrAuth never being advertised, which trips the host code. Could you try the untested hack below? Thanks, M. diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index ce682bcce56f..54301d5ce58c 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -82,6 +82,7 @@ HCR_BSU_IS | HCR_FB | HCR_TAC | \ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO | HCR_PTW ) +#define HCR_GUEST_NV_FILTER_FLAGS (HCR_ATA | HCR_API | HCR_APK | HCR_RW) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) #define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 67f8b7d89db6..bf39bf4ef63c 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -64,6 +64,8 @@ static void __activate_traps(struct kvm_vcpu *vcpu) */ u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2); + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS; + /* * We already set TVM to handle set/way cache maint * ops traps, this somewhat collides with the nested @@ -91,7 +93,10 @@ static void __activate_traps(struct kvm_vcpu *vcpu) SYS_VNCR_EL2); } } else if (nested_virt_in_use(vcpu)) { - hcr |= __vcpu_sys_reg(vcpu, HCR_EL2); + u64 vhcr_el2 = __vcpu_sys_reg(vcpu, HCR_EL2); + + vhcr_el2 &= ~HCR_GUEST_NV_FILTER_FLAGS; + hcr |= vhcr_el2; } ___activate_traps(vcpu, hcr); -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel