From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id 427B0C433F5 for ; Thu, 19 May 2022 13:45:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id DF3F04B3E4; Thu, 19 May 2022 09:45:54 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8V+nxLCr-Apg; Thu, 19 May 2022 09:45:53 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 7E4AA4B3D8; Thu, 19 May 2022 09:45:53 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 555D74B3F8 for ; Thu, 19 May 2022 09:45:52 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VQ1My3smEs5D for ; Thu, 19 May 2022 09:45:51 -0400 (EDT) Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id C4EDB4B416 for ; Thu, 19 May 2022 09:45:49 -0400 (EDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CFC9DB824AB; Thu, 19 May 2022 13:45:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25501C34100; Thu, 19 May 2022 13:45:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652967947; bh=vn94WCmAYZN6zOdSO+ooW8+de2325RR/Q7OATfCvR9k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YWFWQwTQ5O1KPqUp4BW5LPJrK1PuwVjN4EzxwAUnvG9gKXhFK29lUf6lJCynTmQIY uuMF81D9IwMYXDoVfnf80ZErySl4bRkjRdJnmNTXylawqPQbuVSX7ng/D9cFD6o35g InsiV34IuXF/t7k7pEd+BVAkOnKwqbpkxLsqi0/9qBBcGc/p1RAoQOpXNfg/oZUbNE QQAnwIRB783+q8GGoqZRySDQTalK6LEId6R7zRcOepIeV9Pi2pG0wO7Lg3Zeyao50t iB3nCYJg/r5LeOJHjZ1PjccVQpsPbVBB7UNvLWkIbzJ43AdLz8rhzMjROferhnKAoX ZMdk4tF0CqUgA== From: Will Deacon To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 51/89] KVM: arm64: Introduce per-EC entry/exit handlers Date: Thu, 19 May 2022 14:41:26 +0100 Message-Id: <20220519134204.5379-52-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220519134204.5379-1-will@kernel.org> References: <20220519134204.5379-1-will@kernel.org> MIME-Version: 1.0 Cc: Marc Zyngier , kernel-team@android.com, kvm@vger.kernel.org, Andy Lutomirski , linux-arm-kernel@lists.infradead.org, Michael Roth , Catalin Marinas , Chao Peng , Will Deacon X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu From: Marc Zyngier Introduce per-EC entry/exit handlers at EL2 and provide initial implementations to manage the 'flags' and fault information registers. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp/include/nvhe/pkvm.h | 3 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 99 +++++++++++++++++++++++--- 2 files changed, 94 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h index da8de2b7afb4..d070400b5616 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h @@ -25,6 +25,9 @@ struct kvm_shadow_vcpu_state { /* A pointer to the shadow vm. */ struct kvm_shadow_vm *shadow_vm; + /* Tracks exit code for the protected guest. */ + u32 exit_code; + /* * Points to the per-cpu pointer of the cpu where it's loaded, or NULL * if not loaded. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 229ef890d459..bbf2621f1862 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -24,6 +24,51 @@ DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); +typedef void (*shadow_entry_exit_handler_fn)(struct kvm_vcpu *, struct kvm_vcpu *); + +static void handle_vm_entry_generic(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + unsigned long host_flags = READ_ONCE(host_vcpu->arch.flags); + + shadow_vcpu->arch.flags &= ~(KVM_ARM64_PENDING_EXCEPTION | + KVM_ARM64_EXCEPT_MASK); + + if (host_flags & KVM_ARM64_PENDING_EXCEPTION) { + shadow_vcpu->arch.flags |= KVM_ARM64_PENDING_EXCEPTION; + shadow_vcpu->arch.flags |= host_flags & KVM_ARM64_EXCEPT_MASK; + } else if (host_flags & KVM_ARM64_INCREMENT_PC) { + shadow_vcpu->arch.flags |= KVM_ARM64_INCREMENT_PC; + } +} + +static void handle_vm_exit_generic(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + WRITE_ONCE(host_vcpu->arch.fault.esr_el2, + shadow_vcpu->arch.fault.esr_el2); +} + +static void handle_vm_exit_abt(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + WRITE_ONCE(host_vcpu->arch.fault.esr_el2, + shadow_vcpu->arch.fault.esr_el2); + WRITE_ONCE(host_vcpu->arch.fault.far_el2, + shadow_vcpu->arch.fault.far_el2); + WRITE_ONCE(host_vcpu->arch.fault.hpfar_el2, + shadow_vcpu->arch.fault.hpfar_el2); + WRITE_ONCE(host_vcpu->arch.fault.disr_el1, + shadow_vcpu->arch.fault.disr_el1); +} + +static const shadow_entry_exit_handler_fn entry_vm_shadow_handlers[] = { + [0 ... ESR_ELx_EC_MAX] = handle_vm_entry_generic, +}; + +static const shadow_entry_exit_handler_fn exit_vm_shadow_handlers[] = { + [0 ... ESR_ELx_EC_MAX] = handle_vm_exit_generic, + [ESR_ELx_EC_IABT_LOW] = handle_vm_exit_abt, + [ESR_ELx_EC_DABT_LOW] = handle_vm_exit_abt, +}; + static void flush_vgic_state(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) { @@ -99,6 +144,8 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) { struct kvm_vcpu *shadow_vcpu = &shadow_state->shadow_vcpu; struct kvm_vcpu *host_vcpu = shadow_state->host_vcpu; + shadow_entry_exit_handler_fn ec_handler; + u8 esr_ec; shadow_vcpu->arch.ctxt = host_vcpu->arch.ctxt; @@ -109,8 +156,6 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) shadow_vcpu->arch.mdcr_el2 = host_vcpu->arch.mdcr_el2; shadow_vcpu->arch.cptr_el2 = host_vcpu->arch.cptr_el2; - shadow_vcpu->arch.flags = host_vcpu->arch.flags; - shadow_vcpu->arch.debug_ptr = kern_hyp_va(host_vcpu->arch.debug_ptr); shadow_vcpu->arch.host_fpsimd_state = host_vcpu->arch.host_fpsimd_state; @@ -118,24 +163,62 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) flush_vgic_state(host_vcpu, shadow_vcpu); flush_timer_state(shadow_state); + + switch (ARM_EXCEPTION_CODE(shadow_state->exit_code)) { + case ARM_EXCEPTION_IRQ: + case ARM_EXCEPTION_EL1_SERROR: + case ARM_EXCEPTION_IL: + break; + case ARM_EXCEPTION_TRAP: + esr_ec = ESR_ELx_EC(kvm_vcpu_get_esr(shadow_vcpu)); + ec_handler = entry_vm_shadow_handlers[esr_ec]; + if (ec_handler) + ec_handler(host_vcpu, shadow_vcpu); + break; + default: + BUG(); + } + + shadow_state->exit_code = 0; } -static void sync_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) +static void sync_shadow_state(struct kvm_shadow_vcpu_state *shadow_state, + u32 exit_reason) { struct kvm_vcpu *shadow_vcpu = &shadow_state->shadow_vcpu; struct kvm_vcpu *host_vcpu = shadow_state->host_vcpu; + shadow_entry_exit_handler_fn ec_handler; + unsigned long host_flags; + u8 esr_ec; host_vcpu->arch.ctxt = shadow_vcpu->arch.ctxt; host_vcpu->arch.hcr_el2 = shadow_vcpu->arch.hcr_el2; host_vcpu->arch.cptr_el2 = shadow_vcpu->arch.cptr_el2; - host_vcpu->arch.fault = shadow_vcpu->arch.fault; - - host_vcpu->arch.flags = shadow_vcpu->arch.flags; - sync_vgic_state(host_vcpu, shadow_vcpu); sync_timer_state(shadow_state); + + switch (ARM_EXCEPTION_CODE(exit_reason)) { + case ARM_EXCEPTION_IRQ: + break; + case ARM_EXCEPTION_TRAP: + esr_ec = ESR_ELx_EC(kvm_vcpu_get_esr(shadow_vcpu)); + ec_handler = exit_vm_shadow_handlers[esr_ec]; + if (ec_handler) + ec_handler(host_vcpu, shadow_vcpu); + break; + case ARM_EXCEPTION_EL1_SERROR: + case ARM_EXCEPTION_IL: + break; + default: + BUG(); + } + + host_flags = READ_ONCE(host_vcpu->arch.flags) & + ~(KVM_ARM64_PENDING_EXCEPTION | KVM_ARM64_INCREMENT_PC); + WRITE_ONCE(host_vcpu->arch.flags, host_flags); + shadow_state->exit_code = exit_reason; } static void handle___pkvm_vcpu_load(struct kvm_cpu_context *host_ctxt) @@ -231,7 +314,7 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) ret = __kvm_vcpu_run(&shadow_state->shadow_vcpu); - sync_shadow_state(shadow_state); + sync_shadow_state(shadow_state, ret); } else { ret = __kvm_vcpu_run(vcpu); } -- 2.36.1.124.g0e6072fb45-goog _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32EA9C4332F for ; Thu, 19 May 2022 13:47:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231858AbiESNrG (ORCPT ); Thu, 19 May 2022 09:47:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239237AbiESNpy (ORCPT ); Thu, 19 May 2022 09:45:54 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E36511D for ; Thu, 19 May 2022 06:45:50 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C75B7B8235B for ; Thu, 19 May 2022 13:45:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25501C34100; Thu, 19 May 2022 13:45:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652967947; bh=vn94WCmAYZN6zOdSO+ooW8+de2325RR/Q7OATfCvR9k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YWFWQwTQ5O1KPqUp4BW5LPJrK1PuwVjN4EzxwAUnvG9gKXhFK29lUf6lJCynTmQIY uuMF81D9IwMYXDoVfnf80ZErySl4bRkjRdJnmNTXylawqPQbuVSX7ng/D9cFD6o35g InsiV34IuXF/t7k7pEd+BVAkOnKwqbpkxLsqi0/9qBBcGc/p1RAoQOpXNfg/oZUbNE QQAnwIRB783+q8GGoqZRySDQTalK6LEId6R7zRcOepIeV9Pi2pG0wO7Lg3Zeyao50t iB3nCYJg/r5LeOJHjZ1PjccVQpsPbVBB7UNvLWkIbzJ43AdLz8rhzMjROferhnKAoX ZMdk4tF0CqUgA== From: Will Deacon To: kvmarm@lists.cs.columbia.edu Cc: Will Deacon , Ard Biesheuvel , Sean Christopherson , Alexandru Elisei , Andy Lutomirski , Catalin Marinas , James Morse , Chao Peng , Quentin Perret , Suzuki K Poulose , Michael Roth , Mark Rutland , Fuad Tabba , Oliver Upton , Marc Zyngier , kernel-team@android.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 51/89] KVM: arm64: Introduce per-EC entry/exit handlers Date: Thu, 19 May 2022 14:41:26 +0100 Message-Id: <20220519134204.5379-52-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220519134204.5379-1-will@kernel.org> References: <20220519134204.5379-1-will@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Marc Zyngier Introduce per-EC entry/exit handlers at EL2 and provide initial implementations to manage the 'flags' and fault information registers. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp/include/nvhe/pkvm.h | 3 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 99 +++++++++++++++++++++++--- 2 files changed, 94 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h index da8de2b7afb4..d070400b5616 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h @@ -25,6 +25,9 @@ struct kvm_shadow_vcpu_state { /* A pointer to the shadow vm. */ struct kvm_shadow_vm *shadow_vm; + /* Tracks exit code for the protected guest. */ + u32 exit_code; + /* * Points to the per-cpu pointer of the cpu where it's loaded, or NULL * if not loaded. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 229ef890d459..bbf2621f1862 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -24,6 +24,51 @@ DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); +typedef void (*shadow_entry_exit_handler_fn)(struct kvm_vcpu *, struct kvm_vcpu *); + +static void handle_vm_entry_generic(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + unsigned long host_flags = READ_ONCE(host_vcpu->arch.flags); + + shadow_vcpu->arch.flags &= ~(KVM_ARM64_PENDING_EXCEPTION | + KVM_ARM64_EXCEPT_MASK); + + if (host_flags & KVM_ARM64_PENDING_EXCEPTION) { + shadow_vcpu->arch.flags |= KVM_ARM64_PENDING_EXCEPTION; + shadow_vcpu->arch.flags |= host_flags & KVM_ARM64_EXCEPT_MASK; + } else if (host_flags & KVM_ARM64_INCREMENT_PC) { + shadow_vcpu->arch.flags |= KVM_ARM64_INCREMENT_PC; + } +} + +static void handle_vm_exit_generic(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + WRITE_ONCE(host_vcpu->arch.fault.esr_el2, + shadow_vcpu->arch.fault.esr_el2); +} + +static void handle_vm_exit_abt(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + WRITE_ONCE(host_vcpu->arch.fault.esr_el2, + shadow_vcpu->arch.fault.esr_el2); + WRITE_ONCE(host_vcpu->arch.fault.far_el2, + shadow_vcpu->arch.fault.far_el2); + WRITE_ONCE(host_vcpu->arch.fault.hpfar_el2, + shadow_vcpu->arch.fault.hpfar_el2); + WRITE_ONCE(host_vcpu->arch.fault.disr_el1, + shadow_vcpu->arch.fault.disr_el1); +} + +static const shadow_entry_exit_handler_fn entry_vm_shadow_handlers[] = { + [0 ... ESR_ELx_EC_MAX] = handle_vm_entry_generic, +}; + +static const shadow_entry_exit_handler_fn exit_vm_shadow_handlers[] = { + [0 ... ESR_ELx_EC_MAX] = handle_vm_exit_generic, + [ESR_ELx_EC_IABT_LOW] = handle_vm_exit_abt, + [ESR_ELx_EC_DABT_LOW] = handle_vm_exit_abt, +}; + static void flush_vgic_state(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) { @@ -99,6 +144,8 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) { struct kvm_vcpu *shadow_vcpu = &shadow_state->shadow_vcpu; struct kvm_vcpu *host_vcpu = shadow_state->host_vcpu; + shadow_entry_exit_handler_fn ec_handler; + u8 esr_ec; shadow_vcpu->arch.ctxt = host_vcpu->arch.ctxt; @@ -109,8 +156,6 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) shadow_vcpu->arch.mdcr_el2 = host_vcpu->arch.mdcr_el2; shadow_vcpu->arch.cptr_el2 = host_vcpu->arch.cptr_el2; - shadow_vcpu->arch.flags = host_vcpu->arch.flags; - shadow_vcpu->arch.debug_ptr = kern_hyp_va(host_vcpu->arch.debug_ptr); shadow_vcpu->arch.host_fpsimd_state = host_vcpu->arch.host_fpsimd_state; @@ -118,24 +163,62 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) flush_vgic_state(host_vcpu, shadow_vcpu); flush_timer_state(shadow_state); + + switch (ARM_EXCEPTION_CODE(shadow_state->exit_code)) { + case ARM_EXCEPTION_IRQ: + case ARM_EXCEPTION_EL1_SERROR: + case ARM_EXCEPTION_IL: + break; + case ARM_EXCEPTION_TRAP: + esr_ec = ESR_ELx_EC(kvm_vcpu_get_esr(shadow_vcpu)); + ec_handler = entry_vm_shadow_handlers[esr_ec]; + if (ec_handler) + ec_handler(host_vcpu, shadow_vcpu); + break; + default: + BUG(); + } + + shadow_state->exit_code = 0; } -static void sync_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) +static void sync_shadow_state(struct kvm_shadow_vcpu_state *shadow_state, + u32 exit_reason) { struct kvm_vcpu *shadow_vcpu = &shadow_state->shadow_vcpu; struct kvm_vcpu *host_vcpu = shadow_state->host_vcpu; + shadow_entry_exit_handler_fn ec_handler; + unsigned long host_flags; + u8 esr_ec; host_vcpu->arch.ctxt = shadow_vcpu->arch.ctxt; host_vcpu->arch.hcr_el2 = shadow_vcpu->arch.hcr_el2; host_vcpu->arch.cptr_el2 = shadow_vcpu->arch.cptr_el2; - host_vcpu->arch.fault = shadow_vcpu->arch.fault; - - host_vcpu->arch.flags = shadow_vcpu->arch.flags; - sync_vgic_state(host_vcpu, shadow_vcpu); sync_timer_state(shadow_state); + + switch (ARM_EXCEPTION_CODE(exit_reason)) { + case ARM_EXCEPTION_IRQ: + break; + case ARM_EXCEPTION_TRAP: + esr_ec = ESR_ELx_EC(kvm_vcpu_get_esr(shadow_vcpu)); + ec_handler = exit_vm_shadow_handlers[esr_ec]; + if (ec_handler) + ec_handler(host_vcpu, shadow_vcpu); + break; + case ARM_EXCEPTION_EL1_SERROR: + case ARM_EXCEPTION_IL: + break; + default: + BUG(); + } + + host_flags = READ_ONCE(host_vcpu->arch.flags) & + ~(KVM_ARM64_PENDING_EXCEPTION | KVM_ARM64_INCREMENT_PC); + WRITE_ONCE(host_vcpu->arch.flags, host_flags); + shadow_state->exit_code = exit_reason; } static void handle___pkvm_vcpu_load(struct kvm_cpu_context *host_ctxt) @@ -231,7 +314,7 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) ret = __kvm_vcpu_run(&shadow_state->shadow_vcpu); - sync_shadow_state(shadow_state); + sync_shadow_state(shadow_state, ret); } else { ret = __kvm_vcpu_run(vcpu); } -- 2.36.1.124.g0e6072fb45-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C208C433F5 for ; Thu, 19 May 2022 14:21:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GPM60jnkZMoUTJ4JBTmFiW31aLjEPmaCcm/vOZlLfXU=; b=uituDrkap22j9y TfMcngcN36i0e6/TTM+1LUhAN9OdiL5hBfEOvtfJAksy/n+rnPcVY9xUdlZ5qQiN1DfsILNe32Vzb yj1cPRiixOpNbQNPjNZHlprNePpgUtH8Vq8B+5NLOtI6TwoePiRlOaQ6d02Lugwkio08VGDmIeZDZ WcSKQ7moyA5XgxAb5U95aiX+nx6KhUdJCv6Su1fSfkm3juQD1Qjthy8B/+K1OVA4WXLuA7CcaCk2z ALNL0At/mbC8YG07v3isOvxRVLznVgrc87/gIZuR9HbJMdL7rKnMJYnMZv0/ElhSbUV/pdvVZqksp MvuGlv8t55njgCs6umVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrh0m-007MON-Q6; Thu, 19 May 2022 14:20:22 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrgTO-0076Ux-2h for linux-arm-kernel@lists.infradead.org; Thu, 19 May 2022 13:45:54 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CFC9DB824AB; Thu, 19 May 2022 13:45:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25501C34100; Thu, 19 May 2022 13:45:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652967947; bh=vn94WCmAYZN6zOdSO+ooW8+de2325RR/Q7OATfCvR9k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YWFWQwTQ5O1KPqUp4BW5LPJrK1PuwVjN4EzxwAUnvG9gKXhFK29lUf6lJCynTmQIY uuMF81D9IwMYXDoVfnf80ZErySl4bRkjRdJnmNTXylawqPQbuVSX7ng/D9cFD6o35g InsiV34IuXF/t7k7pEd+BVAkOnKwqbpkxLsqi0/9qBBcGc/p1RAoQOpXNfg/oZUbNE QQAnwIRB783+q8GGoqZRySDQTalK6LEId6R7zRcOepIeV9Pi2pG0wO7Lg3Zeyao50t iB3nCYJg/r5LeOJHjZ1PjccVQpsPbVBB7UNvLWkIbzJ43AdLz8rhzMjROferhnKAoX ZMdk4tF0CqUgA== From: Will Deacon To: kvmarm@lists.cs.columbia.edu Cc: Will Deacon , Ard Biesheuvel , Sean Christopherson , Alexandru Elisei , Andy Lutomirski , Catalin Marinas , James Morse , Chao Peng , Quentin Perret , Suzuki K Poulose , Michael Roth , Mark Rutland , Fuad Tabba , Oliver Upton , Marc Zyngier , kernel-team@android.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 51/89] KVM: arm64: Introduce per-EC entry/exit handlers Date: Thu, 19 May 2022 14:41:26 +0100 Message-Id: <20220519134204.5379-52-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220519134204.5379-1-will@kernel.org> References: <20220519134204.5379-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220519_064550_436483_A16E6A81 X-CRM114-Status: GOOD ( 18.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Marc Zyngier Introduce per-EC entry/exit handlers at EL2 and provide initial implementations to manage the 'flags' and fault information registers. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp/include/nvhe/pkvm.h | 3 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 99 +++++++++++++++++++++++--- 2 files changed, 94 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h index da8de2b7afb4..d070400b5616 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h @@ -25,6 +25,9 @@ struct kvm_shadow_vcpu_state { /* A pointer to the shadow vm. */ struct kvm_shadow_vm *shadow_vm; + /* Tracks exit code for the protected guest. */ + u32 exit_code; + /* * Points to the per-cpu pointer of the cpu where it's loaded, or NULL * if not loaded. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 229ef890d459..bbf2621f1862 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -24,6 +24,51 @@ DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); +typedef void (*shadow_entry_exit_handler_fn)(struct kvm_vcpu *, struct kvm_vcpu *); + +static void handle_vm_entry_generic(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + unsigned long host_flags = READ_ONCE(host_vcpu->arch.flags); + + shadow_vcpu->arch.flags &= ~(KVM_ARM64_PENDING_EXCEPTION | + KVM_ARM64_EXCEPT_MASK); + + if (host_flags & KVM_ARM64_PENDING_EXCEPTION) { + shadow_vcpu->arch.flags |= KVM_ARM64_PENDING_EXCEPTION; + shadow_vcpu->arch.flags |= host_flags & KVM_ARM64_EXCEPT_MASK; + } else if (host_flags & KVM_ARM64_INCREMENT_PC) { + shadow_vcpu->arch.flags |= KVM_ARM64_INCREMENT_PC; + } +} + +static void handle_vm_exit_generic(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + WRITE_ONCE(host_vcpu->arch.fault.esr_el2, + shadow_vcpu->arch.fault.esr_el2); +} + +static void handle_vm_exit_abt(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + WRITE_ONCE(host_vcpu->arch.fault.esr_el2, + shadow_vcpu->arch.fault.esr_el2); + WRITE_ONCE(host_vcpu->arch.fault.far_el2, + shadow_vcpu->arch.fault.far_el2); + WRITE_ONCE(host_vcpu->arch.fault.hpfar_el2, + shadow_vcpu->arch.fault.hpfar_el2); + WRITE_ONCE(host_vcpu->arch.fault.disr_el1, + shadow_vcpu->arch.fault.disr_el1); +} + +static const shadow_entry_exit_handler_fn entry_vm_shadow_handlers[] = { + [0 ... ESR_ELx_EC_MAX] = handle_vm_entry_generic, +}; + +static const shadow_entry_exit_handler_fn exit_vm_shadow_handlers[] = { + [0 ... ESR_ELx_EC_MAX] = handle_vm_exit_generic, + [ESR_ELx_EC_IABT_LOW] = handle_vm_exit_abt, + [ESR_ELx_EC_DABT_LOW] = handle_vm_exit_abt, +}; + static void flush_vgic_state(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) { @@ -99,6 +144,8 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) { struct kvm_vcpu *shadow_vcpu = &shadow_state->shadow_vcpu; struct kvm_vcpu *host_vcpu = shadow_state->host_vcpu; + shadow_entry_exit_handler_fn ec_handler; + u8 esr_ec; shadow_vcpu->arch.ctxt = host_vcpu->arch.ctxt; @@ -109,8 +156,6 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) shadow_vcpu->arch.mdcr_el2 = host_vcpu->arch.mdcr_el2; shadow_vcpu->arch.cptr_el2 = host_vcpu->arch.cptr_el2; - shadow_vcpu->arch.flags = host_vcpu->arch.flags; - shadow_vcpu->arch.debug_ptr = kern_hyp_va(host_vcpu->arch.debug_ptr); shadow_vcpu->arch.host_fpsimd_state = host_vcpu->arch.host_fpsimd_state; @@ -118,24 +163,62 @@ static void flush_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) flush_vgic_state(host_vcpu, shadow_vcpu); flush_timer_state(shadow_state); + + switch (ARM_EXCEPTION_CODE(shadow_state->exit_code)) { + case ARM_EXCEPTION_IRQ: + case ARM_EXCEPTION_EL1_SERROR: + case ARM_EXCEPTION_IL: + break; + case ARM_EXCEPTION_TRAP: + esr_ec = ESR_ELx_EC(kvm_vcpu_get_esr(shadow_vcpu)); + ec_handler = entry_vm_shadow_handlers[esr_ec]; + if (ec_handler) + ec_handler(host_vcpu, shadow_vcpu); + break; + default: + BUG(); + } + + shadow_state->exit_code = 0; } -static void sync_shadow_state(struct kvm_shadow_vcpu_state *shadow_state) +static void sync_shadow_state(struct kvm_shadow_vcpu_state *shadow_state, + u32 exit_reason) { struct kvm_vcpu *shadow_vcpu = &shadow_state->shadow_vcpu; struct kvm_vcpu *host_vcpu = shadow_state->host_vcpu; + shadow_entry_exit_handler_fn ec_handler; + unsigned long host_flags; + u8 esr_ec; host_vcpu->arch.ctxt = shadow_vcpu->arch.ctxt; host_vcpu->arch.hcr_el2 = shadow_vcpu->arch.hcr_el2; host_vcpu->arch.cptr_el2 = shadow_vcpu->arch.cptr_el2; - host_vcpu->arch.fault = shadow_vcpu->arch.fault; - - host_vcpu->arch.flags = shadow_vcpu->arch.flags; - sync_vgic_state(host_vcpu, shadow_vcpu); sync_timer_state(shadow_state); + + switch (ARM_EXCEPTION_CODE(exit_reason)) { + case ARM_EXCEPTION_IRQ: + break; + case ARM_EXCEPTION_TRAP: + esr_ec = ESR_ELx_EC(kvm_vcpu_get_esr(shadow_vcpu)); + ec_handler = exit_vm_shadow_handlers[esr_ec]; + if (ec_handler) + ec_handler(host_vcpu, shadow_vcpu); + break; + case ARM_EXCEPTION_EL1_SERROR: + case ARM_EXCEPTION_IL: + break; + default: + BUG(); + } + + host_flags = READ_ONCE(host_vcpu->arch.flags) & + ~(KVM_ARM64_PENDING_EXCEPTION | KVM_ARM64_INCREMENT_PC); + WRITE_ONCE(host_vcpu->arch.flags, host_flags); + shadow_state->exit_code = exit_reason; } static void handle___pkvm_vcpu_load(struct kvm_cpu_context *host_ctxt) @@ -231,7 +314,7 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) ret = __kvm_vcpu_run(&shadow_state->shadow_vcpu); - sync_shadow_state(shadow_state); + sync_shadow_state(shadow_state, ret); } else { ret = __kvm_vcpu_run(vcpu); } -- 2.36.1.124.g0e6072fb45-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel