From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 82FD4C6FA82 for ; Sat, 10 Sep 2022 04:15:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VOkd7aV9If0EJpIEwZzVP//c++ylso3Okul+DITL3xs=; b=emw61uY8x/G9/S nfSfLwXJAm1oGBa9EQp2oHy3inv7mXv3+GRslk3aMWazqUyjASm/Rjnr4VRzpZNFLxG0KPC8flBz6 IzJbjxz9SLd1ROFelmZw+I4yyrWLRrWXIPgak5PKSvRiv2OmZ9k2sNbWdMA+2hzYmO31O6oiHFIu7 GWLIWjt8UpfQ4T9xAgnD0Dy3hdh4xcNVknvRDaM4dBx9cBuWQ3wdWTMXLSYZoZGeCfZlBo8e/aqBl y16y8FYSDrhRL7jn82X5UXUJg/XufyfK/0dolbvWCp3/sEg42hcos8BxrJv68KaeTMMj5HEkOuzBt TOX/y9/nJKL2UlP3TIzg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oWrsD-005tRB-CZ; Sat, 10 Sep 2022 04:13:41 +0000 Received: from mail-vs1-xe2d.google.com ([2607:f8b0:4864:20::e2d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oWrs6-005tL4-AU for linux-arm-kernel@lists.infradead.org; Sat, 10 Sep 2022 04:13:36 +0000 Received: by mail-vs1-xe2d.google.com with SMTP id a129so3687250vsc.0 for ; Fri, 09 Sep 2022 21:13:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=bT+lvXkm6Qe6+nTGGZtBv81BtvgBawrDwZvucM+h94M=; b=Yys0KwKHTO+vzTZHw3AkKTcwnUZvMyJM+/WUxQmJjsT87vwqfEWFuN3IyPDJ8gPx3a NqQi0USCw1ZfeJkzhhFJqaxYFKW8KP5mvIMwBxp3xs0Ax/mlVLTnW74LrtYjlISyP6Nu Qmdv71WFERQ9fnzJbq4556HnCJkVvBESHXxXk9W7J+iVMSukzqttWwGjnwoEpQ8ThYAt WGwU0sCuMBPWjPYQ8EL8Ldn12lXjDxgA7dQ4g8pmMM8zi8rBiCk5PAS/RnTseO3ggzMD ICjcHCnGCHRLdFBfhynEOmkL76h+RA0CoSV3nSLQ88rdhD6P/GMybofb2ADNE6Ehkpmz V/GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=bT+lvXkm6Qe6+nTGGZtBv81BtvgBawrDwZvucM+h94M=; b=oEATx6ujwyZrRAwPzrqCmI420RfpxCbpLf9qiOaaoyONHX8B+QeyrUy/iuHi8asRMq ELthaXZNnlyO9PQ9Q3uFkHE8FP4A/8ejJRW8u2kMDxrvRjcHLpyE+Nzb9eEYAO5rRDyq HTWNjDlM1nkJy+t95YRFPoJCVDuwCZi5IdwiP3uL7wkF7HdBpco1PaU/OYS5kYgszI80 2FLYs7At9r5aDfEfKTaO59TTNdk1Vhu8Y+xR42LajslGiiFCYar0NBf7/EVfsWIOMdV9 DwogL5LAOagLNiDkwmeU2CAEkbHYcjMOpfyVAhLkWi0q7QsMXzXzRBtkfGPKwXzht66x uPcg== X-Gm-Message-State: ACgBeo2oEb8SOtB7u3zRQMAUz/2yp3pHTmZtnzGNaI+0Ku3eyaLJdqqM r1Ldz3Uc5do4oHBz4K9kcwPOv1tgX8pVY1/8CfXmeA== X-Google-Smtp-Source: AA6agR5RrXixyzKCzUB/EJrVm8Zhg8tpR3LDBgWYAO15rG9c2TCzDy3dx942O9xqMNTbp/kZRLU6ckHffxtUmgIfS6w= X-Received: by 2002:a67:c00b:0:b0:390:8e1f:594a with SMTP id v11-20020a67c00b000000b003908e1f594amr6206352vsi.80.1662783193547; Fri, 09 Sep 2022 21:13:13 -0700 (PDT) MIME-Version: 1.0 References: <20220909044636.1997755-1-reijiw@google.com> <20220909044636.1997755-2-reijiw@google.com> <87bkrora8b.wl-maz@kernel.org> In-Reply-To: <87bkrora8b.wl-maz@kernel.org> From: Reiji Watanabe Date: Fri, 9 Sep 2022 21:12:57 -0700 Message-ID: Subject: Re: [PATCH 1/3] KVM: arm64: Don't set PSTATE.SS when Software Step state is Active-pending To: Marc Zyngier Cc: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Linux ARM , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220909_211334_395059_92955D82 X-CRM114-Status: GOOD ( 43.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Marc, On Fri, Sep 9, 2022 at 5:41 AM Marc Zyngier wrote: > > Hi Reiji, > > On Fri, 09 Sep 2022 05:46:34 +0100, > Reiji Watanabe wrote: > > > > Currently, PSTATE.SS is set on every guest entry if single-step is > > enabled for the vCPU by userspace. However, it could cause extra > > single-step execution without returning to userspace, which shouldn't > > be performed, if the Software Step state at the last guest exit was > > Active-pending (i.e. the last exit was not triggered by Software Step > > exception, but by an asynchronous exception after the single-step > > execution is performed). > > For my own enlightenment, could you describe a sequence of events that > leads to this issue? Here is an example of the sequences. [Usersace] | - ioctl(SET_GUEST_DEBUG) | - ioctl(KVM_RUN) (vCPU PC==X) v [KVM] | - *vcpu_cpsr(vcpu) |= SPSR_SS; | - mdscr |= DBG_MDSCR_SS; | - VM Entry v [Guest] vCPU PC==X | - Execute an instruction at PC==X. | PC is updated with X+4, and PSTATE.SS is cleared. | | !! Asynchronous exception !! v [KVM] vCPU PC==X+4 | - The kernel processes the async exception. | - handle_exit() returns 1 (don't return to userspace) | - *vcpu_cpsr(vcpu) |= SPSR_SS; | - mdscr |= DBG_MDSCR_SS; | - VM Entry v [Guest] vCPU PC==X+4 | - Execute an instruction at PC==X+4. | PC is updated with X+8, PSTATE.SS is cleared. | | !! Software Step Exception !! v [KVM] vCPU PC==X+8 | - run->exit_reason = KVM_EXIT_DEBUG; | - Return to userspace v [Userspace] - Userspace sees PC==X+8 (Userspace didn't see PC==X+4). > > Fix this by not setting PSTATE.SS on guest entry if the Software > > Step state at the last exit was Active-pending. > > > > Fixes: 337b99bf7edf ("KVM: arm64: guest debug, add support for single-step") > > Signed-off-by: Reiji Watanabe > > --- > > arch/arm64/include/asm/kvm_host.h | 3 +++ > > arch/arm64/kvm/debug.c | 19 ++++++++++++++++++- > > arch/arm64/kvm/guest.c | 1 + > > arch/arm64/kvm/handle_exit.c | 2 ++ > > 4 files changed, 24 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > > index e9c9388ccc02..4cf6eef02565 100644 > > --- a/arch/arm64/include/asm/kvm_host.h > > +++ b/arch/arm64/include/asm/kvm_host.h > > @@ -535,6 +535,9 @@ struct kvm_vcpu_arch { > > #define IN_WFIT __vcpu_single_flag(sflags, BIT(3)) > > /* vcpu system registers loaded on physical CPU */ > > #define SYSREGS_ON_CPU __vcpu_single_flag(sflags, BIT(4)) > > +/* Software step state is Active-pending */ > > +#define DBG_SS_ACTIVE_PENDING __vcpu_single_flag(sflags, BIT(5)) > > + > > > > /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ > > #define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ > > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c > > index 0b28d7db7c76..125cfb94b4ad 100644 > > --- a/arch/arm64/kvm/debug.c > > +++ b/arch/arm64/kvm/debug.c > > @@ -188,7 +188,16 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu) > > * debugging the system. > > */ > > if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) { > > - *vcpu_cpsr(vcpu) |= DBG_SPSR_SS; > > + /* > > + * If the software step state at the last guest exit > > + * was Active-pending, we don't set DBG_SPSR_SS so > > + * that the state is maintained (to not run another > > + * single-step until the pending Software Step > > + * exception is taken). > > + */ > > + if (!vcpu_get_flag(vcpu, DBG_SS_ACTIVE_PENDING)) > > + *vcpu_cpsr(vcpu) |= DBG_SPSR_SS; > > I guess my confusion stems from my (probably wrong) interpretation if > the SS state is A+P, there is no harm in making it pending again > (setting the SS bit in PSTATE). Setting the SS bit in PSTATE (with MDSCR_EL1.SS=1) makes the state Active-not-pending (not Active-pending) if my interpretation of the spec is correct. > > + > > mdscr = vcpu_read_sys_reg(vcpu, MDSCR_EL1); > > mdscr |= DBG_MDSCR_SS; > > But it looks like the *pending* state is actually stored in MDSCR > instead? The spec only mentions this for the A+P state, so this is > quite likely a bug indeed. MDSCR_EL1.SS is set even in Active-not-pending state (not just Active-pending state). (Or do you mean the state is stored in some other field in MDSCR ??) > Now, where does the asynchronous exception comes into play? I found > this intriguing remark in the ARM ARM: > > > The Software Step exception has higher priority than all other types > of synchronous exception. However, the prioritization of this > exception with respect to any unmasked pending asynchronous exception > is not defined by the architecture. > > > Is this what you were referring to in the commit message? I think you > need to spell it out for us, as I don't fully understand what you are > fixing nor do I understand the gory details of single-stepping... Yes, that is what I was referring to. In "Figure D2-3 Software step state machine" in Arm ARM (DDI 0487I.a), since KVM currently sets PSTATE.SS to 1 on every Guest Entry (when single-step is enabled by userspace), KVM always has the CPU in "Inactive" (the second inactive state from the top) transition to "Active-not-pending" on the Guest Entry. With this patch, KVM have the CPU transitions to "Active-pending" if the state before "Inactive" was "Active-pending", which indicates the step completed but Software Step exception is not taken yet, so that Software Step exception is taken before further single-step is executed. I'm sorry for the unclear explanation, and I hope my comments clarify the problem I'm trying to fix. Thank you, Reiji _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel