From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AA5AC04A68 for ; Wed, 27 Jul 2022 11:56:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 869C94C3AE; Wed, 27 Jul 2022 07:56:46 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gbNf2I4Xycc2; Wed, 27 Jul 2022 07:56:45 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 143C04C38A; Wed, 27 Jul 2022 07:56:45 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E513D4C38A for ; Wed, 27 Jul 2022 07:56:43 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 33VLZczPhVK2 for ; Wed, 27 Jul 2022 07:56:42 -0400 (EDT) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 602C04C371 for ; Wed, 27 Jul 2022 07:56:42 -0400 (EDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A3BED6E; Wed, 27 Jul 2022 04:56:42 -0700 (PDT) Received: from monolith.localdoman (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A15EF3F73B; Wed, 27 Jul 2022 04:56:40 -0700 (PDT) Date: Wed, 27 Jul 2022 12:57:16 +0100 From: Alexandru Elisei To: Marc Zyngier Subject: Re: KVM/arm64: SPE: Translate VA to IPA on a stage 2 fault instead of pinning VM memory Message-ID: References: <20220419141012.GB6143@willie-the-truck> <04dea801e298374fb783bf0760b15241@kernel.org> <2d1f87576ce17b8c72bfac522e2aa101@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <2d1f87576ce17b8c72bfac522e2aa101@kernel.org> Cc: Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hi, On Wed, Jul 27, 2022 at 12:08:11PM +0100, Marc Zyngier wrote: > On 2022-07-27 11:44, Alexandru Elisei wrote: > > On Wed, Jul 27, 2022 at 11:29:03AM +0100, Marc Zyngier wrote: > > > On 2022-07-27 11:19, Alexandru Elisei wrote: > > > > Hi Oliver, > > > > > > > > Thank you for the help, replies below. > > > > > > > > On Tue, Jul 26, 2022 at 10:51:21AM -0700, Oliver Upton wrote: > > > > > Hi Alex, > > > > > > > > > > On Mon, Jul 25, 2022 at 11:06:24AM +0100, Alexandru Elisei wrote: > > > > > > > > > > [...] > > > > > > > > > > > > A funkier approach might be to defer pinning of the buffer until the SPE is > > > > > > > enabled and avoid pinning all of VM memory that way, although I can't > > > > > > > immediately tell how flexible the architecture is in allowing you to cache > > > > > > > the base/limit values. > > > > > > > > > > > > I was investigating this approach, and Mark raised a concern that I think > > > > > > might be a showstopper. > > > > > > > > > > > > Let's consider this scenario: > > > > > > > > > > > > Initial conditions: guest at EL1, profiling disabled (PMBLIMITR_EL1.E = 0, > > > > > > PMBSR_EL1.S = 0, PMSCR_EL1.{E0SPE,E1SPE} = {0,0}). > > > > > > > > > > > > 1. Guest programs the buffer and enables it (PMBLIMITR_EL1.E = 1). > > > > > > 2. Guest programs SPE to enable profiling at **EL0** > > > > > > (PMSCR_EL1.{E0SPE,E1SPE} = {1,0}). > > > > > > 3. Guest changes the translation table entries for the buffer. The > > > > > > architecture allows this. > > > > > > 4. Guest does an ERET to EL0, thus enabling profiling. > > > > > > > > > > > > Since KVM cannot trap the ERET to EL0, it will be impossible for KVM to pin > > > > > > the buffer at stage 2 when profiling gets enabled at EL0. > > > > > > > > > > Not saying we necessarily should, but this is possible with FGT no? > > > > > > > > It doesn't look to me like FEAT_FGT offers any knobs to trap ERET from > > > > EL1. > > > > > > See HFGITR.ERET. > > > > Ah, so that's the register, thanks! > > > > I stil am not sure that having FEAT_SPE, an Armv8.3 extension, depend on > > FEAT_FGT, an Armv8.6 extension, is the best idea. Do you know of any > > machines > > that have FEAT_SPE and FEAT_FGT? > > None. Both are pretty niche, and the combination is nowhere > to be seen at the moment. That was also my impression. > > > On the plus side, KVM could enable the trap only in the case above, and > > disable > > it after the ERET is trapped, so it should be relatively cheap to use. > > This feels pretty horrible. Nothing says *when* will EL1 > alter the PTs. It could take tons of EL1->EL1 exceptions > before returning to EL0. And the change could happen after > an EL1->EL0->EL1 transition. At which point do you stop? ERET trapping is enabled When PMBLIMITR_EL1.E = 1, PMSCR_EL1.{E0SPE,E1SPE} = {1,0}. The first guest ERET from EL1 to EL0 enables profiling, at which point the buffer is pinned and ERET trapping is disabled. Guest messing with the translation tables while profiling is enabled is the guest's problem because that's not permitted by the architecture. Any stage 2 dabt taken when the buffer is pinned would be injected back into the guest as an SPE external abort (or something equivalent). Stage 1 dabts are entirely the guest's problem to solve and would be injected back regardless of the status of the buffer. Yes, I agree, there could be a lot of ERETs from EL1 to EL1 before the ERET to EL0; those ERETs would be uselessly trapped. The above is a moot point anyway, because I believe we both agree that having SPE emulation depend on FEAT_FGT is best to be avoided. Thanks, Alex > > If you want to rely on ERET for that, you need to trap > ERET all the time, because all ERETs to EL0 will be > suspect. And doing that to handle such a corner case feels > pretty horrible. > > M. > -- > Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 50A86C04A68 for ; Wed, 27 Jul 2022 11:58:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nrz/Fb2WvlxTAzc6scQ/efYorIjG5VpsL0vQKjcDdFg=; b=v2MiZqRNDo/gpq q1XQZuVG/8mCZjQD8Qje2qOSJYA0yLTuNaIeKlYIzEmpkXlP1+t/rkiXoZT6rlO1cqAaOI4J1uv/x LiZkAKFJ4V1dunIKG+I+cXzWmVkyrssF8fxL+YlI8kv5PCzzvTUqA4N/TyueIGVVZMyoDWyYmEhIe KlQm8t+mKT6czm3PTEd4Unb2kq2becg5qU/pKWn2327EMZzqkPoB0PU2B7xctivDvNMnj6sHiftH3 bV9KDwL+6cK+embU0QIqZR8Ij1ohBsvUj8+ihtrwUe8hER/xPO151+XA+L9uElcoxpRdZEZAvG71k ZeGEBD5/y7aw4vczwiZA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oGfef-00DCZl-RD; Wed, 27 Jul 2022 11:56:45 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oGfed-00DCXs-3C for linux-arm-kernel@lists.infradead.org; Wed, 27 Jul 2022 11:56:44 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A3BED6E; Wed, 27 Jul 2022 04:56:42 -0700 (PDT) Received: from monolith.localdoman (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A15EF3F73B; Wed, 27 Jul 2022 04:56:40 -0700 (PDT) Date: Wed, 27 Jul 2022 12:57:16 +0100 From: Alexandru Elisei To: Marc Zyngier Cc: linux-arm-kernel@lists.infradead.org, Will Deacon , kvmarm@lists.cs.columbia.edu Subject: Re: KVM/arm64: SPE: Translate VA to IPA on a stage 2 fault instead of pinning VM memory Message-ID: References: <20220419141012.GB6143@willie-the-truck> <04dea801e298374fb783bf0760b15241@kernel.org> <2d1f87576ce17b8c72bfac522e2aa101@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <2d1f87576ce17b8c72bfac522e2aa101@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220727_045643_249887_DBD88BEC X-CRM114-Status: GOOD ( 35.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, On Wed, Jul 27, 2022 at 12:08:11PM +0100, Marc Zyngier wrote: > On 2022-07-27 11:44, Alexandru Elisei wrote: > > On Wed, Jul 27, 2022 at 11:29:03AM +0100, Marc Zyngier wrote: > > > On 2022-07-27 11:19, Alexandru Elisei wrote: > > > > Hi Oliver, > > > > > > > > Thank you for the help, replies below. > > > > > > > > On Tue, Jul 26, 2022 at 10:51:21AM -0700, Oliver Upton wrote: > > > > > Hi Alex, > > > > > > > > > > On Mon, Jul 25, 2022 at 11:06:24AM +0100, Alexandru Elisei wrote: > > > > > > > > > > [...] > > > > > > > > > > > > A funkier approach might be to defer pinning of the buffer until the SPE is > > > > > > > enabled and avoid pinning all of VM memory that way, although I can't > > > > > > > immediately tell how flexible the architecture is in allowing you to cache > > > > > > > the base/limit values. > > > > > > > > > > > > I was investigating this approach, and Mark raised a concern that I think > > > > > > might be a showstopper. > > > > > > > > > > > > Let's consider this scenario: > > > > > > > > > > > > Initial conditions: guest at EL1, profiling disabled (PMBLIMITR_EL1.E = 0, > > > > > > PMBSR_EL1.S = 0, PMSCR_EL1.{E0SPE,E1SPE} = {0,0}). > > > > > > > > > > > > 1. Guest programs the buffer and enables it (PMBLIMITR_EL1.E = 1). > > > > > > 2. Guest programs SPE to enable profiling at **EL0** > > > > > > (PMSCR_EL1.{E0SPE,E1SPE} = {1,0}). > > > > > > 3. Guest changes the translation table entries for the buffer. The > > > > > > architecture allows this. > > > > > > 4. Guest does an ERET to EL0, thus enabling profiling. > > > > > > > > > > > > Since KVM cannot trap the ERET to EL0, it will be impossible for KVM to pin > > > > > > the buffer at stage 2 when profiling gets enabled at EL0. > > > > > > > > > > Not saying we necessarily should, but this is possible with FGT no? > > > > > > > > It doesn't look to me like FEAT_FGT offers any knobs to trap ERET from > > > > EL1. > > > > > > See HFGITR.ERET. > > > > Ah, so that's the register, thanks! > > > > I stil am not sure that having FEAT_SPE, an Armv8.3 extension, depend on > > FEAT_FGT, an Armv8.6 extension, is the best idea. Do you know of any > > machines > > that have FEAT_SPE and FEAT_FGT? > > None. Both are pretty niche, and the combination is nowhere > to be seen at the moment. That was also my impression. > > > On the plus side, KVM could enable the trap only in the case above, and > > disable > > it after the ERET is trapped, so it should be relatively cheap to use. > > This feels pretty horrible. Nothing says *when* will EL1 > alter the PTs. It could take tons of EL1->EL1 exceptions > before returning to EL0. And the change could happen after > an EL1->EL0->EL1 transition. At which point do you stop? ERET trapping is enabled When PMBLIMITR_EL1.E = 1, PMSCR_EL1.{E0SPE,E1SPE} = {1,0}. The first guest ERET from EL1 to EL0 enables profiling, at which point the buffer is pinned and ERET trapping is disabled. Guest messing with the translation tables while profiling is enabled is the guest's problem because that's not permitted by the architecture. Any stage 2 dabt taken when the buffer is pinned would be injected back into the guest as an SPE external abort (or something equivalent). Stage 1 dabts are entirely the guest's problem to solve and would be injected back regardless of the status of the buffer. Yes, I agree, there could be a lot of ERETs from EL1 to EL1 before the ERET to EL0; those ERETs would be uselessly trapped. The above is a moot point anyway, because I believe we both agree that having SPE emulation depend on FEAT_FGT is best to be avoided. Thanks, Alex > > If you want to rely on ERET for that, you need to trap > ERET all the time, because all ERETs to EL0 will be > suspect. And doing that to handle such a corner case feels > pretty horrible. > > M. > -- > Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel