From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B48CC433C1 for ; Tue, 30 Mar 2021 10:14:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CDEFB61921 for ; Tue, 30 Mar 2021 10:14:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231600AbhC3KNh (ORCPT ); Tue, 30 Mar 2021 06:13:37 -0400 Received: from mail.kernel.org ([198.145.29.99]:39066 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230248AbhC3KNU (ORCPT ); Tue, 30 Mar 2021 06:13:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 67A2D61864; Tue, 30 Mar 2021 10:13:17 +0000 (UTC) Date: Tue, 30 Mar 2021 11:13:14 +0100 From: Catalin Marinas To: Steven Price Cc: Marc Zyngier , Will Deacon , James Morse , Julien Thierry , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Dave Martin , Mark Rutland , Thomas Gleixner , qemu-devel@nongnu.org, Juan Quintela , "Dr. David Alan Gilbert" , Richard Henderson , Peter Maydell , Haibo Xu , Andrew Jones Subject: Re: [PATCH v10 1/6] arm64: mte: Sync tags for pages where PTE is untagged Message-ID: <20210330101314.GC18075@arm.com> References: <20210312151902.17853-1-steven.price@arm.com> <20210312151902.17853-2-steven.price@arm.com> <20210326185653.GG5126@arm.com> <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 29, 2021 at 04:55:29PM +0100, Steven Price wrote: > On 26/03/2021 18:56, Catalin Marinas wrote: > > On Fri, Mar 12, 2021 at 03:18:57PM +0000, Steven Price wrote: > > > A KVM guest could store tags in a page even if the VMM hasn't mapped > > > the page with PROT_MTE. So when restoring pages from swap we will > > > need to check to see if there are any saved tags even if !pte_tagged(). > > > > > > However don't check pages which are !pte_valid_user() as these will > > > not have been swapped out. > > > > > > Signed-off-by: Steven Price > > > --- > > > arch/arm64/include/asm/pgtable.h | 2 +- > > > arch/arm64/kernel/mte.c | 16 ++++++++++++---- > > > 2 files changed, 13 insertions(+), 5 deletions(-) > > > > > > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h > > > index e17b96d0e4b5..84166625c989 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -312,7 +312,7 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, > > > __sync_icache_dcache(pte); > > > if (system_supports_mte() && > > > - pte_present(pte) && pte_tagged(pte) && !pte_special(pte)) > > > + pte_present(pte) && pte_valid_user(pte) && !pte_special(pte)) > > > mte_sync_tags(ptep, pte); > > > > With the EPAN patches queued in for-next/epan, pte_valid_user() > > disappeared as its semantics weren't very clear. > > Thanks for pointing that out. > > > So this relies on the set_pte_at() being done on the VMM address space. > > I wonder, if the VMM did an mprotect(PROT_NONE), can the VM still access > > it via stage 2? If yes, the pte_valid_user() test wouldn't work. We need > > something like pte_present() && addr <= user_addr_max(). > > AFAIUI the stage 2 matches the VMM's address space (for the subset that has > memslots). So mprotect(PROT_NONE) would cause the stage 2 mapping to be > invalidated and a subsequent fault would exit to the VMM to sort out. This > sort of thing is done for the lazy migration use case (i.e. pages are > fetched as the VM tries to access them). There's also the protected KVM case which IIUC wouldn't provide any mapping of the guest memory to the host (or maybe the host still thinks it's there but cannot access it without a Stage 2 fault). At least in this case it wouldn't swap pages out and it would be the responsibility of the EL2 code to clear the tags when giving pages to the guest (user_mem_abort() must not touch the page). So basically we either have a valid, accessible mapping in the VMM and we can handle the tags via set_pte_at() or we leave it to whatever is running at EL2 in the pKVM case. I don't remember whether we had a clear conclusion in the past: have we ruled out requiring the VMM to map the guest memory with PROT_MTE entirely? IIRC a potential problem was the VMM using MTE itself and having to disable it when accessing the guest memory. Another potential issue (I haven't got my head around it yet) is a race in mte_sync_tags() as we now defer the PG_mte_tagged bit setting until after the tags had been restored. Can we have the same page mapped by two ptes, each attempting to restore it from swap and one gets it first and starts modifying it? Given that we set the actual pte after setting PG_mte_tagged, it's probably alright but I think we miss some barriers. Also, if a page is not a swap one, we currently clear the tags if mapped as pte_tagged() (prior to this patch). We'd need something similar when mapping it in the guest so that we don't leak tags but to avoid any page ending up with PG_mte_tagged, I think you moved the tag clearing to user_mem_abort() in the KVM code. I presume set_pte_at() in the VMM would be called first and then set in Stage 2. > > BTW, ignoring virtualisation, can we ever bring a page in from swap on a > > PROT_NONE mapping (say fault-around)? It's not too bad if we keep the > > metadata around for when the pte becomes accessible but I suspect we > > remove it if the page is removed from swap. > > There are two stages of bringing data from swap. First is populating the > swap cache by doing the physical read from swap. The second is actually > restoring the page table entries. When is the page metadata removed? I want to make sure we don't drop it for some pte attributes. -- Catalin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C501C433DB for ; Tue, 30 Mar 2021 10:14:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B90A761959 for ; Tue, 30 Mar 2021 10:14:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B90A761959 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42300 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRBO4-0004KC-K6 for qemu-devel@archiver.kernel.org; Tue, 30 Mar 2021 06:14:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRBNF-0003ZH-6X for qemu-devel@nongnu.org; Tue, 30 Mar 2021 06:13:25 -0400 Received: from mail.kernel.org ([198.145.29.99]:50812) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRBND-0001lO-0i for qemu-devel@nongnu.org; Tue, 30 Mar 2021 06:13:24 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 67A2D61864; Tue, 30 Mar 2021 10:13:17 +0000 (UTC) Date: Tue, 30 Mar 2021 11:13:14 +0100 From: Catalin Marinas To: Steven Price Subject: Re: [PATCH v10 1/6] arm64: mte: Sync tags for pages where PTE is untagged Message-ID: <20210330101314.GC18075@arm.com> References: <20210312151902.17853-1-steven.price@arm.com> <20210312151902.17853-2-steven.price@arm.com> <20210326185653.GG5126@arm.com> <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> User-Agent: Mutt/1.10.1 (2018-07-13) Received-SPF: pass client-ip=198.145.29.99; envelope-from=cmarinas@kernel.org; helo=mail.kernel.org X-Spam_score_int: -66 X-Spam_score: -6.7 X-Spam_bar: ------ X-Spam_report: (-6.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Maydell , "Dr. David Alan Gilbert" , Andrew Jones , Haibo Xu , Suzuki K Poulose , qemu-devel@nongnu.org, Marc Zyngier , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Dave Martin , James Morse , linux-arm-kernel@lists.infradead.org, Thomas Gleixner , Will Deacon , kvmarm@lists.cs.columbia.edu, Julien Thierry Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Mon, Mar 29, 2021 at 04:55:29PM +0100, Steven Price wrote: > On 26/03/2021 18:56, Catalin Marinas wrote: > > On Fri, Mar 12, 2021 at 03:18:57PM +0000, Steven Price wrote: > > > A KVM guest could store tags in a page even if the VMM hasn't mapped > > > the page with PROT_MTE. So when restoring pages from swap we will > > > need to check to see if there are any saved tags even if !pte_tagged(). > > > > > > However don't check pages which are !pte_valid_user() as these will > > > not have been swapped out. > > > > > > Signed-off-by: Steven Price > > > --- > > > arch/arm64/include/asm/pgtable.h | 2 +- > > > arch/arm64/kernel/mte.c | 16 ++++++++++++---- > > > 2 files changed, 13 insertions(+), 5 deletions(-) > > > > > > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h > > > index e17b96d0e4b5..84166625c989 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -312,7 +312,7 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, > > > __sync_icache_dcache(pte); > > > if (system_supports_mte() && > > > - pte_present(pte) && pte_tagged(pte) && !pte_special(pte)) > > > + pte_present(pte) && pte_valid_user(pte) && !pte_special(pte)) > > > mte_sync_tags(ptep, pte); > > > > With the EPAN patches queued in for-next/epan, pte_valid_user() > > disappeared as its semantics weren't very clear. > > Thanks for pointing that out. > > > So this relies on the set_pte_at() being done on the VMM address space. > > I wonder, if the VMM did an mprotect(PROT_NONE), can the VM still access > > it via stage 2? If yes, the pte_valid_user() test wouldn't work. We need > > something like pte_present() && addr <= user_addr_max(). > > AFAIUI the stage 2 matches the VMM's address space (for the subset that has > memslots). So mprotect(PROT_NONE) would cause the stage 2 mapping to be > invalidated and a subsequent fault would exit to the VMM to sort out. This > sort of thing is done for the lazy migration use case (i.e. pages are > fetched as the VM tries to access them). There's also the protected KVM case which IIUC wouldn't provide any mapping of the guest memory to the host (or maybe the host still thinks it's there but cannot access it without a Stage 2 fault). At least in this case it wouldn't swap pages out and it would be the responsibility of the EL2 code to clear the tags when giving pages to the guest (user_mem_abort() must not touch the page). So basically we either have a valid, accessible mapping in the VMM and we can handle the tags via set_pte_at() or we leave it to whatever is running at EL2 in the pKVM case. I don't remember whether we had a clear conclusion in the past: have we ruled out requiring the VMM to map the guest memory with PROT_MTE entirely? IIRC a potential problem was the VMM using MTE itself and having to disable it when accessing the guest memory. Another potential issue (I haven't got my head around it yet) is a race in mte_sync_tags() as we now defer the PG_mte_tagged bit setting until after the tags had been restored. Can we have the same page mapped by two ptes, each attempting to restore it from swap and one gets it first and starts modifying it? Given that we set the actual pte after setting PG_mte_tagged, it's probably alright but I think we miss some barriers. Also, if a page is not a swap one, we currently clear the tags if mapped as pte_tagged() (prior to this patch). We'd need something similar when mapping it in the guest so that we don't leak tags but to avoid any page ending up with PG_mte_tagged, I think you moved the tag clearing to user_mem_abort() in the KVM code. I presume set_pte_at() in the VMM would be called first and then set in Stage 2. > > BTW, ignoring virtualisation, can we ever bring a page in from swap on a > > PROT_NONE mapping (say fault-around)? It's not too bad if we keep the > > metadata around for when the pte becomes accessible but I suspect we > > remove it if the page is removed from swap. > > There are two stages of bringing data from swap. First is populating the > swap cache by doing the physical read from swap. The second is actually > restoring the page table entries. When is the page metadata removed? I want to make sure we don't drop it for some pte attributes. -- Catalin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB85CC433DB for ; Tue, 30 Mar 2021 10:13:26 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 34E3A61864 for ; Tue, 30 Mar 2021 10:13:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34E3A61864 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id B323E4B299; Tue, 30 Mar 2021 06:13:25 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PYQGrEa6-C53; Tue, 30 Mar 2021 06:13:24 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 818064B2CC; Tue, 30 Mar 2021 06:13:24 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 072994B299 for ; Tue, 30 Mar 2021 06:13:23 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vGpi79nAIVkt for ; Tue, 30 Mar 2021 06:13:21 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 9C8ED4B1B4 for ; Tue, 30 Mar 2021 06:13:21 -0400 (EDT) Received: by mail.kernel.org (Postfix) with ESMTPSA id 67A2D61864; Tue, 30 Mar 2021 10:13:17 +0000 (UTC) Date: Tue, 30 Mar 2021 11:13:14 +0100 From: Catalin Marinas To: Steven Price Subject: Re: [PATCH v10 1/6] arm64: mte: Sync tags for pages where PTE is untagged Message-ID: <20210330101314.GC18075@arm.com> References: <20210312151902.17853-1-steven.price@arm.com> <20210312151902.17853-2-steven.price@arm.com> <20210326185653.GG5126@arm.com> <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> User-Agent: Mutt/1.10.1 (2018-07-13) Cc: "Dr. David Alan Gilbert" , qemu-devel@nongnu.org, Marc Zyngier , Juan Quintela , Richard Henderson , linux-kernel@vger.kernel.org, Dave Martin , linux-arm-kernel@lists.infradead.org, Thomas Gleixner , Will Deacon , kvmarm@lists.cs.columbia.edu X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Mon, Mar 29, 2021 at 04:55:29PM +0100, Steven Price wrote: > On 26/03/2021 18:56, Catalin Marinas wrote: > > On Fri, Mar 12, 2021 at 03:18:57PM +0000, Steven Price wrote: > > > A KVM guest could store tags in a page even if the VMM hasn't mapped > > > the page with PROT_MTE. So when restoring pages from swap we will > > > need to check to see if there are any saved tags even if !pte_tagged(). > > > > > > However don't check pages which are !pte_valid_user() as these will > > > not have been swapped out. > > > > > > Signed-off-by: Steven Price > > > --- > > > arch/arm64/include/asm/pgtable.h | 2 +- > > > arch/arm64/kernel/mte.c | 16 ++++++++++++---- > > > 2 files changed, 13 insertions(+), 5 deletions(-) > > > > > > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h > > > index e17b96d0e4b5..84166625c989 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -312,7 +312,7 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, > > > __sync_icache_dcache(pte); > > > if (system_supports_mte() && > > > - pte_present(pte) && pte_tagged(pte) && !pte_special(pte)) > > > + pte_present(pte) && pte_valid_user(pte) && !pte_special(pte)) > > > mte_sync_tags(ptep, pte); > > > > With the EPAN patches queued in for-next/epan, pte_valid_user() > > disappeared as its semantics weren't very clear. > > Thanks for pointing that out. > > > So this relies on the set_pte_at() being done on the VMM address space. > > I wonder, if the VMM did an mprotect(PROT_NONE), can the VM still access > > it via stage 2? If yes, the pte_valid_user() test wouldn't work. We need > > something like pte_present() && addr <= user_addr_max(). > > AFAIUI the stage 2 matches the VMM's address space (for the subset that has > memslots). So mprotect(PROT_NONE) would cause the stage 2 mapping to be > invalidated and a subsequent fault would exit to the VMM to sort out. This > sort of thing is done for the lazy migration use case (i.e. pages are > fetched as the VM tries to access them). There's also the protected KVM case which IIUC wouldn't provide any mapping of the guest memory to the host (or maybe the host still thinks it's there but cannot access it without a Stage 2 fault). At least in this case it wouldn't swap pages out and it would be the responsibility of the EL2 code to clear the tags when giving pages to the guest (user_mem_abort() must not touch the page). So basically we either have a valid, accessible mapping in the VMM and we can handle the tags via set_pte_at() or we leave it to whatever is running at EL2 in the pKVM case. I don't remember whether we had a clear conclusion in the past: have we ruled out requiring the VMM to map the guest memory with PROT_MTE entirely? IIRC a potential problem was the VMM using MTE itself and having to disable it when accessing the guest memory. Another potential issue (I haven't got my head around it yet) is a race in mte_sync_tags() as we now defer the PG_mte_tagged bit setting until after the tags had been restored. Can we have the same page mapped by two ptes, each attempting to restore it from swap and one gets it first and starts modifying it? Given that we set the actual pte after setting PG_mte_tagged, it's probably alright but I think we miss some barriers. Also, if a page is not a swap one, we currently clear the tags if mapped as pte_tagged() (prior to this patch). We'd need something similar when mapping it in the guest so that we don't leak tags but to avoid any page ending up with PG_mte_tagged, I think you moved the tag clearing to user_mem_abort() in the KVM code. I presume set_pte_at() in the VMM would be called first and then set in Stage 2. > > BTW, ignoring virtualisation, can we ever bring a page in from swap on a > > PROT_NONE mapping (say fault-around)? It's not too bad if we keep the > > metadata around for when the pte becomes accessible but I suspect we > > remove it if the page is removed from swap. > > There are two stages of bringing data from swap. First is populating the > swap cache by doing the physical read from swap. The second is actually > restoring the page table entries. When is the page metadata removed? I want to make sure we don't drop it for some pte attributes. -- Catalin _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C781CC433C1 for ; Tue, 30 Mar 2021 10:15:09 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3196761959 for ; Tue, 30 Mar 2021 10:15:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3196761959 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=65B9ZRPx37HCMT/kGD6N4BdRd0mhlfWWP7v6/BC+oZo=; b=eMEg8dCDdZbHMjfvR4d/+TXbI /egsdvYlzbORbn0Dc8QAZHP5JQpckeWUh4rZ861i9CCuuTKRv/LUJ66l3y2nth8pB+uJsj5sdvOYp R7Ol+Upb2tPN+QYJHha4YWP/gdLO+stR7RfaJPnJqFFN3Jij7yQtBX2s5WwaFeS1IJ1cV0WevShlW /0+YcBUJ+0LZ/d6Vqu0FTUUja6fIZC2IMdg5A+6+BkRifGws/rgn8uoqrPSAKAdcrOINrIkwmjcOg lmNF50rWb3LVf8WouiSUIGRa8pU9aXETv2NdRRacequ1IeLQiA7nLgMPkUo/RAJ/UFRSdjgSqB37x uE1lDCOnA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lRBNH-003MsA-Ve; Tue, 30 Mar 2021 10:13:28 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lRBNC-003Mqb-69 for linux-arm-kernel@lists.infradead.org; Tue, 30 Mar 2021 10:13:24 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 67A2D61864; Tue, 30 Mar 2021 10:13:17 +0000 (UTC) Date: Tue, 30 Mar 2021 11:13:14 +0100 From: Catalin Marinas To: Steven Price Cc: Marc Zyngier , Will Deacon , James Morse , Julien Thierry , Suzuki K Poulose , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Dave Martin , Mark Rutland , Thomas Gleixner , qemu-devel@nongnu.org, Juan Quintela , "Dr. David Alan Gilbert" , Richard Henderson , Peter Maydell , Haibo Xu , Andrew Jones Subject: Re: [PATCH v10 1/6] arm64: mte: Sync tags for pages where PTE is untagged Message-ID: <20210330101314.GC18075@arm.com> References: <20210312151902.17853-1-steven.price@arm.com> <20210312151902.17853-2-steven.price@arm.com> <20210326185653.GG5126@arm.com> <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <21842e4d-7935-077c-3d6f-fced89b7f2bb@arm.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210330_111322_663611_ED4191DA X-CRM114-Status: GOOD ( 40.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Mar 29, 2021 at 04:55:29PM +0100, Steven Price wrote: > On 26/03/2021 18:56, Catalin Marinas wrote: > > On Fri, Mar 12, 2021 at 03:18:57PM +0000, Steven Price wrote: > > > A KVM guest could store tags in a page even if the VMM hasn't mapped > > > the page with PROT_MTE. So when restoring pages from swap we will > > > need to check to see if there are any saved tags even if !pte_tagged(). > > > > > > However don't check pages which are !pte_valid_user() as these will > > > not have been swapped out. > > > > > > Signed-off-by: Steven Price > > > --- > > > arch/arm64/include/asm/pgtable.h | 2 +- > > > arch/arm64/kernel/mte.c | 16 ++++++++++++---- > > > 2 files changed, 13 insertions(+), 5 deletions(-) > > > > > > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h > > > index e17b96d0e4b5..84166625c989 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -312,7 +312,7 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, > > > __sync_icache_dcache(pte); > > > if (system_supports_mte() && > > > - pte_present(pte) && pte_tagged(pte) && !pte_special(pte)) > > > + pte_present(pte) && pte_valid_user(pte) && !pte_special(pte)) > > > mte_sync_tags(ptep, pte); > > > > With the EPAN patches queued in for-next/epan, pte_valid_user() > > disappeared as its semantics weren't very clear. > > Thanks for pointing that out. > > > So this relies on the set_pte_at() being done on the VMM address space. > > I wonder, if the VMM did an mprotect(PROT_NONE), can the VM still access > > it via stage 2? If yes, the pte_valid_user() test wouldn't work. We need > > something like pte_present() && addr <= user_addr_max(). > > AFAIUI the stage 2 matches the VMM's address space (for the subset that has > memslots). So mprotect(PROT_NONE) would cause the stage 2 mapping to be > invalidated and a subsequent fault would exit to the VMM to sort out. This > sort of thing is done for the lazy migration use case (i.e. pages are > fetched as the VM tries to access them). There's also the protected KVM case which IIUC wouldn't provide any mapping of the guest memory to the host (or maybe the host still thinks it's there but cannot access it without a Stage 2 fault). At least in this case it wouldn't swap pages out and it would be the responsibility of the EL2 code to clear the tags when giving pages to the guest (user_mem_abort() must not touch the page). So basically we either have a valid, accessible mapping in the VMM and we can handle the tags via set_pte_at() or we leave it to whatever is running at EL2 in the pKVM case. I don't remember whether we had a clear conclusion in the past: have we ruled out requiring the VMM to map the guest memory with PROT_MTE entirely? IIRC a potential problem was the VMM using MTE itself and having to disable it when accessing the guest memory. Another potential issue (I haven't got my head around it yet) is a race in mte_sync_tags() as we now defer the PG_mte_tagged bit setting until after the tags had been restored. Can we have the same page mapped by two ptes, each attempting to restore it from swap and one gets it first and starts modifying it? Given that we set the actual pte after setting PG_mte_tagged, it's probably alright but I think we miss some barriers. Also, if a page is not a swap one, we currently clear the tags if mapped as pte_tagged() (prior to this patch). We'd need something similar when mapping it in the guest so that we don't leak tags but to avoid any page ending up with PG_mte_tagged, I think you moved the tag clearing to user_mem_abort() in the KVM code. I presume set_pte_at() in the VMM would be called first and then set in Stage 2. > > BTW, ignoring virtualisation, can we ever bring a page in from swap on a > > PROT_NONE mapping (say fault-around)? It's not too bad if we keep the > > metadata around for when the pte becomes accessible but I suspect we > > remove it if the page is removed from swap. > > There are two stages of bringing data from swap. First is populating the > swap cache by doing the physical read from swap. The second is actually > restoring the page table entries. When is the page metadata removed? I want to make sure we don't drop it for some pte attributes. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel