From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3D81C433E2 for ; Fri, 15 May 2020 17:16:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C2764206C0 for ; Fri, 15 May 2020 17:16:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2764206C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B7138E000D; Fri, 15 May 2020 13:16:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23F5D8E0001; Fri, 15 May 2020 13:16:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0979C8E000D; Fri, 15 May 2020 13:16:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0022.hostedemail.com [216.40.44.22]) by kanga.kvack.org (Postfix) with ESMTP id E22ED8E0001 for ; Fri, 15 May 2020 13:16:39 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9C4692489E for ; Fri, 15 May 2020 17:16:39 +0000 (UTC) X-FDA: 76819607718.21.flock21_9168e88559156 X-HE-Tag: flock21_9168e88559156 X-Filterd-Recvd-Size: 4609 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Fri, 15 May 2020 17:16:39 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 735701063; Fri, 15 May 2020 10:16:38 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D68D93F305; Fri, 15 May 2020 10:16:36 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, Will Deacon , Dave P Martin , Vincenzo Frascino , Szabolcs Nagy , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne Subject: [PATCH v4 08/26] arm64: mte: Tags-aware copy_page() implementation Date: Fri, 15 May 2020 18:15:54 +0100 Message-Id: <20200515171612.1020-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200515171612.1020-1-catalin.marinas@arm.com> References: <20200515171612.1020-1-catalin.marinas@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be preserved across page copy (e.g. for copy-on-write). Introduce MTE-aware copy_page() which preserves the tags across page copy. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v4: - Moved the tag copying to a separate function in mte.S and only call= ed if the source page has the PG_mte_tagged flag set. arch/arm64/include/asm/mte.h | 4 ++++ arch/arm64/lib/mte.S | 19 +++++++++++++++++++ arch/arm64/mm/copypage.c | 14 ++++++++++++-- 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 4310a7ff10c0..c1a09499c678 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -19,6 +19,7 @@ void mte_clear_page_tags(void *addr, size_t size); #define PG_mte_tagged PG_arch_2 =20 void mte_sync_tags(pte_t *ptep, pte_t pte); +void mte_copy_page_tags(void *kto, const void *kfrom); void flush_mte_state(void); =20 #else @@ -29,6 +30,9 @@ void flush_mte_state(void); static inline void mte_sync_tags(pte_t *ptep, pte_t pte) { } +static inline void mte_copy_page_tags(void *kto, const void *kfrom) +{ +} static inline void flush_mte_state(void) { } diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index 130fb7047e17..a531b52fa5ba 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -5,6 +5,7 @@ #include =20 #include +#include =20 .arch armv8.5-a+memtag =20 @@ -21,3 +22,21 @@ SYM_FUNC_START(mte_clear_page_tags) cbnz x1, 1b ret SYM_FUNC_END(mte_clear_page_tags) + +/* + * Copy the tags from the source page to the destination one + * x0 - address of the destination page + * x1 - address of the source page + */ +SYM_FUNC_START(mte_copy_page_tags) + mov x2, x0 + mov x3, x1 + multitag_transfer_size x5, x6 +1: ldgm x4, [x3] + stgm x4, [x2] + add x2, x2, x5 + add x3, x3, x5 + tst x2, #(PAGE_SIZE - 1) + b.ne 1b +2: +SYM_FUNC_END(mte_copy_page_tags) diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c index 2ee7b73433a5..2560ddc479ac 100644 --- a/arch/arm64/mm/copypage.c +++ b/arch/arm64/mm/copypage.c @@ -6,16 +6,26 @@ * Copyright (C) 2012 ARM Ltd. */ =20 +#include #include =20 #include #include +#include +#include =20 void __cpu_copy_user_page(void *kto, const void *kfrom, unsigned long va= ddr) { - struct page *page =3D virt_to_page(kto); + struct page *to_page =3D virt_to_page(kto); + struct page *from_page =3D virt_to_page(kfrom); + copy_page(kto, kfrom); - flush_dcache_page(page); + if (system_supports_mte() && + test_bit(PG_mte_tagged, &from_page->flags)) { + mte_copy_page_tags(kto, kfrom); + set_bit(PG_mte_tagged, &to_page->flags); + } + flush_dcache_page(to_page); } EXPORT_SYMBOL_GPL(__cpu_copy_user_page); =20