From: Catalin Marinas <catalin.marinas@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, Richard Earnshaw <Richard.Earnshaw@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, Marc Zyngier <maz@kernel.org>, Kevin Brodsky <kevin.brodsky@arm.com>, linux-mm@kvack.org, Andrey Konovalov <andreyknvl@google.com>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Will Deacon <will@kernel.org> Subject: [PATCH 11/22] arm64: Tags-aware memcmp_pages() implementation Date: Wed, 11 Dec 2019 18:40:16 +0000 [thread overview] Message-ID: <20191211184027.20130-12-catalin.marinas@arm.com> (raw) In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> When the Memory Tagging Extension is enabled, two pages are identical only if both their data and tags are identical. Make the generic memcmp_pages() a __weak function and add an arm64-specific implementation which takes care of the tags comparison. Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm64/include/asm/mte.h | 11 +++++++++ arch/arm64/lib/Makefile | 2 ++ arch/arm64/lib/mte.S | 46 ++++++++++++++++++++++++++++++++++++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/cmppages.c | 26 ++++++++++++++++++++ mm/util.c | 2 +- 6 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/mte.h create mode 100644 arch/arm64/lib/mte.S create mode 100644 arch/arm64/mm/cmppages.c diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h new file mode 100644 index 000000000000..64e814273659 --- /dev/null +++ b/arch/arm64/include/asm/mte.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_MTE_H +#define __ASM_MTE_H + +#ifndef __ASSEMBLY__ + +/* Memory Tagging API */ +int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_MTE_H */ diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index c21b936dc01d..178879a754b0 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -16,3 +16,5 @@ lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o obj-$(CONFIG_CRC32) += crc32.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o + +obj-$(CONFIG_ARM64_MTE) += mte.o diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S new file mode 100644 index 000000000000..d41955ab4134 --- /dev/null +++ b/arch/arm64/lib/mte.S @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2019 ARM Ltd. + */ +#include <linux/linkage.h> + +#include <asm/assembler.h> + +/* + * Compare tags of two pages + * x0 - page1 address + * x1 - page2 address + * Returns: + * w0 - negative, zero or positive value if the tag in the first page is + * less than, equal to or greater than the tag in the second page + */ +ENTRY(mte_memcmp_pages) + multitag_transfer_size x7, x5 +1: + ldgm x2, [x0] + ldgm x3, [x1] + + eor x4, x2, x3 + cbnz x4, 2f + + add x0, x0, x7 + add x1, x1, x7 + + tst x0, #(PAGE_SIZE - 1) + b.ne 1b + + mov w0, #0 + ret +2: + rbit x4, x4 + clz x4, x4 // count the least significant equal bits + and x4, x4, #~3 // round down to a multiple of 4 (bits per tag) + + lsr x2, x2, x4 // remove equal tags + lsr x3, x3, x4 + + lsl w2, w2, #28 // compare the differing tags + sub w0, w2, w3, lsl #28 + + ret +ENDPROC(mte_memcmp_pages) diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 849c1df3d214..7a556e9a19dc 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_ARM64_PTDUMP_CORE) += dump.o obj-$(CONFIG_ARM64_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o +obj-$(CONFIG_ARM64_MTE) += cmppages.o KASAN_SANITIZE_physaddr.o += n obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/arm64/mm/cmppages.c b/arch/arm64/mm/cmppages.c new file mode 100644 index 000000000000..943c1877e014 --- /dev/null +++ b/arch/arm64/mm/cmppages.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2019 ARM Ltd. + */ + +#include <linux/mm.h> +#include <linux/string.h> + +#include <asm/cpufeature.h> +#include <asm/mte.h> + +int memcmp_pages(struct page *page1, struct page *page2) +{ + char *addr1, *addr2; + int ret; + + addr1 = page_address(page1); + addr2 = page_address(page2); + + ret = memcmp(addr1, addr2, PAGE_SIZE); + /* if page content identical, check the tags */ + if (ret == 0 && system_supports_mte()) + ret = mte_memcmp_pages(addr1, addr2); + + return ret; +} diff --git a/mm/util.c b/mm/util.c index 988d11e6c17c..662fb3da6d01 100644 --- a/mm/util.c +++ b/mm/util.c @@ -899,7 +899,7 @@ int get_cmdline(struct task_struct *task, char *buffer, int buflen) return res; } -int memcmp_pages(struct page *page1, struct page *page2) +int __weak memcmp_pages(struct page *page1, struct page *page2) { char *addr1, *addr2; int ret;
WARNING: multiple messages have this Message-ID (diff)
From: Catalin Marinas <catalin.marinas@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, Richard Earnshaw <Richard.Earnshaw@arm.com>, Kevin Brodsky <kevin.brodsky@arm.com>, Andrey Konovalov <andreyknvl@google.com>, linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 11/22] arm64: Tags-aware memcmp_pages() implementation Date: Wed, 11 Dec 2019 18:40:16 +0000 [thread overview] Message-ID: <20191211184027.20130-12-catalin.marinas@arm.com> (raw) Message-ID: <20191211184016.OP-_YSjDaVOn8GiLrnV7Rm4xCeW4IkLkYeGyS1cVvgQ@z> (raw) In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> When the Memory Tagging Extension is enabled, two pages are identical only if both their data and tags are identical. Make the generic memcmp_pages() a __weak function and add an arm64-specific implementation which takes care of the tags comparison. Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm64/include/asm/mte.h | 11 +++++++++ arch/arm64/lib/Makefile | 2 ++ arch/arm64/lib/mte.S | 46 ++++++++++++++++++++++++++++++++++++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/cmppages.c | 26 ++++++++++++++++++++ mm/util.c | 2 +- 6 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/mte.h create mode 100644 arch/arm64/lib/mte.S create mode 100644 arch/arm64/mm/cmppages.c diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h new file mode 100644 index 000000000000..64e814273659 --- /dev/null +++ b/arch/arm64/include/asm/mte.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_MTE_H +#define __ASM_MTE_H + +#ifndef __ASSEMBLY__ + +/* Memory Tagging API */ +int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_MTE_H */ diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index c21b936dc01d..178879a754b0 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -16,3 +16,5 @@ lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o obj-$(CONFIG_CRC32) += crc32.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o + +obj-$(CONFIG_ARM64_MTE) += mte.o diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S new file mode 100644 index 000000000000..d41955ab4134 --- /dev/null +++ b/arch/arm64/lib/mte.S @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2019 ARM Ltd. + */ +#include <linux/linkage.h> + +#include <asm/assembler.h> + +/* + * Compare tags of two pages + * x0 - page1 address + * x1 - page2 address + * Returns: + * w0 - negative, zero or positive value if the tag in the first page is + * less than, equal to or greater than the tag in the second page + */ +ENTRY(mte_memcmp_pages) + multitag_transfer_size x7, x5 +1: + ldgm x2, [x0] + ldgm x3, [x1] + + eor x4, x2, x3 + cbnz x4, 2f + + add x0, x0, x7 + add x1, x1, x7 + + tst x0, #(PAGE_SIZE - 1) + b.ne 1b + + mov w0, #0 + ret +2: + rbit x4, x4 + clz x4, x4 // count the least significant equal bits + and x4, x4, #~3 // round down to a multiple of 4 (bits per tag) + + lsr x2, x2, x4 // remove equal tags + lsr x3, x3, x4 + + lsl w2, w2, #28 // compare the differing tags + sub w0, w2, w3, lsl #28 + + ret +ENDPROC(mte_memcmp_pages) diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 849c1df3d214..7a556e9a19dc 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_ARM64_PTDUMP_CORE) += dump.o obj-$(CONFIG_ARM64_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o +obj-$(CONFIG_ARM64_MTE) += cmppages.o KASAN_SANITIZE_physaddr.o += n obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/arm64/mm/cmppages.c b/arch/arm64/mm/cmppages.c new file mode 100644 index 000000000000..943c1877e014 --- /dev/null +++ b/arch/arm64/mm/cmppages.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2019 ARM Ltd. + */ + +#include <linux/mm.h> +#include <linux/string.h> + +#include <asm/cpufeature.h> +#include <asm/mte.h> + +int memcmp_pages(struct page *page1, struct page *page2) +{ + char *addr1, *addr2; + int ret; + + addr1 = page_address(page1); + addr2 = page_address(page2); + + ret = memcmp(addr1, addr2, PAGE_SIZE); + /* if page content identical, check the tags */ + if (ret == 0 && system_supports_mte()) + ret = mte_memcmp_pages(addr1, addr2); + + return ret; +} diff --git a/mm/util.c b/mm/util.c index 988d11e6c17c..662fb3da6d01 100644 --- a/mm/util.c +++ b/mm/util.c @@ -899,7 +899,7 @@ int get_cmdline(struct task_struct *task, char *buffer, int buflen) return res; } -int memcmp_pages(struct page *page1, struct page *page2) +int __weak memcmp_pages(struct page *page1, struct page *page2) { char *addr1, *addr2; int ret;
next prev parent reply other threads:[~2019-12-11 18:40 UTC|newest] Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-11 18:40 [PATCH 00/22] arm64: Memory Tagging Extension user-space support Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 01/22] mm: Reserve asm-generic prot flags 0x10 and 0x20 for arch use Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 19:26 ` Arnd Bergmann 2019-12-11 19:26 ` Arnd Bergmann 2019-12-11 18:40 ` [PATCH 02/22] kbuild: Add support for 'as-instr' to be used in Kconfig files Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-12 5:03 ` Masahiro Yamada 2019-12-12 5:03 ` Masahiro Yamada 2019-12-11 18:40 ` [PATCH 03/22] arm64: alternative: Allow alternative_insn to always issue the first instruction Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 04/22] arm64: Use macros instead of hard-coded constants for MAIR_EL1 Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 05/22] arm64: mte: system register definitions Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 06/22] arm64: mte: CPU feature detection and initial sysreg configuration Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 07/22] arm64: mte: Use Normal Tagged attributes for the linear map Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 08/22] arm64: mte: Assembler macros and default architecture for .S files Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 09/22] arm64: mte: Tags-aware clear_page() implementation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 10/22] arm64: mte: Tags-aware copy_page() implementation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas [this message] 2019-12-11 18:40 ` [PATCH 11/22] arm64: Tags-aware memcmp_pages() implementation Catalin Marinas 2019-12-11 18:40 ` [PATCH 12/22] arm64: mte: Add specific SIGSEGV codes Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 19:31 ` Arnd Bergmann 2019-12-11 19:31 ` Arnd Bergmann 2019-12-12 9:34 ` Catalin Marinas 2019-12-12 9:34 ` Catalin Marinas 2019-12-12 18:26 ` Eric W. Biederman 2019-12-12 18:26 ` Eric W. Biederman 2019-12-17 17:48 ` Catalin Marinas 2019-12-17 17:48 ` Catalin Marinas 2019-12-17 20:06 ` Eric W. Biederman 2019-12-17 20:06 ` Eric W. Biederman 2019-12-11 18:40 ` [PATCH 13/22] arm64: mte: Handle synchronous and asynchronous tag check faults Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-14 1:43 ` Peter Collingbourne 2019-12-14 1:43 ` Peter Collingbourne 2019-12-17 18:01 ` Catalin Marinas 2019-12-17 18:01 ` Catalin Marinas 2019-12-20 1:36 ` [PATCH] arm64: mte: Do not service syscalls after async tag fault Peter Collingbourne 2019-12-20 1:36 ` Peter Collingbourne 2020-02-12 11:09 ` Catalin Marinas 2020-02-18 21:59 ` Peter Collingbourne 2020-02-19 16:16 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 14/22] mm: Introduce arch_calc_vm_flag_bits() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 15/22] arm64: mte: Add PROT_MTE support to mmap() and mprotect() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2020-01-21 22:06 ` Peter Collingbourne 2019-12-11 18:40 ` [PATCH 16/22] mm: Introduce arch_validate_flags() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 17/22] arm64: mte: Validate the PROT_MTE request via arch_validate_flags() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 18/22] mm: Allow arm64 mmap(PROT_MTE) on RAM-based files Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 19/22] arm64: mte: Allow user control of the tag check mode via prctl() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-19 20:32 ` Peter Collingbourne 2019-12-19 20:32 ` Peter Collingbourne 2019-12-20 1:48 ` [PATCH] arm64: mte: Clear SCTLR_EL1.TCF0 on exec Peter Collingbourne 2019-12-20 1:48 ` Peter Collingbourne 2020-02-12 17:03 ` Catalin Marinas 2019-12-27 14:34 ` [PATCH 19/22] arm64: mte: Allow user control of the tag check mode via prctl() Kevin Brodsky 2019-12-27 14:34 ` Kevin Brodsky 2020-02-12 11:45 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 20/22] arm64: mte: Allow user control of the excluded tags " Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-16 14:20 ` Kevin Brodsky 2019-12-16 14:20 ` Kevin Brodsky 2019-12-16 17:30 ` Peter Collingbourne 2019-12-16 17:30 ` Peter Collingbourne 2019-12-17 17:56 ` Catalin Marinas 2019-12-17 17:56 ` Catalin Marinas 2020-06-22 17:17 ` Catalin Marinas 2020-06-22 19:00 ` Peter Collingbourne 2020-06-23 16:42 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 21/22] arm64: mte: Kconfig entry Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 22/22] arm64: mte: Add Memory Tagging Extension documentation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-24 15:03 ` Kevin Brodsky 2019-12-24 15:03 ` Kevin Brodsky 2019-12-13 18:05 ` [PATCH 00/22] arm64: Memory Tagging Extension user-space support Peter Collingbourne 2019-12-13 18:05 ` Peter Collingbourne 2020-02-13 11:23 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191211184027.20130-12-catalin.marinas@arm.com \ --to=catalin.marinas@arm.com \ --cc=Richard.Earnshaw@arm.com \ --cc=andreyknvl@google.com \ --cc=kevin.brodsky@arm.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-mm@kvack.org \ --cc=maz@kernel.org \ --cc=szabolcs.nagy@arm.com \ --cc=vincenzo.frascino@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).