From: Vincenzo Frascino <vincenzo.frascino@arm.com> To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Dmitry Vyukov <dvyukov@google.com>, Andrey Ryabinin <aryabinin@virtuozzo.com>, Alexander Potapenko <glider@google.com>, Marco Elver <elver@google.com>, Evgenii Stepanov <eugenis@google.com>, Branislav Rankov <Branislav.Rankov@arm.com>, Andrey Konovalov <andreyknvl@google.com> Subject: [PATCH v4 5/5] arm64: mte: Inline mte_assign_mem_tag_range() Date: Mon, 18 Jan 2021 18:30:33 +0000 [thread overview] Message-ID: <20210118183033.41764-6-vincenzo.frascino@arm.com> (raw) In-Reply-To: <20210118183033.41764-1-vincenzo.frascino@arm.com> mte_assign_mem_tag_range() is called on production KASAN HW hot paths. It makes sense to inline it in an attempt to reduce the overhead. Inline mte_assign_mem_tag_range() based on the indications provided at [1]. [1] https://lore.kernel.org/r/CAAeHK+wCO+J7D1_T89DG+jJrPLk3X9RsGFKxJGd0ZcUFjQT-9Q@mail.gmail.com/ Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com> --- arch/arm64/include/asm/mte.h | 26 +++++++++++++++++++++++++- arch/arm64/lib/mte.S | 15 --------------- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 237bb2f7309d..1a6fd53f82c3 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -49,7 +49,31 @@ long get_mte_ctrl(struct task_struct *task); int mte_ptrace_copy_tags(struct task_struct *child, long request, unsigned long addr, unsigned long data); -void mte_assign_mem_tag_range(void *addr, size_t size); +static inline void mte_assign_mem_tag_range(void *addr, size_t size) +{ + u64 _addr = (u64)addr; + u64 _end = _addr + size; + + /* + * This function must be invoked from an MTE enabled context. + * + * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and + * size must be non-zero and MTE_GRANULE_SIZE aligned. + */ + do { + /* + * 'asm volatile' is required to prevent the compiler to move + * the statement outside of the loop. + */ + asm volatile(__MTE_PREAMBLE "stg %0, [%0]" + : + : "r" (_addr) + : "memory"); + + _addr += MTE_GRANULE_SIZE; + } while (_addr != _end); +} + #else /* CONFIG_ARM64_MTE */ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index 9e1a12e10053..a0a650451510 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -150,18 +150,3 @@ SYM_FUNC_START(mte_restore_page_tags) ret SYM_FUNC_END(mte_restore_page_tags) -/* - * Assign allocation tags for a region of memory based on the pointer tag - * x0 - source pointer - * x1 - size - * - * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and - * size must be non-zero and MTE_GRANULE_SIZE aligned. - */ -SYM_FUNC_START(mte_assign_mem_tag_range) -1: stg x0, [x0] - add x0, x0, #MTE_GRANULE_SIZE - subs x1, x1, #MTE_GRANULE_SIZE - b.gt 1b - ret -SYM_FUNC_END(mte_assign_mem_tag_range) -- 2.30.0
WARNING: multiple messages have this Message-ID (diff)
From: Vincenzo Frascino <vincenzo.frascino@arm.com> To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Marco Elver <elver@google.com>, Catalin Marinas <catalin.marinas@arm.com>, Branislav Rankov <Branislav.Rankov@arm.com>, Alexander Potapenko <glider@google.com>, Evgenii Stepanov <eugenis@google.com>, Andrey Konovalov <andreyknvl@google.com>, Andrey Ryabinin <aryabinin@virtuozzo.com>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Will Deacon <will@kernel.org>, Dmitry Vyukov <dvyukov@google.com> Subject: [PATCH v4 5/5] arm64: mte: Inline mte_assign_mem_tag_range() Date: Mon, 18 Jan 2021 18:30:33 +0000 [thread overview] Message-ID: <20210118183033.41764-6-vincenzo.frascino@arm.com> (raw) In-Reply-To: <20210118183033.41764-1-vincenzo.frascino@arm.com> mte_assign_mem_tag_range() is called on production KASAN HW hot paths. It makes sense to inline it in an attempt to reduce the overhead. Inline mte_assign_mem_tag_range() based on the indications provided at [1]. [1] https://lore.kernel.org/r/CAAeHK+wCO+J7D1_T89DG+jJrPLk3X9RsGFKxJGd0ZcUFjQT-9Q@mail.gmail.com/ Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com> --- arch/arm64/include/asm/mte.h | 26 +++++++++++++++++++++++++- arch/arm64/lib/mte.S | 15 --------------- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 237bb2f7309d..1a6fd53f82c3 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -49,7 +49,31 @@ long get_mte_ctrl(struct task_struct *task); int mte_ptrace_copy_tags(struct task_struct *child, long request, unsigned long addr, unsigned long data); -void mte_assign_mem_tag_range(void *addr, size_t size); +static inline void mte_assign_mem_tag_range(void *addr, size_t size) +{ + u64 _addr = (u64)addr; + u64 _end = _addr + size; + + /* + * This function must be invoked from an MTE enabled context. + * + * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and + * size must be non-zero and MTE_GRANULE_SIZE aligned. + */ + do { + /* + * 'asm volatile' is required to prevent the compiler to move + * the statement outside of the loop. + */ + asm volatile(__MTE_PREAMBLE "stg %0, [%0]" + : + : "r" (_addr) + : "memory"); + + _addr += MTE_GRANULE_SIZE; + } while (_addr != _end); +} + #else /* CONFIG_ARM64_MTE */ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index 9e1a12e10053..a0a650451510 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -150,18 +150,3 @@ SYM_FUNC_START(mte_restore_page_tags) ret SYM_FUNC_END(mte_restore_page_tags) -/* - * Assign allocation tags for a region of memory based on the pointer tag - * x0 - source pointer - * x1 - size - * - * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and - * size must be non-zero and MTE_GRANULE_SIZE aligned. - */ -SYM_FUNC_START(mte_assign_mem_tag_range) -1: stg x0, [x0] - add x0, x0, #MTE_GRANULE_SIZE - subs x1, x1, #MTE_GRANULE_SIZE - b.gt 1b - ret -SYM_FUNC_END(mte_assign_mem_tag_range) -- 2.30.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-01-18 18:35 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-01-18 18:30 [PATCH v4 0/5] arm64: ARMv8.5-A: MTE: Add async mode support Vincenzo Frascino 2021-01-18 18:30 ` Vincenzo Frascino 2021-01-18 18:30 ` [PATCH v4 1/5] arm64: mte: Add asynchronous " Vincenzo Frascino 2021-01-18 18:30 ` Vincenzo Frascino 2021-01-19 12:57 ` Catalin Marinas 2021-01-19 12:57 ` Catalin Marinas 2021-01-19 18:10 ` Andrey Konovalov 2021-01-19 18:10 ` Andrey Konovalov 2021-01-18 18:30 ` [PATCH v4 2/5] kasan: Add KASAN mode kernel parameter Vincenzo Frascino 2021-01-18 18:30 ` Vincenzo Frascino 2021-01-19 18:10 ` Andrey Konovalov 2021-01-19 18:10 ` Andrey Konovalov 2021-01-20 14:45 ` Vincenzo Frascino 2021-01-20 14:45 ` Vincenzo Frascino 2021-01-21 16:45 ` Vincenzo Frascino 2021-01-21 16:45 ` Vincenzo Frascino 2021-01-18 18:30 ` [PATCH v4 3/5] kasan: Add report for async mode Vincenzo Frascino 2021-01-18 18:30 ` Vincenzo Frascino 2021-01-19 13:04 ` Catalin Marinas 2021-01-19 13:04 ` Catalin Marinas 2021-01-19 14:23 ` Vincenzo Frascino 2021-01-19 14:23 ` Vincenzo Frascino 2021-01-19 14:46 ` Mark Rutland 2021-01-19 14:46 ` Mark Rutland 2021-01-19 15:05 ` Vincenzo Frascino 2021-01-19 15:05 ` Vincenzo Frascino 2021-01-19 18:12 ` Andrey Konovalov 2021-01-19 18:12 ` Andrey Konovalov 2021-01-20 14:46 ` Vincenzo Frascino 2021-01-20 14:46 ` Vincenzo Frascino 2021-01-18 18:30 ` [PATCH v4 4/5] arm64: mte: Enable async tag check fault Vincenzo Frascino 2021-01-18 18:30 ` Vincenzo Frascino 2021-01-19 14:34 ` Catalin Marinas 2021-01-19 14:34 ` Catalin Marinas 2021-01-19 14:45 ` Vincenzo Frascino 2021-01-19 14:45 ` Vincenzo Frascino 2021-01-18 18:30 ` Vincenzo Frascino [this message] 2021-01-18 18:30 ` [PATCH v4 5/5] arm64: mte: Inline mte_assign_mem_tag_range() Vincenzo Frascino 2021-01-19 14:45 ` Catalin Marinas 2021-01-19 14:45 ` Catalin Marinas 2021-01-19 15:48 ` Vincenzo Frascino 2021-01-19 15:48 ` Vincenzo Frascino 2021-01-19 18:12 ` Andrey Konovalov 2021-01-19 18:12 ` Andrey Konovalov 2021-01-19 19:00 ` Catalin Marinas 2021-01-19 19:00 ` Catalin Marinas 2021-01-19 19:34 ` Andrey Konovalov 2021-01-19 19:34 ` Andrey Konovalov 2021-01-19 18:09 ` [PATCH v4 0/5] arm64: ARMv8.5-A: MTE: Add async mode support Andrey Konovalov 2021-01-19 18:09 ` Andrey Konovalov 2021-01-21 11:35 ` Vincenzo Frascino 2021-01-21 11:35 ` Vincenzo Frascino 2021-01-21 12:25 ` Andrey Konovalov 2021-01-21 12:25 ` Andrey Konovalov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210118183033.41764-6-vincenzo.frascino@arm.com \ --to=vincenzo.frascino@arm.com \ --cc=Branislav.Rankov@arm.com \ --cc=andreyknvl@google.com \ --cc=aryabinin@virtuozzo.com \ --cc=catalin.marinas@arm.com \ --cc=dvyukov@google.com \ --cc=elver@google.com \ --cc=eugenis@google.com \ --cc=glider@google.com \ --cc=kasan-dev@googlegroups.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.