From: Peter Collingbourne <pcc@google.com> To: Catalin Marinas <catalin.marinas@arm.com>, Evgenii Stepanov <eugenis@google.com>, Kostya Serebryany <kcc@google.com> Cc: linux-arch@vger.kernel.org, Richard Earnshaw <Richard.Earnshaw@arm.com>, Will Deacon <will@kernel.org>, Szabolcs Nagy <szabolcs.nagy@arm.com>, Marc Zyngier <maz@kernel.org>, Kevin Brodsky <kevin.brodsky@arm.com>, linux-mm@kvack.org, Andrey Konovalov <andreyknvl@google.com>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Peter Collingbourne <pcc@google.com>, Linux ARM <linux-arm-kernel@lists.infradead.org> Subject: [PATCH] arm64: mte: Clear SCTLR_EL1.TCF0 on exec Date: Thu, 19 Dec 2019 17:48:53 -0800 [thread overview] Message-ID: <20191220014853.223389-1-pcc@google.com> (raw) In-Reply-To: <CAMn1gO4iv1FsxV+aR3CgU=jgmVjHL0YQF-xJJG0UMv3nJZnOBw@mail.gmail.com> Signed-off-by: Peter Collingbourne <pcc@google.com> --- On Thu, Dec 19, 2019 at 12:32 PM Peter Collingbourne <pcc@google.com> wrote: > > On Wed, Dec 11, 2019 at 10:45 AM Catalin Marinas > <catalin.marinas@arm.com> wrote: > > + if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) > > + update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); > > I don't entirely understand why yet, but I've found that this check is > insufficient for ensuring consistency between SCTLR_EL1.TCF0 and > sctlr_tcf0. In my Android test environment with some processes having > sctlr_tcf0=SCTLR_EL1_TCF0_SYNC and others having sctlr_tcf0=0, I am > seeing intermittent tag failures coming from the sctlr_tcf0=0 > processes. With this patch: > > diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c > index ef3bfa2bf2b1..4e5d02520a51 100644 > --- a/arch/arm64/mm/fault.c > +++ b/arch/arm64/mm/fault.c > @@ -663,6 +663,8 @@ static int do_sea(unsigned long addr, unsigned int > esr, struct pt_regs *regs) > static int do_tag_check_fault(unsigned long addr, unsigned int esr, > struct pt_regs *regs) > { > + printk(KERN_ERR "do_tag_check_fault %lx %lx\n", > + current->thread.sctlr_tcf0, read_sysreg(sctlr_el1)); > do_bad_area(addr, esr, regs); > return 0; > } > > I see dmesg output like this: > > [ 15.249216] do_tag_check_fault 0 c60fc64791d > > showing that SCTLR_EL1.TCF0 became inconsistent with sctlr_tcf0. This > patch fixes the problem for me: > > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > index fba89c9f070b..fb012f0baa12 100644 > --- a/arch/arm64/kernel/process.c > +++ b/arch/arm64/kernel/process.c > @@ -518,9 +518,7 @@ static void mte_thread_switch(struct task_struct *next) > if (!system_supports_mte()) > return; > > - /* avoid expensive SCTLR_EL1 accesses if no change */ > - if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) > - update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); > + update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); > update_gcr_el1_excl(next->thread.gcr_excl); > } > #else > @@ -643,15 +641,8 @@ static long set_mte_ctrl(unsigned long arg) > return -EINVAL; > } > > - /* > - * mte_thread_switch() checks current->thread.sctlr_tcf0 as an > - * optimisation. Disable preemption so that it does not see > - * the variable update before the SCTLR_EL1.TCF0 one. > - */ > - preempt_disable(); > current->thread.sctlr_tcf0 = tcf0; > update_sctlr_el1_tcf0(tcf0); > - preempt_enable(); > > current->thread.gcr_excl = (arg & PR_MTE_EXCL_MASK) >> > PR_MTE_EXCL_SHIFT; > update_gcr_el1_excl(current->thread.gcr_excl); > > Since sysreg_clear_set only sets the sysreg if it ended up changing, I > wouldn't expect this to cause a significant performance hit unless > just reading SCTLR_EL1 is expensive. That being said, if the > inconsistency is indicative of a deeper problem, we should probably > address that. I tracked it down to the flush_mte_state() function setting sctlr_tcf0 but failing to update SCTLR_EL1.TCF0. With this patch I am not seeing any more inconsistencies. Peter arch/arm64/kernel/process.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index fba89c9f070b..07e8e7bd3bec 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -319,6 +319,25 @@ static void flush_tagged_addr_state(void) } #ifdef CONFIG_ARM64_MTE +static void update_sctlr_el1_tcf0(u64 tcf0) +{ + /* no need for ISB since this only affects EL0, implicit with ERET */ + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); +} + +static void set_sctlr_el1_tcf0(u64 tcf0) +{ + /* + * mte_thread_switch() checks current->thread.sctlr_tcf0 as an + * optimisation. Disable preemption so that it does not see + * the variable update before the SCTLR_EL1.TCF0 one. + */ + preempt_disable(); + current->thread.sctlr_tcf0 = tcf0; + update_sctlr_el1_tcf0(tcf0); + preempt_enable(); +} + static void flush_mte_state(void) { if (!system_supports_mte()) @@ -327,7 +346,7 @@ static void flush_mte_state(void) /* clear any pending asynchronous tag fault */ clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking */ - current->thread.sctlr_tcf0 = 0; + set_sctlr_el1_tcf0(0); } #else static void flush_mte_state(void) @@ -497,12 +516,6 @@ static void ssbs_thread_switch(struct task_struct *next) } #ifdef CONFIG_ARM64_MTE -static void update_sctlr_el1_tcf0(u64 tcf0) -{ - /* no need for ISB since this only affects EL0, implicit with ERET */ - sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); -} - static void update_gcr_el1_excl(u64 excl) { /* @@ -643,15 +656,7 @@ static long set_mte_ctrl(unsigned long arg) return -EINVAL; } - /* - * mte_thread_switch() checks current->thread.sctlr_tcf0 as an - * optimisation. Disable preemption so that it does not see - * the variable update before the SCTLR_EL1.TCF0 one. - */ - preempt_disable(); - current->thread.sctlr_tcf0 = tcf0; - update_sctlr_el1_tcf0(tcf0); - preempt_enable(); + set_sctlr_el1_tcf0(tcf0); current->thread.gcr_excl = (arg & PR_MTE_EXCL_MASK) >> PR_MTE_EXCL_SHIFT; update_gcr_el1_excl(current->thread.gcr_excl); -- 2.24.1.735.g03f4e72817-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Peter Collingbourne <pcc@google.com> To: Catalin Marinas <catalin.marinas@arm.com>, Evgenii Stepanov <eugenis@google.com>, Kostya Serebryany <kcc@google.com> Cc: Peter Collingbourne <pcc@google.com>, Linux ARM <linux-arm-kernel@lists.infradead.org>, linux-arch@vger.kernel.org, Richard Earnshaw <Richard.Earnshaw@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, Marc Zyngier <maz@kernel.org>, Kevin Brodsky <kevin.brodsky@arm.com>, linux-mm@kvack.org, Andrey Konovalov <andreyknvl@google.com>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Will Deacon <will@kernel.org> Subject: [PATCH] arm64: mte: Clear SCTLR_EL1.TCF0 on exec Date: Thu, 19 Dec 2019 17:48:53 -0800 [thread overview] Message-ID: <20191220014853.223389-1-pcc@google.com> (raw) Message-ID: <20191220014853.K5ZYk0eejuNmjncflZd0drdj2Wm1xN88D83sgWd_W08@z> (raw) In-Reply-To: <CAMn1gO4iv1FsxV+aR3CgU=jgmVjHL0YQF-xJJG0UMv3nJZnOBw@mail.gmail.com> Signed-off-by: Peter Collingbourne <pcc@google.com> --- On Thu, Dec 19, 2019 at 12:32 PM Peter Collingbourne <pcc@google.com> wrote: > > On Wed, Dec 11, 2019 at 10:45 AM Catalin Marinas > <catalin.marinas@arm.com> wrote: > > + if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) > > + update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); > > I don't entirely understand why yet, but I've found that this check is > insufficient for ensuring consistency between SCTLR_EL1.TCF0 and > sctlr_tcf0. In my Android test environment with some processes having > sctlr_tcf0=SCTLR_EL1_TCF0_SYNC and others having sctlr_tcf0=0, I am > seeing intermittent tag failures coming from the sctlr_tcf0=0 > processes. With this patch: > > diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c > index ef3bfa2bf2b1..4e5d02520a51 100644 > --- a/arch/arm64/mm/fault.c > +++ b/arch/arm64/mm/fault.c > @@ -663,6 +663,8 @@ static int do_sea(unsigned long addr, unsigned int > esr, struct pt_regs *regs) > static int do_tag_check_fault(unsigned long addr, unsigned int esr, > struct pt_regs *regs) > { > + printk(KERN_ERR "do_tag_check_fault %lx %lx\n", > + current->thread.sctlr_tcf0, read_sysreg(sctlr_el1)); > do_bad_area(addr, esr, regs); > return 0; > } > > I see dmesg output like this: > > [ 15.249216] do_tag_check_fault 0 c60fc64791d > > showing that SCTLR_EL1.TCF0 became inconsistent with sctlr_tcf0. This > patch fixes the problem for me: > > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > index fba89c9f070b..fb012f0baa12 100644 > --- a/arch/arm64/kernel/process.c > +++ b/arch/arm64/kernel/process.c > @@ -518,9 +518,7 @@ static void mte_thread_switch(struct task_struct *next) > if (!system_supports_mte()) > return; > > - /* avoid expensive SCTLR_EL1 accesses if no change */ > - if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) > - update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); > + update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); > update_gcr_el1_excl(next->thread.gcr_excl); > } > #else > @@ -643,15 +641,8 @@ static long set_mte_ctrl(unsigned long arg) > return -EINVAL; > } > > - /* > - * mte_thread_switch() checks current->thread.sctlr_tcf0 as an > - * optimisation. Disable preemption so that it does not see > - * the variable update before the SCTLR_EL1.TCF0 one. > - */ > - preempt_disable(); > current->thread.sctlr_tcf0 = tcf0; > update_sctlr_el1_tcf0(tcf0); > - preempt_enable(); > > current->thread.gcr_excl = (arg & PR_MTE_EXCL_MASK) >> > PR_MTE_EXCL_SHIFT; > update_gcr_el1_excl(current->thread.gcr_excl); > > Since sysreg_clear_set only sets the sysreg if it ended up changing, I > wouldn't expect this to cause a significant performance hit unless > just reading SCTLR_EL1 is expensive. That being said, if the > inconsistency is indicative of a deeper problem, we should probably > address that. I tracked it down to the flush_mte_state() function setting sctlr_tcf0 but failing to update SCTLR_EL1.TCF0. With this patch I am not seeing any more inconsistencies. Peter arch/arm64/kernel/process.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index fba89c9f070b..07e8e7bd3bec 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -319,6 +319,25 @@ static void flush_tagged_addr_state(void) } #ifdef CONFIG_ARM64_MTE +static void update_sctlr_el1_tcf0(u64 tcf0) +{ + /* no need for ISB since this only affects EL0, implicit with ERET */ + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); +} + +static void set_sctlr_el1_tcf0(u64 tcf0) +{ + /* + * mte_thread_switch() checks current->thread.sctlr_tcf0 as an + * optimisation. Disable preemption so that it does not see + * the variable update before the SCTLR_EL1.TCF0 one. + */ + preempt_disable(); + current->thread.sctlr_tcf0 = tcf0; + update_sctlr_el1_tcf0(tcf0); + preempt_enable(); +} + static void flush_mte_state(void) { if (!system_supports_mte()) @@ -327,7 +346,7 @@ static void flush_mte_state(void) /* clear any pending asynchronous tag fault */ clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking */ - current->thread.sctlr_tcf0 = 0; + set_sctlr_el1_tcf0(0); } #else static void flush_mte_state(void) @@ -497,12 +516,6 @@ static void ssbs_thread_switch(struct task_struct *next) } #ifdef CONFIG_ARM64_MTE -static void update_sctlr_el1_tcf0(u64 tcf0) -{ - /* no need for ISB since this only affects EL0, implicit with ERET */ - sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); -} - static void update_gcr_el1_excl(u64 excl) { /* @@ -643,15 +656,7 @@ static long set_mte_ctrl(unsigned long arg) return -EINVAL; } - /* - * mte_thread_switch() checks current->thread.sctlr_tcf0 as an - * optimisation. Disable preemption so that it does not see - * the variable update before the SCTLR_EL1.TCF0 one. - */ - preempt_disable(); - current->thread.sctlr_tcf0 = tcf0; - update_sctlr_el1_tcf0(tcf0); - preempt_enable(); + set_sctlr_el1_tcf0(tcf0); current->thread.gcr_excl = (arg & PR_MTE_EXCL_MASK) >> PR_MTE_EXCL_SHIFT; update_gcr_el1_excl(current->thread.gcr_excl); -- 2.24.1.735.g03f4e72817-goog
next prev parent reply other threads:[~2019-12-20 1:48 UTC|newest] Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-11 18:40 [PATCH 00/22] arm64: Memory Tagging Extension user-space support Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 01/22] mm: Reserve asm-generic prot flags 0x10 and 0x20 for arch use Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 19:26 ` Arnd Bergmann 2019-12-11 19:26 ` Arnd Bergmann 2019-12-11 18:40 ` [PATCH 02/22] kbuild: Add support for 'as-instr' to be used in Kconfig files Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-12 5:03 ` Masahiro Yamada 2019-12-12 5:03 ` Masahiro Yamada 2019-12-11 18:40 ` [PATCH 03/22] arm64: alternative: Allow alternative_insn to always issue the first instruction Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 04/22] arm64: Use macros instead of hard-coded constants for MAIR_EL1 Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 05/22] arm64: mte: system register definitions Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 06/22] arm64: mte: CPU feature detection and initial sysreg configuration Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 07/22] arm64: mte: Use Normal Tagged attributes for the linear map Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 08/22] arm64: mte: Assembler macros and default architecture for .S files Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 09/22] arm64: mte: Tags-aware clear_page() implementation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 10/22] arm64: mte: Tags-aware copy_page() implementation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 11/22] arm64: Tags-aware memcmp_pages() implementation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 12/22] arm64: mte: Add specific SIGSEGV codes Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 19:31 ` Arnd Bergmann 2019-12-11 19:31 ` Arnd Bergmann 2019-12-12 9:34 ` Catalin Marinas 2019-12-12 9:34 ` Catalin Marinas 2019-12-12 18:26 ` Eric W. Biederman 2019-12-12 18:26 ` Eric W. Biederman 2019-12-17 17:48 ` Catalin Marinas 2019-12-17 17:48 ` Catalin Marinas 2019-12-17 20:06 ` Eric W. Biederman 2019-12-17 20:06 ` Eric W. Biederman 2019-12-11 18:40 ` [PATCH 13/22] arm64: mte: Handle synchronous and asynchronous tag check faults Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-14 1:43 ` Peter Collingbourne 2019-12-14 1:43 ` Peter Collingbourne 2019-12-17 18:01 ` Catalin Marinas 2019-12-17 18:01 ` Catalin Marinas 2019-12-20 1:36 ` [PATCH] arm64: mte: Do not service syscalls after async tag fault Peter Collingbourne 2019-12-20 1:36 ` Peter Collingbourne 2020-02-12 11:09 ` Catalin Marinas 2020-02-18 21:59 ` Peter Collingbourne 2020-02-19 16:16 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 14/22] mm: Introduce arch_calc_vm_flag_bits() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 15/22] arm64: mte: Add PROT_MTE support to mmap() and mprotect() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2020-01-21 22:06 ` Peter Collingbourne 2019-12-11 18:40 ` [PATCH 16/22] mm: Introduce arch_validate_flags() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 17/22] arm64: mte: Validate the PROT_MTE request via arch_validate_flags() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 18/22] mm: Allow arm64 mmap(PROT_MTE) on RAM-based files Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 19/22] arm64: mte: Allow user control of the tag check mode via prctl() Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-19 20:32 ` Peter Collingbourne 2019-12-19 20:32 ` Peter Collingbourne 2019-12-20 1:48 ` Peter Collingbourne [this message] 2019-12-20 1:48 ` [PATCH] arm64: mte: Clear SCTLR_EL1.TCF0 on exec Peter Collingbourne 2020-02-12 17:03 ` Catalin Marinas 2019-12-27 14:34 ` [PATCH 19/22] arm64: mte: Allow user control of the tag check mode via prctl() Kevin Brodsky 2019-12-27 14:34 ` Kevin Brodsky 2020-02-12 11:45 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 20/22] arm64: mte: Allow user control of the excluded tags " Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-16 14:20 ` Kevin Brodsky 2019-12-16 14:20 ` Kevin Brodsky 2019-12-16 17:30 ` Peter Collingbourne 2019-12-16 17:30 ` Peter Collingbourne 2019-12-17 17:56 ` Catalin Marinas 2019-12-17 17:56 ` Catalin Marinas 2020-06-22 17:17 ` Catalin Marinas 2020-06-22 19:00 ` Peter Collingbourne 2020-06-23 16:42 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 21/22] arm64: mte: Kconfig entry Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-11 18:40 ` [PATCH 22/22] arm64: mte: Add Memory Tagging Extension documentation Catalin Marinas 2019-12-11 18:40 ` Catalin Marinas 2019-12-24 15:03 ` Kevin Brodsky 2019-12-24 15:03 ` Kevin Brodsky 2019-12-13 18:05 ` [PATCH 00/22] arm64: Memory Tagging Extension user-space support Peter Collingbourne 2019-12-13 18:05 ` Peter Collingbourne 2020-02-13 11:23 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191220014853.223389-1-pcc@google.com \ --to=pcc@google.com \ --cc=Richard.Earnshaw@arm.com \ --cc=andreyknvl@google.com \ --cc=catalin.marinas@arm.com \ --cc=eugenis@google.com \ --cc=kcc@google.com \ --cc=kevin.brodsky@arm.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-mm@kvack.org \ --cc=maz@kernel.org \ --cc=szabolcs.nagy@arm.com \ --cc=vincenzo.frascino@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).