From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A981C433E0 for ; Wed, 24 Jun 2020 17:53:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4A9CD207E8 for ; Wed, 24 Jun 2020 17:53:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A9CD207E8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E25036B0010; Wed, 24 Jun 2020 13:53:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFF136B0022; Wed, 24 Jun 2020 13:53:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C9DA36B0023; Wed, 24 Jun 2020 13:53:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0117.hostedemail.com [216.40.44.117]) by kanga.kvack.org (Postfix) with ESMTP id A1F446B0010 for ; Wed, 24 Jun 2020 13:53:01 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 58E031EE6 for ; Wed, 24 Jun 2020 17:53:01 +0000 (UTC) X-FDA: 76964851362.16.berry07_240193126e46 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 1B02D100E6903 for ; Wed, 24 Jun 2020 17:53:01 +0000 (UTC) X-HE-Tag: berry07_240193126e46 X-Filterd-Recvd-Size: 13160 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Wed, 24 Jun 2020 17:53:00 +0000 (UTC) Received: from localhost.localdomain (unknown [2.26.170.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EFEC02080C; Wed, 24 Jun 2020 17:52:57 +0000 (UTC) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, Will Deacon , Dave P Martin , Vincenzo Frascino , Szabolcs Nagy , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , Andrew Morton Subject: [PATCH v5 05/25] arm64: mte: Handle synchronous and asynchronous tag check faults Date: Wed, 24 Jun 2020 18:52:24 +0100 Message-Id: <20200624175244.25837-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624175244.25837-1-catalin.marinas@arm.com> References: <20200624175244.25837-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1B02D100E6903 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino The Memory Tagging Extension has two modes of notifying a tag check fault at EL0, configurable through the SCTLR_EL1.TCF0 field: 1. Synchronous raising of a Data Abort exception with DFSC 17. 2. Asynchronous setting of a cumulative bit in TFSRE0_EL1. Add the exception handler for the synchronous exception and handling of the asynchronous TFSRE0_EL1.TF0 bit setting via a new TIF flag in do_notify_resume(). On a tag check failure in user-space, whether synchronous or asynchronous, a SIGSEGV will be raised on the faulting thread. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v4: - Use send_signal_fault() instead of fault_signal_inject() for asynchronous tag check faults as execution can continue even if thi= s signal is masked. - Add DSB ISH prior to writing TFSRE0_EL1 in the clear_mte_async_tcf macro. - Move clear_mte_async_tcf just after returning to user since do_notify_resume() may still cause async tag faults via do_signal()= . =20 v3: - Asynchronous tag check faults during the uaccess routines in the kernel are ignored. - Fix check_mte_async_tcf calling site as it expects the first argume= nt to be the thread flags. - Move the mte_thread_switch() definition and call to a later patch a= s this became empty with the removal of async uaccess checking. - Add dsb() and clearing of TFSRE0_EL1 in flush_mte_state(), in case execve() triggered a asynchronous tag check fault. - Clear TIF_MTE_ASYNC_FAULT in arch_dup_task_struct() so that the chi= ld does not inherit any pending tag fault in the parent. =20 v2: - Clear PSTATE.TCO on exception entry (automatically set by the hardw= are). - On syscall entry, for asynchronous tag check faults from user space= , generate the signal early via syscall restarting. - Before context switch, save any potential async tag check fault generated by the kernel to the TIF flag (this follows an architectu= re update where the uaccess routines use the TCF0 mode). - Moved the flush_mte_state() and mte_thread_switch() function to a n= ew mte.c file. arch/arm64/include/asm/mte.h | 23 +++++++++++++++++ arch/arm64/include/asm/thread_info.h | 4 ++- arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/entry.S | 37 ++++++++++++++++++++++++++++ arch/arm64/kernel/mte.c | 21 ++++++++++++++++ arch/arm64/kernel/process.c | 5 ++++ arch/arm64/kernel/signal.c | 8 ++++++ arch/arm64/kernel/syscall.c | 10 ++++++++ arch/arm64/mm/fault.c | 9 ++++++- 9 files changed, 116 insertions(+), 2 deletions(-) create mode 100644 arch/arm64/include/asm/mte.h create mode 100644 arch/arm64/kernel/mte.c diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h new file mode 100644 index 000000000000..a0bf310da74b --- /dev/null +++ b/arch/arm64/include/asm/mte.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020 ARM Ltd. + */ +#ifndef __ASM_MTE_H +#define __ASM_MTE_H + +#ifndef __ASSEMBLY__ + +#ifdef CONFIG_ARM64_MTE + +void flush_mte_state(void); + +#else + +static inline void flush_mte_state(void) +{ +} + +#endif + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_MTE_H */ diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/as= m/thread_info.h index 6ea8b6a26ae9..c91605faa9cb 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -67,6 +67,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ +#define TIF_MTE_ASYNC_FAULT 6 /* MTE Asynchronous Tag Check Fault */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -95,10 +96,11 @@ void arch_release_task_struct(struct task_struct *tsk= ); #define _TIF_FSCHECK (1 << TIF_FSCHECK) #define _TIF_32BIT (1 << TIF_32BIT) #define _TIF_SVE (1 << TIF_SVE) +#define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) =20 #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK) + _TIF_UPROBE | _TIF_FSCHECK | _TIF_MTE_ASYNC_FAULT) =20 #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 151f28521f1e..cef728a730ae 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_ARM_SDE_INTERFACE) +=3D sdei.o obj-$(CONFIG_ARM64_SSBD) +=3D ssbd.o obj-$(CONFIG_ARM64_PTR_AUTH) +=3D pointer_auth.o obj-$(CONFIG_SHADOW_CALL_STACK) +=3D scs.o +obj-$(CONFIG_ARM64_MTE) +=3D mte.o =20 obj-y +=3D vdso/ probes/ obj-$(CONFIG_COMPAT_VDSO) +=3D vdso32/ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 5304d193c79d..cde127508e38 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -146,6 +146,32 @@ alternative_cb_end #endif .endm =20 + /* Check for MTE asynchronous tag check faults */ + .macro check_mte_async_tcf, flgs, tmp +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 1f +alternative_else_nop_endif + mrs_s \tmp, SYS_TFSRE0_EL1 + tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f + /* Asynchronous TCF occurred for TTBR0 access, set the TI flag */ + orr \flgs, \flgs, #_TIF_MTE_ASYNC_FAULT + str \flgs, [tsk, #TSK_TI_FLAGS] + msr_s SYS_TFSRE0_EL1, xzr +1: +#endif + .endm + + /* Clear the MTE asynchronous tag check faults */ + .macro clear_mte_async_tcf +#ifdef CONFIG_ARM64_MTE +alternative_if ARM64_MTE + dsb ish + msr_s SYS_TFSRE0_EL1, xzr +alternative_else_nop_endif +#endif + .endm + .macro kernel_entry, el, regsize =3D 64 .if \regsize =3D=3D 32 mov w0, w0 // zero upper 32 bits of x0 @@ -177,6 +203,8 @@ alternative_cb_end ldr x19, [tsk, #TSK_TI_FLAGS] disable_step_tsk x19, x20 =20 + /* Check for asynchronous tag check faults in user space */ + check_mte_async_tcf x19, x22 apply_ssbd 1, x22, x23 =20 ptrauth_keys_install_kernel tsk, x20, x22, x23 @@ -247,6 +275,13 @@ alternative_if ARM64_HAS_IRQ_PRIO_MASKING str x20, [sp, #S_PMR_SAVE] alternative_else_nop_endif =20 + /* Re-enable tag checking (TCO set on exception entry) */ +#ifdef CONFIG_ARM64_MTE +alternative_if ARM64_MTE + SET_PSTATE_TCO(0) +alternative_else_nop_endif +#endif + /* * Registers that may be useful after this macro is invoked: * @@ -755,6 +790,8 @@ SYM_CODE_START_LOCAL(ret_to_user) and x2, x1, #_TIF_WORK_MASK cbnz x2, work_pending finish_ret_to_user: + /* Ignore asynchronous tag check faults in the uaccess routines */ + clear_mte_async_tcf enable_step_tsk x1, x2 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c new file mode 100644 index 000000000000..032016823957 --- /dev/null +++ b/arch/arm64/kernel/mte.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 ARM Ltd. + */ + +#include + +#include +#include +#include + +void flush_mte_state(void) +{ + if (!system_supports_mte()) + return; + + /* clear any pending asynchronous tag fault */ + dsb(ish); + write_sysreg_s(0, SYS_TFSRE0_EL1); + clear_thread_flag(TIF_MTE_ASYNC_FAULT); +} diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 6089638c7d43..695705d1f8e5 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -338,6 +339,7 @@ void flush_thread(void) tls_thread_flush(); flush_ptrace_hw_breakpoint(current); flush_tagged_addr_state(); + flush_mte_state(); } =20 void release_thread(struct task_struct *dead_task) @@ -370,6 +372,9 @@ int arch_dup_task_struct(struct task_struct *dst, str= uct task_struct *src) dst->thread.sve_state =3D NULL; clear_tsk_thread_flag(dst, TIF_SVE); =20 + /* clear any pending asynchronous tag fault raised by the parent */ + clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); + return 0; } =20 diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 801d56cdf701..2fe389e5ee4d 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -748,6 +748,9 @@ static void setup_return(struct pt_regs *regs, struct= k_sigaction *ka, regs->pstate |=3D PSR_BTYPE_C; } =20 + /* TCO (Tag Check Override) always cleared for signal handlers */ + regs->pstate &=3D ~PSR_TCO_BIT; + if (ka->sa.sa_flags & SA_RESTORER) sigtramp =3D ka->sa.sa_restorer; else @@ -939,6 +942,11 @@ asmlinkage void do_notify_resume(struct pt_regs *reg= s, if (thread_flags & _TIF_UPROBE) uprobe_notify_resume(regs); =20 + if (thread_flags & _TIF_MTE_ASYNC_FAULT) { + clear_thread_flag(TIF_MTE_ASYNC_FAULT); + send_sig_fault(SIGSEGV, SEGV_MTEAERR, 0, current); + } + if (thread_flags & _TIF_SIGPENDING) do_signal(regs); =20 diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 5f5b868292f5..e4b977e1cf0b 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -120,6 +120,16 @@ static void el0_svc_common(struct pt_regs *regs, int= scno, int sc_nr, local_daif_restore(DAIF_PROCCTX); user_exit(); =20 + if (system_supports_mte() && (flags & _TIF_MTE_ASYNC_FAULT)) { + /* + * Process the asynchronous tag check fault before the actual + * syscall. do_notify_resume() will send a signal to userspace + * before the syscall is restarted. + */ + regs->regs[0] =3D -ERESTARTNOINTR; + return; + } + if (has_syscall_work(flags)) { /* set default errno for user-issued syscall(-1) */ if (scno =3D=3D NO_SYSCALL) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 8afb238ff335..5e832b3387f1 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -658,6 +658,13 @@ static int do_sea(unsigned long addr, unsigned int e= sr, struct pt_regs *regs) return 0; } =20 +static int do_tag_check_fault(unsigned long addr, unsigned int esr, + struct pt_regs *regs) +{ + do_bad_area(addr, esr, regs); + return 0; +} + static const struct fault_info fault_info[] =3D { { do_bad, SIGKILL, SI_KERNEL, "ttbr address size fault" }, { do_bad, SIGKILL, SI_KERNEL, "level 1 address size fault" }, @@ -676,7 +683,7 @@ static const struct fault_info fault_info[] =3D { { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" }, { do_sea, SIGBUS, BUS_OBJERR, "synchronous external abort" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 17" }, + { do_tag_check_fault, SIGSEGV, SEGV_MTESERR, "synchronous tag check fau= lt" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 18" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 19" }, { do_sea, SIGKILL, SI_KERNEL, "level 0 (translation table walk)" },