From: Andrey Konovalov <andreyknvl@google.com>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>,
Vincenzo Frascino <vincenzo.frascino@arm.com>,
Dmitry Vyukov <dvyukov@google.com>,
Andrey Ryabinin <aryabinin@virtuozzo.com>,
Alexander Potapenko <glider@google.com>,
Marco Elver <elver@google.com>,
Evgenii Stepanov <eugenis@google.com>,
Branislav Rankov <Branislav.Rankov@arm.com>,
Kevin Brodsky <kevin.brodsky@arm.com>,
Andrew Morton <akpm@linux-foundation.org>,
kasan-dev@googlegroups.com,
linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org,
Andrey Konovalov <andreyknvl@google.com>
Subject: [PATCH v9 29/44] arm64: mte: Add in-kernel tag fault handler
Date: Tue, 10 Nov 2020 23:10:26 +0100 [thread overview]
Message-ID: <96694ab6b5b64f4ab2de32cdc4773857966d62f1.1605046192.git.andreyknvl@google.com> (raw)
In-Reply-To: <cover.1605046192.git.andreyknvl@google.com>
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Add the implementation of the in-kernel fault handler.
When a tag fault happens on a kernel address:
* MTE is disabled on the current CPU,
* the execution continues.
When a tag fault happens on a user address:
* the kernel executes do_bad_area() and panics.
The tag fault handler for kernel addresses is currently empty and will be
filled in by a future commit.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I9b8aa79567f7c45f4d6a1290efcf34567e620717
---
arch/arm64/include/asm/uaccess.h | 23 ++++++++++++++++
arch/arm64/mm/fault.c | 45 ++++++++++++++++++++++++++++++++
2 files changed, 68 insertions(+)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 991dd5f031e4..c7fff8daf2a7 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -200,13 +200,36 @@ do { \
CONFIG_ARM64_PAN)); \
} while (0)
+/*
+ * The Tag Check Flag (TCF) mode for MTE is per EL, hence TCF0
+ * affects EL0 and TCF affects EL1 irrespective of which TTBR is
+ * used.
+ * The kernel accesses TTBR0 usually with LDTR/STTR instructions
+ * when UAO is available, so these would act as EL0 accesses using
+ * TCF0.
+ * However futex.h code uses exclusives which would be executed as
+ * EL1, this can potentially cause a tag check fault even if the
+ * user disables TCF0.
+ *
+ * To address the problem we set the PSTATE.TCO bit in uaccess_enable()
+ * and reset it in uaccess_disable().
+ *
+ * The Tag check override (TCO) bit disables temporarily the tag checking
+ * preventing the issue.
+ */
static inline void uaccess_disable(void)
{
+ asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(0),
+ ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
__uaccess_disable(ARM64_HAS_PAN);
}
static inline void uaccess_enable(void)
{
+ asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(1),
+ ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
__uaccess_enable(ARM64_HAS_PAN);
}
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 1ee94002801f..fbceb14d93b1 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -33,6 +33,7 @@
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/kprobes.h>
+#include <asm/mte.h>
#include <asm/processor.h>
#include <asm/sysreg.h>
#include <asm/system_misc.h>
@@ -296,6 +297,44 @@ static void die_kernel_fault(const char *msg, unsigned long addr,
do_exit(SIGKILL);
}
+static void report_tag_fault(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+}
+
+static void do_tag_recovery(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ static bool reported = false;
+
+ if (!READ_ONCE(reported)) {
+ report_tag_fault(addr, esr, regs);
+ WRITE_ONCE(reported, true);
+ }
+
+ /*
+ * Disable MTE Tag Checking on the local CPU for the current EL.
+ * It will be done lazily on the other CPUs when they will hit a
+ * tag fault.
+ */
+ sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_NONE);
+ isb();
+}
+
+static bool is_el1_mte_sync_tag_check_fault(unsigned int esr)
+{
+ unsigned int ec = ESR_ELx_EC(esr);
+ unsigned int fsc = esr & ESR_ELx_FSC;
+
+ if (ec != ESR_ELx_EC_DABT_CUR)
+ return false;
+
+ if (fsc == ESR_ELx_FSC_MTE)
+ return true;
+
+ return false;
+}
+
static void __do_kernel_fault(unsigned long addr, unsigned int esr,
struct pt_regs *regs)
{
@@ -312,6 +351,12 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr,
"Ignoring spurious kernel translation fault at virtual address %016lx\n", addr))
return;
+ if (is_el1_mte_sync_tag_check_fault(esr)) {
+ do_tag_recovery(addr, esr, regs);
+
+ return;
+ }
+
if (is_el1_permission_fault(addr, esr, regs)) {
if (esr & ESR_ELx_WNR)
msg = "write to read-only memory";
--
2.29.2.222.g5d2a92d10f8-goog
next prev parent reply other threads:[~2020-11-10 22:12 UTC|newest]
Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
2020-11-10 22:09 ` [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers Andrey Konovalov
2020-11-11 13:16 ` Alexander Potapenko
2020-11-10 22:09 ` [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrey Konovalov
2020-11-11 13:19 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 03/44] kasan: group vmalloc code Andrey Konovalov
2020-11-11 13:21 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 04/44] s390/kasan: include asm/page.h from asm/kasan.h Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 05/44] kasan: shadow declarations only for software modes Andrey Konovalov
2020-11-11 13:25 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory Andrey Konovalov
2020-11-11 13:26 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrey Konovalov
2020-11-11 13:45 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 08/44] kasan: only build init.c for software modes Andrey Konovalov
2020-11-11 13:46 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 09/44] kasan: split out shadow.c from common.c Andrey Konovalov
2020-11-11 14:07 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE Andrey Konovalov
2020-11-11 14:13 ` Alexander Potapenko
2020-11-11 18:47 ` Andrey Konovalov
2020-11-11 19:04 ` Marco Elver
2020-11-12 0:05 ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 11/44] kasan: rename report and tags files Andrey Konovalov
2020-11-11 14:15 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 12/44] kasan: don't duplicate config dependencies Andrey Konovalov
2020-11-11 14:20 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 13/44] kasan: hide invalid free check implementation Andrey Konovalov
2020-11-11 14:23 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrey Konovalov
2020-11-11 14:25 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 15/44] kasan, arm64: only init shadow for software modes Andrey Konovalov
2020-11-11 14:30 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 16/44] kasan, arm64: only use kasan_depth " Andrey Konovalov
2020-11-11 14:59 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 17/44] kasan, arm64: move initialization message Andrey Konovalov
2020-11-11 15:03 ` Alexander Potapenko
2020-11-11 18:50 ` Andrey Konovalov
2020-11-12 15:09 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init Andrey Konovalov
2020-11-11 15:05 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata Andrey Konovalov
2020-11-11 15:06 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata Andrey Konovalov
2020-11-11 15:07 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes Andrey Konovalov
2020-11-11 15:08 ` Alexander Potapenko
2020-11-11 18:52 ` Andrey Konovalov
2020-11-12 15:16 ` Alexander Potapenko
2020-11-12 19:27 ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 22/44] kasan: rename SHADOW layout macros to META Andrey Konovalov
2020-11-11 15:18 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode Andrey Konovalov
2020-11-11 15:22 ` Alexander Potapenko
2020-11-11 16:19 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE Andrey Konovalov
2020-11-11 15:52 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS Andrey Konovalov
2020-11-11 15:58 ` Alexander Potapenko
2020-11-11 18:53 ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 26/44] arm64: Enable armv8.5-a asm-arch option Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 27/44] arm64: mte: Add in-kernel MTE helpers Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags Andrey Konovalov
2020-11-12 9:31 ` Catalin Marinas
2020-11-12 9:39 ` Vincenzo Frascino
2020-11-10 22:10 ` Andrey Konovalov [this message]
2020-11-10 22:10 ` [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE Andrey Konovalov
2020-11-12 9:43 ` Catalin Marinas
2020-11-12 16:46 ` Vincenzo Frascino
2020-11-13 11:17 ` Vincenzo Frascino
2020-11-13 12:00 ` Catalin Marinas
2020-11-13 12:04 ` Vincenzo Frascino
2020-11-10 22:10 ` [PATCH v9 31/44] arm64: mte: Convert gcr_user into an exclude mask Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit Andrey Konovalov
2020-11-12 9:39 ` Catalin Marinas
2020-11-12 9:45 ` Vincenzo Frascino
2020-11-12 9:45 ` Catalin Marinas
2020-11-12 9:55 ` Vincenzo Frascino
2020-11-10 22:10 ` [PATCH v9 33/44] kasan, mm: untag page address in free_reserved_area Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS Andrey Konovalov
2020-11-11 16:10 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 35/44] arm64: kasan: Add arch layer for memory tagging helpers Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrey Konovalov
2020-11-11 16:13 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN Andrey Konovalov
2020-11-11 16:15 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks Andrey Konovalov
2020-11-11 16:22 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime Andrey Konovalov
2020-11-12 17:37 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 40/44] kasan, arm64: print report from tag fault handler Andrey Konovalov
2020-11-11 16:43 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata Andrey Konovalov
2020-11-12 16:10 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrey Konovalov
2020-11-11 16:49 ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode Andrey Konovalov
2020-11-11 16:57 ` Alexander Potapenko
2020-11-11 18:54 ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch Andrey Konovalov
2020-11-12 9:46 ` Catalin Marinas
2020-11-12 15:59 ` Alexander Potapenko
2020-11-12 16:09 ` Marco Elver
2020-11-12 16:41 ` Alexander Potapenko
2020-11-13 11:47 ` Vincenzo Frascino
2020-11-10 22:54 ` [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrew Morton
2020-11-11 14:35 ` Andrey Konovalov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=96694ab6b5b64f4ab2de32cdc4773857966d62f1.1605046192.git.andreyknvl@google.com \
--to=andreyknvl@google.com \
--cc=Branislav.Rankov@arm.com \
--cc=akpm@linux-foundation.org \
--cc=aryabinin@virtuozzo.com \
--cc=catalin.marinas@arm.com \
--cc=dvyukov@google.com \
--cc=elver@google.com \
--cc=eugenis@google.com \
--cc=glider@google.com \
--cc=kasan-dev@googlegroups.com \
--cc=kevin.brodsky@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=vincenzo.frascino@arm.com \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).