All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, andreyknvl@google.com,
	aryabinin@virtuozzo.com, Branislav.Rankov@arm.com,
	catalin.marinas@arm.com, dvyukov@google.com, elver@google.com,
	eugenis@google.com, glider@google.com, gor@linux.ibm.com,
	kevin.brodsky@arm.com, linux-mm@kvack.org,
	mm-commits@vger.kernel.org, torvalds@linux-foundation.org,
	vincenzo.frascino@arm.com, will.deacon@arm.com
Subject: [patch 39/78] arm64: mte: add in-kernel tag fault handler
Date: Fri, 18 Dec 2020 14:03:33 -0800	[thread overview]
Message-ID: <20201218220333.9v99rO2V0%akpm@linux-foundation.org> (raw)
In-Reply-To: <20201218140046.497484741326828e5b5d46ec@linux-foundation.org>

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: mte: add in-kernel tag fault handler

Add the implementation of the in-kernel fault handler.

When a tag fault happens on a kernel address:
* MTE is disabled on the current CPU,
* the execution continues.

When a tag fault happens on a user address:
* the kernel executes do_bad_area() and panics.

The tag fault handler for kernel addresses is currently empty and will be
filled in by a future commit.

Link: https://lkml.kernel.org/r/ad31529b073e22840b7a2246172c2b67747ed7c4.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
[catalin.marinas@arm.com: ensure CONFIG_ARM64_PAN is enabled with MTE]
  Link: https://lkml.kernel.org/r/20201203102628.GB2224@gaia
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/Kconfig               |    2 +
 arch/arm64/include/asm/uaccess.h |   23 ++++++++++++++
 arch/arm64/mm/fault.c            |   45 +++++++++++++++++++++++++++++
 3 files changed, 70 insertions(+)

--- a/arch/arm64/include/asm/uaccess.h~arm64-mte-add-in-kernel-tag-fault-handler
+++ a/arch/arm64/include/asm/uaccess.h
@@ -159,8 +159,28 @@ static inline void __uaccess_enable_hw_p
 			CONFIG_ARM64_PAN));
 }
 
+/*
+ * The Tag Check Flag (TCF) mode for MTE is per EL, hence TCF0
+ * affects EL0 and TCF affects EL1 irrespective of which TTBR is
+ * used.
+ * The kernel accesses TTBR0 usually with LDTR/STTR instructions
+ * when UAO is available, so these would act as EL0 accesses using
+ * TCF0.
+ * However futex.h code uses exclusives which would be executed as
+ * EL1, this can potentially cause a tag check fault even if the
+ * user disables TCF0.
+ *
+ * To address the problem we set the PSTATE.TCO bit in uaccess_enable()
+ * and reset it in uaccess_disable().
+ *
+ * The Tag check override (TCO) bit disables temporarily the tag checking
+ * preventing the issue.
+ */
 static inline void uaccess_disable_privileged(void)
 {
+	asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(0),
+				 ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
 	if (uaccess_ttbr0_disable())
 		return;
 
@@ -169,6 +189,9 @@ static inline void uaccess_disable_privi
 
 static inline void uaccess_enable_privileged(void)
 {
+	asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(1),
+				 ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
 	if (uaccess_ttbr0_enable())
 		return;
 
--- a/arch/arm64/Kconfig~arm64-mte-add-in-kernel-tag-fault-handler
+++ a/arch/arm64/Kconfig
@@ -1649,6 +1649,8 @@ config ARM64_MTE
 	default y
 	depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI
 	depends on AS_HAS_ARMV8_5
+	# Required for tag checking in the uaccess routines
+	depends on ARM64_PAN
 	select ARCH_USES_HIGH_VMA_FLAGS
 	help
 	  Memory Tagging (part of the ARMv8.5 Extensions) provides
--- a/arch/arm64/mm/fault.c~arm64-mte-add-in-kernel-tag-fault-handler
+++ a/arch/arm64/mm/fault.c
@@ -33,6 +33,7 @@
 #include <asm/debug-monitors.h>
 #include <asm/esr.h>
 #include <asm/kprobes.h>
+#include <asm/mte.h>
 #include <asm/processor.h>
 #include <asm/sysreg.h>
 #include <asm/system_misc.h>
@@ -296,6 +297,44 @@ static void die_kernel_fault(const char
 	do_exit(SIGKILL);
 }
 
+static void report_tag_fault(unsigned long addr, unsigned int esr,
+			     struct pt_regs *regs)
+{
+}
+
+static void do_tag_recovery(unsigned long addr, unsigned int esr,
+			   struct pt_regs *regs)
+{
+	static bool reported;
+
+	if (!READ_ONCE(reported)) {
+		report_tag_fault(addr, esr, regs);
+		WRITE_ONCE(reported, true);
+	}
+
+	/*
+	 * Disable MTE Tag Checking on the local CPU for the current EL.
+	 * It will be done lazily on the other CPUs when they will hit a
+	 * tag fault.
+	 */
+	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_NONE);
+	isb();
+}
+
+static bool is_el1_mte_sync_tag_check_fault(unsigned int esr)
+{
+	unsigned int ec = ESR_ELx_EC(esr);
+	unsigned int fsc = esr & ESR_ELx_FSC;
+
+	if (ec != ESR_ELx_EC_DABT_CUR)
+		return false;
+
+	if (fsc == ESR_ELx_FSC_MTE)
+		return true;
+
+	return false;
+}
+
 static void __do_kernel_fault(unsigned long addr, unsigned int esr,
 			      struct pt_regs *regs)
 {
@@ -312,6 +351,12 @@ static void __do_kernel_fault(unsigned l
 	    "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr))
 		return;
 
+	if (is_el1_mte_sync_tag_check_fault(esr)) {
+		do_tag_recovery(addr, esr, regs);
+
+		return;
+	}
+
 	if (is_el1_permission_fault(addr, esr, regs)) {
 		if (esr & ESR_ELx_WNR)
 			msg = "write to read-only memory";
_

  parent reply	other threads:[~2020-12-18 22:03 UTC|newest]

Thread overview: 98+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-18 22:00 incoming Andrew Morton
2020-12-18 22:01 ` [patch 01/78] mm/memcg: bail early from swap accounting if memcg disabled Andrew Morton
2020-12-18 22:01 ` [patch 02/78] mm/memcg: warning on !memcg after readahead page charged Andrew Morton
2020-12-18 22:01 ` [patch 03/78] mm/memcg: remove unused definitions Andrew Morton
2020-12-18 22:01 ` [patch 04/78] mm, kvm: account kvm_vcpu_mmap to kmemcg Andrew Morton
2020-12-18 22:01 ` [patch 05/78] mm/memcontrol:rewrite mem_cgroup_page_lruvec() Andrew Morton
2020-12-18 22:01 ` [patch 06/78] epoll: check for events when removing a timed out thread from the wait queue Andrew Morton
2020-12-18 22:01 ` [patch 07/78] epoll: simplify signal handling Andrew Morton
2020-12-18 22:01 ` [patch 08/78] epoll: pull fatal signal checks into ep_send_events() Andrew Morton
2020-12-18 22:01 ` [patch 09/78] epoll: move eavail next to the list_empty_careful check Andrew Morton
2020-12-18 22:01 ` [patch 10/78] epoll: simplify and optimize busy loop logic Andrew Morton
2020-12-18 22:02 ` [patch 11/78] epoll: pull all code between fetch_events and send_event into the loop Andrew Morton
2020-12-18 22:02 ` [patch 12/78] epoll: replace gotos with a proper loop Andrew Morton
2020-12-18 22:02 ` [patch 13/78] epoll: eliminate unnecessary lock for zero timeout Andrew Morton
2020-12-18 22:02 ` [patch 14/78] kasan: drop unnecessary GPL text from comment headers Andrew Morton
2020-12-18 22:02 ` [patch 15/78] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrew Morton
2020-12-18 22:02 ` [patch 16/78] kasan: group vmalloc code Andrew Morton
2020-12-18 22:02 ` [patch 17/78] kasan: shadow declarations only for software modes Andrew Morton
2020-12-18 22:02 ` [patch 18/78] kasan: rename (un)poison_shadow to (un)poison_range Andrew Morton
2020-12-18 22:02 ` [patch 19/78] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrew Morton
2020-12-18 22:02 ` [patch 20/78] kasan: only build init.c for software modes Andrew Morton
2020-12-18 22:02 ` [patch 21/78] kasan: split out shadow.c from common.c Andrew Morton
2020-12-19  0:28   ` Marco Elver
2020-12-19  1:13     ` Andrew Morton
2020-12-19 10:01       ` Marco Elver
2020-12-19 10:01         ` Marco Elver
2020-12-19 10:11       ` Marco Elver
2020-12-19 10:11         ` Marco Elver
2020-12-19 18:01       ` Andrey Konovalov
2020-12-19 18:01         ` Andrey Konovalov
2020-12-19 19:17       ` Linus Torvalds
2020-12-19 19:17         ` Linus Torvalds
2020-12-19 19:26         ` Linus Torvalds
2020-12-19 19:26           ` Linus Torvalds
2020-12-21  9:46         ` Alexander Potapenko
2020-12-21  9:46           ` Alexander Potapenko
2020-12-21 17:41           ` Linus Torvalds
2020-12-21 17:41             ` Linus Torvalds
2020-12-22 18:38             ` Andrew Morton
2020-12-22 12:00   ` kernel test robot
2020-12-18 22:02 ` [patch 22/78] kasan: define KASAN_MEMORY_PER_SHADOW_PAGE Andrew Morton
2020-12-18 22:02 ` [patch 23/78] kasan: rename report and tags files Andrew Morton
2020-12-18 22:02 ` [patch 24/78] kasan: don't duplicate config dependencies Andrew Morton
2020-12-18 22:02 ` [patch 25/78] kasan: hide invalid free check implementation Andrew Morton
2020-12-18 22:02 ` [patch 26/78] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrew Morton
2020-12-18 22:02 ` [patch 27/78] kasan, arm64: only init shadow for software modes Andrew Morton
2020-12-18 22:02 ` [patch 28/78] kasan, arm64: only use kasan_depth " Andrew Morton
2020-12-18 22:03 ` [patch 29/78] kasan, arm64: move initialization message Andrew Morton
2020-12-18 22:03 ` [patch 30/78] kasan, arm64: rename kasan_init_tags and mark as __init Andrew Morton
2020-12-18 22:03 ` [patch 31/78] kasan: rename addr_has_shadow to addr_has_metadata Andrew Morton
2020-12-18 22:03 ` [patch 32/78] kasan: rename print_shadow_for_address to print_memory_metadata Andrew Morton
2020-12-18 22:03 ` [patch 33/78] kasan: rename SHADOW layout macros to META Andrew Morton
2020-12-18 22:03 ` [patch 34/78] kasan: separate metadata_fetch_row for each mode Andrew Morton
2020-12-18 22:03 ` [patch 35/78] kasan: introduce CONFIG_KASAN_HW_TAGS Andrew Morton
2020-12-18 22:03 ` [patch 36/78] arm64: enable armv8.5-a asm-arch option Andrew Morton
2020-12-18 22:03 ` [patch 37/78] arm64: mte: add in-kernel MTE helpers Andrew Morton
2020-12-18 22:03 ` [patch 38/78] arm64: mte: reset the page tag in page->flags Andrew Morton
2020-12-18 22:03 ` Andrew Morton [this message]
2020-12-18 22:03 ` [patch 40/78] arm64: kasan: allow enabling in-kernel MTE Andrew Morton
2020-12-18 22:03 ` [patch 41/78] arm64: mte: convert gcr_user into an exclude mask Andrew Morton
2020-12-18 22:03 ` [patch 42/78] arm64: mte: switch GCR_EL1 in kernel entry and exit Andrew Morton
2020-12-18 22:03 ` [patch 43/78] kasan, mm: untag page address in free_reserved_area Andrew Morton
2020-12-18 22:03 ` [patch 44/78] arm64: kasan: align allocations for HW_TAGS Andrew Morton
2020-12-18 22:03 ` [patch 45/78] arm64: kasan: add arch layer for memory tagging helpers Andrew Morton
2020-12-18 22:03 ` [patch 46/78] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrew Morton
2020-12-18 22:03 ` [patch 47/78] kasan, x86, s390: update undef CONFIG_KASAN Andrew Morton
2020-12-18 22:04 ` [patch 48/78] kasan, arm64: expand CONFIG_KASAN checks Andrew Morton
2020-12-18 22:04 ` [patch 49/78] kasan, arm64: implement HW_TAGS runtime Andrew Morton
2020-12-18 22:04 ` [patch 50/78] kasan, arm64: print report from tag fault handler Andrew Morton
2020-12-18 22:04 ` [patch 51/78] kasan, mm: reset tags when accessing metadata Andrew Morton
2020-12-18 22:04 ` [patch 52/78] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrew Morton
2020-12-18 22:04 ` [patch 53/78] kasan: add documentation for hardware tag-based mode Andrew Morton
2020-12-18 22:04 ` [patch 54/78] kselftest/arm64: check GCR_EL1 after context switch Andrew Morton
2020-12-18 22:04 ` [patch 55/78] kasan: simplify quarantine_put call site Andrew Morton
2020-12-18 22:04 ` [patch 56/78] kasan: rename get_alloc/free_info Andrew Morton
2020-12-18 22:04 ` [patch 57/78] kasan: introduce set_alloc_info Andrew Morton
2020-12-18 22:04 ` [patch 58/78] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK Andrew Morton
2020-12-18 22:04 ` [patch 59/78] kasan: allow VMAP_STACK for HW_TAGS mode Andrew Morton
2020-12-18 22:04 ` [patch 60/78] kasan: remove __kasan_unpoison_stack Andrew Morton
2020-12-18 22:04 ` [patch 61/78] kasan: inline kasan_reset_tag for tag-based modes Andrew Morton
2020-12-18 22:04 ` [patch 62/78] kasan: inline random_tag for HW_TAGS Andrew Morton
2020-12-18 22:04 ` [patch 63/78] kasan: open-code kasan_unpoison_slab Andrew Morton
2020-12-18 22:04 ` [patch 64/78] kasan: inline (un)poison_range and check_invalid_free Andrew Morton
2020-12-22 14:02   ` kernel test robot
2020-12-18 22:05 ` [patch 65/78] kasan: add and integrate kasan boot parameters Andrew Morton
2020-12-18 22:05 ` [patch 66/78] kasan, mm: check kasan_enabled in annotations Andrew Morton
2020-12-18 22:05 ` [patch 67/78] kasan, mm: rename kasan_poison_kfree Andrew Morton
2020-12-18 22:05 ` [patch 68/78] kasan: don't round_up too much Andrew Morton
2020-12-18 22:05 ` [patch 69/78] kasan: simplify assign_tag and set_tag calls Andrew Morton
2020-12-18 22:05 ` [patch 70/78] kasan: clarify comment in __kasan_kfree_large Andrew Morton
2020-12-18 22:05 ` [patch 71/78] kasan: sanitize objects when metadata doesn't fit Andrew Morton
2020-12-18 22:05 ` [patch 72/78] kasan, mm: allow cache merging with no metadata Andrew Morton
2020-12-18 22:05 ` [patch 73/78] kasan: update documentation Andrew Morton
2020-12-18 22:05 ` [patch 74/78] mm/Kconfig: fix spelling mistake "whats" -> "what's" Andrew Morton
2020-12-18 22:05 ` [patch 75/78] epoll: convert internal api to timespec64 Andrew Morton
2020-12-18 22:05 ` [patch 76/78] epoll: add syscall epoll_pwait2 Andrew Morton
2020-12-18 22:05 ` [patch 77/78] epoll: wire up " Andrew Morton
2020-12-18 22:05 ` [patch 78/78] selftests/filesystems: expand epoll with epoll_pwait2 Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201218220333.9v99rO2V0%akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=Branislav.Rankov@arm.com \
    --cc=andreyknvl@google.com \
    --cc=aryabinin@virtuozzo.com \
    --cc=catalin.marinas@arm.com \
    --cc=dvyukov@google.com \
    --cc=elver@google.com \
    --cc=eugenis@google.com \
    --cc=glider@google.com \
    --cc=gor@linux.ibm.com \
    --cc=kevin.brodsky@arm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mm-commits@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=vincenzo.frascino@arm.com \
    --cc=will.deacon@arm.com \
    --subject='Re: [patch 39/78] arm64: mte: add in-kernel tag fault handler' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.