From: Marco Elver <elver@google.com>
To: elver@google.com, akpm@linux-foundation.org, glider@google.com
Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com,
aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de,
catalin.marinas@arm.com, cl@linux.com,
dave.hansen@linux.intel.com, rientjes@google.com,
dvyukov@google.com, edumazet@google.com,
gregkh@linuxfoundation.org, hdanton@sina.com, mingo@redhat.com,
jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net,
iamjoonsoo.kim@lge.com, joern@purestorage.com,
keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org,
peterz@infradead.org, sjpark@amazon.com, tglx@linutronix.de,
vbabka@suse.cz, will@kernel.org, x86@kernel.org,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org,
linux-mm@kvack.org
Subject: [PATCH v5 3/9] arm64, kfence: enable KFENCE for ARM64
Date: Tue, 27 Oct 2020 15:16:00 +0100 [thread overview]
Message-ID: <20201027141606.426816-4-elver@google.com> (raw)
In-Reply-To: <20201027141606.426816-1-elver@google.com>
Add architecture specific implementation details for KFENCE and enable
KFENCE for the arm64 architecture. In particular, this implements the
required interface in <asm/kfence.h>.
KFENCE requires that attributes for pages from its memory pool can
individually be set. Therefore, force the entire linear map to be mapped
at page granularity. Doing so may result in extra memory allocated for
page tables in case rodata=full is not set; however, currently
CONFIG_RODATA_FULL_DEFAULT_ENABLED=y is the default, and the common case
is therefore not affected by this change.
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Marco Elver <elver@google.com>
---
v5:
* Move generic page allocation code to core.c [suggested by Jann Horn].
* Remove comment about HAVE_ARCH_KFENCE_STATIC_POOL, since we no longer
support static pools.
* Force page granularity for the linear map [suggested by Mark Rutland].
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/kfence.h | 19 +++++++++++++++++++
arch/arm64/mm/fault.c | 4 ++++
arch/arm64/mm/mmu.c | 7 ++++++-
4 files changed, 30 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/include/asm/kfence.h
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index f858c352f72a..2f8b32dddd8b 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -135,6 +135,7 @@ config ARM64
select HAVE_ARCH_JUMP_LABEL_RELATIVE
select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
+ select HAVE_ARCH_KFENCE if (!ARM64_16K_PAGES && !ARM64_64K_PAGES)
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
diff --git a/arch/arm64/include/asm/kfence.h b/arch/arm64/include/asm/kfence.h
new file mode 100644
index 000000000000..5ac0f599cc9a
--- /dev/null
+++ b/arch/arm64/include/asm/kfence.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_KFENCE_H
+#define __ASM_KFENCE_H
+
+#include <asm/cacheflush.h>
+
+#define KFENCE_SKIP_ARCH_FAULT_HANDLER "el1_sync"
+
+static inline bool arch_kfence_init_pool(void) { return true; }
+
+static inline bool kfence_protect_page(unsigned long addr, bool protect)
+{
+ set_memory_valid(addr, 1, !protect);
+
+ return true;
+}
+
+#endif /* __ASM_KFENCE_H */
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 94c99c1c19e3..ec8ed2943484 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -10,6 +10,7 @@
#include <linux/acpi.h>
#include <linux/bitfield.h>
#include <linux/extable.h>
+#include <linux/kfence.h>
#include <linux/signal.h>
#include <linux/mm.h>
#include <linux/hardirq.h>
@@ -312,6 +313,9 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr,
"Ignoring spurious kernel translation fault at virtual address %016lx\n", addr))
return;
+ if (kfence_handle_page_fault(addr))
+ return;
+
if (is_el1_permission_fault(addr, esr, regs)) {
if (esr & ESR_ELx_WNR)
msg = "write to read-only memory";
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 1c0f3e02f731..86be6d1a78ab 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -1449,7 +1449,12 @@ int arch_add_memory(int nid, u64 start, u64 size,
{
int ret, flags = 0;
- if (rodata_full || debug_pagealloc_enabled())
+ /*
+ * KFENCE requires linear map to be mapped at page granularity, so that
+ * it is possible to protect/unprotect single pages in the KFENCE pool.
+ */
+ if (rodata_full || debug_pagealloc_enabled() ||
+ IS_ENABLED(CONFIG_KFENCE))
flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
__create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start),
--
2.29.0.rc2.309.g374f81d7ae-goog
next prev parent reply other threads:[~2020-10-27 14:19 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-27 14:15 [PATCH v5 0/9] KFENCE: A low-overhead sampling-based memory safety error detector Marco Elver
2020-10-27 14:15 ` [PATCH v5 1/9] mm: add Kernel Electric-Fence infrastructure Marco Elver
2020-10-27 14:15 ` [PATCH v5 2/9] x86, kfence: enable KFENCE for x86 Marco Elver
2020-10-27 14:16 ` Marco Elver [this message]
2020-10-27 14:16 ` [PATCH v5 4/9] mm, kfence: insert KFENCE hooks for SLAB Marco Elver
2020-10-27 14:16 ` [PATCH v5 5/9] mm, kfence: insert KFENCE hooks for SLUB Marco Elver
2020-10-27 14:16 ` [PATCH v5 6/9] kfence, kasan: make KFENCE compatible with KASAN Marco Elver
2020-10-27 14:16 ` [PATCH v5 7/9] kfence, Documentation: add KFENCE documentation Marco Elver
2020-10-27 14:16 ` [PATCH v5 8/9] kfence: add test suite Marco Elver
2020-10-27 14:16 ` [PATCH v5 9/9] MAINTAINERS: Add entry for KFENCE Marco Elver
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201027141606.426816-4-elver@google.com \
--to=elver@google.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=andreyknvl@google.com \
--cc=aryabinin@virtuozzo.com \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=dvyukov@google.com \
--cc=edumazet@google.com \
--cc=glider@google.com \
--cc=gregkh@linuxfoundation.org \
--cc=hdanton@sina.com \
--cc=hpa@zytor.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=jannh@google.com \
--cc=joern@purestorage.com \
--cc=kasan-dev@googlegroups.com \
--cc=keescook@chromium.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=penberg@kernel.org \
--cc=peterz@infradead.org \
--cc=rientjes@google.com \
--cc=sjpark@amazon.com \
--cc=tglx@linutronix.de \
--cc=vbabka@suse.cz \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).