From: Marco Elver <elver@google.com> To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com, aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de, catalin.marinas@arm.com, cl@linux.com, dave.hansen@linux.intel.com, rientjes@google.com, dvyukov@google.com, edumazet@google.com, gregkh@linuxfoundation.org, hdanton@sina.com, mingo@redhat.com, jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net, iamjoonsoo.kim@lge.com, keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org, peterz@infradead.org, sjpark@amazon.com, tglx@linutronix.de, vbabka@suse.cz, will@kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org Subject: [PATCH v4 02/11] x86, kfence: enable KFENCE for x86 Date: Tue, 29 Sep 2020 15:38:05 +0200 [thread overview] Message-ID: <20200929133814.2834621-3-elver@google.com> (raw) In-Reply-To: <20200929133814.2834621-1-elver@google.com> From: Alexander Potapenko <glider@google.com> Add architecture specific implementation details for KFENCE and enable KFENCE for the x86 architecture. In particular, this implements the required interface in <asm/kfence.h> for setting up the pool and providing helper functions for protecting and unprotecting pages. For x86, we need to ensure that the pool uses 4K pages, which is done using the set_memory_4k() helper function. Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Co-developed-by: Marco Elver <elver@google.com> Signed-off-by: Marco Elver <elver@google.com> Signed-off-by: Alexander Potapenko <glider@google.com> --- v4: * Define __kfence_pool_attrs. --- arch/x86/Kconfig | 2 ++ arch/x86/include/asm/kfence.h | 60 +++++++++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 4 +++ 3 files changed, 66 insertions(+) create mode 100644 arch/x86/include/asm/kfence.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7101ac64bb20..e22dc722698c 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -144,6 +144,8 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KFENCE + select HAVE_ARCH_KFENCE_STATIC_POOL select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT diff --git a/arch/x86/include/asm/kfence.h b/arch/x86/include/asm/kfence.h new file mode 100644 index 000000000000..98fb1cd80026 --- /dev/null +++ b/arch/x86/include/asm/kfence.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_X86_KFENCE_H +#define _ASM_X86_KFENCE_H + +#include <linux/bug.h> +#include <linux/kfence.h> + +#include <asm/pgalloc.h> +#include <asm/pgtable.h> +#include <asm/set_memory.h> +#include <asm/tlbflush.h> + +/* The alignment should be at least a 4K page. */ +#define __kfence_pool_attrs __aligned(PAGE_SIZE) + +/* + * The page fault handler entry function, up to which the stack trace is + * truncated in reports. + */ +#define KFENCE_SKIP_ARCH_FAULT_HANDLER "asm_exc_page_fault" + +/* Force 4K pages for __kfence_pool. */ +static inline bool arch_kfence_initialize_pool(void) +{ + unsigned long addr; + + for (addr = (unsigned long)__kfence_pool; is_kfence_address((void *)addr); + addr += PAGE_SIZE) { + unsigned int level; + + if (!lookup_address(addr, &level)) + return false; + + if (level != PG_LEVEL_4K) + set_memory_4k(addr, 1); + } + + return true; +} + +/* Protect the given page and flush TLBs. */ +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + unsigned int level; + pte_t *pte = lookup_address(addr, &level); + + if (!pte || level != PG_LEVEL_4K) + return false; + + if (protect) + set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_PRESENT)); + else + set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT)); + + flush_tlb_one_kernel(addr); + return true; +} + +#endif /* _ASM_X86_KFENCE_H */ diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 6e3e8a124903..423e15ad5eb6 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -9,6 +9,7 @@ #include <linux/kdebug.h> /* oops_begin/end, ... */ #include <linux/extable.h> /* search_exception_tables */ #include <linux/memblock.h> /* max_low_pfn */ +#include <linux/kfence.h> /* kfence_handle_page_fault */ #include <linux/kprobes.h> /* NOKPROBE_SYMBOL, ... */ #include <linux/mmiotrace.h> /* kmmio_handler, ... */ #include <linux/perf_event.h> /* perf_sw_event */ @@ -701,6 +702,9 @@ no_context(struct pt_regs *regs, unsigned long error_code, } #endif + if (kfence_handle_page_fault(address)) + return; + /* * 32-bit: * -- 2.28.0.709.gb0816b6eb0-goog
WARNING: multiple messages have this Message-ID (diff)
From: Marco Elver <elver@google.com> To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: mark.rutland@arm.com, hdanton@sina.com, linux-doc@vger.kernel.org, peterz@infradead.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, edumazet@google.com, hpa@zytor.com, cl@linux.com, will@kernel.org, sjpark@amazon.com, corbet@lwn.net, x86@kernel.org, kasan-dev@googlegroups.com, mingo@redhat.com, vbabka@suse.cz, rientjes@google.com, aryabinin@virtuozzo.com, keescook@chromium.org, paulmck@kernel.org, jannh@google.com, andreyknvl@google.com, bp@alien8.de, luto@kernel.org, Jonathan.Cameron@huawei.com, tglx@linutronix.de, dvyukov@google.com, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, penberg@kernel.org, iamjoonsoo.kim@lge.com Subject: [PATCH v4 02/11] x86, kfence: enable KFENCE for x86 Date: Tue, 29 Sep 2020 15:38:05 +0200 [thread overview] Message-ID: <20200929133814.2834621-3-elver@google.com> (raw) In-Reply-To: <20200929133814.2834621-1-elver@google.com> From: Alexander Potapenko <glider@google.com> Add architecture specific implementation details for KFENCE and enable KFENCE for the x86 architecture. In particular, this implements the required interface in <asm/kfence.h> for setting up the pool and providing helper functions for protecting and unprotecting pages. For x86, we need to ensure that the pool uses 4K pages, which is done using the set_memory_4k() helper function. Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Co-developed-by: Marco Elver <elver@google.com> Signed-off-by: Marco Elver <elver@google.com> Signed-off-by: Alexander Potapenko <glider@google.com> --- v4: * Define __kfence_pool_attrs. --- arch/x86/Kconfig | 2 ++ arch/x86/include/asm/kfence.h | 60 +++++++++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 4 +++ 3 files changed, 66 insertions(+) create mode 100644 arch/x86/include/asm/kfence.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7101ac64bb20..e22dc722698c 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -144,6 +144,8 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KFENCE + select HAVE_ARCH_KFENCE_STATIC_POOL select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT diff --git a/arch/x86/include/asm/kfence.h b/arch/x86/include/asm/kfence.h new file mode 100644 index 000000000000..98fb1cd80026 --- /dev/null +++ b/arch/x86/include/asm/kfence.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_X86_KFENCE_H +#define _ASM_X86_KFENCE_H + +#include <linux/bug.h> +#include <linux/kfence.h> + +#include <asm/pgalloc.h> +#include <asm/pgtable.h> +#include <asm/set_memory.h> +#include <asm/tlbflush.h> + +/* The alignment should be at least a 4K page. */ +#define __kfence_pool_attrs __aligned(PAGE_SIZE) + +/* + * The page fault handler entry function, up to which the stack trace is + * truncated in reports. + */ +#define KFENCE_SKIP_ARCH_FAULT_HANDLER "asm_exc_page_fault" + +/* Force 4K pages for __kfence_pool. */ +static inline bool arch_kfence_initialize_pool(void) +{ + unsigned long addr; + + for (addr = (unsigned long)__kfence_pool; is_kfence_address((void *)addr); + addr += PAGE_SIZE) { + unsigned int level; + + if (!lookup_address(addr, &level)) + return false; + + if (level != PG_LEVEL_4K) + set_memory_4k(addr, 1); + } + + return true; +} + +/* Protect the given page and flush TLBs. */ +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + unsigned int level; + pte_t *pte = lookup_address(addr, &level); + + if (!pte || level != PG_LEVEL_4K) + return false; + + if (protect) + set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_PRESENT)); + else + set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT)); + + flush_tlb_one_kernel(addr); + return true; +} + +#endif /* _ASM_X86_KFENCE_H */ diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 6e3e8a124903..423e15ad5eb6 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -9,6 +9,7 @@ #include <linux/kdebug.h> /* oops_begin/end, ... */ #include <linux/extable.h> /* search_exception_tables */ #include <linux/memblock.h> /* max_low_pfn */ +#include <linux/kfence.h> /* kfence_handle_page_fault */ #include <linux/kprobes.h> /* NOKPROBE_SYMBOL, ... */ #include <linux/mmiotrace.h> /* kmmio_handler, ... */ #include <linux/perf_event.h> /* perf_sw_event */ @@ -701,6 +702,9 @@ no_context(struct pt_regs *regs, unsigned long error_code, } #endif + if (kfence_handle_page_fault(address)) + return; + /* * 32-bit: * -- 2.28.0.709.gb0816b6eb0-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-09-29 13:39 UTC|newest] Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-29 13:38 [PATCH v4 00/11] KFENCE: A low-overhead sampling-based memory safety error detector Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 01/11] mm: add Kernel Electric-Fence infrastructure Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-10-02 6:33 ` Jann Horn 2020-10-02 6:33 ` Jann Horn 2020-10-02 6:33 ` Jann Horn 2020-10-02 7:53 ` Jann Horn 2020-10-02 7:53 ` Jann Horn 2020-10-02 7:53 ` Jann Horn 2020-10-02 14:22 ` Dmitry Vyukov 2020-10-02 14:22 ` Dmitry Vyukov 2020-10-02 14:22 ` Dmitry Vyukov 2020-10-02 15:06 ` Mark Rutland 2020-10-02 15:06 ` Mark Rutland 2020-10-02 18:27 ` Jann Horn 2020-10-02 18:27 ` Jann Horn 2020-10-02 18:27 ` Jann Horn 2020-10-05 18:59 ` Marco Elver 2020-10-05 18:59 ` Marco Elver 2020-10-05 18:59 ` Marco Elver 2020-10-02 17:19 ` Marco Elver 2020-10-02 17:19 ` Marco Elver 2020-10-02 19:31 ` Jann Horn 2020-10-02 19:31 ` Jann Horn 2020-10-02 19:31 ` Jann Horn 2020-10-02 21:12 ` Marco Elver 2020-10-02 21:12 ` Marco Elver 2020-10-02 21:28 ` Marco Elver 2020-10-02 21:28 ` Marco Elver 2020-10-02 21:28 ` Marco Elver 2020-10-02 22:27 ` Jann Horn 2020-10-02 22:27 ` Jann Horn 2020-10-02 22:27 ` Jann Horn 2020-10-12 14:20 ` Marco Elver 2020-10-12 14:20 ` Marco Elver 2020-10-12 14:20 ` Marco Elver 2020-09-29 13:38 ` Marco Elver [this message] 2020-09-29 13:38 ` [PATCH v4 02/11] x86, kfence: enable KFENCE for x86 Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-10-02 5:45 ` Jann Horn 2020-10-02 5:45 ` Jann Horn 2020-10-02 5:45 ` Jann Horn 2020-10-07 13:08 ` Marco Elver 2020-10-07 13:08 ` Marco Elver 2020-10-07 13:08 ` Marco Elver 2020-10-07 14:14 ` Jann Horn 2020-10-07 14:14 ` Jann Horn 2020-10-07 14:14 ` Jann Horn 2020-10-07 14:41 ` Marco Elver 2020-10-07 14:41 ` Marco Elver 2020-10-07 14:41 ` Marco Elver 2020-10-09 17:40 ` Marco Elver 2020-10-09 17:40 ` Marco Elver 2020-10-02 6:08 ` Jann Horn 2020-10-02 6:08 ` Jann Horn 2020-10-02 6:08 ` Jann Horn 2020-09-29 13:38 ` [PATCH v4 03/11] arm64, kfence: enable KFENCE for ARM64 Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-10-02 6:47 ` Jann Horn 2020-10-02 6:47 ` Jann Horn 2020-10-02 6:47 ` Jann Horn 2020-10-02 14:18 ` Marco Elver 2020-10-02 14:18 ` Marco Elver 2020-10-02 14:18 ` Marco Elver 2020-10-02 16:10 ` Jann Horn 2020-10-02 16:10 ` Jann Horn 2020-10-02 16:10 ` Jann Horn 2020-09-29 13:38 ` [PATCH v4 04/11] mm, kfence: insert KFENCE hooks for SLAB Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 05/11] mm, kfence: insert KFENCE hooks for SLUB Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-10-02 7:07 ` Jann Horn 2020-10-02 7:07 ` Jann Horn 2020-10-02 7:07 ` Jann Horn 2020-10-05 9:29 ` Alexander Potapenko 2020-10-05 9:29 ` Alexander Potapenko 2020-10-05 9:29 ` Alexander Potapenko 2020-09-29 13:38 ` [PATCH v4 06/11] kfence, kasan: make KFENCE compatible with KASAN Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 07/11] kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 08/11] kfence, lockdep: make KFENCE compatible with lockdep Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 09/11] kfence, Documentation: add KFENCE documentation Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 10/11] kfence: add test suite Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` [PATCH v4 11/11] MAINTAINERS: Add entry for KFENCE Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 13:38 ` Marco Elver 2020-09-29 14:21 ` SeongJae Park 2020-09-29 14:21 ` SeongJae Park
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200929133814.2834621-3-elver@google.com \ --to=elver@google.com \ --cc=Jonathan.Cameron@huawei.com \ --cc=akpm@linux-foundation.org \ --cc=andreyknvl@google.com \ --cc=aryabinin@virtuozzo.com \ --cc=bp@alien8.de \ --cc=catalin.marinas@arm.com \ --cc=cl@linux.com \ --cc=corbet@lwn.net \ --cc=dave.hansen@linux.intel.com \ --cc=dvyukov@google.com \ --cc=edumazet@google.com \ --cc=glider@google.com \ --cc=gregkh@linuxfoundation.org \ --cc=hdanton@sina.com \ --cc=hpa@zytor.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=jannh@google.com \ --cc=kasan-dev@googlegroups.com \ --cc=keescook@chromium.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=luto@kernel.org \ --cc=mark.rutland@arm.com \ --cc=mingo@redhat.com \ --cc=paulmck@kernel.org \ --cc=penberg@kernel.org \ --cc=peterz@infradead.org \ --cc=rientjes@google.com \ --cc=sjpark@amazon.com \ --cc=tglx@linutronix.de \ --cc=vbabka@suse.cz \ --cc=will@kernel.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.