From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F4B8C2D0E3 for ; Wed, 16 Sep 2020 00:30:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0D32B206DB for ; Wed, 16 Sep 2020 00:30:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="o71HNdv6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726767AbgIPAaC (ORCPT ); Tue, 15 Sep 2020 20:30:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726456AbgIONrx (ORCPT ); Tue, 15 Sep 2020 09:47:53 -0400 Received: from mail-wr1-x44a.google.com (mail-wr1-x44a.google.com [IPv6:2a00:1450:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFC6FC061220 for ; Tue, 15 Sep 2020 06:21:14 -0700 (PDT) Received: by mail-wr1-x44a.google.com with SMTP id l15so1218477wro.10 for ; Tue, 15 Sep 2020 06:21:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=SxvDeyNmcNMA+Gji2vA/rr03EMZqbUrEGQF/tm1mQmw=; b=o71HNdv6DA6PlBzIM+uaF8z2m3O6QnjR7fFrIwTk8hGwPsQoHPNfBsi4j6RVOTV6rB QGcF2GHmu301W9XcwzPaoDPBszh7HLN2r1OIiFtQRGVFZnqyA2WsrMM1zgh6JFiNCqJr NvzWd/WwOgV6fmMR5uq/A4qmXqraWPjGFNRE6s/puLWVA8MCLDtoQoDvPZ0kQxE1qChb IOCBoE3v1vWdNeoekoIvAvNugpkzCPHfHoTpQXIAxPwGsvUJdJlFijp3tI0jQ0k3V4t2 0FXj8e4gRWaJ3ZkSy6Yet3Tywb4OQfFmJbFzAps6z9Hko0ZQIsPi6+rl+DUaj1LgN5Wt MyGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SxvDeyNmcNMA+Gji2vA/rr03EMZqbUrEGQF/tm1mQmw=; b=g4I4w3BYKylt9xNCTqWb9LWqkEUmXmJuHZ0BRfM+I3cWuEw8XHohoSboSRdBQvJlay Hk27HJSmntk5AGNqpf2QczX06VkPL1Fl0J+QXhS2Z6/vmwt1byRZYTyPmLwYA2I0ZfKL AZqogSBFCxMWzCNKofiHNde8eSI5aN8hCtarrKxiQ6Gj0Mq8kRROsJNvLnCcmYwLxRD7 2TqCN0TLJmWqiWtzg1BrDgOIh3HSbGSI8viAV5aKX2NjzBae/8MOPZcZwRn+NM4XS4gV UxLWHodK+neZj3U2lr4pmAzfFQokvLUm6qpAIyaaDx1gHUF8kMEab0MOLntSZ2VNF/A7 gUBA== X-Gm-Message-State: AOAM532F81/f9FRzC43AbBIvgsU5OFfj9nkeaQuveCjHeEro8kulfcNz v4jXaB7ohKQDo5FUvXJAbYS0QRHNTA== X-Google-Smtp-Source: ABdhPJwl0RFLWfJGwXPTH8kxWe6H1Id6npGQWxX1hTs9KXzpJGcEjMyA8kMwTptUfAmNgcKzKeCOFQa4gQ== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:adf:8b1d:: with SMTP id n29mr20637102wra.383.1600176065852; Tue, 15 Sep 2020 06:21:05 -0700 (PDT) Date: Tue, 15 Sep 2020 15:20:39 +0200 In-Reply-To: <20200915132046.3332537-1-elver@google.com> Message-Id: <20200915132046.3332537-4-elver@google.com> Mime-Version: 1.0 References: <20200915132046.3332537-1-elver@google.com> X-Mailer: git-send-email 2.28.0.618.gf4bc123cb7-goog Subject: [PATCH v2 03/10] arm64, kfence: enable KFENCE for ARM64 From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com, aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de, catalin.marinas@arm.com, cl@linux.com, dave.hansen@linux.intel.com, rientjes@google.com, dvyukov@google.com, edumazet@google.com, gregkh@linuxfoundation.org, mingo@redhat.com, jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net, iamjoonsoo.kim@lge.com, keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org, peterz@infradead.org, cai@lca.pw, tglx@linutronix.de, vbabka@suse.cz, will@kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add architecture specific implementation details for KFENCE and enable KFENCE for the arm64 architecture. In particular, this implements the required interface in . Currently, the arm64 version does not yet use a statically allocated memory pool, at the cost of a pointer load for each is_kfence_address(). Co-developed-by: Alexander Potapenko Signed-off-by: Alexander Potapenko Signed-off-by: Marco Elver --- For ARM64, we would like to solicit feedback on what the best option is to obtain a constant address for __kfence_pool. One option is to declare a memory range in the memory layout to be dedicated to KFENCE (like is done for KASAN), however, it is unclear if this is the best available option. We would like to avoid touching the memory layout. --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 +++++++++++++++++++++++++++++++++ arch/arm64/mm/fault.c | 4 ++++ 3 files changed, 44 insertions(+) create mode 100644 arch/arm64/include/asm/kfence.h diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 6d232837cbee..1acc6b2877c3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -132,6 +132,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN + select HAVE_ARCH_KFENCE if (!ARM64_16K_PAGES && !ARM64_64K_PAGES) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/arch/arm64/include/asm/kfence.h b/arch/arm64/include/asm/kfence.h new file mode 100644 index 000000000000..608dde80e5ca --- /dev/null +++ b/arch/arm64/include/asm/kfence.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_KFENCE_H +#define __ASM_KFENCE_H + +#include +#include +#include + +#include + +#define KFENCE_SKIP_ARCH_FAULT_HANDLER "el1_sync" + +/* + * FIXME: Support HAVE_ARCH_KFENCE_STATIC_POOL: Use the statically allocated + * __kfence_pool, to avoid the extra pointer load for is_kfence_address(). By + * default, however, we do not have struct pages for static allocations. + */ + +static inline bool arch_kfence_initialize_pool(void) +{ + const unsigned int num_pages = ilog2(roundup_pow_of_two(KFENCE_POOL_SIZE / PAGE_SIZE)); + struct page *pages = alloc_pages(GFP_KERNEL, num_pages); + + if (!pages) + return false; + + __kfence_pool = page_address(pages); + return true; +} + +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + set_memory_valid(addr, 1, !protect); + + return true; +} + +#endif /* __ASM_KFENCE_H */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index f07333e86c2f..d5b72ecbeeea 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -310,6 +311,9 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr, "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr)) return; + if (kfence_handle_page_fault(addr)) + return; + if (is_el1_permission_fault(addr, esr, regs)) { if (esr & ESR_ELx_WNR) msg = "write to read-only memory"; -- 2.28.0.618.gf4bc123cb7-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62B4BC43461 for ; Tue, 15 Sep 2020 13:21:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E504A21D24 for ; Tue, 15 Sep 2020 13:21:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="o71HNdv6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E504A21D24 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 78DCD900057; Tue, 15 Sep 2020 09:21:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6EE4990004C; Tue, 15 Sep 2020 09:21:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CA00900057; Tue, 15 Sep 2020 09:21:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 28F5A90004C for ; Tue, 15 Sep 2020 09:21:08 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id CE96E2489 for ; Tue, 15 Sep 2020 13:21:07 +0000 (UTC) X-FDA: 77265356574.13.leaf07_4d15e3027111 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 9AA8718140B70 for ; Tue, 15 Sep 2020 13:21:07 +0000 (UTC) X-HE-Tag: leaf07_4d15e3027111 X-Filterd-Recvd-Size: 7011 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Sep 2020 13:21:07 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id d9so1213414wrv.16 for ; Tue, 15 Sep 2020 06:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=SxvDeyNmcNMA+Gji2vA/rr03EMZqbUrEGQF/tm1mQmw=; b=o71HNdv6DA6PlBzIM+uaF8z2m3O6QnjR7fFrIwTk8hGwPsQoHPNfBsi4j6RVOTV6rB QGcF2GHmu301W9XcwzPaoDPBszh7HLN2r1OIiFtQRGVFZnqyA2WsrMM1zgh6JFiNCqJr NvzWd/WwOgV6fmMR5uq/A4qmXqraWPjGFNRE6s/puLWVA8MCLDtoQoDvPZ0kQxE1qChb IOCBoE3v1vWdNeoekoIvAvNugpkzCPHfHoTpQXIAxPwGsvUJdJlFijp3tI0jQ0k3V4t2 0FXj8e4gRWaJ3ZkSy6Yet3Tywb4OQfFmJbFzAps6z9Hko0ZQIsPi6+rl+DUaj1LgN5Wt MyGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SxvDeyNmcNMA+Gji2vA/rr03EMZqbUrEGQF/tm1mQmw=; b=oGJ8EHGhvtfiPDdLux+oG4gJpwNTbydbumlpUzHYRix1NGGgy8BkBcjoQr2hky6QZH ZL1qAKESPv8AIlp95xGvD+APhejtuoFqCn84vLIXFxs9NuKZYwz1mqxQlUxpoxQjHOBT vBFLA3AhGwwC8YdXsx0ruZEJP71AUM950LYEjG5W4RjT3UiysKxD1NEK+xYr+Ei2KC4b Lbd3qwVsoMZSBpN81tERspjqr35zLe72b4ycwWl4kHUKb/jhVSo3vUvwy2vJ+BCrU47I BljCzPdEJ4TDTkRhWSI2V3PlhcJ/CrhOJRiUuXjmtsQ4PfPvPc6ubhTw8iygH+pm9EWq 0pxw== X-Gm-Message-State: AOAM530gu9XsIgNM1UOonbzz3DMT8onNXnhPWH/1fu4huifzOx4SNVzx BHVUTB1QBza0It5DEwHHkMOuB9reww== X-Google-Smtp-Source: ABdhPJwl0RFLWfJGwXPTH8kxWe6H1Id6npGQWxX1hTs9KXzpJGcEjMyA8kMwTptUfAmNgcKzKeCOFQa4gQ== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:adf:8b1d:: with SMTP id n29mr20637102wra.383.1600176065852; Tue, 15 Sep 2020 06:21:05 -0700 (PDT) Date: Tue, 15 Sep 2020 15:20:39 +0200 In-Reply-To: <20200915132046.3332537-1-elver@google.com> Message-Id: <20200915132046.3332537-4-elver@google.com> Mime-Version: 1.0 References: <20200915132046.3332537-1-elver@google.com> X-Mailer: git-send-email 2.28.0.618.gf4bc123cb7-goog Subject: [PATCH v2 03/10] arm64, kfence: enable KFENCE for ARM64 From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com, aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de, catalin.marinas@arm.com, cl@linux.com, dave.hansen@linux.intel.com, rientjes@google.com, dvyukov@google.com, edumazet@google.com, gregkh@linuxfoundation.org, mingo@redhat.com, jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net, iamjoonsoo.kim@lge.com, keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org, peterz@infradead.org, cai@lca.pw, tglx@linutronix.de, vbabka@suse.cz, will@kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 9AA8718140B70 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add architecture specific implementation details for KFENCE and enable KFENCE for the arm64 architecture. In particular, this implements the required interface in . Currently, the arm64 version does not yet use a statically allocated memory pool, at the cost of a pointer load for each is_kfence_address(). Co-developed-by: Alexander Potapenko Signed-off-by: Alexander Potapenko Signed-off-by: Marco Elver --- For ARM64, we would like to solicit feedback on what the best option is to obtain a constant address for __kfence_pool. One option is to declare a memory range in the memory layout to be dedicated to KFENCE (like is done for KASAN), however, it is unclear if this is the best available option. We would like to avoid touching the memory layout. --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 +++++++++++++++++++++++++++++++++ arch/arm64/mm/fault.c | 4 ++++ 3 files changed, 44 insertions(+) create mode 100644 arch/arm64/include/asm/kfence.h diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 6d232837cbee..1acc6b2877c3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -132,6 +132,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN + select HAVE_ARCH_KFENCE if (!ARM64_16K_PAGES && !ARM64_64K_PAGES) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/arch/arm64/include/asm/kfence.h b/arch/arm64/include/asm/kfence.h new file mode 100644 index 000000000000..608dde80e5ca --- /dev/null +++ b/arch/arm64/include/asm/kfence.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_KFENCE_H +#define __ASM_KFENCE_H + +#include +#include +#include + +#include + +#define KFENCE_SKIP_ARCH_FAULT_HANDLER "el1_sync" + +/* + * FIXME: Support HAVE_ARCH_KFENCE_STATIC_POOL: Use the statically allocated + * __kfence_pool, to avoid the extra pointer load for is_kfence_address(). By + * default, however, we do not have struct pages for static allocations. + */ + +static inline bool arch_kfence_initialize_pool(void) +{ + const unsigned int num_pages = ilog2(roundup_pow_of_two(KFENCE_POOL_SIZE / PAGE_SIZE)); + struct page *pages = alloc_pages(GFP_KERNEL, num_pages); + + if (!pages) + return false; + + __kfence_pool = page_address(pages); + return true; +} + +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + set_memory_valid(addr, 1, !protect); + + return true; +} + +#endif /* __ASM_KFENCE_H */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index f07333e86c2f..d5b72ecbeeea 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -310,6 +311,9 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr, "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr)) return; + if (kfence_handle_page_fault(addr)) + return; + if (is_el1_permission_fault(addr, esr, regs)) { if (esr & ESR_ELx_WNR) msg = "write to read-only memory"; -- 2.28.0.618.gf4bc123cb7-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D968DC43461 for ; Tue, 15 Sep 2020 13:24:12 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6DE0920829 for ; Tue, 15 Sep 2020 13:24:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="bd0SBJZ0"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="o71HNdv6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6DE0920829 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kbqt0r2iS5qH4hnTCmzHfHHh5ccmr1LZGKWzNXHKS5o=; b=bd0SBJZ0PYl6mUnXCDA4TKlHk NFsb7evkQlYBwwK8i76lTXFvSxD4EOEV8KeCvRSGCRnZEAM163/a3FqtSwUPL2qbRYuST+3+mbuq9 DUmzU0eU2lxCsy6HT0/u2DoqK2pn8KToHadvl9TwYG/IMZL3VosBHm669WpmD3eN9oOjT8YQbpFil cEOZbbsccP+sJ3Rj9TiHImoTpx3TuGFuwRVfK3+eUYS9rhZL/yP8P+MjJ9vjBQRZVQR4rVpooI9da HRzQuLjfwSUwF6njAVqpWcT789/iITH2CBgzZI4CmoykjtA6OJ6Vjtrh69T76wE56zGhBhi9W6CHY vNMFK0/KQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIAuG-0001ry-05; Tue, 15 Sep 2020 13:22:00 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIAtR-0001XQ-V8 for linux-arm-kernel@lists.infradead.org; Tue, 15 Sep 2020 13:21:26 +0000 Received: by mail-wm1-x349.google.com with SMTP id l26so1158968wmg.7 for ; Tue, 15 Sep 2020 06:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=SxvDeyNmcNMA+Gji2vA/rr03EMZqbUrEGQF/tm1mQmw=; b=o71HNdv6DA6PlBzIM+uaF8z2m3O6QnjR7fFrIwTk8hGwPsQoHPNfBsi4j6RVOTV6rB QGcF2GHmu301W9XcwzPaoDPBszh7HLN2r1OIiFtQRGVFZnqyA2WsrMM1zgh6JFiNCqJr NvzWd/WwOgV6fmMR5uq/A4qmXqraWPjGFNRE6s/puLWVA8MCLDtoQoDvPZ0kQxE1qChb IOCBoE3v1vWdNeoekoIvAvNugpkzCPHfHoTpQXIAxPwGsvUJdJlFijp3tI0jQ0k3V4t2 0FXj8e4gRWaJ3ZkSy6Yet3Tywb4OQfFmJbFzAps6z9Hko0ZQIsPi6+rl+DUaj1LgN5Wt MyGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SxvDeyNmcNMA+Gji2vA/rr03EMZqbUrEGQF/tm1mQmw=; b=k2vw8ETW8T6a36ef3uSS/ElDLOg41fLuIlNFD4FSAcx33JFOid1frasnW2naifyAqC gDS8kBCdYtuhIrfrr5zQnQ0NQmUgA8jBSTzBe2WkeZpkf7eZdp4gGKgBj5BtfN/QwS/o 4Cl/je9yTATegv8KWHGpyRlNk3+cFd1ALXsHcSibBBQ/+fBGsazz81ujPZNLSDDlRKX+ OxkyutFP/GBIrdqG75bk+RAXbpCNmpzWTGIS0ZzmwwPaNlpNcguL7cg7cag6QLpftHww VKLBRl4ZNpgDJOhDGtLaA0RcxIHpEcdg+ROq5BWuQMV03UCw9rvIDvnyWaQue901t+VZ 5LcQ== X-Gm-Message-State: AOAM532/aaNnUUI09l36dvjyRd78S4hwmFvnWzMfmlDyUwrAnsd72rb4 050rleGHECfxjqkh4Yifj0x9W2vfIg== X-Google-Smtp-Source: ABdhPJwl0RFLWfJGwXPTH8kxWe6H1Id6npGQWxX1hTs9KXzpJGcEjMyA8kMwTptUfAmNgcKzKeCOFQa4gQ== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:adf:8b1d:: with SMTP id n29mr20637102wra.383.1600176065852; Tue, 15 Sep 2020 06:21:05 -0700 (PDT) Date: Tue, 15 Sep 2020 15:20:39 +0200 In-Reply-To: <20200915132046.3332537-1-elver@google.com> Message-Id: <20200915132046.3332537-4-elver@google.com> Mime-Version: 1.0 References: <20200915132046.3332537-1-elver@google.com> X-Mailer: git-send-email 2.28.0.618.gf4bc123cb7-goog Subject: [PATCH v2 03/10] arm64, kfence: enable KFENCE for ARM64 From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200915_092110_124807_BE01AE7B X-CRM114-Status: GOOD ( 23.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, linux-doc@vger.kernel.org, peterz@infradead.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, edumazet@google.com, hpa@zytor.com, cl@linux.com, will@kernel.org, corbet@lwn.net, x86@kernel.org, kasan-dev@googlegroups.com, mingo@redhat.com, vbabka@suse.cz, rientjes@google.com, aryabinin@virtuozzo.com, keescook@chromium.org, paulmck@kernel.org, jannh@google.com, andreyknvl@google.com, bp@alien8.de, luto@kernel.org, Jonathan.Cameron@huawei.com, tglx@linutronix.de, dvyukov@google.com, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, penberg@kernel.org, cai@lca.pw, iamjoonsoo.kim@lge.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add architecture specific implementation details for KFENCE and enable KFENCE for the arm64 architecture. In particular, this implements the required interface in . Currently, the arm64 version does not yet use a statically allocated memory pool, at the cost of a pointer load for each is_kfence_address(). Co-developed-by: Alexander Potapenko Signed-off-by: Alexander Potapenko Signed-off-by: Marco Elver --- For ARM64, we would like to solicit feedback on what the best option is to obtain a constant address for __kfence_pool. One option is to declare a memory range in the memory layout to be dedicated to KFENCE (like is done for KASAN), however, it is unclear if this is the best available option. We would like to avoid touching the memory layout. --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 +++++++++++++++++++++++++++++++++ arch/arm64/mm/fault.c | 4 ++++ 3 files changed, 44 insertions(+) create mode 100644 arch/arm64/include/asm/kfence.h diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 6d232837cbee..1acc6b2877c3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -132,6 +132,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN + select HAVE_ARCH_KFENCE if (!ARM64_16K_PAGES && !ARM64_64K_PAGES) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/arch/arm64/include/asm/kfence.h b/arch/arm64/include/asm/kfence.h new file mode 100644 index 000000000000..608dde80e5ca --- /dev/null +++ b/arch/arm64/include/asm/kfence.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_KFENCE_H +#define __ASM_KFENCE_H + +#include +#include +#include + +#include + +#define KFENCE_SKIP_ARCH_FAULT_HANDLER "el1_sync" + +/* + * FIXME: Support HAVE_ARCH_KFENCE_STATIC_POOL: Use the statically allocated + * __kfence_pool, to avoid the extra pointer load for is_kfence_address(). By + * default, however, we do not have struct pages for static allocations. + */ + +static inline bool arch_kfence_initialize_pool(void) +{ + const unsigned int num_pages = ilog2(roundup_pow_of_two(KFENCE_POOL_SIZE / PAGE_SIZE)); + struct page *pages = alloc_pages(GFP_KERNEL, num_pages); + + if (!pages) + return false; + + __kfence_pool = page_address(pages); + return true; +} + +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + set_memory_valid(addr, 1, !protect); + + return true; +} + +#endif /* __ASM_KFENCE_H */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index f07333e86c2f..d5b72ecbeeea 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -310,6 +311,9 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr, "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr)) return; + if (kfence_handle_page_fault(addr)) + return; + if (is_el1_permission_fault(addr, esr, regs)) { if (esr & ESR_ELx_WNR) msg = "write to read-only memory"; -- 2.28.0.618.gf4bc123cb7-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel