From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11313C433DF for ; Tue, 13 Oct 2020 07:00:28 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8A568207C3 for ; Tue, 13 Oct 2020 07:00:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Wonjpauk"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="VYTEMjwV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A568207C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tbGsB3jUbrkyP8m0UMOcymr1GdmFOvazSDaft8EkeKo=; b=Wonjpauk/3Z8n5KuAZ++IoJhp d3xOqDQ/5ND5HTo7JhO7YZ1G7RvQ+x0QsKqT83xHW/RMzLsagh93d6lzsKoqT9aijgwg98rQvhbwG Cp/wtGlnDGc5fHVLhTuz8JbVACi3c4byC99RPirVZSyU/FGniOf4UKvIKs7O01/r1kcfFnXWXRu1u V5JElswbx4bTHMM7GZXpFV65DdTYuNQondkyU404ro+bWDy7ekv9sWXwXwj6XB3isz3pmuxtwPi3l M2ZSOYvycL18cohNeRfclSXZzUyxTUaPRpE9BE0383vGTur5xgH3GO3rirk4UsvUza1ob3Lzh6Rpj LF+juntLQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSEGc-0000YD-LI; Tue, 13 Oct 2020 06:58:38 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kSEGZ-0000Xk-Pk for linux-arm-kernel@lists.infradead.org; Tue, 13 Oct 2020 06:58:37 +0000 Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3794220797 for ; Tue, 13 Oct 2020 06:58:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602572314; bh=oLGxQioGwijsin8+oReqPlBm/hoB9XCsApmyZrC6Gc0=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=VYTEMjwVQJqVOTretd1BeP93iQkFGHQdxqafq35MdYNWPnuCSJt5JEBc0TG+yeVt/ sGkz0zVHRjn7zCUJfru8kA7hWiPZJH7l8NUWNjnYbXPS1+kQ2AiDD7KK24i2ZTy+vx XMIO7/HspBLazZQ/3Kgux4BgQkx4g8md2IPZMbxQ= Received: by mail-ot1-f51.google.com with SMTP id e20so17662710otj.11 for ; Mon, 12 Oct 2020 23:58:34 -0700 (PDT) X-Gm-Message-State: AOAM532hc3si9z6UJqLlQDJ7jYYOAxQeSNaw5TPnXPqWj/Hi2hB+1C+D tYV2WUKnek9ddhelvENvMHUmr8p0ErzDTGlMYiM= X-Google-Smtp-Source: ABdhPJxb/HnFoDrxbw0uMMzZil+m8ykHgJ1z3kR0eD+Yp6o1DSsMWFhq/sT89rFMtVgX5cHUZ4e6ZAY4CJTC5OF8i/0= X-Received: by 2002:a9d:6a85:: with SMTP id l5mr22141135otq.77.1602572313367; Mon, 12 Oct 2020 23:58:33 -0700 (PDT) MIME-Version: 1.0 References: <20201012215701.123389-1-linus.walleij@linaro.org> <20201012215701.123389-5-linus.walleij@linaro.org> In-Reply-To: <20201012215701.123389-5-linus.walleij@linaro.org> From: Ard Biesheuvel Date: Tue, 13 Oct 2020 08:58:22 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 4/5 v15] ARM: Initialize the mapping of KASan shadow memory To: Linus Walleij X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201013_025836_058774_383987B1 X-CRM114-Status: GOOD ( 57.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Florian Fainelli , Arnd Bergmann , Abbott Liu , Russell King , kasan-dev , Mike Rapoport , Alexander Potapenko , Dmitry Vyukov , Andrey Ryabinin , Linux ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Linus, Just a couple of cosmetic tweaks below - no need to resend for this. On Mon, 12 Oct 2020 at 23:59, Linus Walleij wrote: > > This patch initializes KASan shadow region's page table and memory. > There are two stage for KASan initializing: > > 1. At early boot stage the whole shadow region is mapped to just > one physical page (kasan_zero_page). It is finished by the function > kasan_early_init which is called by __mmap_switched(arch/arm/kernel/ > head-common.S) > > 2. After the calling of paging_init, we use kasan_zero_page as zero > shadow for some memory that KASan does not need to track, and we > allocate a new shadow space for the other memory that KASan need to > track. These issues are finished by the function kasan_init which is > call by setup_arch. > > When using KASan we also need to increase the THREAD_SIZE_ORDER > from 1 to 2 as the extra calls for shadow memory uses quite a bit > of stack. > > As we need to make a temporary copy of the PGD when setting up > shadow memory we create a helpful PGD_SIZE definition for both > LPAE and non-LPAE setups. > > The KASan core code unconditionally calls pud_populate() so this > needs to be changed from BUG() to do {} while (0) when building > with KASan enabled. > > After the initial development by Andre Ryabinin several modifications > have been made to this code: > > Abbott Liu > - Add support ARM LPAE: If LPAE is enabled, KASan shadow region's > mapping table need be copied in the pgd_alloc() function. > - Change kasan_pte_populate,kasan_pmd_populate,kasan_pud_populate, > kasan_pgd_populate from .meminit.text section to .init.text section. > Reported by Florian Fainelli > > Linus Walleij : > - Drop the custom mainpulation of TTBR0 and just use > cpu_switch_mm() to switch the pgd table. > - Adopt to handle 4th level page tabel folding. > - Rewrite the entire page directory and page entry initialization > sequence to be recursive based on ARM64:s kasan_init.c. > > Ard Biesheuvel : > - Necessary underlying fixes. > - Crucial bug fixes to the memory set-up code. > > Cc: Alexander Potapenko > Cc: Dmitry Vyukov > Cc: kasan-dev@googlegroups.com > Cc: Mike Rapoport > Co-developed-by: Andrey Ryabinin > Co-developed-by: Abbott Liu > Co-developed-by: Ard Biesheuvel > Acked-by: Mike Rapoport > Reviewed-by: Ard Biesheuvel > Tested-by: Ard Biesheuvel # QEMU/KVM/mach-virt/LPAE/8G > Reported-by: Russell King - ARM Linux > Reported-by: Florian Fainelli > Signed-off-by: Andrey Ryabinin > Signed-off-by: Abbott Liu > Signed-off-by: Florian Fainelli > Signed-off-by: Ard Biesheuvel > Signed-off-by: Linus Walleij > --- > ChangeLog v14->v15: > - Avoids reallocating KASAN blocks when a range gets > mapped twice - this occurs when mapping the DTB space explicitly. > - Insert a missing TLB flush. > - Move the cache flush after switching the MM (which makes logical > sense. > - All these fixes discovered by Ard Bisheuvel. > - Dropped the special mapping around the DTB after using Ard's > patches for remapping the DTB in a special memory area. > - Add asmlinkage prototype for kasan_early_init() to get > rid of some compilation warnings. > ChangeLog v13->v14: > - Provide more elaborate prints of how virtual kernel memory > is mapped to the allocated lowmem pages. > - Make sure to also map the memory around the __atags_pointer: > this memory is used for the device tree blob (DTB) and will be > accessed by the device tree parser. We were just lucky that > this was mostly in some acceptable memory location until now. > ChangeLog v12->v13: > - Rebase on kernel v5.9-rc1 > ChangeLog v11->v12: > - Do not try to shadow highmem memory blocks. (Ard) > - Provoke a build bug if the entire shadow memory doesn't fit > inside a single pgd_index() (Ard) > - Move the pointer to (unsigned long) casts into the create_mapping() > function. (Ard) > - After setting up the shadow memory make sure to issue > local_flush_tlb_all() so that we refresh all the global mappings. (Ard) > - Simplify pte_populate() (Ard) > - Skip over pud population as well as p4d. (Ard) > - Drop the stop condition pmd_none(*pmdp) in the pmd population > loop. (Ard) > - Stop passing around the node (NUMA) parameter in the init code, > we are not expecting any NUMA architectures to be introduced into > ARM32 so just hardcode NUMA_NO_NODE when calling > memblock_alloc_try_nid(). > ChangeLog v10->v11: > - Fix compilation on LPAE systems. > - Move the check for valid pgdp, pudp and pmdp into the loop for > each level moving over the directory pointers: we were just lucky > that we just needed one directory for each level so this fixes > the pmdp issue with LPAE and KASan now works like a charm on > LPAE as well. > - Fold fourth level page directory (p4d) into the global page directory > pgd and just skip into the page upper directory (pud) directly. We > do not anticipate that ARM32 will every use 5-level page tables. > - Simplify the ifdeffery around the temporary pgd. > - Insert a comment about pud_populate() that is unconditionally called > by the KASan core code. > ChangeLog v9->v10: > - Rebase onto v5.8-rc1 > - add support for folded p4d page tables, use the primitives necessary > for the 4th level folding, add (empty) walks of p4d level. > - Use the header file that has now appeared as part > of the VM consolidation series. > - Use a recursive method to walk pgd/p4d/pud/pmd/pte instead of the > separate early/main calls and the flat call structure used in the > old code. This was inspired by the ARM64 KASan init code. > - Assume authorship of this code, I have now written the majority of > it so the blame is on me and noone else. > ChangeLog v8->v9: > - Drop the custom CP15 manipulation and cache flushing for swapping > TTBR0 and instead just use cpu_switch_mm(). > - Collect Ard's tags. > ChangeLog v7->v8: > - Rebased. > ChangeLog v6->v7: > - Use SPDX identifer for the license. > - Move the TTBR0 accessor calls into this patch. > --- > arch/arm/include/asm/kasan.h | 33 ++++ > arch/arm/include/asm/pgalloc.h | 8 +- > arch/arm/include/asm/thread_info.h | 8 + > arch/arm/kernel/head-common.S | 3 + > arch/arm/kernel/setup.c | 2 + > arch/arm/mm/Makefile | 3 + > arch/arm/mm/kasan_init.c | 284 +++++++++++++++++++++++++++++ > arch/arm/mm/pgd.c | 16 +- > 8 files changed, 355 insertions(+), 2 deletions(-) > create mode 100644 arch/arm/include/asm/kasan.h > create mode 100644 arch/arm/mm/kasan_init.c > > diff --git a/arch/arm/include/asm/kasan.h b/arch/arm/include/asm/kasan.h > new file mode 100644 > index 000000000000..303c35df3135 > --- /dev/null > +++ b/arch/arm/include/asm/kasan.h > @@ -0,0 +1,33 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * arch/arm/include/asm/kasan.h > + * > + * Copyright (c) 2015 Samsung Electronics Co., Ltd. > + * Author: Andrey Ryabinin > + * > + */ > + > +#ifndef __ASM_KASAN_H > +#define __ASM_KASAN_H > + > +#ifdef CONFIG_KASAN > + > +#include > + > +#define KASAN_SHADOW_SCALE_SHIFT 3 > + > +/* > + * The compiler uses a shadow offset assuming that addresses start > + * from 0. Kernel addresses don't start from 0, so shadow > + * for kernel really starts from 'compiler's shadow offset' + > + * ('kernel address space start' >> KASAN_SHADOW_SCALE_SHIFT) > + */ > + > +asmlinkage void kasan_early_init(void); > +extern void kasan_init(void); > + > +#else > +static inline void kasan_init(void) { } > +#endif > + > +#endif > diff --git a/arch/arm/include/asm/pgalloc.h b/arch/arm/include/asm/pgalloc.h > index 15f4674715f8..fdee1f04f4f3 100644 > --- a/arch/arm/include/asm/pgalloc.h > +++ b/arch/arm/include/asm/pgalloc.h > @@ -21,6 +21,7 @@ > #define _PAGE_KERNEL_TABLE (PMD_TYPE_TABLE | PMD_BIT4 | PMD_DOMAIN(DOMAIN_KERNEL)) > > #ifdef CONFIG_ARM_LPAE > +#define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t)) > > static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) > { > @@ -28,14 +29,19 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) > } > > #else /* !CONFIG_ARM_LPAE */ > +#define PGD_SIZE (PAGE_SIZE << 2) > > /* > * Since we have only two-level page tables, these are trivial > */ > #define pmd_alloc_one(mm,addr) ({ BUG(); ((pmd_t *)2); }) > #define pmd_free(mm, pmd) do { } while (0) > +#ifdef CONFIG_KASAN > +/* The KASan core unconditionally calls pud_populate() on all architectures */ > +#define pud_populate(mm,pmd,pte) do { } while (0) > +#else > #define pud_populate(mm,pmd,pte) BUG() > - > +#endif > #endif /* CONFIG_ARM_LPAE */ > > extern pgd_t *pgd_alloc(struct mm_struct *mm); > diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h > index 536b6b979f63..56fae7861fd3 100644 > --- a/arch/arm/include/asm/thread_info.h > +++ b/arch/arm/include/asm/thread_info.h > @@ -13,7 +13,15 @@ > #include > #include > > +#ifdef CONFIG_KASAN > +/* > + * KASan uses a lot of extra stack space so the thread size order needs to > + * be increased. > + */ > +#define THREAD_SIZE_ORDER 2 > +#else > #define THREAD_SIZE_ORDER 1 > +#endif > #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) > #define THREAD_START_SP (THREAD_SIZE - 8) > > diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S > index 6840c7c60a85..89c80154b9ef 100644 > --- a/arch/arm/kernel/head-common.S > +++ b/arch/arm/kernel/head-common.S > @@ -111,6 +111,9 @@ __mmap_switched: > str r8, [r2] @ Save atags pointer > cmp r3, #0 > strne r10, [r3] @ Save control register values > +#ifdef CONFIG_KASAN > + bl kasan_early_init > +#endif > mov lr, #0 > b start_kernel > ENDPROC(__mmap_switched) > diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c > index 2a70e4958c14..43d033696e33 100644 > --- a/arch/arm/kernel/setup.c > +++ b/arch/arm/kernel/setup.c > @@ -59,6 +59,7 @@ > #include > #include > #include > +#include > > #include "atags.h" > > @@ -1139,6 +1140,7 @@ void __init setup_arch(char **cmdline_p) > early_ioremap_reset(); > > paging_init(mdesc); > + kasan_init(); > request_standard_resources(mdesc); > > if (mdesc->restart) > diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile > index 99699c32d8a5..4536159bc8fa 100644 > --- a/arch/arm/mm/Makefile > +++ b/arch/arm/mm/Makefile > @@ -113,3 +113,6 @@ obj-$(CONFIG_CACHE_L2X0_PMU) += cache-l2x0-pmu.o > obj-$(CONFIG_CACHE_XSC3L2) += cache-xsc3l2.o > obj-$(CONFIG_CACHE_TAUROS2) += cache-tauros2.o > obj-$(CONFIG_CACHE_UNIPHIER) += cache-uniphier.o > + > +KASAN_SANITIZE_kasan_init.o := n > +obj-$(CONFIG_KASAN) += kasan_init.o > diff --git a/arch/arm/mm/kasan_init.c b/arch/arm/mm/kasan_init.c > new file mode 100644 > index 000000000000..22ac84defa5d > --- /dev/null > +++ b/arch/arm/mm/kasan_init.c > @@ -0,0 +1,284 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * This file contains kasan initialization code for ARM. > + * > + * Copyright (c) 2018 Samsung Electronics Co., Ltd. > + * Author: Andrey Ryabinin > + * Author: Linus Walleij > + */ > + > +#define pr_fmt(fmt) "kasan: " fmt > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "mm.h" > + > +static pgd_t tmp_pgd_table[PTRS_PER_PGD] __initdata __aligned(PGD_SIZE); > + > +pmd_t tmp_pmd_table[PTRS_PER_PMD] __page_aligned_bss; > + > +static __init void *kasan_alloc_block(size_t size) > +{ > + return memblock_alloc_try_nid(size, size, __pa(MAX_DMA_ADDRESS), > + MEMBLOCK_ALLOC_KASAN, NUMA_NO_NODE); > +} > + > +static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, > + unsigned long end, bool early) > +{ > + unsigned long next; > + pte_t *ptep = pte_offset_kernel(pmdp, addr); > + > + do { > + pte_t entry; > + void *p; > + > + next = addr + PAGE_SIZE; > + > + if (!early) { > + if (!pte_none(READ_ONCE(*ptep))) > + continue; > + > + p = kasan_alloc_block(PAGE_SIZE); > + if (!p) { > + panic("%s failed to alloc pte for address 0x%lx\n", This does not allocate a page table but a shadow page. > + __func__, addr); > + return; > + } > + memset(p, KASAN_SHADOW_INIT, PAGE_SIZE); > + entry = pfn_pte(virt_to_pfn(p), > + __pgprot(pgprot_val(PAGE_KERNEL))); > + } else if (pte_none(READ_ONCE(*ptep))) { > + /* > + * The early shadow memory is mapping all KASan > + * operations to one and the same page in memory, > + * "kasan_early_shadow_page" so that the instrumentation > + * will work on a scratch area until we can set up the > + * proper KASan shadow memory. > + */ > + entry = pfn_pte(virt_to_pfn(kasan_early_shadow_page), > + __pgprot(_L_PTE_DEFAULT | L_PTE_DIRTY | L_PTE_XN)); > + } else { > + /* > + * Early shadow mappings are PMD_SIZE aligned, so if the > + * first entry is already set, they must all be set. > + */ > + return; > + } > + > + set_pte_at(&init_mm, addr, ptep, entry); > + } while (ptep++, addr = next, addr != end); > +} > + > +/* > + * The pmd (page middle directory) is only used on LPAE > + */ > +static void __init kasan_pmd_populate(pud_t *pudp, unsigned long addr, > + unsigned long end, bool early) > +{ > + unsigned long next; > + pmd_t *pmdp = pmd_offset(pudp, addr); > + > + do { > + if (pmd_none(*pmdp)) { > + void *p = early ? kasan_early_shadow_pte : > + kasan_alloc_block(PAGE_SIZE); > + > + if (!p) { > + panic("%s failed to allocate pmd for address 0x%lx\n", This allocates a block of PTEs > + __func__, addr); > + return; > + } > + pmd_populate_kernel(&init_mm, pmdp, p); > + flush_pmd_entry(pmdp); > + } > + > + next = pmd_addr_end(addr, end); > + kasan_pte_populate(pmdp, addr, next, early); > + } while (pmdp++, addr = next, addr != end); > +} > + > +static void __init kasan_pgd_populate(unsigned long addr, unsigned long end, > + bool early) > +{ > + unsigned long next; > + pgd_t *pgdp; > + p4d_t *p4dp; > + pud_t *pudp; > + > + pgdp = pgd_offset_k(addr); > + > + do { > + /* Allocate and populate the PGD if it doesn't already exist */ > + if (!early && pgd_none(*pgdp)) { > + void *p = kasan_alloc_block(PAGE_SIZE); > + > + if (!p) { > + panic("%s failed to allocate pgd for address 0x%lx\n", This allocates a block of P4D folded into PUD folded into PMD. In summary, since the __func__ gives us the location of the error, perhaps just drop the pgd here (and pmd above?) > + __func__, addr); > + return; > + } > + pgd_populate(&init_mm, pgdp, p); > + } > + > + next = pgd_addr_end(addr, end); > + /* > + * We just immediately jump over the p4d and pud page > + * directories since we believe ARM32 will never gain four > + * nor five level page tables. > + */ > + p4dp = p4d_offset(pgdp, addr); > + pudp = pud_offset(p4dp, addr); > + > + kasan_pmd_populate(pudp, addr, next, early); > + } while (pgdp++, addr = next, addr != end); > +} > + > +extern struct proc_info_list *lookup_processor_type(unsigned int); > + > +void __init kasan_early_init(void) > +{ > + struct proc_info_list *list; > + > + /* > + * locate processor in the list of supported processor > + * types. The linker builds this table for us from the > + * entries in arch/arm/mm/proc-*.S > + */ > + list = lookup_processor_type(read_cpuid_id()); > + if (list) { > +#ifdef MULTI_CPU > + processor = *list->proc; > +#endif > + } > + > + BUILD_BUG_ON((KASAN_SHADOW_END - (1UL << 29)) != KASAN_SHADOW_OFFSET); > + /* > + * We walk the page table and set all of the shadow memory to point > + * to the scratch page. > + */ > + kasan_pgd_populate(KASAN_SHADOW_START, KASAN_SHADOW_END, true); > +} > + > +static void __init clear_pgds(unsigned long start, > + unsigned long end) > +{ > + for (; start && start < end; start += PMD_SIZE) > + pmd_clear(pmd_off_k(start)); > +} > + > +static int __init create_mapping(void *start, void *end) > +{ > + void *shadow_start, *shadow_end; > + > + shadow_start = kasan_mem_to_shadow(start); > + shadow_end = kasan_mem_to_shadow(end); > + > + pr_info("Mapping kernel virtual memory block: %px-%px at shadow: %px-%px\n", > + start, end, shadow_start, shadow_end); > + > + kasan_pgd_populate((unsigned long)shadow_start & PAGE_MASK, > + (unsigned long)shadow_end, false); As I mentioned in my reply to Florian, we should PAGE__ALIGN() shadow_end here to ensure that we can meet the stop condition in kasan_pgd_populate() > + return 0; > +} > + > +void __init kasan_init(void) > +{ > + struct memblock_region *reg; > + int i; > + > + /* > + * We are going to perform proper setup of shadow memory. > + * > + * At first we should unmap early shadow (clear_pgds() call bellow). > + * However, instrumented code can't execute without shadow memory. > + * > + * To keep the early shadow memory MMU tables around while setting up > + * the proper shadow memory, we copy swapper_pg_dir (the initial page > + * table) to tmp_pgd_table and use that to keep the early shadow memory > + * mapped until the full shadow setup is finished. Then we swap back > + * to the proper swapper_pg_dir. > + */ > + > + memcpy(tmp_pgd_table, swapper_pg_dir, sizeof(tmp_pgd_table)); > +#ifdef CONFIG_ARM_LPAE > + /* We need to be in the same PGD or this won't work */ > + BUILD_BUG_ON(pgd_index(KASAN_SHADOW_START) != > + pgd_index(KASAN_SHADOW_END)); > + memcpy(tmp_pmd_table, > + pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_START)), > + sizeof(tmp_pmd_table)); > + set_pgd(&tmp_pgd_table[pgd_index(KASAN_SHADOW_START)], > + __pgd(__pa(tmp_pmd_table) | PMD_TYPE_TABLE | L_PGD_SWAPPER)); > +#endif > + cpu_switch_mm(tmp_pgd_table, &init_mm); > + local_flush_tlb_all(); > + > + clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END); > + > + kasan_populate_early_shadow(kasan_mem_to_shadow((void *)VMALLOC_START), > + kasan_mem_to_shadow((void *)-1UL) + 1); > + > + for_each_memblock(memory, reg) { > + void *start = __va(reg->base); > + void *end = __va(reg->base + reg->size); > + > + /* Do not attempt to shadow highmem */ > + if (reg->base >= arm_lowmem_limit) { > + pr_info("Skip highmem block %px-%px\n", > + start, end); This gives me [ 0.000000] kasan: Skip highmem block 7f9db000-7fb9d000 [ 0.000000] kasan: Skip highmem block 7fb9d000-c0000000 for [ 0.000000] node 0: [mem 0x00000000ff9db000-0x00000000ffb9cfff] [ 0.000000] node 0: [mem 0x00000000ffb9d000-0x000000023fffffff] which is highly confusing - highmem does not have a VA in the first place, so reporting it here makes no sense. Better use %llx here and print reg->base/size directly. > + continue; > + } > + if (reg->base + reg->size > arm_lowmem_limit) { > + pr_info("Truncate memory block %px-%px\n to %px-%px\n", > + start, end, start, __va(arm_lowmem_limit)); This gives me [ 0.000000] kasan: Truncate memory block c0000000-7f9db000 to c0000000-f0000000 for [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000ff9dafff] which is equally confusing. I think we should also use reg->base/size here, and omit the start and __va(arm_lowmem_limit) entirely, and just print something like kasan: Truncating shadow for 0x0040000000-0x00ff9dafff to lowmem region (note that 0x%10llx should be sufficient as LPAE addresses have at most 40 bits) > + end = __va(arm_lowmem_limit); > + } > + if (start >= end) { > + pr_info("Skipping invalid memory block %px-%px\n", > + start, end); > + continue; > + } > + > + create_mapping(start, end); > + } > + > + /* > + * 1. The module global variables are in MODULES_VADDR ~ MODULES_END, > + * so we need to map this area. > + * 2. PKMAP_BASE ~ PKMAP_BASE+PMD_SIZE's shadow and MODULES_VADDR > + * ~ MODULES_END's shadow is in the same PMD_SIZE, so we can't > + * use kasan_populate_zero_shadow. > + */ > + create_mapping((void *)MODULES_VADDR, (void *)(PKMAP_BASE + PMD_SIZE)); > + > + /* > + * KAsan may reuse the contents of kasan_early_shadow_pte directly, so > + * we should make sure that it maps the zero page read-only. > + */ > + for (i = 0; i < PTRS_PER_PTE; i++) > + set_pte_at(&init_mm, KASAN_SHADOW_START + i*PAGE_SIZE, > + &kasan_early_shadow_pte[i], > + pfn_pte(virt_to_pfn(kasan_early_shadow_page), > + __pgprot(pgprot_val(PAGE_KERNEL) > + | L_PTE_RDONLY))); > + > + cpu_switch_mm(swapper_pg_dir, &init_mm); > + local_flush_tlb_all(); > + > + memset(kasan_early_shadow_page, 0, PAGE_SIZE); > + pr_info("Kernel address sanitizer initialized\n"); > + init_task.kasan_depth = 0; > +} > diff --git a/arch/arm/mm/pgd.c b/arch/arm/mm/pgd.c > index c5e1b27046a8..f8e9bc58a84f 100644 > --- a/arch/arm/mm/pgd.c > +++ b/arch/arm/mm/pgd.c > @@ -66,7 +66,21 @@ pgd_t *pgd_alloc(struct mm_struct *mm) > new_pmd = pmd_alloc(mm, new_pud, 0); > if (!new_pmd) > goto no_pmd; > -#endif > +#ifdef CONFIG_KASAN > + /* > + * Copy PMD table for KASAN shadow mappings. > + */ > + init_pgd = pgd_offset_k(TASK_SIZE); > + init_p4d = p4d_offset(init_pgd, TASK_SIZE); > + init_pud = pud_offset(init_p4d, TASK_SIZE); > + init_pmd = pmd_offset(init_pud, TASK_SIZE); > + new_pmd = pmd_offset(new_pud, TASK_SIZE); > + memcpy(new_pmd, init_pmd, > + (pmd_index(MODULES_VADDR) - pmd_index(TASK_SIZE)) > + * sizeof(pmd_t)); > + clean_dcache_area(new_pmd, PTRS_PER_PMD * sizeof(pmd_t)); > +#endif /* CONFIG_KASAN */ > +#endif /* CONFIG_LPAE */ > > if (!vectors_high()) { > /* > -- > 2.26.2 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel