From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D051CA9EC5 for ; Wed, 30 Oct 2019 14:23:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3139320656 for ; Wed, 30 Oct 2019 14:23:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FMHtD4So" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3139320656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 47F806B0270; Wed, 30 Oct 2019 10:23:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4097C6B0271; Wed, 30 Oct 2019 10:23:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20B836B0272; Wed, 30 Oct 2019 10:23:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id E78856B0270 for ; Wed, 30 Oct 2019 10:23:32 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 95906181AEF31 for ; Wed, 30 Oct 2019 14:23:32 +0000 (UTC) X-FDA: 76100669064.28.house14_3bed5790e235d X-HE-Tag: house14_3bed5790e235d X-Filterd-Recvd-Size: 14491 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:31 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id j14so1409499wrm.6 for ; Wed, 30 Oct 2019 07:23:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Qn9uGyCi8LQXEYobYHMlSmj0M7rPOYDf1BFHM1dbAQA=; b=FMHtD4Sotg86acgtSnsa2e/TOGt7pMsxNIvAdUfCFCiSIw3grhYPVmcBmYcCxFfRev 4sgygEcWAj4ifhsFpYihoECyEQ0kS2LUEl+7cEv2fecVajApB38dOTZXo+5zaEr+SNvR I7ijOzlauNwYDuL07bSSpeodNzACQeO3MUhGNhYMwT+Dt+MYAKsxMIw7iymWcLrqbVgc 4W4Jqmc2Rhkr18kOTPLme4bng7Pm/fPBRDkzbBLd9Hr9ymFxFMPTLFvdD6hJDLWUS26x K3wIc/m7vh6dvxQ7C6pIP9jUKuqrYPbgYLfouU+YACpHC05a6sXjstvkXRDpZA9mNQAU 884A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Qn9uGyCi8LQXEYobYHMlSmj0M7rPOYDf1BFHM1dbAQA=; b=d7kCuLg1GQWrMuvCfTRP0QzcGsL4fpgStugxPbT98lAPGSGDfnkl2uj5teMhFa8tOF a92C/aEz3ZaRLwRDecbA5P5XcPOXklN93F9hJzQIq/4dnhR5HQ6IfcOr1+9W9qTbXktK UPOxKYS0GunaLfY7MdwKfU+x1nlkOLaVt/sAc7IUA+1jY1LdqzCeMCKr4Unp0BlRyilu XIJp9IBHaBCPy1LloKygGU8XmmNwdfOF5oDXWkmUtgU9Zv4lRaAaPtotawRIncIIfxKr xAvMTAk1f97htrrf04Zt1g9887+GULF6GufEcr7c8PS5jFBC0OdWacXxaHrT/0d6vPNB WERA== X-Gm-Message-State: APjAAAX1zAFiElr1ngXMRjQsR2ARBb9XUdCpR49q7hG6v/IFN9l0cxfs /ehzSkP7rNnHo1o/LC5U5MR8wlcFx5I= X-Google-Smtp-Source: APXvYqy07Mg2nnumIPWlIyodUuh3ebh+5mTPYmdCFAAYOWHiBBn4e0Nek1ivfL1bBAcKSU24EYrshIKN6lU= X-Received: by 2002:a5d:50c2:: with SMTP id f2mr105308wrt.147.1572445410331; Wed, 30 Oct 2019 07:23:30 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:27 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-16-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 15/25] kmsan: Changing existing files to enable KMSAN builds From: glider@google.com To: Jens Axboe , Andy Lutomirski , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do the following to make KMSAN usable: - add KMSAN declarations to struct page and struct task_struct; - add Makefile.kmsan to top-level Makefile; - call KMSAN initialization from init/main.c - add asm hooks to arch/x86/entry/entry_64.S; - increase task stack size under KMSAN; - disable page merging in block/blk.h; - disable CONFIG_KMSAN in arch/x86/boot/compressed/misc.h to use default string functions instead of KMSAN ones. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Jens Axboe Cc: Andy Lutomirski Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I37e0b7f2d2f2b0aeac5753ff9d6b411485fc374e --- Makefile | 3 ++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/entry/entry_64.S | 18 ++++++++++++++++++ arch/x86/include/asm/page_64_types.h | 12 +++++++----- block/blk.h | 7 +++++++ include/linux/mm_types.h | 9 +++++++++ include/linux/sched.h | 5 +++++ init/main.c | 3 +++ lib/stackdepot.c | 7 ++++--- mm/Makefile | 1 + scripts/Makefile.lib | 6 ++++++ 11 files changed, 63 insertions(+), 9 deletions(-) diff --git a/Makefile b/Makefile index 79be70bf2899..181ae2dac415 100644 --- a/Makefile +++ b/Makefile @@ -478,7 +478,7 @@ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE -export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN +export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KMSAN export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL @@ -898,6 +898,7 @@ KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none) endif include scripts/Makefile.kasan +include scripts/Makefile.kmsan include scripts/Makefile.extrawarn include scripts/Makefile.ubsan diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index c8181392f70d..dd4bd8c5d97a 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -12,6 +12,7 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN +#undef CONFIG_KMSAN /* cpu_feature_enabled() cannot be used this early */ #define USE_EARLY_PGTABLE_L5 diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index b7c3ea4cb19d..1218672bad85 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -168,12 +169,14 @@ GLOBAL(entry_SYSCALL_64_after_hwframe) PUSH_AND_CLEAR_REGS rax=$-ENOSYS TRACE_IRQS_OFF + KMSAN_SYSCALL_ENTER /* IRQs are off. */ movq %rax, %rdi movq %rsp, %rsi call do_syscall_64 /* returns with IRQs disabled */ + KMSAN_SYSCALL_EXIT TRACE_IRQS_IRETQ /* we're about to change IF */ /* @@ -575,6 +578,7 @@ ENTRY(interrupt_entry) 1: ENTER_IRQ_STACK old_rsp=%rdi save_ret=1 + KMSAN_INTERRUPT_ENTER /* We entered an interrupt context - irqs are off: */ TRACE_IRQS_OFF @@ -604,12 +608,14 @@ common_interrupt: addq $-0x80, (%rsp) /* Adjust vector to [-256, -1] range */ call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call do_IRQ /* rdi points to pt_regs */ /* 0(%rsp): old RSP */ ret_from_intr: DISABLE_INTERRUPTS(CLBR_ANY) TRACE_IRQS_OFF + KMSAN_INTERRUPT_EXIT LEAVE_IRQ_STACK testb $3, CS(%rsp) @@ -802,6 +808,7 @@ ENTRY(\sym) .Lcommon_\sym: call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call \do_sym /* rdi points to pt_regs */ jmp ret_from_intr END(\sym) @@ -909,15 +916,18 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt .if \shift_ist != -1 subq $\ist_offset, CPU_TSS_IST(\shift_ist) + KMSAN_IST_ENTER(\shift_ist) .endif .if \read_cr2 movq %r12, %rdx /* Move CR2 into 3rd argument */ .endif + KMSAN_UNPOISON_PT_REGS call \do_sym .if \shift_ist != -1 + KMSAN_IST_EXIT(\shift_ist) addq $\ist_offset, CPU_TSS_IST(\shift_ist) .endif @@ -1079,7 +1089,9 @@ ENTRY(do_softirq_own_stack) pushq %rbp mov %rsp, %rbp ENTER_IRQ_STACK regs=0 old_rsp=%r11 + KMSAN_SOFTIRQ_ENTER call __do_softirq + KMSAN_SOFTIRQ_EXIT LEAVE_IRQ_STACK regs=0 leaveq ret @@ -1466,9 +1478,12 @@ ENTRY(nmi) * done with the NMI stack. */ + KMSAN_NMI_ENTER movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* * Return back to user mode. We must *not* do the normal exit @@ -1678,10 +1693,13 @@ end_repeat_nmi: call paranoid_entry UNWIND_HINT_REGS + KMSAN_NMI_ENTER /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */ movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* Always restore stashed CR3 value (see paranoid_entry) */ RESTORE_CR3 scratch_reg=%r15 save_reg=%r14 diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h index 288b065955b7..ea9fbf09f43b 100644 --- a/arch/x86/include/asm/page_64_types.h +++ b/arch/x86/include/asm/page_64_types.h @@ -7,18 +7,20 @@ #endif #ifdef CONFIG_KASAN -#define KASAN_STACK_ORDER 1 +#define EXTRA_STACK_ORDER 1 +#elif defined(CONFIG_KMSAN) +#define EXTRA_STACK_ORDER 2 #else -#define KASAN_STACK_ORDER 0 +#define EXTRA_STACK_ORDER 0 #endif -#define THREAD_SIZE_ORDER (2 + KASAN_STACK_ORDER) +#define THREAD_SIZE_ORDER (2 + EXTRA_STACK_ORDER) #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) -#define EXCEPTION_STACK_ORDER (0 + KASAN_STACK_ORDER) +#define EXCEPTION_STACK_ORDER (0 + EXTRA_STACK_ORDER) #define EXCEPTION_STKSZ (PAGE_SIZE << EXCEPTION_STACK_ORDER) -#define IRQ_STACK_ORDER (2 + KASAN_STACK_ORDER) +#define IRQ_STACK_ORDER (2 + EXTRA_STACK_ORDER) #define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER) /* diff --git a/block/blk.h b/block/blk.h index 47fba9362e60..9ee271a22423 100644 --- a/block/blk.h +++ b/block/blk.h @@ -78,6 +78,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q, phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset; phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset; +#ifdef CONFIG_KMSAN + /* + * Merging consequent physical pages may not work correctly under KMSAN + * if their metadata pages aren't consequent. Just disable merging. + */ + return false; +#endif if (addr1 + vec1->bv_len != addr2) return false; if (xen_domain() && !xen_biovec_phys_mergeable(vec1, vec2->bv_page)) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 2222fa795284..c87c5416a802 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -216,6 +216,15 @@ struct page { not kmapped, ie. highmem) */ #endif /* WANT_PAGE_VIRTUAL */ +#ifdef CONFIG_KMSAN + /* + * Bits in struct page are scarce, so the LSB in *shadow is used to + * indicate whether the page should be ignored by KMSAN or not. + */ + struct page *shadow; + struct page *origin; +#endif + #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS int _last_cpupid; #endif diff --git a/include/linux/sched.h b/include/linux/sched.h index 72b20f33c56e..ba705f66f78c 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -1173,6 +1174,10 @@ struct task_struct { unsigned int kasan_depth; #endif +#ifdef CONFIG_KMSAN + struct kmsan_task_state kmsan; +#endif + #ifdef CONFIG_FUNCTION_GRAPH_TRACER /* Index of current stored address in ret_stack: */ int curr_ret_stack; diff --git a/init/main.c b/init/main.c index 91f6ebb30ef0..afcca2a38348 100644 --- a/init/main.c +++ b/init/main.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include #include @@ -554,6 +555,7 @@ static void __init mm_init(void) */ page_ext_init_flatmem(); report_meminit(); + kmsan_initialize_shadow(); mem_init(); kmem_cache_init(); kmemleak_init(); @@ -625,6 +627,7 @@ asmlinkage __visible void __init start_kernel(void) sort_main_extable(); trap_init(); mm_init(); + kmsan_initialize(); ftrace_init(); diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 460efd3ef742..529a86eb1dd1 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -233,8 +233,8 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (parts.slabindex > depot_index) { WARN(1, "slab index %d out of bounds (%d) for stack id %08x\n", parts.slabindex, depot_index, handle); - __memset(trace, 0, sizeof(*trace)); - return; + *entries = NULL; + return 0; } slab = stack_slabs[parts.slabindex]; stack = slab + offset; @@ -299,7 +299,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, * contexts and I/O. */ alloc_flags &= ~GFP_ZONEMASK; - alloc_flags &= (GFP_ATOMIC | GFP_KERNEL); + alloc_flags &= (GFP_ATOMIC | GFP_KERNEL | + __GFP_NO_KMSAN_SHADOW); alloc_flags |= __GFP_NOWARN; page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER); if (page) diff --git a/mm/Makefile b/mm/Makefile index d996846697ef..419e6e02dfaf 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -71,6 +71,7 @@ obj-$(CONFIG_PAGE_POISONING) += page_poison.o obj-$(CONFIG_SLAB) += slab.o obj-$(CONFIG_SLUB) += slub.o obj-$(CONFIG_KASAN) += kasan/ +obj-$(CONFIG_KMSAN) += kmsan/ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_MEMTEST) += memtest.o diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 179d55af5852..f9f38f7c1cd4 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -140,6 +140,12 @@ _c_flags += $(if $(patsubst n%,, \ $(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE)) endif +ifeq ($(CONFIG_KMSAN),y) +_c_flags += $(if $(patsubst n%,, \ + $(KMSAN_SANITIZE_$(basetarget).o)$(KMSAN_SANITIZE)y), \ + $(CFLAGS_KMSAN)) +endif + ifeq ($(CONFIG_UBSAN),y) _c_flags += $(if $(patsubst n%,, \ $(UBSAN_SANITIZE_$(basetarget).o)$(UBSAN_SANITIZE)$(CONFIG_UBSAN_SANITIZE_ALL)), \ -- 2.24.0.rc0.303.g954a862665-goog