From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B71CC432C0 for ; Mon, 2 Dec 2019 16:02:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 439742084F for ; Mon, 2 Dec 2019 16:02:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jm8e5e7U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 439742084F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D25B56B0003; Mon, 2 Dec 2019 11:02:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CAE656B0006; Mon, 2 Dec 2019 11:02:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B78906B0007; Mon, 2 Dec 2019 11:02:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0200.hostedemail.com [216.40.44.200]) by kanga.kvack.org (Postfix) with ESMTP id 97BF66B0003 for ; Mon, 2 Dec 2019 11:02:39 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 36F96180D8739 for ; Mon, 2 Dec 2019 16:02:39 +0000 (UTC) X-FDA: 76220669238.06.tramp94_5128d93e3e75e X-HE-Tag: tramp94_5128d93e3e75e X-Filterd-Recvd-Size: 18833 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 2 Dec 2019 16:02:37 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id w15so14013731wru.4 for ; Mon, 02 Dec 2019 08:02:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=bjc0W3N00MFttNvmtEUzc6+XSjQ0rdqEC5BJE6kC+vE=; b=jm8e5e7UYWYpKeuRI1wqMU8/sVsG/Kpqt9DDI6dtQtYjpkeMkgEZlUSDlFT6RXACcs YaeZoQ3R5BvZ6j+AzS4Lc8RNnWFMoy/IR0ZyZx1D9q5Qm8xAqA3YqjtZKbKcnINELhCc seldXeCEQooy70t5Jv9OU7IHuSa6nzj27z9CwXYN/SUOmFO1Ye/gOP7p7VFh4TE+RSnk 3eyakqTZN2Brrq76Zfl17BpMickqFW/jRdRjy9Trvny1VBknR3PO1Xwu1xTVal5YiCWE WawDpQH812uHrZxhZ+Zm4+fy7VkPPHxDg5PZm/Sv1qh6Jwu/5XjD8pyIaoM50Sptdqv+ kwFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=bjc0W3N00MFttNvmtEUzc6+XSjQ0rdqEC5BJE6kC+vE=; b=TzjbBIY/W/mAlgwWGHOnhLACp8mucSnBs8y5r4Ee91xkSdfHBRW1awXN69BZoD6A5j T5V3Xj7kFj1sbkQE8Bi7Rk/Zt/ybdNwpqkyhOCBhkxXZKxd0i4BRvYtRc8AjBXNgml+2 DSxsIcHLZE0E0m0gFOKxTfKA5/Ld4/VhUiW+YTLeBahsnBHMG9ol9KZkMdbO1SPB3cw8 KowxfewahEUAa3A+IFO88BCoeD4Rar7V7bNcb0sX1jWEvDG2ZdoOz1ttC8mZ8QKV8Mtt DUP8qfj4NPHYnhkz9v3VjHySkiTY8DBZY6aSOj1Q6eZWHf4bEjs1TWvMUMrh4joTHUwE E7jg== X-Gm-Message-State: APjAAAVtDG4IPmQeHSVKU2Xjd3mDhP7FfInVsWfdy+YFdi4xO1VIJj1l 8klHNbCqB19SbWxwDr2zd807jhzP9uLyDm9rLA5afg== X-Google-Smtp-Source: APXvYqwPuEfdQGuTtS41YPslsbhz3Yiq5Tqvf86criXgmqUFhe5yqpDYOyMm/zFabtgd0u4c/Sw5yM6zZU5EudRg05o= X-Received: by 2002:adf:e2cd:: with SMTP id d13mr71427909wrj.221.1575302555654; Mon, 02 Dec 2019 08:02:35 -0800 (PST) MIME-Version: 1.0 References: <20191122112621.204798-1-glider@google.com> In-Reply-To: From: Alexander Potapenko Date: Mon, 2 Dec 2019 17:02:24 +0100 Message-ID: Subject: Re: [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure To: Marco Elver Cc: Alexander Viro , Andreas Dilger , Andrew Morton , Andrey Konovalov , Andrey Ryabinin , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Christoph Hellwig , Christoph Hellwig , "Darrick J. Wong" , "David S. Miller" , Dmitry Torokhov , Dmitry Vyukov , Eric Biggers , Eric Dumazet , Eric Van Hensbergen , Greg Kroah-Hartman , Harry Wentland , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jason Wang , Jens Axboe , Marek Szyprowski , Mark Rutland , "Martin K. Petersen" , Martin Schwidefsky , Matthew Wilcox , "Michael S. Tsirkin" , Michal Simek , Petr Mladek , Qian Cai , Randy Dunlap , Robin Murphy , Sergey Senozhatsky , Steven Rostedt , Takashi Iwai , "Theodore Ts'o" , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Wolfram Sang , Linux Memory Management List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Archived-At: List-Archive: List-Post: On Fri, Nov 29, 2019 at 3:39 PM Marco Elver wrote: > > On Fri, 22 Nov 2019 at 12:26, wrote: > > > > KernelMemorySanitizer (KMSAN) is a detector of errors related to uses o= f > > uninitialized memory. It relies on compile-time Clang instrumentation > > (similar to MSan in the userspace: > > https://clang.llvm.org/docs/MemorySanitizer.html) > > and tracks the state of every bit of kernel memory, being able to repor= t > > an error if uninitialized value is used in a condition, dereferenced or > > copied to userspace, USB or network. > > > > KMSAN has reported more than 200 bugs in the past two years, most of > > them with the help of syzkaller (http://syzkaller.appspot.com). > > > > The proposed patchset contains KMSAN runtime implementation together > > with small changes to other subsystems needed to make KMSAN work. > > The latter changes fall into several categories: > > - nice-to-have features that are independent from KMSAN but simplify > > its implementation (stackdepot changes, CONFIG_GENERIC_CSUM etc.); > > - Kconfig changes that prohibit options incompatible with KMSAN; > > - calls to KMSAN runtime functions that help KMSAN do the bookkeeping > > (e.g. tell it to allocate, copy or delete the metadata); > > - calls to KMSAN runtime functions that tell KMSAN to check memory > > escaping the kernel for uninitialized values. These are required to > > increase the number of true positive error reports; > > - calls to runtime functions that tell KMSAN to ignore certain memory > > ranges to avoid false negative reports. Most certainly there can be > > better ways to deal with every such report. > > > > This patchset allows one to boot and run a defconfig+KMSAN kernel on a = QEMU > > without known major false positives. It however doesn't guarantee there > > are no false positives in drivers of certain devices or less tested > > subsystems, although KMSAN is actively tested on syzbot with quite a > > rich config. > > > > One may find it handy to review these patches in Gerrit: > > https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux= /+/1081 > > I've ensured the Change-Id: tags stay away from commit descriptions. > > > > The patchset was generated relative to Linux v5.4-rc5. > > > > I also apologize for not sending every patch in the previous series > > to all recipients of patches from that series. > > > > Note: checkpatch.pl complains a lot about the use of BUG_ON in KMSAN > > source. I don't have a strong opinion on this, but KMSAN is a debugging > > tool, so any runtime invariant violation in it renders the tool useless= . > > Therefore it doesn't make much sense to not terminate after a bug in > > KMSAN. > > > > Alexander Potapenko (36): > > stackdepot: check depot_index before accessing the stack slab > > stackdepot: build with -fno-builtin > > kasan: stackdepot: move filter_irq_stacks() to stackdepot.c > > stackdepot: reserve 5 extra bits in depot_stack_handle_t > > kmsan: add ReST documentation > > kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW > > kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__ > > kmsan: reduce vmalloc space > > kmsan: add KMSAN bits to struct page and struct task_struct > > kmsan: add KMSAN runtime > > kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot > > kmsan: define READ_ONCE_NOCHECK() > > kmsan: make READ_ONCE_TASK_STACK() return initialized values > > kmsan: x86: sync metadata pages on page fault > > kmsan: add tests for KMSAN > > crypto: kmsan: disable accelerated configs under KMSAN > > kmsan: x86: disable UNWINDER_ORC under KMSAN > > kmsan: disable LOCK_DEBUGGING_SUPPORT > > kmsan: x86/asm: add KMSAN hooks to entry_64.S > > kmsan: x86: increase stack sizes in KMSAN builds > > kmsan: disable KMSAN instrumentation for certain kernel parts > > kmsan: mm: call KMSAN hooks from SLUB code > > kmsan: call KMSAN hooks where needed > > kmsan: disable instrumentation of certain functions > > kmsan: unpoison |tlb| in arch_tlb_gather_mmu() > > kmsan: use __msan_memcpy() where possible. > > kmsan: hooks for copy_to_user() and friends > > kmsan: enable KMSAN builds > > kmsan: handle /dev/[u]random > > kmsan: virtio: check/unpoison scatterlist in vring_map_one_sg() > > kmsan: disable strscpy() optimization under KMSAN > > kmsan: add iomap support > > kmsan: dma: unpoison memory mapped by dma_direct_map_page() > > kmsan: disable physical page merging in biovec > > kmsan: ext4: skip block merging logic in ext4_mpage_readpages for > > KMSAN > > net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for > > KASAN/KMSAN > > > > To: Alexander Potapenko > > Cc: Alexander Viro > > Cc: Andreas Dilger > > Cc: Andrew Morton > > Cc: Andrey Konovalov > > Cc: Andrey Ryabinin > > Cc: Andy Lutomirski > > Cc: Ard Biesheuvel > > Cc: Arnd Bergmann > > Cc: Christoph Hellwig > > Cc: Christoph Hellwig > > Cc: Darrick J. Wong > > Cc: "David S. Miller" > > Cc: Dmitry Torokhov > > Cc: Dmitry Vyukov > > Cc: Eric Biggers > > Cc: Eric Dumazet > > Cc: Eric Van Hensbergen > > Cc: Greg Kroah-Hartman > > Cc: Harry Wentland > > Cc: Herbert Xu > > Cc: Ilya Leoshkevich > > Cc: Ingo Molnar > > Cc: Jason Wang > > Cc: Jens Axboe > > Cc: Marek Szyprowski > > Cc: Marco Elver > > Cc: Mark Rutland > > Cc: Martin K. Petersen > > Cc: Martin Schwidefsky > > Cc: Matthew Wilcox > > Cc: "Michael S. Tsirkin" > > Cc: Michal Simek > > Cc: Petr Mladek > > Cc: Qian Cai > > Cc: Randy Dunlap > > Cc: Robin Murphy > > Cc: Sergey Senozhatsky > > Cc: Steven Rostedt > > Cc: Takashi Iwai > > Cc: "Theodore Ts'o" > > Cc: Thomas Gleixner > > Cc: Vasily Gorbik > > Cc: Vegard Nossum > > Cc: Wolfram Sang > > Cc: linux-mm@kvack.org > > > > Documentation/dev-tools/index.rst | 1 + > > Documentation/dev-tools/kmsan.rst | 418 ++++++++++++++++++ > > Makefile | 3 +- > > arch/x86/Kconfig | 5 + > > arch/x86/Kconfig.debug | 3 + > > arch/x86/boot/Makefile | 2 + > > arch/x86/boot/compressed/Makefile | 2 + > > arch/x86/boot/compressed/misc.h | 1 + > > arch/x86/entry/common.c | 1 + > > arch/x86/entry/entry_64.S | 16 + > > arch/x86/entry/vdso/Makefile | 3 + > > arch/x86/include/asm/checksum.h | 10 +- > > arch/x86/include/asm/irq_regs.h | 1 + > > arch/x86/include/asm/kmsan.h | 117 +++++ > > arch/x86/include/asm/page_64.h | 13 + > > arch/x86/include/asm/page_64_types.h | 12 +- > > arch/x86/include/asm/pgtable_64_types.h | 15 + > > arch/x86/include/asm/string_64.h | 9 +- > > arch/x86/include/asm/syscall_wrapper.h | 1 + > > arch/x86/include/asm/uaccess.h | 12 + > > arch/x86/include/asm/unwind.h | 9 +- > > arch/x86/kernel/Makefile | 4 + > > arch/x86/kernel/apic/apic.c | 1 + > > arch/x86/kernel/cpu/Makefile | 1 + > > arch/x86/kernel/dumpstack_64.c | 1 + > > arch/x86/kernel/process_64.c | 5 + > > arch/x86/kernel/traps.c | 12 +- > > arch/x86/kernel/uprobes.c | 7 +- > > arch/x86/lib/Makefile | 2 + > > arch/x86/mm/Makefile | 2 + > > arch/x86/mm/fault.c | 20 + > > arch/x86/mm/ioremap.c | 3 + > > arch/x86/realmode/rm/Makefile | 2 + > > block/blk.h | 7 + > > crypto/Kconfig | 52 +++ > > drivers/char/random.c | 6 + > > drivers/firmware/efi/libstub/Makefile | 1 + > > drivers/usb/core/urb.c | 2 + > > drivers/virtio/virtio_ring.c | 10 +- > > fs/ext4/readpage.c | 11 + > > include/asm-generic/cacheflush.h | 7 +- > > include/asm-generic/uaccess.h | 12 +- > > include/linux/compiler-clang.h | 8 + > > include/linux/compiler-gcc.h | 5 + > > include/linux/compiler.h | 13 +- > > include/linux/gfp.h | 4 +- > > include/linux/highmem.h | 4 + > > include/linux/kmsan-checks.h | 122 +++++ > > include/linux/kmsan.h | 143 ++++++ > > include/linux/mm_types.h | 9 + > > include/linux/sched.h | 5 + > > include/linux/stackdepot.h | 10 + > > include/linux/string.h | 2 + > > include/linux/uaccess.h | 32 +- > > init/main.c | 3 + > > kernel/Makefile | 1 + > > kernel/dma/direct.c | 1 + > > kernel/exit.c | 2 + > > kernel/fork.c | 2 + > > kernel/kthread.c | 2 + > > kernel/printk/printk.c | 6 + > > kernel/profile.c | 1 + > > kernel/sched/core.c | 6 + > > kernel/softirq.c | 5 + > > lib/Kconfig.debug | 5 + > > lib/Kconfig.kmsan | 22 + > > lib/Makefile | 6 + > > lib/iomap.c | 40 ++ > > lib/ioremap.c | 5 + > > lib/iov_iter.c | 6 + > > lib/stackdepot.c | 69 ++- > > lib/string.c | 5 +- > > lib/test_kmsan.c | 231 ++++++++++ > > lib/usercopy.c | 6 +- > > mm/Makefile | 1 + > > mm/compaction.c | 9 + > > mm/gup.c | 3 + > > mm/kasan/common.c | 23 - > > mm/kmsan/Makefile | 4 + > > mm/kmsan/kmsan.c | 563 ++++++++++++++++++++++++ > > mm/kmsan/kmsan.h | 146 ++++++ > > mm/kmsan/kmsan_entry.c | 118 +++++ > > mm/kmsan/kmsan_hooks.c | 422 ++++++++++++++++++ > > mm/kmsan/kmsan_init.c | 88 ++++ > > mm/kmsan/kmsan_instr.c | 259 +++++++++++ > > mm/kmsan/kmsan_report.c | 133 ++++++ > > mm/kmsan/kmsan_shadow.c | 543 +++++++++++++++++++++++ > > mm/kmsan/kmsan_shadow.h | 30 ++ > > mm/memory.c | 2 + > > mm/mmu_gather.c | 10 + > > mm/page_alloc.c | 16 + > > mm/slub.c | 34 +- > > mm/vmalloc.c | 23 +- > > net/sched/sch_generic.c | 2 + > > scripts/Makefile.kmsan | 12 + > > scripts/Makefile.lib | 6 + > > 96 files changed, 3983 insertions(+), 67 deletions(-) > > create mode 100644 Documentation/dev-tools/kmsan.rst > > create mode 100644 arch/x86/include/asm/kmsan.h > > create mode 100644 include/linux/kmsan-checks.h > > create mode 100644 include/linux/kmsan.h > > create mode 100644 lib/Kconfig.kmsan > > create mode 100644 lib/test_kmsan.c > > create mode 100644 mm/kmsan/Makefile > > create mode 100644 mm/kmsan/kmsan.c > > create mode 100644 mm/kmsan/kmsan.h > > create mode 100644 mm/kmsan/kmsan_entry.c > > create mode 100644 mm/kmsan/kmsan_hooks.c > > create mode 100644 mm/kmsan/kmsan_init.c > > create mode 100644 mm/kmsan/kmsan_instr.c > > create mode 100644 mm/kmsan/kmsan_report.c > > create mode 100644 mm/kmsan/kmsan_shadow.c > > create mode 100644 mm/kmsan/kmsan_shadow.h > > create mode 100644 scripts/Makefile.kmsan > > There are currently lots of UACCESS warnings -- should the core > runtime functions be added to the whitelist in tools/objtool/check.c? I've tried to do this properly, but failed. What I consider to be an intuitive solution is to add KMSAN interface functions (e.g. __msan_metadata_ptr_for_load_X) to the whitelist in tools/objtool/check.c. None of these functions is expected to touch userspace memory, but they can be called in the uaccess context, as the compiler adds them to every memory access. Turns out however it's not enough to whitelist those functions, as they are viral: after whitelisting them I get warnings about their callees. The list of functions called by e.g. kmsan_get_shadow_origin_ptr() include __warn_printk() and other non-KMSAN stuff, so I have to call user_access_save()/user_access_restore() to keep those warnings from spreading. But this slows the whole runtime down by ~50% for no reason. > Thanks, > -- Marco --=20 Alexander Potapenko Software Engineer Google Germany GmbH Erika-Mann-Stra=C3=9Fe, 33 80636 M=C3=BCnchen Gesch=C3=A4ftsf=C3=BChrer: Paul Manicle, Halimah DeLaine Prado Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg