From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.9 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C98FFC4741F for ; Tue, 29 Sep 2020 13:38:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6C9CB216C4 for ; Tue, 29 Sep 2020 13:38:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YQlZfknS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729957AbgI2Niq (ORCPT ); Tue, 29 Sep 2020 09:38:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725306AbgI2Nio (ORCPT ); Tue, 29 Sep 2020 09:38:44 -0400 Received: from mail-wr1-x44a.google.com (mail-wr1-x44a.google.com [IPv6:2a00:1450:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC35AC0613D0 for ; Tue, 29 Sep 2020 06:38:43 -0700 (PDT) Received: by mail-wr1-x44a.google.com with SMTP id b7so1779702wrn.6 for ; Tue, 29 Sep 2020 06:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=z1cXfrxOGzx7xyJIGn8IR6C4kJWaluaZ0MOMBbLzink=; b=YQlZfknSSvNvRczIa3w2KnrV406GAOWv4riuiKZDNf0muEq5IprfLHxEuf4QWG2X6v KVEafDVUZDZQ/lnlrbDUNKx/knoe0JXG3MFvQm3KqRbrBn5QRqhqvGhI0yC0xwPgNC+k Y4P7HPEPuFHow+cjpjzAfQn3BrGuvqTxF3SmyzIX4KIrgBxhqNT5NVw3qJ7UIdn9etXc UzivMLi+g3ro2cTRKuq3xjEFUtrcCDeVEgq/8uVv6TYifMgCu9AdmCnbkKnIgM6dpAYT cfPBKuaPXzQ+F6v8J2dAtTQTTOV2n/MRnS8WB1gJcHzeNwAwNLmiUohJWLw6JRw8ULih 0Ang== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=z1cXfrxOGzx7xyJIGn8IR6C4kJWaluaZ0MOMBbLzink=; b=IwGDgbC+Hd//A362TLohuqkcN7GY13bqA/TGxNwQh7xCGgvHF/+jwYp/UyNI8p/RTn IJ9UgLKNEMfFyWWoCYSU58D8IARQTCfklGgvqZ6tXBpcqEFXDqtHHW1xSBB0RrqzJEYQ Zsr+I8qhuRBOWRUQOtivJBrH2uRc5DTk9PhhkfVZxfazwqXwNaQnClr0hO0I4NrE23oE aujRbVAhEJA8JuZ8UJaUxYxdMyvJXqpITROhnUbJoAeagD1oYOnjHdyBf/YREuktXm3e E6ZoRpI2wRypAHJ2oEWk2Xm6h8i6OpmP6V7kuIH+3lsHuCyJGD0Ep0yZVQZSvBAKq/Ee jWcw== X-Gm-Message-State: AOAM533KlGTdtfH1mzWzOyGBDmoqbwu5LWJ8+dt/7NlXVSZ8GNFIsRHE x4AN/4vCcPYCyj4yEgn6YQpHaVQJgA== X-Google-Smtp-Source: ABdhPJzLfb+1L+DLOD5HANT71QD+TvopcJoj2qHEWsAOEUajKuEbN72wT6s+YP/on4qsVc9AwMoyktwOfw== Sender: "elver via sendgmr" X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:a7b:cd08:: with SMTP id f8mr4611202wmj.124.1601386721987; Tue, 29 Sep 2020 06:38:41 -0700 (PDT) Date: Tue, 29 Sep 2020 15:38:03 +0200 Message-Id: <20200929133814.2834621-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.28.0.709.gb0816b6eb0-goog Subject: [PATCH v4 00/11] KFENCE: A low-overhead sampling-based memory safety error detector From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com, aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de, catalin.marinas@arm.com, cl@linux.com, dave.hansen@linux.intel.com, rientjes@google.com, dvyukov@google.com, edumazet@google.com, gregkh@linuxfoundation.org, hdanton@sina.com, mingo@redhat.com, jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net, iamjoonsoo.kim@lge.com, keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org, peterz@infradead.org, sjpark@amazon.com, tglx@linutronix.de, vbabka@suse.cz, will@kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a low-overhead sampling-based memory safety error detector of heap use-after-free, invalid-free, and out-of-bounds access errors. This series enables KFENCE for the x86 and arm64 architectures, and adds KFENCE hooks to the SLAB and SLUB allocators. KFENCE is designed to be enabled in production kernels, and has near zero performance overhead. Compared to KASAN, KFENCE trades performance for precision. The main motivation behind KFENCE's design, is that with enough total uptime KFENCE will detect bugs in code paths not typically exercised by non-production test workloads. One way to quickly achieve a large enough total uptime is when the tool is deployed across a large fleet of machines. KFENCE objects each reside on a dedicated page, at either the left or right page boundaries. The pages to the left and right of the object page are "guard pages", whose attributes are changed to a protected state, and cause page faults on any attempted access to them. Such page faults are then intercepted by KFENCE, which handles the fault gracefully by reporting a memory access error. Guarded allocations are set up based on a sample interval (can be set via kfence.sample_interval). After expiration of the sample interval, the next allocation through the main allocator (SLAB or SLUB) returns a guarded allocation from the KFENCE object pool. At this point, the timer is reset, and the next allocation is set up after the expiration of the interval. To enable/disable a KFENCE allocation through the main allocator's fast-path without overhead, KFENCE relies on static branches via the static keys infrastructure. The static branch is toggled to redirect the allocation to KFENCE. The KFENCE memory pool is of fixed size, and if the pool is exhausted no further KFENCE allocations occur. The default config is conservative with only 255 objects, resulting in a pool size of 2 MiB (with 4 KiB pages). We have verified by running synthetic benchmarks (sysbench I/O, hackbench) that a kernel with KFENCE is performance-neutral compared to a non-KFENCE baseline kernel. KFENCE is inspired by GWP-ASan [1], a userspace tool with similar properties. The name "KFENCE" is a homage to the Electric Fence Malloc Debugger [2]. For more details, see Documentation/dev-tools/kfence.rst added in the series -- also viewable here: https://raw.githubusercontent.com/google/kasan/kfence/Documentation/dev-tools/kfence.rst [1] http://llvm.org/docs/GwpAsan.html [2] https://linux.die.net/man/3/efence v4: * MAINTAINERS: Split out from first patch. * Make static memory pool's attrs entirely arch-dependent. * Fix report generation if __slab_free tail-called. * Clarify RCU test comment [reported by Paul E. McKenney]. v3: https://lkml.kernel.org/r/20200921132611.1700350-1-elver@google.com * Rewrite SLAB/SLUB patch descriptions to clarify need for 'orig_size'. * Various smaller fixes (see details in patches). v2: https://lkml.kernel.org/r/20200915132046.3332537-1-elver@google.com * Various comment/documentation changes (see details in patches). * Various smaller fixes (see details in patches). * Change all reports to reference the kfence object, "kfence-#nn". * Skip allocation/free internals stack trace. * Rework KMEMLEAK compatibility patch. RFC/v1: https://lkml.kernel.org/r/20200907134055.2878499-1-elver@google.com Alexander Potapenko (6): mm: add Kernel Electric-Fence infrastructure x86, kfence: enable KFENCE for x86 mm, kfence: insert KFENCE hooks for SLAB mm, kfence: insert KFENCE hooks for SLUB kfence, kasan: make KFENCE compatible with KASAN kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver (5): arm64, kfence: enable KFENCE for ARM64 kfence, lockdep: make KFENCE compatible with lockdep kfence, Documentation: add KFENCE documentation kfence: add test suite MAINTAINERS: Add entry for KFENCE Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kfence.rst | 291 +++++++++++ MAINTAINERS | 11 + arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 ++ arch/arm64/mm/fault.c | 4 + arch/x86/Kconfig | 2 + arch/x86/include/asm/kfence.h | 60 +++ arch/x86/mm/fault.c | 4 + include/linux/kfence.h | 174 +++++++ init/main.c | 2 + kernel/locking/lockdep.c | 8 + lib/Kconfig.debug | 1 + lib/Kconfig.kfence | 78 +++ mm/Makefile | 1 + mm/kasan/common.c | 7 + mm/kfence/Makefile | 6 + mm/kfence/core.c | 733 +++++++++++++++++++++++++++ mm/kfence/kfence.h | 102 ++++ mm/kfence/kfence_test.c | 783 +++++++++++++++++++++++++++++ mm/kfence/report.c | 225 +++++++++ mm/kmemleak.c | 6 + mm/slab.c | 46 +- mm/slab_common.c | 6 +- mm/slub.c | 72 ++- 25 files changed, 2631 insertions(+), 32 deletions(-) create mode 100644 Documentation/dev-tools/kfence.rst create mode 100644 arch/arm64/include/asm/kfence.h create mode 100644 arch/x86/include/asm/kfence.h create mode 100644 include/linux/kfence.h create mode 100644 lib/Kconfig.kfence create mode 100644 mm/kfence/Makefile create mode 100644 mm/kfence/core.c create mode 100644 mm/kfence/kfence.h create mode 100644 mm/kfence/kfence_test.c create mode 100644 mm/kfence/report.c -- 2.28.0.709.gb0816b6eb0-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D57CC4727D for ; Tue, 29 Sep 2020 13:38:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 738DB20848 for ; Tue, 29 Sep 2020 13:38:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="YQlZfknS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 738DB20848 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 89C7D8E0001; Tue, 29 Sep 2020 09:38:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84F936B005D; Tue, 29 Sep 2020 09:38:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73DB28E0001; Tue, 29 Sep 2020 09:38:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id 5DD836B005C for ; Tue, 29 Sep 2020 09:38:44 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1AD8E12D5 for ; Tue, 29 Sep 2020 13:38:44 +0000 (UTC) X-FDA: 77316204168.21.sense87_2806da12718a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id ED2B3180458C0 for ; Tue, 29 Sep 2020 13:38:43 +0000 (UTC) X-HE-Tag: sense87_2806da12718a X-Filterd-Recvd-Size: 9069 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 29 Sep 2020 13:38:43 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id c200so1847542wmd.5 for ; Tue, 29 Sep 2020 06:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=z1cXfrxOGzx7xyJIGn8IR6C4kJWaluaZ0MOMBbLzink=; b=YQlZfknSSvNvRczIa3w2KnrV406GAOWv4riuiKZDNf0muEq5IprfLHxEuf4QWG2X6v KVEafDVUZDZQ/lnlrbDUNKx/knoe0JXG3MFvQm3KqRbrBn5QRqhqvGhI0yC0xwPgNC+k Y4P7HPEPuFHow+cjpjzAfQn3BrGuvqTxF3SmyzIX4KIrgBxhqNT5NVw3qJ7UIdn9etXc UzivMLi+g3ro2cTRKuq3xjEFUtrcCDeVEgq/8uVv6TYifMgCu9AdmCnbkKnIgM6dpAYT cfPBKuaPXzQ+F6v8J2dAtTQTTOV2n/MRnS8WB1gJcHzeNwAwNLmiUohJWLw6JRw8ULih 0Ang== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=z1cXfrxOGzx7xyJIGn8IR6C4kJWaluaZ0MOMBbLzink=; b=P+fEhyLHLvhrwVrc4Y5mOBRODlQ0j/IDrW7V/YE2oAzOGDYncghzwJT550eX8/t+dp QYH4DzWrFgVh2TNztjZtpAAPgKLuCwUmtVweRbFNDPIh16rdHim2hvFN/WknLDlgtnEB p8B9txyWCUFm9xqwpdOmPHyhnUZqo2fqjjewBc2J6NBK+AbJoLcQa3LL54fk1ACE6pjb RHgIpi7ToRNck01Tmt8GTk1vplj6xkWAAk/TyUWoG0qja1HI9xeAEMILyo/6TY4MJ40L aa89V/IG35pJxGiqiKqm/uxHWk5OHPVAqp9ILP8dhbGRVuwvfA13KMecOuh+Yi0WleU+ foDQ== X-Gm-Message-State: AOAM530PUdTDHRemR5Q6a3FCCn4osV5xqust4A/I2RmJtFV982A9D8+t AJHaL+LNoaZG6pshcMEn/sqBSK4jKw== X-Google-Smtp-Source: ABdhPJzLfb+1L+DLOD5HANT71QD+TvopcJoj2qHEWsAOEUajKuEbN72wT6s+YP/on4qsVc9AwMoyktwOfw== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:a7b:cd08:: with SMTP id f8mr4611202wmj.124.1601386721987; Tue, 29 Sep 2020 06:38:41 -0700 (PDT) Date: Tue, 29 Sep 2020 15:38:03 +0200 Message-Id: <20200929133814.2834621-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.28.0.709.gb0816b6eb0-goog Subject: [PATCH v4 00/11] KFENCE: A low-overhead sampling-based memory safety error detector From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com Cc: hpa@zytor.com, paulmck@kernel.org, andreyknvl@google.com, aryabinin@virtuozzo.com, luto@kernel.org, bp@alien8.de, catalin.marinas@arm.com, cl@linux.com, dave.hansen@linux.intel.com, rientjes@google.com, dvyukov@google.com, edumazet@google.com, gregkh@linuxfoundation.org, hdanton@sina.com, mingo@redhat.com, jannh@google.com, Jonathan.Cameron@huawei.com, corbet@lwn.net, iamjoonsoo.kim@lge.com, keescook@chromium.org, mark.rutland@arm.com, penberg@kernel.org, peterz@infradead.org, sjpark@amazon.com, tglx@linutronix.de, vbabka@suse.cz, will@kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a low-overhead sampling-based memory safety error detector of heap use-after-free, invalid-free, and out-of-bounds access errors. This series enables KFENCE for the x86 and arm64 architectures, and adds KFENCE hooks to the SLAB and SLUB allocators. KFENCE is designed to be enabled in production kernels, and has near zero performance overhead. Compared to KASAN, KFENCE trades performance for precision. The main motivation behind KFENCE's design, is that with enough total uptime KFENCE will detect bugs in code paths not typically exercised by non-production test workloads. One way to quickly achieve a large enough total uptime is when the tool is deployed across a large fleet of machines. KFENCE objects each reside on a dedicated page, at either the left or right page boundaries. The pages to the left and right of the object page are "guard pages", whose attributes are changed to a protected state, and cause page faults on any attempted access to them. Such page faults are then intercepted by KFENCE, which handles the fault gracefully by reporting a memory access error. Guarded allocations are set up based on a sample interval (can be set via kfence.sample_interval). After expiration of the sample interval, the next allocation through the main allocator (SLAB or SLUB) returns a guarded allocation from the KFENCE object pool. At this point, the timer is reset, and the next allocation is set up after the expiration of the interval. To enable/disable a KFENCE allocation through the main allocator's fast-path without overhead, KFENCE relies on static branches via the static keys infrastructure. The static branch is toggled to redirect the allocation to KFENCE. The KFENCE memory pool is of fixed size, and if the pool is exhausted no further KFENCE allocations occur. The default config is conservative with only 255 objects, resulting in a pool size of 2 MiB (with 4 KiB pages). We have verified by running synthetic benchmarks (sysbench I/O, hackbench) that a kernel with KFENCE is performance-neutral compared to a non-KFENCE baseline kernel. KFENCE is inspired by GWP-ASan [1], a userspace tool with similar properties. The name "KFENCE" is a homage to the Electric Fence Malloc Debugger [2]. For more details, see Documentation/dev-tools/kfence.rst added in the series -- also viewable here: https://raw.githubusercontent.com/google/kasan/kfence/Documentation/dev-tools/kfence.rst [1] http://llvm.org/docs/GwpAsan.html [2] https://linux.die.net/man/3/efence v4: * MAINTAINERS: Split out from first patch. * Make static memory pool's attrs entirely arch-dependent. * Fix report generation if __slab_free tail-called. * Clarify RCU test comment [reported by Paul E. McKenney]. v3: https://lkml.kernel.org/r/20200921132611.1700350-1-elver@google.com * Rewrite SLAB/SLUB patch descriptions to clarify need for 'orig_size'. * Various smaller fixes (see details in patches). v2: https://lkml.kernel.org/r/20200915132046.3332537-1-elver@google.com * Various comment/documentation changes (see details in patches). * Various smaller fixes (see details in patches). * Change all reports to reference the kfence object, "kfence-#nn". * Skip allocation/free internals stack trace. * Rework KMEMLEAK compatibility patch. RFC/v1: https://lkml.kernel.org/r/20200907134055.2878499-1-elver@google.com Alexander Potapenko (6): mm: add Kernel Electric-Fence infrastructure x86, kfence: enable KFENCE for x86 mm, kfence: insert KFENCE hooks for SLAB mm, kfence: insert KFENCE hooks for SLUB kfence, kasan: make KFENCE compatible with KASAN kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver (5): arm64, kfence: enable KFENCE for ARM64 kfence, lockdep: make KFENCE compatible with lockdep kfence, Documentation: add KFENCE documentation kfence: add test suite MAINTAINERS: Add entry for KFENCE Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kfence.rst | 291 +++++++++++ MAINTAINERS | 11 + arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 ++ arch/arm64/mm/fault.c | 4 + arch/x86/Kconfig | 2 + arch/x86/include/asm/kfence.h | 60 +++ arch/x86/mm/fault.c | 4 + include/linux/kfence.h | 174 +++++++ init/main.c | 2 + kernel/locking/lockdep.c | 8 + lib/Kconfig.debug | 1 + lib/Kconfig.kfence | 78 +++ mm/Makefile | 1 + mm/kasan/common.c | 7 + mm/kfence/Makefile | 6 + mm/kfence/core.c | 733 +++++++++++++++++++++++++++ mm/kfence/kfence.h | 102 ++++ mm/kfence/kfence_test.c | 783 +++++++++++++++++++++++++++++ mm/kfence/report.c | 225 +++++++++ mm/kmemleak.c | 6 + mm/slab.c | 46 +- mm/slab_common.c | 6 +- mm/slub.c | 72 ++- 25 files changed, 2631 insertions(+), 32 deletions(-) create mode 100644 Documentation/dev-tools/kfence.rst create mode 100644 arch/arm64/include/asm/kfence.h create mode 100644 arch/x86/include/asm/kfence.h create mode 100644 include/linux/kfence.h create mode 100644 lib/Kconfig.kfence create mode 100644 mm/kfence/Makefile create mode 100644 mm/kfence/core.c create mode 100644 mm/kfence/kfence.h create mode 100644 mm/kfence/kfence_test.c create mode 100644 mm/kfence/report.c -- 2.28.0.709.gb0816b6eb0-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47C1BC47423 for ; Tue, 29 Sep 2020 13:40:31 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AD94B207C4 for ; Tue, 29 Sep 2020 13:40:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="UpKemzzv"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="YQlZfknS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD94B207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:Mime-Version:Message-Id:Date: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=hpWpC6zf6IafEEGQzDaXeGM7b41S1whAUgCdMD4FYRw=; b=UpKemzzvGAZy9Cp9fxBmT3IKeR dt0YOQXTxqF4SsUtCzVkUeCii6gcf1h+MyiBGgZQ3J9jpVMrF1oeHuOpoFS+PrExckctH/SUy8zBR 0ylfR5L8/Oz9qD5OfHiZBVsPiK55GP+xm7+SHy3lRxy9otbFul4IBxpPtlYTqkPk1NekaBtI8EZvP o58feD4QovXupgtVIx0xBLFgRmYOT4mYgUqITLLE8+fM43PhkdwBiKPCDd3w4i8ZLnQ/CjWlpsguW x1RASZAVrXoE+GNaiRMJHxTOGvaljaE2Stv2M0BCoVY8NekcVJojnFkpPoqKJEW463r8b2h+MoKN5 VzrHys/g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kNFqE-0002hD-3M; Tue, 29 Sep 2020 13:38:50 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kNFq9-0002fF-Hr for linux-arm-kernel@lists.infradead.org; Tue, 29 Sep 2020 13:38:46 +0000 Received: by mail-wm1-x349.google.com with SMTP id p20so1730479wmg.0 for ; Tue, 29 Sep 2020 06:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=z1cXfrxOGzx7xyJIGn8IR6C4kJWaluaZ0MOMBbLzink=; b=YQlZfknSSvNvRczIa3w2KnrV406GAOWv4riuiKZDNf0muEq5IprfLHxEuf4QWG2X6v KVEafDVUZDZQ/lnlrbDUNKx/knoe0JXG3MFvQm3KqRbrBn5QRqhqvGhI0yC0xwPgNC+k Y4P7HPEPuFHow+cjpjzAfQn3BrGuvqTxF3SmyzIX4KIrgBxhqNT5NVw3qJ7UIdn9etXc UzivMLi+g3ro2cTRKuq3xjEFUtrcCDeVEgq/8uVv6TYifMgCu9AdmCnbkKnIgM6dpAYT cfPBKuaPXzQ+F6v8J2dAtTQTTOV2n/MRnS8WB1gJcHzeNwAwNLmiUohJWLw6JRw8ULih 0Ang== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=z1cXfrxOGzx7xyJIGn8IR6C4kJWaluaZ0MOMBbLzink=; b=ZsXqnWYqMNFOWiVXKZwAaV1KM7coFUKFZISJoDMoqN/JLINiNc3LzOV+jOlQFnX98e 5x+Es14YOw+Y6r1RcvdAfbbfLZEO2eDZhs618kNQu2/e7M7gcBnwPqxLOEdjnRz4FzID bp4mbVMEU+6VJS76PD1wWzAvvd2mhNQ4RP9nt23+rDPahRYt0qQZOlazBGkSfxuknvjm W1DFxhfEwkZPzlooSJlAG8TduIX07Y0ZxiibuYM9UgYUFv/pniam7KiO/u/H7jtGRWYd sv371CPKNnRgxNn4f8ltNGgxJ2bCnFee7zIXT1yZmN06c+tA3JSCw9/1GsK/0yE7PkEg DtHw== X-Gm-Message-State: AOAM532QFtF+lLZ3YyncrNs+B9366ce+UDONSaTDLTVGyBe2dw3ODQLl rey+SChG4dJ0ZG+Zj4i2AF/Z+XxceQ== X-Google-Smtp-Source: ABdhPJzLfb+1L+DLOD5HANT71QD+TvopcJoj2qHEWsAOEUajKuEbN72wT6s+YP/on4qsVc9AwMoyktwOfw== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:f693:9fff:fef4:2449]) (user=elver job=sendgmr) by 2002:a7b:cd08:: with SMTP id f8mr4611202wmj.124.1601386721987; Tue, 29 Sep 2020 06:38:41 -0700 (PDT) Date: Tue, 29 Sep 2020 15:38:03 +0200 Message-Id: <20200929133814.2834621-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.28.0.709.gb0816b6eb0-goog Subject: [PATCH v4 00/11] KFENCE: A low-overhead sampling-based memory safety error detector From: Marco Elver To: elver@google.com, akpm@linux-foundation.org, glider@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200929_093845_608444_E39093A6 X-CRM114-Status: GOOD ( 17.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, hdanton@sina.com, linux-doc@vger.kernel.org, peterz@infradead.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, edumazet@google.com, hpa@zytor.com, cl@linux.com, will@kernel.org, sjpark@amazon.com, corbet@lwn.net, x86@kernel.org, kasan-dev@googlegroups.com, mingo@redhat.com, vbabka@suse.cz, rientjes@google.com, aryabinin@virtuozzo.com, keescook@chromium.org, paulmck@kernel.org, jannh@google.com, andreyknvl@google.com, bp@alien8.de, luto@kernel.org, Jonathan.Cameron@huawei.com, tglx@linutronix.de, dvyukov@google.com, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, penberg@kernel.org, iamjoonsoo.kim@lge.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This adds the Kernel Electric-Fence (KFENCE) infrastructure. KFENCE is a low-overhead sampling-based memory safety error detector of heap use-after-free, invalid-free, and out-of-bounds access errors. This series enables KFENCE for the x86 and arm64 architectures, and adds KFENCE hooks to the SLAB and SLUB allocators. KFENCE is designed to be enabled in production kernels, and has near zero performance overhead. Compared to KASAN, KFENCE trades performance for precision. The main motivation behind KFENCE's design, is that with enough total uptime KFENCE will detect bugs in code paths not typically exercised by non-production test workloads. One way to quickly achieve a large enough total uptime is when the tool is deployed across a large fleet of machines. KFENCE objects each reside on a dedicated page, at either the left or right page boundaries. The pages to the left and right of the object page are "guard pages", whose attributes are changed to a protected state, and cause page faults on any attempted access to them. Such page faults are then intercepted by KFENCE, which handles the fault gracefully by reporting a memory access error. Guarded allocations are set up based on a sample interval (can be set via kfence.sample_interval). After expiration of the sample interval, the next allocation through the main allocator (SLAB or SLUB) returns a guarded allocation from the KFENCE object pool. At this point, the timer is reset, and the next allocation is set up after the expiration of the interval. To enable/disable a KFENCE allocation through the main allocator's fast-path without overhead, KFENCE relies on static branches via the static keys infrastructure. The static branch is toggled to redirect the allocation to KFENCE. The KFENCE memory pool is of fixed size, and if the pool is exhausted no further KFENCE allocations occur. The default config is conservative with only 255 objects, resulting in a pool size of 2 MiB (with 4 KiB pages). We have verified by running synthetic benchmarks (sysbench I/O, hackbench) that a kernel with KFENCE is performance-neutral compared to a non-KFENCE baseline kernel. KFENCE is inspired by GWP-ASan [1], a userspace tool with similar properties. The name "KFENCE" is a homage to the Electric Fence Malloc Debugger [2]. For more details, see Documentation/dev-tools/kfence.rst added in the series -- also viewable here: https://raw.githubusercontent.com/google/kasan/kfence/Documentation/dev-tools/kfence.rst [1] http://llvm.org/docs/GwpAsan.html [2] https://linux.die.net/man/3/efence v4: * MAINTAINERS: Split out from first patch. * Make static memory pool's attrs entirely arch-dependent. * Fix report generation if __slab_free tail-called. * Clarify RCU test comment [reported by Paul E. McKenney]. v3: https://lkml.kernel.org/r/20200921132611.1700350-1-elver@google.com * Rewrite SLAB/SLUB patch descriptions to clarify need for 'orig_size'. * Various smaller fixes (see details in patches). v2: https://lkml.kernel.org/r/20200915132046.3332537-1-elver@google.com * Various comment/documentation changes (see details in patches). * Various smaller fixes (see details in patches). * Change all reports to reference the kfence object, "kfence-#nn". * Skip allocation/free internals stack trace. * Rework KMEMLEAK compatibility patch. RFC/v1: https://lkml.kernel.org/r/20200907134055.2878499-1-elver@google.com Alexander Potapenko (6): mm: add Kernel Electric-Fence infrastructure x86, kfence: enable KFENCE for x86 mm, kfence: insert KFENCE hooks for SLAB mm, kfence: insert KFENCE hooks for SLUB kfence, kasan: make KFENCE compatible with KASAN kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver (5): arm64, kfence: enable KFENCE for ARM64 kfence, lockdep: make KFENCE compatible with lockdep kfence, Documentation: add KFENCE documentation kfence: add test suite MAINTAINERS: Add entry for KFENCE Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kfence.rst | 291 +++++++++++ MAINTAINERS | 11 + arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kfence.h | 39 ++ arch/arm64/mm/fault.c | 4 + arch/x86/Kconfig | 2 + arch/x86/include/asm/kfence.h | 60 +++ arch/x86/mm/fault.c | 4 + include/linux/kfence.h | 174 +++++++ init/main.c | 2 + kernel/locking/lockdep.c | 8 + lib/Kconfig.debug | 1 + lib/Kconfig.kfence | 78 +++ mm/Makefile | 1 + mm/kasan/common.c | 7 + mm/kfence/Makefile | 6 + mm/kfence/core.c | 733 +++++++++++++++++++++++++++ mm/kfence/kfence.h | 102 ++++ mm/kfence/kfence_test.c | 783 +++++++++++++++++++++++++++++ mm/kfence/report.c | 225 +++++++++ mm/kmemleak.c | 6 + mm/slab.c | 46 +- mm/slab_common.c | 6 +- mm/slub.c | 72 ++- 25 files changed, 2631 insertions(+), 32 deletions(-) create mode 100644 Documentation/dev-tools/kfence.rst create mode 100644 arch/arm64/include/asm/kfence.h create mode 100644 arch/x86/include/asm/kfence.h create mode 100644 include/linux/kfence.h create mode 100644 lib/Kconfig.kfence create mode 100644 mm/kfence/Makefile create mode 100644 mm/kfence/core.c create mode 100644 mm/kfence/kfence.h create mode 100644 mm/kfence/kfence_test.c create mode 100644 mm/kfence/report.c -- 2.28.0.709.gb0816b6eb0-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel