From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6644C63697 for ; Mon, 23 Nov 2020 20:15:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2AB6F2074B for ; Mon, 23 Nov 2020 20:15:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Va6STH1a" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2AB6F2074B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0D8C26B00A4; Mon, 23 Nov 2020 15:15:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 08B9B6B00A5; Mon, 23 Nov 2020 15:15:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E948D6B00A6; Mon, 23 Nov 2020 15:15:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0213.hostedemail.com [216.40.44.213]) by kanga.kvack.org (Postfix) with ESMTP id BE8776B00A4 for ; Mon, 23 Nov 2020 15:15:38 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 60F9B180AD801 for ; Mon, 23 Nov 2020 20:15:38 +0000 (UTC) X-FDA: 77516788356.06.hook82_5f1314d27368 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id D0E401003D991 for ; Mon, 23 Nov 2020 20:15:37 +0000 (UTC) X-HE-Tag: hook82_5f1314d27368 X-Filterd-Recvd-Size: 7164 Received: from mail-qt1-f202.google.com (mail-qt1-f202.google.com [209.85.160.202]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Nov 2020 20:15:37 +0000 (UTC) Received: by mail-qt1-f202.google.com with SMTP id t17so2183342qtp.3 for ; Mon, 23 Nov 2020 12:15:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=jlSh18JA+tyc0YTMYNGOzncWOn17Ui2VWegewVhQQAI=; b=Va6STH1aX4wYunU0iMuz4BvFHnwWptWCMWXmt/xofeqd3owQuGpwtvjYit4rl6mlW2 hhm5iDeTzV3r8r1qtqi5fdwTQs/TSfD/bCoDouUVq/mqil3HrggfywpVxpheL6ZEWpIl zDXMe40hvq88yS3PuIjxsWaxgjZyKl/B05KVNcChbJkk49tAW2TDEREdWyEig5SkLtvs 5K5C3H1Yizp2NapbQ8UOeKQZdiMwc5N0OfbB8Dc3yzVaEvt/fAhl005CJycepaLAgkY1 zyVIgqpv8eZepV66RBshT0P3UBs6jkxe5tAKTYnNSfeugX8uOp+AF7kVSkrFVfKZ4WBw kmbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=jlSh18JA+tyc0YTMYNGOzncWOn17Ui2VWegewVhQQAI=; b=CE3Xrn/+ussqSVL14Ij58m0NwX8+5jVzI1mn81mqD7IXabHC1XB1fudsJp31jBL/D/ fvp6g5+n4ZtmEUEbKGBe13I60Iq3Q4hsxYe9PZLP4/Q7neKB+FPHhrKF/88pKmvN2323 O3IwqEyU74YZtPUEYvWvb7ap3XAyzmZHldm7OyzkNEUSLQjdPwlRCCEGzHJHiaAeKsEj EYy2/jMtw7TxsG23Bn6pMKToAvcdqr6fOQz3JwQ10sJS7hJK8EKky01oLmDD+Vxh+WoR 4T4YfeMvW6BHHa/AC+JvAxyNCE8JJ1W1SHS2hyPEcd7E+Vdl9qwH1zFEVDA9zQuN8+Ue lIHw== X-Gm-Message-State: AOAM530wSR2STWgQHhJLUvgaDBQjf2XrA8HxEY3hZH/o83uEg2e/zP5X T1P6ou6zvyFfXlAZjrUdOtK5L7asuvyQw15D X-Google-Smtp-Source: ABdhPJxiz/yK/mWtOWOXNmT0lUD67I1dkQtTBgb4Q9Y4cmrtA6Kl6akYtyMC6LEbXAx67mmzam5gwyx0dWQN53nE X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:804a:: with SMTP id 68mr1172222qva.1.1606162536399; Mon, 23 Nov 2020 12:15:36 -0800 (PST) Date: Mon, 23 Nov 2020 21:14:48 +0100 In-Reply-To: Message-Id: <37497e940bfd4b32c0a93a702a9ae4cf061d5392.1606162397.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.454.gaff20da3a2-goog Subject: [PATCH mm v4 18/19] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov , Vincenzo Frascino Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 2610438120ce..83860aa4e89c 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 249ccba1ecf5..da79f340f3a6 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) -- 2.29.2.454.gaff20da3a2-goog