From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74D40C43457 for ; Wed, 14 Oct 2020 20:45:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DDB2F22248 for ; Wed, 14 Oct 2020 20:45:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="caicryyE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDB2F22248 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5169E6B0074; Wed, 14 Oct 2020 16:45:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EFAA94000F; Wed, 14 Oct 2020 16:45:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F0E194000E; Wed, 14 Oct 2020 16:45:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0100.hostedemail.com [216.40.44.100]) by kanga.kvack.org (Postfix) with ESMTP id 01CE96B0074 for ; Wed, 14 Oct 2020 16:45:04 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 807A0180AD806 for ; Wed, 14 Oct 2020 20:45:04 +0000 (UTC) X-FDA: 77371710528.22.egg27_1103afb2720e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 596CC18038E67 for ; Wed, 14 Oct 2020 20:45:04 +0000 (UTC) X-HE-Tag: egg27_1103afb2720e X-Filterd-Recvd-Size: 13300 Received: from mail-qv1-f73.google.com (mail-qv1-f73.google.com [209.85.219.73]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Wed, 14 Oct 2020 20:45:03 +0000 (UTC) Received: by mail-qv1-f73.google.com with SMTP id c3so317159qvj.4 for ; Wed, 14 Oct 2020 13:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=SUndFZwmd/UI7ffIXpzXKFFOW5Rrpznt5Bq+QYZ8r0w=; b=caicryyEbUpsIiZ0NMxXzTfuEB2I6l9NimsZ6Lk6Czndwv2e3G7VtJkCRKGYY4FWAT 6uHlkTBwMQGK03tgQxtNK33SmCFmXX/ZiiOfaElRTt2AvB1IuQsUzCGNPm/ESL0z3zsV 3pnR6sdai/eX52O2cMQXO10G9NgkjhnJZbKGbOf3Iu0pJh8LZmkFxOgoJ6A8yB6J+k26 IuIYtakyU4uB3YuLy1C+0IV4/lGcy/Cc/uWNB2t3/fkHb47ZN2IcL3QOcXmACEr/Lx+4 Ul81bm2qjhUIqn2E/9Qde6fHHlfgg+TUNsJKNzGZj4kkfh1nc4orCaR2vfpmo2V/JM68 fsxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SUndFZwmd/UI7ffIXpzXKFFOW5Rrpznt5Bq+QYZ8r0w=; b=nwFFYuR9f2AP2TcTve6K3llqbDGH8ecZvoiIBUFI/L2tcqd8XaTk5hS62i7I+/2Jy3 KHrTILGc06skPyNCWLZrqztUSROFZ9Ies1gyAxFNVGVkIXELKt34ViVLRykJ2pxNMTSy 5x5RRLGsOAR3YA+/LnFlfREz0g7cKZb7O264c91e+zFS3wxHrj2+Q+DkyucD763AWgD1 yXxkBSaWgBdaz1DwJkW4ZULUuM5lbQeMrtABouixZyiYhJqz7NfM4TnCLJPgeGhXx8hn 6MnbgEC9KMnjjnulhot6/ki+1Vhu67jCEO3osSsJ1ant+y8NndsyktRWu8zYFt1TmDyw rTWA== X-Gm-Message-State: AOAM5313UPGEoBq3x6MfZ4hiPuEOZoUjQGP5T/u+3bDP4B+RxhULai17 K9xumU0Z7afIvpnUiVOeA9wmzz7clfxDv0tL X-Google-Smtp-Source: ABdhPJwkH7NZaiymmpkrpu4Oo26FdAmZETt4ONiB1fR/jb04+GN/F+l6yYohbS6UY0kfXjoO7q/MICddWQPbHX7/ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:4f46:: with SMTP id eu6mr1409445qvb.9.1602708302995; Wed, 14 Oct 2020 13:45:02 -0700 (PDT) Date: Wed, 14 Oct 2020 22:44:36 +0200 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.28.0.1011.ga647a8990f-goog Subject: [PATCH RFC 8/8] kasan: add and integrate kasan_mode boot param From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver , Evgenii Stepanov Cc: Andrey Ryabinin , Elena Petrova , Branislav Rankov , Kevin Brodsky , Andrew Morton , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: TODO: no meaningful description here yet, please see the cover letter for this RFC series. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4 --- mm/kasan/common.c | 69 +++++++++++++++++++++++++--------------------- mm/kasan/generic.c | 4 +++ mm/kasan/hw_tags.c | 53 +++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 8 ++++++ mm/kasan/report.c | 10 +++++-- mm/kasan/sw_tags.c | 4 +++ 6 files changed, 115 insertions(+), 33 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a3e67d49b893..d642d5fce1e5 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -135,35 +135,37 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, unsigned int redzone_size; int redzone_adjust; - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC) && - (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta))) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - - redzone_size = optimal_redzone(cache->object_size); - redzone_adjust = redzone_size - (*size - cache->object_size); - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + redzone_size)); + if (static_branch_unlikely(&kasan_debug)) { + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + redzone_size)); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } } *flags |= SLAB_KASAN; @@ -180,6 +182,7 @@ size_t kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { + WARN_ON(!static_branch_unlikely(&kasan_debug)); return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; } @@ -187,6 +190,7 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + WARN_ON(!static_branch_unlikely(&kasan_debug)); return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; } @@ -266,8 +270,10 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, if (!(cache->flags & SLAB_KASAN)) return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + if (static_branch_unlikely(&kasan_debug)) { + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -305,6 +311,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE); if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + !static_branch_unlikely(&kasan_debug) || unlikely(!(cache->flags & SLAB_KASAN))) return false; @@ -351,7 +358,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) + if (static_branch_unlikely(&kasan_debug) && cache->flags & SLAB_KASAN) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index d259e4c3aefd..9d968eaedc98 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -33,6 +33,10 @@ #include "kasan.h" #include "../slab.h" +/* See the comments in hw_tags.c */ +DEFINE_STATIC_KEY_TRUE_RO(kasan_enabled); +DEFINE_STATIC_KEY_TRUE_RO(kasan_debug); + /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index b372421258c8..fc6ab1c8b155 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -8,6 +8,8 @@ #define pr_fmt(fmt) "kasan: " fmt +#include +#include #include #include #include @@ -17,8 +19,57 @@ #include "kasan.h" +enum kasan_mode { + KASAN_MODE_OFF, + KASAN_MODE_ON, + KASAN_MODE_DEBUG, +}; + +static enum kasan_mode kasan_mode __ro_after_init; + +/* Whether KASAN is enabled at all. */ +/* TODO: ideally no KASAN callbacks when this is disabled. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_enabled); + +/* Whether to collect debugging info, e.g. alloc/free stack traces. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_debug); + +/* Whether to use syncronous or asynchronous tag checking. */ +static bool kasan_sync __ro_after_init; + +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (strcmp(arg, "on") == 0) + kasan_mode = KASAN_MODE_ON; + else if (strcmp(arg, "debug") == 0) + kasan_mode = KASAN_MODE_DEBUG; + return 0; +} +early_param("kasan_mode", early_kasan_mode); + void __init kasan_init_tags(void) { + /* TODO: system_supports_tags() always returns 0 here, fix. */ + if (0 /*!system_supports_tags()*/) + return; + + switch (kasan_mode) { + case KASAN_MODE_OFF: + return; + case KASAN_MODE_ON: + static_branch_enable(&kasan_enabled); + break; + case KASAN_MODE_DEBUG: + static_branch_enable(&kasan_enabled); + static_branch_enable(&kasan_debug); + kasan_sync = true; + break; + } + + /* TODO: choose between sync and async based on kasan_sync. */ init_tags(KASAN_TAG_MAX); pr_info("KernelAddressSanitizer initialized\n"); @@ -60,6 +111,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; + WARN_ON(!static_branch_unlikely(&kasan_debug)); alloc_meta = kasan_get_alloc_meta(cache, object); kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } @@ -69,6 +121,7 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; + WARN_ON(!static_branch_unlikely(&kasan_debug)); alloc_meta = kasan_get_alloc_meta(cache, object); return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 47d6074c7958..3712e7a39717 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -279,6 +279,14 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #define get_mem_tag(addr) arch_get_mem_tag(addr) #define set_mem_tag_range(addr, size, tag) arch_set_mem_tag_range((addr), (size), (tag)) +#ifdef CONFIG_KASAN_HW_TAGS +DECLARE_STATIC_KEY_FALSE(kasan_enabled); +DECLARE_STATIC_KEY_FALSE(kasan_debug); +#else +DECLARE_STATIC_KEY_TRUE(kasan_enabled); +DECLARE_STATIC_KEY_TRUE(kasan_debug); +#endif + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/report.c b/mm/kasan/report.c index dee5350b459c..ae956a29ad4e 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -159,8 +159,8 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, (void *)(object_addr + cache->object_size)); } -static void describe_object(struct kmem_cache *cache, void *object, - const void *addr, u8 tag) +static void describe_object_stacks(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) { struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); @@ -188,7 +188,13 @@ static void describe_object(struct kmem_cache *cache, void *object, } #endif } +} +static void describe_object(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) +{ + if (static_branch_unlikely(&kasan_debug)) + describe_object_stacks(cache, object, addr, tag); describe_object_addr(cache, object, addr); } diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 099af6dc8f7e..50e797a16e17 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -33,6 +33,10 @@ #include "kasan.h" #include "../slab.h" +/* See the comments in hw_tags.c */ +DEFINE_STATIC_KEY_TRUE_RO(kasan_enabled); +DEFINE_STATIC_KEY_TRUE_RO(kasan_debug); + static DEFINE_PER_CPU(u32, prng_state); void __init kasan_init_tags(void) -- 2.28.0.1011.ga647a8990f-goog