From: Andrey Konovalov <andreyknvl@google.com> To: Andrew Morton <akpm@linux-foundation.org> Cc: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, Vincenzo Frascino <vincenzo.frascino@arm.com>, Dmitry Vyukov <dvyukov@google.com>, Andrey Ryabinin <aryabinin@virtuozzo.com>, Alexander Potapenko <glider@google.com>, Marco Elver <elver@google.com>, Evgenii Stepanov <eugenis@google.com>, Branislav Rankov <Branislav.Rankov@arm.com>, Kevin Brodsky <kevin.brodsky@arm.com>, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov <andreyknvl@google.com>, Vincenzo Frascino <Vincenzo.Frascino@arm.com> Subject: [PATCH mm v3 18/19] kasan, mm: allow cache merging with no metadata Date: Fri, 13 Nov 2020 23:20:08 +0100 [thread overview] Message-ID: <6f0a1e72783ddac000ac08e7315b1d7c0ca4ec51.1605305978.git.andreyknvl@google.com> (raw) In-Reply-To: <cover.1605305978.git.andreyknvl@google.com> The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com> Signed-off-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com> Signed-off-by: Andrey Konovalov <andreyknvl@google.com> Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 16cf53eac29b..173a8e81d001 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index cf874243efab..a5a4dcb1254d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include <linux/seq_file.h> #include <linux/proc_fs.h> #include <linux/debugfs.h> +#include <linux/kasan.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/page.h> @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) -- 2.29.2.299.gdc1121823c-goog
WARNING: multiple messages have this Message-ID (diff)
From: Andrey Konovalov <andreyknvl@google.com> To: Andrew Morton <akpm@linux-foundation.org> Cc: linux-arm-kernel@lists.infradead.org, Marco Elver <elver@google.com>, Catalin Marinas <catalin.marinas@arm.com>, Kevin Brodsky <kevin.brodsky@arm.com>, Will Deacon <will.deacon@arm.com>, Branislav Rankov <Branislav.Rankov@arm.com>, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko <glider@google.com>, Evgenii Stepanov <eugenis@google.com>, Andrey Konovalov <andreyknvl@google.com>, Andrey Ryabinin <aryabinin@virtuozzo.com>, Vincenzo Frascino <Vincenzo.Frascino@arm.com>, Dmitry Vyukov <dvyukov@google.com> Subject: [PATCH mm v3 18/19] kasan, mm: allow cache merging with no metadata Date: Fri, 13 Nov 2020 23:20:08 +0100 [thread overview] Message-ID: <6f0a1e72783ddac000ac08e7315b1d7c0ca4ec51.1605305978.git.andreyknvl@google.com> (raw) In-Reply-To: <cover.1605305978.git.andreyknvl@google.com> The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com> Signed-off-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com> Signed-off-by: Andrey Konovalov <andreyknvl@google.com> Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 16cf53eac29b..173a8e81d001 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index cf874243efab..a5a4dcb1254d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include <linux/seq_file.h> #include <linux/proc_fs.h> #include <linux/debugfs.h> +#include <linux/kasan.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> #include <asm/page.h> @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) -- 2.29.2.299.gdc1121823c-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-11-13 22:21 UTC|newest] Thread overview: 120+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-13 22:19 [PATCH mm v3 00/19] kasan: boot parameters for hardware tag-based mode Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` [PATCH mm v3 01/19] kasan: simplify quarantine_put call site Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` [PATCH mm v3 02/19] kasan: rename get_alloc/free_info Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` [PATCH mm v3 03/19] kasan: introduce set_alloc_info Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` [PATCH mm v3 04/19] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-16 11:00 ` Dmitry Vyukov 2020-11-16 11:00 ` Dmitry Vyukov 2020-11-16 11:00 ` Dmitry Vyukov 2020-11-13 22:19 ` [PATCH mm v3 05/19] kasan: allow VMAP_STACK for HW_TAGS mode Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-16 11:01 ` Dmitry Vyukov 2020-11-16 11:01 ` Dmitry Vyukov 2020-11-16 11:01 ` Dmitry Vyukov 2020-11-13 22:19 ` [PATCH mm v3 06/19] kasan: remove __kasan_unpoison_stack Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` [PATCH mm v3 07/19] kasan: inline kasan_reset_tag for tag-based modes Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-17 10:56 ` Dmitry Vyukov 2020-11-17 10:56 ` Dmitry Vyukov 2020-11-17 10:56 ` Dmitry Vyukov 2020-11-13 22:19 ` [PATCH mm v3 08/19] kasan: inline random_tag for HW_TAGS Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-17 10:58 ` Dmitry Vyukov 2020-11-17 10:58 ` Dmitry Vyukov 2020-11-17 10:58 ` Dmitry Vyukov 2020-11-13 22:19 ` [PATCH mm v3 09/19] kasan: open-code kasan_unpoison_slab Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-13 22:19 ` Andrey Konovalov 2020-11-16 15:06 ` Marco Elver 2020-11-16 15:06 ` Marco Elver 2020-11-13 22:20 ` [PATCH mm v3 10/19] kasan: inline (un)poison_range and check_invalid_free Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:11 ` Marco Elver 2020-11-16 15:11 ` Marco Elver 2020-11-13 22:20 ` [PATCH mm v3 11/19] kasan: add and integrate kasan boot parameters Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:15 ` Marco Elver 2020-11-16 15:15 ` Marco Elver 2020-11-17 11:09 ` Dmitry Vyukov 2020-11-17 11:09 ` Dmitry Vyukov 2020-11-17 11:09 ` Dmitry Vyukov 2020-11-13 22:20 ` [PATCH mm v3 12/19] kasan, mm: check kasan_enabled in annotations Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:26 ` Marco Elver 2020-11-16 15:26 ` Marco Elver 2020-11-17 11:12 ` Dmitry Vyukov 2020-11-17 11:12 ` Dmitry Vyukov 2020-11-17 11:12 ` Dmitry Vyukov 2020-11-13 22:20 ` [PATCH mm v3 13/19] kasan, mm: rename kasan_poison_kfree Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:43 ` Marco Elver 2020-11-16 15:43 ` Marco Elver 2020-11-13 22:20 ` [PATCH mm v3 14/19] kasan: don't round_up too much Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` [PATCH mm v3 15/19] kasan: simplify assign_tag and set_tag calls Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` [PATCH mm v3 16/19] kasan: clarify comment in __kasan_kfree_large Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` [PATCH mm v3 17/19] kasan: clean up metadata allocation and usage Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:46 ` Marco Elver 2020-11-16 15:46 ` Marco Elver 2020-11-17 13:12 ` Dmitry Vyukov 2020-11-17 13:12 ` Dmitry Vyukov 2020-11-17 13:12 ` Dmitry Vyukov 2020-11-17 13:18 ` Marco Elver 2020-11-17 13:18 ` Marco Elver 2020-11-17 13:18 ` Marco Elver 2020-11-17 13:27 ` Dmitry Vyukov 2020-11-17 13:27 ` Dmitry Vyukov 2020-11-17 13:27 ` Dmitry Vyukov 2020-11-23 18:54 ` Andrey Konovalov 2020-11-23 18:54 ` Andrey Konovalov 2020-11-23 18:54 ` Andrey Konovalov 2020-11-23 19:16 ` Andrey Konovalov 2020-11-23 19:16 ` Andrey Konovalov 2020-11-23 19:16 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov [this message] 2020-11-13 22:20 ` [PATCH mm v3 18/19] kasan, mm: allow cache merging with no metadata Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:45 ` Marco Elver 2020-11-16 15:45 ` Marco Elver 2020-11-17 13:25 ` Dmitry Vyukov 2020-11-17 13:25 ` Dmitry Vyukov 2020-11-17 13:25 ` Dmitry Vyukov 2020-11-23 13:52 ` Andrey Konovalov 2020-11-23 13:52 ` Andrey Konovalov 2020-11-23 13:52 ` Andrey Konovalov 2020-11-13 22:20 ` [PATCH mm v3 19/19] kasan: update documentation Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-13 22:20 ` Andrey Konovalov 2020-11-16 15:47 ` Marco Elver 2020-11-16 15:47 ` Marco Elver 2020-11-17 13:28 ` Dmitry Vyukov 2020-11-17 13:28 ` Dmitry Vyukov 2020-11-17 13:28 ` Dmitry Vyukov 2020-11-16 14:48 ` [PATCH mm v3 00/19] kasan: boot parameters for hardware tag-based mode Vincenzo Frascino 2020-11-16 14:48 ` Vincenzo Frascino
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=6f0a1e72783ddac000ac08e7315b1d7c0ca4ec51.1605305978.git.andreyknvl@google.com \ --to=andreyknvl@google.com \ --cc=Branislav.Rankov@arm.com \ --cc=akpm@linux-foundation.org \ --cc=aryabinin@virtuozzo.com \ --cc=catalin.marinas@arm.com \ --cc=dvyukov@google.com \ --cc=elver@google.com \ --cc=eugenis@google.com \ --cc=glider@google.com \ --cc=kasan-dev@googlegroups.com \ --cc=kevin.brodsky@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=vincenzo.frascino@arm.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.