mm-commits Archive on lore.kernel.org
 help / color / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, andreyknvl@google.com,
	aryabinin@virtuozzo.com, Branislav.Rankov@arm.com,
	catalin.marinas@arm.com, dvyukov@google.com, elver@google.com,
	eugenis@google.com, glider@google.com, gor@linux.ibm.com,
	kevin.brodsky@arm.com, linux-mm@kvack.org,
	mm-commits@vger.kernel.org, torvalds@linux-foundation.org,
	Vincenzo.Frascino@arm.com, will.deacon@arm.com
Subject: [patch 72/78] kasan, mm: allow cache merging with no metadata
Date: Fri, 18 Dec 2020 14:05:25 -0800
Message-ID: <20201218220525.X5G3jCEts%akpm@linux-foundation.org> (raw)
In-Reply-To: <20201218140046.497484741326828e5b5d46ec@linux-foundation.org>

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, mm: allow cache merging with no metadata

The reason cache merging is disabled with KASAN is because KASAN puts its
metadata right after the allocated object. When the merged caches have
slightly different sizes, the metadata ends up in different places, which
KASAN doesn't support.

It might be possible to adjust the metadata allocation algorithm and make
it friendly to the cache merging code. Instead this change takes a simpler
approach and allows merging caches when no metadata is present. Which is
the case for hardware tag-based KASAN with kasan.mode=prod.

Link: https://lkml.kernel.org/r/37497e940bfd4b32c0a93a702a9ae4cf061d5392.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba
Co-developed-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |   21 +++++++++++++++++++--
 mm/kasan/common.c     |   11 +++++++++++
 mm/slab_common.c      |    3 ++-
 3 files changed, 32 insertions(+), 3 deletions(-)

--- a/include/linux/kasan.h~kasan-mm-allow-cache-merging-with-no-metadata
+++ a/include/linux/kasan.h
@@ -82,17 +82,30 @@ struct kasan_cache {
 };
 
 #ifdef CONFIG_KASAN_HW_TAGS
+
 DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled);
+
 static __always_inline bool kasan_enabled(void)
 {
 	return static_branch_likely(&kasan_flag_enabled);
 }
-#else
+
+#else /* CONFIG_KASAN_HW_TAGS */
+
 static inline bool kasan_enabled(void)
 {
 	return true;
 }
-#endif
+
+#endif /* CONFIG_KASAN_HW_TAGS */
+
+slab_flags_t __kasan_never_merge(void);
+static __always_inline slab_flags_t kasan_never_merge(void)
+{
+	if (kasan_enabled())
+		return __kasan_never_merge();
+	return 0;
+}
 
 void __kasan_unpoison_range(const void *addr, size_t size);
 static __always_inline void kasan_unpoison_range(const void *addr, size_t size)
@@ -239,6 +252,10 @@ static inline bool kasan_enabled(void)
 {
 	return false;
 }
+static inline slab_flags_t kasan_never_merge(void)
+{
+	return 0;
+}
 static inline void kasan_unpoison_range(const void *address, size_t size) {}
 static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
 static inline void kasan_free_pages(struct page *page, unsigned int order) {}
--- a/mm/kasan/common.c~kasan-mm-allow-cache-merging-with-no-metadata
+++ a/mm/kasan/common.c
@@ -86,6 +86,17 @@ asmlinkage void kasan_unpoison_task_stac
 }
 #endif /* CONFIG_KASAN_STACK */
 
+/*
+ * Only allow cache merging when stack collection is disabled and no metadata
+ * is present.
+ */
+slab_flags_t __kasan_never_merge(void)
+{
+	if (kasan_stack_collection_enabled())
+		return SLAB_KASAN;
+	return 0;
+}
+
 void __kasan_alloc_pages(struct page *page, unsigned int order)
 {
 	u8 tag;
--- a/mm/slab_common.c~kasan-mm-allow-cache-merging-with-no-metadata
+++ a/mm/slab_common.c
@@ -18,6 +18,7 @@
 #include <linux/seq_file.h>
 #include <linux/proc_fs.h>
 #include <linux/debugfs.h>
+#include <linux/kasan.h>
 #include <asm/cacheflush.h>
 #include <asm/tlbflush.h>
 #include <asm/page.h>
@@ -53,7 +54,7 @@ static DECLARE_WORK(slab_caches_to_rcu_d
  */
 #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
 		SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \
-		SLAB_FAILSLAB | SLAB_KASAN)
+		SLAB_FAILSLAB | kasan_never_merge())
 
 #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
 			 SLAB_CACHE_DMA32 | SLAB_ACCOUNT)
_

  parent reply index

Thread overview: 89+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-18 22:00 incoming Andrew Morton
2020-12-18 22:01 ` [patch 01/78] mm/memcg: bail early from swap accounting if memcg disabled Andrew Morton
2020-12-18 22:01 ` [patch 02/78] mm/memcg: warning on !memcg after readahead page charged Andrew Morton
2020-12-18 22:01 ` [patch 03/78] mm/memcg: remove unused definitions Andrew Morton
2020-12-18 22:01 ` [patch 04/78] mm, kvm: account kvm_vcpu_mmap to kmemcg Andrew Morton
2020-12-18 22:01 ` [patch 05/78] mm/memcontrol:rewrite mem_cgroup_page_lruvec() Andrew Morton
2020-12-18 22:01 ` [patch 06/78] epoll: check for events when removing a timed out thread from the wait queue Andrew Morton
2020-12-18 22:01 ` [patch 07/78] epoll: simplify signal handling Andrew Morton
2020-12-18 22:01 ` [patch 08/78] epoll: pull fatal signal checks into ep_send_events() Andrew Morton
2020-12-18 22:01 ` [patch 09/78] epoll: move eavail next to the list_empty_careful check Andrew Morton
2020-12-18 22:01 ` [patch 10/78] epoll: simplify and optimize busy loop logic Andrew Morton
2020-12-18 22:02 ` [patch 11/78] epoll: pull all code between fetch_events and send_event into the loop Andrew Morton
2020-12-18 22:02 ` [patch 12/78] epoll: replace gotos with a proper loop Andrew Morton
2020-12-18 22:02 ` [patch 13/78] epoll: eliminate unnecessary lock for zero timeout Andrew Morton
2020-12-18 22:02 ` [patch 14/78] kasan: drop unnecessary GPL text from comment headers Andrew Morton
2020-12-18 22:02 ` [patch 15/78] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrew Morton
2020-12-18 22:02 ` [patch 16/78] kasan: group vmalloc code Andrew Morton
2020-12-18 22:02 ` [patch 17/78] kasan: shadow declarations only for software modes Andrew Morton
2020-12-18 22:02 ` [patch 18/78] kasan: rename (un)poison_shadow to (un)poison_range Andrew Morton
2020-12-18 22:02 ` [patch 19/78] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrew Morton
2020-12-18 22:02 ` [patch 20/78] kasan: only build init.c for software modes Andrew Morton
2020-12-18 22:02 ` [patch 21/78] kasan: split out shadow.c from common.c Andrew Morton
2020-12-19  0:28   ` Marco Elver
2020-12-19  1:13     ` Andrew Morton
2020-12-19 10:01       ` Marco Elver
2020-12-19 10:11       ` Marco Elver
2020-12-19 18:01       ` Andrey Konovalov
2020-12-19 19:17       ` Linus Torvalds
2020-12-19 19:26         ` Linus Torvalds
2020-12-21  9:46         ` Alexander Potapenko
2020-12-21 17:41           ` Linus Torvalds
2020-12-22 18:38             ` Andrew Morton
2020-12-18 22:02 ` [patch 22/78] kasan: define KASAN_MEMORY_PER_SHADOW_PAGE Andrew Morton
2020-12-18 22:02 ` [patch 23/78] kasan: rename report and tags files Andrew Morton
2020-12-18 22:02 ` [patch 24/78] kasan: don't duplicate config dependencies Andrew Morton
2020-12-18 22:02 ` [patch 25/78] kasan: hide invalid free check implementation Andrew Morton
2020-12-18 22:02 ` [patch 26/78] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrew Morton
2020-12-18 22:02 ` [patch 27/78] kasan, arm64: only init shadow for software modes Andrew Morton
2020-12-18 22:02 ` [patch 28/78] kasan, arm64: only use kasan_depth " Andrew Morton
2020-12-18 22:03 ` [patch 29/78] kasan, arm64: move initialization message Andrew Morton
2020-12-18 22:03 ` [patch 30/78] kasan, arm64: rename kasan_init_tags and mark as __init Andrew Morton
2020-12-18 22:03 ` [patch 31/78] kasan: rename addr_has_shadow to addr_has_metadata Andrew Morton
2020-12-18 22:03 ` [patch 32/78] kasan: rename print_shadow_for_address to print_memory_metadata Andrew Morton
2020-12-18 22:03 ` [patch 33/78] kasan: rename SHADOW layout macros to META Andrew Morton
2020-12-18 22:03 ` [patch 34/78] kasan: separate metadata_fetch_row for each mode Andrew Morton
2020-12-18 22:03 ` [patch 35/78] kasan: introduce CONFIG_KASAN_HW_TAGS Andrew Morton
2020-12-18 22:03 ` [patch 36/78] arm64: enable armv8.5-a asm-arch option Andrew Morton
2020-12-18 22:03 ` [patch 37/78] arm64: mte: add in-kernel MTE helpers Andrew Morton
2020-12-18 22:03 ` [patch 38/78] arm64: mte: reset the page tag in page->flags Andrew Morton
2020-12-18 22:03 ` [patch 39/78] arm64: mte: add in-kernel tag fault handler Andrew Morton
2020-12-18 22:03 ` [patch 40/78] arm64: kasan: allow enabling in-kernel MTE Andrew Morton
2020-12-18 22:03 ` [patch 41/78] arm64: mte: convert gcr_user into an exclude mask Andrew Morton
2020-12-18 22:03 ` [patch 42/78] arm64: mte: switch GCR_EL1 in kernel entry and exit Andrew Morton
2020-12-18 22:03 ` [patch 43/78] kasan, mm: untag page address in free_reserved_area Andrew Morton
2020-12-18 22:03 ` [patch 44/78] arm64: kasan: align allocations for HW_TAGS Andrew Morton
2020-12-18 22:03 ` [patch 45/78] arm64: kasan: add arch layer for memory tagging helpers Andrew Morton
2020-12-18 22:03 ` [patch 46/78] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrew Morton
2020-12-18 22:03 ` [patch 47/78] kasan, x86, s390: update undef CONFIG_KASAN Andrew Morton
2020-12-18 22:04 ` [patch 48/78] kasan, arm64: expand CONFIG_KASAN checks Andrew Morton
2020-12-18 22:04 ` [patch 49/78] kasan, arm64: implement HW_TAGS runtime Andrew Morton
2020-12-18 22:04 ` [patch 50/78] kasan, arm64: print report from tag fault handler Andrew Morton
2020-12-18 22:04 ` [patch 51/78] kasan, mm: reset tags when accessing metadata Andrew Morton
2020-12-18 22:04 ` [patch 52/78] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrew Morton
2020-12-18 22:04 ` [patch 53/78] kasan: add documentation for hardware tag-based mode Andrew Morton
2020-12-18 22:04 ` [patch 54/78] kselftest/arm64: check GCR_EL1 after context switch Andrew Morton
2020-12-18 22:04 ` [patch 55/78] kasan: simplify quarantine_put call site Andrew Morton
2020-12-18 22:04 ` [patch 56/78] kasan: rename get_alloc/free_info Andrew Morton
2020-12-18 22:04 ` [patch 57/78] kasan: introduce set_alloc_info Andrew Morton
2020-12-18 22:04 ` [patch 58/78] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK Andrew Morton
2020-12-18 22:04 ` [patch 59/78] kasan: allow VMAP_STACK for HW_TAGS mode Andrew Morton
2020-12-18 22:04 ` [patch 60/78] kasan: remove __kasan_unpoison_stack Andrew Morton
2020-12-18 22:04 ` [patch 61/78] kasan: inline kasan_reset_tag for tag-based modes Andrew Morton
2020-12-18 22:04 ` [patch 62/78] kasan: inline random_tag for HW_TAGS Andrew Morton
2020-12-18 22:04 ` [patch 63/78] kasan: open-code kasan_unpoison_slab Andrew Morton
2020-12-18 22:04 ` [patch 64/78] kasan: inline (un)poison_range and check_invalid_free Andrew Morton
2020-12-18 22:05 ` [patch 65/78] kasan: add and integrate kasan boot parameters Andrew Morton
2020-12-18 22:05 ` [patch 66/78] kasan, mm: check kasan_enabled in annotations Andrew Morton
2020-12-18 22:05 ` [patch 67/78] kasan, mm: rename kasan_poison_kfree Andrew Morton
2020-12-18 22:05 ` [patch 68/78] kasan: don't round_up too much Andrew Morton
2020-12-18 22:05 ` [patch 69/78] kasan: simplify assign_tag and set_tag calls Andrew Morton
2020-12-18 22:05 ` [patch 70/78] kasan: clarify comment in __kasan_kfree_large Andrew Morton
2020-12-18 22:05 ` [patch 71/78] kasan: sanitize objects when metadata doesn't fit Andrew Morton
2020-12-18 22:05 ` Andrew Morton [this message]
2020-12-18 22:05 ` [patch 73/78] kasan: update documentation Andrew Morton
2020-12-18 22:05 ` [patch 74/78] mm/Kconfig: fix spelling mistake "whats" -> "what's" Andrew Morton
2020-12-18 22:05 ` [patch 75/78] epoll: convert internal api to timespec64 Andrew Morton
2020-12-18 22:05 ` [patch 76/78] epoll: add syscall epoll_pwait2 Andrew Morton
2020-12-18 22:05 ` [patch 77/78] epoll: wire up " Andrew Morton
2020-12-18 22:05 ` [patch 78/78] selftests/filesystems: expand epoll with epoll_pwait2 Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201218220525.X5G3jCEts%akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=Branislav.Rankov@arm.com \
    --cc=Vincenzo.Frascino@arm.com \
    --cc=andreyknvl@google.com \
    --cc=aryabinin@virtuozzo.com \
    --cc=catalin.marinas@arm.com \
    --cc=dvyukov@google.com \
    --cc=elver@google.com \
    --cc=eugenis@google.com \
    --cc=glider@google.com \
    --cc=gor@linux.ibm.com \
    --cc=kevin.brodsky@arm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mm-commits@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

mm-commits Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/mm-commits/0 mm-commits/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 mm-commits mm-commits/ https://lore.kernel.org/mm-commits \
		mm-commits@vger.kernel.org
	public-inbox-index mm-commits

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.mm-commits


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git