All of lore.kernel.org
 help / color / mirror / Atom feed
* incoming
@ 2020-12-18 22:00 Andrew Morton
  2020-12-18 22:01 ` [patch 01/78] mm/memcg: bail early from swap accounting if memcg disabled Andrew Morton
                   ` (77 more replies)
  0 siblings, 78 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:00 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm


78 patches, based on a409ed156a90093a03fe6a93721ddf4c591eac87.

Subsystems affected by this patch series:

  mm/memcg
  epoll
  mm/kasan
  mm/cleanups
  epoll

Subsystem: mm/memcg

    Alex Shi <alex.shi@linux.alibaba.com>:
    Patch series "bail out early for memcg disable":
      mm/memcg: bail early from swap accounting if memcg disabled
      mm/memcg: warning on !memcg after readahead page charged

    Wei Yang <richard.weiyang@gmail.com>:
      mm/memcg: remove unused definitions

    Shakeel Butt <shakeelb@google.com>:
      mm, kvm: account kvm_vcpu_mmap to kmemcg

    Hui Su <sh_def@163.com>:
      mm/memcontrol:rewrite mem_cgroup_page_lruvec()

Subsystem: epoll

    Soheil Hassas Yeganeh <soheil@google.com>:
    Patch series "simplify ep_poll":
      epoll: check for events when removing a timed out thread from the wait queue
      epoll: simplify signal handling
      epoll: pull fatal signal checks into ep_send_events()
      epoll: move eavail next to the list_empty_careful check
      epoll: simplify and optimize busy loop logic
      epoll: pull all code between fetch_events and send_event into the loop
      epoll: replace gotos with a proper loop
      epoll: eliminate unnecessary lock for zero timeout

Subsystem: mm/kasan

    Andrey Konovalov <andreyknvl@google.com>:
    Patch series "kasan: add hardware tag-based mode for arm64", v11:
      kasan: drop unnecessary GPL text from comment headers
      kasan: KASAN_VMALLOC depends on KASAN_GENERIC
      kasan: group vmalloc code
      kasan: shadow declarations only for software modes
      kasan: rename (un)poison_shadow to (un)poison_range
      kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_*
      kasan: only build init.c for software modes
      kasan: split out shadow.c from common.c
      kasan: define KASAN_MEMORY_PER_SHADOW_PAGE
      kasan: rename report and tags files
      kasan: don't duplicate config dependencies
      kasan: hide invalid free check implementation
      kasan: decode stack frame only with KASAN_STACK_ENABLE
      kasan, arm64: only init shadow for software modes
      kasan, arm64: only use kasan_depth for software modes
      kasan, arm64: move initialization message
      kasan, arm64: rename kasan_init_tags and mark as __init
      kasan: rename addr_has_shadow to addr_has_metadata
      kasan: rename print_shadow_for_address to print_memory_metadata
      kasan: rename SHADOW layout macros to META
      kasan: separate metadata_fetch_row for each mode
      kasan: introduce CONFIG_KASAN_HW_TAGS

    Vincenzo Frascino <vincenzo.frascino@arm.com>:
      arm64: enable armv8.5-a asm-arch option
      arm64: mte: add in-kernel MTE helpers
      arm64: mte: reset the page tag in page->flags
      arm64: mte: add in-kernel tag fault handler
      arm64: kasan: allow enabling in-kernel MTE
      arm64: mte: convert gcr_user into an exclude mask
      arm64: mte: switch GCR_EL1 in kernel entry and exit
      kasan, mm: untag page address in free_reserved_area

    Andrey Konovalov <andreyknvl@google.com>:
      arm64: kasan: align allocations for HW_TAGS
      arm64: kasan: add arch layer for memory tagging helpers
      kasan: define KASAN_GRANULE_SIZE for HW_TAGS
      kasan, x86, s390: update undef CONFIG_KASAN
      kasan, arm64: expand CONFIG_KASAN checks
      kasan, arm64: implement HW_TAGS runtime
      kasan, arm64: print report from tag fault handler
      kasan, mm: reset tags when accessing metadata
      kasan, arm64: enable CONFIG_KASAN_HW_TAGS
      kasan: add documentation for hardware tag-based mode

    Vincenzo Frascino <vincenzo.frascino@arm.com>:
      kselftest/arm64: check GCR_EL1 after context switch

    Andrey Konovalov <andreyknvl@google.com>:
    Patch series "kasan: boot parameters for hardware tag-based mode", v4:
      kasan: simplify quarantine_put call site
      kasan: rename get_alloc/free_info
      kasan: introduce set_alloc_info
      kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK
      kasan: allow VMAP_STACK for HW_TAGS mode
      kasan: remove __kasan_unpoison_stack
      kasan: inline kasan_reset_tag for tag-based modes
      kasan: inline random_tag for HW_TAGS
      kasan: open-code kasan_unpoison_slab
      kasan: inline (un)poison_range and check_invalid_free
      kasan: add and integrate kasan boot parameters
      kasan, mm: check kasan_enabled in annotations
      kasan, mm: rename kasan_poison_kfree
      kasan: don't round_up too much
      kasan: simplify assign_tag and set_tag calls
      kasan: clarify comment in __kasan_kfree_large
      kasan: sanitize objects when metadata doesn't fit
      kasan, mm: allow cache merging with no metadata
      kasan: update documentation

Subsystem: mm/cleanups

    Colin Ian King <colin.king@canonical.com>:
      mm/Kconfig: fix spelling mistake "whats" -> "what's"

Subsystem: epoll

    Willem de Bruijn <willemb@google.com>:
    Patch series "add epoll_pwait2 syscall", v4:
      epoll: convert internal api to timespec64
      epoll: add syscall epoll_pwait2
      epoll: wire up syscall epoll_pwait2
      selftests/filesystems: expand epoll with epoll_pwait2

 Documentation/dev-tools/kasan.rst                             |  274 +-
 arch/Kconfig                                                  |    8 
 arch/alpha/kernel/syscalls/syscall.tbl                        |    1 
 arch/arm/tools/syscall.tbl                                    |    1 
 arch/arm64/Kconfig                                            |    9 
 arch/arm64/Makefile                                           |    7 
 arch/arm64/include/asm/assembler.h                            |    2 
 arch/arm64/include/asm/cache.h                                |    3 
 arch/arm64/include/asm/esr.h                                  |    1 
 arch/arm64/include/asm/kasan.h                                |   17 
 arch/arm64/include/asm/memory.h                               |   15 
 arch/arm64/include/asm/mte-def.h                              |   16 
 arch/arm64/include/asm/mte-kasan.h                            |   67 
 arch/arm64/include/asm/mte.h                                  |   22 
 arch/arm64/include/asm/processor.h                            |    2 
 arch/arm64/include/asm/string.h                               |    5 
 arch/arm64/include/asm/uaccess.h                              |   23 
 arch/arm64/include/asm/unistd.h                               |    2 
 arch/arm64/include/asm/unistd32.h                             |    2 
 arch/arm64/kernel/asm-offsets.c                               |    3 
 arch/arm64/kernel/cpufeature.c                                |    3 
 arch/arm64/kernel/entry.S                                     |   41 
 arch/arm64/kernel/head.S                                      |    2 
 arch/arm64/kernel/hibernate.c                                 |    5 
 arch/arm64/kernel/image-vars.h                                |    2 
 arch/arm64/kernel/kaslr.c                                     |    3 
 arch/arm64/kernel/module.c                                    |    6 
 arch/arm64/kernel/mte.c                                       |  124 +
 arch/arm64/kernel/setup.c                                     |    2 
 arch/arm64/kernel/sleep.S                                     |    2 
 arch/arm64/kernel/smp.c                                       |    2 
 arch/arm64/lib/mte.S                                          |   16 
 arch/arm64/mm/copypage.c                                      |    9 
 arch/arm64/mm/fault.c                                         |   59 
 arch/arm64/mm/kasan_init.c                                    |   41 
 arch/arm64/mm/mteswap.c                                       |    9 
 arch/arm64/mm/proc.S                                          |   23 
 arch/arm64/mm/ptdump.c                                        |    6 
 arch/ia64/kernel/syscalls/syscall.tbl                         |    1 
 arch/m68k/kernel/syscalls/syscall.tbl                         |    1 
 arch/microblaze/kernel/syscalls/syscall.tbl                   |    1 
 arch/mips/kernel/syscalls/syscall_n32.tbl                     |    1 
 arch/mips/kernel/syscalls/syscall_n64.tbl                     |    1 
 arch/mips/kernel/syscalls/syscall_o32.tbl                     |    1 
 arch/parisc/kernel/syscalls/syscall.tbl                       |    1 
 arch/powerpc/kernel/syscalls/syscall.tbl                      |    1 
 arch/s390/boot/string.c                                       |    1 
 arch/s390/kernel/syscalls/syscall.tbl                         |    1 
 arch/sh/kernel/syscalls/syscall.tbl                           |    1 
 arch/sparc/kernel/syscalls/syscall.tbl                        |    1 
 arch/x86/boot/compressed/misc.h                               |    1 
 arch/x86/entry/syscalls/syscall_32.tbl                        |    1 
 arch/x86/entry/syscalls/syscall_64.tbl                        |    1 
 arch/x86/kernel/acpi/wakeup_64.S                              |    2 
 arch/x86/kvm/x86.c                                            |    2 
 arch/xtensa/kernel/syscalls/syscall.tbl                       |    1 
 fs/eventpoll.c                                                |  359 ++-
 include/linux/compat.h                                        |    6 
 include/linux/kasan-checks.h                                  |    2 
 include/linux/kasan.h                                         |  423 ++--
 include/linux/memcontrol.h                                    |  137 -
 include/linux/mm.h                                            |   24 
 include/linux/mmdebug.h                                       |   13 
 include/linux/moduleloader.h                                  |    3 
 include/linux/page-flags-layout.h                             |    2 
 include/linux/sched.h                                         |    2 
 include/linux/string.h                                        |    2 
 include/linux/syscalls.h                                      |    5 
 include/uapi/asm-generic/unistd.h                             |    4 
 init/init_task.c                                              |    2 
 kernel/fork.c                                                 |    4 
 kernel/sys_ni.c                                               |    2 
 lib/Kconfig.kasan                                             |   71 
 lib/test_kasan.c                                              |    2 
 lib/test_kasan_module.c                                       |    2 
 mm/Kconfig                                                    |    2 
 mm/kasan/Makefile                                             |   33 
 mm/kasan/common.c                                             | 1006 ++--------
 mm/kasan/generic.c                                            |   72 
 mm/kasan/generic_report.c                                     |   13 
 mm/kasan/hw_tags.c                                            |  294 ++
 mm/kasan/init.c                                               |   25 
 mm/kasan/kasan.h                                              |  204 +-
 mm/kasan/quarantine.c                                         |   35 
 mm/kasan/report.c                                             |  363 +--
 mm/kasan/report_generic.c                                     |  169 +
 mm/kasan/report_hw_tags.c                                     |   44 
 mm/kasan/report_sw_tags.c                                     |   22 
 mm/kasan/shadow.c                                             |  541 +++++
 mm/kasan/sw_tags.c                                            |   34 
 mm/kasan/tags.c                                               |    7 
 mm/kasan/tags_report.c                                        |    7 
 mm/memcontrol.c                                               |   53 
 mm/mempool.c                                                  |    4 
 mm/page_alloc.c                                               |    9 
 mm/page_poison.c                                              |    2 
 mm/ptdump.c                                                   |   13 
 mm/slab_common.c                                              |    5 
 mm/slub.c                                                     |   29 
 scripts/Makefile.lib                                          |    2 
 tools/testing/selftests/arm64/mte/Makefile                    |    2 
 tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c     |  155 +
 tools/testing/selftests/filesystems/epoll/epoll_wakeup_test.c |   72 
 virt/kvm/coalesced_mmio.c                                     |    2 
 virt/kvm/kvm_main.c                                           |    2 
 105 files changed, 3268 insertions(+), 1873 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 01/78] mm/memcg: bail early from swap accounting if memcg disabled
  2020-12-18 22:00 incoming Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 02/78] mm/memcg: warning on !memcg after readahead page charged Andrew Morton
                   ` (76 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, alex.shi, guro, hannes, hughd, linux-mm, mhocko,
	mm-commits, torvalds, vdavydov.dev

From: Alex Shi <alex.shi@linux.alibaba.com>
Subject: mm/memcg: bail early from swap accounting if memcg disabled

Patch series "bail out early for memcg disable".

These 2 patches are indepenedent from per memcg lru lock, and may
encounter unexpected warning, so let's move out them from per memcg
lru locking patchset.


This patch (of 2):

We could bail out early when memcg wasn't enabled.

Link: https://lkml.kernel.org/r/1604283436-18880-1-git-send-email-alex.shi@linux.alibaba.com
Link: https://lkml.kernel.org/r/1604283436-18880-2-git-send-email-alex.shi@linux.alibaba.com
Signed-off-by: Alex Shi <alex.shi@linux.alibaba.com>
Reviewed-by: Roman Gushchin <guro@fb.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Hugh Dickins <hughd@google.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/memcontrol.c |    6 ++++++
 1 file changed, 6 insertions(+)

--- a/mm/memcontrol.c~mm-memcg-bail-early-from-swap-accounting-if-memcg-disabled
+++ a/mm/memcontrol.c
@@ -7178,6 +7178,9 @@ void mem_cgroup_swapout(struct page *pag
 	VM_BUG_ON_PAGE(PageLRU(page), page);
 	VM_BUG_ON_PAGE(page_count(page), page);
 
+	if (mem_cgroup_disabled())
+		return;
+
 	if (cgroup_subsys_on_dfl(memory_cgrp_subsys))
 		return;
 
@@ -7242,6 +7245,9 @@ int mem_cgroup_try_charge_swap(struct pa
 	struct mem_cgroup *memcg;
 	unsigned short oldid;
 
+	if (mem_cgroup_disabled())
+		return 0;
+
 	if (!cgroup_subsys_on_dfl(memory_cgrp_subsys))
 		return 0;
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 02/78] mm/memcg: warning on !memcg after readahead page charged
  2020-12-18 22:00 incoming Andrew Morton
  2020-12-18 22:01 ` [patch 01/78] mm/memcg: bail early from swap accounting if memcg disabled Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 03/78] mm/memcg: remove unused definitions Andrew Morton
                   ` (75 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, alex.shi, hannes, hughd, linux-mm, mhocko, mm-commits,
	torvalds, vdavydov.dev

From: Alex Shi <alex.shi@linux.alibaba.com>
Subject: mm/memcg: warning on !memcg after readahead page charged

Add VM_WARN_ON_ONCE_PAGE() macro.

Since readahead page is charged on memcg too, in theory we don't have to
check this exception now.  Before safely remove them all, add a warning
for the unexpected !memcg.

Link: https://lkml.kernel.org/r/1604283436-18880-3-git-send-email-alex.shi@linux.alibaba.com
Signed-off-by: Alex Shi <alex.shi@linux.alibaba.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Hugh Dickins <hughd@google.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/mmdebug.h |   13 +++++++++++++
 mm/memcontrol.c         |   10 ++++------
 2 files changed, 17 insertions(+), 6 deletions(-)

--- a/include/linux/mmdebug.h~mm-memcg-warning-on-memcg-after-readahead-page-charged
+++ a/include/linux/mmdebug.h
@@ -37,6 +37,18 @@ void dump_mm(const struct mm_struct *mm)
 			BUG();						\
 		}							\
 	} while (0)
+#define VM_WARN_ON_ONCE_PAGE(cond, page)	({			\
+	static bool __section(".data.once") __warned;			\
+	int __ret_warn_once = !!(cond);					\
+									\
+	if (unlikely(__ret_warn_once && !__warned)) {			\
+		dump_page(page, "VM_WARN_ON_ONCE_PAGE(" __stringify(cond)")");\
+		__warned = true;					\
+		WARN_ON(1);						\
+	}								\
+	unlikely(__ret_warn_once);					\
+})
+
 #define VM_WARN_ON(cond) (void)WARN_ON(cond)
 #define VM_WARN_ON_ONCE(cond) (void)WARN_ON_ONCE(cond)
 #define VM_WARN_ONCE(cond, format...) (void)WARN_ONCE(cond, format)
@@ -48,6 +60,7 @@ void dump_mm(const struct mm_struct *mm)
 #define VM_BUG_ON_MM(cond, mm) VM_BUG_ON(cond)
 #define VM_WARN_ON(cond) BUILD_BUG_ON_INVALID(cond)
 #define VM_WARN_ON_ONCE(cond) BUILD_BUG_ON_INVALID(cond)
+#define VM_WARN_ON_ONCE_PAGE(cond, page)  BUILD_BUG_ON_INVALID(cond)
 #define VM_WARN_ONCE(cond, format...) BUILD_BUG_ON_INVALID(cond)
 #define VM_WARN(cond, format...) BUILD_BUG_ON_INVALID(cond)
 #endif
--- a/mm/memcontrol.c~mm-memcg-warning-on-memcg-after-readahead-page-charged
+++ a/mm/memcontrol.c
@@ -1362,10 +1362,7 @@ struct lruvec *mem_cgroup_page_lruvec(st
 	}
 
 	memcg = page_memcg(page);
-	/*
-	 * Swapcache readahead pages are added to the LRU - and
-	 * possibly migrated - before they are charged.
-	 */
+	VM_WARN_ON_ONCE_PAGE(!memcg, page);
 	if (!memcg)
 		memcg = root_mem_cgroup;
 
@@ -6987,6 +6984,7 @@ void mem_cgroup_migrate(struct page *old
 		return;
 
 	memcg = page_memcg(oldpage);
+	VM_WARN_ON_ONCE_PAGE(!memcg, oldpage);
 	if (!memcg)
 		return;
 
@@ -7186,7 +7184,7 @@ void mem_cgroup_swapout(struct page *pag
 
 	memcg = page_memcg(page);
 
-	/* Readahead page, never charged */
+	VM_WARN_ON_ONCE_PAGE(!memcg, page);
 	if (!memcg)
 		return;
 
@@ -7253,7 +7251,7 @@ int mem_cgroup_try_charge_swap(struct pa
 
 	memcg = page_memcg(page);
 
-	/* Readahead page, never charged */
+	VM_WARN_ON_ONCE_PAGE(!memcg, page);
 	if (!memcg)
 		return 0;
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 03/78] mm/memcg: remove unused definitions
  2020-12-18 22:00 incoming Andrew Morton
  2020-12-18 22:01 ` [patch 01/78] mm/memcg: bail early from swap accounting if memcg disabled Andrew Morton
  2020-12-18 22:01 ` [patch 02/78] mm/memcg: warning on !memcg after readahead page charged Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 04/78] mm, kvm: account kvm_vcpu_mmap to kmemcg Andrew Morton
                   ` (74 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, guro, hannes, linux-mm, mhocko, mm-commits,
	richard.weiyang, shakeelb, torvalds

From: Wei Yang <richard.weiyang@gmail.com>
Subject: mm/memcg: remove unused definitions

Some definitions are left unused, just clean them.

Link: https://lkml.kernel.org/r/20201108003834.12669-1-richard.weiyang@gmail.com
Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Reviewed-by: Roman Gushchin <guro@fb.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/memcontrol.h |  118 -----------------------------------
 1 file changed, 118 deletions(-)

--- a/include/linux/memcontrol.h~mm-memcg-remove-unused-definitions
+++ a/include/linux/memcontrol.h
@@ -913,41 +913,6 @@ static inline void mod_memcg_state(struc
 	local_irq_restore(flags);
 }
 
-/**
- * mod_memcg_page_state - update page state statistics
- * @page: the page
- * @idx: page state item to account
- * @val: number of pages (positive or negative)
- *
- * The @page must be locked or the caller must use lock_page_memcg()
- * to prevent double accounting when the page is concurrently being
- * moved to another memcg:
- *
- *   lock_page(page) or lock_page_memcg(page)
- *   if (TestClearPageState(page))
- *     mod_memcg_page_state(page, state, -1);
- *   unlock_page(page) or unlock_page_memcg(page)
- *
- * Kernel pages are an exception to this, since they'll never move.
- */
-static inline void __mod_memcg_page_state(struct page *page,
-					  int idx, int val)
-{
-	struct mem_cgroup *memcg = page_memcg(page);
-
-	if (memcg)
-		__mod_memcg_state(memcg, idx, val);
-}
-
-static inline void mod_memcg_page_state(struct page *page,
-					int idx, int val)
-{
-	struct mem_cgroup *memcg = page_memcg(page);
-
-	if (memcg)
-		mod_memcg_state(memcg, idx, val);
-}
-
 static inline unsigned long lruvec_page_state(struct lruvec *lruvec,
 					      enum node_stat_item idx)
 {
@@ -1395,18 +1360,6 @@ static inline void mod_memcg_state(struc
 {
 }
 
-static inline void __mod_memcg_page_state(struct page *page,
-					  int idx,
-					  int nr)
-{
-}
-
-static inline void mod_memcg_page_state(struct page *page,
-					int idx,
-					int nr)
-{
-}
-
 static inline unsigned long lruvec_page_state(struct lruvec *lruvec,
 					      enum node_stat_item idx)
 {
@@ -1479,34 +1432,6 @@ static inline void lruvec_memcg_debug(st
 }
 #endif /* CONFIG_MEMCG */
 
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void __inc_memcg_state(struct mem_cgroup *memcg,
-				     int idx)
-{
-	__mod_memcg_state(memcg, idx, 1);
-}
-
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void __dec_memcg_state(struct mem_cgroup *memcg,
-				     int idx)
-{
-	__mod_memcg_state(memcg, idx, -1);
-}
-
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void __inc_memcg_page_state(struct page *page,
-					  int idx)
-{
-	__mod_memcg_page_state(page, idx, 1);
-}
-
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void __dec_memcg_page_state(struct page *page,
-					  int idx)
-{
-	__mod_memcg_page_state(page, idx, -1);
-}
-
 static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
 {
 	__mod_lruvec_kmem_state(p, idx, 1);
@@ -1517,34 +1442,6 @@ static inline void __dec_lruvec_kmem_sta
 	__mod_lruvec_kmem_state(p, idx, -1);
 }
 
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void inc_memcg_state(struct mem_cgroup *memcg,
-				   int idx)
-{
-	mod_memcg_state(memcg, idx, 1);
-}
-
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void dec_memcg_state(struct mem_cgroup *memcg,
-				   int idx)
-{
-	mod_memcg_state(memcg, idx, -1);
-}
-
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void inc_memcg_page_state(struct page *page,
-					int idx)
-{
-	mod_memcg_page_state(page, idx, 1);
-}
-
-/* idx can be of type enum memcg_stat_item or node_stat_item */
-static inline void dec_memcg_page_state(struct page *page,
-					int idx)
-{
-	mod_memcg_page_state(page, idx, -1);
-}
-
 static inline struct lruvec *parent_lruvec(struct lruvec *lruvec)
 {
 	struct mem_cgroup *memcg;
@@ -1733,21 +1630,6 @@ static inline void memcg_kmem_uncharge_p
 		__memcg_kmem_uncharge_page(page, order);
 }
 
-static inline int memcg_kmem_charge(struct mem_cgroup *memcg, gfp_t gfp,
-				    unsigned int nr_pages)
-{
-	if (memcg_kmem_enabled())
-		return __memcg_kmem_charge(memcg, gfp, nr_pages);
-	return 0;
-}
-
-static inline void memcg_kmem_uncharge(struct mem_cgroup *memcg,
-				       unsigned int nr_pages)
-{
-	if (memcg_kmem_enabled())
-		__memcg_kmem_uncharge(memcg, nr_pages);
-}
-
 /*
  * A helper for accessing memcg's kmem_id, used for getting
  * corresponding LRU lists.
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 04/78] mm, kvm: account kvm_vcpu_mmap to kmemcg
  2020-12-18 22:00 incoming Andrew Morton
                   ` (2 preceding siblings ...)
  2020-12-18 22:01 ` [patch 03/78] mm/memcg: remove unused definitions Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 05/78] mm/memcontrol:rewrite mem_cgroup_page_lruvec() Andrew Morton
                   ` (73 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, guro, hannes, linux-mm, mhocko, mm-commits, pbonzini,
	shakeelb, torvalds

From: Shakeel Butt <shakeelb@google.com>
Subject: mm, kvm: account kvm_vcpu_mmap to kmemcg

A VCPU of a VM can allocate couple of pages which can be mmap'ed by the
user space application. At the moment this memory is not charged to the
memcg of the VMM. On a large machine running large number of VMs or
small number of VMs having large number of VCPUs, this unaccounted
memory can be very significant. So, charge this memory to the memcg of
the VMM. Please note that lifetime of these allocations corresponds to
the lifetime of the VMM.

Link: https://lkml.kernel.org/r/20201106202923.2087414-1-shakeelb@google.com
Signed-off-by: Shakeel Butt <shakeelb@google.com>
Acked-by: Roman Gushchin <guro@fb.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/x86/kvm/x86.c        |    2 +-
 virt/kvm/coalesced_mmio.c |    2 +-
 virt/kvm/kvm_main.c       |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

--- a/arch/x86/kvm/x86.c~mm-kvm-account-kvm_vcpu_mmap-to-kmemcg
+++ a/arch/x86/kvm/x86.c
@@ -9869,7 +9869,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu
 
 	r = -ENOMEM;
 
-	page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+	page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
 	if (!page)
 		goto fail_free_lapic;
 	vcpu->arch.pio_data = page_address(page);
--- a/virt/kvm/coalesced_mmio.c~mm-kvm-account-kvm_vcpu_mmap-to-kmemcg
+++ a/virt/kvm/coalesced_mmio.c
@@ -111,7 +111,7 @@ int kvm_coalesced_mmio_init(struct kvm *
 {
 	struct page *page;
 
-	page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+	page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
 	if (!page)
 		return -ENOMEM;
 
--- a/virt/kvm/kvm_main.c~mm-kvm-account-kvm_vcpu_mmap-to-kmemcg
+++ a/virt/kvm/kvm_main.c
@@ -3116,7 +3116,7 @@ static int kvm_vm_ioctl_create_vcpu(stru
 	}
 
 	BUILD_BUG_ON(sizeof(struct kvm_run) > PAGE_SIZE);
-	page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+	page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
 	if (!page) {
 		r = -ENOMEM;
 		goto vcpu_free;
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 05/78] mm/memcontrol:rewrite mem_cgroup_page_lruvec()
  2020-12-18 22:00 incoming Andrew Morton
                   ` (3 preceding siblings ...)
  2020-12-18 22:01 ` [patch 04/78] mm, kvm: account kvm_vcpu_mmap to kmemcg Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 06/78] epoll: check for events when removing a timed out thread from the wait queue Andrew Morton
                   ` (72 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, alex.shi, chris, guro, hannes, laoar.shao, linux-mm,
	lstoakes, mhocko, mm-commits, sh_def, shakeelb, torvalds,
	vdavydov.dev

From: Hui Su <sh_def@163.com>
Subject: mm/memcontrol:rewrite mem_cgroup_page_lruvec()

mem_cgroup_page_lruvec() in memcontrol.c and mem_cgroup_lruvec() in
memcontrol.h is very similar except for the param(page and memcg) which
also can be convert to each other.

So rewrite mem_cgroup_page_lruvec() with mem_cgroup_lruvec().

[alex.shi@linux.alibaba.com: add missed warning in mem_cgroup_lruvec]
  Link: https://lkml.kernel.org/r/94f17bb7-ec61-5b72-3555-fabeb5a4d73b@linux.alibaba.com
[lstoakes@gmail.com: warn on missing memcg on mem_cgroup_page_lruvec()]
  Link: https://lkml.kernel.org/r/20201125112202.387009-1-lstoakes@gmail.com
Link: https://lkml.kernel.org/r/20201108143731.GA74138@rlk
Signed-off-by: Hui Su <sh_def@163.com>
Signed-off-by: Alex Shi <alex.shi@linux.alibaba.com>
Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Acked-by: Roman Gushchin <guro@fb.com>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Yafang Shao <laoar.shao@gmail.com>
Cc: Chris Down <chris@chrisdown.name>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/memcontrol.h |   19 ++++++++++++++++-
 mm/memcontrol.c            |   37 -----------------------------------
 2 files changed, 17 insertions(+), 39 deletions(-)

--- a/include/linux/memcontrol.h~mm-memcontrol-rewrite-mem_cgroup_page_lruvec
+++ a/include/linux/memcontrol.h
@@ -620,9 +620,10 @@ mem_cgroup_nodeinfo(struct mem_cgroup *m
 /**
  * mem_cgroup_lruvec - get the lru list vector for a memcg & node
  * @memcg: memcg of the wanted lruvec
+ * @pgdat: pglist_data
  *
  * Returns the lru list vector holding pages for a given @memcg &
- * @node combination. This can be the node lruvec, if the memory
+ * @pgdat combination. This can be the node lruvec, if the memory
  * controller is disabled.
  */
 static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
@@ -652,7 +653,21 @@ out:
 	return lruvec;
 }
 
-struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_data *);
+/**
+ * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page
+ * @page: the page
+ * @pgdat: pgdat of the page
+ *
+ * This function relies on page->mem_cgroup being stable.
+ */
+static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page,
+						struct pglist_data *pgdat)
+{
+	struct mem_cgroup *memcg = page_memcg(page);
+
+	VM_WARN_ON_ONCE_PAGE(!memcg, page);
+	return mem_cgroup_lruvec(memcg, pgdat);
+}
 
 static inline bool lruvec_holds_page_lru_lock(struct page *page,
 					      struct lruvec *lruvec)
--- a/mm/memcontrol.c~mm-memcontrol-rewrite-mem_cgroup_page_lruvec
+++ a/mm/memcontrol.c
@@ -1343,43 +1343,6 @@ void lruvec_memcg_debug(struct lruvec *l
 #endif
 
 /**
- * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page
- * @page: the page
- * @pgdat: pgdat of the page
- *
- * This function relies on page's memcg being stable - see the
- * access rules in commit_charge().
- */
-struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct pglist_data *pgdat)
-{
-	struct mem_cgroup_per_node *mz;
-	struct mem_cgroup *memcg;
-	struct lruvec *lruvec;
-
-	if (mem_cgroup_disabled()) {
-		lruvec = &pgdat->__lruvec;
-		goto out;
-	}
-
-	memcg = page_memcg(page);
-	VM_WARN_ON_ONCE_PAGE(!memcg, page);
-	if (!memcg)
-		memcg = root_mem_cgroup;
-
-	mz = mem_cgroup_page_nodeinfo(memcg, page);
-	lruvec = &mz->lruvec;
-out:
-	/*
-	 * Since a node can be onlined after the mem_cgroup was created,
-	 * we have to be prepared to initialize lruvec->zone here;
-	 * and if offlined then reonlined, we need to reinitialize it.
-	 */
-	if (unlikely(lruvec->pgdat != pgdat))
-		lruvec->pgdat = pgdat;
-	return lruvec;
-}
-
-/**
  * lock_page_lruvec - lock and return lruvec for a given page.
  * @page: the page
  *
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 06/78] epoll: check for events when removing a timed out thread from the wait queue
  2020-12-18 22:00 incoming Andrew Morton
                   ` (4 preceding siblings ...)
  2020-12-18 22:01 ` [patch 05/78] mm/memcontrol:rewrite mem_cgroup_page_lruvec() Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 07/78] epoll: simplify signal handling Andrew Morton
                   ` (71 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, dbueso, edumazet, guantaol, khazhy, linux-mm, mm-commits,
	soheil, torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: check for events when removing a timed out thread from the wait queue

Patch series "simplify ep_poll".

This patch series is a followup based on the suggestions and feedback by
Linus:
https://lkml.kernel.org/r/CAHk-=wizk=OxUyQPbO8MS41w2Pag1kniUV5WdD5qWL-gq1kjDA@mail.gmail.com

The first patch in the series is a fix for the epoll race in presence of
timeouts, so that it can be cleanly backported to all affected stable
kernels.

The rest of the patch series simplify the ep_poll() implementation.  Some
of these simplifications result in minor performance enhancements as well.
We have kept these changes under self tests and internal benchmarks for a
few days, and there are minor (1-2%) performance enhancements as a result.


This patch (of 8):

After abc610e01c66 ("fs/epoll: avoid barrier after an epoll_wait(2)
timeout"), we break out of the ep_poll loop upon timeout, without checking
whether there is any new events available.  Prior to that patch-series we
always called ep_events_available() after exiting the loop.

This can cause races and missed wakeups.  For example, consider the
following scenario reported by Guantao Liu:

Suppose we have an eventfd added using EPOLLET to an epollfd.

Thread 1: Sleeps for just below 5ms and then writes to an eventfd.
Thread 2: Calls epoll_wait with a timeout of 5 ms. If it sees an
          event of the eventfd, it will write back on that fd.
Thread 3: Calls epoll_wait with a negative timeout.

Prior to abc610e01c66, it is guaranteed that Thread 3 will wake up either
by Thread 1 or Thread 2.  After abc610e01c66, Thread 3 can be blocked
indefinitely if Thread 2 sees a timeout right before the write to the
eventfd by Thread 1.  Thread 2 will be woken up from
schedule_hrtimeout_range and, with evail 0, it will not call
ep_send_events().

To fix this issue:
1) Simplify the timed_out case as suggested by Linus.
2) while holding the lock, recheck whether the thread was woken up
   after its time out has reached.

Note that (2) is different from Linus' original suggestion: It do not set
"eavail = ep_events_available(ep)" to avoid unnecessary contention (when
there are too many timed-out threads and a small number of events), as
well as races mentioned in the discussion thread.

This is the first patch in the series so that the backport to stable
releases is straightforward.

Link: https://lkml.kernel.org/r/20201106231635.3528496-1-soheil.kdev@gmail.com
Link: https://lkml.kernel.org/r/CAHk-=wizk=OxUyQPbO8MS41w2Pag1kniUV5WdD5qWL-gq1kjDA@mail.gmail.com
Link: https://lkml.kernel.org/r/20201106231635.3528496-2-soheil.kdev@gmail.com
Fixes: abc610e01c66 ("fs/epoll: avoid barrier after an epoll_wait(2) timeout")
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Tested-by: Guantao Liu <guantaol@google.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reported-by: Guantao Liu <guantaol@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Reviewed-by: Davidlohr Bueso <dbueso@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

--- a/fs/eventpoll.c~epoll-check-for-events-when-removing-a-timed-out-thread-from-the-wait-queue
+++ a/fs/eventpoll.c
@@ -1817,23 +1817,30 @@ fetch_events:
 		}
 		write_unlock_irq(&ep->lock);
 
-		if (eavail || res)
-			break;
-
-		if (!schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS)) {
-			timed_out = 1;
-			break;
-		}
-
-		/* We were woken up, thus go and try to harvest some events */
+		if (!eavail && !res)
+			timed_out = !schedule_hrtimeout_range(to, slack,
+							      HRTIMER_MODE_ABS);
+
+		/*
+		 * We were woken up, thus go and try to harvest some events.
+		 * If timed out and still on the wait queue, recheck eavail
+		 * carefully under lock, below.
+		 */
 		eavail = 1;
-
 	} while (0);
 
 	__set_current_state(TASK_RUNNING);
 
 	if (!list_empty_careful(&wait.entry)) {
 		write_lock_irq(&ep->lock);
+		/*
+		 * If the thread timed out and is not on the wait queue, it
+		 * means that the thread was woken up after its timeout expired
+		 * before it could reacquire the lock. Thus, when wait.entry is
+		 * empty, it needs to harvest events.
+		 */
+		if (timed_out)
+			eavail = list_empty(&wait.entry);
 		__remove_wait_queue(&ep->wq, &wait);
 		write_unlock_irq(&ep->lock);
 	}
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 07/78] epoll: simplify signal handling
  2020-12-18 22:00 incoming Andrew Morton
                   ` (5 preceding siblings ...)
  2020-12-18 22:01 ` [patch 06/78] epoll: check for events when removing a timed out thread from the wait queue Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 08/78] epoll: pull fatal signal checks into ep_send_events() Andrew Morton
                   ` (70 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: simplify signal handling

Check signals before locking ep->lock, and immediately return -EINTR if
there is any signal pending.

This saves a few loads, stores, and branches from the hot path and
simplifies the loop structure for follow up patches.

Link: https://lkml.kernel.org/r/20201106231635.3528496-3-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

--- a/fs/eventpoll.c~epoll-simplify-signal-handling
+++ a/fs/eventpoll.c
@@ -1733,7 +1733,7 @@ static inline struct timespec64 ep_set_m
 static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
 		   int maxevents, long timeout)
 {
-	int res = 0, eavail, timed_out = 0;
+	int res, eavail, timed_out = 0;
 	u64 slack = 0;
 	wait_queue_entry_t wait;
 	ktime_t expires, *to = NULL;
@@ -1780,6 +1780,9 @@ fetch_events:
 	ep_reset_busy_poll_napi_id(ep);
 
 	do {
+		if (signal_pending(current))
+			return -EINTR;
+
 		/*
 		 * Internally init_wait() uses autoremove_wake_function(),
 		 * thus wait entry is removed from the wait queue on each
@@ -1809,15 +1812,12 @@ fetch_events:
 		 * important.
 		 */
 		eavail = ep_events_available(ep);
-		if (!eavail) {
-			if (signal_pending(current))
-				res = -EINTR;
-			else
-				__add_wait_queue_exclusive(&ep->wq, &wait);
-		}
+		if (!eavail)
+			__add_wait_queue_exclusive(&ep->wq, &wait);
+
 		write_unlock_irq(&ep->lock);
 
-		if (!eavail && !res)
+		if (!eavail)
 			timed_out = !schedule_hrtimeout_range(to, slack,
 							      HRTIMER_MODE_ABS);
 
@@ -1853,14 +1853,14 @@ send_events:
 		 * finding more events available and fetching
 		 * repeatedly.
 		 */
-		res = -EINTR;
+		return -EINTR;
 	}
 	/*
 	 * Try to transfer events to user space. In case we get 0 events and
 	 * there's still timeout left over, we go trying again in search of
 	 * more luck.
 	 */
-	if (!res && eavail &&
+	if (eavail &&
 	    !(res = ep_send_events(ep, events, maxevents)) && !timed_out)
 		goto fetch_events;
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 08/78] epoll: pull fatal signal checks into ep_send_events()
  2020-12-18 22:00 incoming Andrew Morton
                   ` (6 preceding siblings ...)
  2020-12-18 22:01 ` [patch 07/78] epoll: simplify signal handling Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 09/78] epoll: move eavail next to the list_empty_careful check Andrew Morton
                   ` (69 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: pull fatal signal checks into ep_send_events()

To simplify the code, pull in checking the fatal signals into
ep_send_events().  ep_send_events() is called only from ep_poll().

Note that, previously, we were always checking fatal events, but it is
checked only if eavail is true.  This should be fine because the goal of
that check is to quickly return from epoll_wait() when there is a pending
fatal signal.

Link: https://lkml.kernel.org/r/20201106231635.3528496-4-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Suggested-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

--- a/fs/eventpoll.c~epoll-pull-fatal-signal-checks-into-ep_send_events
+++ a/fs/eventpoll.c
@@ -1625,6 +1625,14 @@ static int ep_send_events(struct eventpo
 	poll_table pt;
 	int res = 0;
 
+	/*
+	 * Always short-circuit for fatal signals to allow threads to make a
+	 * timely exit without the chance of finding more events available and
+	 * fetching repeatedly.
+	 */
+	if (fatal_signal_pending(current))
+		return -EINTR;
+
 	init_poll_funcptr(&pt, NULL);
 
 	mutex_lock(&ep->mtx);
@@ -1846,15 +1854,6 @@ fetch_events:
 	}
 
 send_events:
-	if (fatal_signal_pending(current)) {
-		/*
-		 * Always short-circuit for fatal signals to allow
-		 * threads to make a timely exit without the chance of
-		 * finding more events available and fetching
-		 * repeatedly.
-		 */
-		return -EINTR;
-	}
 	/*
 	 * Try to transfer events to user space. In case we get 0 events and
 	 * there's still timeout left over, we go trying again in search of
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 09/78] epoll: move eavail next to the list_empty_careful check
  2020-12-18 22:00 incoming Andrew Morton
                   ` (7 preceding siblings ...)
  2020-12-18 22:01 ` [patch 08/78] epoll: pull fatal signal checks into ep_send_events() Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:01 ` [patch 10/78] epoll: simplify and optimize busy loop logic Andrew Morton
                   ` (68 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: move eavail next to the list_empty_careful check

This is a no-op change and simply to make the code more coherent.

Link: https://lkml.kernel.org/r/20201106231635.3528496-5-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/fs/eventpoll.c~epoll-move-eavail-next-to-the-list_empty_careful-check
+++ a/fs/eventpoll.c
@@ -1828,6 +1828,7 @@ fetch_events:
 		if (!eavail)
 			timed_out = !schedule_hrtimeout_range(to, slack,
 							      HRTIMER_MODE_ABS);
+		__set_current_state(TASK_RUNNING);
 
 		/*
 		 * We were woken up, thus go and try to harvest some events.
@@ -1837,8 +1838,6 @@ fetch_events:
 		eavail = 1;
 	} while (0);
 
-	__set_current_state(TASK_RUNNING);
-
 	if (!list_empty_careful(&wait.entry)) {
 		write_lock_irq(&ep->lock);
 		/*
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 10/78] epoll: simplify and optimize busy loop logic
  2020-12-18 22:00 incoming Andrew Morton
                   ` (8 preceding siblings ...)
  2020-12-18 22:01 ` [patch 09/78] epoll: move eavail next to the list_empty_careful check Andrew Morton
@ 2020-12-18 22:01 ` Andrew Morton
  2020-12-18 22:02 ` [patch 11/78] epoll: pull all code between fetch_events and send_event into the loop Andrew Morton
                   ` (67 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:01 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: simplify and optimize busy loop logic

ep_events_available() is called multiple times around the busy loop logic,
even though the logic is generally not used.  ep_reset_busy_poll_napi_id()
is similarly always called, even when busy loop is not used.

Eliminate ep_reset_busy_poll_napi_id() and inline it inside
ep_busy_loop().  Make ep_busy_loop() return whether there are any events
available after the busy loop.  This will eliminate unnecessary loads and
branches, and simplifies the loop.

Link: https://lkml.kernel.org/r/20201106231635.3528496-6-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   40 +++++++++++++++++-----------------------
 1 file changed, 17 insertions(+), 23 deletions(-)

--- a/fs/eventpoll.c~epoll-simplify-and-optimize-busy-loop-logic
+++ a/fs/eventpoll.c
@@ -389,19 +389,24 @@ static bool ep_busy_loop_end(void *p, un
  *
  * we must do our busy polling with irqs enabled
  */
-static void ep_busy_loop(struct eventpoll *ep, int nonblock)
+static bool ep_busy_loop(struct eventpoll *ep, int nonblock)
 {
 	unsigned int napi_id = READ_ONCE(ep->napi_id);
 
-	if ((napi_id >= MIN_NAPI_ID) && net_busy_loop_on())
+	if ((napi_id >= MIN_NAPI_ID) && net_busy_loop_on()) {
 		napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false,
 			       BUSY_POLL_BUDGET);
-}
-
-static inline void ep_reset_busy_poll_napi_id(struct eventpoll *ep)
-{
-	if (ep->napi_id)
+		if (ep_events_available(ep))
+			return true;
+		/*
+		 * Busy poll timed out.  Drop NAPI ID for now, we can add
+		 * it back in when we have moved a socket with a valid NAPI
+		 * ID onto the ready list.
+		 */
 		ep->napi_id = 0;
+		return false;
+	}
+	return false;
 }
 
 /*
@@ -441,12 +446,9 @@ static inline void ep_set_busy_poll_napi
 
 #else
 
-static inline void ep_busy_loop(struct eventpoll *ep, int nonblock)
-{
-}
-
-static inline void ep_reset_busy_poll_napi_id(struct eventpoll *ep)
+static inline bool ep_busy_loop(struct eventpoll *ep, int nonblock)
 {
+	return false;
 }
 
 static inline void ep_set_busy_poll_napi_id(struct epitem *epi)
@@ -1772,21 +1774,13 @@ static int ep_poll(struct eventpoll *ep,
 	}
 
 fetch_events:
-
-	if (!ep_events_available(ep))
-		ep_busy_loop(ep, timed_out);
-
 	eavail = ep_events_available(ep);
+	if (!eavail)
+		eavail = ep_busy_loop(ep, timed_out);
+
 	if (eavail)
 		goto send_events;
 
-	/*
-	 * Busy poll timed out.  Drop NAPI ID for now, we can add
-	 * it back in when we have moved a socket with a valid NAPI
-	 * ID onto the ready list.
-	 */
-	ep_reset_busy_poll_napi_id(ep);
-
 	do {
 		if (signal_pending(current))
 			return -EINTR;
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 11/78] epoll: pull all code between fetch_events and send_event into the loop
  2020-12-18 22:00 incoming Andrew Morton
                   ` (9 preceding siblings ...)
  2020-12-18 22:01 ` [patch 10/78] epoll: simplify and optimize busy loop logic Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 12/78] epoll: replace gotos with a proper loop Andrew Morton
                   ` (66 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: pull all code between fetch_events and send_event into the loop

This is a no-op change which simplifies the follow up patches.

Link: https://lkml.kernel.org/r/20201106231635.3528496-7-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   41 +++++++++++++++++++++--------------------
 1 file changed, 21 insertions(+), 20 deletions(-)

--- a/fs/eventpoll.c~epoll-pull-all-code-between-fetch_events-and-send_event-into-the-loop
+++ a/fs/eventpoll.c
@@ -1774,14 +1774,14 @@ static int ep_poll(struct eventpoll *ep,
 	}
 
 fetch_events:
-	eavail = ep_events_available(ep);
-	if (!eavail)
-		eavail = ep_busy_loop(ep, timed_out);
+	do {
+		eavail = ep_events_available(ep);
+		if (!eavail)
+			eavail = ep_busy_loop(ep, timed_out);
 
-	if (eavail)
-		goto send_events;
+		if (eavail)
+			goto send_events;
 
-	do {
 		if (signal_pending(current))
 			return -EINTR;
 
@@ -1830,21 +1830,22 @@ fetch_events:
 		 * carefully under lock, below.
 		 */
 		eavail = 1;
-	} while (0);
 
-	if (!list_empty_careful(&wait.entry)) {
-		write_lock_irq(&ep->lock);
-		/*
-		 * If the thread timed out and is not on the wait queue, it
-		 * means that the thread was woken up after its timeout expired
-		 * before it could reacquire the lock. Thus, when wait.entry is
-		 * empty, it needs to harvest events.
-		 */
-		if (timed_out)
-			eavail = list_empty(&wait.entry);
-		__remove_wait_queue(&ep->wq, &wait);
-		write_unlock_irq(&ep->lock);
-	}
+		if (!list_empty_careful(&wait.entry)) {
+			write_lock_irq(&ep->lock);
+			/*
+			 * If the thread timed out and is not on the wait queue,
+			 * it means that the thread was woken up after its
+			 * timeout expired before it could reacquire the lock.
+			 * Thus, when wait.entry is empty, it needs to harvest
+			 * events.
+			 */
+			if (timed_out)
+				eavail = list_empty(&wait.entry);
+			__remove_wait_queue(&ep->wq, &wait);
+			write_unlock_irq(&ep->lock);
+		}
+	} while (0);
 
 send_events:
 	/*
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 12/78] epoll: replace gotos with a proper loop
  2020-12-18 22:00 incoming Andrew Morton
                   ` (10 preceding siblings ...)
  2020-12-18 22:02 ` [patch 11/78] epoll: pull all code between fetch_events and send_event into the loop Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 13/78] epoll: eliminate unnecessary lock for zero timeout Andrew Morton
                   ` (65 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: replace gotos with a proper loop

The existing loop is pointless, and the labels make it really hard to
follow the structure.

Replace that control structure with a simple loop that returns when there
are new events, there is a signal, or the thread has timed out.

Link: https://lkml.kernel.org/r/20201106231635.3528496-8-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   42 +++++++++++++++++++++---------------------
 1 file changed, 21 insertions(+), 21 deletions(-)

--- a/fs/eventpoll.c~epoll-replace-gotos-with-a-proper-loop
+++ a/fs/eventpoll.c
@@ -1743,7 +1743,7 @@ static inline struct timespec64 ep_set_m
 static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
 		   int maxevents, long timeout)
 {
-	int res, eavail, timed_out = 0;
+	int res, eavail = 0, timed_out = 0;
 	u64 slack = 0;
 	wait_queue_entry_t wait;
 	ktime_t expires, *to = NULL;
@@ -1769,18 +1769,30 @@ static int ep_poll(struct eventpoll *ep,
 		write_lock_irq(&ep->lock);
 		eavail = ep_events_available(ep);
 		write_unlock_irq(&ep->lock);
-
-		goto send_events;
 	}
 
-fetch_events:
-	do {
+	while (1) {
+		if (eavail) {
+			/*
+			 * Try to transfer events to user space. In case we get
+			 * 0 events and there's still timeout left over, we go
+			 * trying again in search of more luck.
+			 */
+			res = ep_send_events(ep, events, maxevents);
+			if (res)
+				return res;
+		}
+
+		if (timed_out)
+			return 0;
+
 		eavail = ep_events_available(ep);
-		if (!eavail)
-			eavail = ep_busy_loop(ep, timed_out);
+		if (eavail)
+			continue;
 
+		eavail = ep_busy_loop(ep, timed_out);
 		if (eavail)
-			goto send_events;
+			continue;
 
 		if (signal_pending(current))
 			return -EINTR;
@@ -1845,19 +1857,7 @@ fetch_events:
 			__remove_wait_queue(&ep->wq, &wait);
 			write_unlock_irq(&ep->lock);
 		}
-	} while (0);
-
-send_events:
-	/*
-	 * Try to transfer events to user space. In case we get 0 events and
-	 * there's still timeout left over, we go trying again in search of
-	 * more luck.
-	 */
-	if (eavail &&
-	    !(res = ep_send_events(ep, events, maxevents)) && !timed_out)
-		goto fetch_events;
-
-	return res;
+	}
 }
 
 /**
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 13/78] epoll: eliminate unnecessary lock for zero timeout
  2020-12-18 22:00 incoming Andrew Morton
                   ` (11 preceding siblings ...)
  2020-12-18 22:02 ` [patch 12/78] epoll: replace gotos with a proper loop Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 14/78] kasan: drop unnecessary GPL text from comment headers Andrew Morton
                   ` (64 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, edumazet, guantaol, khazhy, linux-mm, mm-commits, soheil,
	torvalds, willemb

From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: eliminate unnecessary lock for zero timeout

We call ep_events_available() under lock when timeout is 0, and then call
it without locks in the loop for the other cases.

Instead, call ep_events_available() without lock for all cases.  For
non-zero timeouts, we will recheck after adding the thread to the wait
queue.  For zero timeout cases, by definition, user is opportunistically
polling and will have to call epoll_wait again in the future.

Note that this lock was kept in c5a282e9635e9 because the whole loop was
historically under lock.

This patch results in a 1% CPU/RPC reduction in RPC benchmarks.

Link: https://lkml.kernel.org/r/20201106231635.3528496-9-soheil.kdev@gmail.com
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Suggested-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Guantao Liu <guantaol@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

--- a/fs/eventpoll.c~epoll-eliminate-unnecessary-lock-for-zero-timeout
+++ a/fs/eventpoll.c
@@ -1743,7 +1743,7 @@ static inline struct timespec64 ep_set_m
 static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
 		   int maxevents, long timeout)
 {
-	int res, eavail = 0, timed_out = 0;
+	int res, eavail, timed_out = 0;
 	u64 slack = 0;
 	wait_queue_entry_t wait;
 	ktime_t expires, *to = NULL;
@@ -1759,18 +1759,21 @@ static int ep_poll(struct eventpoll *ep,
 	} else if (timeout == 0) {
 		/*
 		 * Avoid the unnecessary trip to the wait queue loop, if the
-		 * caller specified a non blocking operation. We still need
-		 * lock because we could race and not see an epi being added
-		 * to the ready list while in irq callback. Thus incorrectly
-		 * returning 0 back to userspace.
+		 * caller specified a non blocking operation.
 		 */
 		timed_out = 1;
-
-		write_lock_irq(&ep->lock);
-		eavail = ep_events_available(ep);
-		write_unlock_irq(&ep->lock);
 	}
 
+	/*
+	 * This call is racy: We may or may not see events that are being added
+	 * to the ready list under the lock (e.g., in IRQ callbacks). For, cases
+	 * with a non-zero timeout, this thread will check the ready list under
+	 * lock and will added to the wait queue.  For, cases with a zero
+	 * timeout, the user by definition should not care and will have to
+	 * recheck again.
+	 */
+	eavail = ep_events_available(ep);
+
 	while (1) {
 		if (eavail) {
 			/*
@@ -1786,10 +1789,6 @@ static int ep_poll(struct eventpoll *ep,
 		if (timed_out)
 			return 0;
 
-		eavail = ep_events_available(ep);
-		if (eavail)
-			continue;
-
 		eavail = ep_busy_loop(ep, timed_out);
 		if (eavail)
 			continue;
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 14/78] kasan: drop unnecessary GPL text from comment headers
  2020-12-18 22:00 incoming Andrew Morton
                   ` (12 preceding siblings ...)
  2020-12-18 22:02 ` [patch 13/78] epoll: eliminate unnecessary lock for zero timeout Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 15/78] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrew Morton
                   ` (63 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: drop unnecessary GPL text from comment headers

Patch series "kasan: add hardware tag-based mode for arm64", v11.

This patchset adds a new hardware tag-based mode to KASAN [1].  The new
mode is similar to the existing software tag-based KASAN, but relies on
arm64 Memory Tagging Extension (MTE) [2] to perform memory and pointer
tagging (instead of shadow memory and compiler instrumentation).

This patchset is co-developed and tested by
Vincenzo Frascino <vincenzo.frascino@arm.com>.

This patchset is available here:

https://github.com/xairy/linux/tree/up-kasan-mte-v11

For testing in QEMU hardware tag-based KASAN requires:

1. QEMU built from master [4] (use "-machine virt,mte=on -cpu max" arguments
   to run).
2. GCC version 10.

[1] https://www.kernel.org/doc/html/latest/dev-tools/kasan.html
[2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety
[3] git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/mte
[4] https://github.com/qemu/qemu

====== Overview

The underlying ideas of the approach used by hardware tag-based KASAN are:

1. By relying on the Top Byte Ignore (TBI) arm64 CPU feature, pointer tags
   are stored in the top byte of each kernel pointer.

2. With the Memory Tagging Extension (MTE) arm64 CPU feature, memory tags
   for kernel memory allocations are stored in a dedicated memory not
   accessible via normal instuctions.

3. On each memory allocation, a random tag is generated, embedded it into
   the returned pointer, and the corresponding memory is tagged with the
   same tag value.

4. With MTE the CPU performs a check on each memory access to make sure
   that the pointer tag matches the memory tag.

5. On a tag mismatch the CPU generates a tag fault, and a KASAN report is
   printed.

Same as other KASAN modes, hardware tag-based KASAN is intended as a
debugging feature at this point.

====== Rationale

There are two main reasons for this new hardware tag-based mode:

1. Previously implemented software tag-based KASAN is being successfully
   used on dogfood testing devices due to its low memory overhead (as
   initially planned). The new hardware mode keeps the same low memory
   overhead, and is expected to have significantly lower performance
   impact, due to the tag checks being performed by the hardware.
   Therefore the new mode can be used as a better alternative in dogfood
   testing for hardware that supports MTE.

2. The new mode lays the groundwork for the planned in-kernel MTE-based
   memory corruption mitigation to be used in production.

====== Technical details

Considering the implementation perspective, hardware tag-based KASAN is
almost identical to the software mode.  The key difference is using MTE
for assigning and checking tags.

Compared to the software mode, the hardware mode uses 4 bits per tag, as
dictated by MTE.  Pointer tags are stored in bits [56:60), the top 4 bits
have the normal value 0xF.  Having less distict tags increases the
probablity of false negatives (from ~1/256 to ~1/16) in certain cases.

Only synchronous exceptions are set up and used by hardware tag-based KASAN.

====== Benchmarks

Note: all measurements have been performed with software emulation of Memory
Tagging Extension, performance numbers for hardware tag-based KASAN on the
actual hardware are expected to be better.

Boot time [1]:
* 2.8 sec for clean kernel
* 5.7 sec for hardware tag-based KASAN
* 11.8 sec for software tag-based KASAN
* 11.6 sec for generic KASAN

Slab memory usage after boot [2]:
* 7.0 kb for clean kernel
* 9.7 kb for hardware tag-based KASAN
* 9.7 kb for software tag-based KASAN
* 41.3 kb for generic KASAN

Measurements have been performed with:
* defconfig-based configs
* Manually built QEMU master
* QEMU arguments: -machine virt,mte=on -cpu max
* CONFIG_KASAN_STACK_ENABLE disabled
* CONFIG_KASAN_INLINE enabled
* clang-10 as the compiler and gcc-10 as the assembler
    
[1] Time before the ext4 driver is initialized.
[2] Measured as `cat /proc/meminfo | grep Slab`.

====== Notes

The cover letter for software tag-based KASAN patchset can be found here:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0116523cfffa62aeb5aa3b85ce7419f3dae0c1b8

===== Tags

Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>


This patch (of 41):

Don't mention "GNU General Public License version 2" text explicitly, as
it's already covered by the SPDX-License-Identifier.

Link: https://lkml.kernel.org/r/cover.1606161801.git.andreyknvl@google.com
Link: https://lkml.kernel.org/r/6ea9f5f4aa9dbbffa0d0c0a780b37699a4531034.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c         |    5 -----
 mm/kasan/generic.c        |    5 -----
 mm/kasan/generic_report.c |    5 -----
 mm/kasan/init.c           |    5 -----
 mm/kasan/quarantine.c     |   10 ----------
 mm/kasan/report.c         |    5 -----
 mm/kasan/tags.c           |    5 -----
 mm/kasan/tags_report.c    |    5 -----
 8 files changed, 45 deletions(-)

--- a/mm/kasan/common.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/common.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/export.h>
--- a/mm/kasan/generic.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/generic.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
--- a/mm/kasan/generic_report.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/generic_report.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/bitops.h>
--- a/mm/kasan/init.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/init.c
@@ -4,11 +4,6 @@
  *
  * Copyright (c) 2015 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/memblock.h>
--- a/mm/kasan/quarantine.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/quarantine.c
@@ -6,16 +6,6 @@
  * Copyright (C) 2016 Google, Inc.
  *
  * Based on code by Dmitry Chernenkov.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License for more details.
- *
  */
 
 #include <linux/gfp.h>
--- a/mm/kasan/report.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/report.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/bitops.h>
--- a/mm/kasan/tags.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/tags.c
@@ -4,11 +4,6 @@
  *
  * Copyright (c) 2018 Google, Inc.
  * Author: Andrey Konovalov <andreyknvl@google.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
--- a/mm/kasan/tags_report.c~kasan-drop-unnecessary-gpl-text-from-comment-headers
+++ a/mm/kasan/tags_report.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/bitops.h>
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 15/78] kasan: KASAN_VMALLOC depends on KASAN_GENERIC
  2020-12-18 22:00 incoming Andrew Morton
                   ` (13 preceding siblings ...)
  2020-12-18 22:02 ` [patch 14/78] kasan: drop unnecessary GPL text from comment headers Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 16/78] kasan: group vmalloc code Andrew Morton
                   ` (62 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: KASAN_VMALLOC depends on KASAN_GENERIC

Currently only generic KASAN mode supports vmalloc, reflect that in the
config.

Link: https://lkml.kernel.org/r/0c493d3a065ad95b04313d00244e884a7e2498ff.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 lib/Kconfig.kasan |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/lib/Kconfig.kasan~kasan-kasan_vmalloc-depends-on-kasan_generic
+++ a/lib/Kconfig.kasan
@@ -146,7 +146,7 @@ config KASAN_SW_TAGS_IDENTIFY
 
 config KASAN_VMALLOC
 	bool "Back mappings in vmalloc space with real shadow memory"
-	depends on HAVE_ARCH_KASAN_VMALLOC
+	depends on KASAN_GENERIC && HAVE_ARCH_KASAN_VMALLOC
 	help
 	  By default, the shadow region for vmalloc space is the read-only
 	  zero page. This means that KASAN cannot detect errors involving
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 16/78] kasan: group vmalloc code
  2020-12-18 22:00 incoming Andrew Morton
                   ` (14 preceding siblings ...)
  2020-12-18 22:02 ` [patch 15/78] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 17/78] kasan: shadow declarations only for software modes Andrew Morton
                   ` (61 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: group vmalloc code

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Group all vmalloc-related function declarations in include/linux/kasan.h,
and their implementations in mm/kasan/common.c.

No functional changes.

Link: https://lkml.kernel.org/r/80a6fdd29b039962843bd6cf22ce2643a7c8904e.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |   41 +++++++++++----------
 mm/kasan/common.c     |   78 ++++++++++++++++++++--------------------
 2 files changed, 63 insertions(+), 56 deletions(-)

--- a/include/linux/kasan.h~kasan-group-vmalloc-code
+++ a/include/linux/kasan.h
@@ -75,19 +75,6 @@ struct kasan_cache {
 	int free_meta_offset;
 };
 
-/*
- * These functions provide a special case to support backing module
- * allocations with real shadow memory. With KASAN vmalloc, the special
- * case is unnecessary, as the work is handled in the generic case.
- */
-#ifndef CONFIG_KASAN_VMALLOC
-int kasan_module_alloc(void *addr, size_t size);
-void kasan_free_shadow(const struct vm_struct *vm);
-#else
-static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
-static inline void kasan_free_shadow(const struct vm_struct *vm) {}
-#endif
-
 int kasan_add_zero_shadow(void *start, unsigned long size);
 void kasan_remove_zero_shadow(void *start, unsigned long size);
 
@@ -156,9 +143,6 @@ static inline bool kasan_slab_free(struc
 	return false;
 }
 
-static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
-static inline void kasan_free_shadow(const struct vm_struct *vm) {}
-
 static inline int kasan_add_zero_shadow(void *start, unsigned long size)
 {
 	return 0;
@@ -211,13 +195,16 @@ static inline void *kasan_reset_tag(cons
 #endif /* CONFIG_KASAN_SW_TAGS */
 
 #ifdef CONFIG_KASAN_VMALLOC
+
 int kasan_populate_vmalloc(unsigned long addr, unsigned long size);
 void kasan_poison_vmalloc(const void *start, unsigned long size);
 void kasan_unpoison_vmalloc(const void *start, unsigned long size);
 void kasan_release_vmalloc(unsigned long start, unsigned long end,
 			   unsigned long free_region_start,
 			   unsigned long free_region_end);
-#else
+
+#else /* CONFIG_KASAN_VMALLOC */
+
 static inline int kasan_populate_vmalloc(unsigned long start,
 					unsigned long size)
 {
@@ -232,7 +219,25 @@ static inline void kasan_release_vmalloc
 					 unsigned long end,
 					 unsigned long free_region_start,
 					 unsigned long free_region_end) {}
-#endif
+
+#endif /* CONFIG_KASAN_VMALLOC */
+
+#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
+
+/*
+ * These functions provide a special case to support backing module
+ * allocations with real shadow memory. With KASAN vmalloc, the special
+ * case is unnecessary, as the work is handled in the generic case.
+ */
+int kasan_module_alloc(void *addr, size_t size);
+void kasan_free_shadow(const struct vm_struct *vm);
+
+#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
+
+static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
+static inline void kasan_free_shadow(const struct vm_struct *vm) {}
+
+#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
 
 #ifdef CONFIG_KASAN_INLINE
 void kasan_non_canonical_hook(unsigned long addr);
--- a/mm/kasan/common.c~kasan-group-vmalloc-code
+++ a/mm/kasan/common.c
@@ -536,44 +536,6 @@ void kasan_kfree_large(void *ptr, unsign
 	/* The object will be poisoned by page_alloc. */
 }
 
-#ifndef CONFIG_KASAN_VMALLOC
-int kasan_module_alloc(void *addr, size_t size)
-{
-	void *ret;
-	size_t scaled_size;
-	size_t shadow_size;
-	unsigned long shadow_start;
-
-	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
-	scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
-	shadow_size = round_up(scaled_size, PAGE_SIZE);
-
-	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
-		return -EINVAL;
-
-	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
-			shadow_start + shadow_size,
-			GFP_KERNEL,
-			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
-			__builtin_return_address(0));
-
-	if (ret) {
-		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
-		find_vm_area(addr)->flags |= VM_KASAN;
-		kmemleak_ignore(ret);
-		return 0;
-	}
-
-	return -ENOMEM;
-}
-
-void kasan_free_shadow(const struct vm_struct *vm)
-{
-	if (vm->flags & VM_KASAN)
-		vfree(kasan_mem_to_shadow(vm->addr));
-}
-#endif
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 static bool shadow_mapped(unsigned long addr)
 {
@@ -685,6 +647,7 @@ core_initcall(kasan_memhotplug_init);
 #endif
 
 #ifdef CONFIG_KASAN_VMALLOC
+
 static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
 				      void *unused)
 {
@@ -923,4 +886,43 @@ void kasan_release_vmalloc(unsigned long
 				       (unsigned long)shadow_end);
 	}
 }
+
+#else /* CONFIG_KASAN_VMALLOC */
+
+int kasan_module_alloc(void *addr, size_t size)
+{
+	void *ret;
+	size_t scaled_size;
+	size_t shadow_size;
+	unsigned long shadow_start;
+
+	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
+	scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
+	shadow_size = round_up(scaled_size, PAGE_SIZE);
+
+	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
+		return -EINVAL;
+
+	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
+			shadow_start + shadow_size,
+			GFP_KERNEL,
+			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
+			__builtin_return_address(0));
+
+	if (ret) {
+		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
+		find_vm_area(addr)->flags |= VM_KASAN;
+		kmemleak_ignore(ret);
+		return 0;
+	}
+
+	return -ENOMEM;
+}
+
+void kasan_free_shadow(const struct vm_struct *vm)
+{
+	if (vm->flags & VM_KASAN)
+		vfree(kasan_mem_to_shadow(vm->addr));
+}
+
 #endif
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 17/78] kasan: shadow declarations only for software modes
  2020-12-18 22:00 incoming Andrew Morton
                   ` (15 preceding siblings ...)
  2020-12-18 22:02 ` [patch 16/78] kasan: group vmalloc code Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 18/78] kasan: rename (un)poison_shadow to (un)poison_range Andrew Morton
                   ` (60 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, valdis.kletnieks, vincenzo.frascino,
	will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: shadow declarations only for software modes

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Group shadow-related KASAN function declarations and only define them for
the two existing software modes.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/e88d94eff94db883a65dca52e1736d80d28dd9bc.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Valdis Kletnieks <valdis.kletnieks@vt.edu>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
[valdis.kletnieks@vt.edu: fix build issue with asmlinkage]
  Link: https://lkml.kernel.org/r/35126.1606402815@turing-police
  Link: https://lore.kernel.org/linux-arm-kernel/24105.1606397102@turing-police/
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |   48 ++++++++++++++++++++++++++--------------
 1 file changed, 32 insertions(+), 16 deletions(-)

--- a/include/linux/kasan.h~kasan-shadow-declarations-only-for-software-modes
+++ a/include/linux/kasan.h
@@ -11,7 +11,7 @@ struct task_struct;
 
 #ifdef CONFIG_KASAN
 
-#include <linux/pgtable.h>
+#include <linux/linkage.h>
 #include <asm/kasan.h>
 
 /* kasan_data struct is used in KUnit tests for KASAN expected failures */
@@ -20,6 +20,20 @@ struct kunit_kasan_expectation {
 	bool report_found;
 };
 
+#endif
+
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
+
+#include <linux/pgtable.h>
+
+/* Software KASAN implementations use shadow memory. */
+
+#ifdef CONFIG_KASAN_SW_TAGS
+#define KASAN_SHADOW_INIT 0xFF
+#else
+#define KASAN_SHADOW_INIT 0
+#endif
+
 extern unsigned char kasan_early_shadow_page[PAGE_SIZE];
 extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE];
 extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD];
@@ -35,6 +49,23 @@ static inline void *kasan_mem_to_shadow(
 		+ KASAN_SHADOW_OFFSET;
 }
 
+int kasan_add_zero_shadow(void *start, unsigned long size);
+void kasan_remove_zero_shadow(void *start, unsigned long size);
+
+#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+static inline int kasan_add_zero_shadow(void *start, unsigned long size)
+{
+	return 0;
+}
+static inline void kasan_remove_zero_shadow(void *start,
+					unsigned long size)
+{}
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+#ifdef CONFIG_KASAN
+
 /* Enable reporting bugs after kasan_disable_current() */
 extern void kasan_enable_current(void);
 
@@ -75,9 +106,6 @@ struct kasan_cache {
 	int free_meta_offset;
 };
 
-int kasan_add_zero_shadow(void *start, unsigned long size);
-void kasan_remove_zero_shadow(void *start, unsigned long size);
-
 size_t __ksize(const void *);
 static inline void kasan_unpoison_slab(const void *ptr)
 {
@@ -143,14 +171,6 @@ static inline bool kasan_slab_free(struc
 	return false;
 }
 
-static inline int kasan_add_zero_shadow(void *start, unsigned long size)
-{
-	return 0;
-}
-static inline void kasan_remove_zero_shadow(void *start,
-					unsigned long size)
-{}
-
 static inline void kasan_unpoison_slab(const void *ptr) { }
 static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
 
@@ -158,8 +178,6 @@ static inline size_t kasan_metadata_size
 
 #ifdef CONFIG_KASAN_GENERIC
 
-#define KASAN_SHADOW_INIT 0
-
 void kasan_cache_shrink(struct kmem_cache *cache);
 void kasan_cache_shutdown(struct kmem_cache *cache);
 void kasan_record_aux_stack(void *ptr);
@@ -174,8 +192,6 @@ static inline void kasan_record_aux_stac
 
 #ifdef CONFIG_KASAN_SW_TAGS
 
-#define KASAN_SHADOW_INIT 0xFF
-
 void kasan_init_tags(void);
 
 void *kasan_reset_tag(const void *addr);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 18/78] kasan: rename (un)poison_shadow to (un)poison_range
  2020-12-18 22:00 incoming Andrew Morton
                   ` (16 preceding siblings ...)
  2020-12-18 22:02 ` [patch 17/78] kasan: shadow declarations only for software modes Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 19/78] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrew Morton
                   ` (59 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename (un)poison_shadow to (un)poison_range

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory.  Rename external annotation
kasan_unpoison_shadow() to kasan_unpoison_range(), and introduce internal
functions (un)poison_range() (without kasan_ prefix).

Co-developed-by: Marco Elver <elver@google.com>
Link: https://lkml.kernel.org/r/fccdcaa13dc6b2211bf363d6c6d499279a54fe3a.1606161801.git.andreyknvl@google.com
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |    6 ++--
 kernel/fork.c         |    4 +--
 mm/kasan/common.c     |   49 ++++++++++++++++++++++------------------
 mm/kasan/generic.c    |   23 ++++++++----------
 mm/kasan/kasan.h      |    3 +-
 mm/kasan/tags.c       |    2 -
 mm/slab_common.c      |    2 -
 7 files changed, 47 insertions(+), 42 deletions(-)

--- a/include/linux/kasan.h~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/include/linux/kasan.h
@@ -72,7 +72,7 @@ extern void kasan_enable_current(void);
 /* Disable reporting bugs for current task */
 extern void kasan_disable_current(void);
 
-void kasan_unpoison_shadow(const void *address, size_t size);
+void kasan_unpoison_range(const void *address, size_t size);
 
 void kasan_unpoison_task_stack(struct task_struct *task);
 
@@ -109,7 +109,7 @@ struct kasan_cache {
 size_t __ksize(const void *);
 static inline void kasan_unpoison_slab(const void *ptr)
 {
-	kasan_unpoison_shadow(ptr, __ksize(ptr));
+	kasan_unpoison_range(ptr, __ksize(ptr));
 }
 size_t kasan_metadata_size(struct kmem_cache *cache);
 
@@ -118,7 +118,7 @@ void kasan_restore_multi_shot(bool enabl
 
 #else /* CONFIG_KASAN */
 
-static inline void kasan_unpoison_shadow(const void *address, size_t size) {}
+static inline void kasan_unpoison_range(const void *address, size_t size) {}
 
 static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
 
--- a/kernel/fork.c~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/kernel/fork.c
@@ -225,8 +225,8 @@ static unsigned long *alloc_thread_stack
 		if (!s)
 			continue;
 
-		/* Clear the KASAN shadow of the stack. */
-		kasan_unpoison_shadow(s->addr, THREAD_SIZE);
+		/* Mark stack accessible for KASAN. */
+		kasan_unpoison_range(s->addr, THREAD_SIZE);
 
 		/* Clear stale pointers from reused stack. */
 		memset(s->addr, 0, THREAD_SIZE);
--- a/mm/kasan/common.c~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/mm/kasan/common.c
@@ -108,7 +108,7 @@ void *memcpy(void *dest, const void *src
  * Poisons the shadow memory for 'size' bytes starting from 'addr'.
  * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
  */
-void kasan_poison_shadow(const void *address, size_t size, u8 value)
+void poison_range(const void *address, size_t size, u8 value)
 {
 	void *shadow_start, *shadow_end;
 
@@ -125,7 +125,7 @@ void kasan_poison_shadow(const void *add
 	__memset(shadow_start, value, shadow_end - shadow_start);
 }
 
-void kasan_unpoison_shadow(const void *address, size_t size)
+void unpoison_range(const void *address, size_t size)
 {
 	u8 tag = get_tag(address);
 
@@ -136,7 +136,7 @@ void kasan_unpoison_shadow(const void *a
 	 */
 	address = reset_tag(address);
 
-	kasan_poison_shadow(address, size, tag);
+	poison_range(address, size, tag);
 
 	if (size & KASAN_SHADOW_MASK) {
 		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
@@ -148,12 +148,17 @@ void kasan_unpoison_shadow(const void *a
 	}
 }
 
+void kasan_unpoison_range(const void *address, size_t size)
+{
+	unpoison_range(address, size);
+}
+
 static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
 {
 	void *base = task_stack_page(task);
 	size_t size = sp - base;
 
-	kasan_unpoison_shadow(base, size);
+	unpoison_range(base, size);
 }
 
 /* Unpoison the entire stack for a task. */
@@ -172,7 +177,7 @@ asmlinkage void kasan_unpoison_task_stac
 	 */
 	void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1));
 
-	kasan_unpoison_shadow(base, watermark - base);
+	unpoison_range(base, watermark - base);
 }
 
 void kasan_alloc_pages(struct page *page, unsigned int order)
@@ -186,13 +191,13 @@ void kasan_alloc_pages(struct page *page
 	tag = random_tag();
 	for (i = 0; i < (1 << order); i++)
 		page_kasan_tag_set(page + i, tag);
-	kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
+	unpoison_range(page_address(page), PAGE_SIZE << order);
 }
 
 void kasan_free_pages(struct page *page, unsigned int order)
 {
 	if (likely(!PageHighMem(page)))
-		kasan_poison_shadow(page_address(page),
+		poison_range(page_address(page),
 				PAGE_SIZE << order,
 				KASAN_FREE_PAGE);
 }
@@ -284,18 +289,18 @@ void kasan_poison_slab(struct page *page
 
 	for (i = 0; i < compound_nr(page); i++)
 		page_kasan_tag_reset(page + i);
-	kasan_poison_shadow(page_address(page), page_size(page),
-			KASAN_KMALLOC_REDZONE);
+	poison_range(page_address(page), page_size(page),
+		     KASAN_KMALLOC_REDZONE);
 }
 
 void kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
 {
-	kasan_unpoison_shadow(object, cache->object_size);
+	unpoison_range(object, cache->object_size);
 }
 
 void kasan_poison_object_data(struct kmem_cache *cache, void *object)
 {
-	kasan_poison_shadow(object,
+	poison_range(object,
 			round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
 			KASAN_KMALLOC_REDZONE);
 }
@@ -408,7 +413,7 @@ static bool __kasan_slab_free(struct kme
 	}
 
 	rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
-	kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
+	poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE);
 
 	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
 			unlikely(!(cache->flags & SLAB_KASAN)))
@@ -448,9 +453,9 @@ static void *__kasan_kmalloc(struct kmem
 		tag = assign_tag(cache, object, false, keep_tag);
 
 	/* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
-	kasan_unpoison_shadow(set_tag(object, tag), size);
-	kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
-		KASAN_KMALLOC_REDZONE);
+	unpoison_range(set_tag(object, tag), size);
+	poison_range((void *)redzone_start, redzone_end - redzone_start,
+		     KASAN_KMALLOC_REDZONE);
 
 	if (cache->flags & SLAB_KASAN)
 		kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags);
@@ -489,9 +494,9 @@ void * __must_check kasan_kmalloc_large(
 				KASAN_SHADOW_SCALE_SIZE);
 	redzone_end = (unsigned long)ptr + page_size(page);
 
-	kasan_unpoison_shadow(ptr, size);
-	kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
-		KASAN_PAGE_REDZONE);
+	unpoison_range(ptr, size);
+	poison_range((void *)redzone_start, redzone_end - redzone_start,
+		     KASAN_PAGE_REDZONE);
 
 	return (void *)ptr;
 }
@@ -523,7 +528,7 @@ void kasan_poison_kfree(void *ptr, unsig
 			kasan_report_invalid_free(ptr, ip);
 			return;
 		}
-		kasan_poison_shadow(ptr, page_size(page), KASAN_FREE_PAGE);
+		poison_range(ptr, page_size(page), KASAN_FREE_PAGE);
 	} else {
 		__kasan_slab_free(page->slab_cache, ptr, ip, false);
 	}
@@ -709,7 +714,7 @@ int kasan_populate_vmalloc(unsigned long
 	 * // vmalloc() allocates memory
 	 * // let a = area->addr
 	 * // we reach kasan_populate_vmalloc
-	 * // and call kasan_unpoison_shadow:
+	 * // and call unpoison_range:
 	 * STORE shadow(a), unpoison_val
 	 * ...
 	 * STORE shadow(a+99), unpoison_val	x = LOAD p
@@ -744,7 +749,7 @@ void kasan_poison_vmalloc(const void *st
 		return;
 
 	size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
-	kasan_poison_shadow(start, size, KASAN_VMALLOC_INVALID);
+	poison_range(start, size, KASAN_VMALLOC_INVALID);
 }
 
 void kasan_unpoison_vmalloc(const void *start, unsigned long size)
@@ -752,7 +757,7 @@ void kasan_unpoison_vmalloc(const void *
 	if (!is_vmalloc_or_module_addr(start))
 		return;
 
-	kasan_unpoison_shadow(start, size);
+	unpoison_range(start, size);
 }
 
 static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
--- a/mm/kasan/generic.c~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/mm/kasan/generic.c
@@ -202,11 +202,11 @@ static void register_global(struct kasan
 {
 	size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
 
-	kasan_unpoison_shadow(global->beg, global->size);
+	unpoison_range(global->beg, global->size);
 
-	kasan_poison_shadow(global->beg + aligned_size,
-		global->size_with_redzone - aligned_size,
-		KASAN_GLOBAL_REDZONE);
+	poison_range(global->beg + aligned_size,
+		     global->size_with_redzone - aligned_size,
+		     KASAN_GLOBAL_REDZONE);
 }
 
 void __asan_register_globals(struct kasan_global *globals, size_t size)
@@ -285,13 +285,12 @@ void __asan_alloca_poison(unsigned long
 
 	WARN_ON(!IS_ALIGNED(addr, KASAN_ALLOCA_REDZONE_SIZE));
 
-	kasan_unpoison_shadow((const void *)(addr + rounded_down_size),
-			      size - rounded_down_size);
-	kasan_poison_shadow(left_redzone, KASAN_ALLOCA_REDZONE_SIZE,
-			KASAN_ALLOCA_LEFT);
-	kasan_poison_shadow(right_redzone,
-			padding_size + KASAN_ALLOCA_REDZONE_SIZE,
-			KASAN_ALLOCA_RIGHT);
+	unpoison_range((const void *)(addr + rounded_down_size),
+		       size - rounded_down_size);
+	poison_range(left_redzone, KASAN_ALLOCA_REDZONE_SIZE,
+		     KASAN_ALLOCA_LEFT);
+	poison_range(right_redzone, padding_size + KASAN_ALLOCA_REDZONE_SIZE,
+		     KASAN_ALLOCA_RIGHT);
 }
 EXPORT_SYMBOL(__asan_alloca_poison);
 
@@ -301,7 +300,7 @@ void __asan_allocas_unpoison(const void
 	if (unlikely(!stack_top || stack_top > stack_bottom))
 		return;
 
-	kasan_unpoison_shadow(stack_top, stack_bottom - stack_top);
+	unpoison_range(stack_top, stack_bottom - stack_top);
 }
 EXPORT_SYMBOL(__asan_allocas_unpoison);
 
--- a/mm/kasan/kasan.h~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/mm/kasan/kasan.h
@@ -150,7 +150,8 @@ static inline bool addr_has_shadow(const
 	return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
 }
 
-void kasan_poison_shadow(const void *address, size_t size, u8 value);
+void poison_range(const void *address, size_t size, u8 value);
+void unpoison_range(const void *address, size_t size);
 
 /**
  * check_memory_region - Check memory region, and report if invalid access.
--- a/mm/kasan/tags.c~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/mm/kasan/tags.c
@@ -153,7 +153,7 @@ EXPORT_SYMBOL(__hwasan_storeN_noabort);
 
 void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size)
 {
-	kasan_poison_shadow((void *)addr, size, tag);
+	poison_range((void *)addr, size, tag);
 }
 EXPORT_SYMBOL(__hwasan_tag_memory);
 
--- a/mm/slab_common.c~kasan-rename-unpoison_shadow-to-unpoison_range
+++ a/mm/slab_common.c
@@ -1176,7 +1176,7 @@ size_t ksize(const void *objp)
 	 * We assume that ksize callers could use whole allocated area,
 	 * so we need to unpoison this area.
 	 */
-	kasan_unpoison_shadow(objp, size);
+	kasan_unpoison_range(objp, size);
 	return size;
 }
 EXPORT_SYMBOL(ksize);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 19/78] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_*
  2020-12-18 22:00 incoming Andrew Morton
                   ` (17 preceding siblings ...)
  2020-12-18 22:02 ` [patch 18/78] kasan: rename (un)poison_shadow to (un)poison_range Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 20/78] kasan: only build init.c for software modes Andrew Morton
                   ` (58 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_*

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory, but will still use the concept
of memory granules.  Each memory granule maps to a single metadata entry:
8 bytes per one shadow byte for generic mode, 16 bytes per one shadow byte
for software tag-based mode, and 16 bytes per one allocation tag for
hardware tag-based mode.

Rename KASAN_SHADOW_SCALE_SIZE to KASAN_GRANULE_SIZE, and
KASAN_SHADOW_MASK to KASAN_GRANULE_MASK.

Also use MASK when used as a mask, otherwise use SIZE.

No functional changes.

Link: https://lkml.kernel.org/r/939b5754e47f528a6e6a6f28ffc5815d8d128033.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 Documentation/dev-tools/kasan.rst |    2 -
 lib/test_kasan.c                  |    2 -
 lib/test_kasan_module.c           |    2 -
 mm/kasan/common.c                 |   39 ++++++++++++++--------------
 mm/kasan/generic.c                |   14 +++++-----
 mm/kasan/generic_report.c         |    8 ++---
 mm/kasan/init.c                   |    8 ++---
 mm/kasan/kasan.h                  |    4 +-
 mm/kasan/report.c                 |   10 +++----
 mm/kasan/tags_report.c            |    2 -
 10 files changed, 46 insertions(+), 45 deletions(-)

--- a/Documentation/dev-tools/kasan.rst~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/Documentation/dev-tools/kasan.rst
@@ -265,7 +265,7 @@ Most mappings in vmalloc space are small
 page of shadow space. Allocating a full shadow page per mapping would
 therefore be wasteful. Furthermore, to ensure that different mappings
 use different shadow pages, mappings would have to be aligned to
-``KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE``.
+``KASAN_GRANULE_SIZE * PAGE_SIZE``.
 
 Instead, we share backing space across multiple mappings. We allocate
 a backing page when a mapping in vmalloc space uses a particular page
--- a/lib/test_kasan.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/lib/test_kasan.c
@@ -25,7 +25,7 @@
 
 #include "../mm/kasan/kasan.h"
 
-#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE)
+#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE)
 
 /*
  * We assign some test results to these globals to make sure the tests
--- a/lib/test_kasan_module.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/lib/test_kasan_module.c
@@ -15,7 +15,7 @@
 
 #include "../mm/kasan/kasan.h"
 
-#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE)
+#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE)
 
 static noinline void __init copy_user_test(void)
 {
--- a/mm/kasan/common.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/common.c
@@ -106,7 +106,7 @@ void *memcpy(void *dest, const void *src
 
 /*
  * Poisons the shadow memory for 'size' bytes starting from 'addr'.
- * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
+ * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
  */
 void poison_range(const void *address, size_t size, u8 value)
 {
@@ -138,13 +138,13 @@ void unpoison_range(const void *address,
 
 	poison_range(address, size, tag);
 
-	if (size & KASAN_SHADOW_MASK) {
+	if (size & KASAN_GRANULE_MASK) {
 		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
 
 		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 			*shadow = tag;
 		else
-			*shadow = size & KASAN_SHADOW_MASK;
+			*shadow = size & KASAN_GRANULE_MASK;
 	}
 }
 
@@ -301,7 +301,7 @@ void kasan_unpoison_object_data(struct k
 void kasan_poison_object_data(struct kmem_cache *cache, void *object)
 {
 	poison_range(object,
-			round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
+			round_up(cache->object_size, KASAN_GRANULE_SIZE),
 			KASAN_KMALLOC_REDZONE);
 }
 
@@ -373,7 +373,7 @@ static inline bool shadow_invalid(u8 tag
 {
 	if (IS_ENABLED(CONFIG_KASAN_GENERIC))
 		return shadow_byte < 0 ||
-			shadow_byte >= KASAN_SHADOW_SCALE_SIZE;
+			shadow_byte >= KASAN_GRANULE_SIZE;
 
 	/* else CONFIG_KASAN_SW_TAGS: */
 	if ((u8)shadow_byte == KASAN_TAG_INVALID)
@@ -412,7 +412,7 @@ static bool __kasan_slab_free(struct kme
 		return true;
 	}
 
-	rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
+	rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE);
 	poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE);
 
 	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
@@ -445,9 +445,9 @@ static void *__kasan_kmalloc(struct kmem
 		return NULL;
 
 	redzone_start = round_up((unsigned long)(object + size),
-				KASAN_SHADOW_SCALE_SIZE);
+				KASAN_GRANULE_SIZE);
 	redzone_end = round_up((unsigned long)object + cache->object_size,
-				KASAN_SHADOW_SCALE_SIZE);
+				KASAN_GRANULE_SIZE);
 
 	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		tag = assign_tag(cache, object, false, keep_tag);
@@ -491,7 +491,7 @@ void * __must_check kasan_kmalloc_large(
 
 	page = virt_to_page(ptr);
 	redzone_start = round_up((unsigned long)(ptr + size),
-				KASAN_SHADOW_SCALE_SIZE);
+				KASAN_GRANULE_SIZE);
 	redzone_end = (unsigned long)ptr + page_size(page);
 
 	unpoison_range(ptr, size);
@@ -589,8 +589,8 @@ static int __meminit kasan_mem_notifier(
 	shadow_size = nr_shadow_pages << PAGE_SHIFT;
 	shadow_end = shadow_start + shadow_size;
 
-	if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) ||
-		WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT)))
+	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
+		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
 		return NOTIFY_BAD;
 
 	switch (action) {
@@ -748,7 +748,7 @@ void kasan_poison_vmalloc(const void *st
 	if (!is_vmalloc_or_module_addr(start))
 		return;
 
-	size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
+	size = round_up(size, KASAN_GRANULE_SIZE);
 	poison_range(start, size, KASAN_VMALLOC_INVALID);
 }
 
@@ -861,22 +861,22 @@ void kasan_release_vmalloc(unsigned long
 	unsigned long region_start, region_end;
 	unsigned long size;
 
-	region_start = ALIGN(start, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
-	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
+	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
+	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
 
 	free_region_start = ALIGN(free_region_start,
-				  PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
+				  PAGE_SIZE * KASAN_GRANULE_SIZE);
 
 	if (start != region_start &&
 	    free_region_start < region_start)
-		region_start -= PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE;
+		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
 
 	free_region_end = ALIGN_DOWN(free_region_end,
-				     PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
+				     PAGE_SIZE * KASAN_GRANULE_SIZE);
 
 	if (end != region_end &&
 	    free_region_end > region_end)
-		region_end += PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE;
+		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
 
 	shadow_start = kasan_mem_to_shadow((void *)region_start);
 	shadow_end = kasan_mem_to_shadow((void *)region_end);
@@ -902,7 +902,8 @@ int kasan_module_alloc(void *addr, size_
 	unsigned long shadow_start;
 
 	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
-	scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
+	scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
+				KASAN_SHADOW_SCALE_SHIFT;
 	shadow_size = round_up(scaled_size, PAGE_SIZE);
 
 	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
--- a/mm/kasan/generic.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/generic.c
@@ -46,7 +46,7 @@ static __always_inline bool memory_is_po
 	s8 shadow_value = *(s8 *)kasan_mem_to_shadow((void *)addr);
 
 	if (unlikely(shadow_value)) {
-		s8 last_accessible_byte = addr & KASAN_SHADOW_MASK;
+		s8 last_accessible_byte = addr & KASAN_GRANULE_MASK;
 		return unlikely(last_accessible_byte >= shadow_value);
 	}
 
@@ -62,7 +62,7 @@ static __always_inline bool memory_is_po
 	 * Access crosses 8(shadow size)-byte boundary. Such access maps
 	 * into 2 shadow bytes, so we need to check them both.
 	 */
-	if (unlikely(((addr + size - 1) & KASAN_SHADOW_MASK) < size - 1))
+	if (unlikely(((addr + size - 1) & KASAN_GRANULE_MASK) < size - 1))
 		return *shadow_addr || memory_is_poisoned_1(addr + size - 1);
 
 	return memory_is_poisoned_1(addr + size - 1);
@@ -73,7 +73,7 @@ static __always_inline bool memory_is_po
 	u16 *shadow_addr = (u16 *)kasan_mem_to_shadow((void *)addr);
 
 	/* Unaligned 16-bytes access maps into 3 shadow bytes. */
-	if (unlikely(!IS_ALIGNED(addr, KASAN_SHADOW_SCALE_SIZE)))
+	if (unlikely(!IS_ALIGNED(addr, KASAN_GRANULE_SIZE)))
 		return *shadow_addr || memory_is_poisoned_1(addr + 15);
 
 	return *shadow_addr;
@@ -134,7 +134,7 @@ static __always_inline bool memory_is_po
 		s8 *last_shadow = (s8 *)kasan_mem_to_shadow((void *)last_byte);
 
 		if (unlikely(ret != (unsigned long)last_shadow ||
-			((long)(last_byte & KASAN_SHADOW_MASK) >= *last_shadow)))
+			((long)(last_byte & KASAN_GRANULE_MASK) >= *last_shadow)))
 			return true;
 	}
 	return false;
@@ -200,7 +200,7 @@ void kasan_cache_shutdown(struct kmem_ca
 
 static void register_global(struct kasan_global *global)
 {
-	size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
+	size_t aligned_size = round_up(global->size, KASAN_GRANULE_SIZE);
 
 	unpoison_range(global->beg, global->size);
 
@@ -274,10 +274,10 @@ EXPORT_SYMBOL(__asan_handle_no_return);
 /* Emitted by compiler to poison alloca()ed objects. */
 void __asan_alloca_poison(unsigned long addr, size_t size)
 {
-	size_t rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
+	size_t rounded_up_size = round_up(size, KASAN_GRANULE_SIZE);
 	size_t padding_size = round_up(size, KASAN_ALLOCA_REDZONE_SIZE) -
 			rounded_up_size;
-	size_t rounded_down_size = round_down(size, KASAN_SHADOW_SCALE_SIZE);
+	size_t rounded_down_size = round_down(size, KASAN_GRANULE_SIZE);
 
 	const void *left_redzone = (const void *)(addr -
 			KASAN_ALLOCA_REDZONE_SIZE);
--- a/mm/kasan/generic_report.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/generic_report.c
@@ -34,7 +34,7 @@ void *find_first_bad_addr(void *addr, si
 	void *p = addr;
 
 	while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p)))
-		p += KASAN_SHADOW_SCALE_SIZE;
+		p += KASAN_GRANULE_SIZE;
 	return p;
 }
 
@@ -46,14 +46,14 @@ static const char *get_shadow_bug_type(s
 	shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr);
 
 	/*
-	 * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look
+	 * If shadow byte value is in [0, KASAN_GRANULE_SIZE) we can look
 	 * at the next shadow byte to determine the type of the bad access.
 	 */
-	if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1)
+	if (*shadow_addr > 0 && *shadow_addr <= KASAN_GRANULE_SIZE - 1)
 		shadow_addr++;
 
 	switch (*shadow_addr) {
-	case 0 ... KASAN_SHADOW_SCALE_SIZE - 1:
+	case 0 ... KASAN_GRANULE_SIZE - 1:
 		/*
 		 * In theory it's still possible to see these shadow values
 		 * due to a data race in the kernel code.
--- a/mm/kasan/init.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/init.c
@@ -442,8 +442,8 @@ void kasan_remove_zero_shadow(void *star
 	end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
 	if (WARN_ON((unsigned long)start %
-			(KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)))
+			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
+	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
 		return;
 
 	for (; addr < end; addr = next) {
@@ -477,8 +477,8 @@ int kasan_add_zero_shadow(void *start, u
 	shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
 	if (WARN_ON((unsigned long)start %
-			(KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)))
+			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
+	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
 		return -EINVAL;
 
 	ret = kasan_populate_early_shadow(shadow_start, shadow_end);
--- a/mm/kasan/kasan.h~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/kasan.h
@@ -5,8 +5,8 @@
 #include <linux/kasan.h>
 #include <linux/stackdepot.h>
 
-#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT)
-#define KASAN_SHADOW_MASK       (KASAN_SHADOW_SCALE_SIZE - 1)
+#define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
+#define KASAN_GRANULE_MASK	(KASAN_GRANULE_SIZE - 1)
 
 #define KASAN_TAG_KERNEL	0xFF /* native kernel pointers tag */
 #define KASAN_TAG_INVALID	0xFE /* inaccessible memory tag */
--- a/mm/kasan/report.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/report.c
@@ -314,24 +314,24 @@ static bool __must_check get_address_sta
 		return false;
 
 	aligned_addr = round_down((unsigned long)addr, sizeof(long));
-	mem_ptr = round_down(aligned_addr, KASAN_SHADOW_SCALE_SIZE);
+	mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
 	shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
 	shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
 
 	while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
 		shadow_ptr--;
-		mem_ptr -= KASAN_SHADOW_SCALE_SIZE;
+		mem_ptr -= KASAN_GRANULE_SIZE;
 	}
 
 	while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
 		shadow_ptr--;
-		mem_ptr -= KASAN_SHADOW_SCALE_SIZE;
+		mem_ptr -= KASAN_GRANULE_SIZE;
 	}
 
 	if (shadow_ptr < shadow_bottom)
 		return false;
 
-	frame = (const unsigned long *)(mem_ptr + KASAN_SHADOW_SCALE_SIZE);
+	frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
 	if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
 		pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
 		       frame[0]);
@@ -599,6 +599,6 @@ void kasan_non_canonical_hook(unsigned l
 	else
 		bug_type = "maybe wild-memory-access";
 	pr_alert("KASAN: %s in range [0x%016lx-0x%016lx]\n", bug_type,
-		 orig_addr, orig_addr + KASAN_SHADOW_MASK);
+		 orig_addr, orig_addr + KASAN_GRANULE_SIZE - 1);
 }
 #endif
--- a/mm/kasan/tags_report.c~kasan-rename-kasan_shadow_-to-kasan_granule_
+++ a/mm/kasan/tags_report.c
@@ -76,7 +76,7 @@ void *find_first_bad_addr(void *addr, si
 	void *end = p + size;
 
 	while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p))
-		p += KASAN_SHADOW_SCALE_SIZE;
+		p += KASAN_GRANULE_SIZE;
 	return p;
 }
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 20/78] kasan: only build init.c for software modes
  2020-12-18 22:00 incoming Andrew Morton
                   ` (18 preceding siblings ...)
  2020-12-18 22:02 ` [patch 19/78] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 21/78] kasan: split out shadow.c from common.c Andrew Morton
                   ` (57 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: only build init.c for software modes

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory, so only build init.c that
contains shadow initialization code for software modes.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/bae0a6a35b7a9b1a443803c1a55e6e3fecc311c9.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/Makefile |    6 +++---
 mm/kasan/init.c   |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

--- a/mm/kasan/init.c~kasan-only-build-initc-for-software-modes
+++ a/mm/kasan/init.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains some kasan initialization code.
+ * This file contains KASAN shadow initialization code.
  *
  * Copyright (c) 2015 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
--- a/mm/kasan/Makefile~kasan-only-build-initc-for-software-modes
+++ a/mm/kasan/Makefile
@@ -29,6 +29,6 @@ CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNT
 CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 
-obj-$(CONFIG_KASAN) := common.o init.o report.o
-obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o
-obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o
+obj-$(CONFIG_KASAN) := common.o report.o
+obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o
+obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-18 22:00 incoming Andrew Morton
                   ` (19 preceding siblings ...)
  2020-12-18 22:02 ` [patch 20/78] kasan: only build init.c for software modes Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-19  0:28   ` Marco Elver
  2020-12-22 12:00   ` kernel test robot
  2020-12-18 22:02 ` [patch 22/78] kasan: define KASAN_MEMORY_PER_SHADOW_PAGE Andrew Morton
                   ` (56 subsequent siblings)
  77 siblings, 2 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: split out shadow.c from common.c

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory.  Move all shadow-related code
to shadow.c, which is only enabled for software KASAN modes that use
shadow memory.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/17d95cfa7d5cf9c4fcd9bf415f2a8dea911668df.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/Makefile |    6 
 mm/kasan/common.c |  486 -----------------------------------------
 mm/kasan/shadow.c |  518 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 523 insertions(+), 487 deletions(-)

--- a/mm/kasan/common.c~kasan-split-out-shadowc-from-commonc
+++ a/mm/kasan/common.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains common generic and tag-based KASAN code.
+ * This file contains common KASAN code.
  *
  * Copyright (c) 2014 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
@@ -13,7 +13,6 @@
 #include <linux/init.h>
 #include <linux/kasan.h>
 #include <linux/kernel.h>
-#include <linux/kmemleak.h>
 #include <linux/linkage.h>
 #include <linux/memblock.h>
 #include <linux/memory.h>
@@ -26,12 +25,8 @@
 #include <linux/stacktrace.h>
 #include <linux/string.h>
 #include <linux/types.h>
-#include <linux/vmalloc.h>
 #include <linux/bug.h>
 
-#include <asm/cacheflush.h>
-#include <asm/tlbflush.h>
-
 #include "kasan.h"
 #include "../slab.h"
 
@@ -61,93 +56,6 @@ void kasan_disable_current(void)
 	current->kasan_depth--;
 }
 
-bool __kasan_check_read(const volatile void *p, unsigned int size)
-{
-	return check_memory_region((unsigned long)p, size, false, _RET_IP_);
-}
-EXPORT_SYMBOL(__kasan_check_read);
-
-bool __kasan_check_write(const volatile void *p, unsigned int size)
-{
-	return check_memory_region((unsigned long)p, size, true, _RET_IP_);
-}
-EXPORT_SYMBOL(__kasan_check_write);
-
-#undef memset
-void *memset(void *addr, int c, size_t len)
-{
-	if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_))
-		return NULL;
-
-	return __memset(addr, c, len);
-}
-
-#ifdef __HAVE_ARCH_MEMMOVE
-#undef memmove
-void *memmove(void *dest, const void *src, size_t len)
-{
-	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
-	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
-		return NULL;
-
-	return __memmove(dest, src, len);
-}
-#endif
-
-#undef memcpy
-void *memcpy(void *dest, const void *src, size_t len)
-{
-	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
-	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
-		return NULL;
-
-	return __memcpy(dest, src, len);
-}
-
-/*
- * Poisons the shadow memory for 'size' bytes starting from 'addr'.
- * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
- */
-void poison_range(const void *address, size_t size, u8 value)
-{
-	void *shadow_start, *shadow_end;
-
-	/*
-	 * Perform shadow offset calculation based on untagged address, as
-	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
-	 * addresses to this function.
-	 */
-	address = reset_tag(address);
-
-	shadow_start = kasan_mem_to_shadow(address);
-	shadow_end = kasan_mem_to_shadow(address + size);
-
-	__memset(shadow_start, value, shadow_end - shadow_start);
-}
-
-void unpoison_range(const void *address, size_t size)
-{
-	u8 tag = get_tag(address);
-
-	/*
-	 * Perform shadow offset calculation based on untagged address, as
-	 * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
-	 * addresses to this function.
-	 */
-	address = reset_tag(address);
-
-	poison_range(address, size, tag);
-
-	if (size & KASAN_GRANULE_MASK) {
-		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
-
-		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
-			*shadow = tag;
-		else
-			*shadow = size & KASAN_GRANULE_MASK;
-	}
-}
-
 void kasan_unpoison_range(const void *address, size_t size)
 {
 	unpoison_range(address, size);
@@ -540,395 +448,3 @@ void kasan_kfree_large(void *ptr, unsign
 		kasan_report_invalid_free(ptr, ip);
 	/* The object will be poisoned by page_alloc. */
 }
-
-#ifdef CONFIG_MEMORY_HOTPLUG
-static bool shadow_mapped(unsigned long addr)
-{
-	pgd_t *pgd = pgd_offset_k(addr);
-	p4d_t *p4d;
-	pud_t *pud;
-	pmd_t *pmd;
-	pte_t *pte;
-
-	if (pgd_none(*pgd))
-		return false;
-	p4d = p4d_offset(pgd, addr);
-	if (p4d_none(*p4d))
-		return false;
-	pud = pud_offset(p4d, addr);
-	if (pud_none(*pud))
-		return false;
-
-	/*
-	 * We can't use pud_large() or pud_huge(), the first one is
-	 * arch-specific, the last one depends on HUGETLB_PAGE.  So let's abuse
-	 * pud_bad(), if pud is bad then it's bad because it's huge.
-	 */
-	if (pud_bad(*pud))
-		return true;
-	pmd = pmd_offset(pud, addr);
-	if (pmd_none(*pmd))
-		return false;
-
-	if (pmd_bad(*pmd))
-		return true;
-	pte = pte_offset_kernel(pmd, addr);
-	return !pte_none(*pte);
-}
-
-static int __meminit kasan_mem_notifier(struct notifier_block *nb,
-			unsigned long action, void *data)
-{
-	struct memory_notify *mem_data = data;
-	unsigned long nr_shadow_pages, start_kaddr, shadow_start;
-	unsigned long shadow_end, shadow_size;
-
-	nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT;
-	start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn);
-	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr);
-	shadow_size = nr_shadow_pages << PAGE_SHIFT;
-	shadow_end = shadow_start + shadow_size;
-
-	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
-		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
-		return NOTIFY_BAD;
-
-	switch (action) {
-	case MEM_GOING_ONLINE: {
-		void *ret;
-
-		/*
-		 * If shadow is mapped already than it must have been mapped
-		 * during the boot. This could happen if we onlining previously
-		 * offlined memory.
-		 */
-		if (shadow_mapped(shadow_start))
-			return NOTIFY_OK;
-
-		ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start,
-					shadow_end, GFP_KERNEL,
-					PAGE_KERNEL, VM_NO_GUARD,
-					pfn_to_nid(mem_data->start_pfn),
-					__builtin_return_address(0));
-		if (!ret)
-			return NOTIFY_BAD;
-
-		kmemleak_ignore(ret);
-		return NOTIFY_OK;
-	}
-	case MEM_CANCEL_ONLINE:
-	case MEM_OFFLINE: {
-		struct vm_struct *vm;
-
-		/*
-		 * shadow_start was either mapped during boot by kasan_init()
-		 * or during memory online by __vmalloc_node_range().
-		 * In the latter case we can use vfree() to free shadow.
-		 * Non-NULL result of the find_vm_area() will tell us if
-		 * that was the second case.
-		 *
-		 * Currently it's not possible to free shadow mapped
-		 * during boot by kasan_init(). It's because the code
-		 * to do that hasn't been written yet. So we'll just
-		 * leak the memory.
-		 */
-		vm = find_vm_area((void *)shadow_start);
-		if (vm)
-			vfree((void *)shadow_start);
-	}
-	}
-
-	return NOTIFY_OK;
-}
-
-static int __init kasan_memhotplug_init(void)
-{
-	hotplug_memory_notifier(kasan_mem_notifier, 0);
-
-	return 0;
-}
-
-core_initcall(kasan_memhotplug_init);
-#endif
-
-#ifdef CONFIG_KASAN_VMALLOC
-
-static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
-				      void *unused)
-{
-	unsigned long page;
-	pte_t pte;
-
-	if (likely(!pte_none(*ptep)))
-		return 0;
-
-	page = __get_free_page(GFP_KERNEL);
-	if (!page)
-		return -ENOMEM;
-
-	memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
-	pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
-
-	spin_lock(&init_mm.page_table_lock);
-	if (likely(pte_none(*ptep))) {
-		set_pte_at(&init_mm, addr, ptep, pte);
-		page = 0;
-	}
-	spin_unlock(&init_mm.page_table_lock);
-	if (page)
-		free_page(page);
-	return 0;
-}
-
-int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
-{
-	unsigned long shadow_start, shadow_end;
-	int ret;
-
-	if (!is_vmalloc_or_module_addr((void *)addr))
-		return 0;
-
-	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
-	shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
-	shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
-	shadow_end = ALIGN(shadow_end, PAGE_SIZE);
-
-	ret = apply_to_page_range(&init_mm, shadow_start,
-				  shadow_end - shadow_start,
-				  kasan_populate_vmalloc_pte, NULL);
-	if (ret)
-		return ret;
-
-	flush_cache_vmap(shadow_start, shadow_end);
-
-	/*
-	 * We need to be careful about inter-cpu effects here. Consider:
-	 *
-	 *   CPU#0				  CPU#1
-	 * WRITE_ONCE(p, vmalloc(100));		while (x = READ_ONCE(p)) ;
-	 *					p[99] = 1;
-	 *
-	 * With compiler instrumentation, that ends up looking like this:
-	 *
-	 *   CPU#0				  CPU#1
-	 * // vmalloc() allocates memory
-	 * // let a = area->addr
-	 * // we reach kasan_populate_vmalloc
-	 * // and call unpoison_range:
-	 * STORE shadow(a), unpoison_val
-	 * ...
-	 * STORE shadow(a+99), unpoison_val	x = LOAD p
-	 * // rest of vmalloc process		<data dependency>
-	 * STORE p, a				LOAD shadow(x+99)
-	 *
-	 * If there is no barrier between the end of unpoisioning the shadow
-	 * and the store of the result to p, the stores could be committed
-	 * in a different order by CPU#0, and CPU#1 could erroneously observe
-	 * poison in the shadow.
-	 *
-	 * We need some sort of barrier between the stores.
-	 *
-	 * In the vmalloc() case, this is provided by a smp_wmb() in
-	 * clear_vm_uninitialized_flag(). In the per-cpu allocator and in
-	 * get_vm_area() and friends, the caller gets shadow allocated but
-	 * doesn't have any pages mapped into the virtual address space that
-	 * has been reserved. Mapping those pages in will involve taking and
-	 * releasing a page-table lock, which will provide the barrier.
-	 */
-
-	return 0;
-}
-
-/*
- * Poison the shadow for a vmalloc region. Called as part of the
- * freeing process at the time the region is freed.
- */
-void kasan_poison_vmalloc(const void *start, unsigned long size)
-{
-	if (!is_vmalloc_or_module_addr(start))
-		return;
-
-	size = round_up(size, KASAN_GRANULE_SIZE);
-	poison_range(start, size, KASAN_VMALLOC_INVALID);
-}
-
-void kasan_unpoison_vmalloc(const void *start, unsigned long size)
-{
-	if (!is_vmalloc_or_module_addr(start))
-		return;
-
-	unpoison_range(start, size);
-}
-
-static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
-					void *unused)
-{
-	unsigned long page;
-
-	page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT);
-
-	spin_lock(&init_mm.page_table_lock);
-
-	if (likely(!pte_none(*ptep))) {
-		pte_clear(&init_mm, addr, ptep);
-		free_page(page);
-	}
-	spin_unlock(&init_mm.page_table_lock);
-
-	return 0;
-}
-
-/*
- * Release the backing for the vmalloc region [start, end), which
- * lies within the free region [free_region_start, free_region_end).
- *
- * This can be run lazily, long after the region was freed. It runs
- * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap
- * infrastructure.
- *
- * How does this work?
- * -------------------
- *
- * We have a region that is page aligned, labelled as A.
- * That might not map onto the shadow in a way that is page-aligned:
- *
- *                    start                     end
- *                    v                         v
- * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc
- *  -------- -------- --------          -------- --------
- *      |        |       |                 |        |
- *      |        |       |         /-------/        |
- *      \-------\|/------/         |/---------------/
- *              |||                ||
- *             |??AAAAAA|AAAAAAAA|AA??????|                < shadow
- *                 (1)      (2)      (3)
- *
- * First we align the start upwards and the end downwards, so that the
- * shadow of the region aligns with shadow page boundaries. In the
- * example, this gives us the shadow page (2). This is the shadow entirely
- * covered by this allocation.
- *
- * Then we have the tricky bits. We want to know if we can free the
- * partially covered shadow pages - (1) and (3) in the example. For this,
- * we are given the start and end of the free region that contains this
- * allocation. Extending our previous example, we could have:
- *
- *  free_region_start                                    free_region_end
- *  |                 start                     end      |
- *  v                 v                         v        v
- * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc
- *  -------- -------- --------          -------- --------
- *      |        |       |                 |        |
- *      |        |       |         /-------/        |
- *      \-------\|/------/         |/---------------/
- *              |||                ||
- *             |FFAAAAAA|AAAAAAAA|AAF?????|                < shadow
- *                 (1)      (2)      (3)
- *
- * Once again, we align the start of the free region up, and the end of
- * the free region down so that the shadow is page aligned. So we can free
- * page (1) - we know no allocation currently uses anything in that page,
- * because all of it is in the vmalloc free region. But we cannot free
- * page (3), because we can't be sure that the rest of it is unused.
- *
- * We only consider pages that contain part of the original region for
- * freeing: we don't try to free other pages from the free region or we'd
- * end up trying to free huge chunks of virtual address space.
- *
- * Concurrency
- * -----------
- *
- * How do we know that we're not freeing a page that is simultaneously
- * being used for a fresh allocation in kasan_populate_vmalloc(_pte)?
- *
- * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running
- * at the same time. While we run under free_vmap_area_lock, the population
- * code does not.
- *
- * free_vmap_area_lock instead operates to ensure that the larger range
- * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and
- * the per-cpu region-finding algorithm both run under free_vmap_area_lock,
- * no space identified as free will become used while we are running. This
- * means that so long as we are careful with alignment and only free shadow
- * pages entirely covered by the free region, we will not run in to any
- * trouble - any simultaneous allocations will be for disjoint regions.
- */
-void kasan_release_vmalloc(unsigned long start, unsigned long end,
-			   unsigned long free_region_start,
-			   unsigned long free_region_end)
-{
-	void *shadow_start, *shadow_end;
-	unsigned long region_start, region_end;
-	unsigned long size;
-
-	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
-	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
-
-	free_region_start = ALIGN(free_region_start,
-				  PAGE_SIZE * KASAN_GRANULE_SIZE);
-
-	if (start != region_start &&
-	    free_region_start < region_start)
-		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
-
-	free_region_end = ALIGN_DOWN(free_region_end,
-				     PAGE_SIZE * KASAN_GRANULE_SIZE);
-
-	if (end != region_end &&
-	    free_region_end > region_end)
-		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
-
-	shadow_start = kasan_mem_to_shadow((void *)region_start);
-	shadow_end = kasan_mem_to_shadow((void *)region_end);
-
-	if (shadow_end > shadow_start) {
-		size = shadow_end - shadow_start;
-		apply_to_existing_page_range(&init_mm,
-					     (unsigned long)shadow_start,
-					     size, kasan_depopulate_vmalloc_pte,
-					     NULL);
-		flush_tlb_kernel_range((unsigned long)shadow_start,
-				       (unsigned long)shadow_end);
-	}
-}
-
-#else /* CONFIG_KASAN_VMALLOC */
-
-int kasan_module_alloc(void *addr, size_t size)
-{
-	void *ret;
-	size_t scaled_size;
-	size_t shadow_size;
-	unsigned long shadow_start;
-
-	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
-	scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
-				KASAN_SHADOW_SCALE_SHIFT;
-	shadow_size = round_up(scaled_size, PAGE_SIZE);
-
-	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
-		return -EINVAL;
-
-	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
-			shadow_start + shadow_size,
-			GFP_KERNEL,
-			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
-			__builtin_return_address(0));
-
-	if (ret) {
-		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
-		find_vm_area(addr)->flags |= VM_KASAN;
-		kmemleak_ignore(ret);
-		return 0;
-	}
-
-	return -ENOMEM;
-}
-
-void kasan_free_shadow(const struct vm_struct *vm)
-{
-	if (vm->flags & VM_KASAN)
-		vfree(kasan_mem_to_shadow(vm->addr));
-}
-
-#endif
--- a/mm/kasan/Makefile~kasan-split-out-shadowc-from-commonc
+++ a/mm/kasan/Makefile
@@ -10,6 +10,7 @@ CFLAGS_REMOVE_generic_report.o = $(CC_FL
 CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE)
 
@@ -26,9 +27,10 @@ CFLAGS_generic_report.o := $(CC_FLAGS_KA
 CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 
 obj-$(CONFIG_KASAN) := common.o report.o
-obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o
-obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o
+obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o
+obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o
--- /dev/null
+++ a/mm/kasan/shadow.c
@@ -0,0 +1,518 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This file contains KASAN runtime code that manages shadow memory for
+ * generic and software tag-based KASAN modes.
+ *
+ * Copyright (c) 2014 Samsung Electronics Co., Ltd.
+ * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
+ *
+ * Some code borrowed from https://github.com/xairy/kasan-prototype by
+ *        Andrey Konovalov <andreyknvl@gmail.com>
+ */
+
+#include <linux/init.h>
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/kfence.h>
+#include <linux/kmemleak.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include <asm/cacheflush.h>
+#include <asm/tlbflush.h>
+
+#include "kasan.h"
+
+bool __kasan_check_read(const volatile void *p, unsigned int size)
+{
+	return check_memory_region((unsigned long)p, size, false, _RET_IP_);
+}
+EXPORT_SYMBOL(__kasan_check_read);
+
+bool __kasan_check_write(const volatile void *p, unsigned int size)
+{
+	return check_memory_region((unsigned long)p, size, true, _RET_IP_);
+}
+EXPORT_SYMBOL(__kasan_check_write);
+
+#undef memset
+void *memset(void *addr, int c, size_t len)
+{
+	if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_))
+		return NULL;
+
+	return __memset(addr, c, len);
+}
+
+#ifdef __HAVE_ARCH_MEMMOVE
+#undef memmove
+void *memmove(void *dest, const void *src, size_t len)
+{
+	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
+	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
+		return NULL;
+
+	return __memmove(dest, src, len);
+}
+#endif
+
+#undef memcpy
+void *memcpy(void *dest, const void *src, size_t len)
+{
+	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
+	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
+		return NULL;
+
+	return __memcpy(dest, src, len);
+}
+
+/*
+ * Poisons the shadow memory for 'size' bytes starting from 'addr'.
+ * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
+ */
+void poison_range(const void *address, size_t size, u8 value)
+{
+	void *shadow_start, *shadow_end;
+
+	/*
+	 * Perform shadow offset calculation based on untagged address, as
+	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
+	 * addresses to this function.
+	 */
+	address = reset_tag(address);
+
+	/* Skip KFENCE memory if called explicitly outside of sl*b. */
+	if (is_kfence_address(address))
+		return;
+
+	shadow_start = kasan_mem_to_shadow(address);
+	shadow_end = kasan_mem_to_shadow(address + size);
+
+	__memset(shadow_start, value, shadow_end - shadow_start);
+}
+
+void unpoison_range(const void *address, size_t size)
+{
+	u8 tag = get_tag(address);
+
+	/*
+	 * Perform shadow offset calculation based on untagged address, as
+	 * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
+	 * addresses to this function.
+	 */
+	address = reset_tag(address);
+
+	/*
+	 * Skip KFENCE memory if called explicitly outside of sl*b. Also note
+	 * that calls to ksize(), where size is not a multiple of machine-word
+	 * size, would otherwise poison the invalid portion of the word.
+	 */
+	if (is_kfence_address(address))
+		return;
+
+	poison_range(address, size, tag);
+
+	if (size & KASAN_GRANULE_MASK) {
+		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
+
+		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
+			*shadow = tag;
+		else
+			*shadow = size & KASAN_GRANULE_MASK;
+	}
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static bool shadow_mapped(unsigned long addr)
+{
+	pgd_t *pgd = pgd_offset_k(addr);
+	p4d_t *p4d;
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+
+	if (pgd_none(*pgd))
+		return false;
+	p4d = p4d_offset(pgd, addr);
+	if (p4d_none(*p4d))
+		return false;
+	pud = pud_offset(p4d, addr);
+	if (pud_none(*pud))
+		return false;
+
+	/*
+	 * We can't use pud_large() or pud_huge(), the first one is
+	 * arch-specific, the last one depends on HUGETLB_PAGE.  So let's abuse
+	 * pud_bad(), if pud is bad then it's bad because it's huge.
+	 */
+	if (pud_bad(*pud))
+		return true;
+	pmd = pmd_offset(pud, addr);
+	if (pmd_none(*pmd))
+		return false;
+
+	if (pmd_bad(*pmd))
+		return true;
+	pte = pte_offset_kernel(pmd, addr);
+	return !pte_none(*pte);
+}
+
+static int __meminit kasan_mem_notifier(struct notifier_block *nb,
+			unsigned long action, void *data)
+{
+	struct memory_notify *mem_data = data;
+	unsigned long nr_shadow_pages, start_kaddr, shadow_start;
+	unsigned long shadow_end, shadow_size;
+
+	nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT;
+	start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn);
+	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr);
+	shadow_size = nr_shadow_pages << PAGE_SHIFT;
+	shadow_end = shadow_start + shadow_size;
+
+	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
+		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
+		return NOTIFY_BAD;
+
+	switch (action) {
+	case MEM_GOING_ONLINE: {
+		void *ret;
+
+		/*
+		 * If shadow is mapped already than it must have been mapped
+		 * during the boot. This could happen if we onlining previously
+		 * offlined memory.
+		 */
+		if (shadow_mapped(shadow_start))
+			return NOTIFY_OK;
+
+		ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start,
+					shadow_end, GFP_KERNEL,
+					PAGE_KERNEL, VM_NO_GUARD,
+					pfn_to_nid(mem_data->start_pfn),
+					__builtin_return_address(0));
+		if (!ret)
+			return NOTIFY_BAD;
+
+		kmemleak_ignore(ret);
+		return NOTIFY_OK;
+	}
+	case MEM_CANCEL_ONLINE:
+	case MEM_OFFLINE: {
+		struct vm_struct *vm;
+
+		/*
+		 * shadow_start was either mapped during boot by kasan_init()
+		 * or during memory online by __vmalloc_node_range().
+		 * In the latter case we can use vfree() to free shadow.
+		 * Non-NULL result of the find_vm_area() will tell us if
+		 * that was the second case.
+		 *
+		 * Currently it's not possible to free shadow mapped
+		 * during boot by kasan_init(). It's because the code
+		 * to do that hasn't been written yet. So we'll just
+		 * leak the memory.
+		 */
+		vm = find_vm_area((void *)shadow_start);
+		if (vm)
+			vfree((void *)shadow_start);
+	}
+	}
+
+	return NOTIFY_OK;
+}
+
+static int __init kasan_memhotplug_init(void)
+{
+	hotplug_memory_notifier(kasan_mem_notifier, 0);
+
+	return 0;
+}
+
+core_initcall(kasan_memhotplug_init);
+#endif
+
+#ifdef CONFIG_KASAN_VMALLOC
+
+static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+				      void *unused)
+{
+	unsigned long page;
+	pte_t pte;
+
+	if (likely(!pte_none(*ptep)))
+		return 0;
+
+	page = __get_free_page(GFP_KERNEL);
+	if (!page)
+		return -ENOMEM;
+
+	memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
+	pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
+
+	spin_lock(&init_mm.page_table_lock);
+	if (likely(pte_none(*ptep))) {
+		set_pte_at(&init_mm, addr, ptep, pte);
+		page = 0;
+	}
+	spin_unlock(&init_mm.page_table_lock);
+	if (page)
+		free_page(page);
+	return 0;
+}
+
+int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
+{
+	unsigned long shadow_start, shadow_end;
+	int ret;
+
+	if (!is_vmalloc_or_module_addr((void *)addr))
+		return 0;
+
+	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
+	shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
+	shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
+	shadow_end = ALIGN(shadow_end, PAGE_SIZE);
+
+	ret = apply_to_page_range(&init_mm, shadow_start,
+				  shadow_end - shadow_start,
+				  kasan_populate_vmalloc_pte, NULL);
+	if (ret)
+		return ret;
+
+	flush_cache_vmap(shadow_start, shadow_end);
+
+	/*
+	 * We need to be careful about inter-cpu effects here. Consider:
+	 *
+	 *   CPU#0				  CPU#1
+	 * WRITE_ONCE(p, vmalloc(100));		while (x = READ_ONCE(p)) ;
+	 *					p[99] = 1;
+	 *
+	 * With compiler instrumentation, that ends up looking like this:
+	 *
+	 *   CPU#0				  CPU#1
+	 * // vmalloc() allocates memory
+	 * // let a = area->addr
+	 * // we reach kasan_populate_vmalloc
+	 * // and call unpoison_range:
+	 * STORE shadow(a), unpoison_val
+	 * ...
+	 * STORE shadow(a+99), unpoison_val	x = LOAD p
+	 * // rest of vmalloc process		<data dependency>
+	 * STORE p, a				LOAD shadow(x+99)
+	 *
+	 * If there is no barrier between the end of unpoisioning the shadow
+	 * and the store of the result to p, the stores could be committed
+	 * in a different order by CPU#0, and CPU#1 could erroneously observe
+	 * poison in the shadow.
+	 *
+	 * We need some sort of barrier between the stores.
+	 *
+	 * In the vmalloc() case, this is provided by a smp_wmb() in
+	 * clear_vm_uninitialized_flag(). In the per-cpu allocator and in
+	 * get_vm_area() and friends, the caller gets shadow allocated but
+	 * doesn't have any pages mapped into the virtual address space that
+	 * has been reserved. Mapping those pages in will involve taking and
+	 * releasing a page-table lock, which will provide the barrier.
+	 */
+
+	return 0;
+}
+
+/*
+ * Poison the shadow for a vmalloc region. Called as part of the
+ * freeing process at the time the region is freed.
+ */
+void kasan_poison_vmalloc(const void *start, unsigned long size)
+{
+	if (!is_vmalloc_or_module_addr(start))
+		return;
+
+	size = round_up(size, KASAN_GRANULE_SIZE);
+	poison_range(start, size, KASAN_VMALLOC_INVALID);
+}
+
+void kasan_unpoison_vmalloc(const void *start, unsigned long size)
+{
+	if (!is_vmalloc_or_module_addr(start))
+		return;
+
+	unpoison_range(start, size);
+}
+
+static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+					void *unused)
+{
+	unsigned long page;
+
+	page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT);
+
+	spin_lock(&init_mm.page_table_lock);
+
+	if (likely(!pte_none(*ptep))) {
+		pte_clear(&init_mm, addr, ptep);
+		free_page(page);
+	}
+	spin_unlock(&init_mm.page_table_lock);
+
+	return 0;
+}
+
+/*
+ * Release the backing for the vmalloc region [start, end), which
+ * lies within the free region [free_region_start, free_region_end).
+ *
+ * This can be run lazily, long after the region was freed. It runs
+ * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap
+ * infrastructure.
+ *
+ * How does this work?
+ * -------------------
+ *
+ * We have a region that is page aligned, labelled as A.
+ * That might not map onto the shadow in a way that is page-aligned:
+ *
+ *                    start                     end
+ *                    v                         v
+ * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc
+ *  -------- -------- --------          -------- --------
+ *      |        |       |                 |        |
+ *      |        |       |         /-------/        |
+ *      \-------\|/------/         |/---------------/
+ *              |||                ||
+ *             |??AAAAAA|AAAAAAAA|AA??????|                < shadow
+ *                 (1)      (2)      (3)
+ *
+ * First we align the start upwards and the end downwards, so that the
+ * shadow of the region aligns with shadow page boundaries. In the
+ * example, this gives us the shadow page (2). This is the shadow entirely
+ * covered by this allocation.
+ *
+ * Then we have the tricky bits. We want to know if we can free the
+ * partially covered shadow pages - (1) and (3) in the example. For this,
+ * we are given the start and end of the free region that contains this
+ * allocation. Extending our previous example, we could have:
+ *
+ *  free_region_start                                    free_region_end
+ *  |                 start                     end      |
+ *  v                 v                         v        v
+ * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc
+ *  -------- -------- --------          -------- --------
+ *      |        |       |                 |        |
+ *      |        |       |         /-------/        |
+ *      \-------\|/------/         |/---------------/
+ *              |||                ||
+ *             |FFAAAAAA|AAAAAAAA|AAF?????|                < shadow
+ *                 (1)      (2)      (3)
+ *
+ * Once again, we align the start of the free region up, and the end of
+ * the free region down so that the shadow is page aligned. So we can free
+ * page (1) - we know no allocation currently uses anything in that page,
+ * because all of it is in the vmalloc free region. But we cannot free
+ * page (3), because we can't be sure that the rest of it is unused.
+ *
+ * We only consider pages that contain part of the original region for
+ * freeing: we don't try to free other pages from the free region or we'd
+ * end up trying to free huge chunks of virtual address space.
+ *
+ * Concurrency
+ * -----------
+ *
+ * How do we know that we're not freeing a page that is simultaneously
+ * being used for a fresh allocation in kasan_populate_vmalloc(_pte)?
+ *
+ * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running
+ * at the same time. While we run under free_vmap_area_lock, the population
+ * code does not.
+ *
+ * free_vmap_area_lock instead operates to ensure that the larger range
+ * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and
+ * the per-cpu region-finding algorithm both run under free_vmap_area_lock,
+ * no space identified as free will become used while we are running. This
+ * means that so long as we are careful with alignment and only free shadow
+ * pages entirely covered by the free region, we will not run in to any
+ * trouble - any simultaneous allocations will be for disjoint regions.
+ */
+void kasan_release_vmalloc(unsigned long start, unsigned long end,
+			   unsigned long free_region_start,
+			   unsigned long free_region_end)
+{
+	void *shadow_start, *shadow_end;
+	unsigned long region_start, region_end;
+	unsigned long size;
+
+	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
+	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
+
+	free_region_start = ALIGN(free_region_start,
+				  PAGE_SIZE * KASAN_GRANULE_SIZE);
+
+	if (start != region_start &&
+	    free_region_start < region_start)
+		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
+
+	free_region_end = ALIGN_DOWN(free_region_end,
+				     PAGE_SIZE * KASAN_GRANULE_SIZE);
+
+	if (end != region_end &&
+	    free_region_end > region_end)
+		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
+
+	shadow_start = kasan_mem_to_shadow((void *)region_start);
+	shadow_end = kasan_mem_to_shadow((void *)region_end);
+
+	if (shadow_end > shadow_start) {
+		size = shadow_end - shadow_start;
+		apply_to_existing_page_range(&init_mm,
+					     (unsigned long)shadow_start,
+					     size, kasan_depopulate_vmalloc_pte,
+					     NULL);
+		flush_tlb_kernel_range((unsigned long)shadow_start,
+				       (unsigned long)shadow_end);
+	}
+}
+
+#else /* CONFIG_KASAN_VMALLOC */
+
+int kasan_module_alloc(void *addr, size_t size)
+{
+	void *ret;
+	size_t scaled_size;
+	size_t shadow_size;
+	unsigned long shadow_start;
+
+	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
+	scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
+				KASAN_SHADOW_SCALE_SHIFT;
+	shadow_size = round_up(scaled_size, PAGE_SIZE);
+
+	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
+		return -EINVAL;
+
+	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
+			shadow_start + shadow_size,
+			GFP_KERNEL,
+			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
+			__builtin_return_address(0));
+
+	if (ret) {
+		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
+		find_vm_area(addr)->flags |= VM_KASAN;
+		kmemleak_ignore(ret);
+		return 0;
+	}
+
+	return -ENOMEM;
+}
+
+void kasan_free_shadow(const struct vm_struct *vm)
+{
+	if (vm->flags & VM_KASAN)
+		vfree(kasan_mem_to_shadow(vm->addr));
+}
+
+#endif
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 22/78] kasan: define KASAN_MEMORY_PER_SHADOW_PAGE
  2020-12-18 22:00 incoming Andrew Morton
                   ` (20 preceding siblings ...)
  2020-12-18 22:02 ` [patch 21/78] kasan: split out shadow.c from common.c Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 23/78] kasan: rename report and tags files Andrew Morton
                   ` (55 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: define KASAN_MEMORY_PER_SHADOW_PAGE

Define KASAN_MEMORY_PER_SHADOW_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT),
which is the same as (KASAN_GRANULE_SIZE * PAGE_SIZE) for software modes
that use shadow memory, and use it across KASAN code to simplify it.

Link: https://lkml.kernel.org/r/8329391cfe14b5cffd3decf3b5c535b6ce21eef6.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/init.c   |   10 ++++------
 mm/kasan/kasan.h  |    2 ++
 mm/kasan/shadow.c |   16 +++++++---------
 3 files changed, 13 insertions(+), 15 deletions(-)

--- a/mm/kasan/init.c~kasan-define-kasan_memory_per_shadow_page
+++ a/mm/kasan/init.c
@@ -441,9 +441,8 @@ void kasan_remove_zero_shadow(void *star
 	addr = (unsigned long)kasan_mem_to_shadow(start);
 	end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
-	if (WARN_ON((unsigned long)start %
-			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
+	if (WARN_ON((unsigned long)start % KASAN_MEMORY_PER_SHADOW_PAGE) ||
+	    WARN_ON(size % KASAN_MEMORY_PER_SHADOW_PAGE))
 		return;
 
 	for (; addr < end; addr = next) {
@@ -476,9 +475,8 @@ int kasan_add_zero_shadow(void *start, u
 	shadow_start = kasan_mem_to_shadow(start);
 	shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
-	if (WARN_ON((unsigned long)start %
-			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
+	if (WARN_ON((unsigned long)start % KASAN_MEMORY_PER_SHADOW_PAGE) ||
+	    WARN_ON(size % KASAN_MEMORY_PER_SHADOW_PAGE))
 		return -EINVAL;
 
 	ret = kasan_populate_early_shadow(shadow_start, shadow_end);
--- a/mm/kasan/kasan.h~kasan-define-kasan_memory_per_shadow_page
+++ a/mm/kasan/kasan.h
@@ -8,6 +8,8 @@
 #define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
 #define KASAN_GRANULE_MASK	(KASAN_GRANULE_SIZE - 1)
 
+#define KASAN_MEMORY_PER_SHADOW_PAGE	(KASAN_GRANULE_SIZE << PAGE_SHIFT)
+
 #define KASAN_TAG_KERNEL	0xFF /* native kernel pointers tag */
 #define KASAN_TAG_INVALID	0xFE /* inaccessible memory tag */
 #define KASAN_TAG_MAX		0xFD /* maximum value for random tags */
--- a/mm/kasan/shadow.c~kasan-define-kasan_memory_per_shadow_page
+++ a/mm/kasan/shadow.c
@@ -174,7 +174,7 @@ static int __meminit kasan_mem_notifier(
 	shadow_end = shadow_start + shadow_size;
 
 	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
-		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
+		WARN_ON(start_kaddr % KASAN_MEMORY_PER_SHADOW_PAGE))
 		return NOTIFY_BAD;
 
 	switch (action) {
@@ -445,22 +445,20 @@ void kasan_release_vmalloc(unsigned long
 	unsigned long region_start, region_end;
 	unsigned long size;
 
-	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
-	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
+	region_start = ALIGN(start, KASAN_MEMORY_PER_SHADOW_PAGE);
+	region_end = ALIGN_DOWN(end, KASAN_MEMORY_PER_SHADOW_PAGE);
 
-	free_region_start = ALIGN(free_region_start,
-				  PAGE_SIZE * KASAN_GRANULE_SIZE);
+	free_region_start = ALIGN(free_region_start, KASAN_MEMORY_PER_SHADOW_PAGE);
 
 	if (start != region_start &&
 	    free_region_start < region_start)
-		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
+		region_start -= KASAN_MEMORY_PER_SHADOW_PAGE;
 
-	free_region_end = ALIGN_DOWN(free_region_end,
-				     PAGE_SIZE * KASAN_GRANULE_SIZE);
+	free_region_end = ALIGN_DOWN(free_region_end, KASAN_MEMORY_PER_SHADOW_PAGE);
 
 	if (end != region_end &&
 	    free_region_end > region_end)
-		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
+		region_end += KASAN_MEMORY_PER_SHADOW_PAGE;
 
 	shadow_start = kasan_mem_to_shadow((void *)region_start);
 	shadow_end = kasan_mem_to_shadow((void *)region_end);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 23/78] kasan: rename report and tags files
  2020-12-18 22:00 incoming Andrew Morton
                   ` (21 preceding siblings ...)
  2020-12-18 22:02 ` [patch 22/78] kasan: define KASAN_MEMORY_PER_SHADOW_PAGE Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 24/78] kasan: don't duplicate config dependencies Andrew Morton
                   ` (54 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename report and tags files

Rename generic_report.c to report_generic.c and tags_report.c to
report_sw_tags.c, as their content is more relevant to report.c file. 
Also rename tags.c to sw_tags.c to better reflect that this file contains
code for software tag-based mode.

No functional changes.

Link: https://lkml.kernel.org/r/a6105d416da97d389580015afed66c4c3cfd4c08.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/Makefile         |   16 +-
 mm/kasan/generic_report.c |  160 -----------------------------
 mm/kasan/report.c         |    2 
 mm/kasan/report_generic.c |  160 +++++++++++++++++++++++++++++
 mm/kasan/report_sw_tags.c |   88 ++++++++++++++++
 mm/kasan/sw_tags.c        |  195 ++++++++++++++++++++++++++++++++++++
 mm/kasan/tags.c           |  195 ------------------------------------
 mm/kasan/tags_report.c    |   88 ----------------
 8 files changed, 452 insertions(+), 452 deletions(-)

--- a/mm/kasan/Makefile~kasan-rename-report-and-tags-files
+++ a/mm/kasan/Makefile
@@ -6,13 +6,13 @@ KCOV_INSTRUMENT := n
 # Disable ftrace to avoid recursion.
 CFLAGS_REMOVE_common.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_generic.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_generic_report.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE)
 
 # Function splitter causes unnecessary splits in __asan_load1/__asan_store1
 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533
@@ -23,14 +23,14 @@ CC_FLAGS_KASAN_RUNTIME += -DDISABLE_BRAN
 
 CFLAGS_common.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
-CFLAGS_generic_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
-CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
-CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 
 obj-$(CONFIG_KASAN) := common.o report.o
-obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o
-obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o
+obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o
+obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o
--- a/mm/kasan/report.c~kasan-rename-report-and-tags-files
+++ a/mm/kasan/report.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains common generic and tag-based KASAN error reporting code.
+ * This file contains common KASAN error reporting code.
  *
  * Copyright (c) 2014 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
diff --git a/mm/kasan/generic_report.c b/mm/kasan/report_generic.c
similarity index 100%
rename from mm/kasan/generic_report.c
rename to mm/kasan/report_generic.c
diff --git a/mm/kasan/tags_report.c b/mm/kasan/report_sw_tags.c
similarity index 100%
rename from mm/kasan/tags_report.c
rename to mm/kasan/report_sw_tags.c
diff --git a/mm/kasan/tags.c b/mm/kasan/sw_tags.c
similarity index 100%
rename from mm/kasan/tags.c
rename to mm/kasan/sw_tags.c

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 24/78] kasan: don't duplicate config dependencies
  2020-12-18 22:00 incoming Andrew Morton
                   ` (22 preceding siblings ...)
  2020-12-18 22:02 ` [patch 23/78] kasan: rename report and tags files Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 25/78] kasan: hide invalid free check implementation Andrew Morton
                   ` (53 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: don't duplicate config dependencies

Both KASAN_GENERIC and KASAN_SW_TAGS have common dependencies, move those
to KASAN.

Link: https://lkml.kernel.org/r/c1cc0d562608a318c607afe22db5ec2a7af72e47.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 lib/Kconfig.kasan |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

--- a/lib/Kconfig.kasan~kasan-dont-duplicate-config-dependencies
+++ a/lib/Kconfig.kasan
@@ -24,6 +24,8 @@ menuconfig KASAN
 		   (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)
 	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
 	depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS
+	select CONSTRUCTORS
+	select STACKDEPOT
 	help
 	  Enables KASAN (KernelAddressSANitizer) - runtime memory debugger,
 	  designed to find out-of-bounds accesses and use-after-free bugs.
@@ -46,10 +48,7 @@ choice
 config KASAN_GENERIC
 	bool "Generic mode"
 	depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC
-	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
 	select SLUB_DEBUG if SLUB
-	select CONSTRUCTORS
-	select STACKDEPOT
 	help
 	  Enables generic KASAN mode.
 
@@ -70,10 +69,7 @@ config KASAN_GENERIC
 config KASAN_SW_TAGS
 	bool "Software tag-based mode"
 	depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS
-	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
 	select SLUB_DEBUG if SLUB
-	select CONSTRUCTORS
-	select STACKDEPOT
 	help
 	  Enables software tag-based KASAN mode.
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 25/78] kasan: hide invalid free check implementation
  2020-12-18 22:00 incoming Andrew Morton
                   ` (23 preceding siblings ...)
  2020-12-18 22:02 ` [patch 24/78] kasan: don't duplicate config dependencies Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 26/78] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrew Morton
                   ` (52 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: hide invalid free check implementation

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

For software KASAN modes the check is based on the value in the shadow
memory.  Hardware tag-based KASAN won't be using shadow, so hide the
implementation of the check in check_invalid_free().

Also simplify the code for software tag-based mode.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/d01534a4b977f97d87515dc590e6348e1406de81.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c  |   19 +------------------
 mm/kasan/generic.c |    7 +++++++
 mm/kasan/kasan.h   |    2 ++
 mm/kasan/sw_tags.c |    9 +++++++++
 4 files changed, 19 insertions(+), 18 deletions(-)

--- a/mm/kasan/common.c~kasan-hide-invalid-free-check-implementation
+++ a/mm/kasan/common.c
@@ -277,25 +277,9 @@ void * __must_check kasan_init_slab_obj(
 	return (void *)object;
 }
 
-static inline bool shadow_invalid(u8 tag, s8 shadow_byte)
-{
-	if (IS_ENABLED(CONFIG_KASAN_GENERIC))
-		return shadow_byte < 0 ||
-			shadow_byte >= KASAN_GRANULE_SIZE;
-
-	/* else CONFIG_KASAN_SW_TAGS: */
-	if ((u8)shadow_byte == KASAN_TAG_INVALID)
-		return true;
-	if ((tag != KASAN_TAG_KERNEL) && (tag != (u8)shadow_byte))
-		return true;
-
-	return false;
-}
-
 static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
 			      unsigned long ip, bool quarantine)
 {
-	s8 shadow_byte;
 	u8 tag;
 	void *tagged_object;
 	unsigned long rounded_up_size;
@@ -314,8 +298,7 @@ static bool __kasan_slab_free(struct kme
 	if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU))
 		return false;
 
-	shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object));
-	if (shadow_invalid(tag, shadow_byte)) {
+	if (check_invalid_free(tagged_object)) {
 		kasan_report_invalid_free(tagged_object, ip);
 		return true;
 	}
--- a/mm/kasan/generic.c~kasan-hide-invalid-free-check-implementation
+++ a/mm/kasan/generic.c
@@ -187,6 +187,13 @@ bool check_memory_region(unsigned long a
 	return check_memory_region_inline(addr, size, write, ret_ip);
 }
 
+bool check_invalid_free(void *addr)
+{
+	s8 shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr));
+
+	return shadow_byte < 0 || shadow_byte >= KASAN_GRANULE_SIZE;
+}
+
 void kasan_cache_shrink(struct kmem_cache *cache)
 {
 	quarantine_remove_cache(cache);
--- a/mm/kasan/kasan.h~kasan-hide-invalid-free-check-implementation
+++ a/mm/kasan/kasan.h
@@ -166,6 +166,8 @@ void unpoison_range(const void *address,
 bool check_memory_region(unsigned long addr, size_t size, bool write,
 				unsigned long ret_ip);
 
+bool check_invalid_free(void *addr);
+
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
 
--- a/mm/kasan/sw_tags.c~kasan-hide-invalid-free-check-implementation
+++ a/mm/kasan/sw_tags.c
@@ -121,6 +121,15 @@ bool check_memory_region(unsigned long a
 	return true;
 }
 
+bool check_invalid_free(void *addr)
+{
+	u8 tag = get_tag(addr);
+	u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr)));
+
+	return (shadow_byte == KASAN_TAG_INVALID) ||
+		(tag != KASAN_TAG_KERNEL && tag != shadow_byte);
+}
+
 #define DEFINE_HWASAN_LOAD_STORE(size)					\
 	void __hwasan_load##size##_noabort(unsigned long addr)		\
 	{								\
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 26/78] kasan: decode stack frame only with KASAN_STACK_ENABLE
  2020-12-18 22:00 incoming Andrew Morton
                   ` (24 preceding siblings ...)
  2020-12-18 22:02 ` [patch 25/78] kasan: hide invalid free check implementation Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 27/78] kasan, arm64: only init shadow for software modes Andrew Morton
                   ` (51 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: decode stack frame only with KASAN_STACK_ENABLE

Decoding routines aren't needed when CONFIG_KASAN_STACK_ENABLE is not
enabled.  Currently only generic KASAN mode implements stack error
reporting.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/05a24db36f5ec876af876a299bbea98c29468ebd.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/kasan.h          |    6 +
 mm/kasan/report.c         |  162 ------------------------------------
 mm/kasan/report_generic.c |  162 ++++++++++++++++++++++++++++++++++++
 3 files changed, 168 insertions(+), 162 deletions(-)

--- a/mm/kasan/kasan.h~kasan-decode-stack-frame-only-with-kasan_stack_enable
+++ a/mm/kasan/kasan.h
@@ -171,6 +171,12 @@ bool check_invalid_free(void *addr);
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
 
+#if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK
+void print_address_stack_frame(const void *addr);
+#else
+static inline void print_address_stack_frame(const void *addr) { }
+#endif
+
 bool kasan_report(unsigned long addr, size_t size,
 		bool is_write, unsigned long ip);
 void kasan_report_invalid_free(void *object, unsigned long ip);
--- a/mm/kasan/report.c~kasan-decode-stack-frame-only-with-kasan_stack_enable
+++ a/mm/kasan/report.c
@@ -211,168 +211,6 @@ static inline bool init_task_stack_addr(
 			sizeof(init_thread_union.stack));
 }
 
-static bool __must_check tokenize_frame_descr(const char **frame_descr,
-					      char *token, size_t max_tok_len,
-					      unsigned long *value)
-{
-	const char *sep = strchr(*frame_descr, ' ');
-
-	if (sep == NULL)
-		sep = *frame_descr + strlen(*frame_descr);
-
-	if (token != NULL) {
-		const size_t tok_len = sep - *frame_descr;
-
-		if (tok_len + 1 > max_tok_len) {
-			pr_err("KASAN internal error: frame description too long: %s\n",
-			       *frame_descr);
-			return false;
-		}
-
-		/* Copy token (+ 1 byte for '\0'). */
-		strlcpy(token, *frame_descr, tok_len + 1);
-	}
-
-	/* Advance frame_descr past separator. */
-	*frame_descr = sep + 1;
-
-	if (value != NULL && kstrtoul(token, 10, value)) {
-		pr_err("KASAN internal error: not a valid number: %s\n", token);
-		return false;
-	}
-
-	return true;
-}
-
-static void print_decoded_frame_descr(const char *frame_descr)
-{
-	/*
-	 * We need to parse the following string:
-	 *    "n alloc_1 alloc_2 ... alloc_n"
-	 * where alloc_i looks like
-	 *    "offset size len name"
-	 * or "offset size len name:line".
-	 */
-
-	char token[64];
-	unsigned long num_objects;
-
-	if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-				  &num_objects))
-		return;
-
-	pr_err("\n");
-	pr_err("this frame has %lu %s:\n", num_objects,
-	       num_objects == 1 ? "object" : "objects");
-
-	while (num_objects--) {
-		unsigned long offset;
-		unsigned long size;
-
-		/* access offset */
-		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-					  &offset))
-			return;
-		/* access size */
-		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-					  &size))
-			return;
-		/* name length (unused) */
-		if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL))
-			return;
-		/* object name */
-		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-					  NULL))
-			return;
-
-		/* Strip line number; without filename it's not very helpful. */
-		strreplace(token, ':', '\0');
-
-		/* Finally, print object information. */
-		pr_err(" [%lu, %lu) '%s'", offset, offset + size, token);
-	}
-}
-
-static bool __must_check get_address_stack_frame_info(const void *addr,
-						      unsigned long *offset,
-						      const char **frame_descr,
-						      const void **frame_pc)
-{
-	unsigned long aligned_addr;
-	unsigned long mem_ptr;
-	const u8 *shadow_bottom;
-	const u8 *shadow_ptr;
-	const unsigned long *frame;
-
-	BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP));
-
-	/*
-	 * NOTE: We currently only support printing frame information for
-	 * accesses to the task's own stack.
-	 */
-	if (!object_is_on_stack(addr))
-		return false;
-
-	aligned_addr = round_down((unsigned long)addr, sizeof(long));
-	mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
-	shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
-	shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
-
-	while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
-		shadow_ptr--;
-		mem_ptr -= KASAN_GRANULE_SIZE;
-	}
-
-	while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
-		shadow_ptr--;
-		mem_ptr -= KASAN_GRANULE_SIZE;
-	}
-
-	if (shadow_ptr < shadow_bottom)
-		return false;
-
-	frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
-	if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
-		pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
-		       frame[0]);
-		return false;
-	}
-
-	*offset = (unsigned long)addr - (unsigned long)frame;
-	*frame_descr = (const char *)frame[1];
-	*frame_pc = (void *)frame[2];
-
-	return true;
-}
-
-static void print_address_stack_frame(const void *addr)
-{
-	unsigned long offset;
-	const char *frame_descr;
-	const void *frame_pc;
-
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
-		return;
-
-	if (!get_address_stack_frame_info(addr, &offset, &frame_descr,
-					  &frame_pc))
-		return;
-
-	/*
-	 * get_address_stack_frame_info only returns true if the given addr is
-	 * on the current task's stack.
-	 */
-	pr_err("\n");
-	pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n",
-	       addr, current->comm, task_pid_nr(current), offset);
-	pr_err(" %pS\n", frame_pc);
-
-	if (!frame_descr)
-		return;
-
-	print_decoded_frame_descr(frame_descr);
-}
-
 static void print_address_description(void *addr, u8 tag)
 {
 	struct page *page = kasan_addr_to_page(addr);
--- a/mm/kasan/report_generic.c~kasan-decode-stack-frame-only-with-kasan_stack_enable
+++ a/mm/kasan/report_generic.c
@@ -16,6 +16,7 @@
 #include <linux/mm.h>
 #include <linux/printk.h>
 #include <linux/sched.h>
+#include <linux/sched/task_stack.h>
 #include <linux/slab.h>
 #include <linux/stackdepot.h>
 #include <linux/stacktrace.h>
@@ -122,6 +123,167 @@ const char *get_bug_type(struct kasan_ac
 	return get_wild_bug_type(info);
 }
 
+#if CONFIG_KASAN_STACK
+static bool __must_check tokenize_frame_descr(const char **frame_descr,
+					      char *token, size_t max_tok_len,
+					      unsigned long *value)
+{
+	const char *sep = strchr(*frame_descr, ' ');
+
+	if (sep == NULL)
+		sep = *frame_descr + strlen(*frame_descr);
+
+	if (token != NULL) {
+		const size_t tok_len = sep - *frame_descr;
+
+		if (tok_len + 1 > max_tok_len) {
+			pr_err("KASAN internal error: frame description too long: %s\n",
+			       *frame_descr);
+			return false;
+		}
+
+		/* Copy token (+ 1 byte for '\0'). */
+		strlcpy(token, *frame_descr, tok_len + 1);
+	}
+
+	/* Advance frame_descr past separator. */
+	*frame_descr = sep + 1;
+
+	if (value != NULL && kstrtoul(token, 10, value)) {
+		pr_err("KASAN internal error: not a valid number: %s\n", token);
+		return false;
+	}
+
+	return true;
+}
+
+static void print_decoded_frame_descr(const char *frame_descr)
+{
+	/*
+	 * We need to parse the following string:
+	 *    "n alloc_1 alloc_2 ... alloc_n"
+	 * where alloc_i looks like
+	 *    "offset size len name"
+	 * or "offset size len name:line".
+	 */
+
+	char token[64];
+	unsigned long num_objects;
+
+	if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+				  &num_objects))
+		return;
+
+	pr_err("\n");
+	pr_err("this frame has %lu %s:\n", num_objects,
+	       num_objects == 1 ? "object" : "objects");
+
+	while (num_objects--) {
+		unsigned long offset;
+		unsigned long size;
+
+		/* access offset */
+		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+					  &offset))
+			return;
+		/* access size */
+		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+					  &size))
+			return;
+		/* name length (unused) */
+		if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL))
+			return;
+		/* object name */
+		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+					  NULL))
+			return;
+
+		/* Strip line number; without filename it's not very helpful. */
+		strreplace(token, ':', '\0');
+
+		/* Finally, print object information. */
+		pr_err(" [%lu, %lu) '%s'", offset, offset + size, token);
+	}
+}
+
+static bool __must_check get_address_stack_frame_info(const void *addr,
+						      unsigned long *offset,
+						      const char **frame_descr,
+						      const void **frame_pc)
+{
+	unsigned long aligned_addr;
+	unsigned long mem_ptr;
+	const u8 *shadow_bottom;
+	const u8 *shadow_ptr;
+	const unsigned long *frame;
+
+	BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP));
+
+	/*
+	 * NOTE: We currently only support printing frame information for
+	 * accesses to the task's own stack.
+	 */
+	if (!object_is_on_stack(addr))
+		return false;
+
+	aligned_addr = round_down((unsigned long)addr, sizeof(long));
+	mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
+	shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
+	shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
+
+	while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
+		shadow_ptr--;
+		mem_ptr -= KASAN_GRANULE_SIZE;
+	}
+
+	while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
+		shadow_ptr--;
+		mem_ptr -= KASAN_GRANULE_SIZE;
+	}
+
+	if (shadow_ptr < shadow_bottom)
+		return false;
+
+	frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
+	if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
+		pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
+		       frame[0]);
+		return false;
+	}
+
+	*offset = (unsigned long)addr - (unsigned long)frame;
+	*frame_descr = (const char *)frame[1];
+	*frame_pc = (void *)frame[2];
+
+	return true;
+}
+
+void print_address_stack_frame(const void *addr)
+{
+	unsigned long offset;
+	const char *frame_descr;
+	const void *frame_pc;
+
+	if (!get_address_stack_frame_info(addr, &offset, &frame_descr,
+					  &frame_pc))
+		return;
+
+	/*
+	 * get_address_stack_frame_info only returns true if the given addr is
+	 * on the current task's stack.
+	 */
+	pr_err("\n");
+	pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n",
+	       addr, current->comm, task_pid_nr(current), offset);
+	pr_err(" %pS\n", frame_pc);
+
+	if (!frame_descr)
+		return;
+
+	print_decoded_frame_descr(frame_descr);
+}
+#endif /* CONFIG_KASAN_STACK */
+
 #define DEFINE_ASAN_REPORT_LOAD(size)                     \
 void __asan_report_load##size##_noabort(unsigned long addr) \
 {                                                         \
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 27/78] kasan, arm64: only init shadow for software modes
  2020-12-18 22:00 incoming Andrew Morton
                   ` (25 preceding siblings ...)
  2020-12-18 22:02 ` [patch 26/78] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:02 ` [patch 28/78] kasan, arm64: only use kasan_depth " Andrew Morton
                   ` (50 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: only init shadow for software modes

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory.  Only initialize it
when one of the software KASAN modes are enabled.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/d1742eea2cd728d150d49b144e49b6433405c7ba.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/kasan.h |    8 ++++++--
 arch/arm64/mm/kasan_init.c     |   15 ++++++++++++++-
 2 files changed, 20 insertions(+), 3 deletions(-)

--- a/arch/arm64/include/asm/kasan.h~kasan-arm64-only-init-shadow-for-software-modes
+++ a/arch/arm64/include/asm/kasan.h
@@ -13,6 +13,12 @@
 #define arch_kasan_get_tag(addr)	__tag_get(addr)
 
 #ifdef CONFIG_KASAN
+void kasan_init(void);
+#else
+static inline void kasan_init(void) { }
+#endif
+
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 
 /*
  * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
@@ -33,12 +39,10 @@
 #define _KASAN_SHADOW_START(va)	(KASAN_SHADOW_END - (1UL << ((va) - KASAN_SHADOW_SCALE_SHIFT)))
 #define KASAN_SHADOW_START      _KASAN_SHADOW_START(vabits_actual)
 
-void kasan_init(void);
 void kasan_copy_shadow(pgd_t *pgdir);
 asmlinkage void kasan_early_init(void);
 
 #else
-static inline void kasan_init(void) { }
 static inline void kasan_copy_shadow(pgd_t *pgdir) { }
 #endif
 
--- a/arch/arm64/mm/kasan_init.c~kasan-arm64-only-init-shadow-for-software-modes
+++ a/arch/arm64/mm/kasan_init.c
@@ -21,6 +21,8 @@
 #include <asm/sections.h>
 #include <asm/tlbflush.h>
 
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
+
 static pgd_t tmp_pg_dir[PTRS_PER_PGD] __initdata __aligned(PGD_SIZE);
 
 /*
@@ -208,7 +210,7 @@ static void __init clear_pgds(unsigned l
 		set_pgd(pgd_offset_k(start), __pgd(0));
 }
 
-void __init kasan_init(void)
+static void __init kasan_init_shadow(void)
 {
 	u64 kimg_shadow_start, kimg_shadow_end;
 	u64 mod_shadow_start, mod_shadow_end;
@@ -269,6 +271,17 @@ void __init kasan_init(void)
 
 	memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE);
 	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
+}
+
+#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
+
+static inline void __init kasan_init_shadow(void) { }
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+void __init kasan_init(void)
+{
+	kasan_init_shadow();
 
 	/* At this point kasan is fully initialized. Enable error messages */
 	init_task.kasan_depth = 0;
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 28/78] kasan, arm64: only use kasan_depth for software modes
  2020-12-18 22:00 incoming Andrew Morton
                   ` (26 preceding siblings ...)
  2020-12-18 22:02 ` [patch 27/78] kasan, arm64: only init shadow for software modes Andrew Morton
@ 2020-12-18 22:02 ` Andrew Morton
  2020-12-18 22:03 ` [patch 29/78] kasan, arm64: move initialization message Andrew Morton
                   ` (49 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:02 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: only use kasan_depth for software modes

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't use kasan_depth.  Only define and use it
when one of the software KASAN modes are enabled.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/e16f15aeda90bc7fb4dfc2e243a14b74cc5c8219.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/mm/kasan_init.c |   11 ++++++++---
 include/linux/kasan.h      |   18 +++++++++---------
 include/linux/sched.h      |    2 +-
 init/init_task.c           |    2 +-
 mm/kasan/common.c          |    2 ++
 mm/kasan/report.c          |    2 ++
 6 files changed, 23 insertions(+), 14 deletions(-)

--- a/arch/arm64/mm/kasan_init.c~kasan-arm64-only-use-kasan_depth-for-software-modes
+++ a/arch/arm64/mm/kasan_init.c
@@ -273,17 +273,22 @@ static void __init kasan_init_shadow(voi
 	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
 }
 
+static void __init kasan_init_depth(void)
+{
+	init_task.kasan_depth = 0;
+}
+
 #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
 
 static inline void __init kasan_init_shadow(void) { }
 
+static inline void __init kasan_init_depth(void) { }
+
 #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 void __init kasan_init(void)
 {
 	kasan_init_shadow();
-
-	/* At this point kasan is fully initialized. Enable error messages */
-	init_task.kasan_depth = 0;
+	kasan_init_depth();
 	pr_info("KernelAddressSanitizer initialized\n");
 }
--- a/include/linux/kasan.h~kasan-arm64-only-use-kasan_depth-for-software-modes
+++ a/include/linux/kasan.h
@@ -52,6 +52,12 @@ static inline void *kasan_mem_to_shadow(
 int kasan_add_zero_shadow(void *start, unsigned long size);
 void kasan_remove_zero_shadow(void *start, unsigned long size);
 
+/* Enable reporting bugs after kasan_disable_current() */
+extern void kasan_enable_current(void);
+
+/* Disable reporting bugs for current task */
+extern void kasan_disable_current(void);
+
 #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 static inline int kasan_add_zero_shadow(void *start, unsigned long size)
@@ -62,16 +68,13 @@ static inline void kasan_remove_zero_sha
 					unsigned long size)
 {}
 
+static inline void kasan_enable_current(void) {}
+static inline void kasan_disable_current(void) {}
+
 #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 #ifdef CONFIG_KASAN
 
-/* Enable reporting bugs after kasan_disable_current() */
-extern void kasan_enable_current(void);
-
-/* Disable reporting bugs for current task */
-extern void kasan_disable_current(void);
-
 void kasan_unpoison_range(const void *address, size_t size);
 
 void kasan_unpoison_task_stack(struct task_struct *task);
@@ -122,9 +125,6 @@ static inline void kasan_unpoison_range(
 
 static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
 
-static inline void kasan_enable_current(void) {}
-static inline void kasan_disable_current(void) {}
-
 static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
 static inline void kasan_free_pages(struct page *page, unsigned int order) {}
 
--- a/include/linux/sched.h~kasan-arm64-only-use-kasan_depth-for-software-modes
+++ a/include/linux/sched.h
@@ -1234,7 +1234,7 @@ struct task_struct {
 	u64				timer_slack_ns;
 	u64				default_timer_slack_ns;
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	unsigned int			kasan_depth;
 #endif
 
--- a/init/init_task.c~kasan-arm64-only-use-kasan_depth-for-software-modes
+++ a/init/init_task.c
@@ -176,7 +176,7 @@ struct task_struct init_task
 	.numa_group	= NULL,
 	.numa_faults	= NULL,
 #endif
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	.kasan_depth	= 1,
 #endif
 #ifdef CONFIG_KCSAN
--- a/mm/kasan/common.c~kasan-arm64-only-use-kasan_depth-for-software-modes
+++ a/mm/kasan/common.c
@@ -46,6 +46,7 @@ void kasan_set_track(struct kasan_track
 	track->stack = kasan_save_stack(flags);
 }
 
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 void kasan_enable_current(void)
 {
 	current->kasan_depth++;
@@ -55,6 +56,7 @@ void kasan_disable_current(void)
 {
 	current->kasan_depth--;
 }
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 void kasan_unpoison_range(const void *address, size_t size)
 {
--- a/mm/kasan/report.c~kasan-arm64-only-use-kasan_depth-for-software-modes
+++ a/mm/kasan/report.c
@@ -292,8 +292,10 @@ static void print_shadow_for_address(con
 
 static bool report_enabled(void)
 {
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	if (current->kasan_depth)
 		return false;
+#endif
 	if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags))
 		return true;
 	return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 29/78] kasan, arm64: move initialization message
  2020-12-18 22:00 incoming Andrew Morton
                   ` (27 preceding siblings ...)
  2020-12-18 22:02 ` [patch 28/78] kasan, arm64: only use kasan_depth " Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 30/78] kasan, arm64: rename kasan_init_tags and mark as __init Andrew Morton
                   ` (48 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: move initialization message

Software tag-based KASAN mode is fully initialized with kasan_init_tags(),
while the generic mode only requires kasan_init().  Move the
initialization message for tag-based mode into kasan_init_tags().

Also fix pr_fmt() usage for KASAN code: generic.c doesn't need it as it
doesn't use any printing functions; tag-based mode should use "kasan:"
instead of KBUILD_MODNAME (which stands for file name).

Link: https://lkml.kernel.org/r/29a30ea4e1750450dd1f693d25b7b6cb05913ecf.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/kasan.h |    9 +++------
 arch/arm64/mm/kasan_init.c     |   13 +++++--------
 mm/kasan/generic.c             |    2 --
 mm/kasan/sw_tags.c             |    4 +++-
 4 files changed, 11 insertions(+), 17 deletions(-)

--- a/arch/arm64/include/asm/kasan.h~kasan-arm64-move-initialization-message
+++ a/arch/arm64/include/asm/kasan.h
@@ -12,14 +12,10 @@
 #define arch_kasan_reset_tag(addr)	__tag_reset(addr)
 #define arch_kasan_get_tag(addr)	__tag_get(addr)
 
-#ifdef CONFIG_KASAN
-void kasan_init(void);
-#else
-static inline void kasan_init(void) { }
-#endif
-
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 
+void kasan_init(void);
+
 /*
  * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
  * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses,
@@ -43,6 +39,7 @@ void kasan_copy_shadow(pgd_t *pgdir);
 asmlinkage void kasan_early_init(void);
 
 #else
+static inline void kasan_init(void) { }
 static inline void kasan_copy_shadow(pgd_t *pgdir) { }
 #endif
 
--- a/arch/arm64/mm/kasan_init.c~kasan-arm64-move-initialization-message
+++ a/arch/arm64/mm/kasan_init.c
@@ -278,17 +278,14 @@ static void __init kasan_init_depth(void
 	init_task.kasan_depth = 0;
 }
 
-#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
-
-static inline void __init kasan_init_shadow(void) { }
-
-static inline void __init kasan_init_depth(void) { }
-
-#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
-
 void __init kasan_init(void)
 {
 	kasan_init_shadow();
 	kasan_init_depth();
+#if defined(CONFIG_KASAN_GENERIC)
+	/* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
 	pr_info("KernelAddressSanitizer initialized\n");
+#endif
 }
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
--- a/mm/kasan/generic.c~kasan-arm64-move-initialization-message
+++ a/mm/kasan/generic.c
@@ -9,8 +9,6 @@
  *        Andrey Konovalov <andreyknvl@gmail.com>
  */
 
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/init.h>
--- a/mm/kasan/sw_tags.c~kasan-arm64-move-initialization-message
+++ a/mm/kasan/sw_tags.c
@@ -6,7 +6,7 @@
  * Author: Andrey Konovalov <andreyknvl@google.com>
  */
 
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#define pr_fmt(fmt) "kasan: " fmt
 
 #include <linux/export.h>
 #include <linux/interrupt.h>
@@ -41,6 +41,8 @@ void kasan_init_tags(void)
 
 	for_each_possible_cpu(cpu)
 		per_cpu(prng_state, cpu) = (u32)get_cycles();
+
+	pr_info("KernelAddressSanitizer initialized\n");
 }
 
 /*
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 30/78] kasan, arm64: rename kasan_init_tags and mark as __init
  2020-12-18 22:00 incoming Andrew Morton
                   ` (28 preceding siblings ...)
  2020-12-18 22:03 ` [patch 29/78] kasan, arm64: move initialization message Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 31/78] kasan: rename addr_has_shadow to addr_has_metadata Andrew Morton
                   ` (47 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: rename kasan_init_tags and mark as __init

Rename kasan_init_tags() to kasan_init_sw_tags() as the upcoming hardware
tag-based KASAN mode will have its own initialization routine.  Also
similarly to kasan_init() mark kasan_init_tags() as __init.

Link: https://lkml.kernel.org/r/71e52af72a09f4b50c8042f16101c60e50649fbb.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/kernel/setup.c  |    2 +-
 arch/arm64/mm/kasan_init.c |    2 +-
 include/linux/kasan.h      |    4 ++--
 mm/kasan/sw_tags.c         |    2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

--- a/arch/arm64/kernel/setup.c~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init
+++ a/arch/arm64/kernel/setup.c
@@ -358,7 +358,7 @@ void __init __no_sanitize_address setup_
 	smp_build_mpidr_hash();
 
 	/* Init percpu seeds for random tags after cpus are set up. */
-	kasan_init_tags();
+	kasan_init_sw_tags();
 
 #ifdef CONFIG_ARM64_SW_TTBR0_PAN
 	/*
--- a/arch/arm64/mm/kasan_init.c~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init
+++ a/arch/arm64/mm/kasan_init.c
@@ -283,7 +283,7 @@ void __init kasan_init(void)
 	kasan_init_shadow();
 	kasan_init_depth();
 #if defined(CONFIG_KASAN_GENERIC)
-	/* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
+	/* CONFIG_KASAN_SW_TAGS also requires kasan_init_sw_tags(). */
 	pr_info("KernelAddressSanitizer initialized\n");
 #endif
 }
--- a/include/linux/kasan.h~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init
+++ a/include/linux/kasan.h
@@ -192,7 +192,7 @@ static inline void kasan_record_aux_stac
 
 #ifdef CONFIG_KASAN_SW_TAGS
 
-void kasan_init_tags(void);
+void __init kasan_init_sw_tags(void);
 
 void *kasan_reset_tag(const void *addr);
 
@@ -201,7 +201,7 @@ bool kasan_report(unsigned long addr, si
 
 #else /* CONFIG_KASAN_SW_TAGS */
 
-static inline void kasan_init_tags(void) { }
+static inline void kasan_init_sw_tags(void) { }
 
 static inline void *kasan_reset_tag(const void *addr)
 {
--- a/mm/kasan/sw_tags.c~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init
+++ a/mm/kasan/sw_tags.c
@@ -35,7 +35,7 @@
 
 static DEFINE_PER_CPU(u32, prng_state);
 
-void kasan_init_tags(void)
+void __init kasan_init_sw_tags(void)
 {
 	int cpu;
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 31/78] kasan: rename addr_has_shadow to addr_has_metadata
  2020-12-18 22:00 incoming Andrew Morton
                   ` (29 preceding siblings ...)
  2020-12-18 22:03 ` [patch 30/78] kasan, arm64: rename kasan_init_tags and mark as __init Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 32/78] kasan: rename print_shadow_for_address to print_memory_metadata Andrew Morton
                   ` (46 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename addr_has_shadow to addr_has_metadata

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory, but will reuse this
function.  Rename "shadow" to implementation-neutral "metadata".

No functional changes.

Link: https://lkml.kernel.org/r/370466fba590a4596b55ffd38adfd990f8886db4.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/kasan.h          |    2 +-
 mm/kasan/report.c         |    6 +++---
 mm/kasan/report_generic.c |    2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

--- a/mm/kasan/kasan.h~kasan-rename-addr_has_shadow-to-addr_has_metadata
+++ a/mm/kasan/kasan.h
@@ -147,7 +147,7 @@ static inline const void *kasan_shadow_t
 		<< KASAN_SHADOW_SCALE_SHIFT);
 }
 
-static inline bool addr_has_shadow(const void *addr)
+static inline bool addr_has_metadata(const void *addr)
 {
 	return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
 }
--- a/mm/kasan/report.c~kasan-rename-addr_has_shadow-to-addr_has_metadata
+++ a/mm/kasan/report.c
@@ -361,7 +361,7 @@ static void __kasan_report(unsigned long
 	untagged_addr = reset_tag(tagged_addr);
 
 	info.access_addr = tagged_addr;
-	if (addr_has_shadow(untagged_addr))
+	if (addr_has_metadata(untagged_addr))
 		info.first_bad_addr = find_first_bad_addr(tagged_addr, size);
 	else
 		info.first_bad_addr = untagged_addr;
@@ -372,11 +372,11 @@ static void __kasan_report(unsigned long
 	start_report(&flags);
 
 	print_error_description(&info);
-	if (addr_has_shadow(untagged_addr))
+	if (addr_has_metadata(untagged_addr))
 		print_tags(get_tag(tagged_addr), info.first_bad_addr);
 	pr_err("\n");
 
-	if (addr_has_shadow(untagged_addr)) {
+	if (addr_has_metadata(untagged_addr)) {
 		print_address_description(untagged_addr, get_tag(tagged_addr));
 		pr_err("\n");
 		print_shadow_for_address(info.first_bad_addr);
--- a/mm/kasan/report_generic.c~kasan-rename-addr_has_shadow-to-addr_has_metadata
+++ a/mm/kasan/report_generic.c
@@ -118,7 +118,7 @@ const char *get_bug_type(struct kasan_ac
 	if (info->access_addr + info->access_size < info->access_addr)
 		return "out-of-bounds";
 
-	if (addr_has_shadow(info->access_addr))
+	if (addr_has_metadata(info->access_addr))
 		return get_shadow_bug_type(info);
 	return get_wild_bug_type(info);
 }
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 32/78] kasan: rename print_shadow_for_address to print_memory_metadata
  2020-12-18 22:00 incoming Andrew Morton
                   ` (30 preceding siblings ...)
  2020-12-18 22:03 ` [patch 31/78] kasan: rename addr_has_shadow to addr_has_metadata Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 33/78] kasan: rename SHADOW layout macros to META Andrew Morton
                   ` (45 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename print_shadow_for_address to print_memory_metadata

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory, but will reuse this
function.  Rename "shadow" to implementation-neutral "metadata".

No functional changes.

Link: https://lkml.kernel.org/r/dd955c5aadaee16aef451a6189d19172166a23f5.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/report.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/mm/kasan/report.c~kasan-rename-print_shadow_for_address-to-print_memory_metadata
+++ a/mm/kasan/report.c
@@ -252,7 +252,7 @@ static int shadow_pointer_offset(const v
 		(shadow - row) / SHADOW_BYTES_PER_BLOCK + 1;
 }
 
-static void print_shadow_for_address(const void *addr)
+static void print_memory_metadata(const void *addr)
 {
 	int i;
 	const void *shadow = kasan_mem_to_shadow(addr);
@@ -338,7 +338,7 @@ void kasan_report_invalid_free(void *obj
 	pr_err("\n");
 	print_address_description(object, tag);
 	pr_err("\n");
-	print_shadow_for_address(object);
+	print_memory_metadata(object);
 	end_report(&flags);
 }
 
@@ -379,7 +379,7 @@ static void __kasan_report(unsigned long
 	if (addr_has_metadata(untagged_addr)) {
 		print_address_description(untagged_addr, get_tag(tagged_addr));
 		pr_err("\n");
-		print_shadow_for_address(info.first_bad_addr);
+		print_memory_metadata(info.first_bad_addr);
 	} else {
 		dump_stack();
 	}
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 33/78] kasan: rename SHADOW layout macros to META
  2020-12-18 22:00 incoming Andrew Morton
                   ` (31 preceding siblings ...)
  2020-12-18 22:03 ` [patch 32/78] kasan: rename print_shadow_for_address to print_memory_metadata Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 34/78] kasan: separate metadata_fetch_row for each mode Andrew Morton
                   ` (44 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename SHADOW layout macros to META

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory, but will reuse
these macros.  Rename "SHADOW" to implementation-neutral "META".

No functional changes.

Link: https://lkml.kernel.org/r/f96244ec59dc17db35173ec352c5592b14aefaf8.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/report.c |   30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

--- a/mm/kasan/report.c~kasan-rename-shadow-layout-macros-to-meta
+++ a/mm/kasan/report.c
@@ -33,11 +33,11 @@
 #include "kasan.h"
 #include "../slab.h"
 
-/* Shadow layout customization. */
-#define SHADOW_BYTES_PER_BLOCK 1
-#define SHADOW_BLOCKS_PER_ROW 16
-#define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK)
-#define SHADOW_ROWS_AROUND_ADDR 2
+/* Metadata layout customization. */
+#define META_BYTES_PER_BLOCK 1
+#define META_BLOCKS_PER_ROW 16
+#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
+#define META_ROWS_AROUND_ADDR 2
 
 static unsigned long kasan_flags;
 
@@ -240,7 +240,7 @@ static void print_address_description(vo
 
 static bool row_is_guilty(const void *row, const void *guilty)
 {
-	return (row <= guilty) && (guilty < row + SHADOW_BYTES_PER_ROW);
+	return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW);
 }
 
 static int shadow_pointer_offset(const void *row, const void *shadow)
@@ -249,7 +249,7 @@ static int shadow_pointer_offset(const v
 	 *    3 + (BITS_PER_LONG/8)*2 chars.
 	 */
 	return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 +
-		(shadow - row) / SHADOW_BYTES_PER_BLOCK + 1;
+		(shadow - row) / META_BYTES_PER_BLOCK + 1;
 }
 
 static void print_memory_metadata(const void *addr)
@@ -259,15 +259,15 @@ static void print_memory_metadata(const
 	const void *shadow_row;
 
 	shadow_row = (void *)round_down((unsigned long)shadow,
-					SHADOW_BYTES_PER_ROW)
-		- SHADOW_ROWS_AROUND_ADDR * SHADOW_BYTES_PER_ROW;
+					META_BYTES_PER_ROW)
+		- META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW;
 
 	pr_err("Memory state around the buggy address:\n");
 
-	for (i = -SHADOW_ROWS_AROUND_ADDR; i <= SHADOW_ROWS_AROUND_ADDR; i++) {
+	for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) {
 		const void *kaddr = kasan_shadow_to_mem(shadow_row);
 		char buffer[4 + (BITS_PER_LONG/8)*2];
-		char shadow_buf[SHADOW_BYTES_PER_ROW];
+		char shadow_buf[META_BYTES_PER_ROW];
 
 		snprintf(buffer, sizeof(buffer),
 			(i == 0) ? ">%px: " : " %px: ", kaddr);
@@ -276,17 +276,17 @@ static void print_memory_metadata(const
 		 * function, because generic functions may try to
 		 * access kasan mapping for the passed address.
 		 */
-		memcpy(shadow_buf, shadow_row, SHADOW_BYTES_PER_ROW);
+		memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW);
 		print_hex_dump(KERN_ERR, buffer,
-			DUMP_PREFIX_NONE, SHADOW_BYTES_PER_ROW, 1,
-			shadow_buf, SHADOW_BYTES_PER_ROW, 0);
+			DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1,
+			shadow_buf, META_BYTES_PER_ROW, 0);
 
 		if (row_is_guilty(shadow_row, shadow))
 			pr_err("%*c\n",
 				shadow_pointer_offset(shadow_row, shadow),
 				'^');
 
-		shadow_row += SHADOW_BYTES_PER_ROW;
+		shadow_row += META_BYTES_PER_ROW;
 	}
 }
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 34/78] kasan: separate metadata_fetch_row for each mode
  2020-12-18 22:00 incoming Andrew Morton
                   ` (32 preceding siblings ...)
  2020-12-18 22:03 ` [patch 33/78] kasan: rename SHADOW layout macros to META Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 35/78] kasan: introduce CONFIG_KASAN_HW_TAGS Andrew Morton
                   ` (43 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: separate metadata_fetch_row for each mode

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Rework print_memory_metadata() to make it agnostic with regard to the way
metadata is stored.  Allow providing a separate metadata_fetch_row()
implementation for each KASAN mode.  Hardware tag-based KASAN will provide
its own implementation that doesn't use shadow memory.

No functional changes for software modes.

Link: https://lkml.kernel.org/r/5fb1ec0152bb1f521505017800387ec3e36ffe18.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/kasan.h          |    8 +++++
 mm/kasan/report.c         |   56 +++++++++++++++++-------------------
 mm/kasan/report_generic.c |    5 +++
 mm/kasan/report_sw_tags.c |    5 +++
 4 files changed, 45 insertions(+), 29 deletions(-)

--- a/mm/kasan/kasan.h~kasan-separate-metadata_fetch_row-for-each-mode
+++ a/mm/kasan/kasan.h
@@ -58,6 +58,13 @@
 #define KASAN_ABI_VERSION 1
 #endif
 
+/* Metadata layout customization. */
+#define META_BYTES_PER_BLOCK 1
+#define META_BLOCKS_PER_ROW 16
+#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
+#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_GRANULE_SIZE)
+#define META_ROWS_AROUND_ADDR 2
+
 struct kasan_access_info {
 	const void *access_addr;
 	const void *first_bad_addr;
@@ -170,6 +177,7 @@ bool check_invalid_free(void *addr);
 
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
+void metadata_fetch_row(char *buffer, void *row);
 
 #if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK
 void print_address_stack_frame(const void *addr);
--- a/mm/kasan/report.c~kasan-separate-metadata_fetch_row-for-each-mode
+++ a/mm/kasan/report.c
@@ -33,12 +33,6 @@
 #include "kasan.h"
 #include "../slab.h"
 
-/* Metadata layout customization. */
-#define META_BYTES_PER_BLOCK 1
-#define META_BLOCKS_PER_ROW 16
-#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
-#define META_ROWS_AROUND_ADDR 2
-
 static unsigned long kasan_flags;
 
 #define KASAN_BIT_REPORTED	0
@@ -238,55 +232,59 @@ static void print_address_description(vo
 	print_address_stack_frame(addr);
 }
 
-static bool row_is_guilty(const void *row, const void *guilty)
+static bool meta_row_is_guilty(const void *row, const void *addr)
 {
-	return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW);
+	return (row <= addr) && (addr < row + META_MEM_BYTES_PER_ROW);
 }
 
-static int shadow_pointer_offset(const void *row, const void *shadow)
+static int meta_pointer_offset(const void *row, const void *addr)
 {
-	/* The length of ">ff00ff00ff00ff00: " is
-	 *    3 + (BITS_PER_LONG/8)*2 chars.
+	/*
+	 * Memory state around the buggy address:
+	 *  ff00ff00ff00ff00: 00 00 00 05 fe fe fe fe fe fe fe fe fe fe fe fe
+	 *  ...
+	 *
+	 * The length of ">ff00ff00ff00ff00: " is
+	 *    3 + (BITS_PER_LONG / 8) * 2 chars.
+	 * The length of each granule metadata is 2 bytes
+	 *    plus 1 byte for space.
 	 */
-	return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 +
-		(shadow - row) / META_BYTES_PER_BLOCK + 1;
+	return 3 + (BITS_PER_LONG / 8) * 2 +
+		(addr - row) / KASAN_GRANULE_SIZE * 3 + 1;
 }
 
 static void print_memory_metadata(const void *addr)
 {
 	int i;
-	const void *shadow = kasan_mem_to_shadow(addr);
-	const void *shadow_row;
+	void *row;
 
-	shadow_row = (void *)round_down((unsigned long)shadow,
-					META_BYTES_PER_ROW)
-		- META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW;
+	row = (void *)round_down((unsigned long)addr, META_MEM_BYTES_PER_ROW)
+			- META_ROWS_AROUND_ADDR * META_MEM_BYTES_PER_ROW;
 
 	pr_err("Memory state around the buggy address:\n");
 
 	for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) {
-		const void *kaddr = kasan_shadow_to_mem(shadow_row);
-		char buffer[4 + (BITS_PER_LONG/8)*2];
-		char shadow_buf[META_BYTES_PER_ROW];
+		char buffer[4 + (BITS_PER_LONG / 8) * 2];
+		char metadata[META_BYTES_PER_ROW];
 
 		snprintf(buffer, sizeof(buffer),
-			(i == 0) ? ">%px: " : " %px: ", kaddr);
+				(i == 0) ? ">%px: " : " %px: ", row);
+
 		/*
 		 * We should not pass a shadow pointer to generic
 		 * function, because generic functions may try to
 		 * access kasan mapping for the passed address.
 		 */
-		memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW);
+		metadata_fetch_row(&metadata[0], row);
+
 		print_hex_dump(KERN_ERR, buffer,
 			DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1,
-			shadow_buf, META_BYTES_PER_ROW, 0);
+			metadata, META_BYTES_PER_ROW, 0);
 
-		if (row_is_guilty(shadow_row, shadow))
-			pr_err("%*c\n",
-				shadow_pointer_offset(shadow_row, shadow),
-				'^');
+		if (meta_row_is_guilty(row, addr))
+			pr_err("%*c\n", meta_pointer_offset(row, addr), '^');
 
-		shadow_row += META_BYTES_PER_ROW;
+		row += META_MEM_BYTES_PER_ROW;
 	}
 }
 
--- a/mm/kasan/report_generic.c~kasan-separate-metadata_fetch_row-for-each-mode
+++ a/mm/kasan/report_generic.c
@@ -123,6 +123,11 @@ const char *get_bug_type(struct kasan_ac
 	return get_wild_bug_type(info);
 }
 
+void metadata_fetch_row(char *buffer, void *row)
+{
+	memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);
+}
+
 #if CONFIG_KASAN_STACK
 static bool __must_check tokenize_frame_descr(const char **frame_descr,
 					      char *token, size_t max_tok_len,
--- a/mm/kasan/report_sw_tags.c~kasan-separate-metadata_fetch_row-for-each-mode
+++ a/mm/kasan/report_sw_tags.c
@@ -80,6 +80,11 @@ void *find_first_bad_addr(void *addr, si
 	return p;
 }
 
+void metadata_fetch_row(char *buffer, void *row)
+{
+	memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);
+}
+
 void print_tags(u8 addr_tag, const void *addr)
 {
 	u8 *shadow = (u8 *)kasan_mem_to_shadow(addr);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 35/78] kasan: introduce CONFIG_KASAN_HW_TAGS
  2020-12-18 22:00 incoming Andrew Morton
                   ` (33 preceding siblings ...)
  2020-12-18 22:03 ` [patch 34/78] kasan: separate metadata_fetch_row for each mode Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 36/78] arm64: enable armv8.5-a asm-arch option Andrew Morton
                   ` (42 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: introduce CONFIG_KASAN_HW_TAGS

This patch adds a configuration option for a new KASAN mode called
hardware tag-based KASAN.  This mode uses the memory tagging approach like
the software tag-based mode, but relies on arm64 Memory Tagging Extension
feature for tag management and access checking.

Link: https://lkml.kernel.org/r/44906a209d3a44f9c6f5a21841e90988e365601e.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 lib/Kconfig.kasan |   61 +++++++++++++++++++++++++++++++-------------
 1 file changed, 44 insertions(+), 17 deletions(-)

--- a/lib/Kconfig.kasan~kasan-introduce-config_kasan_hw_tags
+++ a/lib/Kconfig.kasan
@@ -6,7 +6,10 @@ config HAVE_ARCH_KASAN
 config HAVE_ARCH_KASAN_SW_TAGS
 	bool
 
-config	HAVE_ARCH_KASAN_VMALLOC
+config HAVE_ARCH_KASAN_HW_TAGS
+	bool
+
+config HAVE_ARCH_KASAN_VMALLOC
 	bool
 
 config CC_HAS_KASAN_GENERIC
@@ -15,16 +18,19 @@ config CC_HAS_KASAN_GENERIC
 config CC_HAS_KASAN_SW_TAGS
 	def_bool $(cc-option, -fsanitize=kernel-hwaddress)
 
+# This option is only required for software KASAN modes.
+# Old GCC versions don't have proper support for no_sanitize_address.
+# See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89124 for details.
 config CC_HAS_WORKING_NOSANITIZE_ADDRESS
 	def_bool !CC_IS_GCC || GCC_VERSION >= 80300
 
 menuconfig KASAN
 	bool "KASAN: runtime memory debugger"
-	depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \
-		   (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)
+	depends on (((HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \
+		     (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)) && \
+		    CC_HAS_WORKING_NOSANITIZE_ADDRESS) || \
+		   HAVE_ARCH_KASAN_HW_TAGS
 	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
-	depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS
-	select CONSTRUCTORS
 	select STACKDEPOT
 	help
 	  Enables KASAN (KernelAddressSANitizer) - runtime memory debugger,
@@ -37,18 +43,24 @@ choice
 	prompt "KASAN mode"
 	default KASAN_GENERIC
 	help
-	  KASAN has two modes: generic KASAN (similar to userspace ASan,
-	  x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and
-	  software tag-based KASAN (a version based on software memory
-	  tagging, arm64 only, similar to userspace HWASan, enabled with
-	  CONFIG_KASAN_SW_TAGS).
+	  KASAN has three modes:
+	  1. generic KASAN (similar to userspace ASan,
+	     x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC),
+	  2. software tag-based KASAN (arm64 only, based on software
+	     memory tagging (similar to userspace HWASan), enabled with
+	     CONFIG_KASAN_SW_TAGS), and
+	  3. hardware tag-based KASAN (arm64 only, based on hardware
+	     memory tagging, enabled with CONFIG_KASAN_HW_TAGS).
+
+	  All KASAN modes are strictly debugging features.
 
-	  Both generic and tag-based KASAN are strictly debugging features.
+	  For better error reports enable CONFIG_STACKTRACE.
 
 config KASAN_GENERIC
 	bool "Generic mode"
 	depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC
 	select SLUB_DEBUG if SLUB
+	select CONSTRUCTORS
 	help
 	  Enables generic KASAN mode.
 
@@ -61,8 +73,6 @@ config KASAN_GENERIC
 	  and introduces an overhead of ~x1.5 for the rest of the allocations.
 	  The performance slowdown is ~x3.
 
-	  For better error detection enable CONFIG_STACKTRACE.
-
 	  Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB
 	  (the resulting kernel does not boot).
 
@@ -70,11 +80,15 @@ config KASAN_SW_TAGS
 	bool "Software tag-based mode"
 	depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS
 	select SLUB_DEBUG if SLUB
+	select CONSTRUCTORS
 	help
 	  Enables software tag-based KASAN mode.
 
-	  This mode requires Top Byte Ignore support by the CPU and therefore
-	  is only supported for arm64. This mode requires Clang.
+	  This mode require software memory tagging support in the form of
+	  HWASan-like compiler instrumentation.
+
+	  Currently this mode is only implemented for arm64 CPUs and relies on
+	  Top Byte Ignore. This mode requires Clang.
 
 	  This mode consumes about 1/16th of available memory at kernel start
 	  and introduces an overhead of ~20% for the rest of the allocations.
@@ -82,15 +96,27 @@ config KASAN_SW_TAGS
 	  casting and comparison, as it embeds tags into the top byte of each
 	  pointer.
 
-	  For better error detection enable CONFIG_STACKTRACE.
-
 	  Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB
 	  (the resulting kernel does not boot).
 
+config KASAN_HW_TAGS
+	bool "Hardware tag-based mode"
+	depends on HAVE_ARCH_KASAN_HW_TAGS
+	depends on SLUB
+	help
+	  Enables hardware tag-based KASAN mode.
+
+	  This mode requires hardware memory tagging support, and can be used
+	  by any architecture that provides it.
+
+	  Currently this mode is only implemented for arm64 CPUs starting from
+	  ARMv8.5 and relies on Memory Tagging Extension and Top Byte Ignore.
+
 endchoice
 
 choice
 	prompt "Instrumentation type"
+	depends on KASAN_GENERIC || KASAN_SW_TAGS
 	default KASAN_OUTLINE
 
 config KASAN_OUTLINE
@@ -114,6 +140,7 @@ endchoice
 
 config KASAN_STACK_ENABLE
 	bool "Enable stack instrumentation (unsafe)" if CC_IS_CLANG && !COMPILE_TEST
+	depends on KASAN_GENERIC || KASAN_SW_TAGS
 	help
 	  The LLVM stack address sanitizer has a know problem that
 	  causes excessive stack usage in a lot of functions, see
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 36/78] arm64: enable armv8.5-a asm-arch option
  2020-12-18 22:00 incoming Andrew Morton
                   ` (34 preceding siblings ...)
  2020-12-18 22:03 ` [patch 35/78] kasan: introduce CONFIG_KASAN_HW_TAGS Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 37/78] arm64: mte: add in-kernel MTE helpers Andrew Morton
                   ` (41 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: enable armv8.5-a asm-arch option

Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) which is
an armv8.5-a architecture extension.

Enable the correct asm option when the compiler supports it in order to
allow the usage of ALTERNATIVE()s with MTE instructions.

Link: https://lkml.kernel.org/r/d03d1157124ea3532eaeb77507988733f5734986.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/Kconfig  |    4 ++++
 arch/arm64/Makefile |    5 +++++
 2 files changed, 9 insertions(+)

--- a/arch/arm64/Kconfig~arm64-enable-armv85-a-asm-arch-option
+++ a/arch/arm64/Kconfig
@@ -1571,6 +1571,9 @@ endmenu
 
 menu "ARMv8.5 architectural features"
 
+config AS_HAS_ARMV8_5
+	def_bool $(cc-option,-Wa$(comma)-march=armv8.5-a)
+
 config ARM64_BTI
 	bool "Branch Target Identification support"
 	default y
@@ -1645,6 +1648,7 @@ config ARM64_MTE
 	bool "Memory Tagging Extension support"
 	default y
 	depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI
+	depends on AS_HAS_ARMV8_5
 	select ARCH_USES_HIGH_VMA_FLAGS
 	help
 	  Memory Tagging (part of the ARMv8.5 Extensions) provides
--- a/arch/arm64/Makefile~arm64-enable-armv85-a-asm-arch-option
+++ a/arch/arm64/Makefile
@@ -96,6 +96,11 @@ ifeq ($(CONFIG_AS_HAS_ARMV8_4), y)
 asm-arch := armv8.4-a
 endif
 
+ifeq ($(CONFIG_AS_HAS_ARMV8_5), y)
+# make sure to pass the newest target architecture to -march.
+asm-arch := armv8.5-a
+endif
+
 ifdef asm-arch
 KBUILD_CFLAGS	+= -Wa,-march=$(asm-arch) \
 		   -DARM64_ASM_ARCH='"$(asm-arch)"'
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 37/78] arm64: mte: add in-kernel MTE helpers
  2020-12-18 22:00 incoming Andrew Morton
                   ` (35 preceding siblings ...)
  2020-12-18 22:03 ` [patch 36/78] arm64: enable armv8.5-a asm-arch option Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 38/78] arm64: mte: reset the page tag in page->flags Andrew Morton
                   ` (40 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: mte: add in-kernel MTE helpers

Provide helper functions to manipulate allocation and pointer tags for
kernel addresses.

Low-level helper functions (mte_assign_*, written in assembly) operate tag
values from the [0x0, 0xF] range.  High-level helper functions
(mte_get/set_*) use the [0xF0, 0xFF] range to preserve compatibility with
normal kernel pointers that have 0xFF in their top byte.

MTE_GRANULE_SIZE and related definitions are moved to mte-def.h header
that doesn't have any dependencies and is safe to include into any
low-level header.

Link: https://lkml.kernel.org/r/c31bf759b4411b2d98cdd801eb928e241584fd1f.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/esr.h       |    1 
 arch/arm64/include/asm/mte-def.h   |   15 +++++++
 arch/arm64/include/asm/mte-kasan.h |   56 +++++++++++++++++++++++++++
 arch/arm64/include/asm/mte.h       |   20 ++++++---
 arch/arm64/kernel/mte.c            |   48 +++++++++++++++++++++++
 arch/arm64/lib/mte.S               |   16 +++++++
 6 files changed, 150 insertions(+), 6 deletions(-)

--- a/arch/arm64/include/asm/esr.h~arm64-mte-add-in-kernel-mte-helpers
+++ a/arch/arm64/include/asm/esr.h
@@ -106,6 +106,7 @@
 #define ESR_ELx_FSC_TYPE	(0x3C)
 #define ESR_ELx_FSC_LEVEL	(0x03)
 #define ESR_ELx_FSC_EXTABT	(0x10)
+#define ESR_ELx_FSC_MTE		(0x11)
 #define ESR_ELx_FSC_SERROR	(0x11)
 #define ESR_ELx_FSC_ACCESS	(0x08)
 #define ESR_ELx_FSC_FAULT	(0x04)
--- /dev/null
+++ a/arch/arm64/include/asm/mte-def.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 ARM Ltd.
+ */
+#ifndef __ASM_MTE_DEF_H
+#define __ASM_MTE_DEF_H
+
+#define MTE_GRANULE_SIZE	UL(16)
+#define MTE_GRANULE_MASK	(~(MTE_GRANULE_SIZE - 1))
+#define MTE_TAG_SHIFT		56
+#define MTE_TAG_SIZE		4
+#define MTE_TAG_MASK		GENMASK((MTE_TAG_SHIFT + (MTE_TAG_SIZE - 1)), MTE_TAG_SHIFT)
+#define MTE_TAG_MAX		(MTE_TAG_MASK >> MTE_TAG_SHIFT)
+
+#endif /* __ASM_MTE_DEF_H  */
--- a/arch/arm64/include/asm/mte.h~arm64-mte-add-in-kernel-mte-helpers
+++ a/arch/arm64/include/asm/mte.h
@@ -5,14 +5,16 @@
 #ifndef __ASM_MTE_H
 #define __ASM_MTE_H
 
-#define MTE_GRANULE_SIZE	UL(16)
-#define MTE_GRANULE_MASK	(~(MTE_GRANULE_SIZE - 1))
-#define MTE_TAG_SHIFT		56
-#define MTE_TAG_SIZE		4
+#include <asm/compiler.h>
+#include <asm/mte-def.h>
+
+#define __MTE_PREAMBLE		ARM64_ASM_PREAMBLE ".arch_extension memtag\n"
 
 #ifndef __ASSEMBLY__
 
+#include <linux/bitfield.h>
 #include <linux/page-flags.h>
+#include <linux/types.h>
 
 #include <asm/pgtable-types.h>
 
@@ -45,7 +47,9 @@ long get_mte_ctrl(struct task_struct *ta
 int mte_ptrace_copy_tags(struct task_struct *child, long request,
 			 unsigned long addr, unsigned long data);
 
-#else
+void mte_assign_mem_tag_range(void *addr, size_t size);
+
+#else /* CONFIG_ARM64_MTE */
 
 /* unused if !CONFIG_ARM64_MTE, silence the compiler */
 #define PG_mte_tagged	0
@@ -80,7 +84,11 @@ static inline int mte_ptrace_copy_tags(s
 	return -EIO;
 }
 
-#endif
+static inline void mte_assign_mem_tag_range(void *addr, size_t size)
+{
+}
+
+#endif /* CONFIG_ARM64_MTE */
 
 #endif /* __ASSEMBLY__ */
 #endif /* __ASM_MTE_H  */
--- /dev/null
+++ a/arch/arm64/include/asm/mte-kasan.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 ARM Ltd.
+ */
+#ifndef __ASM_MTE_KASAN_H
+#define __ASM_MTE_KASAN_H
+
+#include <asm/mte-def.h>
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+/*
+ * The functions below are meant to be used only for the
+ * KASAN_HW_TAGS interface defined in asm/memory.h.
+ */
+#ifdef CONFIG_ARM64_MTE
+
+static inline u8 mte_get_ptr_tag(void *ptr)
+{
+	/* Note: The format of KASAN tags is 0xF<x> */
+	u8 tag = 0xF0 | (u8)(((u64)(ptr)) >> MTE_TAG_SHIFT);
+
+	return tag;
+}
+
+u8 mte_get_mem_tag(void *addr);
+u8 mte_get_random_tag(void);
+void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag);
+
+#else /* CONFIG_ARM64_MTE */
+
+static inline u8 mte_get_ptr_tag(void *ptr)
+{
+	return 0xFF;
+}
+
+static inline u8 mte_get_mem_tag(void *addr)
+{
+	return 0xFF;
+}
+static inline u8 mte_get_random_tag(void)
+{
+	return 0xFF;
+}
+static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
+{
+	return addr;
+}
+
+#endif /* CONFIG_ARM64_MTE */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_MTE_KASAN_H  */
--- a/arch/arm64/kernel/mte.c~arm64-mte-add-in-kernel-mte-helpers
+++ a/arch/arm64/kernel/mte.c
@@ -13,10 +13,13 @@
 #include <linux/swap.h>
 #include <linux/swapops.h>
 #include <linux/thread_info.h>
+#include <linux/types.h>
 #include <linux/uio.h>
 
+#include <asm/barrier.h>
 #include <asm/cpufeature.h>
 #include <asm/mte.h>
+#include <asm/mte-kasan.h>
 #include <asm/ptrace.h>
 #include <asm/sysreg.h>
 
@@ -72,6 +75,51 @@ int memcmp_pages(struct page *page1, str
 	return ret;
 }
 
+u8 mte_get_mem_tag(void *addr)
+{
+	if (!system_supports_mte())
+		return 0xFF;
+
+	asm(__MTE_PREAMBLE "ldg %0, [%0]"
+	    : "+r" (addr));
+
+	return mte_get_ptr_tag(addr);
+}
+
+u8 mte_get_random_tag(void)
+{
+	void *addr;
+
+	if (!system_supports_mte())
+		return 0xFF;
+
+	asm(__MTE_PREAMBLE "irg %0, %0"
+	    : "+r" (addr));
+
+	return mte_get_ptr_tag(addr);
+}
+
+void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
+{
+	void *ptr = addr;
+
+	if ((!system_supports_mte()) || (size == 0))
+		return addr;
+
+	/* Make sure that size is MTE granule aligned. */
+	WARN_ON(size & (MTE_GRANULE_SIZE - 1));
+
+	/* Make sure that the address is MTE granule aligned. */
+	WARN_ON((u64)addr & (MTE_GRANULE_SIZE - 1));
+
+	tag = 0xF0 | tag;
+	ptr = (void *)__tag_set(ptr, tag);
+
+	mte_assign_mem_tag_range(ptr, size);
+
+	return ptr;
+}
+
 static void update_sctlr_el1_tcf0(u64 tcf0)
 {
 	/* ISB required for the kernel uaccess routines */
--- a/arch/arm64/lib/mte.S~arm64-mte-add-in-kernel-mte-helpers
+++ a/arch/arm64/lib/mte.S
@@ -149,3 +149,19 @@ SYM_FUNC_START(mte_restore_page_tags)
 
 	ret
 SYM_FUNC_END(mte_restore_page_tags)
+
+/*
+ * Assign allocation tags for a region of memory based on the pointer tag
+ *   x0 - source pointer
+ *   x1 - size
+ *
+ * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and
+ * size must be non-zero and MTE_GRANULE_SIZE aligned.
+ */
+SYM_FUNC_START(mte_assign_mem_tag_range)
+1:	stg	x0, [x0]
+	add	x0, x0, #MTE_GRANULE_SIZE
+	subs	x1, x1, #MTE_GRANULE_SIZE
+	b.gt	1b
+	ret
+SYM_FUNC_END(mte_assign_mem_tag_range)
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 38/78] arm64: mte: reset the page tag in page->flags
  2020-12-18 22:00 incoming Andrew Morton
                   ` (36 preceding siblings ...)
  2020-12-18 22:03 ` [patch 37/78] arm64: mte: add in-kernel MTE helpers Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 39/78] arm64: mte: add in-kernel tag fault handler Andrew Morton
                   ` (39 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: mte: reset the page tag in page->flags

The hardware tag-based KASAN for compatibility with the other modes stores
the tag associated to a page in page->flags.  Due to this the kernel
faults on access when it allocates a page with an initial tag and the user
changes the tags.

Reset the tag associated by the kernel to a page in all the meaningful
places to prevent kernel faults on access.

Note: An alternative to this approach could be to modify page_to_virt(). 
This though could end up being racy, in fact if a CPU checks the
PG_mte_tagged bit and decides that the page is not tagged but another CPU
maps the same with PROT_MTE and becomes tagged the subsequent kernel
access would fail.

Link: https://lkml.kernel.org/r/9073d4e973747a6f78d5bdd7ebe17f290d087096.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/kernel/hibernate.c |    5 +++++
 arch/arm64/kernel/mte.c       |    9 +++++++++
 arch/arm64/mm/copypage.c      |    9 +++++++++
 arch/arm64/mm/mteswap.c       |    9 +++++++++
 4 files changed, 32 insertions(+)

--- a/arch/arm64/kernel/hibernate.c~arm64-mte-reset-the-page-tag-in-page-flags
+++ a/arch/arm64/kernel/hibernate.c
@@ -371,6 +371,11 @@ static void swsusp_mte_restore_tags(void
 		unsigned long pfn = xa_state.xa_index;
 		struct page *page = pfn_to_online_page(pfn);
 
+		/*
+		 * It is not required to invoke page_kasan_tag_reset(page)
+		 * at this point since the tags stored in page->flags are
+		 * already restored.
+		 */
 		mte_restore_page_tags(page_address(page), tags);
 
 		mte_free_tag_storage(tags);
--- a/arch/arm64/kernel/mte.c~arm64-mte-reset-the-page-tag-in-page-flags
+++ a/arch/arm64/kernel/mte.c
@@ -34,6 +34,15 @@ static void mte_sync_page_tags(struct pa
 			return;
 	}
 
+	page_kasan_tag_reset(page);
+	/*
+	 * We need smp_wmb() in between setting the flags and clearing the
+	 * tags because if another thread reads page->flags and builds a
+	 * tagged address out of it, there is an actual dependency to the
+	 * memory access, but on the current thread we do not guarantee that
+	 * the new page->flags are visible before the tags were updated.
+	 */
+	smp_wmb();
 	mte_clear_page_tags(page_address(page));
 }
 
--- a/arch/arm64/mm/copypage.c~arm64-mte-reset-the-page-tag-in-page-flags
+++ a/arch/arm64/mm/copypage.c
@@ -23,6 +23,15 @@ void copy_highpage(struct page *to, stru
 
 	if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) {
 		set_bit(PG_mte_tagged, &to->flags);
+		page_kasan_tag_reset(to);
+		/*
+		 * We need smp_wmb() in between setting the flags and clearing the
+		 * tags because if another thread reads page->flags and builds a
+		 * tagged address out of it, there is an actual dependency to the
+		 * memory access, but on the current thread we do not guarantee that
+		 * the new page->flags are visible before the tags were updated.
+		 */
+		smp_wmb();
 		mte_copy_page_tags(kto, kfrom);
 	}
 }
--- a/arch/arm64/mm/mteswap.c~arm64-mte-reset-the-page-tag-in-page-flags
+++ a/arch/arm64/mm/mteswap.c
@@ -53,6 +53,15 @@ bool mte_restore_tags(swp_entry_t entry,
 	if (!tags)
 		return false;
 
+	page_kasan_tag_reset(page);
+	/*
+	 * We need smp_wmb() in between setting the flags and clearing the
+	 * tags because if another thread reads page->flags and builds a
+	 * tagged address out of it, there is an actual dependency to the
+	 * memory access, but on the current thread we do not guarantee that
+	 * the new page->flags are visible before the tags were updated.
+	 */
+	smp_wmb();
 	mte_restore_page_tags(page_address(page), tags);
 
 	return true;
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 39/78] arm64: mte: add in-kernel tag fault handler
  2020-12-18 22:00 incoming Andrew Morton
                   ` (37 preceding siblings ...)
  2020-12-18 22:03 ` [patch 38/78] arm64: mte: reset the page tag in page->flags Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 40/78] arm64: kasan: allow enabling in-kernel MTE Andrew Morton
                   ` (38 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: mte: add in-kernel tag fault handler

Add the implementation of the in-kernel fault handler.

When a tag fault happens on a kernel address:
* MTE is disabled on the current CPU,
* the execution continues.

When a tag fault happens on a user address:
* the kernel executes do_bad_area() and panics.

The tag fault handler for kernel addresses is currently empty and will be
filled in by a future commit.

Link: https://lkml.kernel.org/r/ad31529b073e22840b7a2246172c2b67747ed7c4.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
[catalin.marinas@arm.com: ensure CONFIG_ARM64_PAN is enabled with MTE]
  Link: https://lkml.kernel.org/r/20201203102628.GB2224@gaia
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/Kconfig               |    2 +
 arch/arm64/include/asm/uaccess.h |   23 ++++++++++++++
 arch/arm64/mm/fault.c            |   45 +++++++++++++++++++++++++++++
 3 files changed, 70 insertions(+)

--- a/arch/arm64/include/asm/uaccess.h~arm64-mte-add-in-kernel-tag-fault-handler
+++ a/arch/arm64/include/asm/uaccess.h
@@ -159,8 +159,28 @@ static inline void __uaccess_enable_hw_p
 			CONFIG_ARM64_PAN));
 }
 
+/*
+ * The Tag Check Flag (TCF) mode for MTE is per EL, hence TCF0
+ * affects EL0 and TCF affects EL1 irrespective of which TTBR is
+ * used.
+ * The kernel accesses TTBR0 usually with LDTR/STTR instructions
+ * when UAO is available, so these would act as EL0 accesses using
+ * TCF0.
+ * However futex.h code uses exclusives which would be executed as
+ * EL1, this can potentially cause a tag check fault even if the
+ * user disables TCF0.
+ *
+ * To address the problem we set the PSTATE.TCO bit in uaccess_enable()
+ * and reset it in uaccess_disable().
+ *
+ * The Tag check override (TCO) bit disables temporarily the tag checking
+ * preventing the issue.
+ */
 static inline void uaccess_disable_privileged(void)
 {
+	asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(0),
+				 ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
 	if (uaccess_ttbr0_disable())
 		return;
 
@@ -169,6 +189,9 @@ static inline void uaccess_disable_privi
 
 static inline void uaccess_enable_privileged(void)
 {
+	asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(1),
+				 ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
 	if (uaccess_ttbr0_enable())
 		return;
 
--- a/arch/arm64/Kconfig~arm64-mte-add-in-kernel-tag-fault-handler
+++ a/arch/arm64/Kconfig
@@ -1649,6 +1649,8 @@ config ARM64_MTE
 	default y
 	depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI
 	depends on AS_HAS_ARMV8_5
+	# Required for tag checking in the uaccess routines
+	depends on ARM64_PAN
 	select ARCH_USES_HIGH_VMA_FLAGS
 	help
 	  Memory Tagging (part of the ARMv8.5 Extensions) provides
--- a/arch/arm64/mm/fault.c~arm64-mte-add-in-kernel-tag-fault-handler
+++ a/arch/arm64/mm/fault.c
@@ -33,6 +33,7 @@
 #include <asm/debug-monitors.h>
 #include <asm/esr.h>
 #include <asm/kprobes.h>
+#include <asm/mte.h>
 #include <asm/processor.h>
 #include <asm/sysreg.h>
 #include <asm/system_misc.h>
@@ -296,6 +297,44 @@ static void die_kernel_fault(const char
 	do_exit(SIGKILL);
 }
 
+static void report_tag_fault(unsigned long addr, unsigned int esr,
+			     struct pt_regs *regs)
+{
+}
+
+static void do_tag_recovery(unsigned long addr, unsigned int esr,
+			   struct pt_regs *regs)
+{
+	static bool reported;
+
+	if (!READ_ONCE(reported)) {
+		report_tag_fault(addr, esr, regs);
+		WRITE_ONCE(reported, true);
+	}
+
+	/*
+	 * Disable MTE Tag Checking on the local CPU for the current EL.
+	 * It will be done lazily on the other CPUs when they will hit a
+	 * tag fault.
+	 */
+	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_NONE);
+	isb();
+}
+
+static bool is_el1_mte_sync_tag_check_fault(unsigned int esr)
+{
+	unsigned int ec = ESR_ELx_EC(esr);
+	unsigned int fsc = esr & ESR_ELx_FSC;
+
+	if (ec != ESR_ELx_EC_DABT_CUR)
+		return false;
+
+	if (fsc == ESR_ELx_FSC_MTE)
+		return true;
+
+	return false;
+}
+
 static void __do_kernel_fault(unsigned long addr, unsigned int esr,
 			      struct pt_regs *regs)
 {
@@ -312,6 +351,12 @@ static void __do_kernel_fault(unsigned l
 	    "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr))
 		return;
 
+	if (is_el1_mte_sync_tag_check_fault(esr)) {
+		do_tag_recovery(addr, esr, regs);
+
+		return;
+	}
+
 	if (is_el1_permission_fault(addr, esr, regs)) {
 		if (esr & ESR_ELx_WNR)
 			msg = "write to read-only memory";
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 40/78] arm64: kasan: allow enabling in-kernel MTE
  2020-12-18 22:00 incoming Andrew Morton
                   ` (38 preceding siblings ...)
  2020-12-18 22:03 ` [patch 39/78] arm64: mte: add in-kernel tag fault handler Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 41/78] arm64: mte: convert gcr_user into an exclude mask Andrew Morton
                   ` (37 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: kasan: allow enabling in-kernel MTE

Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) feature
and requires it to be enabled.  MTE supports

This patch adds a new mte_enable_kernel() helper, that enables MTE in
Synchronous mode in EL1 and is intended to be called from KASAN runtime
during initialization.

The Tag Checking operation causes a synchronous data abort as a
consequence of a tag check fault when MTE is configured in synchronous
mode.

As part of this change enable match-all tag for EL1 to allow the kernel to
access user pages without faulting.  This is required because the kernel
does not have knowledge of the tags set by the user in a page.

Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a
similar way as TCF0 affects EL0.

MTE that is built on top of the Top Byte Ignore (TBI) feature hence we
enable it as part of this patch as well.

Link: https://lkml.kernel.org/r/7352b0a0899af65c2785416c8ca6bf3845b66fa1.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/mte-kasan.h |    6 ++++++
 arch/arm64/kernel/mte.c            |    7 +++++++
 arch/arm64/mm/proc.S               |   23 ++++++++++++++++++++---
 3 files changed, 33 insertions(+), 3 deletions(-)

--- a/arch/arm64/include/asm/mte-kasan.h~arm64-kasan-allow-enabling-in-kernel-mte
+++ a/arch/arm64/include/asm/mte-kasan.h
@@ -29,6 +29,8 @@ u8 mte_get_mem_tag(void *addr);
 u8 mte_get_random_tag(void);
 void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag);
 
+void mte_enable_kernel(void);
+
 #else /* CONFIG_ARM64_MTE */
 
 static inline u8 mte_get_ptr_tag(void *ptr)
@@ -49,6 +51,10 @@ static inline void *mte_set_mem_tag_rang
 	return addr;
 }
 
+static inline void mte_enable_kernel(void)
+{
+}
+
 #endif /* CONFIG_ARM64_MTE */
 
 #endif /* __ASSEMBLY__ */
--- a/arch/arm64/kernel/mte.c~arm64-kasan-allow-enabling-in-kernel-mte
+++ a/arch/arm64/kernel/mte.c
@@ -129,6 +129,13 @@ void *mte_set_mem_tag_range(void *addr,
 	return ptr;
 }
 
+void mte_enable_kernel(void)
+{
+	/* Enable MTE Sync Mode for EL1. */
+	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC);
+	isb();
+}
+
 static void update_sctlr_el1_tcf0(u64 tcf0)
 {
 	/* ISB required for the kernel uaccess routines */
--- a/arch/arm64/mm/proc.S~arm64-kasan-allow-enabling-in-kernel-mte
+++ a/arch/arm64/mm/proc.S
@@ -40,9 +40,15 @@
 #define TCR_CACHE_FLAGS	TCR_IRGN_WBWA | TCR_ORGN_WBWA
 
 #ifdef CONFIG_KASAN_SW_TAGS
-#define TCR_KASAN_FLAGS TCR_TBI1 | TCR_TBID1
+#define TCR_KASAN_SW_FLAGS TCR_TBI1 | TCR_TBID1
 #else
-#define TCR_KASAN_FLAGS 0
+#define TCR_KASAN_SW_FLAGS 0
+#endif
+
+#ifdef CONFIG_KASAN_HW_TAGS
+#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1
+#else
+#define TCR_KASAN_HW_FLAGS 0
 #endif
 
 /*
@@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup)
 	 */
 	mov_q	x5, MAIR_EL1_SET
 #ifdef CONFIG_ARM64_MTE
+	mte_tcr	.req	x20
+
+	mov	mte_tcr, #0
+
 	/*
 	 * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported
 	 * (ID_AA64PFR1_EL1[11:8] > 1).
@@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup)
 	/* clear any pending tag check faults in TFSR*_EL1 */
 	msr_s	SYS_TFSR_EL1, xzr
 	msr_s	SYS_TFSRE0_EL1, xzr
+
+	/* set the TCR_EL1 bits */
+	mov_q	mte_tcr, TCR_KASAN_HW_FLAGS
 1:
 #endif
 	msr	mair_el1, x5
@@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup)
 	 */
 	mov_q	x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \
 			TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \
-			TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS
+			TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS
+#ifdef CONFIG_ARM64_MTE
+	orr	x10, x10, mte_tcr
+	.unreq	mte_tcr
+#endif
 	tcr_clear_errata_bits x10, x9, x5
 
 #ifdef CONFIG_ARM64_VA_BITS_52
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 41/78] arm64: mte: convert gcr_user into an exclude mask
  2020-12-18 22:00 incoming Andrew Morton
                   ` (39 preceding siblings ...)
  2020-12-18 22:03 ` [patch 40/78] arm64: kasan: allow enabling in-kernel MTE Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 42/78] arm64: mte: switch GCR_EL1 in kernel entry and exit Andrew Morton
                   ` (36 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: mte: convert gcr_user into an exclude mask

The gcr_user mask is a per thread mask that represents the tags that are
excluded from random generation when the Memory Tagging Extension is
present and an 'irg' instruction is invoked.

gcr_user affects the behavior on EL0 only.

Currently that mask is an include mask and it is controlled by the user
via prctl() while GCR_EL1 accepts an exclude mask.

Convert the include mask into an exclude one to make it easier the
register setting.

Note: This change will affect gcr_kernel (for EL1) introduced with a
future patch.

Link: https://lkml.kernel.org/r/946dd31be833b660334c4f93410acf6d6c4cf3c4.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/processor.h |    2 -
 arch/arm64/kernel/mte.c            |   29 +++++++++++++--------------
 2 files changed, 16 insertions(+), 15 deletions(-)

--- a/arch/arm64/include/asm/processor.h~arm64-mte-convert-gcr_user-into-an-exclude-mask
+++ a/arch/arm64/include/asm/processor.h
@@ -152,7 +152,7 @@ struct thread_struct {
 #endif
 #ifdef CONFIG_ARM64_MTE
 	u64			sctlr_tcf0;
-	u64			gcr_user_incl;
+	u64			gcr_user_excl;
 #endif
 };
 
--- a/arch/arm64/kernel/mte.c~arm64-mte-convert-gcr_user-into-an-exclude-mask
+++ a/arch/arm64/kernel/mte.c
@@ -156,23 +156,22 @@ static void set_sctlr_el1_tcf0(u64 tcf0)
 	preempt_enable();
 }
 
-static void update_gcr_el1_excl(u64 incl)
+static void update_gcr_el1_excl(u64 excl)
 {
-	u64 excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
 
 	/*
-	 * Note that 'incl' is an include mask (controlled by the user via
-	 * prctl()) while GCR_EL1 accepts an exclude mask.
+	 * Note that the mask controlled by the user via prctl() is an
+	 * include while GCR_EL1 accepts an exclude mask.
 	 * No need for ISB since this only affects EL0 currently, implicit
 	 * with ERET.
 	 */
 	sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl);
 }
 
-static void set_gcr_el1_excl(u64 incl)
+static void set_gcr_el1_excl(u64 excl)
 {
-	current->thread.gcr_user_incl = incl;
-	update_gcr_el1_excl(incl);
+	current->thread.gcr_user_excl = excl;
+	update_gcr_el1_excl(excl);
 }
 
 void flush_mte_state(void)
@@ -187,7 +186,7 @@ void flush_mte_state(void)
 	/* disable tag checking */
 	set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE);
 	/* reset tag generation mask */
-	set_gcr_el1_excl(0);
+	set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK);
 }
 
 void mte_thread_switch(struct task_struct *next)
@@ -198,7 +197,7 @@ void mte_thread_switch(struct task_struc
 	/* avoid expensive SCTLR_EL1 accesses if no change */
 	if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0)
 		update_sctlr_el1_tcf0(next->thread.sctlr_tcf0);
-	update_gcr_el1_excl(next->thread.gcr_user_incl);
+	update_gcr_el1_excl(next->thread.gcr_user_excl);
 }
 
 void mte_suspend_exit(void)
@@ -206,13 +205,14 @@ void mte_suspend_exit(void)
 	if (!system_supports_mte())
 		return;
 
-	update_gcr_el1_excl(current->thread.gcr_user_incl);
+	update_gcr_el1_excl(current->thread.gcr_user_excl);
 }
 
 long set_mte_ctrl(struct task_struct *task, unsigned long arg)
 {
 	u64 tcf0;
-	u64 gcr_incl = (arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT;
+	u64 gcr_excl = ~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) &
+		       SYS_GCR_EL1_EXCL_MASK;
 
 	if (!system_supports_mte())
 		return 0;
@@ -233,10 +233,10 @@ long set_mte_ctrl(struct task_struct *ta
 
 	if (task != current) {
 		task->thread.sctlr_tcf0 = tcf0;
-		task->thread.gcr_user_incl = gcr_incl;
+		task->thread.gcr_user_excl = gcr_excl;
 	} else {
 		set_sctlr_el1_tcf0(tcf0);
-		set_gcr_el1_excl(gcr_incl);
+		set_gcr_el1_excl(gcr_excl);
 	}
 
 	return 0;
@@ -245,11 +245,12 @@ long set_mte_ctrl(struct task_struct *ta
 long get_mte_ctrl(struct task_struct *task)
 {
 	unsigned long ret;
+	u64 incl = ~task->thread.gcr_user_excl & SYS_GCR_EL1_EXCL_MASK;
 
 	if (!system_supports_mte())
 		return 0;
 
-	ret = task->thread.gcr_user_incl << PR_MTE_TAG_SHIFT;
+	ret = incl << PR_MTE_TAG_SHIFT;
 
 	switch (task->thread.sctlr_tcf0) {
 	case SCTLR_EL1_TCF0_NONE:
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 42/78] arm64: mte: switch GCR_EL1 in kernel entry and exit
  2020-12-18 22:00 incoming Andrew Morton
                   ` (40 preceding siblings ...)
  2020-12-18 22:03 ` [patch 41/78] arm64: mte: convert gcr_user into an exclude mask Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 43/78] kasan, mm: untag page address in free_reserved_area Andrew Morton
                   ` (35 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: arm64: mte: switch GCR_EL1 in kernel entry and exit

When MTE is present, the GCR_EL1 register contains the tags mask that
allows to exclude tags from the random generation via the IRG instruction.

With the introduction of the new Tag-Based KASAN API that provides a
mechanism to reserve tags for special reasons, the MTE implementation has
to make sure that the GCR_EL1 setting for the kernel does not affect the
userspace processes and viceversa.

Save and restore the kernel/user mask in GCR_EL1 in kernel entry and exit.

Link: https://lkml.kernel.org/r/578b03294708cc7258fad0dc9c2a2e809e5a8214.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/mte-def.h   |    1 
 arch/arm64/include/asm/mte-kasan.h |    5 +++
 arch/arm64/include/asm/mte.h       |    2 +
 arch/arm64/kernel/asm-offsets.c    |    3 +
 arch/arm64/kernel/entry.S          |   41 +++++++++++++++++++++++++++
 arch/arm64/kernel/mte.c            |   31 ++++++++++++++++++--
 6 files changed, 79 insertions(+), 4 deletions(-)

--- a/arch/arm64/include/asm/mte-def.h~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit
+++ a/arch/arm64/include/asm/mte-def.h
@@ -10,6 +10,5 @@
 #define MTE_TAG_SHIFT		56
 #define MTE_TAG_SIZE		4
 #define MTE_TAG_MASK		GENMASK((MTE_TAG_SHIFT + (MTE_TAG_SIZE - 1)), MTE_TAG_SHIFT)
-#define MTE_TAG_MAX		(MTE_TAG_MASK >> MTE_TAG_SHIFT)
 
 #endif /* __ASM_MTE_DEF_H  */
--- a/arch/arm64/include/asm/mte.h~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit
+++ a/arch/arm64/include/asm/mte.h
@@ -18,6 +18,8 @@
 
 #include <asm/pgtable-types.h>
 
+extern u64 gcr_kernel_excl;
+
 void mte_clear_page_tags(void *addr);
 unsigned long mte_copy_tags_from_user(void *to, const void __user *from,
 				      unsigned long n);
--- a/arch/arm64/include/asm/mte-kasan.h~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit
+++ a/arch/arm64/include/asm/mte-kasan.h
@@ -30,6 +30,7 @@ u8 mte_get_random_tag(void);
 void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag);
 
 void mte_enable_kernel(void);
+void mte_init_tags(u64 max_tag);
 
 #else /* CONFIG_ARM64_MTE */
 
@@ -55,6 +56,10 @@ static inline void mte_enable_kernel(voi
 {
 }
 
+static inline void mte_init_tags(u64 max_tag)
+{
+}
+
 #endif /* CONFIG_ARM64_MTE */
 
 #endif /* __ASSEMBLY__ */
--- a/arch/arm64/kernel/asm-offsets.c~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit
+++ a/arch/arm64/kernel/asm-offsets.c
@@ -47,6 +47,9 @@ int main(void)
   DEFINE(THREAD_KEYS_USER,	offsetof(struct task_struct, thread.keys_user));
   DEFINE(THREAD_KEYS_KERNEL,	offsetof(struct task_struct, thread.keys_kernel));
 #endif
+#ifdef CONFIG_ARM64_MTE
+  DEFINE(THREAD_GCR_EL1_USER,	offsetof(struct task_struct, thread.gcr_user_excl));
+#endif
   BLANK();
   DEFINE(S_X0,			offsetof(struct pt_regs, regs[0]));
   DEFINE(S_X2,			offsetof(struct pt_regs, regs[2]));
--- a/arch/arm64/kernel/entry.S~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit
+++ a/arch/arm64/kernel/entry.S
@@ -173,6 +173,43 @@ alternative_else_nop_endif
 #endif
 	.endm
 
+	.macro mte_set_gcr, tmp, tmp2
+#ifdef CONFIG_ARM64_MTE
+	/*
+	 * Calculate and set the exclude mask preserving
+	 * the RRND (bit[16]) setting.
+	 */
+	mrs_s	\tmp2, SYS_GCR_EL1
+	bfi	\tmp2, \tmp, #0, #16
+	msr_s	SYS_GCR_EL1, \tmp2
+	isb
+#endif
+	.endm
+
+	.macro mte_set_kernel_gcr, tmp, tmp2
+#ifdef CONFIG_KASAN_HW_TAGS
+alternative_if_not ARM64_MTE
+	b	1f
+alternative_else_nop_endif
+	ldr_l	\tmp, gcr_kernel_excl
+
+	mte_set_gcr \tmp, \tmp2
+1:
+#endif
+	.endm
+
+	.macro mte_set_user_gcr, tsk, tmp, tmp2
+#ifdef CONFIG_ARM64_MTE
+alternative_if_not ARM64_MTE
+	b	1f
+alternative_else_nop_endif
+	ldr	\tmp, [\tsk, #THREAD_GCR_EL1_USER]
+
+	mte_set_gcr \tmp, \tmp2
+1:
+#endif
+	.endm
+
 	.macro	kernel_entry, el, regsize = 64
 	.if	\regsize == 32
 	mov	w0, w0				// zero upper 32 bits of x0
@@ -212,6 +249,8 @@ alternative_else_nop_endif
 
 	ptrauth_keys_install_kernel tsk, x20, x22, x23
 
+	mte_set_kernel_gcr x22, x23
+
 	scs_load tsk, x20
 	.else
 	add	x21, sp, #S_FRAME_SIZE
@@ -315,6 +354,8 @@ alternative_else_nop_endif
 	/* No kernel C function calls after this as user keys are set. */
 	ptrauth_keys_install_user tsk, x0, x1, x2
 
+	mte_set_user_gcr tsk, x0, x1
+
 	apply_ssbd 0, x0, x1
 	.endif
 
--- a/arch/arm64/kernel/mte.c~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit
+++ a/arch/arm64/kernel/mte.c
@@ -23,6 +23,8 @@
 #include <asm/ptrace.h>
 #include <asm/sysreg.h>
 
+u64 gcr_kernel_excl __ro_after_init;
+
 static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_swap)
 {
 	pte_t old_pte = READ_ONCE(*ptep);
@@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr,
 	return ptr;
 }
 
+void mte_init_tags(u64 max_tag)
+{
+	static bool gcr_kernel_excl_initialized;
+
+	if (!gcr_kernel_excl_initialized) {
+		/*
+		 * The format of the tags in KASAN is 0xFF and in MTE is 0xF.
+		 * This conversion extracts an MTE tag from a KASAN tag.
+		 */
+		u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT,
+					     max_tag), 0);
+
+		gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
+		gcr_kernel_excl_initialized = true;
+	}
+
+	/* Enable the kernel exclude mask for random tags generation. */
+	write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1);
+}
+
 void mte_enable_kernel(void)
 {
 	/* Enable MTE Sync Mode for EL1. */
@@ -171,7 +193,11 @@ static void update_gcr_el1_excl(u64 excl
 static void set_gcr_el1_excl(u64 excl)
 {
 	current->thread.gcr_user_excl = excl;
-	update_gcr_el1_excl(excl);
+
+	/*
+	 * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value
+	 * by mte_set_user_gcr() in kernel_exit,
+	 */
 }
 
 void flush_mte_state(void)
@@ -197,7 +223,6 @@ void mte_thread_switch(struct task_struc
 	/* avoid expensive SCTLR_EL1 accesses if no change */
 	if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0)
 		update_sctlr_el1_tcf0(next->thread.sctlr_tcf0);
-	update_gcr_el1_excl(next->thread.gcr_user_excl);
 }
 
 void mte_suspend_exit(void)
@@ -205,7 +230,7 @@ void mte_suspend_exit(void)
 	if (!system_supports_mte())
 		return;
 
-	update_gcr_el1_excl(current->thread.gcr_user_excl);
+	update_gcr_el1_excl(gcr_kernel_excl);
 }
 
 long set_mte_ctrl(struct task_struct *task, unsigned long arg)
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 43/78] kasan, mm: untag page address in free_reserved_area
  2020-12-18 22:00 incoming Andrew Morton
                   ` (41 preceding siblings ...)
  2020-12-18 22:03 ` [patch 42/78] arm64: mte: switch GCR_EL1 in kernel entry and exit Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 44/78] arm64: kasan: align allocations for HW_TAGS Andrew Morton
                   ` (34 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: kasan, mm: untag page address in free_reserved_area

free_reserved_area() memsets the pages belonging to a given memory area. 
As that memory hasn't been allocated via page_alloc, the KASAN tags that
those pages have are 0x00.  As the result the memset might result in a tag
mismatch.

Untag the address to avoid spurious faults.

Link: https://lkml.kernel.org/r/ebef6425f4468d063e2f09c1b62ccbb2236b71d3.1606161801.git.andreyknvl@google.com
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/page_alloc.c |    5 +++++
 1 file changed, 5 insertions(+)

--- a/mm/page_alloc.c~kasan-mm-untag-page-address-in-free_reserved_area
+++ a/mm/page_alloc.c
@@ -7671,6 +7671,11 @@ unsigned long free_reserved_area(void *s
 		 * alias for the memset().
 		 */
 		direct_map_addr = page_address(page);
+		/*
+		 * Perform a kasan-unchecked memset() since this memory
+		 * has not been initialized.
+		 */
+		direct_map_addr = kasan_reset_tag(direct_map_addr);
 		if ((unsigned int)poison <= 0xFF)
 			memset(direct_map_addr, poison, PAGE_SIZE);
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 44/78] arm64: kasan: align allocations for HW_TAGS
  2020-12-18 22:00 incoming Andrew Morton
                   ` (42 preceding siblings ...)
  2020-12-18 22:03 ` [patch 43/78] kasan, mm: untag page address in free_reserved_area Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 45/78] arm64: kasan: add arch layer for memory tagging helpers Andrew Morton
                   ` (33 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: arm64: kasan: align allocations for HW_TAGS

Hardware tag-based KASAN uses the memory tagging approach, which requires
all allocations to be aligned to the memory granule size.  Align the
allocations to MTE_GRANULE_SIZE via ARCH_SLAB_MINALIGN when
CONFIG_KASAN_HW_TAGS is enabled.

Link: https://lkml.kernel.org/r/fe64131606b1c2aabfd34ae99554c0d9df18eb19.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/cache.h |    3 +++
 1 file changed, 3 insertions(+)

--- a/arch/arm64/include/asm/cache.h~arm64-kasan-align-allocations-for-hw_tags
+++ a/arch/arm64/include/asm/cache.h
@@ -6,6 +6,7 @@
 #define __ASM_CACHE_H
 
 #include <asm/cputype.h>
+#include <asm/mte-kasan.h>
 
 #define CTR_L1IP_SHIFT		14
 #define CTR_L1IP_MASK		3
@@ -51,6 +52,8 @@
 
 #ifdef CONFIG_KASAN_SW_TAGS
 #define ARCH_SLAB_MINALIGN	(1ULL << KASAN_SHADOW_SCALE_SHIFT)
+#elif defined(CONFIG_KASAN_HW_TAGS)
+#define ARCH_SLAB_MINALIGN	MTE_GRANULE_SIZE
 #endif
 
 #ifndef __ASSEMBLY__
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 45/78] arm64: kasan: add arch layer for memory tagging helpers
  2020-12-18 22:00 incoming Andrew Morton
                   ` (43 preceding siblings ...)
  2020-12-18 22:03 ` [patch 44/78] arm64: kasan: align allocations for HW_TAGS Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 46/78] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrew Morton
                   ` (32 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: arm64: kasan: add arch layer for memory tagging helpers

This patch add a set of arch_*() memory tagging helpers currently only
defined for arm64 when hardware tag-based KASAN is enabled.  These helpers
will be used by KASAN runtime to implement the hardware tag-based mode.

The arch-level indirection level is introduced to simplify adding hardware
tag-based KASAN support for other architectures in the future by defining
the appropriate arch_*() macros.

Link: https://lkml.kernel.org/r/fc9e5bb71201c03131a2fc00a74125723568dda9.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/memory.h |    9 +++++++++
 mm/kasan/kasan.h                |   26 ++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)

--- a/arch/arm64/include/asm/memory.h~arm64-kasan-add-arch-layer-for-memory-tagging-helpers
+++ a/arch/arm64/include/asm/memory.h
@@ -230,6 +230,15 @@ static inline const void *__tag_set(cons
 	return (const void *)(__addr | __tag_shifted(tag));
 }
 
+#ifdef CONFIG_KASAN_HW_TAGS
+#define arch_enable_tagging()			mte_enable_kernel()
+#define arch_init_tags(max_tag)			mte_init_tags(max_tag)
+#define arch_get_random_tag()			mte_get_random_tag()
+#define arch_get_mem_tag(addr)			mte_get_mem_tag(addr)
+#define arch_set_mem_tag_range(addr, size, tag)	\
+			mte_set_mem_tag_range((addr), (size), (tag))
+#endif /* CONFIG_KASAN_HW_TAGS */
+
 /*
  * Physical vs virtual RAM address space conversion.  These are
  * private definitions which should NOT be used outside memory.h
--- a/mm/kasan/kasan.h~arm64-kasan-add-arch-layer-for-memory-tagging-helpers
+++ a/mm/kasan/kasan.h
@@ -243,6 +243,32 @@ static inline const void *arch_kasan_set
 #define reset_tag(addr)		((void *)arch_kasan_reset_tag(addr))
 #define get_tag(addr)		arch_kasan_get_tag(addr)
 
+#ifdef CONFIG_KASAN_HW_TAGS
+
+#ifndef arch_enable_tagging
+#define arch_enable_tagging()
+#endif
+#ifndef arch_init_tags
+#define arch_init_tags(max_tag)
+#endif
+#ifndef arch_get_random_tag
+#define arch_get_random_tag()	(0xFF)
+#endif
+#ifndef arch_get_mem_tag
+#define arch_get_mem_tag(addr)	(0xFF)
+#endif
+#ifndef arch_set_mem_tag_range
+#define arch_set_mem_tag_range(addr, size, tag) ((void *)(addr))
+#endif
+
+#define hw_enable_tagging()			arch_enable_tagging()
+#define hw_init_tags(max_tag)			arch_init_tags(max_tag)
+#define hw_get_random_tag()			arch_get_random_tag()
+#define hw_get_mem_tag(addr)			arch_get_mem_tag(addr)
+#define hw_set_mem_tag_range(addr, size, tag)	arch_set_mem_tag_range((addr), (size), (tag))
+
+#endif /* CONFIG_KASAN_HW_TAGS */
+
 /*
  * Exported functions for interfaces called from assembly or from generated
  * code. Declarations here to avoid warning about missing declarations.
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 46/78] kasan: define KASAN_GRANULE_SIZE for HW_TAGS
  2020-12-18 22:00 incoming Andrew Morton
                   ` (44 preceding siblings ...)
  2020-12-18 22:03 ` [patch 45/78] arm64: kasan: add arch layer for memory tagging helpers Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:03 ` [patch 47/78] kasan, x86, s390: update undef CONFIG_KASAN Andrew Morton
                   ` (31 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: define KASAN_GRANULE_SIZE for HW_TAGS

Hardware tag-based KASAN has granules of MTE_GRANULE_SIZE.  Define
KASAN_GRANULE_SIZE to MTE_GRANULE_SIZE for CONFIG_KASAN_HW_TAGS.

Link: https://lkml.kernel.org/r/3d15794b3d1b27447fd7fdf862c073192ba657bd.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/kasan.h |    6 ++++++
 1 file changed, 6 insertions(+)

--- a/mm/kasan/kasan.h~kasan-define-kasan_granule_size-for-hw_tags
+++ a/mm/kasan/kasan.h
@@ -5,7 +5,13 @@
 #include <linux/kasan.h>
 #include <linux/stackdepot.h>
 
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
+#else
+#include <asm/mte-kasan.h>
+#define KASAN_GRANULE_SIZE	MTE_GRANULE_SIZE
+#endif
+
 #define KASAN_GRANULE_MASK	(KASAN_GRANULE_SIZE - 1)
 
 #define KASAN_MEMORY_PER_SHADOW_PAGE	(KASAN_GRANULE_SIZE << PAGE_SHIFT)
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 47/78] kasan, x86, s390: update undef CONFIG_KASAN
  2020-12-18 22:00 incoming Andrew Morton
                   ` (45 preceding siblings ...)
  2020-12-18 22:03 ` [patch 46/78] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrew Morton
@ 2020-12-18 22:03 ` Andrew Morton
  2020-12-18 22:04 ` [patch 48/78] kasan, arm64: expand CONFIG_KASAN checks Andrew Morton
                   ` (30 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:03 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, x86, s390: update undef CONFIG_KASAN

With the intoduction of hardware tag-based KASAN some kernel checks of
this kind:

  ifdef CONFIG_KASAN

will be updated to:

  if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)

x86 and s390 use a trick to #undef CONFIG_KASAN for some of the code
that isn't linked with KASAN runtime and shouldn't have any KASAN
annotations.

Also #undef CONFIG_KASAN_GENERIC with CONFIG_KASAN.

Link: https://lkml.kernel.org/r/9d84bfaaf8fabe0fc89f913c9e420a30bd31a260.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/s390/boot/string.c         |    1 +
 arch/x86/boot/compressed/misc.h |    1 +
 2 files changed, 2 insertions(+)

--- a/arch/s390/boot/string.c~kasan-x86-s390-update-undef-config_kasan
+++ a/arch/s390/boot/string.c
@@ -3,6 +3,7 @@
 #include <linux/kernel.h>
 #include <linux/errno.h>
 #undef CONFIG_KASAN
+#undef CONFIG_KASAN_GENERIC
 #include "../lib/string.c"
 
 int strncmp(const char *cs, const char *ct, size_t count)
--- a/arch/x86/boot/compressed/misc.h~kasan-x86-s390-update-undef-config_kasan
+++ a/arch/x86/boot/compressed/misc.h
@@ -12,6 +12,7 @@
 #undef CONFIG_PARAVIRT_XXL
 #undef CONFIG_PARAVIRT_SPINLOCKS
 #undef CONFIG_KASAN
+#undef CONFIG_KASAN_GENERIC
 
 /* cpu_feature_enabled() cannot be used this early */
 #define USE_EARLY_PGTABLE_L5
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 48/78] kasan, arm64: expand CONFIG_KASAN checks
  2020-12-18 22:00 incoming Andrew Morton
                   ` (46 preceding siblings ...)
  2020-12-18 22:03 ` [patch 47/78] kasan, x86, s390: update undef CONFIG_KASAN Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 49/78] kasan, arm64: implement HW_TAGS runtime Andrew Morton
                   ` (29 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: expand CONFIG_KASAN checks

Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN modes
(either related to shadow memory or compiler instrumentation).  Expand
those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS.

Link: https://lkml.kernel.org/r/e6971e432dbd72bb897ff14134ebb7e169bdcf0c.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/Kconfig                 |    2 +-
 arch/arm64/Makefile                |    2 +-
 arch/arm64/include/asm/assembler.h |    2 +-
 arch/arm64/include/asm/memory.h    |    2 +-
 arch/arm64/include/asm/string.h    |    5 +++--
 arch/arm64/kernel/head.S           |    2 +-
 arch/arm64/kernel/image-vars.h     |    2 +-
 arch/arm64/kernel/kaslr.c          |    3 ++-
 arch/arm64/kernel/module.c         |    6 ++++--
 arch/arm64/mm/ptdump.c             |    6 +++---
 include/linux/kasan-checks.h       |    2 +-
 include/linux/kasan.h              |    7 ++++---
 include/linux/moduleloader.h       |    3 ++-
 include/linux/string.h             |    2 +-
 mm/ptdump.c                        |   13 ++++++++-----
 scripts/Makefile.lib               |    2 ++
 16 files changed, 36 insertions(+), 25 deletions(-)

--- a/arch/arm64/include/asm/assembler.h~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/include/asm/assembler.h
@@ -473,7 +473,7 @@ USER(\label, ic	ivau, \tmp2)			// invali
 #define NOKPROBE(x)
 #endif
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define EXPORT_SYMBOL_NOKASAN(name)
 #else
 #define EXPORT_SYMBOL_NOKASAN(name)	EXPORT_SYMBOL(name)
--- a/arch/arm64/include/asm/memory.h~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/include/asm/memory.h
@@ -72,7 +72,7 @@
  * address space for the shadow region respectively. They can bloat the stack
  * significantly, so double the (minimum) stack size when they are in use.
  */
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define KASAN_SHADOW_OFFSET	_AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
 #define KASAN_SHADOW_END	((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) \
 					+ KASAN_SHADOW_OFFSET)
--- a/arch/arm64/include/asm/string.h~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/include/asm/string.h
@@ -5,7 +5,7 @@
 #ifndef __ASM_STRING_H
 #define __ASM_STRING_H
 
-#ifndef CONFIG_KASAN
+#if !(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 #define __HAVE_ARCH_STRRCHR
 extern char *strrchr(const char *, int c);
 
@@ -48,7 +48,8 @@ extern void *__memset(void *, int, __ker
 void memcpy_flushcache(void *dst, const void *src, size_t cnt);
 #endif
 
-#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+	!defined(__SANITIZE_ADDRESS__)
 
 /*
  * For files that are not instrumented (e.g. mm/slub.c) we
--- a/arch/arm64/Kconfig~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/Kconfig
@@ -334,7 +334,7 @@ config BROKEN_GAS_INST
 
 config KASAN_SHADOW_OFFSET
 	hex
-	depends on KASAN
+	depends on KASAN_GENERIC || KASAN_SW_TAGS
 	default 0xdfff800000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && !KASAN_SW_TAGS
 	default 0xdfffc00000000000 if ARM64_VA_BITS_47 && !KASAN_SW_TAGS
 	default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS
--- a/arch/arm64/kernel/head.S~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/kernel/head.S
@@ -433,7 +433,7 @@ SYM_FUNC_START_LOCAL(__primary_switched)
 	bl	__pi_memset
 	dsb	ishst				// Make zero page visible to PTW
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	bl	kasan_early_init
 #endif
 #ifdef CONFIG_RANDOMIZE_BASE
--- a/arch/arm64/kernel/image-vars.h~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/kernel/image-vars.h
@@ -37,7 +37,7 @@ __efistub_strncmp		= __pi_strncmp;
 __efistub_strrchr		= __pi_strrchr;
 __efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc;
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 __efistub___memcpy		= __pi_memcpy;
 __efistub___memmove		= __pi_memmove;
 __efistub___memset		= __pi_memset;
--- a/arch/arm64/kernel/kaslr.c~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/kernel/kaslr.c
@@ -161,7 +161,8 @@ u64 __init kaslr_early_init(u64 dt_phys)
 	/* use the top 16 bits to randomize the linear region */
 	memstart_offset_seed = seed >> 48;
 
-	if (IS_ENABLED(CONFIG_KASAN))
+	if (IS_ENABLED(CONFIG_KASAN_GENERIC) ||
+	    IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		/*
 		 * KASAN does not expect the module region to intersect the
 		 * vmalloc region, since shadow memory is allocated for each
--- a/arch/arm64/kernel/module.c~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/kernel/module.c
@@ -30,7 +30,8 @@ void *module_alloc(unsigned long size)
 	if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS))
 		gfp_mask |= __GFP_NOWARN;
 
-	if (IS_ENABLED(CONFIG_KASAN))
+	if (IS_ENABLED(CONFIG_KASAN_GENERIC) ||
+	    IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		/* don't exceed the static module region - see below */
 		module_alloc_end = MODULES_END;
 
@@ -39,7 +40,8 @@ void *module_alloc(unsigned long size)
 				NUMA_NO_NODE, __builtin_return_address(0));
 
 	if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) &&
-	    !IS_ENABLED(CONFIG_KASAN))
+	    !IS_ENABLED(CONFIG_KASAN_GENERIC) &&
+	    !IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		/*
 		 * KASAN can only deal with module allocations being served
 		 * from the reserved module region, since the remainder of
--- a/arch/arm64/Makefile~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/Makefile
@@ -137,7 +137,7 @@ head-y		:= arch/arm64/kernel/head.o
 
 ifeq ($(CONFIG_KASAN_SW_TAGS), y)
 KASAN_SHADOW_SCALE_SHIFT := 4
-else
+else ifeq ($(CONFIG_KASAN_GENERIC), y)
 KASAN_SHADOW_SCALE_SHIFT := 3
 endif
 
--- a/arch/arm64/mm/ptdump.c~kasan-arm64-expand-config_kasan-checks
+++ a/arch/arm64/mm/ptdump.c
@@ -29,7 +29,7 @@
 enum address_markers_idx {
 	PAGE_OFFSET_NR = 0,
 	PAGE_END_NR,
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	KASAN_START_NR,
 #endif
 };
@@ -37,7 +37,7 @@ enum address_markers_idx {
 static struct addr_marker address_markers[] = {
 	{ PAGE_OFFSET,			"Linear Mapping start" },
 	{ 0 /* PAGE_END */,		"Linear Mapping end" },
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	{ 0 /* KASAN_SHADOW_START */,	"Kasan shadow start" },
 	{ KASAN_SHADOW_END,		"Kasan shadow end" },
 #endif
@@ -383,7 +383,7 @@ void ptdump_check_wx(void)
 static int ptdump_init(void)
 {
 	address_markers[PAGE_END_NR].start_address = PAGE_END;
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	address_markers[KASAN_START_NR].start_address = KASAN_SHADOW_START;
 #endif
 	ptdump_initialize();
--- a/include/linux/kasan-checks.h~kasan-arm64-expand-config_kasan-checks
+++ a/include/linux/kasan-checks.h
@@ -9,7 +9,7 @@
  * even in compilation units that selectively disable KASAN, but must use KASAN
  * to validate access to an address.   Never use these in header files!
  */
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 bool __kasan_check_read(const volatile void *p, unsigned int size);
 bool __kasan_check_write(const volatile void *p, unsigned int size);
 #else
--- a/include/linux/kasan.h~kasan-arm64-expand-config_kasan-checks
+++ a/include/linux/kasan.h
@@ -238,7 +238,8 @@ static inline void kasan_release_vmalloc
 
 #endif /* CONFIG_KASAN_VMALLOC */
 
-#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+		!defined(CONFIG_KASAN_VMALLOC)
 
 /*
  * These functions provide a special case to support backing module
@@ -248,12 +249,12 @@ static inline void kasan_release_vmalloc
 int kasan_module_alloc(void *addr, size_t size);
 void kasan_free_shadow(const struct vm_struct *vm);
 
-#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
+#else /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */
 
 static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
 static inline void kasan_free_shadow(const struct vm_struct *vm) {}
 
-#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
+#endif /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */
 
 #ifdef CONFIG_KASAN_INLINE
 void kasan_non_canonical_hook(unsigned long addr);
--- a/include/linux/moduleloader.h~kasan-arm64-expand-config_kasan-checks
+++ a/include/linux/moduleloader.h
@@ -96,7 +96,8 @@ void module_arch_cleanup(struct module *
 /* Any cleanup before freeing mod->module_init */
 void module_arch_freeing_init(struct module *mod);
 
-#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+		!defined(CONFIG_KASAN_VMALLOC)
 #include <linux/kasan.h>
 #define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT)
 #else
--- a/include/linux/string.h~kasan-arm64-expand-config_kasan-checks
+++ a/include/linux/string.h
@@ -267,7 +267,7 @@ void __write_overflow(void) __compiletim
 
 #if !defined(__NO_FORTIFY) && defined(__OPTIMIZE__) && defined(CONFIG_FORTIFY_SOURCE)
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 extern void *__underlying_memchr(const void *p, int c, __kernel_size_t size) __RENAME(memchr);
 extern int __underlying_memcmp(const void *p, const void *q, __kernel_size_t size) __RENAME(memcmp);
 extern void *__underlying_memcpy(void *p, const void *q, __kernel_size_t size) __RENAME(memcpy);
--- a/mm/ptdump.c~kasan-arm64-expand-config_kasan-checks
+++ a/mm/ptdump.c
@@ -4,7 +4,7 @@
 #include <linux/ptdump.h>
 #include <linux/kasan.h>
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 /*
  * This is an optimization for KASAN=y case. Since all kasan page tables
  * eventually point to the kasan_early_shadow_page we could call note_page()
@@ -31,7 +31,8 @@ static int ptdump_pgd_entry(pgd_t *pgd,
 	struct ptdump_state *st = walk->private;
 	pgd_t val = READ_ONCE(*pgd);
 
-#if CONFIG_PGTABLE_LEVELS > 4 && defined(CONFIG_KASAN)
+#if CONFIG_PGTABLE_LEVELS > 4 && \
+		(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 	if (pgd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_p4d)))
 		return note_kasan_page_table(walk, addr);
 #endif
@@ -51,7 +52,8 @@ static int ptdump_p4d_entry(p4d_t *p4d,
 	struct ptdump_state *st = walk->private;
 	p4d_t val = READ_ONCE(*p4d);
 
-#if CONFIG_PGTABLE_LEVELS > 3 && defined(CONFIG_KASAN)
+#if CONFIG_PGTABLE_LEVELS > 3 && \
+		(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 	if (p4d_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pud)))
 		return note_kasan_page_table(walk, addr);
 #endif
@@ -71,7 +73,8 @@ static int ptdump_pud_entry(pud_t *pud,
 	struct ptdump_state *st = walk->private;
 	pud_t val = READ_ONCE(*pud);
 
-#if CONFIG_PGTABLE_LEVELS > 2 && defined(CONFIG_KASAN)
+#if CONFIG_PGTABLE_LEVELS > 2 && \
+		(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 	if (pud_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pmd)))
 		return note_kasan_page_table(walk, addr);
 #endif
@@ -91,7 +94,7 @@ static int ptdump_pmd_entry(pmd_t *pmd,
 	struct ptdump_state *st = walk->private;
 	pmd_t val = READ_ONCE(*pmd);
 
-#if defined(CONFIG_KASAN)
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	if (pmd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pte)))
 		return note_kasan_page_table(walk, addr);
 #endif
--- a/scripts/Makefile.lib~kasan-arm64-expand-config_kasan-checks
+++ a/scripts/Makefile.lib
@@ -148,10 +148,12 @@ endif
 # we don't want to check (depends on variables KASAN_SANITIZE_obj.o, KASAN_SANITIZE)
 #
 ifeq ($(CONFIG_KASAN),y)
+ifneq ($(CONFIG_KASAN_HW_TAGS),y)
 _c_flags += $(if $(patsubst n%,, \
 		$(KASAN_SANITIZE_$(basetarget).o)$(KASAN_SANITIZE)y), \
 		$(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE))
 endif
+endif
 
 ifeq ($(CONFIG_UBSAN),y)
 _c_flags += $(if $(patsubst n%,, \
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 49/78] kasan, arm64: implement HW_TAGS runtime
  2020-12-18 22:00 incoming Andrew Morton
                   ` (47 preceding siblings ...)
  2020-12-18 22:04 ` [patch 48/78] kasan, arm64: expand CONFIG_KASAN checks Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 50/78] kasan, arm64: print report from tag fault handler Andrew Morton
                   ` (28 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: implement HW_TAGS runtime

Provide implementation of KASAN functions required for the hardware
tag-based mode.  Those include core functions for memory and pointer
tagging (tags_hw.c) and bug reporting (report_tags_hw.c).  Also adapt
common KASAN code to support the new mode.

Link: https://lkml.kernel.org/r/cfd0fbede579a6b66755c98c88c108e54f9c56bf.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/include/asm/memory.h   |    4 -
 arch/arm64/kernel/cpufeature.c    |    3 
 arch/arm64/kernel/smp.c           |    2 
 include/linux/kasan.h             |   24 +++++--
 include/linux/mm.h                |    2 
 include/linux/page-flags-layout.h |    2 
 mm/kasan/Makefile                 |    5 +
 mm/kasan/common.c                 |   15 ++--
 mm/kasan/hw_tags.c                |   89 ++++++++++++++++++++++++++++
 mm/kasan/kasan.h                  |   19 ++++-
 mm/kasan/report_hw_tags.c         |   42 +++++++++++++
 mm/kasan/report_sw_tags.c         |    2 
 mm/kasan/shadow.c                 |    2 
 mm/kasan/sw_tags.c                |    2 
 14 files changed, 187 insertions(+), 26 deletions(-)

--- a/arch/arm64/include/asm/memory.h~kasan-arm64-implement-hw_tags-runtime
+++ a/arch/arm64/include/asm/memory.h
@@ -214,7 +214,7 @@ static inline unsigned long kaslr_offset
 	(__force __typeof__(addr))__addr;				\
 })
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 #define __tag_shifted(tag)	((u64)(tag) << 56)
 #define __tag_reset(addr)	__untagged_addr(addr)
 #define __tag_get(addr)		(__u8)((u64)(addr) >> 56)
@@ -222,7 +222,7 @@ static inline unsigned long kaslr_offset
 #define __tag_shifted(tag)	0UL
 #define __tag_reset(addr)	(addr)
 #define __tag_get(addr)		0
-#endif /* CONFIG_KASAN_SW_TAGS */
+#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
 
 static inline const void *__tag_set(const void *addr, u8 tag)
 {
--- a/arch/arm64/kernel/cpufeature.c~kasan-arm64-implement-hw_tags-runtime
+++ a/arch/arm64/kernel/cpufeature.c
@@ -70,6 +70,7 @@
 #include <linux/types.h>
 #include <linux/mm.h>
 #include <linux/cpu.h>
+#include <linux/kasan.h>
 #include <asm/cpu.h>
 #include <asm/cpufeature.h>
 #include <asm/cpu_ops.h>
@@ -1709,6 +1710,8 @@ static void cpu_enable_mte(struct arm64_
 		cleared_zero_page = true;
 		mte_clear_page_tags(lm_alias(empty_zero_page));
 	}
+
+	kasan_init_hw_tags_cpu();
 }
 #endif /* CONFIG_ARM64_MTE */
 
--- a/arch/arm64/kernel/smp.c~kasan-arm64-implement-hw_tags-runtime
+++ a/arch/arm64/kernel/smp.c
@@ -462,6 +462,8 @@ void __init smp_prepare_boot_cpu(void)
 	/* Conditionally switch to GIC PMR for interrupt masking */
 	if (system_uses_irq_prio_masking())
 		init_gic_priority_masking();
+
+	kasan_init_hw_tags();
 }
 
 static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--- a/include/linux/kasan.h~kasan-arm64-implement-hw_tags-runtime
+++ a/include/linux/kasan.h
@@ -190,25 +190,35 @@ static inline void kasan_record_aux_stac
 
 #endif /* CONFIG_KASAN_GENERIC */
 
-#ifdef CONFIG_KASAN_SW_TAGS
-
-void __init kasan_init_sw_tags(void);
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 
 void *kasan_reset_tag(const void *addr);
 
 bool kasan_report(unsigned long addr, size_t size,
 		bool is_write, unsigned long ip);
 
-#else /* CONFIG_KASAN_SW_TAGS */
-
-static inline void kasan_init_sw_tags(void) { }
+#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
 
 static inline void *kasan_reset_tag(const void *addr)
 {
 	return (void *)addr;
 }
 
-#endif /* CONFIG_KASAN_SW_TAGS */
+#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS*/
+
+#ifdef CONFIG_KASAN_SW_TAGS
+void __init kasan_init_sw_tags(void);
+#else
+static inline void kasan_init_sw_tags(void) { }
+#endif
+
+#ifdef CONFIG_KASAN_HW_TAGS
+void kasan_init_hw_tags_cpu(void);
+void __init kasan_init_hw_tags(void);
+#else
+static inline void kasan_init_hw_tags_cpu(void) { }
+static inline void kasan_init_hw_tags(void) { }
+#endif
 
 #ifdef CONFIG_KASAN_VMALLOC
 
--- a/include/linux/mm.h~kasan-arm64-implement-hw_tags-runtime
+++ a/include/linux/mm.h
@@ -1421,7 +1421,7 @@ static inline bool cpupid_match_pid(stru
 }
 #endif /* CONFIG_NUMA_BALANCING */
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 static inline u8 page_kasan_tag(const struct page *page)
 {
 	return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
--- a/include/linux/page-flags-layout.h~kasan-arm64-implement-hw_tags-runtime
+++ a/include/linux/page-flags-layout.h
@@ -77,7 +77,7 @@
 #define LAST_CPUPID_SHIFT 0
 #endif
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 #define KASAN_TAG_WIDTH 8
 #else
 #define KASAN_TAG_WIDTH 0
--- a/mm/kasan/common.c~kasan-arm64-implement-hw_tags-runtime
+++ a/mm/kasan/common.c
@@ -118,7 +118,7 @@ void kasan_free_pages(struct page *page,
  */
 static inline unsigned int optimal_redzone(unsigned int object_size)
 {
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
+	if (!IS_ENABLED(CONFIG_KASAN_GENERIC))
 		return 0;
 
 	return
@@ -183,14 +183,14 @@ size_t kasan_metadata_size(struct kmem_c
 struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
 					const void *object)
 {
-	return (void *)object + cache->kasan_info.alloc_meta_offset;
+	return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset;
 }
 
 struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
 				      const void *object)
 {
 	BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
-	return (void *)object + cache->kasan_info.free_meta_offset;
+	return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset;
 }
 
 void kasan_poison_slab(struct page *page)
@@ -272,9 +272,8 @@ void * __must_check kasan_init_slab_obj(
 	alloc_info = get_alloc_info(cache, object);
 	__memset(alloc_info, 0, sizeof(*alloc_info));
 
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
-		object = set_tag(object,
-				assign_tag(cache, object, true, false));
+	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
+		object = set_tag(object, assign_tag(cache, object, true, false));
 
 	return (void *)object;
 }
@@ -342,10 +341,10 @@ static void *__kasan_kmalloc(struct kmem
 	redzone_end = round_up((unsigned long)object + cache->object_size,
 				KASAN_GRANULE_SIZE);
 
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
+	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
 		tag = assign_tag(cache, object, false, keep_tag);
 
-	/* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
+	/* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */
 	unpoison_range(set_tag(object, tag), size);
 	poison_range((void *)redzone_start, redzone_end - redzone_start,
 		     KASAN_KMALLOC_REDZONE);
--- /dev/null
+++ a/mm/kasan/hw_tags.c
@@ -0,0 +1,89 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This file contains core hardware tag-based KASAN code.
+ *
+ * Copyright (c) 2020 Google, Inc.
+ * Author: Andrey Konovalov <andreyknvl@google.com>
+ */
+
+#define pr_fmt(fmt) "kasan: " fmt
+
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/kfence.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "kasan.h"
+
+/* kasan_init_hw_tags_cpu() is called for each CPU. */
+void kasan_init_hw_tags_cpu(void)
+{
+	hw_init_tags(KASAN_TAG_MAX);
+	hw_enable_tagging();
+}
+
+/* kasan_init_hw_tags() is called once on boot CPU. */
+void __init kasan_init_hw_tags(void)
+{
+	pr_info("KernelAddressSanitizer initialized\n");
+}
+
+void *kasan_reset_tag(const void *addr)
+{
+	return reset_tag(addr);
+}
+
+void poison_range(const void *address, size_t size, u8 value)
+{
+	/* Skip KFENCE memory if called explicitly outside of sl*b. */
+	if (is_kfence_address(address))
+		return;
+
+	hw_set_mem_tag_range(reset_tag(address),
+			round_up(size, KASAN_GRANULE_SIZE), value);
+}
+
+void unpoison_range(const void *address, size_t size)
+{
+	/* Skip KFENCE memory if called explicitly outside of sl*b. */
+	if (is_kfence_address(address))
+		return;
+
+	hw_set_mem_tag_range(reset_tag(address),
+			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
+}
+
+u8 random_tag(void)
+{
+	return hw_get_random_tag();
+}
+
+bool check_invalid_free(void *addr)
+{
+	u8 ptr_tag = get_tag(addr);
+	u8 mem_tag = hw_get_mem_tag(addr);
+
+	return (mem_tag == KASAN_TAG_INVALID) ||
+		(ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
+}
+
+void kasan_set_free_info(struct kmem_cache *cache,
+				void *object, u8 tag)
+{
+	struct kasan_alloc_meta *alloc_meta;
+
+	alloc_meta = get_alloc_info(cache, object);
+	kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT);
+}
+
+struct kasan_track *kasan_get_free_track(struct kmem_cache *cache,
+				void *object, u8 tag)
+{
+	struct kasan_alloc_meta *alloc_meta;
+
+	alloc_meta = get_alloc_info(cache, object);
+	return &alloc_meta->free_track[0];
+}
--- a/mm/kasan/kasan.h~kasan-arm64-implement-hw_tags-runtime
+++ a/mm/kasan/kasan.h
@@ -154,6 +154,11 @@ struct kasan_alloc_meta *get_alloc_info(
 struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
 					const void *object);
 
+void poison_range(const void *address, size_t size, u8 value);
+void unpoison_range(const void *address, size_t size);
+
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
+
 static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
 {
 	return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET)
@@ -165,9 +170,6 @@ static inline bool addr_has_metadata(con
 	return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
 }
 
-void poison_range(const void *address, size_t size, u8 value);
-void unpoison_range(const void *address, size_t size);
-
 /**
  * check_memory_region - Check memory region, and report if invalid access.
  * @addr: the accessed address
@@ -179,6 +181,15 @@ void unpoison_range(const void *address,
 bool check_memory_region(unsigned long addr, size_t size, bool write,
 				unsigned long ret_ip);
 
+#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+static inline bool addr_has_metadata(const void *addr)
+{
+	return true;
+}
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
 bool check_invalid_free(void *addr);
 
 void *find_first_bad_addr(void *addr, size_t size);
@@ -215,7 +226,7 @@ static inline void quarantine_reduce(voi
 static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
 #endif
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 
 void print_tags(u8 addr_tag, const void *addr);
 
--- a/mm/kasan/Makefile~kasan-arm64-implement-hw_tags-runtime
+++ a/mm/kasan/Makefile
@@ -10,8 +10,10 @@ CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE
 CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report_hw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_hw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE)
 
 # Function splitter causes unnecessary splits in __asan_load1/__asan_store1
@@ -27,10 +29,13 @@ CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIM
 CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_report_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 
 obj-$(CONFIG_KASAN) := common.o report.o
 obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o
+obj-$(CONFIG_KASAN_HW_TAGS) += hw_tags.o report_hw_tags.o
 obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o
--- /dev/null
+++ a/mm/kasan/report_hw_tags.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This file contains hardware tag-based KASAN specific error reporting code.
+ *
+ * Copyright (c) 2020 Google, Inc.
+ * Author: Andrey Konovalov <andreyknvl@google.com>
+ */
+
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "kasan.h"
+
+const char *get_bug_type(struct kasan_access_info *info)
+{
+	return "invalid-access";
+}
+
+void *find_first_bad_addr(void *addr, size_t size)
+{
+	return reset_tag(addr);
+}
+
+void metadata_fetch_row(char *buffer, void *row)
+{
+	int i;
+
+	for (i = 0; i < META_BYTES_PER_ROW; i++)
+		buffer[i] = hw_get_mem_tag(row + i * KASAN_GRANULE_SIZE);
+}
+
+void print_tags(u8 addr_tag, const void *addr)
+{
+	u8 memory_tag = hw_get_mem_tag((void *)addr);
+
+	pr_err("Pointer tag: [%02x], memory tag: [%02x]\n",
+		addr_tag, memory_tag);
+}
--- a/mm/kasan/report_sw_tags.c~kasan-arm64-implement-hw_tags-runtime
+++ a/mm/kasan/report_sw_tags.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains tag-based KASAN specific error reporting code.
+ * This file contains software tag-based KASAN specific error reporting code.
  *
  * Copyright (c) 2014 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
--- a/mm/kasan/shadow.c~kasan-arm64-implement-hw_tags-runtime
+++ a/mm/kasan/shadow.c
@@ -120,7 +120,7 @@ void unpoison_range(const void *address,
 
 		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 			*shadow = tag;
-		else
+		else /* CONFIG_KASAN_GENERIC */
 			*shadow = size & KASAN_GRANULE_MASK;
 	}
 }
--- a/mm/kasan/sw_tags.c~kasan-arm64-implement-hw_tags-runtime
+++ a/mm/kasan/sw_tags.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains core tag-based KASAN code.
+ * This file contains core software tag-based KASAN code.
  *
  * Copyright (c) 2018 Google, Inc.
  * Author: Andrey Konovalov <andreyknvl@google.com>
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 50/78] kasan, arm64: print report from tag fault handler
  2020-12-18 22:00 incoming Andrew Morton
                   ` (48 preceding siblings ...)
  2020-12-18 22:04 ` [patch 49/78] kasan, arm64: implement HW_TAGS runtime Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 51/78] kasan, mm: reset tags when accessing metadata Andrew Morton
                   ` (27 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: print report from tag fault handler

Add error reporting for hardware tag-based KASAN.  When
CONFIG_KASAN_HW_TAGS is enabled, print KASAN report from the arm64 tag
fault handler.

SAS bits aren't set in ESR for all faults reported in EL1, so it's
impossible to find out the size of the access the caused the fault.  Adapt
KASAN reporting code to handle this case.

Link: https://lkml.kernel.org/r/b559c82b6a969afedf53b4694b475f0234067a1a.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/mm/fault.c |   14 ++++++++++++++
 mm/kasan/report.c     |   11 ++++++++---
 2 files changed, 22 insertions(+), 3 deletions(-)

--- a/arch/arm64/mm/fault.c~kasan-arm64-print-report-from-tag-fault-handler
+++ a/arch/arm64/mm/fault.c
@@ -14,6 +14,7 @@
 #include <linux/mm.h>
 #include <linux/hardirq.h>
 #include <linux/init.h>
+#include <linux/kasan.h>
 #include <linux/kprobes.h>
 #include <linux/uaccess.h>
 #include <linux/page-flags.h>
@@ -297,10 +298,23 @@ static void die_kernel_fault(const char
 	do_exit(SIGKILL);
 }
 
+#ifdef CONFIG_KASAN_HW_TAGS
 static void report_tag_fault(unsigned long addr, unsigned int esr,
 			     struct pt_regs *regs)
 {
+	bool is_write  = ((esr & ESR_ELx_WNR) >> ESR_ELx_WNR_SHIFT) != 0;
+
+	/*
+	 * SAS bits aren't set for all faults reported in EL1, so we can't
+	 * find out access size.
+	 */
+	kasan_report(addr, 0, is_write, regs->pc);
 }
+#else
+/* Tag faults aren't enabled without CONFIG_KASAN_HW_TAGS. */
+static inline void report_tag_fault(unsigned long addr, unsigned int esr,
+				    struct pt_regs *regs) { }
+#endif
 
 static void do_tag_recovery(unsigned long addr, unsigned int esr,
 			   struct pt_regs *regs)
--- a/mm/kasan/report.c~kasan-arm64-print-report-from-tag-fault-handler
+++ a/mm/kasan/report.c
@@ -62,9 +62,14 @@ static void print_error_description(stru
 {
 	pr_err("BUG: KASAN: %s in %pS\n",
 		get_bug_type(info), (void *)info->ip);
-	pr_err("%s of size %zu at addr %px by task %s/%d\n",
-		info->is_write ? "Write" : "Read", info->access_size,
-		info->access_addr, current->comm, task_pid_nr(current));
+	if (info->access_size)
+		pr_err("%s of size %zu at addr %px by task %s/%d\n",
+			info->is_write ? "Write" : "Read", info->access_size,
+			info->access_addr, current->comm, task_pid_nr(current));
+	else
+		pr_err("%s at addr %px by task %s/%d\n",
+			info->is_write ? "Write" : "Read",
+			info->access_addr, current->comm, task_pid_nr(current));
 }
 
 static DEFINE_SPINLOCK(report_lock);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 51/78] kasan, mm: reset tags when accessing metadata
  2020-12-18 22:00 incoming Andrew Morton
                   ` (49 preceding siblings ...)
  2020-12-18 22:04 ` [patch 50/78] kasan, arm64: print report from tag fault handler Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 52/78] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrew Morton
                   ` (26 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, mm: reset tags when accessing metadata

Kernel allocator code accesses metadata for slab objects, that may lie
out-of-bounds of the object itself, or be accessed when an object is
freed.  Such accesses trigger tag faults and lead to false-positive
reports with hardware tag-based KASAN.

Software KASAN modes disable instrumentation for allocator code via
KASAN_SANITIZE Makefile macro, and rely on kasan_enable/disable_current()
annotations which are used to ignore KASAN reports.

With hardware tag-based KASAN neither of those options are available, as
it doesn't use compiler instrumetation, no tag faults are ignored, and MTE
is disabled after the first one.

Instead, reset tags when accessing metadata (currently only for SLUB).

Link: https://lkml.kernel.org/r/a0f3cefbc49f34c843b664110842de4db28179d0.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Acked-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/page_alloc.c  |    4 +++-
 mm/page_poison.c |    2 +-
 mm/slub.c        |   29 ++++++++++++++++-------------
 3 files changed, 20 insertions(+), 15 deletions(-)

--- a/mm/page_alloc.c~kasan-mm-reset-tags-when-accessing-metadata
+++ a/mm/page_alloc.c
@@ -1204,8 +1204,10 @@ static void kernel_init_free_pages(struc
 
 	/* s390's use of memset() could override KASAN redzones. */
 	kasan_disable_current();
-	for (i = 0; i < numpages; i++)
+	for (i = 0; i < numpages; i++) {
+		page_kasan_tag_reset(page + i);
 		clear_highpage(page + i);
+	}
 	kasan_enable_current();
 }
 
--- a/mm/page_poison.c~kasan-mm-reset-tags-when-accessing-metadata
+++ a/mm/page_poison.c
@@ -25,7 +25,7 @@ static void poison_page(struct page *pag
 
 	/* KASAN still think the page is in-use, so skip it. */
 	kasan_disable_current();
-	memset(addr, PAGE_POISON, PAGE_SIZE);
+	memset(kasan_reset_tag(addr), PAGE_POISON, PAGE_SIZE);
 	kasan_enable_current();
 	kunmap_atomic(addr);
 }
--- a/mm/slub.c~kasan-mm-reset-tags-when-accessing-metadata
+++ a/mm/slub.c
@@ -249,7 +249,7 @@ static inline void *freelist_ptr(const s
 {
 #ifdef CONFIG_SLAB_FREELIST_HARDENED
 	/*
-	 * When CONFIG_KASAN_SW_TAGS is enabled, ptr_addr might be tagged.
+	 * When CONFIG_KASAN_SW/HW_TAGS is enabled, ptr_addr might be tagged.
 	 * Normally, this doesn't cause any issues, as both set_freepointer()
 	 * and get_freepointer() are called with a pointer with the same tag.
 	 * However, there are some issues with CONFIG_SLUB_DEBUG code. For
@@ -275,6 +275,7 @@ static inline void *freelist_dereference
 
 static inline void *get_freepointer(struct kmem_cache *s, void *object)
 {
+	object = kasan_reset_tag(object);
 	return freelist_dereference(s, object + s->offset);
 }
 
@@ -304,6 +305,7 @@ static inline void set_freepointer(struc
 	BUG_ON(object == fp); /* naive detection of double free or corruption */
 #endif
 
+	freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr);
 	*(void **)freeptr_addr = freelist_ptr(s, fp, freeptr_addr);
 }
 
@@ -538,8 +540,8 @@ static void print_section(char *level, c
 			  unsigned int length)
 {
 	metadata_access_enable();
-	print_hex_dump(level, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
-			length, 1);
+	print_hex_dump(level, kasan_reset_tag(text), DUMP_PREFIX_ADDRESS,
+			16, 1, addr, length, 1);
 	metadata_access_disable();
 }
 
@@ -570,7 +572,7 @@ static struct track *get_track(struct km
 
 	p = object + get_info_end(s);
 
-	return p + alloc;
+	return kasan_reset_tag(p + alloc);
 }
 
 static void set_track(struct kmem_cache *s, void *object,
@@ -583,7 +585,8 @@ static void set_track(struct kmem_cache
 		unsigned int nr_entries;
 
 		metadata_access_enable();
-		nr_entries = stack_trace_save(p->addrs, TRACK_ADDRS_COUNT, 3);
+		nr_entries = stack_trace_save(kasan_reset_tag(p->addrs),
+					      TRACK_ADDRS_COUNT, 3);
 		metadata_access_disable();
 
 		if (nr_entries < TRACK_ADDRS_COUNT)
@@ -747,7 +750,7 @@ static __printf(3, 4) void slab_err(stru
 
 static void init_object(struct kmem_cache *s, void *object, u8 val)
 {
-	u8 *p = object;
+	u8 *p = kasan_reset_tag(object);
 
 	if (s->flags & SLAB_RED_ZONE)
 		memset(p - s->red_left_pad, val, s->red_left_pad);
@@ -777,7 +780,7 @@ static int check_bytes_and_report(struct
 	u8 *addr = page_address(page);
 
 	metadata_access_enable();
-	fault = memchr_inv(start, value, bytes);
+	fault = memchr_inv(kasan_reset_tag(start), value, bytes);
 	metadata_access_disable();
 	if (!fault)
 		return 1;
@@ -873,7 +876,7 @@ static int slab_pad_check(struct kmem_ca
 
 	pad = end - remainder;
 	metadata_access_enable();
-	fault = memchr_inv(pad, POISON_INUSE, remainder);
+	fault = memchr_inv(kasan_reset_tag(pad), POISON_INUSE, remainder);
 	metadata_access_disable();
 	if (!fault)
 		return 1;
@@ -1118,7 +1121,7 @@ void setup_page_debug(struct kmem_cache
 		return;
 
 	metadata_access_enable();
-	memset(addr, POISON_INUSE, page_size(page));
+	memset(kasan_reset_tag(addr), POISON_INUSE, page_size(page));
 	metadata_access_disable();
 }
 
@@ -1566,10 +1569,10 @@ static inline bool slab_free_freelist_ho
 			 * Clear the object and the metadata, but don't touch
 			 * the redzone.
 			 */
-			memset(object, 0, s->object_size);
+			memset(kasan_reset_tag(object), 0, s->object_size);
 			rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad
 							   : 0;
-			memset((char *)object + s->inuse, 0,
+			memset((char *)kasan_reset_tag(object) + s->inuse, 0,
 			       s->size - s->inuse - rsize);
 
 		}
@@ -2881,10 +2884,10 @@ redo:
 		stat(s, ALLOC_FASTPATH);
 	}
 
-	maybe_wipe_obj_freeptr(s, object);
+	maybe_wipe_obj_freeptr(s, kasan_reset_tag(object));
 
 	if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object)
-		memset(object, 0, s->object_size);
+		memset(kasan_reset_tag(object), 0, s->object_size);
 
 	slab_post_alloc_hook(s, objcg, gfpflags, 1, &object);
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 52/78] kasan, arm64: enable CONFIG_KASAN_HW_TAGS
  2020-12-18 22:00 incoming Andrew Morton
                   ` (50 preceding siblings ...)
  2020-12-18 22:04 ` [patch 51/78] kasan, mm: reset tags when accessing metadata Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 53/78] kasan: add documentation for hardware tag-based mode Andrew Morton
                   ` (25 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: enable CONFIG_KASAN_HW_TAGS

Hardware tag-based KASAN is now ready, enable the configuration option.

Link: https://lkml.kernel.org/r/a6fa50d3bb6b318e05c6389a44095be96442b8b0.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/Kconfig |    1 +
 1 file changed, 1 insertion(+)

--- a/arch/arm64/Kconfig~kasan-arm64-enable-config_kasan_hw_tags
+++ a/arch/arm64/Kconfig
@@ -137,6 +137,7 @@ config ARM64
 	select HAVE_ARCH_JUMP_LABEL_RELATIVE
 	select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
 	select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
+	select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_MMAP_RND_BITS
 	select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 53/78] kasan: add documentation for hardware tag-based mode
  2020-12-18 22:00 incoming Andrew Morton
                   ` (51 preceding siblings ...)
  2020-12-18 22:04 ` [patch 52/78] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 54/78] kselftest/arm64: check GCR_EL1 after context switch Andrew Morton
                   ` (24 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: add documentation for hardware tag-based mode

Add documentation for hardware tag-based KASAN mode and also add some
clarifications for software tag-based mode.

Link: https://lkml.kernel.org/r/20ed1d387685e89fc31be068f890f070ef9fd5d5.1606161801.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 Documentation/dev-tools/kasan.rst |   84 ++++++++++++++++++++--------
 1 file changed, 61 insertions(+), 23 deletions(-)

--- a/Documentation/dev-tools/kasan.rst~kasan-add-documentation-for-hardware-tag-based-mode
+++ a/Documentation/dev-tools/kasan.rst
@@ -5,12 +5,14 @@ Overview
 --------
 
 KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to
-find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN
-(similar to userspace ASan) and software tag-based KASAN (similar to userspace
-HWASan).
-
-KASAN uses compile-time instrumentation to insert validity checks before every
-memory access, and therefore requires a compiler version that supports that.
+find out-of-bound and use-after-free bugs. KASAN has three modes:
+1. generic KASAN (similar to userspace ASan),
+2. software tag-based KASAN (similar to userspace HWASan),
+3. hardware tag-based KASAN (based on hardware memory tagging).
+
+Software KASAN modes (1 and 2) use compile-time instrumentation to insert
+validity checks before every memory access, and therefore require a compiler
+version that supports that.
 
 Generic KASAN is supported in both GCC and Clang. With GCC it requires version
 8.3.0 or later. Any supported Clang version is compatible, but detection of
@@ -19,7 +21,7 @@ out-of-bounds accesses for global variab
 Tag-based KASAN is only supported in Clang.
 
 Currently generic KASAN is supported for the x86_64, arm64, xtensa, s390 and
-riscv architectures, and tag-based KASAN is supported only for arm64.
+and riscv architectures, and tag-based KASAN modes are supported only for arm64.
 
 Usage
 -----
@@ -28,14 +30,16 @@ To enable KASAN configure kernel with::
 
 	  CONFIG_KASAN = y
 
-and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and
-CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN).
-
-You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE.
-Outline and inline are compiler instrumentation types. The former produces
-smaller binary while the latter is 1.1 - 2 times faster.
+and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN),
+CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN), and
+CONFIG_KASAN_HW_TAGS (to enable hardware tag-based KASAN).
+
+For software modes, you also need to choose between CONFIG_KASAN_OUTLINE and
+CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types.
+The former produces smaller binary while the latter is 1.1 - 2 times faster.
 
-Both KASAN modes work with both SLUB and SLAB memory allocators.
+Both software KASAN modes work with both SLUB and SLAB memory allocators,
+hardware tag-based KASAN currently only support SLUB.
 For better bug detection and nicer reporting, enable CONFIG_STACKTRACE.
 
 To augment reports with last allocation and freeing stack of the physical page,
@@ -197,17 +201,24 @@ call_rcu() and workqueue queuing.
 Software tag-based KASAN
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
-Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to
-store a pointer tag in the top byte of kernel pointers. Like generic KASAN it
-uses shadow memory to store memory tags associated with each 16-byte memory
+Software tag-based KASAN requires software memory tagging support in the form
+of HWASan-like compiler instrumentation (see HWASan documentation for details).
+
+Software tag-based KASAN is currently only implemented for arm64 architecture.
+
+Software tag-based KASAN uses the Top Byte Ignore (TBI) feature of arm64 CPUs
+to store a pointer tag in the top byte of kernel pointers. Like generic KASAN
+it uses shadow memory to store memory tags associated with each 16-byte memory
 cell (therefore it dedicates 1/16th of the kernel memory for shadow memory).
 
-On each memory allocation tag-based KASAN generates a random tag, tags the
-allocated memory with this tag, and embeds this tag into the returned pointer.
+On each memory allocation software tag-based KASAN generates a random tag, tags
+the allocated memory with this tag, and embeds this tag into the returned
+pointer.
+
 Software tag-based KASAN uses compile-time instrumentation to insert checks
 before each memory access. These checks make sure that tag of the memory that
 is being accessed is equal to tag of the pointer that is used to access this
-memory. In case of a tag mismatch tag-based KASAN prints a bug report.
+memory. In case of a tag mismatch software tag-based KASAN prints a bug report.
 
 Software tag-based KASAN also has two instrumentation modes (outline, that
 emits callbacks to check memory accesses; and inline, that performs the shadow
@@ -216,9 +227,36 @@ simply printed from the function that pe
 instrumentation a brk instruction is emitted by the compiler, and a dedicated
 brk handler is used to print bug reports.
 
-A potential expansion of this mode is a hardware tag-based mode, which would
-use hardware memory tagging support instead of compiler instrumentation and
-manual shadow memory manipulation.
+Software tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through
+pointers with 0xFF pointer tag aren't checked). The value 0xFE is currently
+reserved to tag freed memory regions.
+
+Software tag-based KASAN currently only supports tagging of
+kmem_cache_alloc/kmalloc and page_alloc memory.
+
+Hardware tag-based KASAN
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Hardware tag-based KASAN is similar to the software mode in concept, but uses
+hardware memory tagging support instead of compiler instrumentation and
+shadow memory.
+
+Hardware tag-based KASAN is currently only implemented for arm64 architecture
+and based on both arm64 Memory Tagging Extension (MTE) introduced in ARMv8.5
+Instruction Set Architecture, and Top Byte Ignore (TBI).
+
+Special arm64 instructions are used to assign memory tags for each allocation.
+Same tags are assigned to pointers to those allocations. On every memory
+access, hardware makes sure that tag of the memory that is being accessed is
+equal to tag of the pointer that is used to access this memory. In case of a
+tag mismatch a fault is generated and a report is printed.
+
+Hardware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through
+pointers with 0xFF pointer tag aren't checked). The value 0xFE is currently
+reserved to tag freed memory regions.
+
+Hardware tag-based KASAN currently only supports tagging of
+kmem_cache_alloc/kmalloc and page_alloc memory.
 
 What memory accesses are sanitised by KASAN?
 --------------------------------------------
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 54/78] kselftest/arm64: check GCR_EL1 after context switch
  2020-12-18 22:00 incoming Andrew Morton
                   ` (52 preceding siblings ...)
  2020-12-18 22:04 ` [patch 53/78] kasan: add documentation for hardware tag-based mode Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 55/78] kasan: simplify quarantine_put call site Andrew Morton
                   ` (23 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Subject: kselftest/arm64: check GCR_EL1 after context switch

This test is specific to MTE and verifies that the GCR_EL1 register is
context switched correctly.

It spawns 1024 processes and each process spawns 5 threads.  Each thread
writes a random setting of GCR_EL1 through the prctl() system call and
reads it back verifying that it is the same.  If the values are not the
same it reports a failure.

Note: The test has been extended to verify that even SYNC and ASYNC mode
setting is preserved correctly over context switching.

Link: https://lkml.kernel.org/r/b51a165426e906e7ec8a68d806ef3f8cd92581a6.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Marco Elver <elver@google.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 tools/testing/selftests/arm64/mte/Makefile                |    2 
 tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c |  155 ++++++++++
 2 files changed, 156 insertions(+), 1 deletion(-)

--- /dev/null
+++ a/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2020 ARM Limited
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <unistd.h>
+#include <sys/auxv.h>
+#include <sys/mman.h>
+#include <sys/prctl.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "kselftest.h"
+#include "mte_common_util.h"
+
+#define PR_SET_TAGGED_ADDR_CTRL 55
+#define PR_GET_TAGGED_ADDR_CTRL 56
+# define PR_TAGGED_ADDR_ENABLE  (1UL << 0)
+# define PR_MTE_TCF_SHIFT	1
+# define PR_MTE_TCF_NONE	(0UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_SYNC	(1UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_ASYNC	(2UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_MASK	(3UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TAG_SHIFT	3
+# define PR_MTE_TAG_MASK	(0xffffUL << PR_MTE_TAG_SHIFT)
+
+#include "mte_def.h"
+
+#define NUM_ITERATIONS		1024
+#define MAX_THREADS		5
+#define THREAD_ITERATIONS	1000
+
+void *execute_thread(void *x)
+{
+	pid_t pid = *((pid_t *)x);
+	pid_t tid = gettid();
+	uint64_t prctl_tag_mask;
+	uint64_t prctl_set;
+	uint64_t prctl_get;
+	uint64_t prctl_tcf;
+
+	srand(time(NULL) ^ (pid << 16) ^ (tid << 16));
+
+	prctl_tag_mask = rand() & 0xffff;
+
+	if (prctl_tag_mask % 2)
+		prctl_tcf = PR_MTE_TCF_SYNC;
+	else
+		prctl_tcf = PR_MTE_TCF_ASYNC;
+
+	prctl_set = PR_TAGGED_ADDR_ENABLE | prctl_tcf | (prctl_tag_mask << PR_MTE_TAG_SHIFT);
+
+	for (int j = 0; j < THREAD_ITERATIONS; j++) {
+		if (prctl(PR_SET_TAGGED_ADDR_CTRL, prctl_set, 0, 0, 0)) {
+			perror("prctl() failed");
+			goto fail;
+		}
+
+		prctl_get = prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0);
+
+		if (prctl_set != prctl_get) {
+			ksft_print_msg("Error: prctl_set: 0x%lx != prctl_get: 0x%lx\n",
+						prctl_set, prctl_get);
+			goto fail;
+		}
+	}
+
+	return (void *)KSFT_PASS;
+
+fail:
+	return (void *)KSFT_FAIL;
+}
+
+int execute_test(pid_t pid)
+{
+	pthread_t thread_id[MAX_THREADS];
+	int thread_data[MAX_THREADS];
+
+	for (int i = 0; i < MAX_THREADS; i++)
+		pthread_create(&thread_id[i], NULL,
+			       execute_thread, (void *)&pid);
+
+	for (int i = 0; i < MAX_THREADS; i++)
+		pthread_join(thread_id[i], (void *)&thread_data[i]);
+
+	for (int i = 0; i < MAX_THREADS; i++)
+		if (thread_data[i] == KSFT_FAIL)
+			return KSFT_FAIL;
+
+	return KSFT_PASS;
+}
+
+int mte_gcr_fork_test(void)
+{
+	pid_t pid;
+	int results[NUM_ITERATIONS];
+	pid_t cpid;
+	int res;
+
+	for (int i = 0; i < NUM_ITERATIONS; i++) {
+		pid = fork();
+
+		if (pid < 0)
+			return KSFT_FAIL;
+
+		if (pid == 0) {
+			cpid = getpid();
+
+			res = execute_test(cpid);
+
+			exit(res);
+		}
+	}
+
+	for (int i = 0; i < NUM_ITERATIONS; i++) {
+		wait(&res);
+
+		if (WIFEXITED(res))
+			results[i] = WEXITSTATUS(res);
+		else
+			--i;
+	}
+
+	for (int i = 0; i < NUM_ITERATIONS; i++)
+		if (results[i] == KSFT_FAIL)
+			return KSFT_FAIL;
+
+	return KSFT_PASS;
+}
+
+int main(int argc, char *argv[])
+{
+	int err;
+
+	err = mte_default_setup();
+	if (err)
+		return err;
+
+	ksft_set_plan(1);
+
+	evaluate_test(mte_gcr_fork_test(),
+		"Verify that GCR_EL1 is set correctly on context switch\n");
+
+	mte_restore_setup();
+	ksft_print_cnts();
+
+	return ksft_get_fail_cnt() == 0 ? KSFT_PASS : KSFT_FAIL;
+}
+
--- a/tools/testing/selftests/arm64/mte/Makefile~kselftest-arm64-check-gcr_el1-after-context-switch
+++ a/tools/testing/selftests/arm64/mte/Makefile
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 # Copyright (C) 2020 ARM Limited
 
-CFLAGS += -std=gnu99 -I.
+CFLAGS += -std=gnu99 -I. -lpthread
 SRCS := $(filter-out mte_common_util.c,$(wildcard *.c))
 PROGS := $(patsubst %.c,%,$(SRCS))
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 55/78] kasan: simplify quarantine_put call site
  2020-12-18 22:00 incoming Andrew Morton
                   ` (53 preceding siblings ...)
  2020-12-18 22:04 ` [patch 54/78] kselftest/arm64: check GCR_EL1 after context switch Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 56/78] kasan: rename get_alloc/free_info Andrew Morton
                   ` (22 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: simplify quarantine_put call site

Patch series "kasan: boot parameters for hardware tag-based mode", v4.

=== Overview

Hardware tag-based KASAN mode [1] is intended to eventually be used in
production as a security mitigation. Therefore there's a need for finer
control over KASAN features and for an existence of a kill switch.

This patchset adds a few boot parameters for hardware tag-based KASAN that
allow to disable or otherwise control particular KASAN features, as well
as provides some initial optimizations for running KASAN in production.

There's another planned patchset what will further optimize hardware
tag-based KASAN, provide proper benchmarking and tests, and will fully
enable tag-based KASAN for production use.

Hardware tag-based KASAN relies on arm64 Memory Tagging Extension (MTE)
[2] to perform memory and pointer tagging. Please see [3] and [4] for
detailed analysis of how MTE helps to fight memory safety problems.

The features that can be controlled are:

1. Whether KASAN is enabled at all.
2. Whether KASAN collects and saves alloc/free stacks.
3. Whether KASAN panics on a detected bug or not.

The patch titled "kasan: add and integrate kasan boot parameters" of this
series adds a few new boot parameters.

kasan.mode allows to choose one of three main modes:

- kasan.mode=off - KASAN is disabled, no tag checks are performed
- kasan.mode=prod - only essential production features are enabled
- kasan.mode=full - all KASAN features are enabled

The chosen mode provides default control values for the features mentioned
above. However it's also possible to override the default values by
providing:

- kasan.stacktrace=off/on - enable stacks collection
                            (default: on for mode=full, otherwise off)
- kasan.fault=report/panic - only report tag fault or also panic
                             (default: report)

If kasan.mode parameter is not provided, it defaults to full when
CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise.

It is essential that switching between these modes doesn't require
rebuilding the kernel with different configs, as this is required by
the Android GKI (Generic Kernel Image) initiative.

=== Benchmarks

For now I've only performed a few simple benchmarks such as measuring
kernel boot time and slab memory usage after boot. There's an upcoming
patchset which will optimize KASAN further and include more detailed
benchmarking results.

The benchmarks were performed in QEMU and the results below exclude the
slowdown caused by QEMU memory tagging emulation (as it's different from
the slowdown that will be introduced by hardware and is therefore
irrelevant).

KASAN_HW_TAGS=y + kasan.mode=off introduces no performance or memory
impact compared to KASAN_HW_TAGS=n.

kasan.mode=prod (manually excluding tagging) introduces 3% of performance
and no memory impact (except memory used by hardware to store tags)
compared to kasan.mode=off.

kasan.mode=full has about 40% performance and 30% memory impact over
kasan.mode=prod. Both come from alloc/free stack collection.

=== Notes

This patchset is available here:

https://github.com/xairy/linux/tree/up-boot-mte-v4

This patchset is based on v11 of "kasan: add hardware tag-based mode for
arm64" patchset [1].

For testing in QEMU hardware tag-based KASAN requires:

1. QEMU built from master [6] (use "-machine virt,mte=on -cpu max" arguments
   to run).
2. GCC version 10.

[1] https://lore.kernel.org/linux-arm-kernel/cover.1606161801.git.andreyknvl@google.com/T/#t
[2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety
[3] https://arxiv.org/pdf/1802.09517.pdf
[4] https://github.com/microsoft/MSRC-Security-Research/blob/master/papers/2020/Security%20analysis%20of%20memory%20tagging.pdf
[5] https://source.android.com/devices/architecture/kernel/generic-kernel-image
[6] https://github.com/qemu/qemu

=== Tags

Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>


This patch (of 19):

Move get_free_info() call into quarantine_put() to simplify the call site.

No functional changes.

Link: https://lkml.kernel.org/r/cover.1606162397.git.andreyknvl@google.com
Link: https://lkml.kernel.org/r/312d0a3ef92cc6dc4fa5452cbc1714f9393ca239.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Iab0f04e7ebf8d83247024b7190c67c3c34c7940f
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c     |    2 +-
 mm/kasan/kasan.h      |    5 ++---
 mm/kasan/quarantine.c |    3 ++-
 3 files changed, 5 insertions(+), 5 deletions(-)

--- a/mm/kasan/common.c~kasan-simplify-quarantine_put-call-site
+++ a/mm/kasan/common.c
@@ -313,7 +313,7 @@ static bool __kasan_slab_free(struct kme
 
 	kasan_set_free_info(cache, object, tag);
 
-	quarantine_put(get_free_info(cache, object), cache);
+	quarantine_put(cache, object);
 
 	return IS_ENABLED(CONFIG_KASAN_GENERIC);
 }
--- a/mm/kasan/kasan.h~kasan-simplify-quarantine_put-call-site
+++ a/mm/kasan/kasan.h
@@ -216,12 +216,11 @@ struct kasan_track *kasan_get_free_track
 
 #if defined(CONFIG_KASAN_GENERIC) && \
 	(defined(CONFIG_SLAB) || defined(CONFIG_SLUB))
-void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache);
+void quarantine_put(struct kmem_cache *cache, void *object);
 void quarantine_reduce(void);
 void quarantine_remove_cache(struct kmem_cache *cache);
 #else
-static inline void quarantine_put(struct kasan_free_meta *info,
-				struct kmem_cache *cache) { }
+static inline void quarantine_put(struct kmem_cache *cache, void *object) { }
 static inline void quarantine_reduce(void) { }
 static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
 #endif
--- a/mm/kasan/quarantine.c~kasan-simplify-quarantine_put-call-site
+++ a/mm/kasan/quarantine.c
@@ -163,11 +163,12 @@ static void qlist_free_all(struct qlist_
 	qlist_init(q);
 }
 
-void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache)
+void quarantine_put(struct kmem_cache *cache, void *object)
 {
 	unsigned long flags;
 	struct qlist_head *q;
 	struct qlist_head temp = QLIST_INIT;
+	struct kasan_free_meta *info = get_free_info(cache, object);
 
 	/*
 	 * Note: irq must be disabled until after we move the batch to the
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 56/78] kasan: rename get_alloc/free_info
  2020-12-18 22:00 incoming Andrew Morton
                   ` (54 preceding siblings ...)
  2020-12-18 22:04 ` [patch 55/78] kasan: simplify quarantine_put call site Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 57/78] kasan: introduce set_alloc_info Andrew Morton
                   ` (21 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: rename get_alloc/free_info

Rename get_alloc_info() and get_free_info() to kasan_get_alloc_meta() and
kasan_get_free_meta() to better reflect what those do and avoid confusion
with kasan_set_free_info().

No functional changes.

Link: https://lkml.kernel.org/r/27b7c036b754af15a2839e945f6d8bfce32b4c2f.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ib6e4ba61c8b12112b403d3479a9799ac8fff8de1
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c         |   16 ++++++++--------
 mm/kasan/generic.c        |   12 ++++++------
 mm/kasan/hw_tags.c        |    4 ++--
 mm/kasan/kasan.h          |    8 ++++----
 mm/kasan/quarantine.c     |    4 ++--
 mm/kasan/report.c         |   12 ++++++------
 mm/kasan/report_sw_tags.c |    2 +-
 mm/kasan/sw_tags.c        |    4 ++--
 8 files changed, 31 insertions(+), 31 deletions(-)

--- a/mm/kasan/common.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/common.c
@@ -180,14 +180,14 @@ size_t kasan_metadata_size(struct kmem_c
 		sizeof(struct kasan_free_meta) : 0);
 }
 
-struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
-					const void *object)
+struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
+					      const void *object)
 {
 	return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset;
 }
 
-struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
-				      const void *object)
+struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
+					    const void *object)
 {
 	BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
 	return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset;
@@ -264,13 +264,13 @@ static u8 assign_tag(struct kmem_cache *
 void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
 						const void *object)
 {
-	struct kasan_alloc_meta *alloc_info;
+	struct kasan_alloc_meta *alloc_meta;
 
 	if (!(cache->flags & SLAB_KASAN))
 		return (void *)object;
 
-	alloc_info = get_alloc_info(cache, object);
-	__memset(alloc_info, 0, sizeof(*alloc_info));
+	alloc_meta = kasan_get_alloc_meta(cache, object);
+	__memset(alloc_meta, 0, sizeof(*alloc_meta));
 
 	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
 		object = set_tag(object, assign_tag(cache, object, true, false));
@@ -350,7 +350,7 @@ static void *__kasan_kmalloc(struct kmem
 		     KASAN_KMALLOC_REDZONE);
 
 	if (cache->flags & SLAB_KASAN)
-		kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags);
+		kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
 
 	return set_tag(object, tag);
 }
--- a/mm/kasan/generic.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/generic.c
@@ -328,7 +328,7 @@ void kasan_record_aux_stack(void *addr)
 {
 	struct page *page = kasan_addr_to_page(addr);
 	struct kmem_cache *cache;
-	struct kasan_alloc_meta *alloc_info;
+	struct kasan_alloc_meta *alloc_meta;
 	void *object;
 
 	if (!(page && PageSlab(page)))
@@ -336,10 +336,10 @@ void kasan_record_aux_stack(void *addr)
 
 	cache = page->slab_cache;
 	object = nearest_obj(cache, page, addr);
-	alloc_info = get_alloc_info(cache, object);
+	alloc_meta = kasan_get_alloc_meta(cache, object);
 
-	alloc_info->aux_stack[1] = alloc_info->aux_stack[0];
-	alloc_info->aux_stack[0] = kasan_save_stack(GFP_NOWAIT);
+	alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0];
+	alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT);
 }
 
 void kasan_set_free_info(struct kmem_cache *cache,
@@ -347,7 +347,7 @@ void kasan_set_free_info(struct kmem_cac
 {
 	struct kasan_free_meta *free_meta;
 
-	free_meta = get_free_info(cache, object);
+	free_meta = kasan_get_free_meta(cache, object);
 	kasan_set_track(&free_meta->free_track, GFP_NOWAIT);
 
 	/*
@@ -361,5 +361,5 @@ struct kasan_track *kasan_get_free_track
 {
 	if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK)
 		return NULL;
-	return &get_free_info(cache, object)->free_track;
+	return &kasan_get_free_meta(cache, object)->free_track;
 }
--- a/mm/kasan/hw_tags.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/hw_tags.c
@@ -75,7 +75,7 @@ void kasan_set_free_info(struct kmem_cac
 {
 	struct kasan_alloc_meta *alloc_meta;
 
-	alloc_meta = get_alloc_info(cache, object);
+	alloc_meta = kasan_get_alloc_meta(cache, object);
 	kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT);
 }
 
@@ -84,6 +84,6 @@ struct kasan_track *kasan_get_free_track
 {
 	struct kasan_alloc_meta *alloc_meta;
 
-	alloc_meta = get_alloc_info(cache, object);
+	alloc_meta = kasan_get_alloc_meta(cache, object);
 	return &alloc_meta->free_track[0];
 }
--- a/mm/kasan/kasan.h~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/kasan.h
@@ -149,10 +149,10 @@ struct kasan_free_meta {
 #endif
 };
 
-struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
-					const void *object);
-struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
-					const void *object);
+struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
+						const void *object);
+struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
+						const void *object);
 
 void poison_range(const void *address, size_t size, u8 value);
 void unpoison_range(const void *address, size_t size);
--- a/mm/kasan/quarantine.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/quarantine.c
@@ -168,7 +168,7 @@ void quarantine_put(struct kmem_cache *c
 	unsigned long flags;
 	struct qlist_head *q;
 	struct qlist_head temp = QLIST_INIT;
-	struct kasan_free_meta *info = get_free_info(cache, object);
+	struct kasan_free_meta *meta = kasan_get_free_meta(cache, object);
 
 	/*
 	 * Note: irq must be disabled until after we move the batch to the
@@ -185,7 +185,7 @@ void quarantine_put(struct kmem_cache *c
 		local_irq_restore(flags);
 		return;
 	}
-	qlist_put(q, &info->quarantine_link, cache->size);
+	qlist_put(q, &meta->quarantine_link, cache->size);
 	if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) {
 		qlist_move_all(q, &temp);
 
--- a/mm/kasan/report.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/report.c
@@ -164,12 +164,12 @@ static void describe_object_addr(struct
 static void describe_object(struct kmem_cache *cache, void *object,
 				const void *addr, u8 tag)
 {
-	struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object);
+	struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object);
 
 	if (cache->flags & SLAB_KASAN) {
 		struct kasan_track *free_track;
 
-		print_track(&alloc_info->alloc_track, "Allocated");
+		print_track(&alloc_meta->alloc_track, "Allocated");
 		pr_err("\n");
 		free_track = kasan_get_free_track(cache, object, tag);
 		if (free_track) {
@@ -178,14 +178,14 @@ static void describe_object(struct kmem_
 		}
 
 #ifdef CONFIG_KASAN_GENERIC
-		if (alloc_info->aux_stack[0]) {
+		if (alloc_meta->aux_stack[0]) {
 			pr_err("Last potentially related work creation:\n");
-			print_stack(alloc_info->aux_stack[0]);
+			print_stack(alloc_meta->aux_stack[0]);
 			pr_err("\n");
 		}
-		if (alloc_info->aux_stack[1]) {
+		if (alloc_meta->aux_stack[1]) {
 			pr_err("Second to last potentially related work creation:\n");
-			print_stack(alloc_info->aux_stack[1]);
+			print_stack(alloc_meta->aux_stack[1]);
 			pr_err("\n");
 		}
 #endif
--- a/mm/kasan/report_sw_tags.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/report_sw_tags.c
@@ -46,7 +46,7 @@ const char *get_bug_type(struct kasan_ac
 	if (page && PageSlab(page)) {
 		cache = page->slab_cache;
 		object = nearest_obj(cache, page, (void *)addr);
-		alloc_meta = get_alloc_info(cache, object);
+		alloc_meta = kasan_get_alloc_meta(cache, object);
 
 		for (i = 0; i < KASAN_NR_FREE_STACKS; i++)
 			if (alloc_meta->free_pointer_tag[i] == tag)
--- a/mm/kasan/sw_tags.c~kasan-rename-get_alloc-free_info
+++ a/mm/kasan/sw_tags.c
@@ -174,7 +174,7 @@ void kasan_set_free_info(struct kmem_cac
 	struct kasan_alloc_meta *alloc_meta;
 	u8 idx = 0;
 
-	alloc_meta = get_alloc_info(cache, object);
+	alloc_meta = kasan_get_alloc_meta(cache, object);
 
 #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY
 	idx = alloc_meta->free_track_idx;
@@ -191,7 +191,7 @@ struct kasan_track *kasan_get_free_track
 	struct kasan_alloc_meta *alloc_meta;
 	int i = 0;
 
-	alloc_meta = get_alloc_info(cache, object);
+	alloc_meta = kasan_get_alloc_meta(cache, object);
 
 #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY
 	for (i = 0; i < KASAN_NR_FREE_STACKS; i++) {
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 57/78] kasan: introduce set_alloc_info
  2020-12-18 22:00 incoming Andrew Morton
                   ` (55 preceding siblings ...)
  2020-12-18 22:04 ` [patch 56/78] kasan: rename get_alloc/free_info Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 58/78] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK Andrew Morton
                   ` (20 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: introduce set_alloc_info

Add set_alloc_info() helper and move kasan_set_track() into it. This will
simplify the code for one of the upcoming changes.

No functional changes.

Link: https://lkml.kernel.org/r/b2393e8f1e311a70fc3aaa2196461b6acdee7d21.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I0316193cbb4ecc9b87b7c2eee0dd79f8ec908c1a
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

--- a/mm/kasan/common.c~kasan-introduce-set_alloc_info
+++ a/mm/kasan/common.c
@@ -323,6 +323,11 @@ bool kasan_slab_free(struct kmem_cache *
 	return __kasan_slab_free(cache, object, ip, true);
 }
 
+static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)
+{
+	kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
+}
+
 static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
 				size_t size, gfp_t flags, bool keep_tag)
 {
@@ -350,7 +355,7 @@ static void *__kasan_kmalloc(struct kmem
 		     KASAN_KMALLOC_REDZONE);
 
 	if (cache->flags & SLAB_KASAN)
-		kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
+		set_alloc_info(cache, (void *)object, flags);
 
 	return set_tag(object, tag);
 }
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 58/78] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK
  2020-12-18 22:00 incoming Andrew Morton
                   ` (56 preceding siblings ...)
  2020-12-18 22:04 ` [patch 57/78] kasan: introduce set_alloc_info Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 59/78] kasan: allow VMAP_STACK for HW_TAGS mode Andrew Morton
                   ` (19 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK

There's a config option CONFIG_KASAN_STACK that has to be enabled for
KASAN to use stack instrumentation and perform validity checks for
stack variables.

There's no need to unpoison stack when CONFIG_KASAN_STACK is not enabled.
Only call kasan_unpoison_task_stack[_below]() when CONFIG_KASAN_STACK is
enabled.

Note, that CONFIG_KASAN_STACK is an option that is currently always
defined when CONFIG_KASAN is enabled, and therefore has to be tested
with #if instead of #ifdef.

Link: https://lkml.kernel.org/r/d09dd3f8abb388da397fd11598c5edeaa83fe559.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/If8a891e9fe01ea543e00b576852685afec0887e3
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/kernel/sleep.S        |    2 +-
 arch/x86/kernel/acpi/wakeup_64.S |    2 +-
 include/linux/kasan.h            |   10 ++++++----
 mm/kasan/common.c                |    2 ++
 4 files changed, 10 insertions(+), 6 deletions(-)

--- a/arch/arm64/kernel/sleep.S~kasan-arm64-unpoison-stack-only-with-config_kasan_stack
+++ a/arch/arm64/kernel/sleep.S
@@ -133,7 +133,7 @@ SYM_FUNC_START(_cpu_resume)
 	 */
 	bl	cpu_do_resume
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK
 	mov	x0, sp
 	bl	kasan_unpoison_task_stack_below
 #endif
--- a/arch/x86/kernel/acpi/wakeup_64.S~kasan-arm64-unpoison-stack-only-with-config_kasan_stack
+++ a/arch/x86/kernel/acpi/wakeup_64.S
@@ -112,7 +112,7 @@ SYM_FUNC_START(do_suspend_lowlevel)
 	movq	pt_regs_r14(%rax), %r14
 	movq	pt_regs_r15(%rax), %r15
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK
 	/*
 	 * The suspend path may have poisoned some areas deeper in the stack,
 	 * which we now need to unpoison.
--- a/include/linux/kasan.h~kasan-arm64-unpoison-stack-only-with-config_kasan_stack
+++ a/include/linux/kasan.h
@@ -77,8 +77,6 @@ static inline void kasan_disable_current
 
 void kasan_unpoison_range(const void *address, size_t size);
 
-void kasan_unpoison_task_stack(struct task_struct *task);
-
 void kasan_alloc_pages(struct page *page, unsigned int order);
 void kasan_free_pages(struct page *page, unsigned int order);
 
@@ -123,8 +121,6 @@ void kasan_restore_multi_shot(bool enabl
 
 static inline void kasan_unpoison_range(const void *address, size_t size) {}
 
-static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
-
 static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
 static inline void kasan_free_pages(struct page *page, unsigned int order) {}
 
@@ -176,6 +172,12 @@ static inline size_t kasan_metadata_size
 
 #endif /* CONFIG_KASAN */
 
+#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK
+void kasan_unpoison_task_stack(struct task_struct *task);
+#else
+static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
+#endif
+
 #ifdef CONFIG_KASAN_GENERIC
 
 void kasan_cache_shrink(struct kmem_cache *cache);
--- a/mm/kasan/common.c~kasan-arm64-unpoison-stack-only-with-config_kasan_stack
+++ a/mm/kasan/common.c
@@ -63,6 +63,7 @@ void kasan_unpoison_range(const void *ad
 	unpoison_range(address, size);
 }
 
+#if CONFIG_KASAN_STACK
 static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
 {
 	void *base = task_stack_page(task);
@@ -89,6 +90,7 @@ asmlinkage void kasan_unpoison_task_stac
 
 	unpoison_range(base, watermark - base);
 }
+#endif /* CONFIG_KASAN_STACK */
 
 void kasan_alloc_pages(struct page *page, unsigned int order)
 {
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 59/78] kasan: allow VMAP_STACK for HW_TAGS mode
  2020-12-18 22:00 incoming Andrew Morton
                   ` (57 preceding siblings ...)
  2020-12-18 22:04 ` [patch 58/78] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 60/78] kasan: remove __kasan_unpoison_stack Andrew Morton
                   ` (18 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: allow VMAP_STACK for HW_TAGS mode

Even though hardware tag-based mode currently doesn't support checking
vmalloc allocations, it doesn't use shadow memory and works with
VMAP_STACK as is. Change VMAP_STACK definition accordingly.

Link: https://lkml.kernel.org/r/ecdb2a1658ebd88eb276dee2493518ac0e82de41.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I3552cbc12321dec82cd7372676e9372a2eb452ac
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/Kconfig |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/arch/Kconfig~kasan-allow-vmap_stack-for-hw_tags-mode
+++ a/arch/Kconfig
@@ -976,16 +976,16 @@ config VMAP_STACK
 	default y
 	bool "Use a virtually-mapped stack"
 	depends on HAVE_ARCH_VMAP_STACK
-	depends on !KASAN || KASAN_VMALLOC
+	depends on !KASAN || KASAN_HW_TAGS || KASAN_VMALLOC
 	help
 	  Enable this if you want the use virtually-mapped kernel stacks
 	  with guard pages.  This causes kernel stack overflows to be
 	  caught immediately rather than causing difficult-to-diagnose
 	  corruption.
 
-	  To use this with KASAN, the architecture must support backing
-	  virtual mappings with real shadow memory, and KASAN_VMALLOC must
-	  be enabled.
+	  To use this with software KASAN modes, the architecture must support
+	  backing virtual mappings with real shadow memory, and KASAN_VMALLOC
+	  must be enabled.
 
 config ARCH_OPTIONAL_KERNEL_RWX
 	def_bool n
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 60/78] kasan: remove __kasan_unpoison_stack
  2020-12-18 22:00 incoming Andrew Morton
                   ` (58 preceding siblings ...)
  2020-12-18 22:04 ` [patch 59/78] kasan: allow VMAP_STACK for HW_TAGS mode Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 61/78] kasan: inline kasan_reset_tag for tag-based modes Andrew Morton
                   ` (17 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: remove __kasan_unpoison_stack

There's no need for __kasan_unpoison_stack() helper, as it's only
currently used in a single place. Removing it also removes unneeded
arithmetic.

No functional changes.

Link: https://lkml.kernel.org/r/93e78948704a42ea92f6248ff8a725613d721161.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ie5ba549d445292fe629b4a96735e4034957bcc50
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c |   12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

--- a/mm/kasan/common.c~kasan-remove-__kasan_unpoison_stack
+++ a/mm/kasan/common.c
@@ -64,18 +64,12 @@ void kasan_unpoison_range(const void *ad
 }
 
 #if CONFIG_KASAN_STACK
-static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
-{
-	void *base = task_stack_page(task);
-	size_t size = sp - base;
-
-	unpoison_range(base, size);
-}
-
 /* Unpoison the entire stack for a task. */
 void kasan_unpoison_task_stack(struct task_struct *task)
 {
-	__kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE);
+	void *base = task_stack_page(task);
+
+	unpoison_range(base, THREAD_SIZE);
 }
 
 /* Unpoison the stack for the current task beyond a watermark sp value. */
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 61/78] kasan: inline kasan_reset_tag for tag-based modes
  2020-12-18 22:00 incoming Andrew Morton
                   ` (59 preceding siblings ...)
  2020-12-18 22:04 ` [patch 60/78] kasan: remove __kasan_unpoison_stack Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 62/78] kasan: inline random_tag for HW_TAGS Andrew Morton
                   ` (16 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: inline kasan_reset_tag for tag-based modes

Using kasan_reset_tag() currently results in a function call. As it's
called quite often from the allocator code, this leads to a noticeable
slowdown. Move it to include/linux/kasan.h and turn it into a static
inline function. Also remove the now unneeded reset_tag() internal KASAN
macro and use kasan_reset_tag() instead.

Link: https://lkml.kernel.org/r/6940383a3a9dfb416134d338d8fac97a9ebb8686.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I4d2061acfe91d480a75df00b07c22d8494ef14b5
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h     |    5 ++++-
 mm/kasan/common.c         |    6 +++---
 mm/kasan/hw_tags.c        |    9 ++-------
 mm/kasan/kasan.h          |    4 ----
 mm/kasan/report.c         |    4 ++--
 mm/kasan/report_hw_tags.c |    2 +-
 mm/kasan/report_sw_tags.c |    4 ++--
 mm/kasan/shadow.c         |    4 ++--
 mm/kasan/sw_tags.c        |    9 ++-------
 9 files changed, 18 insertions(+), 29 deletions(-)

--- a/include/linux/kasan.h~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/include/linux/kasan.h
@@ -194,7 +194,10 @@ static inline void kasan_record_aux_stac
 
 #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 
-void *kasan_reset_tag(const void *addr);
+static inline void *kasan_reset_tag(const void *addr)
+{
+	return (void *)arch_kasan_reset_tag(addr);
+}
 
 bool kasan_report(unsigned long addr, size_t size,
 		bool is_write, unsigned long ip);
--- a/mm/kasan/common.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/common.c
@@ -179,14 +179,14 @@ size_t kasan_metadata_size(struct kmem_c
 struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
 					      const void *object)
 {
-	return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset;
+	return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset;
 }
 
 struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
 					    const void *object)
 {
 	BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
-	return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset;
+	return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset;
 }
 
 void kasan_poison_slab(struct page *page)
@@ -283,7 +283,7 @@ static bool __kasan_slab_free(struct kme
 
 	tag = get_tag(object);
 	tagged_object = object;
-	object = reset_tag(object);
+	object = kasan_reset_tag(object);
 
 	if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) !=
 	    object)) {
--- a/mm/kasan/hw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/hw_tags.c
@@ -31,18 +31,13 @@ void __init kasan_init_hw_tags(void)
 	pr_info("KernelAddressSanitizer initialized\n");
 }
 
-void *kasan_reset_tag(const void *addr)
-{
-	return reset_tag(addr);
-}
-
 void poison_range(const void *address, size_t size, u8 value)
 {
 	/* Skip KFENCE memory if called explicitly outside of sl*b. */
 	if (is_kfence_address(address))
 		return;
 
-	hw_set_mem_tag_range(reset_tag(address),
+	hw_set_mem_tag_range(kasan_reset_tag(address),
 			round_up(size, KASAN_GRANULE_SIZE), value);
 }
 
@@ -52,7 +47,7 @@ void unpoison_range(const void *address,
 	if (is_kfence_address(address))
 		return;
 
-	hw_set_mem_tag_range(reset_tag(address),
+	hw_set_mem_tag_range(kasan_reset_tag(address),
 			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
 }
 
--- a/mm/kasan/kasan.h~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/kasan.h
@@ -248,15 +248,11 @@ static inline const void *arch_kasan_set
 	return addr;
 }
 #endif
-#ifndef arch_kasan_reset_tag
-#define arch_kasan_reset_tag(addr)	((void *)(addr))
-#endif
 #ifndef arch_kasan_get_tag
 #define arch_kasan_get_tag(addr)	0
 #endif
 
 #define set_tag(addr, tag)	((void *)arch_kasan_set_tag((addr), (tag)))
-#define reset_tag(addr)		((void *)arch_kasan_reset_tag(addr))
 #define get_tag(addr)		arch_kasan_get_tag(addr)
 
 #ifdef CONFIG_KASAN_HW_TAGS
--- a/mm/kasan/report.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/report.c
@@ -328,7 +328,7 @@ void kasan_report_invalid_free(void *obj
 	unsigned long flags;
 	u8 tag = get_tag(object);
 
-	object = reset_tag(object);
+	object = kasan_reset_tag(object);
 
 #if IS_ENABLED(CONFIG_KUNIT)
 	if (current->kunit_test)
@@ -361,7 +361,7 @@ static void __kasan_report(unsigned long
 	disable_trace_on_warning();
 
 	tagged_addr = (void *)addr;
-	untagged_addr = reset_tag(tagged_addr);
+	untagged_addr = kasan_reset_tag(tagged_addr);
 
 	info.access_addr = tagged_addr;
 	if (addr_has_metadata(untagged_addr))
--- a/mm/kasan/report_hw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/report_hw_tags.c
@@ -22,7 +22,7 @@ const char *get_bug_type(struct kasan_ac
 
 void *find_first_bad_addr(void *addr, size_t size)
 {
-	return reset_tag(addr);
+	return kasan_reset_tag(addr);
 }
 
 void metadata_fetch_row(char *buffer, void *row)
--- a/mm/kasan/report_sw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/report_sw_tags.c
@@ -41,7 +41,7 @@ const char *get_bug_type(struct kasan_ac
 	int i;
 
 	tag = get_tag(info->access_addr);
-	addr = reset_tag(info->access_addr);
+	addr = kasan_reset_tag(info->access_addr);
 	page = kasan_addr_to_page(addr);
 	if (page && PageSlab(page)) {
 		cache = page->slab_cache;
@@ -72,7 +72,7 @@ const char *get_bug_type(struct kasan_ac
 void *find_first_bad_addr(void *addr, size_t size)
 {
 	u8 tag = get_tag(addr);
-	void *p = reset_tag(addr);
+	void *p = kasan_reset_tag(addr);
 	void *end = p + size;
 
 	while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p))
--- a/mm/kasan/shadow.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/shadow.c
@@ -82,7 +82,7 @@ void poison_range(const void *address, s
 	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
 	 * addresses to this function.
 	 */
-	address = reset_tag(address);
+	address = kasan_reset_tag(address);
 
 	/* Skip KFENCE memory if called explicitly outside of sl*b. */
 	if (is_kfence_address(address))
@@ -103,7 +103,7 @@ void unpoison_range(const void *address,
 	 * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
 	 * addresses to this function.
 	 */
-	address = reset_tag(address);
+	address = kasan_reset_tag(address);
 
 	/*
 	 * Skip KFENCE memory if called explicitly outside of sl*b. Also note
--- a/mm/kasan/sw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes
+++ a/mm/kasan/sw_tags.c
@@ -67,11 +67,6 @@ u8 random_tag(void)
 	return (u8)(state % (KASAN_TAG_MAX + 1));
 }
 
-void *kasan_reset_tag(const void *addr)
-{
-	return reset_tag(addr);
-}
-
 bool check_memory_region(unsigned long addr, size_t size, bool write,
 				unsigned long ret_ip)
 {
@@ -107,7 +102,7 @@ bool check_memory_region(unsigned long a
 	if (tag == KASAN_TAG_KERNEL)
 		return true;
 
-	untagged_addr = reset_tag((const void *)addr);
+	untagged_addr = kasan_reset_tag((const void *)addr);
 	if (unlikely(untagged_addr <
 			kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) {
 		return !kasan_report(addr, size, write, ret_ip);
@@ -126,7 +121,7 @@ bool check_memory_region(unsigned long a
 bool check_invalid_free(void *addr)
 {
 	u8 tag = get_tag(addr);
-	u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr)));
+	u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr)));
 
 	return (shadow_byte == KASAN_TAG_INVALID) ||
 		(tag != KASAN_TAG_KERNEL && tag != shadow_byte);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 62/78] kasan: inline random_tag for HW_TAGS
  2020-12-18 22:00 incoming Andrew Morton
                   ` (60 preceding siblings ...)
  2020-12-18 22:04 ` [patch 61/78] kasan: inline kasan_reset_tag for tag-based modes Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 63/78] kasan: open-code kasan_unpoison_slab Andrew Morton
                   ` (15 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: inline random_tag for HW_TAGS

Using random_tag() currently results in a function call. Move its
definition to mm/kasan/kasan.h and turn it into a static inline function
for hardware tag-based mode to avoid uneeded function calls.

Link: https://lkml.kernel.org/r/be438471690e351e1d792e6bb432e8c03ccb15d3.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Iac5b2faf9a912900e16cca6834d621f5d4abf427
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/hw_tags.c |    5 -----
 mm/kasan/kasan.h   |   31 ++++++++++++++-----------------
 2 files changed, 14 insertions(+), 22 deletions(-)

--- a/mm/kasan/hw_tags.c~kasan-inline-random_tag-for-hw_tags
+++ a/mm/kasan/hw_tags.c
@@ -51,11 +51,6 @@ void unpoison_range(const void *address,
 			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
 }
 
-u8 random_tag(void)
-{
-	return hw_get_random_tag();
-}
-
 bool check_invalid_free(void *addr)
 {
 	u8 ptr_tag = get_tag(addr);
--- a/mm/kasan/kasan.h~kasan-inline-random_tag-for-hw_tags
+++ a/mm/kasan/kasan.h
@@ -190,6 +190,12 @@ static inline bool addr_has_metadata(con
 
 #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
+void print_tags(u8 addr_tag, const void *addr);
+#else
+static inline void print_tags(u8 addr_tag, const void *addr) { }
+#endif
+
 bool check_invalid_free(void *addr);
 
 void *find_first_bad_addr(void *addr, size_t size);
@@ -225,23 +231,6 @@ static inline void quarantine_reduce(voi
 static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
 #endif
 
-#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
-
-void print_tags(u8 addr_tag, const void *addr);
-
-u8 random_tag(void);
-
-#else
-
-static inline void print_tags(u8 addr_tag, const void *addr) { }
-
-static inline u8 random_tag(void)
-{
-	return 0;
-}
-
-#endif
-
 #ifndef arch_kasan_set_tag
 static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)
 {
@@ -281,6 +270,14 @@ static inline const void *arch_kasan_set
 
 #endif /* CONFIG_KASAN_HW_TAGS */
 
+#ifdef CONFIG_KASAN_SW_TAGS
+u8 random_tag(void);
+#elif defined(CONFIG_KASAN_HW_TAGS)
+static inline u8 random_tag(void) { return hw_get_random_tag(); }
+#else
+static inline u8 random_tag(void) { return 0; }
+#endif
+
 /*
  * Exported functions for interfaces called from assembly or from generated
  * code. Declarations here to avoid warning about missing declarations.
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 63/78] kasan: open-code kasan_unpoison_slab
  2020-12-18 22:00 incoming Andrew Morton
                   ` (61 preceding siblings ...)
  2020-12-18 22:04 ` [patch 62/78] kasan: inline random_tag for HW_TAGS Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-18 22:04 ` [patch 64/78] kasan: inline (un)poison_range and check_invalid_free Andrew Morton
                   ` (14 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: open-code kasan_unpoison_slab

There's the external annotation kasan_unpoison_slab() that is currently
defined as static inline and uses kasan_unpoison_range(). Open-code this
function in mempool.c. Otherwise with an upcoming change this function
will result in an unnecessary function call.

Link: https://lkml.kernel.org/r/131a6694a978a9a8b150187e539eecc8bcbf759b.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ia7c8b659f79209935cbaab3913bf7f082cc43a0e
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |    6 ------
 mm/mempool.c          |    2 +-
 2 files changed, 1 insertion(+), 7 deletions(-)

--- a/include/linux/kasan.h~kasan-open-code-kasan_unpoison_slab
+++ a/include/linux/kasan.h
@@ -107,11 +107,6 @@ struct kasan_cache {
 	int free_meta_offset;
 };
 
-size_t __ksize(const void *);
-static inline void kasan_unpoison_slab(const void *ptr)
-{
-	kasan_unpoison_range(ptr, __ksize(ptr));
-}
 size_t kasan_metadata_size(struct kmem_cache *cache);
 
 bool kasan_save_enable_multi_shot(void);
@@ -167,7 +162,6 @@ static inline bool kasan_slab_free(struc
 	return false;
 }
 
-static inline void kasan_unpoison_slab(const void *ptr) { }
 static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
 
 #endif /* CONFIG_KASAN */
--- a/mm/mempool.c~kasan-open-code-kasan_unpoison_slab
+++ a/mm/mempool.c
@@ -112,7 +112,7 @@ static __always_inline void kasan_poison
 static void kasan_unpoison_element(mempool_t *pool, void *element)
 {
 	if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc)
-		kasan_unpoison_slab(element);
+		kasan_unpoison_range(element, __ksize(element));
 	else if (pool->alloc == mempool_alloc_pages)
 		kasan_alloc_pages(element, (unsigned long)pool->pool_data);
 }
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 64/78] kasan: inline (un)poison_range and check_invalid_free
  2020-12-18 22:00 incoming Andrew Morton
                   ` (62 preceding siblings ...)
  2020-12-18 22:04 ` [patch 63/78] kasan: open-code kasan_unpoison_slab Andrew Morton
@ 2020-12-18 22:04 ` Andrew Morton
  2020-12-22 14:02   ` kernel test robot
  2020-12-18 22:05 ` [patch 65/78] kasan: add and integrate kasan boot parameters Andrew Morton
                   ` (13 subsequent siblings)
  77 siblings, 1 reply; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:04 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: inline (un)poison_range and check_invalid_free

Using (un)poison_range() or check_invalid_free() currently results in
function calls. Move their definitions to mm/kasan/kasan.h and turn them
into static inline functions for hardware tag-based mode to avoid
unneeded function calls.

Link: https://lkml.kernel.org/r/7007955b69eb31b5376a7dc1e0f4ac49138504f2.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ia9d8191024a12d1374675b3d27197f10193f50bb
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/hw_tags.c |   30 ----------------------------
 mm/kasan/kasan.h   |   45 ++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 40 insertions(+), 35 deletions(-)

--- a/mm/kasan/hw_tags.c~kasan-inline-unpoison_range-and-check_invalid_free
+++ a/mm/kasan/hw_tags.c
@@ -10,7 +10,6 @@
 
 #include <linux/kasan.h>
 #include <linux/kernel.h>
-#include <linux/kfence.h>
 #include <linux/memory.h>
 #include <linux/mm.h>
 #include <linux/string.h>
@@ -31,35 +30,6 @@ void __init kasan_init_hw_tags(void)
 	pr_info("KernelAddressSanitizer initialized\n");
 }
 
-void poison_range(const void *address, size_t size, u8 value)
-{
-	/* Skip KFENCE memory if called explicitly outside of sl*b. */
-	if (is_kfence_address(address))
-		return;
-
-	hw_set_mem_tag_range(kasan_reset_tag(address),
-			round_up(size, KASAN_GRANULE_SIZE), value);
-}
-
-void unpoison_range(const void *address, size_t size)
-{
-	/* Skip KFENCE memory if called explicitly outside of sl*b. */
-	if (is_kfence_address(address))
-		return;
-
-	hw_set_mem_tag_range(kasan_reset_tag(address),
-			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
-}
-
-bool check_invalid_free(void *addr)
-{
-	u8 ptr_tag = get_tag(addr);
-	u8 mem_tag = hw_get_mem_tag(addr);
-
-	return (mem_tag == KASAN_TAG_INVALID) ||
-		(ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
-}
-
 void kasan_set_free_info(struct kmem_cache *cache,
 				void *object, u8 tag)
 {
--- a/mm/kasan/kasan.h~kasan-inline-unpoison_range-and-check_invalid_free
+++ a/mm/kasan/kasan.h
@@ -3,6 +3,7 @@
 #define __MM_KASAN_KASAN_H
 
 #include <linux/kasan.h>
+#include <linux/kfence.h>
 #include <linux/stackdepot.h>
 
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
@@ -154,9 +155,6 @@ struct kasan_alloc_meta *kasan_get_alloc
 struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
 						const void *object);
 
-void poison_range(const void *address, size_t size, u8 value);
-void unpoison_range(const void *address, size_t size);
-
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 
 static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
@@ -196,8 +194,6 @@ void print_tags(u8 addr_tag, const void
 static inline void print_tags(u8 addr_tag, const void *addr) { }
 #endif
 
-bool check_invalid_free(void *addr);
-
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
 void metadata_fetch_row(char *buffer, void *row);
@@ -278,6 +274,45 @@ static inline u8 random_tag(void) { retu
 static inline u8 random_tag(void) { return 0; }
 #endif
 
+#ifdef CONFIG_KASAN_HW_TAGS
+
+static inline void poison_range(const void *address, size_t size, u8 value)
+{
+	/* Skip KFENCE memory if called explicitly outside of sl*b. */
+	if (is_kfence_address(address))
+		return;
+
+	hw_set_mem_tag_range(kasan_reset_tag(address),
+			round_up(size, KASAN_GRANULE_SIZE), value);
+}
+
+static inline void unpoison_range(const void *address, size_t size)
+{
+	/* Skip KFENCE memory if called explicitly outside of sl*b. */
+	if (is_kfence_address(address))
+		return;
+
+	hw_set_mem_tag_range(kasan_reset_tag(address),
+			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
+}
+
+static inline bool check_invalid_free(void *addr)
+{
+	u8 ptr_tag = get_tag(addr);
+	u8 mem_tag = hw_get_mem_tag(addr);
+
+	return (mem_tag == KASAN_TAG_INVALID) ||
+		(ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
+}
+
+#else /* CONFIG_KASAN_HW_TAGS */
+
+void poison_range(const void *address, size_t size, u8 value);
+void unpoison_range(const void *address, size_t size);
+bool check_invalid_free(void *addr);
+
+#endif /* CONFIG_KASAN_HW_TAGS */
+
 /*
  * Exported functions for interfaces called from assembly or from generated
  * code. Declarations here to avoid warning about missing declarations.
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 65/78] kasan: add and integrate kasan boot parameters
  2020-12-18 22:00 incoming Andrew Morton
                   ` (63 preceding siblings ...)
  2020-12-18 22:04 ` [patch 64/78] kasan: inline (un)poison_range and check_invalid_free Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 66/78] kasan, mm: check kasan_enabled in annotations Andrew Morton
                   ` (12 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: add and integrate kasan boot parameters

Hardware tag-based KASAN mode is intended to eventually be used in
production as a security mitigation. Therefore there's a need for finer
control over KASAN features and for an existence of a kill switch.

This change adds a few boot parameters for hardware tag-based KASAN that
allow to disable or otherwise control particular KASAN features.

The features that can be controlled are:

1. Whether KASAN is enabled at all.
2. Whether KASAN collects and saves alloc/free stacks.
3. Whether KASAN panics on a detected bug or not.

With this change a new boot parameter kasan.mode allows to choose one of
three main modes:

- kasan.mode=off - KASAN is disabled, no tag checks are performed
- kasan.mode=prod - only essential production features are enabled
- kasan.mode=full - all KASAN features are enabled

The chosen mode provides default control values for the features mentioned
above. However it's also possible to override the default values by
providing:

- kasan.stacktrace=off/on - enable alloc/free stack collection
                            (default: on for mode=full, otherwise off)
- kasan.fault=report/panic - only report tag fault or also panic
                             (default: report)

If kasan.mode parameter is not provided, it defaults to full when
CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise.

It is essential that switching between these modes doesn't require
rebuilding the kernel with different configs, as this is required by
the Android GKI (Generic Kernel Image) initiative [1].

[1] https://source.android.com/devices/architecture/kernel/generic-kernel-image

[andreyknvl@google.com: don't use read-only static keys]
  Link: https://lkml.kernel.org/r/f2ded589eba1597f7360a972226083de9afd86e2.1607537948.git.andreyknvl@google.com
Link: https://lkml.kernel.org/r/cb093613879d8d8841173f090133eddeb4c35f1f.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c  |   22 ++++--
 mm/kasan/hw_tags.c |  151 +++++++++++++++++++++++++++++++++++++++++++
 mm/kasan/kasan.h   |   16 ++++
 mm/kasan/report.c  |   14 +++
 4 files changed, 196 insertions(+), 7 deletions(-)

--- a/mm/kasan/common.c~kasan-add-and-integrate-kasan-boot-parameters
+++ a/mm/kasan/common.c
@@ -134,6 +134,11 @@ void kasan_cache_create(struct kmem_cach
 	unsigned int redzone_size;
 	int redzone_adjust;
 
+	if (!kasan_stack_collection_enabled()) {
+		*flags |= SLAB_KASAN;
+		return;
+	}
+
 	/* Add alloc meta. */
 	cache->kasan_info.alloc_meta_offset = *size;
 	*size += sizeof(struct kasan_alloc_meta);
@@ -170,6 +175,8 @@ void kasan_cache_create(struct kmem_cach
 
 size_t kasan_metadata_size(struct kmem_cache *cache)
 {
+	if (!kasan_stack_collection_enabled())
+		return 0;
 	return (cache->kasan_info.alloc_meta_offset ?
 		sizeof(struct kasan_alloc_meta) : 0) +
 		(cache->kasan_info.free_meta_offset ?
@@ -262,11 +269,13 @@ void * __must_check kasan_init_slab_obj(
 {
 	struct kasan_alloc_meta *alloc_meta;
 
-	if (!(cache->flags & SLAB_KASAN))
-		return (void *)object;
+	if (kasan_stack_collection_enabled()) {
+		if (!(cache->flags & SLAB_KASAN))
+			return (void *)object;
 
-	alloc_meta = kasan_get_alloc_meta(cache, object);
-	__memset(alloc_meta, 0, sizeof(*alloc_meta));
+		alloc_meta = kasan_get_alloc_meta(cache, object);
+		__memset(alloc_meta, 0, sizeof(*alloc_meta));
+	}
 
 	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
 		object = set_tag(object, assign_tag(cache, object, true, false));
@@ -303,6 +312,9 @@ static bool __kasan_slab_free(struct kme
 	rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE);
 	poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE);
 
+	if (!kasan_stack_collection_enabled())
+		return false;
+
 	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
 			unlikely(!(cache->flags & SLAB_KASAN)))
 		return false;
@@ -350,7 +362,7 @@ static void *__kasan_kmalloc(struct kmem
 	poison_range((void *)redzone_start, redzone_end - redzone_start,
 		     KASAN_KMALLOC_REDZONE);
 
-	if (cache->flags & SLAB_KASAN)
+	if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN))
 		set_alloc_info(cache, (void *)object, flags);
 
 	return set_tag(object, tag);
--- a/mm/kasan/hw_tags.c~kasan-add-and-integrate-kasan-boot-parameters
+++ a/mm/kasan/hw_tags.c
@@ -8,18 +8,115 @@
 
 #define pr_fmt(fmt) "kasan: " fmt
 
+#include <linux/init.h>
 #include <linux/kasan.h>
 #include <linux/kernel.h>
 #include <linux/memory.h>
 #include <linux/mm.h>
+#include <linux/static_key.h>
 #include <linux/string.h>
 #include <linux/types.h>
 
 #include "kasan.h"
 
+enum kasan_arg_mode {
+	KASAN_ARG_MODE_DEFAULT,
+	KASAN_ARG_MODE_OFF,
+	KASAN_ARG_MODE_PROD,
+	KASAN_ARG_MODE_FULL,
+};
+
+enum kasan_arg_stacktrace {
+	KASAN_ARG_STACKTRACE_DEFAULT,
+	KASAN_ARG_STACKTRACE_OFF,
+	KASAN_ARG_STACKTRACE_ON,
+};
+
+enum kasan_arg_fault {
+	KASAN_ARG_FAULT_DEFAULT,
+	KASAN_ARG_FAULT_REPORT,
+	KASAN_ARG_FAULT_PANIC,
+};
+
+static enum kasan_arg_mode kasan_arg_mode __ro_after_init;
+static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init;
+static enum kasan_arg_fault kasan_arg_fault __ro_after_init;
+
+/* Whether KASAN is enabled at all. */
+DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled);
+EXPORT_SYMBOL(kasan_flag_enabled);
+
+/* Whether to collect alloc/free stack traces. */
+DEFINE_STATIC_KEY_FALSE(kasan_flag_stacktrace);
+
+/* Whether panic or disable tag checking on fault. */
+bool kasan_flag_panic __ro_after_init;
+
+/* kasan.mode=off/prod/full */
+static int __init early_kasan_mode(char *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	if (!strcmp(arg, "off"))
+		kasan_arg_mode = KASAN_ARG_MODE_OFF;
+	else if (!strcmp(arg, "prod"))
+		kasan_arg_mode = KASAN_ARG_MODE_PROD;
+	else if (!strcmp(arg, "full"))
+		kasan_arg_mode = KASAN_ARG_MODE_FULL;
+	else
+		return -EINVAL;
+
+	return 0;
+}
+early_param("kasan.mode", early_kasan_mode);
+
+/* kasan.stack=off/on */
+static int __init early_kasan_flag_stacktrace(char *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	if (!strcmp(arg, "off"))
+		kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_OFF;
+	else if (!strcmp(arg, "on"))
+		kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_ON;
+	else
+		return -EINVAL;
+
+	return 0;
+}
+early_param("kasan.stacktrace", early_kasan_flag_stacktrace);
+
+/* kasan.fault=report/panic */
+static int __init early_kasan_fault(char *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	if (!strcmp(arg, "report"))
+		kasan_arg_fault = KASAN_ARG_FAULT_REPORT;
+	else if (!strcmp(arg, "panic"))
+		kasan_arg_fault = KASAN_ARG_FAULT_PANIC;
+	else
+		return -EINVAL;
+
+	return 0;
+}
+early_param("kasan.fault", early_kasan_fault);
+
 /* kasan_init_hw_tags_cpu() is called for each CPU. */
 void kasan_init_hw_tags_cpu(void)
 {
+	/*
+	 * There's no need to check that the hardware is MTE-capable here,
+	 * as this function is only called for MTE-capable hardware.
+	 */
+
+	/* If KASAN is disabled, do nothing. */
+	if (kasan_arg_mode == KASAN_ARG_MODE_OFF)
+		return;
+
 	hw_init_tags(KASAN_TAG_MAX);
 	hw_enable_tagging();
 }
@@ -27,6 +124,60 @@ void kasan_init_hw_tags_cpu(void)
 /* kasan_init_hw_tags() is called once on boot CPU. */
 void __init kasan_init_hw_tags(void)
 {
+	/* If hardware doesn't support MTE, do nothing. */
+	if (!system_supports_mte())
+		return;
+
+	/* Choose KASAN mode if kasan boot parameter is not provided. */
+	if (kasan_arg_mode == KASAN_ARG_MODE_DEFAULT) {
+		if (IS_ENABLED(CONFIG_DEBUG_KERNEL))
+			kasan_arg_mode = KASAN_ARG_MODE_FULL;
+		else
+			kasan_arg_mode = KASAN_ARG_MODE_PROD;
+	}
+
+	/* Preset parameter values based on the mode. */
+	switch (kasan_arg_mode) {
+	case KASAN_ARG_MODE_DEFAULT:
+		/* Shouldn't happen as per the check above. */
+		WARN_ON(1);
+		return;
+	case KASAN_ARG_MODE_OFF:
+		/* If KASAN is disabled, do nothing. */
+		return;
+	case KASAN_ARG_MODE_PROD:
+		static_branch_enable(&kasan_flag_enabled);
+		break;
+	case KASAN_ARG_MODE_FULL:
+		static_branch_enable(&kasan_flag_enabled);
+		static_branch_enable(&kasan_flag_stacktrace);
+		break;
+	}
+
+	/* Now, optionally override the presets. */
+
+	switch (kasan_arg_stacktrace) {
+	case KASAN_ARG_STACKTRACE_DEFAULT:
+		break;
+	case KASAN_ARG_STACKTRACE_OFF:
+		static_branch_disable(&kasan_flag_stacktrace);
+		break;
+	case KASAN_ARG_STACKTRACE_ON:
+		static_branch_enable(&kasan_flag_stacktrace);
+		break;
+	}
+
+	switch (kasan_arg_fault) {
+	case KASAN_ARG_FAULT_DEFAULT:
+		break;
+	case KASAN_ARG_FAULT_REPORT:
+		kasan_flag_panic = false;
+		break;
+	case KASAN_ARG_FAULT_PANIC:
+		kasan_flag_panic = true;
+		break;
+	}
+
 	pr_info("KernelAddressSanitizer initialized\n");
 }
 
--- a/mm/kasan/kasan.h~kasan-add-and-integrate-kasan-boot-parameters
+++ a/mm/kasan/kasan.h
@@ -6,6 +6,22 @@
 #include <linux/kfence.h>
 #include <linux/stackdepot.h>
 
+#ifdef CONFIG_KASAN_HW_TAGS
+#include <linux/static_key.h>
+DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace);
+static inline bool kasan_stack_collection_enabled(void)
+{
+	return static_branch_unlikely(&kasan_flag_stacktrace);
+}
+#else
+static inline bool kasan_stack_collection_enabled(void)
+{
+	return true;
+}
+#endif
+
+extern bool kasan_flag_panic __ro_after_init;
+
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
 #else
--- a/mm/kasan/report.c~kasan-add-and-integrate-kasan-boot-parameters
+++ a/mm/kasan/report.c
@@ -99,6 +99,10 @@ static void end_report(unsigned long *fl
 		panic_on_warn = 0;
 		panic("panic_on_warn set ...\n");
 	}
+#ifdef CONFIG_KASAN_HW_TAGS
+	if (kasan_flag_panic)
+		panic("kasan.fault=panic set ...\n");
+#endif
 	kasan_enable_current();
 }
 
@@ -161,8 +165,8 @@ static void describe_object_addr(struct
 		(void *)(object_addr + cache->object_size));
 }
 
-static void describe_object(struct kmem_cache *cache, void *object,
-				const void *addr, u8 tag)
+static void describe_object_stacks(struct kmem_cache *cache, void *object,
+					const void *addr, u8 tag)
 {
 	struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object);
 
@@ -190,7 +194,13 @@ static void describe_object(struct kmem_
 		}
 #endif
 	}
+}
 
+static void describe_object(struct kmem_cache *cache, void *object,
+				const void *addr, u8 tag)
+{
+	if (kasan_stack_collection_enabled())
+		describe_object_stacks(cache, object, addr, tag);
 	describe_object_addr(cache, object, addr);
 }
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 66/78] kasan, mm: check kasan_enabled in annotations
  2020-12-18 22:00 incoming Andrew Morton
                   ` (64 preceding siblings ...)
  2020-12-18 22:05 ` [patch 65/78] kasan: add and integrate kasan boot parameters Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 67/78] kasan, mm: rename kasan_poison_kfree Andrew Morton
                   ` (11 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, Vincenzo.Frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, mm: check kasan_enabled in annotations

Declare the kasan_enabled static key in include/linux/kasan.h and in
include/linux/mm.h and check it in all kasan annotations. This allows to
avoid any slowdown caused by function calls when kasan_enabled is
disabled.

Link: https://lkml.kernel.org/r/9f90e3c0aa840dbb4833367c2335193299f69023.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I2589451d3c96c97abbcbf714baabe6161c6f153e
Co-developed-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |  217 ++++++++++++++++++++++++++++++----------
 include/linux/mm.h    |   22 ++--
 mm/kasan/common.c     |   56 +++++-----
 3 files changed, 212 insertions(+), 83 deletions(-)

--- a/include/linux/kasan.h~kasan-mm-check-kasan_enabled-in-annotations
+++ a/include/linux/kasan.h
@@ -2,6 +2,7 @@
 #ifndef _LINUX_KASAN_H
 #define _LINUX_KASAN_H
 
+#include <linux/static_key.h>
 #include <linux/types.h>
 
 struct kmem_cache;
@@ -75,54 +76,176 @@ static inline void kasan_disable_current
 
 #ifdef CONFIG_KASAN
 
-void kasan_unpoison_range(const void *address, size_t size);
-
-void kasan_alloc_pages(struct page *page, unsigned int order);
-void kasan_free_pages(struct page *page, unsigned int order);
-
-void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
-			slab_flags_t *flags);
-
-void kasan_poison_slab(struct page *page);
-void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
-void kasan_poison_object_data(struct kmem_cache *cache, void *object);
-void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
-					const void *object);
-
-void * __must_check kasan_kmalloc_large(const void *ptr, size_t size,
-						gfp_t flags);
-void kasan_kfree_large(void *ptr, unsigned long ip);
-void kasan_poison_kfree(void *ptr, unsigned long ip);
-void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object,
-					size_t size, gfp_t flags);
-void * __must_check kasan_krealloc(const void *object, size_t new_size,
-					gfp_t flags);
-
-void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object,
-					gfp_t flags);
-bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip);
-
 struct kasan_cache {
 	int alloc_meta_offset;
 	int free_meta_offset;
 };
 
-size_t kasan_metadata_size(struct kmem_cache *cache);
+#ifdef CONFIG_KASAN_HW_TAGS
+DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled);
+static __always_inline bool kasan_enabled(void)
+{
+	return static_branch_likely(&kasan_flag_enabled);
+}
+#else
+static inline bool kasan_enabled(void)
+{
+	return true;
+}
+#endif
+
+void __kasan_unpoison_range(const void *addr, size_t size);
+static __always_inline void kasan_unpoison_range(const void *addr, size_t size)
+{
+	if (kasan_enabled())
+		__kasan_unpoison_range(addr, size);
+}
+
+void __kasan_alloc_pages(struct page *page, unsigned int order);
+static __always_inline void kasan_alloc_pages(struct page *page,
+						unsigned int order)
+{
+	if (kasan_enabled())
+		__kasan_alloc_pages(page, order);
+}
+
+void __kasan_free_pages(struct page *page, unsigned int order);
+static __always_inline void kasan_free_pages(struct page *page,
+						unsigned int order)
+{
+	if (kasan_enabled())
+		__kasan_free_pages(page, order);
+}
+
+void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
+				slab_flags_t *flags);
+static __always_inline void kasan_cache_create(struct kmem_cache *cache,
+				unsigned int *size, slab_flags_t *flags)
+{
+	if (kasan_enabled())
+		__kasan_cache_create(cache, size, flags);
+}
+
+size_t __kasan_metadata_size(struct kmem_cache *cache);
+static __always_inline size_t kasan_metadata_size(struct kmem_cache *cache)
+{
+	if (kasan_enabled())
+		return __kasan_metadata_size(cache);
+	return 0;
+}
+
+void __kasan_poison_slab(struct page *page);
+static __always_inline void kasan_poison_slab(struct page *page)
+{
+	if (kasan_enabled())
+		__kasan_poison_slab(page);
+}
+
+void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
+static __always_inline void kasan_unpoison_object_data(struct kmem_cache *cache,
+							void *object)
+{
+	if (kasan_enabled())
+		__kasan_unpoison_object_data(cache, object);
+}
+
+void __kasan_poison_object_data(struct kmem_cache *cache, void *object);
+static __always_inline void kasan_poison_object_data(struct kmem_cache *cache,
+							void *object)
+{
+	if (kasan_enabled())
+		__kasan_poison_object_data(cache, object);
+}
+
+void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache,
+					  const void *object);
+static __always_inline void * __must_check kasan_init_slab_obj(
+				struct kmem_cache *cache, const void *object)
+{
+	if (kasan_enabled())
+		return __kasan_init_slab_obj(cache, object);
+	return (void *)object;
+}
+
+bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip);
+static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object,
+						unsigned long ip)
+{
+	if (kasan_enabled())
+		return __kasan_slab_free(s, object, ip);
+	return false;
+}
+
+void * __must_check __kasan_slab_alloc(struct kmem_cache *s,
+				       void *object, gfp_t flags);
+static __always_inline void * __must_check kasan_slab_alloc(
+				struct kmem_cache *s, void *object, gfp_t flags)
+{
+	if (kasan_enabled())
+		return __kasan_slab_alloc(s, object, flags);
+	return object;
+}
+
+void * __must_check __kasan_kmalloc(struct kmem_cache *s, const void *object,
+				    size_t size, gfp_t flags);
+static __always_inline void * __must_check kasan_kmalloc(struct kmem_cache *s,
+				const void *object, size_t size, gfp_t flags)
+{
+	if (kasan_enabled())
+		return __kasan_kmalloc(s, object, size, flags);
+	return (void *)object;
+}
+
+void * __must_check __kasan_kmalloc_large(const void *ptr,
+					  size_t size, gfp_t flags);
+static __always_inline void * __must_check kasan_kmalloc_large(const void *ptr,
+						      size_t size, gfp_t flags)
+{
+	if (kasan_enabled())
+		return __kasan_kmalloc_large(ptr, size, flags);
+	return (void *)ptr;
+}
+
+void * __must_check __kasan_krealloc(const void *object,
+				     size_t new_size, gfp_t flags);
+static __always_inline void * __must_check kasan_krealloc(const void *object,
+						 size_t new_size, gfp_t flags)
+{
+	if (kasan_enabled())
+		return __kasan_krealloc(object, new_size, flags);
+	return (void *)object;
+}
+
+void __kasan_poison_kfree(void *ptr, unsigned long ip);
+static __always_inline void kasan_poison_kfree(void *ptr, unsigned long ip)
+{
+	if (kasan_enabled())
+		__kasan_poison_kfree(ptr, ip);
+}
+
+void __kasan_kfree_large(void *ptr, unsigned long ip);
+static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip)
+{
+	if (kasan_enabled())
+		__kasan_kfree_large(ptr, ip);
+}
 
 bool kasan_save_enable_multi_shot(void);
 void kasan_restore_multi_shot(bool enabled);
 
 #else /* CONFIG_KASAN */
 
+static inline bool kasan_enabled(void)
+{
+	return false;
+}
 static inline void kasan_unpoison_range(const void *address, size_t size) {}
-
 static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
 static inline void kasan_free_pages(struct page *page, unsigned int order) {}
-
 static inline void kasan_cache_create(struct kmem_cache *cache,
 				      unsigned int *size,
 				      slab_flags_t *flags) {}
-
+static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
 static inline void kasan_poison_slab(struct page *page) {}
 static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
 					void *object) {}
@@ -133,36 +256,32 @@ static inline void *kasan_init_slab_obj(
 {
 	return (void *)object;
 }
-
-static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags)
+static inline bool kasan_slab_free(struct kmem_cache *s, void *object,
+				   unsigned long ip)
 {
-	return ptr;
+	return false;
+}
+static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object,
+				   gfp_t flags)
+{
+	return object;
 }
-static inline void kasan_kfree_large(void *ptr, unsigned long ip) {}
-static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {}
 static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object,
 				size_t size, gfp_t flags)
 {
 	return (void *)object;
 }
+static inline void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
+{
+	return (void *)ptr;
+}
 static inline void *kasan_krealloc(const void *object, size_t new_size,
 				 gfp_t flags)
 {
 	return (void *)object;
 }
-
-static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object,
-				   gfp_t flags)
-{
-	return object;
-}
-static inline bool kasan_slab_free(struct kmem_cache *s, void *object,
-				   unsigned long ip)
-{
-	return false;
-}
-
-static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
+static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {}
+static inline void kasan_kfree_large(void *ptr, unsigned long ip) {}
 
 #endif /* CONFIG_KASAN */
 
--- a/include/linux/mm.h~kasan-mm-check-kasan_enabled-in-annotations
+++ a/include/linux/mm.h
@@ -31,6 +31,7 @@
 #include <linux/sizes.h>
 #include <linux/sched.h>
 #include <linux/pgtable.h>
+#include <linux/kasan.h>
 
 struct mempolicy;
 struct anon_vma;
@@ -1422,22 +1423,30 @@ static inline bool cpupid_match_pid(stru
 #endif /* CONFIG_NUMA_BALANCING */
 
 #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
+
 static inline u8 page_kasan_tag(const struct page *page)
 {
-	return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
+	if (kasan_enabled())
+		return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
+	return 0xff;
 }
 
 static inline void page_kasan_tag_set(struct page *page, u8 tag)
 {
-	page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT);
-	page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT;
+	if (kasan_enabled()) {
+		page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT);
+		page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT;
+	}
 }
 
 static inline void page_kasan_tag_reset(struct page *page)
 {
-	page_kasan_tag_set(page, 0xff);
+	if (kasan_enabled())
+		page_kasan_tag_set(page, 0xff);
 }
-#else
+
+#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
+
 static inline u8 page_kasan_tag(const struct page *page)
 {
 	return 0xff;
@@ -1445,7 +1454,8 @@ static inline u8 page_kasan_tag(const st
 
 static inline void page_kasan_tag_set(struct page *page, u8 tag) { }
 static inline void page_kasan_tag_reset(struct page *page) { }
-#endif
+
+#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
 
 static inline struct zone *page_zone(const struct page *page)
 {
--- a/mm/kasan/common.c~kasan-mm-check-kasan_enabled-in-annotations
+++ a/mm/kasan/common.c
@@ -58,7 +58,7 @@ void kasan_disable_current(void)
 }
 #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
-void kasan_unpoison_range(const void *address, size_t size)
+void __kasan_unpoison_range(const void *address, size_t size)
 {
 	unpoison_range(address, size);
 }
@@ -86,7 +86,7 @@ asmlinkage void kasan_unpoison_task_stac
 }
 #endif /* CONFIG_KASAN_STACK */
 
-void kasan_alloc_pages(struct page *page, unsigned int order)
+void __kasan_alloc_pages(struct page *page, unsigned int order)
 {
 	u8 tag;
 	unsigned long i;
@@ -100,7 +100,7 @@ void kasan_alloc_pages(struct page *page
 	unpoison_range(page_address(page), PAGE_SIZE << order);
 }
 
-void kasan_free_pages(struct page *page, unsigned int order)
+void __kasan_free_pages(struct page *page, unsigned int order)
 {
 	if (likely(!PageHighMem(page)))
 		poison_range(page_address(page),
@@ -127,8 +127,8 @@ static inline unsigned int optimal_redzo
 		object_size <= (1 << 16) - 1024 ? 1024 : 2048;
 }
 
-void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
-			slab_flags_t *flags)
+void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
+			  slab_flags_t *flags)
 {
 	unsigned int orig_size = *size;
 	unsigned int redzone_size;
@@ -173,7 +173,7 @@ void kasan_cache_create(struct kmem_cach
 	*flags |= SLAB_KASAN;
 }
 
-size_t kasan_metadata_size(struct kmem_cache *cache)
+size_t __kasan_metadata_size(struct kmem_cache *cache)
 {
 	if (!kasan_stack_collection_enabled())
 		return 0;
@@ -196,7 +196,7 @@ struct kasan_free_meta *kasan_get_free_m
 	return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset;
 }
 
-void kasan_poison_slab(struct page *page)
+void __kasan_poison_slab(struct page *page)
 {
 	unsigned long i;
 
@@ -206,12 +206,12 @@ void kasan_poison_slab(struct page *page
 		     KASAN_KMALLOC_REDZONE);
 }
 
-void kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
+void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
 {
 	unpoison_range(object, cache->object_size);
 }
 
-void kasan_poison_object_data(struct kmem_cache *cache, void *object)
+void __kasan_poison_object_data(struct kmem_cache *cache, void *object)
 {
 	poison_range(object,
 			round_up(cache->object_size, KASAN_GRANULE_SIZE),
@@ -264,7 +264,7 @@ static u8 assign_tag(struct kmem_cache *
 #endif
 }
 
-void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
+void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache,
 						const void *object)
 {
 	struct kasan_alloc_meta *alloc_meta;
@@ -283,7 +283,7 @@ void * __must_check kasan_init_slab_obj(
 	return (void *)object;
 }
 
-static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
+static bool ____kasan_slab_free(struct kmem_cache *cache, void *object,
 			      unsigned long ip, bool quarantine)
 {
 	u8 tag;
@@ -326,9 +326,9 @@ static bool __kasan_slab_free(struct kme
 	return IS_ENABLED(CONFIG_KASAN_GENERIC);
 }
 
-bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip)
+bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip)
 {
-	return __kasan_slab_free(cache, object, ip, true);
+	return ____kasan_slab_free(cache, object, ip, true);
 }
 
 static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)
@@ -336,7 +336,7 @@ static void set_alloc_info(struct kmem_c
 	kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
 }
 
-static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
+static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object,
 				size_t size, gfp_t flags, bool keep_tag)
 {
 	unsigned long redzone_start;
@@ -368,20 +368,20 @@ static void *__kasan_kmalloc(struct kmem
 	return set_tag(object, tag);
 }
 
-void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object,
-					gfp_t flags)
+void * __must_check __kasan_slab_alloc(struct kmem_cache *cache,
+					void *object, gfp_t flags)
 {
-	return __kasan_kmalloc(cache, object, cache->object_size, flags, false);
+	return ____kasan_kmalloc(cache, object, cache->object_size, flags, false);
 }
 
-void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object,
-				size_t size, gfp_t flags)
+void * __must_check __kasan_kmalloc(struct kmem_cache *cache, const void *object,
+					size_t size, gfp_t flags)
 {
-	return __kasan_kmalloc(cache, object, size, flags, true);
+	return ____kasan_kmalloc(cache, object, size, flags, true);
 }
-EXPORT_SYMBOL(kasan_kmalloc);
+EXPORT_SYMBOL(__kasan_kmalloc);
 
-void * __must_check kasan_kmalloc_large(const void *ptr, size_t size,
+void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size,
 						gfp_t flags)
 {
 	struct page *page;
@@ -406,7 +406,7 @@ void * __must_check kasan_kmalloc_large(
 	return (void *)ptr;
 }
 
-void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags)
+void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flags)
 {
 	struct page *page;
 
@@ -416,13 +416,13 @@ void * __must_check kasan_krealloc(const
 	page = virt_to_head_page(object);
 
 	if (unlikely(!PageSlab(page)))
-		return kasan_kmalloc_large(object, size, flags);
+		return __kasan_kmalloc_large(object, size, flags);
 	else
-		return __kasan_kmalloc(page->slab_cache, object, size,
+		return ____kasan_kmalloc(page->slab_cache, object, size,
 						flags, true);
 }
 
-void kasan_poison_kfree(void *ptr, unsigned long ip)
+void __kasan_poison_kfree(void *ptr, unsigned long ip)
 {
 	struct page *page;
 
@@ -435,11 +435,11 @@ void kasan_poison_kfree(void *ptr, unsig
 		}
 		poison_range(ptr, page_size(page), KASAN_FREE_PAGE);
 	} else {
-		__kasan_slab_free(page->slab_cache, ptr, ip, false);
+		____kasan_slab_free(page->slab_cache, ptr, ip, false);
 	}
 }
 
-void kasan_kfree_large(void *ptr, unsigned long ip)
+void __kasan_kfree_large(void *ptr, unsigned long ip)
 {
 	if (ptr != page_address(virt_to_head_page(ptr)))
 		kasan_report_invalid_free(ptr, ip);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 67/78] kasan, mm: rename kasan_poison_kfree
  2020-12-18 22:00 incoming Andrew Morton
                   ` (65 preceding siblings ...)
  2020-12-18 22:05 ` [patch 66/78] kasan, mm: check kasan_enabled in annotations Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 68/78] kasan: don't round_up too much Andrew Morton
                   ` (10 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, mm: rename kasan_poison_kfree

Rename kasan_poison_kfree() to kasan_slab_free_mempool() as it better
reflects what this annotation does. Also add a comment that explains the
PageSlab() check.

No functional changes.

Link: https://lkml.kernel.org/r/141675fb493555e984c5dca555e9d9f768c7bbaa.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I5026f87364e556b506ef1baee725144bb04b8810
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |   16 ++++++++--------
 mm/kasan/common.c     |   40 +++++++++++++++++++++++-----------------
 mm/mempool.c          |    2 +-
 3 files changed, 32 insertions(+), 26 deletions(-)

--- a/include/linux/kasan.h~kasan-mm-rename-kasan_poison_kfree
+++ a/include/linux/kasan.h
@@ -176,6 +176,13 @@ static __always_inline bool kasan_slab_f
 	return false;
 }
 
+void __kasan_slab_free_mempool(void *ptr, unsigned long ip);
+static __always_inline void kasan_slab_free_mempool(void *ptr, unsigned long ip)
+{
+	if (kasan_enabled())
+		__kasan_slab_free_mempool(ptr, ip);
+}
+
 void * __must_check __kasan_slab_alloc(struct kmem_cache *s,
 				       void *object, gfp_t flags);
 static __always_inline void * __must_check kasan_slab_alloc(
@@ -216,13 +223,6 @@ static __always_inline void * __must_che
 	return (void *)object;
 }
 
-void __kasan_poison_kfree(void *ptr, unsigned long ip);
-static __always_inline void kasan_poison_kfree(void *ptr, unsigned long ip)
-{
-	if (kasan_enabled())
-		__kasan_poison_kfree(ptr, ip);
-}
-
 void __kasan_kfree_large(void *ptr, unsigned long ip);
 static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip)
 {
@@ -261,6 +261,7 @@ static inline bool kasan_slab_free(struc
 {
 	return false;
 }
+static inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) {}
 static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object,
 				   gfp_t flags)
 {
@@ -280,7 +281,6 @@ static inline void *kasan_krealloc(const
 {
 	return (void *)object;
 }
-static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {}
 static inline void kasan_kfree_large(void *ptr, unsigned long ip) {}
 
 #endif /* CONFIG_KASAN */
--- a/mm/kasan/common.c~kasan-mm-rename-kasan_poison_kfree
+++ a/mm/kasan/common.c
@@ -331,6 +331,29 @@ bool __kasan_slab_free(struct kmem_cache
 	return ____kasan_slab_free(cache, object, ip, true);
 }
 
+void __kasan_slab_free_mempool(void *ptr, unsigned long ip)
+{
+	struct page *page;
+
+	page = virt_to_head_page(ptr);
+
+	/*
+	 * Even though this function is only called for kmem_cache_alloc and
+	 * kmalloc backed mempool allocations, those allocations can still be
+	 * !PageSlab() when the size provided to kmalloc is larger than
+	 * KMALLOC_MAX_SIZE, and kmalloc falls back onto page_alloc.
+	 */
+	if (unlikely(!PageSlab(page))) {
+		if (ptr != page_address(page)) {
+			kasan_report_invalid_free(ptr, ip);
+			return;
+		}
+		poison_range(ptr, page_size(page), KASAN_FREE_PAGE);
+	} else {
+		____kasan_slab_free(page->slab_cache, ptr, ip, false);
+	}
+}
+
 static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)
 {
 	kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
@@ -422,23 +445,6 @@ void * __must_check __kasan_krealloc(con
 						flags, true);
 }
 
-void __kasan_poison_kfree(void *ptr, unsigned long ip)
-{
-	struct page *page;
-
-	page = virt_to_head_page(ptr);
-
-	if (unlikely(!PageSlab(page))) {
-		if (ptr != page_address(page)) {
-			kasan_report_invalid_free(ptr, ip);
-			return;
-		}
-		poison_range(ptr, page_size(page), KASAN_FREE_PAGE);
-	} else {
-		____kasan_slab_free(page->slab_cache, ptr, ip, false);
-	}
-}
-
 void __kasan_kfree_large(void *ptr, unsigned long ip)
 {
 	if (ptr != page_address(virt_to_head_page(ptr)))
--- a/mm/mempool.c~kasan-mm-rename-kasan_poison_kfree
+++ a/mm/mempool.c
@@ -104,7 +104,7 @@ static inline void poison_element(mempoo
 static __always_inline void kasan_poison_element(mempool_t *pool, void *element)
 {
 	if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc)
-		kasan_poison_kfree(element, _RET_IP_);
+		kasan_slab_free_mempool(element, _RET_IP_);
 	else if (pool->alloc == mempool_alloc_pages)
 		kasan_free_pages(element, (unsigned long)pool->pool_data);
 }
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 68/78] kasan: don't round_up too much
  2020-12-18 22:00 incoming Andrew Morton
                   ` (66 preceding siblings ...)
  2020-12-18 22:05 ` [patch 67/78] kasan, mm: rename kasan_poison_kfree Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 69/78] kasan: simplify assign_tag and set_tag calls Andrew Morton
                   ` (9 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: don't round_up too much

For hardware tag-based mode kasan_poison_memory() already rounds up the
size. Do the same for software modes and remove round_up() from the common
code.

Link: https://lkml.kernel.org/r/47b232474f1f89dc072aeda0fa58daa6efade377.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ib397128fac6eba874008662b4964d65352db4aa4
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c |    8 ++------
 mm/kasan/shadow.c |    1 +
 2 files changed, 3 insertions(+), 6 deletions(-)

--- a/mm/kasan/common.c~kasan-dont-round_up-too-much
+++ a/mm/kasan/common.c
@@ -213,9 +213,7 @@ void __kasan_unpoison_object_data(struct
 
 void __kasan_poison_object_data(struct kmem_cache *cache, void *object)
 {
-	poison_range(object,
-			round_up(cache->object_size, KASAN_GRANULE_SIZE),
-			KASAN_KMALLOC_REDZONE);
+	poison_range(object, cache->object_size, KASAN_KMALLOC_REDZONE);
 }
 
 /*
@@ -288,7 +286,6 @@ static bool ____kasan_slab_free(struct k
 {
 	u8 tag;
 	void *tagged_object;
-	unsigned long rounded_up_size;
 
 	tag = get_tag(object);
 	tagged_object = object;
@@ -309,8 +306,7 @@ static bool ____kasan_slab_free(struct k
 		return true;
 	}
 
-	rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE);
-	poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE);
+	poison_range(object, cache->object_size, KASAN_KMALLOC_FREE);
 
 	if (!kasan_stack_collection_enabled())
 		return false;
--- a/mm/kasan/shadow.c~kasan-dont-round_up-too-much
+++ a/mm/kasan/shadow.c
@@ -83,6 +83,7 @@ void poison_range(const void *address, s
 	 * addresses to this function.
 	 */
 	address = kasan_reset_tag(address);
+	size = round_up(size, KASAN_GRANULE_SIZE);
 
 	/* Skip KFENCE memory if called explicitly outside of sl*b. */
 	if (is_kfence_address(address))
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 69/78] kasan: simplify assign_tag and set_tag calls
  2020-12-18 22:00 incoming Andrew Morton
                   ` (67 preceding siblings ...)
  2020-12-18 22:05 ` [patch 68/78] kasan: don't round_up too much Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 70/78] kasan: clarify comment in __kasan_kfree_large Andrew Morton
                   ` (8 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: simplify assign_tag and set_tag calls

set_tag() already ignores the tag for the generic mode, so just call it
as is. Add a check for the generic mode to assign_tag(), and simplify its
call in ____kasan_kmalloc().

Link: https://lkml.kernel.org/r/121eeab245f98555862b289d2ba9269c868fbbcf.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I18905ca78fb4a3d60e1a34a4ca00247272480438
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c |   13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

--- a/mm/kasan/common.c~kasan-simplify-assign_tag-and-set_tag-calls
+++ a/mm/kasan/common.c
@@ -233,6 +233,9 @@ void __kasan_poison_object_data(struct k
 static u8 assign_tag(struct kmem_cache *cache, const void *object,
 			bool init, bool keep_tag)
 {
+	if (IS_ENABLED(CONFIG_KASAN_GENERIC))
+		return 0xff;
+
 	/*
 	 * 1. When an object is kmalloc()'ed, two hooks are called:
 	 *    kasan_slab_alloc() and kasan_kmalloc(). We assign the
@@ -275,8 +278,8 @@ void * __must_check __kasan_init_slab_ob
 		__memset(alloc_meta, 0, sizeof(*alloc_meta));
 	}
 
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
-		object = set_tag(object, assign_tag(cache, object, true, false));
+	/* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */
+	object = set_tag(object, assign_tag(cache, object, true, false));
 
 	return (void *)object;
 }
@@ -360,7 +363,7 @@ static void *____kasan_kmalloc(struct km
 {
 	unsigned long redzone_start;
 	unsigned long redzone_end;
-	u8 tag = 0xff;
+	u8 tag;
 
 	if (gfpflags_allow_blocking(flags))
 		quarantine_reduce();
@@ -372,9 +375,7 @@ static void *____kasan_kmalloc(struct km
 				KASAN_GRANULE_SIZE);
 	redzone_end = round_up((unsigned long)object + cache->object_size,
 				KASAN_GRANULE_SIZE);
-
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
-		tag = assign_tag(cache, object, false, keep_tag);
+	tag = assign_tag(cache, object, false, keep_tag);
 
 	/* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */
 	unpoison_range(set_tag(object, tag), size);
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 70/78] kasan: clarify comment in __kasan_kfree_large
  2020-12-18 22:00 incoming Andrew Morton
                   ` (68 preceding siblings ...)
  2020-12-18 22:05 ` [patch 69/78] kasan: simplify assign_tag and set_tag calls Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 71/78] kasan: sanitize objects when metadata doesn't fit Andrew Morton
                   ` (7 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: clarify comment in __kasan_kfree_large

Currently it says that the memory gets poisoned by page_alloc code. 
Clarify this by mentioning the specific callback that poisons the memory.

Link: https://lkml.kernel.org/r/1c8380fe0332a3bcc720fe29f1e0bef2e2974416.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/I1334dffb69b87d7986fab88a1a039cc3ea764725
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/mm/kasan/common.c~kasan-clarify-comment-in-__kasan_kfree_large
+++ a/mm/kasan/common.c
@@ -446,5 +446,5 @@ void __kasan_kfree_large(void *ptr, unsi
 {
 	if (ptr != page_address(virt_to_head_page(ptr)))
 		kasan_report_invalid_free(ptr, ip);
-	/* The object will be poisoned by page_alloc. */
+	/* The object will be poisoned by kasan_free_pages(). */
 }
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 71/78] kasan: sanitize objects when metadata doesn't fit
  2020-12-18 22:00 incoming Andrew Morton
                   ` (69 preceding siblings ...)
  2020-12-18 22:05 ` [patch 70/78] kasan: clarify comment in __kasan_kfree_large Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 72/78] kasan, mm: allow cache merging with no metadata Andrew Morton
                   ` (6 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, Vincenzo.Frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: sanitize objects when metadata doesn't fit

KASAN marks caches that are sanitized with the SLAB_KASAN cache flag. 
Currently if the metadata that is appended after the object (stores e.g. 
stack trace ids) doesn't fit into KMALLOC_MAX_SIZE (can only happen with
SLAB, see the comment in the patch), KASAN turns off sanitization
completely.

With this change sanitization of the object data is always enabled. 
However the metadata is only stored when it fits.  Instead of checking for
SLAB_KASAN flag accross the code to find out whether the metadata is
there, use cache->kasan_info.alloc/free_meta_offset.  As 0 can be a valid
value for free_meta_offset, introduce KASAN_NO_FREE_META as an indicator
that the free metadata is missing.

Without this change all sanitized KASAN objects would be put into
quarantine with generic KASAN.  With this change, only the objects that
have metadata (i.e.  when it fits) are put into quarantine, the rest is
freed right away.

Along the way rework __kasan_cache_create() and add claryfying comments.

Link: https://lkml.kernel.org/r/aee34b87a5e4afe586c2ac6a0b32db8dc4dcc2dc.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Icd947e2bea054cb5cfbdc6cf6652227d97032dcb
Co-developed-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/kasan/common.c         |  116 ++++++++++++++++++++++--------------
 mm/kasan/generic.c        |    9 +-
 mm/kasan/hw_tags.c        |    6 +
 mm/kasan/kasan.h          |   17 ++++-
 mm/kasan/quarantine.c     |   18 ++++-
 mm/kasan/report.c         |   43 +++++++------
 mm/kasan/report_sw_tags.c |    9 +-
 mm/kasan/sw_tags.c        |    4 +
 8 files changed, 147 insertions(+), 75 deletions(-)

--- a/mm/kasan/common.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/common.c
@@ -114,9 +114,6 @@ void __kasan_free_pages(struct page *pag
  */
 static inline unsigned int optimal_redzone(unsigned int object_size)
 {
-	if (!IS_ENABLED(CONFIG_KASAN_GENERIC))
-		return 0;
-
 	return
 		object_size <= 64        - 16   ? 16 :
 		object_size <= 128       - 32   ? 32 :
@@ -130,47 +127,77 @@ static inline unsigned int optimal_redzo
 void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size,
 			  slab_flags_t *flags)
 {
-	unsigned int orig_size = *size;
-	unsigned int redzone_size;
-	int redzone_adjust;
+	unsigned int ok_size;
+	unsigned int optimal_size;
+
+	/*
+	 * SLAB_KASAN is used to mark caches as ones that are sanitized by
+	 * KASAN. Currently this flag is used in two places:
+	 * 1. In slab_ksize() when calculating the size of the accessible
+	 *    memory within the object.
+	 * 2. In slab_common.c to prevent merging of sanitized caches.
+	 */
+	*flags |= SLAB_KASAN;
 
-	if (!kasan_stack_collection_enabled()) {
-		*flags |= SLAB_KASAN;
+	if (!kasan_stack_collection_enabled())
 		return;
-	}
 
-	/* Add alloc meta. */
+	ok_size = *size;
+
+	/* Add alloc meta into redzone. */
 	cache->kasan_info.alloc_meta_offset = *size;
 	*size += sizeof(struct kasan_alloc_meta);
 
-	/* Add free meta. */
-	if (IS_ENABLED(CONFIG_KASAN_GENERIC) &&
-	    (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor ||
-	     cache->object_size < sizeof(struct kasan_free_meta))) {
-		cache->kasan_info.free_meta_offset = *size;
-		*size += sizeof(struct kasan_free_meta);
+	/*
+	 * If alloc meta doesn't fit, don't add it.
+	 * This can only happen with SLAB, as it has KMALLOC_MAX_SIZE equal
+	 * to KMALLOC_MAX_CACHE_SIZE and doesn't fall back to page_alloc for
+	 * larger sizes.
+	 */
+	if (*size > KMALLOC_MAX_SIZE) {
+		cache->kasan_info.alloc_meta_offset = 0;
+		*size = ok_size;
+		/* Continue, since free meta might still fit. */
 	}
 
-	redzone_size = optimal_redzone(cache->object_size);
-	redzone_adjust = redzone_size -	(*size - cache->object_size);
-	if (redzone_adjust > 0)
-		*size += redzone_adjust;
-
-	*size = min_t(unsigned int, KMALLOC_MAX_SIZE,
-			max(*size, cache->object_size + redzone_size));
+	/* Only the generic mode uses free meta or flexible redzones. */
+	if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) {
+		cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META;
+		return;
+	}
 
 	/*
-	 * If the metadata doesn't fit, don't enable KASAN at all.
+	 * Add free meta into redzone when it's not possible to store
+	 * it in the object. This is the case when:
+	 * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that it can
+	 *    be touched after it was freed, or
+	 * 2. Object has a constructor, which means it's expected to
+	 *    retain its content until the next allocation, or
+	 * 3. Object is too small.
+	 * Otherwise cache->kasan_info.free_meta_offset = 0 is implied.
 	 */
-	if (*size <= cache->kasan_info.alloc_meta_offset ||
-			*size <= cache->kasan_info.free_meta_offset) {
-		cache->kasan_info.alloc_meta_offset = 0;
-		cache->kasan_info.free_meta_offset = 0;
-		*size = orig_size;
-		return;
+	if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor ||
+	    cache->object_size < sizeof(struct kasan_free_meta)) {
+		ok_size = *size;
+
+		cache->kasan_info.free_meta_offset = *size;
+		*size += sizeof(struct kasan_free_meta);
+
+		/* If free meta doesn't fit, don't add it. */
+		if (*size > KMALLOC_MAX_SIZE) {
+			cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META;
+			*size = ok_size;
+		}
 	}
 
-	*flags |= SLAB_KASAN;
+	/* Calculate size with optimal redzone. */
+	optimal_size = cache->object_size + optimal_redzone(cache->object_size);
+	/* Limit it with KMALLOC_MAX_SIZE (relevant for SLAB only). */
+	if (optimal_size > KMALLOC_MAX_SIZE)
+		optimal_size = KMALLOC_MAX_SIZE;
+	/* Use optimal size if the size with added metas is not large enough. */
+	if (*size < optimal_size)
+		*size = optimal_size;
 }
 
 size_t __kasan_metadata_size(struct kmem_cache *cache)
@@ -186,15 +213,21 @@ size_t __kasan_metadata_size(struct kmem
 struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
 					      const void *object)
 {
+	if (!cache->kasan_info.alloc_meta_offset)
+		return NULL;
 	return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset;
 }
 
+#ifdef CONFIG_KASAN_GENERIC
 struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
 					    const void *object)
 {
 	BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
+	if (cache->kasan_info.free_meta_offset == KASAN_NO_FREE_META)
+		return NULL;
 	return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset;
 }
+#endif
 
 void __kasan_poison_slab(struct page *page)
 {
@@ -271,11 +304,9 @@ void * __must_check __kasan_init_slab_ob
 	struct kasan_alloc_meta *alloc_meta;
 
 	if (kasan_stack_collection_enabled()) {
-		if (!(cache->flags & SLAB_KASAN))
-			return (void *)object;
-
 		alloc_meta = kasan_get_alloc_meta(cache, object);
-		__memset(alloc_meta, 0, sizeof(*alloc_meta));
+		if (alloc_meta)
+			__memset(alloc_meta, 0, sizeof(*alloc_meta));
 	}
 
 	/* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */
@@ -314,15 +345,12 @@ static bool ____kasan_slab_free(struct k
 	if (!kasan_stack_collection_enabled())
 		return false;
 
-	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
-			unlikely(!(cache->flags & SLAB_KASAN)))
+	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine))
 		return false;
 
 	kasan_set_free_info(cache, object, tag);
 
-	quarantine_put(cache, object);
-
-	return IS_ENABLED(CONFIG_KASAN_GENERIC);
+	return quarantine_put(cache, object);
 }
 
 bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip)
@@ -355,7 +383,11 @@ void __kasan_slab_free_mempool(void *ptr
 
 static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)
 {
-	kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
+	struct kasan_alloc_meta *alloc_meta;
+
+	alloc_meta = kasan_get_alloc_meta(cache, object);
+	if (alloc_meta)
+		kasan_set_track(&alloc_meta->alloc_track, flags);
 }
 
 static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object,
@@ -382,7 +414,7 @@ static void *____kasan_kmalloc(struct km
 	poison_range((void *)redzone_start, redzone_end - redzone_start,
 		     KASAN_KMALLOC_REDZONE);
 
-	if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN))
+	if (kasan_stack_collection_enabled())
 		set_alloc_info(cache, (void *)object, flags);
 
 	return set_tag(object, tag);
--- a/mm/kasan/generic.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/generic.c
@@ -348,11 +348,11 @@ void kasan_set_free_info(struct kmem_cac
 	struct kasan_free_meta *free_meta;
 
 	free_meta = kasan_get_free_meta(cache, object);
-	kasan_set_track(&free_meta->free_track, GFP_NOWAIT);
+	if (!free_meta)
+		return;
 
-	/*
-	 *  the object was freed and has free track set
-	 */
+	kasan_set_track(&free_meta->free_track, GFP_NOWAIT);
+	/* The object was freed and has free track set. */
 	*(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREETRACK;
 }
 
@@ -361,5 +361,6 @@ struct kasan_track *kasan_get_free_track
 {
 	if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK)
 		return NULL;
+	/* Free meta must be present with KASAN_KMALLOC_FREETRACK. */
 	return &kasan_get_free_meta(cache, object)->free_track;
 }
--- a/mm/kasan/hw_tags.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/hw_tags.c
@@ -187,7 +187,8 @@ void kasan_set_free_info(struct kmem_cac
 	struct kasan_alloc_meta *alloc_meta;
 
 	alloc_meta = kasan_get_alloc_meta(cache, object);
-	kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT);
+	if (alloc_meta)
+		kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT);
 }
 
 struct kasan_track *kasan_get_free_track(struct kmem_cache *cache,
@@ -196,5 +197,8 @@ struct kasan_track *kasan_get_free_track
 	struct kasan_alloc_meta *alloc_meta;
 
 	alloc_meta = kasan_get_alloc_meta(cache, object);
+	if (!alloc_meta)
+		return NULL;
+
 	return &alloc_meta->free_track[0];
 }
--- a/mm/kasan/kasan.h~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/kasan.h
@@ -156,20 +156,31 @@ struct kasan_alloc_meta {
 struct qlist_node {
 	struct qlist_node *next;
 };
+
+/*
+ * Generic mode either stores free meta in the object itself or in the redzone
+ * after the object. In the former case free meta offset is 0, in the latter
+ * case it has some sane value smaller than INT_MAX. Use INT_MAX as free meta
+ * offset when free meta isn't present.
+ */
+#define KASAN_NO_FREE_META INT_MAX
+
 struct kasan_free_meta {
+#ifdef CONFIG_KASAN_GENERIC
 	/* This field is used while the object is in the quarantine.
 	 * Otherwise it might be used for the allocator freelist.
 	 */
 	struct qlist_node quarantine_link;
-#ifdef CONFIG_KASAN_GENERIC
 	struct kasan_track free_track;
 #endif
 };
 
 struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
 						const void *object);
+#ifdef CONFIG_KASAN_GENERIC
 struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
 						const void *object);
+#endif
 
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 
@@ -234,11 +245,11 @@ struct kasan_track *kasan_get_free_track
 
 #if defined(CONFIG_KASAN_GENERIC) && \
 	(defined(CONFIG_SLAB) || defined(CONFIG_SLUB))
-void quarantine_put(struct kmem_cache *cache, void *object);
+bool quarantine_put(struct kmem_cache *cache, void *object);
 void quarantine_reduce(void);
 void quarantine_remove_cache(struct kmem_cache *cache);
 #else
-static inline void quarantine_put(struct kmem_cache *cache, void *object) { }
+static inline bool quarantine_put(struct kmem_cache *cache, void *object) { return false; }
 static inline void quarantine_reduce(void) { }
 static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
 #endif
--- a/mm/kasan/quarantine.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/quarantine.c
@@ -137,7 +137,12 @@ static void qlink_free(struct qlist_node
 	if (IS_ENABLED(CONFIG_SLAB))
 		local_irq_save(flags);
 
+	/*
+	 * As the object now gets freed from the quaratine, assume that its
+	 * free track is no longer valid.
+	 */
 	*(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREE;
+
 	___cache_free(cache, object, _THIS_IP_);
 
 	if (IS_ENABLED(CONFIG_SLAB))
@@ -163,7 +168,7 @@ static void qlist_free_all(struct qlist_
 	qlist_init(q);
 }
 
-void quarantine_put(struct kmem_cache *cache, void *object)
+bool quarantine_put(struct kmem_cache *cache, void *object)
 {
 	unsigned long flags;
 	struct qlist_head *q;
@@ -171,6 +176,13 @@ void quarantine_put(struct kmem_cache *c
 	struct kasan_free_meta *meta = kasan_get_free_meta(cache, object);
 
 	/*
+	 * If there's no metadata for this object, don't put it into
+	 * quarantine.
+	 */
+	if (!meta)
+		return false;
+
+	/*
 	 * Note: irq must be disabled until after we move the batch to the
 	 * global quarantine. Otherwise quarantine_remove_cache() can miss
 	 * some objects belonging to the cache if they are in our local temp
@@ -183,7 +195,7 @@ void quarantine_put(struct kmem_cache *c
 	q = this_cpu_ptr(&cpu_quarantine);
 	if (q->offline) {
 		local_irq_restore(flags);
-		return;
+		return false;
 	}
 	qlist_put(q, &meta->quarantine_link, cache->size);
 	if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) {
@@ -206,6 +218,8 @@ void quarantine_put(struct kmem_cache *c
 	}
 
 	local_irq_restore(flags);
+
+	return true;
 }
 
 void quarantine_reduce(void)
--- a/mm/kasan/report.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/report.c
@@ -168,32 +168,35 @@ static void describe_object_addr(struct
 static void describe_object_stacks(struct kmem_cache *cache, void *object,
 					const void *addr, u8 tag)
 {
-	struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object);
-
-	if (cache->flags & SLAB_KASAN) {
-		struct kasan_track *free_track;
+	struct kasan_alloc_meta *alloc_meta;
+	struct kasan_track *free_track;
 
+	alloc_meta = kasan_get_alloc_meta(cache, object);
+	if (alloc_meta) {
 		print_track(&alloc_meta->alloc_track, "Allocated");
 		pr_err("\n");
-		free_track = kasan_get_free_track(cache, object, tag);
-		if (free_track) {
-			print_track(free_track, "Freed");
-			pr_err("\n");
-		}
+	}
+
+	free_track = kasan_get_free_track(cache, object, tag);
+	if (free_track) {
+		print_track(free_track, "Freed");
+		pr_err("\n");
+	}
 
 #ifdef CONFIG_KASAN_GENERIC
-		if (alloc_meta->aux_stack[0]) {
-			pr_err("Last potentially related work creation:\n");
-			print_stack(alloc_meta->aux_stack[0]);
-			pr_err("\n");
-		}
-		if (alloc_meta->aux_stack[1]) {
-			pr_err("Second to last potentially related work creation:\n");
-			print_stack(alloc_meta->aux_stack[1]);
-			pr_err("\n");
-		}
-#endif
+	if (!alloc_meta)
+		return;
+	if (alloc_meta->aux_stack[0]) {
+		pr_err("Last potentially related work creation:\n");
+		print_stack(alloc_meta->aux_stack[0]);
+		pr_err("\n");
 	}
+	if (alloc_meta->aux_stack[1]) {
+		pr_err("Second to last potentially related work creation:\n");
+		print_stack(alloc_meta->aux_stack[1]);
+		pr_err("\n");
+	}
+#endif
 }
 
 static void describe_object(struct kmem_cache *cache, void *object,
--- a/mm/kasan/report_sw_tags.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/report_sw_tags.c
@@ -48,9 +48,12 @@ const char *get_bug_type(struct kasan_ac
 		object = nearest_obj(cache, page, (void *)addr);
 		alloc_meta = kasan_get_alloc_meta(cache, object);
 
-		for (i = 0; i < KASAN_NR_FREE_STACKS; i++)
-			if (alloc_meta->free_pointer_tag[i] == tag)
-				return "use-after-free";
+		if (alloc_meta) {
+			for (i = 0; i < KASAN_NR_FREE_STACKS; i++) {
+				if (alloc_meta->free_pointer_tag[i] == tag)
+					return "use-after-free";
+			}
+		}
 		return "out-of-bounds";
 	}
 
--- a/mm/kasan/sw_tags.c~kasan-sanitize-objects-when-metadata-doesnt-fit
+++ a/mm/kasan/sw_tags.c
@@ -170,6 +170,8 @@ void kasan_set_free_info(struct kmem_cac
 	u8 idx = 0;
 
 	alloc_meta = kasan_get_alloc_meta(cache, object);
+	if (!alloc_meta)
+		return;
 
 #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY
 	idx = alloc_meta->free_track_idx;
@@ -187,6 +189,8 @@ struct kasan_track *kasan_get_free_track
 	int i = 0;
 
 	alloc_meta = kasan_get_alloc_meta(cache, object);
+	if (!alloc_meta)
+		return NULL;
 
 #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY
 	for (i = 0; i < KASAN_NR_FREE_STACKS; i++) {
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 72/78] kasan, mm: allow cache merging with no metadata
  2020-12-18 22:00 incoming Andrew Morton
                   ` (70 preceding siblings ...)
  2020-12-18 22:05 ` [patch 71/78] kasan: sanitize objects when metadata doesn't fit Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 73/78] kasan: update documentation Andrew Morton
                   ` (5 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, Vincenzo.Frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan, mm: allow cache merging with no metadata

The reason cache merging is disabled with KASAN is because KASAN puts its
metadata right after the allocated object. When the merged caches have
slightly different sizes, the metadata ends up in different places, which
KASAN doesn't support.

It might be possible to adjust the metadata allocation algorithm and make
it friendly to the cache merging code. Instead this change takes a simpler
approach and allows merging caches when no metadata is present. Which is
the case for hardware tag-based KASAN with kasan.mode=prod.

Link: https://lkml.kernel.org/r/37497e940bfd4b32c0a93a702a9ae4cf061d5392.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba
Co-developed-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Vincenzo Frascino <Vincenzo.Frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/kasan.h |   21 +++++++++++++++++++--
 mm/kasan/common.c     |   11 +++++++++++
 mm/slab_common.c      |    3 ++-
 3 files changed, 32 insertions(+), 3 deletions(-)

--- a/include/linux/kasan.h~kasan-mm-allow-cache-merging-with-no-metadata
+++ a/include/linux/kasan.h
@@ -82,17 +82,30 @@ struct kasan_cache {
 };
 
 #ifdef CONFIG_KASAN_HW_TAGS
+
 DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled);
+
 static __always_inline bool kasan_enabled(void)
 {
 	return static_branch_likely(&kasan_flag_enabled);
 }
-#else
+
+#else /* CONFIG_KASAN_HW_TAGS */
+
 static inline bool kasan_enabled(void)
 {
 	return true;
 }
-#endif
+
+#endif /* CONFIG_KASAN_HW_TAGS */
+
+slab_flags_t __kasan_never_merge(void);
+static __always_inline slab_flags_t kasan_never_merge(void)
+{
+	if (kasan_enabled())
+		return __kasan_never_merge();
+	return 0;
+}
 
 void __kasan_unpoison_range(const void *addr, size_t size);
 static __always_inline void kasan_unpoison_range(const void *addr, size_t size)
@@ -239,6 +252,10 @@ static inline bool kasan_enabled(void)
 {
 	return false;
 }
+static inline slab_flags_t kasan_never_merge(void)
+{
+	return 0;
+}
 static inline void kasan_unpoison_range(const void *address, size_t size) {}
 static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
 static inline void kasan_free_pages(struct page *page, unsigned int order) {}
--- a/mm/kasan/common.c~kasan-mm-allow-cache-merging-with-no-metadata
+++ a/mm/kasan/common.c
@@ -86,6 +86,17 @@ asmlinkage void kasan_unpoison_task_stac
 }
 #endif /* CONFIG_KASAN_STACK */
 
+/*
+ * Only allow cache merging when stack collection is disabled and no metadata
+ * is present.
+ */
+slab_flags_t __kasan_never_merge(void)
+{
+	if (kasan_stack_collection_enabled())
+		return SLAB_KASAN;
+	return 0;
+}
+
 void __kasan_alloc_pages(struct page *page, unsigned int order)
 {
 	u8 tag;
--- a/mm/slab_common.c~kasan-mm-allow-cache-merging-with-no-metadata
+++ a/mm/slab_common.c
@@ -18,6 +18,7 @@
 #include <linux/seq_file.h>
 #include <linux/proc_fs.h>
 #include <linux/debugfs.h>
+#include <linux/kasan.h>
 #include <asm/cacheflush.h>
 #include <asm/tlbflush.h>
 #include <asm/page.h>
@@ -53,7 +54,7 @@ static DECLARE_WORK(slab_caches_to_rcu_d
  */
 #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
 		SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \
-		SLAB_FAILSLAB | SLAB_KASAN)
+		SLAB_FAILSLAB | kasan_never_merge())
 
 #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
 			 SLAB_CACHE_DMA32 | SLAB_ACCOUNT)
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 73/78] kasan: update documentation
  2020-12-18 22:00 incoming Andrew Morton
                   ` (71 preceding siblings ...)
  2020-12-18 22:05 ` [patch 72/78] kasan, mm: allow cache merging with no metadata Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 74/78] mm/Kconfig: fix spelling mistake "whats" -> "what's" Andrew Morton
                   ` (4 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, elver, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

From: Andrey Konovalov <andreyknvl@google.com>
Subject: kasan: update documentation

This change updates KASAN documentation to reflect the addition of boot
parameters and also reworks and clarifies some of the existing sections,
in particular: defines what a memory granule is, mentions quarantine,
makes Kunit section more readable.

Link: https://lkml.kernel.org/r/748daf013e17d925b0fe00c1c3b5dce726dd2430.1606162397.git.andreyknvl@google.com
Link: https://linux-review.googlesource.com/id/Ib1f83e91be273264b25f42b04448ac96b858849f
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Branislav Rankov <Branislav.Rankov@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 Documentation/dev-tools/kasan.rst |  188 +++++++++++++++++-----------
 1 file changed, 117 insertions(+), 71 deletions(-)

--- a/Documentation/dev-tools/kasan.rst~kasan-update-documentation
+++ a/Documentation/dev-tools/kasan.rst
@@ -4,8 +4,9 @@ The Kernel Address Sanitizer (KASAN)
 Overview
 --------
 
-KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to
-find out-of-bound and use-after-free bugs. KASAN has three modes:
+KernelAddressSANitizer (KASAN) is a dynamic memory safety error detector
+designed to find out-of-bound and use-after-free bugs. KASAN has three modes:
+
 1. generic KASAN (similar to userspace ASan),
 2. software tag-based KASAN (similar to userspace HWASan),
 3. hardware tag-based KASAN (based on hardware memory tagging).
@@ -39,23 +40,13 @@ CONFIG_KASAN_INLINE. Outline and inline
 The former produces smaller binary while the latter is 1.1 - 2 times faster.
 
 Both software KASAN modes work with both SLUB and SLAB memory allocators,
-hardware tag-based KASAN currently only support SLUB.
-For better bug detection and nicer reporting, enable CONFIG_STACKTRACE.
+while the hardware tag-based KASAN currently only support SLUB.
+
+For better error reports that include stack traces, enable CONFIG_STACKTRACE.
 
 To augment reports with last allocation and freeing stack of the physical page,
 it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on.
 
-To disable instrumentation for specific files or directories, add a line
-similar to the following to the respective kernel Makefile:
-
-- For a single file (e.g. main.o)::
-
-    KASAN_SANITIZE_main.o := n
-
-- For all files in one directory::
-
-    KASAN_SANITIZE := n
-
 Error reports
 ~~~~~~~~~~~~~
 
@@ -140,22 +131,75 @@ freed (in case of a use-after-free bug r
 the accessed slab object and information about the accessed memory page.
 
 In the last section the report shows memory state around the accessed address.
-Reading this part requires some understanding of how KASAN works.
-
-The state of each 8 aligned bytes of memory is encoded in one shadow byte.
-Those 8 bytes can be accessible, partially accessible, freed or be a redzone.
-We use the following encoding for each shadow byte: 0 means that all 8 bytes
-of the corresponding memory region are accessible; number N (1 <= N <= 7) means
-that the first N bytes are accessible, and other (8 - N) bytes are not;
-any negative value indicates that the entire 8-byte word is inaccessible.
-We use different negative values to distinguish between different kinds of
-inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h).
+Internally KASAN tracks memory state separately for each memory granule, which
+is either 8 or 16 aligned bytes depending on KASAN mode. Each number in the
+memory state section of the report shows the state of one of the memory
+granules that surround the accessed address.
+
+For generic KASAN the size of each memory granule is 8. The state of each
+granule is encoded in one shadow byte. Those 8 bytes can be accessible,
+partially accessible, freed or be a part of a redzone. KASAN uses the following
+encoding for each shadow byte: 0 means that all 8 bytes of the corresponding
+memory region are accessible; number N (1 <= N <= 7) means that the first N
+bytes are accessible, and other (8 - N) bytes are not; any negative value
+indicates that the entire 8-byte word is inaccessible. KASAN uses different
+negative values to distinguish between different kinds of inaccessible memory
+like redzones or freed memory (see mm/kasan/kasan.h).
 
 In the report above the arrows point to the shadow byte 03, which means that
 the accessed address is partially accessible.
 
 For tag-based KASAN this last report section shows the memory tags around the
-accessed address (see Implementation details section).
+accessed address (see `Implementation details`_ section).
+
+Boot parameters
+~~~~~~~~~~~~~~~
+
+Hardware tag-based KASAN mode (see the section about different mode below) is
+intended for use in production as a security mitigation. Therefore it supports
+boot parameters that allow to disable KASAN competely or otherwise control
+particular KASAN features.
+
+The things that can be controlled are:
+
+1. Whether KASAN is enabled at all.
+2. Whether KASAN collects and saves alloc/free stacks.
+3. Whether KASAN panics on a detected bug or not.
+
+The ``kasan.mode`` boot parameter allows to choose one of three main modes:
+
+- ``kasan.mode=off`` - KASAN is disabled, no tag checks are performed
+- ``kasan.mode=prod`` - only essential production features are enabled
+- ``kasan.mode=full`` - all KASAN features are enabled
+
+The chosen mode provides default control values for the features mentioned
+above. However it's also possible to override the default values by providing:
+
+- ``kasan.stacktrace=off`` or ``=on`` - enable alloc/free stack collection
+					(default: ``on`` for ``mode=full``,
+					 otherwise ``off``)
+- ``kasan.fault=report`` or ``=panic`` - only print KASAN report or also panic
+					 (default: ``report``)
+
+If ``kasan.mode`` parameter is not provided, it defaults to ``full`` when
+``CONFIG_DEBUG_KERNEL`` is enabled, and to ``prod`` otherwise.
+
+For developers
+~~~~~~~~~~~~~~
+
+Software KASAN modes use compiler instrumentation to insert validity checks.
+Such instrumentation might be incompatible with some part of the kernel, and
+therefore needs to be disabled. To disable instrumentation for specific files
+or directories, add a line similar to the following to the respective kernel
+Makefile:
+
+- For a single file (e.g. main.o)::
+
+    KASAN_SANITIZE_main.o := n
+
+- For all files in one directory::
+
+    KASAN_SANITIZE := n
 
 
 Implementation details
@@ -164,10 +208,10 @@ Implementation details
 Generic KASAN
 ~~~~~~~~~~~~~
 
-From a high level, our approach to memory error detection is similar to that
-of kmemcheck: use shadow memory to record whether each byte of memory is safe
-to access, and use compile-time instrumentation to insert checks of shadow
-memory on each memory access.
+From a high level perspective, KASAN's approach to memory error detection is
+similar to that of kmemcheck: use shadow memory to record whether each byte of
+memory is safe to access, and use compile-time instrumentation to insert checks
+of shadow memory on each memory access.
 
 Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB
 to cover 128TB on x86_64) and uses direct mapping with a scale and offset to
@@ -198,6 +242,9 @@ Generic KASAN also reports the last 2 ca
 potentially has access to an object. Call stacks for the following are shown:
 call_rcu() and workqueue queuing.
 
+Generic KASAN is the only mode that delays the reuse of freed object via
+quarantine (see mm/kasan/quarantine.c for implementation).
+
 Software tag-based KASAN
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -305,15 +352,15 @@ therefore be wasteful. Furthermore, to e
 use different shadow pages, mappings would have to be aligned to
 ``KASAN_GRANULE_SIZE * PAGE_SIZE``.
 
-Instead, we share backing space across multiple mappings. We allocate
+Instead, KASAN shares backing space across multiple mappings. It allocates
 a backing page when a mapping in vmalloc space uses a particular page
 of the shadow region. This page can be shared by other vmalloc
 mappings later on.
 
-We hook in to the vmap infrastructure to lazily clean up unused shadow
+KASAN hooks into the vmap infrastructure to lazily clean up unused shadow
 memory.
 
-To avoid the difficulties around swapping mappings around, we expect
+To avoid the difficulties around swapping mappings around, KASAN expects
 that the part of the shadow region that covers the vmalloc space will
 not be covered by the early shadow page, but will be left
 unmapped. This will require changes in arch-specific code.
@@ -324,24 +371,31 @@ architectures that do not have a fixed m
 CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE
 --------------------------------------------------
 
-``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing.
-This means each test focuses on a small unit of functionality and
-there are a few ways these tests can be run.
+KASAN tests consist on two parts:
+
+1. Tests that are integrated with the KUnit Test Framework. Enabled with
+``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified
+automatically in a few different ways, see the instructions below.
+
+2. Tests that are currently incompatible with KUnit. Enabled with
+``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can
+only be verified manually, by loading the kernel module and inspecting the
+kernel log for KASAN reports.
 
-Each test will print the KASAN report if an error is detected and then
-print the number of the test and the status of the test:
+Each KUnit-compatible KASAN test prints a KASAN report if an error is detected.
+Then the test prints its number and status.
 
-pass::
+When a test passes::
 
         ok 28 - kmalloc_double_kzfree
 
-or, if kmalloc failed::
+When a test fails due to a failed ``kmalloc``::
 
         # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163
         Expected ptr is not null, but is
         not ok 4 - kmalloc_large_oob_right
 
-or, if a KASAN report was expected, but not found::
+When a test fails due to a missing KASAN report::
 
         # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629
         Expected kasan_data->report_expected == kasan_data->report_found, but
@@ -349,46 +403,38 @@ or, if a KASAN report was expected, but
         kasan_data->report_found == 0
         not ok 28 - kmalloc_double_kzfree
 
-All test statuses are tracked as they run and an overall status will
-be printed at the end::
+At the end the cumulative status of all KASAN tests is printed. On success::
 
         ok 1 - kasan
 
-or::
+Or, if one of the tests failed::
 
         not ok 1 - kasan
 
-(1) Loadable Module
-~~~~~~~~~~~~~~~~~~~~
+
+There are a few ways to run KUnit-compatible KASAN tests.
+
+1. Loadable module
+~~~~~~~~~~~~~~~~~~
 
 With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as
-a loadable module and run on any architecture that supports KASAN
-using something like insmod or modprobe. The module is called ``test_kasan``.
+a loadable module and run on any architecture that supports KASAN by loading
+the module with insmod or modprobe. The module is called ``test_kasan``.
 
-(2) Built-In
-~~~~~~~~~~~~~
+2. Built-In
+~~~~~~~~~~~
 
 With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in
-on any architecture that supports KASAN. These and any other KUnit
-tests enabled will run and print the results at boot as a late-init
-call.
-
-(3) Using kunit_tool
-~~~~~~~~~~~~~~~~~~~~~
-
-With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also
-use kunit_tool to see the results of these along with other KUnit
-tests in a more readable way. This will not print the KASAN reports
-of tests that passed. Use `KUnit documentation <https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>`_ for more up-to-date
-information on kunit_tool.
+on any architecure that supports KASAN. These and any other KUnit tests enabled
+will run and print the results at boot as a late-init call.
 
-.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html
+3. Using kunit_tool
+~~~~~~~~~~~~~~~~~~~
 
-``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be
-converted to KUnit. These tests can be run only as a module with
-``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and
-``CONFIG_KASAN`` built-in. The type of error expected and the
-function being run is printed before the expression expected to give
-an error. Then the error is printed, if found, and that test
-should be interpreted to pass only if the error was the one expected
-by the test.
+With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also
+possible use ``kunit_tool`` to see the results of these and other KUnit tests
+in a more readable way. This will not print the KASAN reports of the tests that
+passed. Use `KUnit documentation <https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>`_
+for more up-to-date information on ``kunit_tool``.
+
+.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 74/78] mm/Kconfig: fix spelling mistake "whats" -> "what's"
  2020-12-18 22:00 incoming Andrew Morton
                   ` (72 preceding siblings ...)
  2020-12-18 22:05 ` [patch 73/78] kasan: update documentation Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 75/78] epoll: convert internal api to timespec64 Andrew Morton
                   ` (3 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, colin.king, linux-mm, mm-commits, torvalds

From: Colin Ian King <colin.king@canonical.com>
Subject: mm/Kconfig: fix spelling mistake "whats" -> "what's"

There is a spelling mistake in the Kconfig help text. Fix it.

Link: https://lkml.kernel.org/r/20201217172717.58203-1-colin.king@canonical.com
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/Kconfig |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/mm/Kconfig~mm-fix-spelling-mistake-in-kconfig-whats-whats
+++ a/mm/Kconfig
@@ -713,7 +713,7 @@ config ZSMALLOC_STAT
 	select DEBUG_FS
 	help
 	  This option enables code in the zsmalloc to collect various
-	  statistics about whats happening in zsmalloc and exports that
+	  statistics about what's happening in zsmalloc and exports that
 	  information to userspace via debugfs.
 	  If unsure, say N.
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 75/78] epoll: convert internal api to timespec64
  2020-12-18 22:00 incoming Andrew Morton
                   ` (73 preceding siblings ...)
  2020-12-18 22:05 ` [patch 74/78] mm/Kconfig: fix spelling mistake "whats" -> "what's" Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 76/78] epoll: add syscall epoll_pwait2 Andrew Morton
                   ` (2 subsequent siblings)
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, arnd, linux-mm, mm-commits, torvalds, viro, willemb, willy

From: Willem de Bruijn <willemb@google.com>
Subject: epoll: convert internal api to timespec64

Patch series "add epoll_pwait2 syscall", v4.

Enable nanosecond timeouts for epoll.

Analogous to pselect and ppoll, introduce an epoll_wait syscall
variant that takes a struct timespec instead of int timeout.


This patch (of 4):

Make epoll more consistent with select/poll: pass along the timeout as
timespec64 pointer.

In anticipation of additional changes affecting all three polling
mechanisms:

- add epoll_pwait2 syscall with timespec semantics,
  and share poll_select_set_timeout implementation.
- compute slack before conversion to absolute time,
  to save one ktime_get_ts64 call.

Link: https://lkml.kernel.org/r/20201121144401.3727659-1-willemdebruijn.kernel@gmail.com
Link: https://lkml.kernel.org/r/20201121144401.3727659-2-willemdebruijn.kernel@gmail.com
Signed-off-by: Willem de Bruijn <willemb@google.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   57 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 37 insertions(+), 20 deletions(-)

--- a/fs/eventpoll.c~epoll-convert-internal-api-to-timespec64
+++ a/fs/eventpoll.c
@@ -1712,15 +1712,25 @@ static int ep_send_events(struct eventpo
 	return res;
 }
 
-static inline struct timespec64 ep_set_mstimeout(long ms)
+static struct timespec64 *ep_timeout_to_timespec(struct timespec64 *to, long ms)
 {
-	struct timespec64 now, ts = {
-		.tv_sec = ms / MSEC_PER_SEC,
-		.tv_nsec = NSEC_PER_MSEC * (ms % MSEC_PER_SEC),
-	};
+	struct timespec64 now;
+
+	if (ms < 0)
+		return NULL;
+
+	if (!ms) {
+		to->tv_sec = 0;
+		to->tv_nsec = 0;
+		return to;
+	}
+
+	to->tv_sec = ms / MSEC_PER_SEC;
+	to->tv_nsec = NSEC_PER_MSEC * (ms % MSEC_PER_SEC);
 
 	ktime_get_ts64(&now);
-	return timespec64_add_safe(now, ts);
+	*to = timespec64_add_safe(now, *to);
+	return to;
 }
 
 /**
@@ -1732,8 +1742,8 @@ static inline struct timespec64 ep_set_m
  *          stored.
  * @maxevents: Size (in terms of number of events) of the caller event buffer.
  * @timeout: Maximum timeout for the ready events fetch operation, in
- *           milliseconds. If the @timeout is zero, the function will not block,
- *           while if the @timeout is less than zero, the function will block
+ *           timespec. If the timeout is zero, the function will not block,
+ *           while if the @timeout ptr is NULL, the function will block
  *           until at least one event has been retrieved (or an error
  *           occurred).
  *
@@ -1741,7 +1751,7 @@ static inline struct timespec64 ep_set_m
  *          error code, in case of error.
  */
 static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
-		   int maxevents, long timeout)
+		   int maxevents, struct timespec64 *timeout)
 {
 	int res, eavail, timed_out = 0;
 	u64 slack = 0;
@@ -1750,13 +1760,11 @@ static int ep_poll(struct eventpoll *ep,
 
 	lockdep_assert_irqs_enabled();
 
-	if (timeout > 0) {
-		struct timespec64 end_time = ep_set_mstimeout(timeout);
-
-		slack = select_estimate_accuracy(&end_time);
+	if (timeout && (timeout->tv_sec | timeout->tv_nsec)) {
+		slack = select_estimate_accuracy(timeout);
 		to = &expires;
-		*to = timespec64_to_ktime(end_time);
-	} else if (timeout == 0) {
+		*to = timespec64_to_ktime(*timeout);
+	} else if (timeout) {
 		/*
 		 * Avoid the unnecessary trip to the wait queue loop, if the
 		 * caller specified a non blocking operation.
@@ -2175,7 +2183,7 @@ SYSCALL_DEFINE4(epoll_ctl, int, epfd, in
  * part of the user space epoll_wait(2).
  */
 static int do_epoll_wait(int epfd, struct epoll_event __user *events,
-			 int maxevents, int timeout)
+			 int maxevents, struct timespec64 *to)
 {
 	int error;
 	struct fd f;
@@ -2209,7 +2217,7 @@ static int do_epoll_wait(int epfd, struc
 	ep = f.file->private_data;
 
 	/* Time to fish for events ... */
-	error = ep_poll(ep, events, maxevents, timeout);
+	error = ep_poll(ep, events, maxevents, to);
 
 error_fput:
 	fdput(f);
@@ -2219,7 +2227,10 @@ error_fput:
 SYSCALL_DEFINE4(epoll_wait, int, epfd, struct epoll_event __user *, events,
 		int, maxevents, int, timeout)
 {
-	return do_epoll_wait(epfd, events, maxevents, timeout);
+	struct timespec64 to;
+
+	return do_epoll_wait(epfd, events, maxevents,
+			     ep_timeout_to_timespec(&to, timeout));
 }
 
 /*
@@ -2230,6 +2241,7 @@ SYSCALL_DEFINE6(epoll_pwait, int, epfd,
 		int, maxevents, int, timeout, const sigset_t __user *, sigmask,
 		size_t, sigsetsize)
 {
+	struct timespec64 to;
 	int error;
 
 	/*
@@ -2240,7 +2252,9 @@ SYSCALL_DEFINE6(epoll_pwait, int, epfd,
 	if (error)
 		return error;
 
-	error = do_epoll_wait(epfd, events, maxevents, timeout);
+	error = do_epoll_wait(epfd, events, maxevents,
+			      ep_timeout_to_timespec(&to, timeout));
+
 	restore_saved_sigmask_unless(error == -EINTR);
 
 	return error;
@@ -2253,6 +2267,7 @@ COMPAT_SYSCALL_DEFINE6(epoll_pwait, int,
 			const compat_sigset_t __user *, sigmask,
 			compat_size_t, sigsetsize)
 {
+	struct timespec64 to;
 	long err;
 
 	/*
@@ -2263,7 +2278,9 @@ COMPAT_SYSCALL_DEFINE6(epoll_pwait, int,
 	if (err)
 		return err;
 
-	err = do_epoll_wait(epfd, events, maxevents, timeout);
+	err = do_epoll_wait(epfd, events, maxevents,
+			    ep_timeout_to_timespec(&to, timeout));
+
 	restore_saved_sigmask_unless(err == -EINTR);
 
 	return err;
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 76/78] epoll: add syscall epoll_pwait2
  2020-12-18 22:00 incoming Andrew Morton
                   ` (74 preceding siblings ...)
  2020-12-18 22:05 ` [patch 75/78] epoll: convert internal api to timespec64 Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 77/78] epoll: wire up " Andrew Morton
  2020-12-18 22:05 ` [patch 78/78] selftests/filesystems: expand epoll with epoll_pwait2 Andrew Morton
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, arnd, linux-mm, mm-commits, torvalds, viro, willemb, willy

From: Willem de Bruijn <willemb@google.com>
Subject: epoll: add syscall epoll_pwait2

Add syscall epoll_pwait2, an epoll_wait variant with nsec resolution that
replaces int timeout with struct timespec.  It is equivalent otherwise.

    int epoll_pwait2(int fd, struct epoll_event *events,
                     int maxevents,
                     const struct timespec *timeout,
                     const sigset_t *sigset);

The underlying hrtimer is already programmed with nsec resolution. 
pselect and ppoll also set nsec resolution timeout with timespec.

The sigset_t in epoll_pwait has a compat variant. epoll_pwait2 needs
the same.

For timespec, only support this new interface on 2038 aware platforms
that define __kernel_timespec_t. So no CONFIG_COMPAT_32BIT_TIME.

Link: https://lkml.kernel.org/r/20201121144401.3727659-3-willemdebruijn.kernel@gmail.com
Signed-off-by: Willem de Bruijn <willemb@google.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 fs/eventpoll.c |   87 +++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 73 insertions(+), 14 deletions(-)

--- a/fs/eventpoll.c~epoll-add-syscall-epoll_pwait2
+++ a/fs/eventpoll.c
@@ -2237,11 +2237,10 @@ SYSCALL_DEFINE4(epoll_wait, int, epfd, s
  * Implement the event wait interface for the eventpoll file. It is the kernel
  * part of the user space epoll_pwait(2).
  */
-SYSCALL_DEFINE6(epoll_pwait, int, epfd, struct epoll_event __user *, events,
-		int, maxevents, int, timeout, const sigset_t __user *, sigmask,
-		size_t, sigsetsize)
+static int do_epoll_pwait(int epfd, struct epoll_event __user *events,
+			  int maxevents, struct timespec64 *to,
+			  const sigset_t __user *sigmask, size_t sigsetsize)
 {
-	struct timespec64 to;
 	int error;
 
 	/*
@@ -2252,22 +2251,48 @@ SYSCALL_DEFINE6(epoll_pwait, int, epfd,
 	if (error)
 		return error;
 
-	error = do_epoll_wait(epfd, events, maxevents,
-			      ep_timeout_to_timespec(&to, timeout));
+	error = do_epoll_wait(epfd, events, maxevents, to);
 
 	restore_saved_sigmask_unless(error == -EINTR);
 
 	return error;
 }
 
-#ifdef CONFIG_COMPAT
-COMPAT_SYSCALL_DEFINE6(epoll_pwait, int, epfd,
-			struct epoll_event __user *, events,
-			int, maxevents, int, timeout,
-			const compat_sigset_t __user *, sigmask,
-			compat_size_t, sigsetsize)
+SYSCALL_DEFINE6(epoll_pwait, int, epfd, struct epoll_event __user *, events,
+		int, maxevents, int, timeout, const sigset_t __user *, sigmask,
+		size_t, sigsetsize)
 {
 	struct timespec64 to;
+
+	return do_epoll_pwait(epfd, events, maxevents,
+			      ep_timeout_to_timespec(&to, timeout),
+			      sigmask, sigsetsize);
+}
+
+SYSCALL_DEFINE6(epoll_pwait2, int, epfd, struct epoll_event __user *, events,
+		int, maxevents, const struct __kernel_timespec __user *, timeout,
+		const sigset_t __user *, sigmask, size_t, sigsetsize)
+{
+	struct timespec64 ts, *to = NULL;
+
+	if (timeout) {
+		if (get_timespec64(&ts, timeout))
+			return -EFAULT;
+		to = &ts;
+		if (poll_select_set_timeout(to, ts.tv_sec, ts.tv_nsec))
+			return -EINVAL;
+	}
+
+	return do_epoll_pwait(epfd, events, maxevents, to,
+			      sigmask, sigsetsize);
+}
+
+#ifdef CONFIG_COMPAT
+static int do_compat_epoll_pwait(int epfd, struct epoll_event __user *events,
+				 int maxevents, struct timespec64 *timeout,
+				 const compat_sigset_t __user *sigmask,
+				 compat_size_t sigsetsize)
+{
 	long err;
 
 	/*
@@ -2278,13 +2303,47 @@ COMPAT_SYSCALL_DEFINE6(epoll_pwait, int,
 	if (err)
 		return err;
 
-	err = do_epoll_wait(epfd, events, maxevents,
-			    ep_timeout_to_timespec(&to, timeout));
+	err = do_epoll_wait(epfd, events, maxevents, timeout);
 
 	restore_saved_sigmask_unless(err == -EINTR);
 
 	return err;
 }
+
+COMPAT_SYSCALL_DEFINE6(epoll_pwait, int, epfd,
+		       struct epoll_event __user *, events,
+		       int, maxevents, int, timeout,
+		       const compat_sigset_t __user *, sigmask,
+		       compat_size_t, sigsetsize)
+{
+	struct timespec64 to;
+
+	return do_compat_epoll_pwait(epfd, events, maxevents,
+				     ep_timeout_to_timespec(&to, timeout),
+				     sigmask, sigsetsize);
+}
+
+COMPAT_SYSCALL_DEFINE6(epoll_pwait2, int, epfd,
+		       struct epoll_event __user *, events,
+		       int, maxevents,
+		       const struct __kernel_timespec __user *, timeout,
+		       const compat_sigset_t __user *, sigmask,
+		       compat_size_t, sigsetsize)
+{
+	struct timespec64 ts, *to = NULL;
+
+	if (timeout) {
+		if (get_timespec64(&ts, timeout))
+			return -EFAULT;
+		to = &ts;
+		if (poll_select_set_timeout(to, ts.tv_sec, ts.tv_nsec))
+			return -EINVAL;
+	}
+
+	return do_compat_epoll_pwait(epfd, events, maxevents, to,
+				     sigmask, sigsetsize);
+}
+
 #endif
 
 static int __init eventpoll_init(void)
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 77/78] epoll: wire up syscall epoll_pwait2
  2020-12-18 22:00 incoming Andrew Morton
                   ` (75 preceding siblings ...)
  2020-12-18 22:05 ` [patch 76/78] epoll: add syscall epoll_pwait2 Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  2020-12-18 22:05 ` [patch 78/78] selftests/filesystems: expand epoll with epoll_pwait2 Andrew Morton
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, arnd, linux-mm, mm-commits, torvalds, viro, willemb, willy

From: Willem de Bruijn <willemb@google.com>
Subject: epoll: wire up syscall epoll_pwait2

Split off from prev patch in the series that implements the syscall.

Link: https://lkml.kernel.org/r/20201121144401.3727659-4-willemdebruijn.kernel@gmail.com
Signed-off-by: Willem de Bruijn <willemb@google.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/alpha/kernel/syscalls/syscall.tbl      |    1 +
 arch/arm/tools/syscall.tbl                  |    1 +
 arch/arm64/include/asm/unistd.h             |    2 +-
 arch/arm64/include/asm/unistd32.h           |    2 ++
 arch/ia64/kernel/syscalls/syscall.tbl       |    1 +
 arch/m68k/kernel/syscalls/syscall.tbl       |    1 +
 arch/microblaze/kernel/syscalls/syscall.tbl |    1 +
 arch/mips/kernel/syscalls/syscall_n32.tbl   |    1 +
 arch/mips/kernel/syscalls/syscall_n64.tbl   |    1 +
 arch/mips/kernel/syscalls/syscall_o32.tbl   |    1 +
 arch/parisc/kernel/syscalls/syscall.tbl     |    1 +
 arch/powerpc/kernel/syscalls/syscall.tbl    |    1 +
 arch/s390/kernel/syscalls/syscall.tbl       |    1 +
 arch/sh/kernel/syscalls/syscall.tbl         |    1 +
 arch/sparc/kernel/syscalls/syscall.tbl      |    1 +
 arch/x86/entry/syscalls/syscall_32.tbl      |    1 +
 arch/x86/entry/syscalls/syscall_64.tbl      |    1 +
 arch/xtensa/kernel/syscalls/syscall.tbl     |    1 +
 include/linux/compat.h                      |    6 ++++++
 include/linux/syscalls.h                    |    5 +++++
 include/uapi/asm-generic/unistd.h           |    4 +++-
 kernel/sys_ni.c                             |    2 ++
 22 files changed, 35 insertions(+), 2 deletions(-)

--- a/arch/alpha/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/alpha/kernel/syscalls/syscall.tbl
@@ -480,3 +480,4 @@
 548	common	pidfd_getfd			sys_pidfd_getfd
 549	common	faccessat2			sys_faccessat2
 550	common	process_madvise			sys_process_madvise
+551	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/arm64/include/asm/unistd32.h~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/arm64/include/asm/unistd32.h
@@ -889,6 +889,8 @@ __SYSCALL(__NR_pidfd_getfd, sys_pidfd_ge
 __SYSCALL(__NR_faccessat2, sys_faccessat2)
 #define __NR_process_madvise 440
 __SYSCALL(__NR_process_madvise, sys_process_madvise)
+#define __NR_epoll_pwait2 441
+__SYSCALL(__NR_epoll_pwait2, sys_epoll_pwait2)
 
 /*
  * Please add new compat syscalls above this comment and update
--- a/arch/arm64/include/asm/unistd.h~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/arm64/include/asm/unistd.h
@@ -38,7 +38,7 @@
 #define __ARM_NR_compat_set_tls		(__ARM_NR_COMPAT_BASE + 5)
 #define __ARM_NR_COMPAT_END		(__ARM_NR_COMPAT_BASE + 0x800)
 
-#define __NR_compat_syscalls		441
+#define __NR_compat_syscalls		442
 #endif
 
 #define __ARCH_WANT_SYS_CLONE
--- a/arch/arm/tools/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/arm/tools/syscall.tbl
@@ -454,3 +454,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/ia64/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/ia64/kernel/syscalls/syscall.tbl
@@ -361,3 +361,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/m68k/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/m68k/kernel/syscalls/syscall.tbl
@@ -440,3 +440,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/microblaze/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/microblaze/kernel/syscalls/syscall.tbl
@@ -446,3 +446,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/mips/kernel/syscalls/syscall_n32.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/mips/kernel/syscalls/syscall_n32.tbl
@@ -379,3 +379,4 @@
 438	n32	pidfd_getfd			sys_pidfd_getfd
 439	n32	faccessat2			sys_faccessat2
 440	n32	process_madvise			sys_process_madvise
+441	n32	epoll_pwait2			sys_epoll_pwait2
--- a/arch/mips/kernel/syscalls/syscall_n64.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/mips/kernel/syscalls/syscall_n64.tbl
@@ -355,3 +355,4 @@
 438	n64	pidfd_getfd			sys_pidfd_getfd
 439	n64	faccessat2			sys_faccessat2
 440	n64	process_madvise			sys_process_madvise
+441	n64	epoll_pwait2			sys_epoll_pwait2
--- a/arch/mips/kernel/syscalls/syscall_o32.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/mips/kernel/syscalls/syscall_o32.tbl
@@ -428,3 +428,4 @@
 438	o32	pidfd_getfd			sys_pidfd_getfd
 439	o32	faccessat2			sys_faccessat2
 440	o32	process_madvise			sys_process_madvise
+441	o32	epoll_pwait2			sys_epoll_pwait2		compat_sys_epoll_pwait2
--- a/arch/parisc/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/parisc/kernel/syscalls/syscall.tbl
@@ -438,3 +438,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2		compat_sys_epoll_pwait2
--- a/arch/powerpc/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/powerpc/kernel/syscalls/syscall.tbl
@@ -530,3 +530,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2		compat_sys_epoll_pwait2
--- a/arch/s390/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/s390/kernel/syscalls/syscall.tbl
@@ -443,3 +443,4 @@
 438  common	pidfd_getfd		sys_pidfd_getfd			sys_pidfd_getfd
 439  common	faccessat2		sys_faccessat2			sys_faccessat2
 440  common	process_madvise		sys_process_madvise		sys_process_madvise
+441  common	epoll_pwait2		sys_epoll_pwait2		sys_epoll_pwait2
--- a/arch/sh/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/sh/kernel/syscalls/syscall.tbl
@@ -443,3 +443,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/sparc/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/sparc/kernel/syscalls/syscall.tbl
@@ -486,3 +486,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/arch/x86/entry/syscalls/syscall_32.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/x86/entry/syscalls/syscall_32.tbl
@@ -445,3 +445,4 @@
 438	i386	pidfd_getfd		sys_pidfd_getfd
 439	i386	faccessat2		sys_faccessat2
 440	i386	process_madvise		sys_process_madvise
+441	i386	epoll_pwait2		sys_epoll_pwait2		compat_sys_epoll_pwait2
--- a/arch/x86/entry/syscalls/syscall_64.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/x86/entry/syscalls/syscall_64.tbl
@@ -362,6 +362,7 @@
 438	common	pidfd_getfd		sys_pidfd_getfd
 439	common	faccessat2		sys_faccessat2
 440	common	process_madvise		sys_process_madvise
+441	common	epoll_pwait2		sys_epoll_pwait2
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
--- a/arch/xtensa/kernel/syscalls/syscall.tbl~epoll-wire-up-syscall-epoll_pwait2
+++ a/arch/xtensa/kernel/syscalls/syscall.tbl
@@ -411,3 +411,4 @@
 438	common	pidfd_getfd			sys_pidfd_getfd
 439	common	faccessat2			sys_faccessat2
 440	common	process_madvise			sys_process_madvise
+441	common	epoll_pwait2			sys_epoll_pwait2
--- a/include/linux/compat.h~epoll-wire-up-syscall-epoll_pwait2
+++ a/include/linux/compat.h
@@ -537,6 +537,12 @@ asmlinkage long compat_sys_epoll_pwait(i
 			int maxevents, int timeout,
 			const compat_sigset_t __user *sigmask,
 			compat_size_t sigsetsize);
+asmlinkage long compat_sys_epoll_pwait2(int epfd,
+			struct epoll_event __user *events,
+			int maxevents,
+			const struct __kernel_timespec __user *timeout,
+			const compat_sigset_t __user *sigmask,
+			compat_size_t sigsetsize);
 
 /* fs/fcntl.c */
 asmlinkage long compat_sys_fcntl(unsigned int fd, unsigned int cmd,
--- a/include/linux/syscalls.h~epoll-wire-up-syscall-epoll_pwait2
+++ a/include/linux/syscalls.h
@@ -362,6 +362,11 @@ asmlinkage long sys_epoll_pwait(int epfd
 				int maxevents, int timeout,
 				const sigset_t __user *sigmask,
 				size_t sigsetsize);
+asmlinkage long sys_epoll_pwait2(int epfd, struct epoll_event __user *events,
+				 int maxevents,
+				 const struct __kernel_timespec __user *timeout,
+				 const sigset_t __user *sigmask,
+				 size_t sigsetsize);
 
 /* fs/fcntl.c */
 asmlinkage long sys_dup(unsigned int fildes);
--- a/include/uapi/asm-generic/unistd.h~epoll-wire-up-syscall-epoll_pwait2
+++ a/include/uapi/asm-generic/unistd.h
@@ -859,9 +859,11 @@ __SYSCALL(__NR_pidfd_getfd, sys_pidfd_ge
 __SYSCALL(__NR_faccessat2, sys_faccessat2)
 #define __NR_process_madvise 440
 __SYSCALL(__NR_process_madvise, sys_process_madvise)
+#define __NR_epoll_pwait2 441
+__SC_COMP(__NR_epoll_pwait2, sys_epoll_pwait2, compat_sys_epoll_pwait2)
 
 #undef __NR_syscalls
-#define __NR_syscalls 441
+#define __NR_syscalls 442
 
 /*
  * 32 bit systems traditionally used different
--- a/kernel/sys_ni.c~epoll-wire-up-syscall-epoll_pwait2
+++ a/kernel/sys_ni.c
@@ -68,6 +68,8 @@ COND_SYSCALL(epoll_create1);
 COND_SYSCALL(epoll_ctl);
 COND_SYSCALL(epoll_pwait);
 COND_SYSCALL_COMPAT(epoll_pwait);
+COND_SYSCALL(epoll_pwait2);
+COND_SYSCALL_COMPAT(epoll_pwait2);
 
 /* fs/fcntl.c */
 
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* [patch 78/78] selftests/filesystems: expand epoll with epoll_pwait2
  2020-12-18 22:00 incoming Andrew Morton
                   ` (76 preceding siblings ...)
  2020-12-18 22:05 ` [patch 77/78] epoll: wire up " Andrew Morton
@ 2020-12-18 22:05 ` Andrew Morton
  77 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-18 22:05 UTC (permalink / raw)
  To: akpm, arnd, linux-mm, mm-commits, torvalds, viro, willemb, willy

From: Willem de Bruijn <willemb@google.com>
Subject: selftests/filesystems: expand epoll with epoll_pwait2

Code coverage for the epoll_pwait2 syscall.

epoll62: Repeat basic test epoll1, but exercising the new syscall.
epoll63: Pass a timespec and exercise the timeout wakeup path.

Link: https://lkml.kernel.org/r/20201121144401.3727659-5-willemdebruijn.kernel@gmail.com
Signed-off-by: Willem de Bruijn <willemb@google.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 tools/testing/selftests/filesystems/epoll/epoll_wakeup_test.c |   72 ++++++++++
 1 file changed, 72 insertions(+)

--- a/tools/testing/selftests/filesystems/epoll/epoll_wakeup_test.c~selftests-filesystems-expand-epoll-with-epoll_pwait2
+++ a/tools/testing/selftests/filesystems/epoll/epoll_wakeup_test.c
@@ -1,6 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0
 
 #define _GNU_SOURCE
+#include <asm/unistd.h>
+#include <linux/time_types.h>
 #include <poll.h>
 #include <unistd.h>
 #include <assert.h>
@@ -21,6 +23,19 @@ struct epoll_mtcontext
 	pthread_t waiter;
 };
 
+#ifndef __NR_epoll_pwait2
+#define __NR_epoll_pwait2 -1
+#endif
+
+static inline int sys_epoll_pwait2(int fd, struct epoll_event *events,
+				   int maxevents,
+				   const struct __kernel_timespec *timeout,
+				   const sigset_t *sigset, size_t sigsetsize)
+{
+	return syscall(__NR_epoll_pwait2, fd, events, maxevents, timeout,
+		       sigset, sigsetsize);
+}
+
 static void signal_handler(int signum)
 {
 }
@@ -3377,4 +3392,61 @@ TEST(epoll61)
 	close(ctx.evfd);
 }
 
+/* Equivalent to basic test epoll1, but exercising epoll_pwait2. */
+TEST(epoll62)
+{
+	int efd;
+	int sfd[2];
+	struct epoll_event e;
+
+	ASSERT_EQ(socketpair(AF_UNIX, SOCK_STREAM, 0, sfd), 0);
+
+	efd = epoll_create(1);
+	ASSERT_GE(efd, 0);
+
+	e.events = EPOLLIN;
+	ASSERT_EQ(epoll_ctl(efd, EPOLL_CTL_ADD, sfd[0], &e), 0);
+
+	ASSERT_EQ(write(sfd[1], "w", 1), 1);
+
+	EXPECT_EQ(sys_epoll_pwait2(efd, &e, 1, NULL, NULL, 0), 1);
+	EXPECT_EQ(sys_epoll_pwait2(efd, &e, 1, NULL, NULL, 0), 1);
+
+	close(efd);
+	close(sfd[0]);
+	close(sfd[1]);
+}
+
+/* Epoll_pwait2 basic timeout test. */
+TEST(epoll63)
+{
+	const int cfg_delay_ms = 10;
+	unsigned long long tdiff;
+	struct __kernel_timespec ts;
+	int efd;
+	int sfd[2];
+	struct epoll_event e;
+
+	ASSERT_EQ(socketpair(AF_UNIX, SOCK_STREAM, 0, sfd), 0);
+
+	efd = epoll_create(1);
+	ASSERT_GE(efd, 0);
+
+	e.events = EPOLLIN;
+	ASSERT_EQ(epoll_ctl(efd, EPOLL_CTL_ADD, sfd[0], &e), 0);
+
+	ts.tv_sec = 0;
+	ts.tv_nsec = cfg_delay_ms * 1000 * 1000;
+
+	tdiff = msecs();
+	EXPECT_EQ(sys_epoll_pwait2(efd, &e, 1, &ts, NULL, 0), 0);
+	tdiff = msecs() - tdiff;
+
+	EXPECT_GE(tdiff, cfg_delay_ms);
+
+	close(efd);
+	close(sfd[0]);
+	close(sfd[1]);
+}
+
 TEST_HARNESS_MAIN
_

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-18 22:02 ` [patch 21/78] kasan: split out shadow.c from common.c Andrew Morton
@ 2020-12-19  0:28   ` Marco Elver
  2020-12-19  1:13     ` Andrew Morton
  2020-12-22 12:00   ` kernel test robot
  1 sibling, 1 reply; 395+ messages in thread
From: Marco Elver @ 2020-12-19  0:28 UTC (permalink / raw)
  To: Andrew Morton
  Cc: andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

On Fri, Dec 18, 2020 at 02:02PM -0800, Andrew Morton wrote:
> From: Andrey Konovalov <andreyknvl@google.com>
> Subject: kasan: split out shadow.c from common.c
> 
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
> 
> The new mode won't be using shadow memory.  Move all shadow-related code
> to shadow.c, which is only enabled for software KASAN modes that use
> shadow memory.
> 
> No functional changes for software modes.
> 
> Link: https://lkml.kernel.org/r/17d95cfa7d5cf9c4fcd9bf415f2a8dea911668df.1606161801.git.andreyknvl@google.com
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> Reviewed-by: Alexander Potapenko <glider@google.com>
> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Branislav Rankov <Branislav.Rankov@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Evgenii Stepanov <eugenis@google.com>
> Cc: Kevin Brodsky <kevin.brodsky@arm.com>
> Cc: Vasily Gorbik <gor@linux.ibm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  mm/kasan/Makefile |    6 
>  mm/kasan/common.c |  486 -----------------------------------------
>  mm/kasan/shadow.c |  518 ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 523 insertions(+), 487 deletions(-)
> 
> --- a/mm/kasan/common.c~kasan-split-out-shadowc-from-commonc
> +++ a/mm/kasan/common.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * This file contains common generic and tag-based KASAN code.
> + * This file contains common KASAN code.
>   *
>   * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>   * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kmemleak.h>
>  #include <linux/linkage.h>
>  #include <linux/memblock.h>
>  #include <linux/memory.h>
> @@ -26,12 +25,8 @@
>  #include <linux/stacktrace.h>
>  #include <linux/string.h>
>  #include <linux/types.h>
> -#include <linux/vmalloc.h>
>  #include <linux/bug.h>
>  
> -#include <asm/cacheflush.h>
> -#include <asm/tlbflush.h>
> -
>  #include "kasan.h"
>  #include "../slab.h"
>  
[...]
> -/*
> - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> - */
> -void poison_range(const void *address, size_t size, u8 value)
> -{
> -	void *shadow_start, *shadow_end;
> -
> -	/*
> -	 * Perform shadow offset calculation based on untagged address, as
> -	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
> -	 * addresses to this function.
> -	 */
> -	address = reset_tag(address);
> -

The moved lines do not mention kfence...
(The same commit in -next does.)

> -	shadow_start = kasan_mem_to_shadow(address);
> -	shadow_end = kasan_mem_to_shadow(address + size);
> -
> -	__memset(shadow_start, value, shadow_end - shadow_start);
> -}
[...]
> --- /dev/null
> +++ a/mm/kasan/shadow.c
> @@ -0,0 +1,518 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * This file contains KASAN runtime code that manages shadow memory for
> + * generic and software tag-based KASAN modes.
> + *
> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> + *
> + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> + *        Andrey Konovalov <andreyknvl@gmail.com>
> + */
> +
> +#include <linux/init.h>
> +#include <linux/kasan.h>
> +#include <linux/kernel.h>
> +#include <linux/kfence.h>

This is the first time kfence is mentioned. Is this correct?

Is my assumption correct that the kasan changes and kfence changes are
to be swapped?

Thanks,
-- Marco

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19  0:28   ` Marco Elver
@ 2020-12-19  1:13     ` Andrew Morton
  2020-12-19 10:01         ` Marco Elver
                         ` (3 more replies)
  0 siblings, 4 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-19  1:13 UTC (permalink / raw)
  To: Marco Elver
  Cc: andreyknvl, aryabinin, Branislav.Rankov, catalin.marinas,
	dvyukov, eugenis, glider, gor, kevin.brodsky, linux-mm,
	mm-commits, torvalds, vincenzo.frascino, will.deacon

On Sat, 19 Dec 2020 01:28:29 +0100 Marco Elver <elver@google.com> wrote:

> [...]
> > -/*
> > - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> > - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> > - */
> > -void poison_range(const void *address, size_t size, u8 value)
> > -{
> > -	void *shadow_start, *shadow_end;
> > -
> > -	/*
> > -	 * Perform shadow offset calculation based on untagged address, as
> > -	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
> > -	 * addresses to this function.
> > -	 */
> > -	address = reset_tag(address);
> > -
> 
> The moved lines do not mention kfence...
> (The same commit in -next does.)

They shouldn't.

> > -	shadow_start = kasan_mem_to_shadow(address);
> > -	shadow_end = kasan_mem_to_shadow(address + size);
> > -
> > -	__memset(shadow_start, value, shadow_end - shadow_start);
> > -}
> [...]
> > --- /dev/null
> > +++ a/mm/kasan/shadow.c
> > @@ -0,0 +1,518 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * This file contains KASAN runtime code that manages shadow memory for
> > + * generic and software tag-based KASAN modes.
> > + *
> > + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> > + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> > + *
> > + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> > + *        Andrey Konovalov <andreyknvl@gmail.com>
> > + */
> > +
> > +#include <linux/init.h>
> > +#include <linux/kasan.h>
> > +#include <linux/kernel.h>
> > +#include <linux/kfence.h>
> 
> This is the first time kfence is mentioned. Is this correct?

Yes.

> Is my assumption correct that the kasan changes and kfence changes are
> to be swapped?

Yes, kfence came in fairly late and seems a bit fresh.  I was planning
on holding it off until next cycle.

Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
present :(  We'll need this, yes?

--- a/mm/kasan/kasan.h~a
+++ a/mm/kasan/kasan.h
@@ -3,7 +3,6 @@
 #define __MM_KASAN_KASAN_H
 
 #include <linux/kasan.h>
-#include <linux/kfence.h>
 #include <linux/stackdepot.h>
 
 #ifdef CONFIG_KASAN_HW_TAGS
@@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
 
 static inline void poison_range(const void *address, size_t size, u8 value)
 {
-	/* Skip KFENCE memory if called explicitly outside of sl*b. */
-	if (is_kfence_address(address))
-		return;
-
 	hw_set_mem_tag_range(kasan_reset_tag(address),
 			round_up(size, KASAN_GRANULE_SIZE), value);
 }
 
 static inline void unpoison_range(const void *address, size_t size)
 {
-	/* Skip KFENCE memory if called explicitly outside of sl*b. */
-	if (is_kfence_address(address))
-		return;
-
 	hw_set_mem_tag_range(kasan_reset_tag(address),
 			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
 }
--- a/mm/kasan/shadow.c~a
+++ a/mm/kasan/shadow.c
@@ -13,7 +13,6 @@
 #include <linux/init.h>
 #include <linux/kasan.h>
 #include <linux/kernel.h>
-#include <linux/kfence.h>
 #include <linux/kmemleak.h>
 #include <linux/memory.h>
 #include <linux/mm.h>
@@ -85,10 +84,6 @@ void poison_range(const void *address, s
 	address = kasan_reset_tag(address);
 	size = round_up(size, KASAN_GRANULE_SIZE);
 
-	/* Skip KFENCE memory if called explicitly outside of sl*b. */
-	if (is_kfence_address(address))
-		return;
-
 	shadow_start = kasan_mem_to_shadow(address);
 	shadow_end = kasan_mem_to_shadow(address + size);
 
@@ -106,14 +101,6 @@ void unpoison_range(const void *address,
 	 */
 	address = kasan_reset_tag(address);
 
-	/*
-	 * Skip KFENCE memory if called explicitly outside of sl*b. Also note
-	 * that calls to ksize(), where size is not a multiple of machine-word
-	 * size, would otherwise poison the invalid portion of the word.
-	 */
-	if (is_kfence_address(address))
-		return;
-
 	poison_range(address, size, tag);
 
 	if (size & KASAN_GRANULE_MASK) {
_


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19  1:13     ` Andrew Morton
@ 2020-12-19 10:01         ` Marco Elver
  2020-12-19 10:11         ` Marco Elver
                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 395+ messages in thread
From: Marco Elver @ 2020-12-19 10:01 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Andrey Konovalov, Andrey Ryabinin, Branislav Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgenii Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky,
	Linux Memory Management List, mm-commits, Linus Torvalds,
	Vincenzo Frascino, Will Deacon

On Sat, 19 Dec 2020 at 02:13, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 19 Dec 2020 01:28:29 +0100 Marco Elver <elver@google.com> wrote:
>
> > [...]
> > > -/*
> > > - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> > > - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> > > - */
> > > -void poison_range(const void *address, size_t size, u8 value)
> > > -{
> > > -   void *shadow_start, *shadow_end;
> > > -
> > > -   /*
> > > -    * Perform shadow offset calculation based on untagged address, as
> > > -    * some of the callers (e.g. kasan_poison_object_data) pass tagged
> > > -    * addresses to this function.
> > > -    */
> > > -   address = reset_tag(address);
> > > -
> >
> > The moved lines do not mention kfence...
> > (The same commit in -next does.)
>
> They shouldn't.
>
> > > -   shadow_start = kasan_mem_to_shadow(address);
> > > -   shadow_end = kasan_mem_to_shadow(address + size);
> > > -
> > > -   __memset(shadow_start, value, shadow_end - shadow_start);
> > > -}
> > [...]
> > > --- /dev/null
> > > +++ a/mm/kasan/shadow.c
> > > @@ -0,0 +1,518 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * This file contains KASAN runtime code that manages shadow memory for
> > > + * generic and software tag-based KASAN modes.
> > > + *
> > > + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> > > + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> > > + *
> > > + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> > > + *        Andrey Konovalov <andreyknvl@gmail.com>
> > > + */
> > > +
> > > +#include <linux/init.h>
> > > +#include <linux/kasan.h>
> > > +#include <linux/kernel.h>
> > > +#include <linux/kfence.h>
> >
> > This is the first time kfence is mentioned. Is this correct?
>
> Yes.
>
> > Is my assumption correct that the kasan changes and kfence changes are
> > to be swapped?
>
> Yes, kfence came in fairly late and seems a bit fresh.  I was planning
> on holding it off until next cycle.
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?



> --- a/mm/kasan/kasan.h~a
> +++ a/mm/kasan/kasan.h
> @@ -3,7 +3,6 @@
>  #define __MM_KASAN_KASAN_H
>
>  #include <linux/kasan.h>
> -#include <linux/kfence.h>
>  #include <linux/stackdepot.h>
>
>  #ifdef CONFIG_KASAN_HW_TAGS
> @@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
>
>  static inline void poison_range(const void *address, size_t size, u8 value)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), value);
>  }
>
>  static inline void unpoison_range(const void *address, size_t size)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
>  }
> --- a/mm/kasan/shadow.c~a
> +++ a/mm/kasan/shadow.c
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kfence.h>
>  #include <linux/kmemleak.h>
>  #include <linux/memory.h>
>  #include <linux/mm.h>
> @@ -85,10 +84,6 @@ void poison_range(const void *address, s
>         address = kasan_reset_tag(address);
>         size = round_up(size, KASAN_GRANULE_SIZE);
>
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         shadow_start = kasan_mem_to_shadow(address);
>         shadow_end = kasan_mem_to_shadow(address + size);
>
> @@ -106,14 +101,6 @@ void unpoison_range(const void *address,
>          */
>         address = kasan_reset_tag(address);
>
> -       /*
> -        * Skip KFENCE memory if called explicitly outside of sl*b. Also note
> -        * that calls to ksize(), where size is not a multiple of machine-word
> -        * size, would otherwise poison the invalid portion of the word.
> -        */
> -       if (is_kfence_address(address))
> -               return;
> -
>         poison_range(address, size, tag);
>
>         if (size & KASAN_GRANULE_MASK) {
> _
>

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-19 10:01         ` Marco Elver
  0 siblings, 0 replies; 395+ messages in thread
From: Marco Elver @ 2020-12-19 10:01 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Andrey Konovalov, Andrey Ryabinin, Branislav Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgenii Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky,
	Linux Memory Management List, mm-commits, Linus Torvalds,
	Vincenzo Frascino, Will Deacon

On Sat, 19 Dec 2020 at 02:13, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 19 Dec 2020 01:28:29 +0100 Marco Elver <elver@google.com> wrote:
>
> > [...]
> > > -/*
> > > - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> > > - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> > > - */
> > > -void poison_range(const void *address, size_t size, u8 value)
> > > -{
> > > -   void *shadow_start, *shadow_end;
> > > -
> > > -   /*
> > > -    * Perform shadow offset calculation based on untagged address, as
> > > -    * some of the callers (e.g. kasan_poison_object_data) pass tagged
> > > -    * addresses to this function.
> > > -    */
> > > -   address = reset_tag(address);
> > > -
> >
> > The moved lines do not mention kfence...
> > (The same commit in -next does.)
>
> They shouldn't.
>
> > > -   shadow_start = kasan_mem_to_shadow(address);
> > > -   shadow_end = kasan_mem_to_shadow(address + size);
> > > -
> > > -   __memset(shadow_start, value, shadow_end - shadow_start);
> > > -}
> > [...]
> > > --- /dev/null
> > > +++ a/mm/kasan/shadow.c
> > > @@ -0,0 +1,518 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * This file contains KASAN runtime code that manages shadow memory for
> > > + * generic and software tag-based KASAN modes.
> > > + *
> > > + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> > > + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> > > + *
> > > + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> > > + *        Andrey Konovalov <andreyknvl@gmail.com>
> > > + */
> > > +
> > > +#include <linux/init.h>
> > > +#include <linux/kasan.h>
> > > +#include <linux/kernel.h>
> > > +#include <linux/kfence.h>
> >
> > This is the first time kfence is mentioned. Is this correct?
>
> Yes.
>
> > Is my assumption correct that the kasan changes and kfence changes are
> > to be swapped?
>
> Yes, kfence came in fairly late and seems a bit fresh.  I was planning
> on holding it off until next cycle.
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?



> --- a/mm/kasan/kasan.h~a
> +++ a/mm/kasan/kasan.h
> @@ -3,7 +3,6 @@
>  #define __MM_KASAN_KASAN_H
>
>  #include <linux/kasan.h>
> -#include <linux/kfence.h>
>  #include <linux/stackdepot.h>
>
>  #ifdef CONFIG_KASAN_HW_TAGS
> @@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
>
>  static inline void poison_range(const void *address, size_t size, u8 value)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), value);
>  }
>
>  static inline void unpoison_range(const void *address, size_t size)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
>  }
> --- a/mm/kasan/shadow.c~a
> +++ a/mm/kasan/shadow.c
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kfence.h>
>  #include <linux/kmemleak.h>
>  #include <linux/memory.h>
>  #include <linux/mm.h>
> @@ -85,10 +84,6 @@ void poison_range(const void *address, s
>         address = kasan_reset_tag(address);
>         size = round_up(size, KASAN_GRANULE_SIZE);
>
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         shadow_start = kasan_mem_to_shadow(address);
>         shadow_end = kasan_mem_to_shadow(address + size);
>
> @@ -106,14 +101,6 @@ void unpoison_range(const void *address,
>          */
>         address = kasan_reset_tag(address);
>
> -       /*
> -        * Skip KFENCE memory if called explicitly outside of sl*b. Also note
> -        * that calls to ksize(), where size is not a multiple of machine-word
> -        * size, would otherwise poison the invalid portion of the word.
> -        */
> -       if (is_kfence_address(address))
> -               return;
> -
>         poison_range(address, size, tag);
>
>         if (size & KASAN_GRANULE_MASK) {
> _
>


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19  1:13     ` Andrew Morton
@ 2020-12-19 10:11         ` Marco Elver
  2020-12-19 10:11         ` Marco Elver
                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 395+ messages in thread
From: Marco Elver @ 2020-12-19 10:11 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Andrey Konovalov, Andrey Ryabinin, Branislav Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgenii Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky,
	Linux Memory Management List, mm-commits, Linus Torvalds,
	Vincenzo Frascino, Will Deacon

[Ignore previous email without reply -- this time with actual reply]

On Sat, 19 Dec 2020 at 02:13, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 19 Dec 2020 01:28:29 +0100 Marco Elver <elver@google.com> wrote:
> > [...]
> > > -/*
> > > - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> > > - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> > > - */
> > > -void poison_range(const void *address, size_t size, u8 value)
> > > -{
> > > -   void *shadow_start, *shadow_end;
> > > -
> > > -   /*
> > > -    * Perform shadow offset calculation based on untagged address, as
> > > -    * some of the callers (e.g. kasan_poison_object_data) pass tagged
> > > -    * addresses to this function.
> > > -    */
> > > -   address = reset_tag(address);
> > > -
> >
> > The moved lines do not mention kfence...
> > (The same commit in -next does.)
>
> They shouldn't.
>
> > > -   shadow_start = kasan_mem_to_shadow(address);
> > > -   shadow_end = kasan_mem_to_shadow(address + size);
> > > -
> > > -   __memset(shadow_start, value, shadow_end - shadow_start);
> > > -}
> > [...]
> > > --- /dev/null
> > > +++ a/mm/kasan/shadow.c
> > > @@ -0,0 +1,518 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * This file contains KASAN runtime code that manages shadow memory for
> > > + * generic and software tag-based KASAN modes.
> > > + *
> > > + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> > > + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> > > + *
> > > + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> > > + *        Andrey Konovalov <andreyknvl@gmail.com>
> > > + */
> > > +
> > > +#include <linux/init.h>
> > > +#include <linux/kasan.h>
> > > +#include <linux/kernel.h>
> > > +#include <linux/kfence.h>
> >
> > This is the first time kfence is mentioned. Is this correct?
>
> Yes.
>
> > Is my assumption correct that the kasan changes and kfence changes are
> > to be swapped?
>
> Yes, kfence came in fairly late and seems a bit fresh.  I was planning
> on holding it off until next cycle.
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?

Looks reasonable; any mention of kfence should be removed from any of
the kasan patches if the kasan series goes before kfence. And kfence's
"kfence, kasan: make KFENCE compatible with KASAN" should absorb any
of those reverted changes.

Because kfence was picked up earlier, and appeared in -next before the
kasan series, the kasan series was rebased to not conflict with those
changes from kfence. Sorry for the inconvenience, and thank you for
sorting it out.

Thanks,
-- Marco

> --- a/mm/kasan/kasan.h~a
> +++ a/mm/kasan/kasan.h
> @@ -3,7 +3,6 @@
>  #define __MM_KASAN_KASAN_H
>
>  #include <linux/kasan.h>
> -#include <linux/kfence.h>
>  #include <linux/stackdepot.h>
>
>  #ifdef CONFIG_KASAN_HW_TAGS
> @@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
>
>  static inline void poison_range(const void *address, size_t size, u8 value)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), value);
>  }
>
>  static inline void unpoison_range(const void *address, size_t size)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
>  }
> --- a/mm/kasan/shadow.c~a
> +++ a/mm/kasan/shadow.c
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kfence.h>
>  #include <linux/kmemleak.h>
>  #include <linux/memory.h>
>  #include <linux/mm.h>
> @@ -85,10 +84,6 @@ void poison_range(const void *address, s
>         address = kasan_reset_tag(address);
>         size = round_up(size, KASAN_GRANULE_SIZE);
>
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         shadow_start = kasan_mem_to_shadow(address);
>         shadow_end = kasan_mem_to_shadow(address + size);
>
> @@ -106,14 +101,6 @@ void unpoison_range(const void *address,
>          */
>         address = kasan_reset_tag(address);
>
> -       /*
> -        * Skip KFENCE memory if called explicitly outside of sl*b. Also note
> -        * that calls to ksize(), where size is not a multiple of machine-word
> -        * size, would otherwise poison the invalid portion of the word.
> -        */
> -       if (is_kfence_address(address))
> -               return;
> -
>         poison_range(address, size, tag);
>
>         if (size & KASAN_GRANULE_MASK) {
> _
>

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-19 10:11         ` Marco Elver
  0 siblings, 0 replies; 395+ messages in thread
From: Marco Elver @ 2020-12-19 10:11 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Andrey Konovalov, Andrey Ryabinin, Branislav Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgenii Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky,
	Linux Memory Management List, mm-commits, Linus Torvalds,
	Vincenzo Frascino, Will Deacon

[Ignore previous email without reply -- this time with actual reply]

On Sat, 19 Dec 2020 at 02:13, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Sat, 19 Dec 2020 01:28:29 +0100 Marco Elver <elver@google.com> wrote:
> > [...]
> > > -/*
> > > - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> > > - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> > > - */
> > > -void poison_range(const void *address, size_t size, u8 value)
> > > -{
> > > -   void *shadow_start, *shadow_end;
> > > -
> > > -   /*
> > > -    * Perform shadow offset calculation based on untagged address, as
> > > -    * some of the callers (e.g. kasan_poison_object_data) pass tagged
> > > -    * addresses to this function.
> > > -    */
> > > -   address = reset_tag(address);
> > > -
> >
> > The moved lines do not mention kfence...
> > (The same commit in -next does.)
>
> They shouldn't.
>
> > > -   shadow_start = kasan_mem_to_shadow(address);
> > > -   shadow_end = kasan_mem_to_shadow(address + size);
> > > -
> > > -   __memset(shadow_start, value, shadow_end - shadow_start);
> > > -}
> > [...]
> > > --- /dev/null
> > > +++ a/mm/kasan/shadow.c
> > > @@ -0,0 +1,518 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * This file contains KASAN runtime code that manages shadow memory for
> > > + * generic and software tag-based KASAN modes.
> > > + *
> > > + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> > > + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> > > + *
> > > + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> > > + *        Andrey Konovalov <andreyknvl@gmail.com>
> > > + */
> > > +
> > > +#include <linux/init.h>
> > > +#include <linux/kasan.h>
> > > +#include <linux/kernel.h>
> > > +#include <linux/kfence.h>
> >
> > This is the first time kfence is mentioned. Is this correct?
>
> Yes.
>
> > Is my assumption correct that the kasan changes and kfence changes are
> > to be swapped?
>
> Yes, kfence came in fairly late and seems a bit fresh.  I was planning
> on holding it off until next cycle.
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?

Looks reasonable; any mention of kfence should be removed from any of
the kasan patches if the kasan series goes before kfence. And kfence's
"kfence, kasan: make KFENCE compatible with KASAN" should absorb any
of those reverted changes.

Because kfence was picked up earlier, and appeared in -next before the
kasan series, the kasan series was rebased to not conflict with those
changes from kfence. Sorry for the inconvenience, and thank you for
sorting it out.

Thanks,
-- Marco

> --- a/mm/kasan/kasan.h~a
> +++ a/mm/kasan/kasan.h
> @@ -3,7 +3,6 @@
>  #define __MM_KASAN_KASAN_H
>
>  #include <linux/kasan.h>
> -#include <linux/kfence.h>
>  #include <linux/stackdepot.h>
>
>  #ifdef CONFIG_KASAN_HW_TAGS
> @@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
>
>  static inline void poison_range(const void *address, size_t size, u8 value)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), value);
>  }
>
>  static inline void unpoison_range(const void *address, size_t size)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
>  }
> --- a/mm/kasan/shadow.c~a
> +++ a/mm/kasan/shadow.c
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kfence.h>
>  #include <linux/kmemleak.h>
>  #include <linux/memory.h>
>  #include <linux/mm.h>
> @@ -85,10 +84,6 @@ void poison_range(const void *address, s
>         address = kasan_reset_tag(address);
>         size = round_up(size, KASAN_GRANULE_SIZE);
>
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         shadow_start = kasan_mem_to_shadow(address);
>         shadow_end = kasan_mem_to_shadow(address + size);
>
> @@ -106,14 +101,6 @@ void unpoison_range(const void *address,
>          */
>         address = kasan_reset_tag(address);
>
> -       /*
> -        * Skip KFENCE memory if called explicitly outside of sl*b. Also note
> -        * that calls to ksize(), where size is not a multiple of machine-word
> -        * size, would otherwise poison the invalid portion of the word.
> -        */
> -       if (is_kfence_address(address))
> -               return;
> -
>         poison_range(address, size, tag);
>
>         if (size & KASAN_GRANULE_MASK) {
> _
>


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19  1:13     ` Andrew Morton
@ 2020-12-19 18:01         ` Andrey Konovalov
  2020-12-19 10:11         ` Marco Elver
                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 395+ messages in thread
From: Andrey Konovalov @ 2020-12-19 18:01 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Ryabinin, Branislav Rankov, Catalin Marinas,
	Dmitry Vyukov, Evgenii Stepanov, Alexander Potapenko,
	Vasily Gorbik, Kevin Brodsky, Linux Memory Management List,
	mm-commits, Linus Torvalds, Vincenzo Frascino, Will Deacon

On Sat, Dec 19, 2020 at 2:13 AM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?
>
> --- a/mm/kasan/kasan.h~a
> +++ a/mm/kasan/kasan.h
> @@ -3,7 +3,6 @@
>  #define __MM_KASAN_KASAN_H
>
>  #include <linux/kasan.h>
> -#include <linux/kfence.h>
>  #include <linux/stackdepot.h>
>
>  #ifdef CONFIG_KASAN_HW_TAGS
> @@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
>
>  static inline void poison_range(const void *address, size_t size, u8 value)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), value);
>  }
>
>  static inline void unpoison_range(const void *address, size_t size)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
>  }
> --- a/mm/kasan/shadow.c~a
> +++ a/mm/kasan/shadow.c
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kfence.h>
>  #include <linux/kmemleak.h>
>  #include <linux/memory.h>
>  #include <linux/mm.h>
> @@ -85,10 +84,6 @@ void poison_range(const void *address, s
>         address = kasan_reset_tag(address);
>         size = round_up(size, KASAN_GRANULE_SIZE);
>
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         shadow_start = kasan_mem_to_shadow(address);
>         shadow_end = kasan_mem_to_shadow(address + size);
>
> @@ -106,14 +101,6 @@ void unpoison_range(const void *address,
>          */
>         address = kasan_reset_tag(address);
>
> -       /*
> -        * Skip KFENCE memory if called explicitly outside of sl*b. Also note
> -        * that calls to ksize(), where size is not a multiple of machine-word
> -        * size, would otherwise poison the invalid portion of the word.
> -        */
> -       if (is_kfence_address(address))
> -               return;
> -
>         poison_range(address, size, tag);
>
>         if (size & KASAN_GRANULE_MASK) {
> _
>

Yes, this should be it.

Thanks!

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-19 18:01         ` Andrey Konovalov
  0 siblings, 0 replies; 395+ messages in thread
From: Andrey Konovalov @ 2020-12-19 18:01 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Ryabinin, Branislav Rankov, Catalin Marinas,
	Dmitry Vyukov, Evgenii Stepanov, Alexander Potapenko,
	Vasily Gorbik, Kevin Brodsky, Linux Memory Management List,
	mm-commits, Linus Torvalds, Vincenzo Frascino, Will Deacon

On Sat, Dec 19, 2020 at 2:13 AM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?
>
> --- a/mm/kasan/kasan.h~a
> +++ a/mm/kasan/kasan.h
> @@ -3,7 +3,6 @@
>  #define __MM_KASAN_KASAN_H
>
>  #include <linux/kasan.h>
> -#include <linux/kfence.h>
>  #include <linux/stackdepot.h>
>
>  #ifdef CONFIG_KASAN_HW_TAGS
> @@ -305,20 +304,12 @@ static inline u8 random_tag(void) { retu
>
>  static inline void poison_range(const void *address, size_t size, u8 value)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), value);
>  }
>
>  static inline void unpoison_range(const void *address, size_t size)
>  {
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         hw_set_mem_tag_range(kasan_reset_tag(address),
>                         round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
>  }
> --- a/mm/kasan/shadow.c~a
> +++ a/mm/kasan/shadow.c
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kfence.h>
>  #include <linux/kmemleak.h>
>  #include <linux/memory.h>
>  #include <linux/mm.h>
> @@ -85,10 +84,6 @@ void poison_range(const void *address, s
>         address = kasan_reset_tag(address);
>         size = round_up(size, KASAN_GRANULE_SIZE);
>
> -       /* Skip KFENCE memory if called explicitly outside of sl*b. */
> -       if (is_kfence_address(address))
> -               return;
> -
>         shadow_start = kasan_mem_to_shadow(address);
>         shadow_end = kasan_mem_to_shadow(address + size);
>
> @@ -106,14 +101,6 @@ void unpoison_range(const void *address,
>          */
>         address = kasan_reset_tag(address);
>
> -       /*
> -        * Skip KFENCE memory if called explicitly outside of sl*b. Also note
> -        * that calls to ksize(), where size is not a multiple of machine-word
> -        * size, would otherwise poison the invalid portion of the word.
> -        */
> -       if (is_kfence_address(address))
> -               return;
> -
>         poison_range(address, size, tag);
>
>         if (size & KASAN_GRANULE_MASK) {
> _
>

Yes, this should be it.

Thanks!


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19  1:13     ` Andrew Morton
@ 2020-12-19 19:17         ` Linus Torvalds
  2020-12-19 10:11         ` Marco Elver
                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2020-12-19 19:17 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Konovalov, Andrey Ryabinin, Branislav.Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgeniy Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky, Linux-MM,
	mm-commits, Vincenzo Frascino, Will Deacon

On Fri, Dec 18, 2020 at 5:13 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?

Four of the patches in this patch-bomb mention the not-yet-existing
kfence.h header. This plus two others have it in the actual patch. The
fourth has it only as context.

Two others mention kfence in some form or another.

I think I'll drop all the kasan-related patches (and that's most of
it), and basically only keep the memcg and epoll ones from this
series.

(That cuts the series of 78 down to 17 - and I haven't actually
verified that even that smaller subset is entirely independent of the
dropped patches yet).

            Linus

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-19 19:17         ` Linus Torvalds
  0 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2020-12-19 19:17 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Konovalov, Andrey Ryabinin, Branislav.Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgeniy Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky, Linux-MM,
	mm-commits, Vincenzo Frascino, Will Deacon

On Fri, Dec 18, 2020 at 5:13 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> present :(  We'll need this, yes?

Four of the patches in this patch-bomb mention the not-yet-existing
kfence.h header. This plus two others have it in the actual patch. The
fourth has it only as context.

Two others mention kfence in some form or another.

I think I'll drop all the kasan-related patches (and that's most of
it), and basically only keep the memcg and epoll ones from this
series.

(That cuts the series of 78 down to 17 - and I haven't actually
verified that even that smaller subset is entirely independent of the
dropped patches yet).

            Linus


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19 19:17         ` Linus Torvalds
@ 2020-12-19 19:26           ` Linus Torvalds
  -1 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2020-12-19 19:26 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Konovalov, Andrey Ryabinin, Branislav.Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgeniy Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky, Linux-MM,
	mm-commits, Vincenzo Frascino, Will Deacon

On Sat, Dec 19, 2020 at 11:17 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> (That cuts the series of 78 down to 17

18, actually.

>                       - and I haven't actually
> verified that even that smaller subset is entirely independent of the
> dropped patches yet).

Looks like that small remainder is fine.

           Linus

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-19 19:26           ` Linus Torvalds
  0 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2020-12-19 19:26 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Konovalov, Andrey Ryabinin, Branislav.Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgeniy Stepanov,
	Alexander Potapenko, Vasily Gorbik, Kevin Brodsky, Linux-MM,
	mm-commits, Vincenzo Frascino, Will Deacon

On Sat, Dec 19, 2020 at 11:17 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> (That cuts the series of 78 down to 17

18, actually.

>                       - and I haven't actually
> verified that even that smaller subset is entirely independent of the
> dropped patches yet).

Looks like that small remainder is fine.

           Linus


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-19 19:17         ` Linus Torvalds
@ 2020-12-21  9:46           ` Alexander Potapenko
  -1 siblings, 0 replies; 395+ messages in thread
From: Alexander Potapenko @ 2020-12-21  9:46 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Konovalov, Andrey Ryabinin, Branislav Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgeniy Stepanov, Vasily Gorbik,
	Kevin Brodsky, Linux-MM, mm-commits, Vincenzo Frascino,
	Will Deacon, Linus Torvalds

On Sat, Dec 19, 2020 at 8:17 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Fri, Dec 18, 2020 at 5:13 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> > present :(  We'll need this, yes?
>
> Four of the patches in this patch-bomb mention the not-yet-existing
> kfence.h header. This plus two others have it in the actual patch. The
> fourth has it only as context.
>
> Two others mention kfence in some form or another.
>
> I think I'll drop all the kasan-related patches (and that's most of
> it), and basically only keep the memcg and epoll ones from this
> series.

@Andrew, were you planning to resend the KASAN series for 5.11? Do you
need help with that?

It would be a pity to punt that big chunk of MTE-related work the
folks did just because of the leftover KFENCE bits, we didn't mean to.

-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-21  9:46           ` Alexander Potapenko
  0 siblings, 0 replies; 395+ messages in thread
From: Alexander Potapenko @ 2020-12-21  9:46 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Marco Elver, Andrey Konovalov, Andrey Ryabinin, Branislav Rankov,
	Catalin Marinas, Dmitry Vyukov, Evgeniy Stepanov, Vasily Gorbik,
	Kevin Brodsky, Linux-MM, mm-commits, Vincenzo Frascino,
	Will Deacon, Linus Torvalds

On Sat, Dec 19, 2020 at 8:17 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Fri, Dec 18, 2020 at 5:13 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > Sigh.  I don't have access to my capable-of-compiling-KASAN machine at
> > present :(  We'll need this, yes?
>
> Four of the patches in this patch-bomb mention the not-yet-existing
> kfence.h header. This plus two others have it in the actual patch. The
> fourth has it only as context.
>
> Two others mention kfence in some form or another.
>
> I think I'll drop all the kasan-related patches (and that's most of
> it), and basically only keep the memcg and epoll ones from this
> series.

@Andrew, were you planning to resend the KASAN series for 5.11? Do you
need help with that?

It would be a pity to punt that big chunk of MTE-related work the
folks did just because of the leftover KFENCE bits, we didn't mean to.

-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-21  9:46           ` Alexander Potapenko
@ 2020-12-21 17:41             ` Linus Torvalds
  -1 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2020-12-21 17:41 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Andrew Morton, Marco Elver, Andrey Konovalov, Andrey Ryabinin,
	Branislav Rankov, Catalin Marinas, Dmitry Vyukov,
	Evgeniy Stepanov, Vasily Gorbik, Kevin Brodsky, Linux-MM,
	mm-commits, Vincenzo Frascino, Will Deacon

On Mon, Dec 21, 2020 at 1:46 AM Alexander Potapenko <glider@google.com> wrote:
>
> It would be a pity to punt that big chunk of MTE-related work the
> folks did just because of the leftover KFENCE bits, we didn't mean to.

Note that it wasn't just the kfence bits - that was only a symptom of
the problem.

The real problem is that Andrew apparently doesn't have a
KASAN-capable setup right now, so he isn't building what he sends me.

linux-next ends up building the end result, of course, but that's the
full series, and the smaller sequences don't get any build testing.

              Linus

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
@ 2020-12-21 17:41             ` Linus Torvalds
  0 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2020-12-21 17:41 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Andrew Morton, Marco Elver, Andrey Konovalov, Andrey Ryabinin,
	Branislav Rankov, Catalin Marinas, Dmitry Vyukov,
	Evgeniy Stepanov, Vasily Gorbik, Kevin Brodsky, Linux-MM,
	mm-commits, Vincenzo Frascino, Will Deacon

On Mon, Dec 21, 2020 at 1:46 AM Alexander Potapenko <glider@google.com> wrote:
>
> It would be a pity to punt that big chunk of MTE-related work the
> folks did just because of the leftover KFENCE bits, we didn't mean to.

Note that it wasn't just the kfence bits - that was only a symptom of
the problem.

The real problem is that Andrew apparently doesn't have a
KASAN-capable setup right now, so he isn't building what he sends me.

linux-next ends up building the end result, of course, but that's the
full series, and the smaller sequences don't get any build testing.

              Linus


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-18 22:02 ` [patch 21/78] kasan: split out shadow.c from common.c Andrew Morton
  2020-12-19  0:28   ` Marco Elver
@ 2020-12-22 12:00   ` kernel test robot
  1 sibling, 0 replies; 395+ messages in thread
From: kernel test robot @ 2020-12-22 12:00 UTC (permalink / raw)
  To: Andrew Morton; +Cc: kbuild-all, clang-built-linux, Linux Memory Management List

[-- Attachment #1: Type: text/plain, Size: 2289 bytes --]

Hi Andrew,

I love your patch! Yet something to improve:

[auto build test ERROR on linus/master]
[cannot apply to mmotm/master arm64/for-next/core kvm/linux-next hnaz-linux-mm/master v5.10 next-20201222]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Andrew-Morton/mm-memcg-bail-early-from-swap-accounting-if-memcg-disabled/20201219-070247
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 3644e2d2dda78e21edd8f5415b6d7ab03f5f54f3
config: powerpc64-randconfig-r023-20201217 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project cee1e7d14f4628d6174b33640d502bff3b54ae45)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install powerpc64 cross compiling tool for clang build
        # apt-get install binutils-powerpc64-linux-gnu
        # https://github.com/0day-ci/linux/commit/a9dd3c7e7e03d1caf5a0235f2bbba3b8dd2e6579
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Andrew-Morton/mm-memcg-bail-early-from-swap-accounting-if-memcg-disabled/20201219-070247
        git checkout a9dd3c7e7e03d1caf5a0235f2bbba3b8dd2e6579
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=powerpc64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

>> mm/kasan/shadow.c:16:10: fatal error: 'linux/kfence.h' file not found
   #include <linux/kfence.h>
            ^~~~~~~~~~~~~~~~
   1 error generated.


vim +16 mm/kasan/shadow.c

  > 16	#include <linux/kfence.h>
    17	#include <linux/kmemleak.h>
    18	#include <linux/memory.h>
    19	#include <linux/mm.h>
    20	#include <linux/string.h>
    21	#include <linux/types.h>
    22	#include <linux/vmalloc.h>
    23	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31525 bytes --]

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 64/78] kasan: inline (un)poison_range and check_invalid_free
  2020-12-18 22:04 ` [patch 64/78] kasan: inline (un)poison_range and check_invalid_free Andrew Morton
@ 2020-12-22 14:02   ` kernel test robot
  0 siblings, 0 replies; 395+ messages in thread
From: kernel test robot @ 2020-12-22 14:02 UTC (permalink / raw)
  To: Andrew Morton; +Cc: kbuild-all, clang-built-linux, Linux Memory Management List

[-- Attachment #1: Type: text/plain, Size: 2213 bytes --]

Hi Andrew,

I love your patch! Yet something to improve:

[auto build test ERROR on linus/master]
[cannot apply to mmotm/master arm64/for-next/core kvm/linux-next hnaz-linux-mm/master v5.10 next-20201222]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Andrew-Morton/mm-memcg-bail-early-from-swap-accounting-if-memcg-disabled/20201219-070247
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 3644e2d2dda78e21edd8f5415b6d7ab03f5f54f3
config: powerpc64-randconfig-r023-20201217 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project cee1e7d14f4628d6174b33640d502bff3b54ae45)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install powerpc64 cross compiling tool for clang build
        # apt-get install binutils-powerpc64-linux-gnu
        # https://github.com/0day-ci/linux/commit/f7e9cd914d0ea9f9621ac544fdcb0672aa4a8fc9
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Andrew-Morton/mm-memcg-bail-early-from-swap-accounting-if-memcg-disabled/20201219-070247
        git checkout f7e9cd914d0ea9f9621ac544fdcb0672aa4a8fc9
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=powerpc64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from mm/kasan/common.c:30:
>> mm/kasan/kasan.h:6:10: fatal error: 'linux/kfence.h' file not found
   #include <linux/kfence.h>
            ^~~~~~~~~~~~~~~~
   1 error generated.


vim +6 mm/kasan/kasan.h

     4	
     5	#include <linux/kasan.h>
   > 6	#include <linux/kfence.h>
     7	#include <linux/stackdepot.h>
     8	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31525 bytes --]

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: [patch 21/78] kasan: split out shadow.c from common.c
  2020-12-21 17:41             ` Linus Torvalds
  (?)
@ 2020-12-22 18:38             ` Andrew Morton
  -1 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2020-12-22 18:38 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexander Potapenko, Marco Elver, Andrey Konovalov,
	Andrey Ryabinin, Branislav Rankov, Catalin Marinas,
	Dmitry Vyukov, Evgeniy Stepanov, Vasily Gorbik, Kevin Brodsky,
	Linux-MM, mm-commits, Vincenzo Frascino, Will Deacon

On Mon, 21 Dec 2020 09:41:37 -0800 Linus Torvalds <torvalds@linux-foundation.org> wrote:

> On Mon, Dec 21, 2020 at 1:46 AM Alexander Potapenko <glider@google.com> wrote:
> >
> > It would be a pity to punt that big chunk of MTE-related work the
> > folks did just because of the leftover KFENCE bits, we didn't mean to.
> 
> Note that it wasn't just the kfence bits - that was only a symptom of
> the problem.
> 
> The real problem is that Andrew apparently doesn't have a
> KASAN-capable setup right now, so he isn't building what he sends me.
>
> linux-next ends up building the end result, of course, but that's the
> full series, and the smaller sequences don't get any build testing.
> 

All sorted out now - I'll send the kasan work along later today.

^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-11-20  0:42 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-11-20  0:42 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

15 patches, based on a90af8f15bdc9449ee2d24e1d73fa3f7e8633f81.

Subsystems affected by this patch series:

  mm/swap
  ipc
  mm/slab-generic
  hexagon
  mm/kmemleak
  mm/hugetlb
  mm/kasan
  mm/damon
  mm/highmem
  proc

Subsystem: mm/swap

    Matthew Wilcox <willy@infradead.org>:
      mm/swap.c:put_pages_list(): reinitialise the page list

Subsystem: ipc

    Alexander Mikhalitsyn <alexander.mikhalitsyn@virtuozzo.com>:
    Patch series "shm: shm_rmid_forced feature fixes":
      ipc: WARN if trying to remove ipc object which is absent
      shm: extend forced shm destroy to support objects from several IPC nses

Subsystem: mm/slab-generic

    Yunfeng Ye <yeyunfeng@huawei.com>:
      mm: emit the "free" trace report before freeing memory in kmem_cache_free()

Subsystem: hexagon

    Nathan Chancellor <nathan@kernel.org>:
    Patch series "Fixes for ARCH=hexagon allmodconfig", v2:
      hexagon: export raw I/O routines for modules
      hexagon: clean up timer-regs.h
      hexagon: ignore vmlinux.lds

Subsystem: mm/kmemleak

    Rustam Kovhaev <rkovhaev@gmail.com>:
      mm: kmemleak: slob: respect SLAB_NOLEAKTRACE flag

Subsystem: mm/hugetlb

    Bui Quang Minh <minhquangbui99@gmail.com>:
      hugetlb: fix hugetlb cgroup refcounting during mremap

    Mina Almasry <almasrymina@google.com>:
      hugetlb, userfaultfd: fix reservation restore on userfaultfd error

Subsystem: mm/kasan

    Kees Cook <keescook@chromium.org>:
      kasan: test: silence intentional read overflow warnings

Subsystem: mm/damon

    SeongJae Park <sj@kernel.org>:
    Patch series "DAMON fixes":
      mm/damon/dbgfs: use '__GFP_NOWARN' for user-specified size buffer allocation
      mm/damon/dbgfs: fix missed use of damon_dbgfs_lock

Subsystem: mm/highmem

    Ard Biesheuvel <ardb@kernel.org>:
      kmap_local: don't assume kmap PTEs are linear arrays in memory

Subsystem: proc

    David Hildenbrand <david@redhat.com>:
      proc/vmcore: fix clearing user buffer by properly using clear_user()

 arch/arm/Kconfig                      |    1 
 arch/hexagon/include/asm/timer-regs.h |   26 ----
 arch/hexagon/include/asm/timex.h      |    3 
 arch/hexagon/kernel/.gitignore        |    1 
 arch/hexagon/kernel/time.c            |   12 +-
 arch/hexagon/lib/io.c                 |    4 
 fs/proc/vmcore.c                      |   20 ++-
 include/linux/hugetlb_cgroup.h        |   12 ++
 include/linux/ipc_namespace.h         |   15 ++
 include/linux/sched/task.h            |    2 
 ipc/shm.c                             |  189 +++++++++++++++++++++++++---------
 ipc/util.c                            |    6 -
 lib/test_kasan.c                      |    2 
 mm/Kconfig                            |    3 
 mm/damon/dbgfs.c                      |   20 ++-
 mm/highmem.c                          |   32 +++--
 mm/hugetlb.c                          |   11 +
 mm/slab.c                             |    3 
 mm/slab.h                             |    2 
 mm/slob.c                             |    3 
 mm/slub.c                             |    2 
 mm/swap.c                             |    1 
 22 files changed, 254 insertions(+), 116 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-11-11  4:32 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-11-11  4:32 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

The post-linux-next material.

7 patches, based on debe436e77c72fcee804fb867f275e6d31aa999c.

Subsystems affected by this patch series:

  mm/debug
  mm/slab-generic
  mm/migration
  mm/memcg
  mm/kasan

Subsystem: mm/debug

    Yixuan Cao <caoyixuan2019@email.szu.edu.cn>:
      mm/page_owner.c: modify the type of argument "order" in some functions

Subsystem: mm/slab-generic

    Ingo Molnar <mingo@kernel.org>:
      mm: allow only SLUB on PREEMPT_RT

Subsystem: mm/migration

    Baolin Wang <baolin.wang@linux.alibaba.com>:
      mm: migrate: simplify the file-backed pages validation when migrating its mapping

    Alistair Popple <apopple@nvidia.com>:
      mm/migrate.c: remove MIGRATE_PFN_LOCKED

Subsystem: mm/memcg

    Christoph Hellwig <hch@lst.de>:
    Patch series "unexport memcg locking helpers":
      mm: unexport folio_memcg_{,un}lock
      mm: unexport {,un}lock_page_memcg

Subsystem: mm/kasan

    Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>:
      kasan: add kasan mode messages when kasan init

 Documentation/vm/hmm.rst                 |    2 
 arch/arm64/mm/kasan_init.c               |    2 
 arch/powerpc/kvm/book3s_hv_uvmem.c       |    4 
 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c |    2 
 drivers/gpu/drm/nouveau/nouveau_dmem.c   |    4 
 include/linux/migrate.h                  |    1 
 include/linux/page_owner.h               |   12 +-
 init/Kconfig                             |    2 
 lib/test_hmm.c                           |    5 -
 mm/kasan/hw_tags.c                       |   14 ++
 mm/kasan/sw_tags.c                       |    2 
 mm/memcontrol.c                          |    4 
 mm/migrate.c                             |  151 +++++--------------------------
 mm/page_owner.c                          |    6 -
 14 files changed, 61 insertions(+), 150 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-11-09  2:30 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-11-09  2:30 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

87 patches, based on 8bb7eca972ad531c9b149c0a51ab43a417385813, plus
previously sent material.

Subsystems affected by this patch series:

  mm/pagecache
  mm/hugetlb
  procfs
  misc
  MAINTAINERS
  lib
  checkpatch
  binfmt
  kallsyms
  ramfs
  init
  codafs
  nilfs2
  hfs
  crash_dump
  signals
  seq_file
  fork
  sysvfs
  kcov
  gdb
  resource
  selftests
  ipc

Subsystem: mm/pagecache

    Johannes Weiner <hannes@cmpxchg.org>:
      vfs: keep inodes with page cache off the inode shrinker LRU

Subsystem: mm/hugetlb

    zhangyiru <zhangyiru3@huawei.com>:
      mm,hugetlb: remove mlock ulimit for SHM_HUGETLB

Subsystem: procfs

    Florian Weimer <fweimer@redhat.com>:
      procfs: do not list TID 0 in /proc/<pid>/task

    David Hildenbrand <david@redhat.com>:
      x86/xen: update xen_oldmem_pfn_is_ram() documentation
      x86/xen: simplify xen_oldmem_pfn_is_ram()
      x86/xen: print a warning when HVMOP_get_mem_type fails
      proc/vmcore: let pfn_is_ram() return a bool
      proc/vmcore: convert oldmem_pfn_is_ram callback to more generic vmcore callbacks
      virtio-mem: factor out hotplug specifics from virtio_mem_init() into virtio_mem_init_hotplug()
      virtio-mem: factor out hotplug specifics from virtio_mem_probe() into virtio_mem_init_hotplug()
      virtio-mem: factor out hotplug specifics from virtio_mem_remove() into virtio_mem_deinit_hotplug()
      virtio-mem: kdump mode to sanitize /proc/vmcore access

    Stephen Brennan <stephen.s.brennan@oracle.com>:
      proc: allow pid_revalidate() during LOOKUP_RCU

Subsystem: misc

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
    Patch series "kernel.h further split", v5:
      kernel.h: drop unneeded <linux/kernel.h> inclusion from other headers
      kernel.h: split out container_of() and typeof_member() macros
      include/kunit/test.h: replace kernel.h with the necessary inclusions
      include/linux/list.h: replace kernel.h with the necessary inclusions
      include/linux/llist.h: replace kernel.h with the necessary inclusions
      include/linux/plist.h: replace kernel.h with the necessary inclusions
      include/media/media-entity.h: replace kernel.h with the necessary inclusions
      include/linux/delay.h: replace kernel.h with the necessary inclusions
      include/linux/sbitmap.h: replace kernel.h with the necessary inclusions
      include/linux/radix-tree.h: replace kernel.h with the necessary inclusions
      include/linux/generic-radix-tree.h: replace kernel.h with the necessary inclusions

    Stephen Rothwell <sfr@canb.auug.org.au>:
      kernel.h: split out instruction pointer accessors

    Rasmus Villemoes <linux@rasmusvillemoes.dk>:
      linux/container_of.h: switch to static_assert

    Colin Ian King <colin.i.king@googlemail.com>:
      mailmap: update email address for Colin King

Subsystem: MAINTAINERS

    Kees Cook <keescook@chromium.org>:
      MAINTAINERS: add "exec & binfmt" section with myself and Eric

    Lukas Bulwahn <lukas.bulwahn@gmail.com>:
    Patch series "Rectify file references for dt-bindings in MAINTAINERS", v5:
      MAINTAINERS: rectify entry for ARM/TOSHIBA VISCONTI ARCHITECTURE
      MAINTAINERS: rectify entry for HIKEY960 ONBOARD USB GPIO HUB DRIVER
      MAINTAINERS: rectify entry for INTEL KEEM BAY DRM DRIVER
      MAINTAINERS: rectify entry for ALLWINNER HARDWARE SPINLOCK SUPPORT

Subsystem: lib

    Imran Khan <imran.f.khan@oracle.com>:
    Patch series "lib, stackdepot: check stackdepot handle before accessing slabs", v2:
      lib, stackdepot: check stackdepot handle before accessing slabs
      lib, stackdepot: add helper to print stack entries
      lib, stackdepot: add helper to print stack entries into buffer

    Lucas De Marchi <lucas.demarchi@intel.com>:
      include/linux/string_helpers.h: add linux/string.h for strlen()

    Alexey Dobriyan <adobriyan@gmail.com>:
      lib: uninline simple_strntoull() as well

    Thomas Gleixner <tglx@linutronix.de>:
      mm/scatterlist: replace the !preemptible warning in sg_miter_stop()

Subsystem: checkpatch

    Rikard Falkeborn <rikard.falkeborn@gmail.com>:
      const_structs.checkpatch: add a few sound ops structs

    Joe Perches <joe@perches.com>:
      checkpatch: improve EXPORT_SYMBOL test for EXPORT_SYMBOL_NS uses

    Peter Ujfalusi <peter.ujfalusi@linux.intel.com>:
      checkpatch: get default codespell dictionary path from package location

Subsystem: binfmt

    Kees Cook <keescook@chromium.org>:
      binfmt_elf: reintroduce using MAP_FIXED_NOREPLACE

    Alexey Dobriyan <adobriyan@gmail.com>:
      ELF: simplify STACK_ALLOC macro

Subsystem: kallsyms

    Kefeng Wang <wangkefeng.wang@huawei.com>:
    Patch series "sections: Unify kernel sections range check and use", v4:
      kallsyms: remove arch specific text and data check
      kallsyms: fix address-checks for kernel related range
      sections: move and rename core_kernel_data() to is_kernel_core_data()
      sections: move is_kernel_inittext() into sections.h
      x86: mm: rename __is_kernel_text() to is_x86_32_kernel_text()
      sections: provide internal __is_kernel() and __is_kernel_text() helper
      mm: kasan: use is_kernel() helper
      extable: use is_kernel_text() helper
      powerpc/mm: use core_kernel_text() helper
      microblaze: use is_kernel_text() helper
      alpha: use is_kernel_text() helper

Subsystem: ramfs

    yangerkun <yangerkun@huawei.com>:
      ramfs: fix mount source show for ramfs

Subsystem: init

    Andrew Halaney <ahalaney@redhat.com>:
      init: make unknown command line param message clearer

Subsystem: codafs

    Jan Harkes <jaharkes@cs.cmu.edu>:
    Patch series "Coda updates for -next":
      coda: avoid NULL pointer dereference from a bad inode
      coda: check for async upcall request using local state

    Alex Shi <alex.shi@linux.alibaba.com>:
      coda: remove err which no one care

    Jan Harkes <jaharkes@cs.cmu.edu>:
      coda: avoid flagging NULL inodes
      coda: avoid hidden code duplication in rename
      coda: avoid doing bad things on inode type changes during revalidation

    Xiyu Yang <xiyuyang19@fudan.edu.cn>:
      coda: convert from atomic_t to refcount_t on coda_vm_ops->refcnt

    Jing Yangyang <jing.yangyang@zte.com.cn>:
      coda: use vmemdup_user to replace the open code

    Jan Harkes <jaharkes@cs.cmu.edu>:
      coda: bump module version to 7.2

Subsystem: nilfs2

    Qing Wang <wangqing@vivo.com>:
    Patch series "nilfs2 updates":
      nilfs2: replace snprintf in show functions with sysfs_emit

    Ryusuke Konishi <konishi.ryusuke@gmail.com>:
      nilfs2: remove filenames from file comments

Subsystem: hfs

    Arnd Bergmann <arnd@arndb.de>:
      hfs/hfsplus: use WARN_ON for sanity check

Subsystem: crash_dump

    Changcheng Deng <deng.changcheng@zte.com.cn>:
      crash_dump: fix boolreturn.cocci warning

    Ye Guojin <ye.guojin@zte.com.cn>:
      crash_dump: remove duplicate include in crash_dump.h

Subsystem: signals

    Ye Guojin <ye.guojin@zte.com.cn>:
      signal: remove duplicate include in signal.h

Subsystem: seq_file

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
      seq_file: move seq_escape() to a header

    Muchun Song <songmuchun@bytedance.com>:
      seq_file: fix passing wrong private data

Subsystem: fork

    Ran Xiaokai <ran.xiaokai@zte.com.cn>:
      kernel/fork.c: unshare(): use swap() to make code cleaner

Subsystem: sysvfs

    Pavel Skripkin <paskripkin@gmail.com>:
      sysv: use BUILD_BUG_ON instead of runtime check

Subsystem: kcov

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
    Patch series "kcov: PREEMPT_RT fixup + misc", v2:
      Documentation/kcov: include types.h in the example
      Documentation/kcov: define `ip' in the example
      kcov: allocate per-CPU memory on the relevant node
      kcov: avoid enable+disable interrupts if !in_task()
      kcov: replace local_irq_save() with a local_lock_t

Subsystem: gdb

    Douglas Anderson <dianders@chromium.org>:
      scripts/gdb: handle split debug for vmlinux

Subsystem: resource

    David Hildenbrand <david@redhat.com>:
    Patch series "virtio-mem: disallow mapping virtio-mem memory via /dev/mem", v5:
      kernel/resource: clean up and optimize iomem_is_exclusive()
      kernel/resource: disallow access to exclusive system RAM regions
      virtio-mem: disallow mapping virtio-mem memory via /dev/mem

Subsystem: selftests

    SeongJae Park <sjpark@amazon.de>:
      selftests/kselftest/runner/run_one(): allow running non-executable files

Subsystem: ipc

    Michal Clapinski <mclapinski@google.com>:
      ipc: check checkpoint_restore_ns_capable() to modify C/R proc files

    Manfred Spraul <manfred@colorfullife.com>:
      ipc/ipc_sysctl.c: remove fallback for !CONFIG_PROC_SYSCTL

 .mailmap                                             |    2 
 Documentation/dev-tools/kcov.rst                     |    5 
 MAINTAINERS                                          |   21 +
 arch/alpha/kernel/traps.c                            |    4 
 arch/microblaze/mm/pgtable.c                         |    3 
 arch/powerpc/mm/pgtable_32.c                         |    7 
 arch/riscv/lib/delay.c                               |    4 
 arch/s390/include/asm/facility.h                     |    4 
 arch/x86/kernel/aperture_64.c                        |   13 
 arch/x86/kernel/unwind_orc.c                         |    2 
 arch/x86/mm/init_32.c                                |   14 
 arch/x86/xen/mmu_hvm.c                               |   39 --
 drivers/gpu/drm/drm_dp_mst_topology.c                |    5 
 drivers/gpu/drm/drm_mm.c                             |    5 
 drivers/gpu/drm/i915/i915_vma.c                      |    5 
 drivers/gpu/drm/i915/intel_runtime_pm.c              |   20 -
 drivers/media/dvb-frontends/cxd2880/cxd2880_common.h |    1 
 drivers/virtio/Kconfig                               |    1 
 drivers/virtio/virtio_mem.c                          |  321 +++++++++++++------
 fs/binfmt_elf.c                                      |   33 +
 fs/coda/cnode.c                                      |   13 
 fs/coda/coda_linux.c                                 |   39 +-
 fs/coda/coda_linux.h                                 |    6 
 fs/coda/dir.c                                        |   20 -
 fs/coda/file.c                                       |   12 
 fs/coda/psdev.c                                      |   14 
 fs/coda/upcall.c                                     |    3 
 fs/hfs/inode.c                                       |    6 
 fs/hfsplus/inode.c                                   |   12 
 fs/hugetlbfs/inode.c                                 |   23 -
 fs/inode.c                                           |   46 +-
 fs/internal.h                                        |    1 
 fs/nilfs2/alloc.c                                    |    2 
 fs/nilfs2/alloc.h                                    |    2 
 fs/nilfs2/bmap.c                                     |    2 
 fs/nilfs2/bmap.h                                     |    2 
 fs/nilfs2/btnode.c                                   |    2 
 fs/nilfs2/btnode.h                                   |    2 
 fs/nilfs2/btree.c                                    |    2 
 fs/nilfs2/btree.h                                    |    2 
 fs/nilfs2/cpfile.c                                   |    2 
 fs/nilfs2/cpfile.h                                   |    2 
 fs/nilfs2/dat.c                                      |    2 
 fs/nilfs2/dat.h                                      |    2 
 fs/nilfs2/dir.c                                      |    2 
 fs/nilfs2/direct.c                                   |    2 
 fs/nilfs2/direct.h                                   |    2 
 fs/nilfs2/file.c                                     |    2 
 fs/nilfs2/gcinode.c                                  |    2 
 fs/nilfs2/ifile.c                                    |    2 
 fs/nilfs2/ifile.h                                    |    2 
 fs/nilfs2/inode.c                                    |    2 
 fs/nilfs2/ioctl.c                                    |    2 
 fs/nilfs2/mdt.c                                      |    2 
 fs/nilfs2/mdt.h                                      |    2 
 fs/nilfs2/namei.c                                    |    2 
 fs/nilfs2/nilfs.h                                    |    2 
 fs/nilfs2/page.c                                     |    2 
 fs/nilfs2/page.h                                     |    2 
 fs/nilfs2/recovery.c                                 |    2 
 fs/nilfs2/segbuf.c                                   |    2 
 fs/nilfs2/segbuf.h                                   |    2 
 fs/nilfs2/segment.c                                  |    2 
 fs/nilfs2/segment.h                                  |    2 
 fs/nilfs2/sufile.c                                   |    2 
 fs/nilfs2/sufile.h                                   |    2 
 fs/nilfs2/super.c                                    |    2 
 fs/nilfs2/sysfs.c                                    |   78 ++--
 fs/nilfs2/sysfs.h                                    |    2 
 fs/nilfs2/the_nilfs.c                                |    2 
 fs/nilfs2/the_nilfs.h                                |    2 
 fs/proc/base.c                                       |   21 -
 fs/proc/vmcore.c                                     |  109 ++++--
 fs/ramfs/inode.c                                     |   11 
 fs/seq_file.c                                        |   16 
 fs/sysv/super.c                                      |    6 
 include/asm-generic/sections.h                       |   75 +++-
 include/kunit/test.h                                 |   13 
 include/linux/bottom_half.h                          |    3 
 include/linux/container_of.h                         |   52 ++-
 include/linux/crash_dump.h                           |   30 +
 include/linux/delay.h                                |    2 
 include/linux/fs.h                                   |    1 
 include/linux/fwnode.h                               |    1 
 include/linux/generic-radix-tree.h                   |    3 
 include/linux/hugetlb.h                              |    6 
 include/linux/instruction_pointer.h                  |    8 
 include/linux/kallsyms.h                             |   21 -
 include/linux/kernel.h                               |   39 --
 include/linux/list.h                                 |    4 
 include/linux/llist.h                                |    4 
 include/linux/pagemap.h                              |   50 ++
 include/linux/plist.h                                |    5 
 include/linux/radix-tree.h                           |    4 
 include/linux/rwsem.h                                |    1 
 include/linux/sbitmap.h                              |   11 
 include/linux/seq_file.h                             |   19 +
 include/linux/signal.h                               |    1 
 include/linux/smp.h                                  |    1 
 include/linux/spinlock.h                             |    1 
 include/linux/stackdepot.h                           |    5 
 include/linux/string_helpers.h                       |    1 
 include/media/media-entity.h                         |    3 
 init/main.c                                          |    4 
 ipc/ipc_sysctl.c                                     |   42 +-
 ipc/shm.c                                            |    8 
 kernel/extable.c                                     |   33 -
 kernel/fork.c                                        |    9 
 kernel/kcov.c                                        |   40 +-
 kernel/locking/lockdep.c                             |    3 
 kernel/resource.c                                    |   54 ++-
 kernel/trace/ftrace.c                                |    2 
 lib/scatterlist.c                                    |   11 
 lib/stackdepot.c                                     |   46 ++
 lib/vsprintf.c                                       |    3 
 mm/Kconfig                                           |    7 
 mm/filemap.c                                         |    8 
 mm/kasan/report.c                                    |   17 -
 mm/memfd.c                                           |    4 
 mm/mmap.c                                            |    3 
 mm/page_owner.c                                      |   18 -
 mm/truncate.c                                        |   19 +
 mm/vmscan.c                                          |    7 
 mm/workingset.c                                      |   10 
 net/sysctl_net.c                                     |    2 
 scripts/checkpatch.pl                                |   33 +
 scripts/const_structs.checkpatch                     |    4 
 scripts/gdb/linux/symbols.py                         |    3 
 tools/testing/selftests/kselftest/runner.sh          |   28 +
 tools/testing/selftests/proc/.gitignore              |    1 
 tools/testing/selftests/proc/Makefile                |    2 
 tools/testing/selftests/proc/proc-tid0.c             |   81 ++++
 132 files changed, 1206 insertions(+), 681 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-11-05 20:34 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-11-05 20:34 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

262 patches, based on 8bb7eca972ad531c9b149c0a51ab43a417385813

Subsystems affected by this patch series:

  scripts
  ocfs2
  vfs
  mm/slab-generic
  mm/slab
  mm/slub
  mm/kconfig
  mm/dax
  mm/kasan
  mm/debug
  mm/pagecache
  mm/gup
  mm/swap
  mm/memcg
  mm/pagemap
  mm/mprotect
  mm/mremap
  mm/iomap
  mm/tracing
  mm/vmalloc
  mm/pagealloc
  mm/memory-failure
  mm/hugetlb
  mm/userfaultfd
  mm/vmscan
  mm/tools
  mm/memblock
  mm/oom-kill
  mm/hugetlbfs
  mm/migration
  mm/thp
  mm/readahead
  mm/nommu
  mm/ksm
  mm/vmstat
  mm/madvise
  mm/memory-hotplug
  mm/rmap
  mm/zsmalloc
  mm/highmem
  mm/zram
  mm/cleanups
  mm/kfence
  mm/damon

Subsystem: scripts

    Colin Ian King <colin.king@canonical.com>:
      scripts/spelling.txt: add more spellings to spelling.txt

    Sven Eckelmann <sven@narfation.org>:
      scripts/spelling.txt: fix "mistake" version of "synchronization"

    weidonghui <weidonghui@allwinnertech.com>:
      scripts/decodecode: fix faulting instruction no print when opps.file is DOS format

Subsystem: ocfs2

    Chenyuan Mi <cymi20@fudan.edu.cn>:
      ocfs2: fix handle refcount leak in two exception handling paths

    Valentin Vidic <vvidic@valentin-vidic.from.hr>:
      ocfs2: cleanup journal init and shutdown

    Colin Ian King <colin.king@canonical.com>:
      ocfs2/dlm: remove redundant assignment of variable ret

    Jan Kara <jack@suse.cz>:
    Patch series "ocfs2: Truncate data corruption fix":
      ocfs2: fix data corruption on truncate
      ocfs2: do not zero pages beyond i_size

Subsystem: vfs

    Arnd Bergmann <arnd@arndb.de>:
      fs/posix_acl.c: avoid -Wempty-body warning

    Jia He <justin.he@arm.com>:
      d_path: fix Kernel doc validator complaining

Subsystem: mm/slab-generic

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm: move kvmalloc-related functions to slab.h

Subsystem: mm/slab

    Shi Lei <shi_lei@massclouds.com>:
      mm/slab.c: remove useless lines in enable_cpucache()

Subsystem: mm/slub

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      slub: add back check for free nonslab objects

    Vlastimil Babka <vbabka@suse.cz>:
      mm, slub: change percpu partial accounting from objects to pages
      mm/slub: increase default cpu partial list sizes

    Hyeonggon Yoo <42.hyeyoo@gmail.com>:
      mm, slub: use prefetchw instead of prefetch

Subsystem: mm/kconfig

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
      mm: disable NUMA_BALANCING_DEFAULT_ENABLED and TRANSPARENT_HUGEPAGE on PREEMPT_RT

Subsystem: mm/dax

    Christoph Hellwig <hch@lst.de>:
      mm: don't include <linux/dax.h> in <linux/mempolicy.h>

Subsystem: mm/kasan

    Marco Elver <elver@google.com>:
    Patch series "stackdepot, kasan, workqueue: Avoid expanding stackdepot slabs when holding raw_spin_lock", v2:
      lib/stackdepot: include gfp.h
      lib/stackdepot: remove unused function argument
      lib/stackdepot: introduce __stack_depot_save()
      kasan: common: provide can_alloc in kasan_save_stack()
      kasan: generic: introduce kasan_record_aux_stack_noalloc()
      workqueue, kasan: avoid alloc_pages() when recording stack

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      kasan: fix tag for large allocations when using CONFIG_SLAB

    Peter Collingbourne <pcc@google.com>:
      kasan: test: add memcpy test that avoids out-of-bounds write

Subsystem: mm/debug

    Peter Xu <peterx@redhat.com>:
    Patch series "mm/smaps: Fixes and optimizations on shmem swap handling":
      mm/smaps: fix shmem pte hole swap calculation
      mm/smaps: use vma->vm_pgoff directly when counting partial swap
      mm/smaps: simplify shmem handling of pte holes

    Guo Ren <guoren@linux.alibaba.com>:
      mm: debug_vm_pgtable: don't use __P000 directly

    Kees Cook <keescook@chromium.org>:
      kasan: test: bypass __alloc_size checks
    Patch series "Add __alloc_size()", v3:
      rapidio: avoid bogus __alloc_size warning
      Compiler Attributes: add __alloc_size() for better bounds checking
      slab: clean up function prototypes
      slab: add __alloc_size attributes for better bounds checking
      mm/kvmalloc: add __alloc_size attributes for better bounds checking
      mm/vmalloc: add __alloc_size attributes for better bounds checking
      mm/page_alloc: add __alloc_size attributes for better bounds checking
      percpu: add __alloc_size attributes for better bounds checking

    Yinan Zhang <zhangyinan2019@email.szu.edu.cn>:
      mm/page_ext.c: fix a comment

Subsystem: mm/pagecache

    David Howells <dhowells@redhat.com>:
      mm: stop filemap_read() from grabbing a superfluous page

    Christoph Hellwig <hch@lst.de>:
    Patch series "simplify bdi unregistation":
      mm: export bdi_unregister
      mtd: call bdi_unregister explicitly
      fs: explicitly unregister per-superblock BDIs
      mm: don't automatically unregister bdis
      mm: simplify bdi refcounting

    Jens Axboe <axboe@kernel.dk>:
      mm: don't read i_size of inode unless we need it

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm/filemap.c: remove bogus VM_BUG_ON

    Jens Axboe <axboe@kernel.dk>:
      mm: move more expensive part of XA setup out of mapping check

Subsystem: mm/gup

    John Hubbard <jhubbard@nvidia.com>:
      mm/gup: further simplify __gup_device_huge()

Subsystem: mm/swap

    Xu Wang <vulab@iscas.ac.cn>:
      mm/swapfile: remove needless request_queue NULL pointer check

    Rafael Aquini <aquini@redhat.com>:
      mm/swapfile: fix an integer overflow in swap_show()

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm: optimise put_pages_list()

Subsystem: mm/memcg

    Peter Xu <peterx@redhat.com>:
      mm/memcg: drop swp_entry_t* in mc_handle_file_pte()

    Shakeel Butt <shakeelb@google.com>:
      memcg: flush stats only if updated
      memcg: unify memcg stat flushing

    Waiman Long <longman@redhat.com>:
      mm/memcg: remove obsolete memcg_free_kmem()

    Len Baker <len.baker@gmx.com>:
      mm/list_lru.c: prefer struct_size over open coded arithmetic

    Shakeel Butt <shakeelb@google.com>:
      memcg, kmem: further deprecate kmem.limit_in_bytes

    Muchun Song <songmuchun@bytedance.com>:
      mm: list_lru: remove holding lru lock
      mm: list_lru: fix the return value of list_lru_count_one()
      mm: memcontrol: remove kmemcg_id reparenting
      mm: memcontrol: remove the kmem states
      mm: list_lru: only add memcg-aware lrus to the global lru list

    Vasily Averin <vvs@virtuozzo.com>:
    Patch series "memcg: prohibit unconditional exceeding the limit of dying tasks", v3:
      mm, oom: pagefault_out_of_memory: don't force global OOM for dying tasks

    Michal Hocko <mhocko@suse.com>:
      mm, oom: do not trigger out_of_memory from the #PF

    Vasily Averin <vvs@virtuozzo.com>:
      memcg: prohibit unconditional exceeding the limit of dying tasks

Subsystem: mm/pagemap

    Peng Liu <liupeng256@huawei.com>:
      mm/mmap.c: fix a data race of mm->total_vm

    Rolf Eike Beer <eb@emlix.com>:
      mm: use __pfn_to_section() instead of open coding it

    Amit Daniel Kachhap <amit.kachhap@arm.com>:
      mm/memory.c: avoid unnecessary kernel/user pointer conversion

    Nadav Amit <namit@vmware.com>:
      mm/memory.c: use correct VMA flags when freeing page-tables

    Peter Xu <peterx@redhat.com>:
    Patch series "mm: A few cleanup patches around zap, shmem and uffd", v4:
      mm/shmem: unconditionally set pte dirty in mfill_atomic_install_pte
      mm: clear vmf->pte after pte_unmap_same() returns
      mm: drop first_index/last_index in zap_details
      mm: add zap_skip_check_mapping() helper

    Qi Zheng <zhengqi.arch@bytedance.com>:
    Patch series "Do some code cleanups related to mm", v3:
      mm: introduce pmd_install() helper
      mm: remove redundant smp_wmb()

    Tiberiu A Georgescu <tiberiu.georgescu@nutanix.com>:
      Documentation: update pagemap with shmem exceptions

    Nicholas Piggin <npiggin@gmail.com>:
    Patch series "shoot lazy tlbs", v4:
      lazy tlb: introduce lazy mm refcount helper functions
      lazy tlb: allow lazy tlb mm refcounting to be configurable
      lazy tlb: shoot lazies, a non-refcounting lazy tlb option
      powerpc/64s: enable MMU_LAZY_TLB_SHOOTDOWN

    Lukas Bulwahn <lukas.bulwahn@gmail.com>:
      memory: remove unused CONFIG_MEM_BLOCK_SIZE

Subsystem: mm/mprotect

    Liu Song <liu.song11@zte.com.cn>:
      mm/mprotect.c: avoid repeated assignment in do_mprotect_pkey()

Subsystem: mm/mremap

    Dmitry Safonov <dima@arista.com>:
      mm/mremap: don't account pages in vma_to_resize()

Subsystem: mm/iomap

    Lucas De Marchi <lucas.demarchi@intel.com>:
      include/linux/io-mapping.h: remove fallback for writecombine

Subsystem: mm/tracing

    Gang Li <ligang.bdlg@bytedance.com>:
      mm: mmap_lock: remove redundant newline  in TP_printk
      mm: mmap_lock: use DECLARE_EVENT_CLASS and DEFINE_EVENT_FN

Subsystem: mm/vmalloc

    Vasily Averin <vvs@virtuozzo.com>:
      mm/vmalloc: repair warn_alloc()s in __vmalloc_area_node()

    Peter Zijlstra <peterz@infradead.org>:
      mm/vmalloc: don't allow VM_NO_GUARD on vmap()

    Eric Dumazet <edumazet@google.com>:
      mm/vmalloc: make show_numa_info() aware of hugepage mappings
      mm/vmalloc: make sure to dump unpurged areas in /proc/vmallocinfo

    "Uladzislau Rezki (Sony)" <urezki@gmail.com>:
      mm/vmalloc: do not adjust the search size for alignment overhead
      mm/vmalloc: check various alignments when debugging

    Vasily Averin <vvs@virtuozzo.com>:
      vmalloc: back off when the current task is OOM-killed

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      vmalloc: choose a better start address in vm_area_register_early()
      arm64: support page mapping percpu first chunk allocator
      kasan: arm64: fix pcpu_page_first_chunk crash with KASAN_VMALLOC

    Michal Hocko <mhocko@suse.com>:
      mm/vmalloc: be more explicit about supported gfp flags

    Chen Wandun <chenwandun@huawei.com>:
      mm/vmalloc: introduce alloc_pages_bulk_array_mempolicy to accelerate memory allocation

    Changcheng Deng <deng.changcheng@zte.com.cn>:
      lib/test_vmalloc.c: use swap() to make code cleaner

Subsystem: mm/pagealloc

    Eric Dumazet <edumazet@google.com>:
      mm/large system hash: avoid possible NULL deref in alloc_large_system_hash

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanups and fixup for page_alloc", v2:
      mm/page_alloc.c: remove meaningless VM_BUG_ON() in pindex_to_order()
      mm/page_alloc.c: simplify the code by using macro K()
      mm/page_alloc.c: fix obsolete comment in free_pcppages_bulk()
      mm/page_alloc.c: use helper function zone_spans_pfn()
      mm/page_alloc.c: avoid allocating highmem pages via alloc_pages_exact[_nid]

    Bharata B Rao <bharata@amd.com>:
    Patch series "Fix NUMA nodes fallback list ordering":
      mm/page_alloc: print node fallback order

    Krupa Ramakrishnan <krupa.ramakrishnan@amd.com>:
      mm/page_alloc: use accumulated load when building node fallback list

    Geert Uytterhoeven <geert+renesas@glider.be>:
    Patch series "Fix NUMA without SMP":
      mm: move node_reclaim_distance to fix NUMA without SMP
      mm: move fold_vm_numa_events() to fix NUMA without SMP

    Eric Dumazet <edumazet@google.com>:
      mm/page_alloc.c: do not acquire zone lock in is_free_buddy_page()

    Feng Tang <feng.tang@intel.com>:
      mm/page_alloc: detect allocation forbidden by cpuset and bail out early

    Liangcai Fan <liangcaifan19@gmail.com>:
      mm/page_alloc.c: show watermark_boost of zone in zoneinfo

    Christophe Leroy <christophe.leroy@csgroup.eu>:
      mm: create a new system state and fix core_kernel_text()
      mm: make generic arch_is_kernel_initmem_freed() do what it says
      powerpc: use generic version of arch_is_kernel_initmem_freed()
      s390: use generic version of arch_is_kernel_initmem_freed()

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
      mm: page_alloc: use migrate_disable() in drain_local_pages_wq()

    Wang ShaoBo <bobo.shaobowang@huawei.com>:
      mm/page_alloc: use clamp() to simplify code

Subsystem: mm/memory-failure

    Marco Elver <elver@google.com>:
      mm: fix data race in PagePoisoned()

    Rikard Falkeborn <rikard.falkeborn@gmail.com>:
      mm/memory_failure: constify static mm_walk_ops

    Yang Shi <shy828301@gmail.com>:
    Patch series "Solve silent data loss caused by poisoned page cache (shmem/tmpfs)", v5:
      mm: filemap: coding style cleanup for filemap_map_pmd()
      mm: hwpoison: refactor refcount check handling
      mm: shmem: don't truncate page if memory failure happens
      mm: hwpoison: handle non-anonymous THP correctly

Subsystem: mm/hugetlb

    Peter Xu <peterx@redhat.com>:
      mm/hugetlb: drop __unmap_hugepage_range definition from hugetlb.h

    Mike Kravetz <mike.kravetz@oracle.com>:
    Patch series "hugetlb: add demote/split page functionality", v4:
      hugetlb: add demote hugetlb page sysfs interfaces
      mm/cma: add cma_pages_valid to determine if pages are in CMA
      hugetlb: be sure to free demoted CMA pages to CMA
      hugetlb: add demote bool to gigantic page routines
      hugetlb: add hugetlb demote page support

    Liangcai Fan <liangcaifan19@gmail.com>:
      mm: khugepaged: recalculate min_free_kbytes after stopping khugepaged

    Mina Almasry <almasrymina@google.com>:
      mm, hugepages: add mremap() support for hugepage backed vma
      mm, hugepages: add hugetlb vma mremap() test

    Baolin Wang <baolin.wang@linux.alibaba.com>:
      hugetlb: support node specified when using cma for gigantic hugepages

    Ran Jianping <ran.jianping@zte.com.cn>:
      mm: remove duplicate include in hugepage-mremap.c

    Baolin Wang <baolin.wang@linux.alibaba.com>:
    Patch series "Some cleanups and improvements for hugetlb":
      hugetlb_cgroup: remove unused hugetlb_cgroup_from_counter macro
      hugetlb: replace the obsolete hugetlb_instantiation_mutex in the comments
      hugetlb: remove redundant validation in has_same_uncharge_info()
      hugetlb: remove redundant VM_BUG_ON() in add_reservation_in_range()

    Mike Kravetz <mike.kravetz@oracle.com>:
      hugetlb: remove unnecessary set_page_count in prep_compound_gigantic_page

Subsystem: mm/userfaultfd

    Axel Rasmussen <axelrasmussen@google.com>:
    Patch series "Small userfaultfd selftest fixups", v2:
      userfaultfd/selftests: don't rely on GNU extensions for random numbers
      userfaultfd/selftests: fix feature support detection
      userfaultfd/selftests: fix calculation of expected ioctls

Subsystem: mm/vmscan

    Miaohe Lin <linmiaohe@huawei.com>:
      mm/page_isolation: fix potential missing call to unset_migratetype_isolate()
      mm/page_isolation: guard against possible putback unisolated page

    Kai Song <songkai01@inspur.com>:
      mm/vmscan.c: fix -Wunused-but-set-variable warning

    Mel Gorman <mgorman@techsingularity.net>:
    Patch series "Remove dependency on congestion_wait in mm/", v5. Patch series:
      mm/vmscan: throttle reclaim until some writeback completes if congested
      mm/vmscan: throttle reclaim and compaction when too may pages are isolated
      mm/vmscan: throttle reclaim when no progress is being made
      mm/writeback: throttle based on page writeback instead of congestion
      mm/page_alloc: remove the throttling logic from the page allocator
      mm/vmscan: centralise timeout values for reclaim_throttle
      mm/vmscan: increase the timeout if page reclaim is not making progress
      mm/vmscan: delay waking of tasks throttled on NOPROGRESS

    Yuanzheng Song <songyuanzheng@huawei.com>:
      mm/vmpressure: fix data-race with memcg->socket_pressure

Subsystem: mm/tools

    Zhenliang Wei <weizhenliang@huawei.com>:
      tools/vm/page_owner_sort.c: count and sort by mem

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
    Patch series "tools/vm/page-types.c: a few improvements":
      tools/vm/page-types.c: make walk_file() aware of address range option
      tools/vm/page-types.c: move show_file() to summary output
      tools/vm/page-types.c: print file offset in hexadecimal

Subsystem: mm/memblock

    Mike Rapoport <rppt@linux.ibm.com>:
    Patch series "memblock: cleanup memblock_free interface", v2:
      arch_numa: simplify numa_distance allocation
      xen/x86: free_p2m_page: use memblock_free_ptr() to free a virtual pointer
      memblock: drop memblock_free_early_nid() and memblock_free_early()
      memblock: stop aliasing __memblock_free_late with memblock_free_late
      memblock: rename memblock_free to memblock_phys_free
      memblock: use memblock_free for freeing virtual pointers

Subsystem: mm/oom-kill

    Sultan Alsawaf <sultan@kerneltoast.com>:
      mm: mark the OOM reaper thread as freezable

Subsystem: mm/hugetlbfs

    Zhenguo Yao <yaozhenguo1@gmail.com>:
      hugetlbfs: extend the definition of hugepages parameter to support node allocation

Subsystem: mm/migration

    John Hubbard <jhubbard@nvidia.com>:
      mm/migrate: de-duplicate migrate_reason strings

    Yang Shi <shy828301@gmail.com>:
      mm: migrate: make demotion knob depend on migration

Subsystem: mm/thp

    "George G. Davis" <davis.george@siemens.com>:
      selftests/vm/transhuge-stress: fix ram size thinko

    Rongwei Wang <rongwei.wang@linux.alibaba.com>:
    Patch series "fix two bugs for file THP":
      mm, thp: lock filemap when truncating page cache
      mm, thp: fix incorrect unmap behavior for private pages

Subsystem: mm/readahead

    Lin Feng <linf@wangsu.com>:
      mm/readahead.c: fix incorrect comments for get_init_ra_size

Subsystem: mm/nommu

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      mm: nommu: kill arch_get_unmapped_area()

Subsystem: mm/ksm

    "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>:
      selftest/vm: fix ksm selftest to run with different NUMA topologies

    Pedro Demarchi Gomes <pedrodemargomes@gmail.com>:
      selftests: vm: add KSM huge pages merging time test

Subsystem: mm/vmstat

    Liu Shixin <liushixin2@huawei.com>:
      mm/vmstat: annotate data race for zone->free_area[order].nr_free

    Lin Feng <linf@wangsu.com>:
      mm: vmstat.c: make extfrag_index show more pretty

Subsystem: mm/madvise

    David Hildenbrand <david@redhat.com>:
      selftests/vm: make MADV_POPULATE_(READ|WRITE) use in-tree headers

Subsystem: mm/memory-hotplug

    Tang Yizhou <tangyizhou@huawei.com>:
      mm/memory_hotplug: add static qualifier for online_policy_to_str()

    David Hildenbrand <david@redhat.com>:
    Patch series "memory-hotplug.rst: document the "auto-movable" online policy":
      memory-hotplug.rst: fix two instances of "movablecore" that should be "movable_node"
      memory-hotplug.rst: fix wrong /sys/module/memory_hotplug/parameters/ path
      memory-hotplug.rst: document the "auto-movable" online policy
    Patch series "mm/memory_hotplug: Kconfig and 32 bit cleanups":
      mm/memory_hotplug: remove CONFIG_X86_64_ACPI_NUMA dependency from CONFIG_MEMORY_HOTPLUG
      mm/memory_hotplug: remove CONFIG_MEMORY_HOTPLUG_SPARSE
      mm/memory_hotplug: restrict CONFIG_MEMORY_HOTPLUG to 64 bit
      mm/memory_hotplug: remove HIGHMEM leftovers
      mm/memory_hotplug: remove stale function declarations
      x86: remove memory hotplug support on X86_32
    Patch series "mm/memory_hotplug: full support for add_memory_driver_managed() with CONFIG_ARCH_KEEP_MEMBLOCK", v2:
      mm/memory_hotplug: handle memblock_add_node() failures in add_memory_resource()
      memblock: improve MEMBLOCK_HOTPLUG documentation
      memblock: allow to specify flags with memblock_add_node()
      memblock: add MEMBLOCK_DRIVER_MANAGED to mimic IORESOURCE_SYSRAM_DRIVER_MANAGED
      mm/memory_hotplug: indicate MEMBLOCK_DRIVER_MANAGED with IORESOURCE_SYSRAM_DRIVER_MANAGED

Subsystem: mm/rmap

    Alistair Popple <apopple@nvidia.com>:
      mm/rmap.c: avoid double faults migrating device private pages

Subsystem: mm/zsmalloc

    Miaohe Lin <linmiaohe@huawei.com>:
      mm/zsmalloc.c: close race window between zs_pool_dec_isolated() and zs_unregister_migration()

Subsystem: mm/highmem

    Ira Weiny <ira.weiny@intel.com>:
      mm/highmem: remove deprecated kmap_atomic

Subsystem: mm/zram

    Jaewon Kim <jaewon31.kim@samsung.com>:
      zram_drv: allow reclaim on bio_alloc

    Dan Carpenter <dan.carpenter@oracle.com>:
      zram: off by one in read_block_state()

    Brian Geffon <bgeffon@google.com>:
      zram: introduce an aged idle interface

Subsystem: mm/cleanups

    Stephen Kitt <steve@sk2.org>:
      mm: remove HARDENED_USERCOPY_FALLBACK

    Mianhan Liu <liumh1@shanghaitech.edu.cn>:
      include/linux/mm.h: move nr_free_buffer_pages from swap.h to mm.h

Subsystem: mm/kfence

    Marco Elver <elver@google.com>:
      stacktrace: move filter_irq_stacks() to kernel/stacktrace.c
      kfence: count unexpectedly skipped allocations
      kfence: move saving stack trace of allocations into __kfence_alloc()
      kfence: limit currently covered allocations when pool nearly full
      kfence: add note to documentation about skipping covered allocations
      kfence: test: use kunit_skip() to skip tests
      kfence: shorten critical sections of alloc/free
      kfence: always use static branches to guard kfence_alloc()
      kfence: default to dynamic branch instead of static keys mode

Subsystem: mm/damon

    Geert Uytterhoeven <geert@linux-m68k.org>:
      mm/damon: grammar s/works/work/

    SeongJae Park <sjpark@amazon.de>:
      Documentation/vm: move user guides to admin-guide/mm/

    SeongJae Park <sj@kernel.org>:
      MAINTAINERS: update SeongJae's email address

    SeongJae Park <sjpark@amazon.de>:
      docs/vm/damon: remove broken reference
      include/linux/damon.h: fix kernel-doc comments for 'damon_callback'

    SeongJae Park <sj@kernel.org>:
      mm/damon/core: print kdamond start log in debug mode only

    Changbin Du <changbin.du@gmail.com>:
      mm/damon: remove unnecessary do_exit() from kdamond
      mm/damon: needn't hold kdamond_lock to print pid of kdamond

    Colin Ian King <colin.king@canonical.com>:
      mm/damon/core: nullify pointer ctx->kdamond with a NULL

    SeongJae Park <sj@kernel.org>:
    Patch series "Implement Data Access Monitoring-based Memory Operation Schemes":
      mm/damon/core: account age of target regions
      mm/damon/core: implement DAMON-based Operation Schemes (DAMOS)
      mm/damon/vaddr: support DAMON-based Operation Schemes
      mm/damon/dbgfs: support DAMON-based Operation Schemes
      mm/damon/schemes: implement statistics feature
      selftests/damon: add 'schemes' debugfs tests
      Docs/admin-guide/mm/damon: document DAMON-based Operation Schemes
    Patch series "DAMON: Support Physical Memory Address Space Monitoring::
      mm/damon/dbgfs: allow users to set initial monitoring target regions
      mm/damon/dbgfs-test: add a unit test case for 'init_regions'
      Docs/admin-guide/mm/damon: document 'init_regions' feature
      mm/damon/vaddr: separate commonly usable functions
      mm/damon: implement primitives for physical address space monitoring
      mm/damon/dbgfs: support physical memory monitoring
      Docs/DAMON: document physical memory monitoring support

    Rikard Falkeborn <rikard.falkeborn@gmail.com>:
      mm/damon/vaddr: constify static mm_walk_ops

    Rongwei Wang <rongwei.wang@linux.alibaba.com>:
      mm/damon/dbgfs: remove unnecessary variables

    SeongJae Park <sj@kernel.org>:
      mm/damon/paddr: support the pageout scheme
      mm/damon/schemes: implement size quota for schemes application speed control
      mm/damon/schemes: skip already charged targets and regions
      mm/damon/schemes: implement time quota
      mm/damon/dbgfs: support quotas of schemes
      mm/damon/selftests: support schemes quotas
      mm/damon/schemes: prioritize regions within the quotas
      mm/damon/vaddr,paddr: support pageout prioritization
      mm/damon/dbgfs: support prioritization weights
      tools/selftests/damon: update for regions prioritization of schemes
      mm/damon/schemes: activate schemes based on a watermarks mechanism
      mm/damon/dbgfs: support watermarks
      selftests/damon: support watermarks
      mm/damon: introduce DAMON-based Reclamation (DAMON_RECLAIM)
      Documentation/admin-guide/mm/damon: add a document for DAMON_RECLAIM

    Xin Hao <xhao@linux.alibaba.com>:
    Patch series "mm/damon: Fix some small bugs", v4:
      mm/damon: remove unnecessary variable initialization
      mm/damon/dbgfs: add adaptive_targets list check before enable monitor_on

    SeongJae Park <sj@kernel.org>:
    Patch series "Fix trivial nits in Documentation/admin-guide/mm":
      Docs/admin-guide/mm/damon/start: fix wrong example commands
      Docs/admin-guide/mm/damon/start: fix a wrong link
      Docs/admin-guide/mm/damon/start: simplify the content
      Docs/admin-guide/mm/pagemap: wordsmith page flags descriptions

    Changbin Du <changbin.du@gmail.com>:
      mm/damon: simplify stop mechanism

    Colin Ian King <colin.i.king@googlemail.com>:
      mm/damon: fix a few spelling mistakes in comments and a pr_debug message

    Changbin Du <changbin.du@gmail.com>:
      mm/damon: remove return value from before_terminate callback

 a/Documentation/admin-guide/blockdev/zram.rst                  |    8 
 a/Documentation/admin-guide/cgroup-v1/memory.rst               |   11 
 a/Documentation/admin-guide/kernel-parameters.txt              |   14 
 a/Documentation/admin-guide/mm/damon/index.rst                 |    1 
 a/Documentation/admin-guide/mm/damon/reclaim.rst               |  235 +++
 a/Documentation/admin-guide/mm/damon/start.rst                 |  140 +
 a/Documentation/admin-guide/mm/damon/usage.rst                 |  117 +
 a/Documentation/admin-guide/mm/hugetlbpage.rst                 |   42 
 a/Documentation/admin-guide/mm/memory-hotplug.rst              |  147 +-
 a/Documentation/admin-guide/mm/pagemap.rst                     |   75 -
 a/Documentation/core-api/memory-hotplug.rst                    |    3 
 a/Documentation/dev-tools/kfence.rst                           |   23 
 a/Documentation/translations/zh_CN/core-api/memory-hotplug.rst |    4 
 a/Documentation/vm/damon/design.rst                            |   29 
 a/Documentation/vm/damon/faq.rst                               |    5 
 a/Documentation/vm/damon/index.rst                             |    1 
 a/Documentation/vm/page_owner.rst                              |   23 
 a/MAINTAINERS                                                  |    2 
 a/Makefile                                                     |   15 
 a/arch/Kconfig                                                 |   28 
 a/arch/alpha/kernel/core_irongate.c                            |    6 
 a/arch/arc/mm/init.c                                           |    6 
 a/arch/arm/mach-hisi/platmcpm.c                                |    2 
 a/arch/arm/mach-rpc/ecard.c                                    |    2 
 a/arch/arm/mm/init.c                                           |    2 
 a/arch/arm64/Kconfig                                           |    4 
 a/arch/arm64/mm/kasan_init.c                                   |   16 
 a/arch/arm64/mm/mmu.c                                          |    4 
 a/arch/ia64/mm/contig.c                                        |    2 
 a/arch/ia64/mm/init.c                                          |    2 
 a/arch/m68k/mm/mcfmmu.c                                        |    3 
 a/arch/m68k/mm/motorola.c                                      |    6 
 a/arch/mips/loongson64/init.c                                  |    4 
 a/arch/mips/mm/init.c                                          |    6 
 a/arch/mips/sgi-ip27/ip27-memory.c                             |    3 
 a/arch/mips/sgi-ip30/ip30-setup.c                              |    6 
 a/arch/powerpc/Kconfig                                         |    1 
 a/arch/powerpc/configs/skiroot_defconfig                       |    1 
 a/arch/powerpc/include/asm/machdep.h                           |    2 
 a/arch/powerpc/include/asm/sections.h                          |   13 
 a/arch/powerpc/kernel/dt_cpu_ftrs.c                            |    8 
 a/arch/powerpc/kernel/paca.c                                   |    8 
 a/arch/powerpc/kernel/setup-common.c                           |    4 
 a/arch/powerpc/kernel/setup_64.c                               |    6 
 a/arch/powerpc/kernel/smp.c                                    |    2 
 a/arch/powerpc/mm/book3s64/radix_tlb.c                         |    4 
 a/arch/powerpc/mm/hugetlbpage.c                                |    9 
 a/arch/powerpc/platforms/powernv/pci-ioda.c                    |    4 
 a/arch/powerpc/platforms/powernv/setup.c                       |    4 
 a/arch/powerpc/platforms/pseries/setup.c                       |    2 
 a/arch/powerpc/platforms/pseries/svm.c                         |    9 
 a/arch/riscv/kernel/setup.c                                    |   10 
 a/arch/s390/include/asm/sections.h                             |   12 
 a/arch/s390/kernel/setup.c                                     |   11 
 a/arch/s390/kernel/smp.c                                       |    6 
 a/arch/s390/kernel/uv.c                                        |    2 
 a/arch/s390/mm/init.c                                          |    3 
 a/arch/s390/mm/kasan_init.c                                    |    2 
 a/arch/sh/boards/mach-ap325rxa/setup.c                         |    2 
 a/arch/sh/boards/mach-ecovec24/setup.c                         |    4 
 a/arch/sh/boards/mach-kfr2r09/setup.c                          |    2 
 a/arch/sh/boards/mach-migor/setup.c                            |    2 
 a/arch/sh/boards/mach-se/7724/setup.c                          |    4 
 a/arch/sparc/kernel/smp_64.c                                   |    4 
 a/arch/um/kernel/mem.c                                         |    4 
 a/arch/x86/Kconfig                                             |    6 
 a/arch/x86/kernel/setup.c                                      |    4 
 a/arch/x86/kernel/setup_percpu.c                               |    2 
 a/arch/x86/mm/init.c                                           |    2 
 a/arch/x86/mm/init_32.c                                        |   31 
 a/arch/x86/mm/kasan_init_64.c                                  |    4 
 a/arch/x86/mm/numa.c                                           |    2 
 a/arch/x86/mm/numa_emulation.c                                 |    2 
 a/arch/x86/xen/mmu_pv.c                                        |    8 
 a/arch/x86/xen/p2m.c                                           |    4 
 a/arch/x86/xen/setup.c                                         |    6 
 a/drivers/base/Makefile                                        |    2 
 a/drivers/base/arch_numa.c                                     |   96 +
 a/drivers/base/node.c                                          |    9 
 a/drivers/block/zram/zram_drv.c                                |   66 
 a/drivers/firmware/efi/memmap.c                                |    2 
 a/drivers/hwmon/occ/p9_sbe.c                                   |    1 
 a/drivers/macintosh/smu.c                                      |    2 
 a/drivers/mmc/core/mmc_test.c                                  |    1 
 a/drivers/mtd/mtdcore.c                                        |    1 
 a/drivers/of/kexec.c                                           |    4 
 a/drivers/of/of_reserved_mem.c                                 |    5 
 a/drivers/rapidio/devices/rio_mport_cdev.c                     |    9 
 a/drivers/s390/char/sclp_early.c                               |    4 
 a/drivers/usb/early/xhci-dbc.c                                 |   10 
 a/drivers/virtio/Kconfig                                       |    2 
 a/drivers/xen/swiotlb-xen.c                                    |    4 
 a/fs/d_path.c                                                  |    8 
 a/fs/exec.c                                                    |    4 
 a/fs/ocfs2/alloc.c                                             |   21 
 a/fs/ocfs2/dlm/dlmrecovery.c                                   |    1 
 a/fs/ocfs2/file.c                                              |    8 
 a/fs/ocfs2/inode.c                                             |    4 
 a/fs/ocfs2/journal.c                                           |   28 
 a/fs/ocfs2/journal.h                                           |    3 
 a/fs/ocfs2/super.c                                             |   40 
 a/fs/open.c                                                    |   16 
 a/fs/posix_acl.c                                               |    3 
 a/fs/proc/task_mmu.c                                           |   28 
 a/fs/super.c                                                   |    3 
 a/include/asm-generic/sections.h                               |   14 
 a/include/linux/backing-dev-defs.h                             |    3 
 a/include/linux/backing-dev.h                                  |    1 
 a/include/linux/cma.h                                          |    1 
 a/include/linux/compiler-gcc.h                                 |    8 
 a/include/linux/compiler_attributes.h                          |   10 
 a/include/linux/compiler_types.h                               |   12 
 a/include/linux/cpuset.h                                       |   17 
 a/include/linux/damon.h                                        |  258 +++
 a/include/linux/fs.h                                           |    1 
 a/include/linux/gfp.h                                          |    8 
 a/include/linux/highmem.h                                      |   28 
 a/include/linux/hugetlb.h                                      |   36 
 a/include/linux/io-mapping.h                                   |    6 
 a/include/linux/kasan.h                                        |    8 
 a/include/linux/kernel.h                                       |    1 
 a/include/linux/kfence.h                                       |   21 
 a/include/linux/memblock.h                                     |   48 
 a/include/linux/memcontrol.h                                   |    9 
 a/include/linux/memory.h                                       |   26 
 a/include/linux/memory_hotplug.h                               |    3 
 a/include/linux/mempolicy.h                                    |    5 
 a/include/linux/migrate.h                                      |   23 
 a/include/linux/migrate_mode.h                                 |   13 
 a/include/linux/mm.h                                           |   57 
 a/include/linux/mm_types.h                                     |    2 
 a/include/linux/mmzone.h                                       |   41 
 a/include/linux/node.h                                         |    4 
 a/include/linux/page-flags.h                                   |    2 
 a/include/linux/percpu.h                                       |    6 
 a/include/linux/sched/mm.h                                     |   25 
 a/include/linux/slab.h                                         |  181 +-
 a/include/linux/slub_def.h                                     |   13 
 a/include/linux/stackdepot.h                                   |    8 
 a/include/linux/stacktrace.h                                   |    1 
 a/include/linux/swap.h                                         |    1 
 a/include/linux/vmalloc.h                                      |   24 
 a/include/trace/events/mmap_lock.h                             |   50 
 a/include/trace/events/vmscan.h                                |   42 
 a/include/trace/events/writeback.h                             |    7 
 a/init/Kconfig                                                 |    2 
 a/init/initramfs.c                                             |    4 
 a/init/main.c                                                  |    6 
 a/kernel/cgroup/cpuset.c                                       |   23 
 a/kernel/cpu.c                                                 |    2 
 a/kernel/dma/swiotlb.c                                         |    6 
 a/kernel/exit.c                                                |    2 
 a/kernel/extable.c                                             |    2 
 a/kernel/fork.c                                                |   51 
 a/kernel/kexec_file.c                                          |    5 
 a/kernel/kthread.c                                             |   21 
 a/kernel/locking/lockdep.c                                     |   15 
 a/kernel/printk/printk.c                                       |    4 
 a/kernel/sched/core.c                                          |   37 
 a/kernel/sched/sched.h                                         |    4 
 a/kernel/sched/topology.c                                      |    1 
 a/kernel/stacktrace.c                                          |   30 
 a/kernel/tsacct.c                                              |    2 
 a/kernel/workqueue.c                                           |    2 
 a/lib/Kconfig.debug                                            |    2 
 a/lib/Kconfig.kfence                                           |   26 
 a/lib/bootconfig.c                                             |    2 
 a/lib/cpumask.c                                                |    6 
 a/lib/stackdepot.c                                             |   76 -
 a/lib/test_kasan.c                                             |   26 
 a/lib/test_kasan_module.c                                      |    2 
 a/lib/test_vmalloc.c                                           |    6 
 a/mm/Kconfig                                                   |   10 
 a/mm/backing-dev.c                                             |   65 
 a/mm/cma.c                                                     |   26 
 a/mm/compaction.c                                              |   12 
 a/mm/damon/Kconfig                                             |   24 
 a/mm/damon/Makefile                                            |    4 
 a/mm/damon/core.c                                              |  500 ++++++-
 a/mm/damon/dbgfs-test.h                                        |   56 
 a/mm/damon/dbgfs.c                                             |  486 +++++-
 a/mm/damon/paddr.c                                             |  275 +++
 a/mm/damon/prmtv-common.c                                      |  133 +
 a/mm/damon/prmtv-common.h                                      |   20 
 a/mm/damon/reclaim.c                                           |  356 ++++
 a/mm/damon/vaddr-test.h                                        |    2 
 a/mm/damon/vaddr.c                                             |  167 +-
 a/mm/debug.c                                                   |   20 
 a/mm/debug_vm_pgtable.c                                        |    7 
 a/mm/filemap.c                                                 |   78 -
 a/mm/gup.c                                                     |    5 
 a/mm/highmem.c                                                 |    6 
 a/mm/hugetlb.c                                                 |  713 +++++++++-
 a/mm/hugetlb_cgroup.c                                          |    3 
 a/mm/internal.h                                                |   26 
 a/mm/kasan/common.c                                            |    8 
 a/mm/kasan/generic.c                                           |   16 
 a/mm/kasan/kasan.h                                             |    2 
 a/mm/kasan/shadow.c                                            |    5 
 a/mm/kfence/core.c                                             |  214 ++-
 a/mm/kfence/kfence.h                                           |    2 
 a/mm/kfence/kfence_test.c                                      |   14 
 a/mm/khugepaged.c                                              |   10 
 a/mm/list_lru.c                                                |   58 
 a/mm/memblock.c                                                |   35 
 a/mm/memcontrol.c                                              |  217 +--
 a/mm/memory-failure.c                                          |  117 +
 a/mm/memory.c                                                  |  166 +-
 a/mm/memory_hotplug.c                                          |   57 
 a/mm/mempolicy.c                                               |  143 +-
 a/mm/migrate.c                                                 |   61 
 a/mm/mmap.c                                                    |    2 
 a/mm/mprotect.c                                                |    5 
 a/mm/mremap.c                                                  |   86 -
 a/mm/nommu.c                                                   |    6 
 a/mm/oom_kill.c                                                |   27 
 a/mm/page-writeback.c                                          |   13 
 a/mm/page_alloc.c                                              |  119 -
 a/mm/page_ext.c                                                |    2 
 a/mm/page_isolation.c                                          |   29 
 a/mm/percpu.c                                                  |   24 
 a/mm/readahead.c                                               |    2 
 a/mm/rmap.c                                                    |    8 
 a/mm/shmem.c                                                   |   44 
 a/mm/slab.c                                                    |   16 
 a/mm/slab_common.c                                             |    8 
 a/mm/slub.c                                                    |  117 -
 a/mm/sparse-vmemmap.c                                          |    2 
 a/mm/sparse.c                                                  |    6 
 a/mm/swap.c                                                    |   23 
 a/mm/swapfile.c                                                |    6 
 a/mm/userfaultfd.c                                             |    8 
 a/mm/vmalloc.c                                                 |  107 +
 a/mm/vmpressure.c                                              |    2 
 a/mm/vmscan.c                                                  |  194 ++
 a/mm/vmstat.c                                                  |   76 -
 a/mm/zsmalloc.c                                                |    7 
 a/net/ipv4/tcp.c                                               |    1 
 a/net/ipv4/udp.c                                               |    1 
 a/net/netfilter/ipvs/ip_vs_ctl.c                               |    1 
 a/net/openvswitch/meter.c                                      |    1 
 a/net/sctp/protocol.c                                          |    1 
 a/scripts/checkpatch.pl                                        |    3 
 a/scripts/decodecode                                           |    2 
 a/scripts/spelling.txt                                         |   18 
 a/security/Kconfig                                             |   14 
 a/tools/testing/selftests/damon/debugfs_attrs.sh               |   25 
 a/tools/testing/selftests/memory-hotplug/config                |    1 
 a/tools/testing/selftests/vm/.gitignore                        |    1 
 a/tools/testing/selftests/vm/Makefile                          |    1 
 a/tools/testing/selftests/vm/hugepage-mremap.c                 |  161 ++
 a/tools/testing/selftests/vm/ksm_tests.c                       |  154 ++
 a/tools/testing/selftests/vm/madv_populate.c                   |   15 
 a/tools/testing/selftests/vm/run_vmtests.sh                    |   11 
 a/tools/testing/selftests/vm/transhuge-stress.c                |    2 
 a/tools/testing/selftests/vm/userfaultfd.c                     |  157 +-
 a/tools/vm/page-types.c                                        |   38 
 a/tools/vm/page_owner_sort.c                                   |   94 +
 b/Documentation/admin-guide/mm/index.rst                       |    2 
 b/Documentation/vm/index.rst                                   |   26 
 260 files changed, 6448 insertions(+), 2327 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-10-28 21:35 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-10-28 21:35 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

11 patches, based on 411a44c24a561e449b592ff631b7ae321f1eb559.

Subsystems affected by this patch series:

  mm/memcg
  mm/memory-failure
  mm/oom-kill
  ocfs2
  mm/secretmem
  mm/vmalloc
  mm/hugetlb
  mm/damon
  mm/tools

Subsystem: mm/memcg

    Shakeel Butt <shakeelb@google.com>:
      memcg: page_alloc: skip bulk allocator for __GFP_ACCOUNT

Subsystem: mm/memory-failure

    Yang Shi <shy828301@gmail.com>:
      mm: hwpoison: remove the unnecessary THP check
      mm: filemap: check if THP has hwpoisoned subpage for PMD page fault

Subsystem: mm/oom-kill

    Suren Baghdasaryan <surenb@google.com>:
      mm/oom_kill.c: prevent a race between process_mrelease and exit_mmap

Subsystem: ocfs2

    Gautham Ananthakrishna <gautham.ananthakrishna@oracle.com>:
      ocfs2: fix race between searching chunks and release journal_head from buffer_head

Subsystem: mm/secretmem

    Kees Cook <keescook@chromium.org>:
      mm/secretmem: avoid letting secretmem_users drop to zero

Subsystem: mm/vmalloc

    Chen Wandun <chenwandun@huawei.com>:
      mm/vmalloc: fix numa spreading for large hash tables

Subsystem: mm/hugetlb

    Rongwei Wang <rongwei.wang@linux.alibaba.com>:
      mm, thp: bail out early in collapse_file for writeback page

    Yang Shi <shy828301@gmail.com>:
      mm: khugepaged: skip huge page collapse for special files

Subsystem: mm/damon

    SeongJae Park <sj@kernel.org>:
      mm/damon/core-test: fix wrong expectations for 'damon_split_regions_of()'

Subsystem: mm/tools

    David Yang <davidcomponentone@gmail.com>:
      tools/testing/selftests/vm/split_huge_page_test.c: fix application of sizeof to pointer

 fs/ocfs2/suballoc.c                               |   22 ++++++++++-------
 include/linux/page-flags.h                        |   23 ++++++++++++++++++
 mm/damon/core-test.h                              |    4 +--
 mm/huge_memory.c                                  |    2 +
 mm/khugepaged.c                                   |   26 +++++++++++++-------
 mm/memory-failure.c                               |   28 +++++++++++-----------
 mm/memory.c                                       |    9 +++++++
 mm/oom_kill.c                                     |   23 +++++++++---------
 mm/page_alloc.c                                   |    8 +++++-
 mm/secretmem.c                                    |    2 -
 mm/vmalloc.c                                      |   15 +++++++----
 tools/testing/selftests/vm/split_huge_page_test.c |    2 -
 12 files changed, 110 insertions(+), 54 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-10-18 22:14 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-10-18 22:14 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits


19 patches, based on 519d81956ee277b4419c723adfb154603c2565ba.

Subsystems affected by this patch series:

  mm/userfaultfd
  mm/migration
  ocfs2
  mm/memblock
  mm/mempolicy
  mm/slub
  binfmt
  vfs
  mm/secretmem
  mm/thp
  misc

Subsystem: mm/userfaultfd

    Peter Xu <peterx@redhat.com>:
      mm/userfaultfd: selftests: fix memory corruption with thp enabled

    Nadav Amit <namit@vmware.com>:
      userfaultfd: fix a race between writeprotect and exit_mmap()

Subsystem: mm/migration

    Dave Hansen <dave.hansen@linux.intel.com>:
    Patch series "mm/migrate: 5.15 fixes for automatic demotion", v2:
      mm/migrate: optimize hotplug-time demotion order updates
      mm/migrate: add CPU hotplug to demotion #ifdef

    Huang Ying <ying.huang@intel.com>:
      mm/migrate: fix CPUHP state to update node demotion order

Subsystem: ocfs2

    Jan Kara <jack@suse.cz>:
      ocfs2: fix data corruption after conversion from inline format

    Valentin Vidic <vvidic@valentin-vidic.from.hr>:
      ocfs2: mount fails with buffer overflow in strlen

Subsystem: mm/memblock

    Peng Fan <peng.fan@nxp.com>:
      memblock: check memory total_size

Subsystem: mm/mempolicy

    Eric Dumazet <edumazet@google.com>:
      mm/mempolicy: do not allow illegal MPOL_F_NUMA_BALANCING | MPOL_LOCAL in mbind()

Subsystem: mm/slub

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Fixups for slub":
      mm, slub: fix two bugs in slab_debug_trace_open()
      mm, slub: fix mismatch between reconstructed freelist depth and cnt
      mm, slub: fix potential memoryleak in kmem_cache_open()
      mm, slub: fix potential use-after-free in slab_debugfs_fops
      mm, slub: fix incorrect memcg slab count for bulk free

Subsystem: binfmt

    Lukas Bulwahn <lukas.bulwahn@gmail.com>:
      elfcore: correct reference to CONFIG_UML

Subsystem: vfs

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      vfs: check fd has read access in kernel_read_file_from_fd()

Subsystem: mm/secretmem

    Sean Christopherson <seanjc@google.com>:
      mm/secretmem: fix NULL page->mapping dereference in page_is_secretmem()

Subsystem: mm/thp

    Marek Szyprowski <m.szyprowski@samsung.com>:
      mm/thp: decrease nr_thps in file's mapping on THP split

Subsystem: misc

    Andrej Shadura <andrew.shadura@collabora.co.uk>:
      mailmap: add Andrej Shadura

 .mailmap                                 |    2 +
 fs/kernel_read_file.c                    |    2 -
 fs/ocfs2/alloc.c                         |   46 ++++++-----------------
 fs/ocfs2/super.c                         |   14 +++++--
 fs/userfaultfd.c                         |   12 ++++--
 include/linux/cpuhotplug.h               |    4 ++
 include/linux/elfcore.h                  |    2 -
 include/linux/memory.h                   |    5 ++
 include/linux/secretmem.h                |    2 -
 mm/huge_memory.c                         |    6 ++-
 mm/memblock.c                            |    2 -
 mm/mempolicy.c                           |   16 ++------
 mm/migrate.c                             |   62 ++++++++++++++++++-------------
 mm/page_ext.c                            |    4 --
 mm/slab.c                                |    4 +-
 mm/slub.c                                |   31 ++++++++++++---
 tools/testing/selftests/vm/userfaultfd.c |   23 ++++++++++-
 17 files changed, 138 insertions(+), 99 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-09-24 22:42 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-09-24 22:42 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

16 patches, based on 7d42e98182586f57f376406d033f05fe135edb75.

Subsystems affected by this patch series:

  mm/memory-failure
  mm/kasan
  mm/damon
  xtensa
  mm/shmem
  ocfs2
  scripts
  mm/tools
  lib
  mm/pagecache
  mm/debug
  sh
  mm/kasan
  mm/memory-failure
  mm/pagemap

Subsystem: mm/memory-failure

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm, hwpoison: add is_free_buddy_page() in HWPoisonHandlable()

Subsystem: mm/kasan

    Marco Elver <elver@google.com>:
      kasan: fix Kconfig check of CC_HAS_WORKING_NOSANITIZE_ADDRESS

Subsystem: mm/damon

    Adam Borowski <kilobyte@angband.pl>:
      mm/damon: don't use strnlen() with known-bogus source length

Subsystem: xtensa

    Guenter Roeck <linux@roeck-us.net>:
      xtensa: increase size of gcc stack frame check

Subsystem: mm/shmem

    Liu Yuntao <liuyuntao10@huawei.com>:
      mm/shmem.c: fix judgment error in shmem_is_huge()

Subsystem: ocfs2

    Wengang Wang <wen.gang.wang@oracle.com>:
      ocfs2: drop acl cache for directories too

Subsystem: scripts

    Miles Chen <miles.chen@mediatek.com>:
      scripts/sorttable: riscv: fix undeclared identifier 'EM_RISCV' error

Subsystem: mm/tools

    Changbin Du <changbin.du@gmail.com>:
      tools/vm/page-types: remove dependency on opt_file for idle page tracking

Subsystem: lib

    Paul Menzel <pmenzel@molgen.mpg.de>:
      lib/zlib_inflate/inffast: check config in C to avoid unused function warning

Subsystem: mm/pagecache

    Minchan Kim <minchan@kernel.org>:
      mm: fs: invalidate bh_lrus for only cold path

Subsystem: mm/debug

    Weizhao Ouyang <o451686892@gmail.com>:
      mm/debug: sync up MR_CONTIG_RANGE and MR_LONGTERM_PIN
      mm/debug: sync up latest migrate_reason to migrate_reason_names

Subsystem: sh

    Geert Uytterhoeven <geert+renesas@glider.be>:
      sh: pgtable-3level: fix cast to pointer from integer of different size

Subsystem: mm/kasan

    Nathan Chancellor <nathan@kernel.org>:
      kasan: always respect CONFIG_KASAN_STACK

Subsystem: mm/memory-failure

    Qi Zheng <zhengqi.arch@bytedance.com>:
      mm/memory_failure: fix the missing pte_unmap() call

Subsystem: mm/pagemap

    Chen Jun <chenjun102@huawei.com>:
      mm: fix uninitialized use in overcommit_policy_handler

 arch/sh/include/asm/pgtable-3level.h |    2 +-
 fs/buffer.c                          |    8 ++++++--
 fs/ocfs2/dlmglue.c                   |    3 ++-
 include/linux/buffer_head.h          |    4 ++--
 include/linux/migrate.h              |    6 +++++-
 lib/Kconfig.debug                    |    2 +-
 lib/Kconfig.kasan                    |    2 ++
 lib/zlib_inflate/inffast.c           |   13 ++++++-------
 mm/damon/dbgfs-test.h                |   16 ++++++++--------
 mm/debug.c                           |    4 +++-
 mm/memory-failure.c                  |   12 ++++++------
 mm/shmem.c                           |    4 ++--
 mm/swap.c                            |   19 ++++++++++++++++---
 mm/util.c                            |    4 ++--
 scripts/Makefile.kasan               |    3 ++-
 scripts/sorttable.c                  |    4 ++++
 tools/vm/page-types.c                |    2 +-
 17 files changed, 69 insertions(+), 39 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-09-10 17:11 ` incoming Kees Cook
@ 2021-09-10 20:13   ` Kees Cook
  0 siblings, 0 replies; 395+ messages in thread
From: Kees Cook @ 2021-09-10 20:13 UTC (permalink / raw)
  To: linux-kernel; +Cc: Linus Torvalds, Andrew Morton, linux-mm, mm-commits

On Fri, Sep 10, 2021 at 10:11:53AM -0700, Kees Cook wrote:
> On Thu, Sep 09, 2021 at 08:09:48PM -0700, Andrew Morton wrote:
> > 
> > More post linux-next material.
> > 
> > 9 patches, based on f154c806676ad7153c6e161f30c53a44855329d6.
> > 
> > Subsystems affected by this patch series:
> > 
> >   mm/slab-generic
> >   rapidio
> >   mm/debug
> > 
> > Subsystem: mm/slab-generic
> > 
> >     "Matthew Wilcox (Oracle)" <willy@infradead.org>:
> >       mm: move kvmalloc-related functions to slab.h
> > 
> > Subsystem: rapidio
> > 
> >     Kees Cook <keescook@chromium.org>:
> >       rapidio: avoid bogus __alloc_size warning
> > 
> > Subsystem: mm/debug
> > 
> >     Kees Cook <keescook@chromium.org>:
> >     Patch series "Add __alloc_size() for better bounds checking", v2:
> >       Compiler Attributes: add __alloc_size() for better bounds checking
> >       checkpatch: add __alloc_size() to known $Attribute
> >       slab: clean up function declarations
> >       slab: add __alloc_size attributes for better bounds checking
> >       mm/page_alloc: add __alloc_size attributes for better bounds checking
> >       percpu: add __alloc_size attributes for better bounds checking
> >       mm/vmalloc: add __alloc_size attributes for better bounds checking
> 
> Hi,
> 
> FYI, in overnight build testing I found yet another corner case in
> GCC's handling of the __alloc_size attribute. It's the gift that keeps
> on giving. The fix is here:
> 
> https://lore.kernel.org/lkml/20210910165851.3296624-1-keescook@chromium.org/

I'm so glad it's Friday. Here's the v2 fix... *sigh*

https://lore.kernel.org/lkml/20210910201132.3809437-1-keescook@chromium.org/

-Kees

> 
> > 
> >  Makefile                                 |   15 +++
> >  drivers/of/kexec.c                       |    1 
> >  drivers/rapidio/devices/rio_mport_cdev.c |    9 +-
> >  include/linux/compiler_attributes.h      |    6 +
> >  include/linux/gfp.h                      |    2 
> >  include/linux/mm.h                       |   34 --------
> >  include/linux/percpu.h                   |    3 
> >  include/linux/slab.h                     |  122 ++++++++++++++++++++++---------
> >  include/linux/vmalloc.h                  |   11 ++
> >  scripts/checkpatch.pl                    |    3 
> >  10 files changed, 132 insertions(+), 74 deletions(-)
> > 
> 
> -- 
> Kees Cook

-- 
Kees Cook

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-09-10  3:09 incoming Andrew Morton
@ 2021-09-10 17:11 ` Kees Cook
  2021-09-10 20:13   ` incoming Kees Cook
  0 siblings, 1 reply; 395+ messages in thread
From: Kees Cook @ 2021-09-10 17:11 UTC (permalink / raw)
  To: Linus Torvalds, Andrew Morton; +Cc: linux-mm, mm-commits

On Thu, Sep 09, 2021 at 08:09:48PM -0700, Andrew Morton wrote:
> 
> More post linux-next material.
> 
> 9 patches, based on f154c806676ad7153c6e161f30c53a44855329d6.
> 
> Subsystems affected by this patch series:
> 
>   mm/slab-generic
>   rapidio
>   mm/debug
> 
> Subsystem: mm/slab-generic
> 
>     "Matthew Wilcox (Oracle)" <willy@infradead.org>:
>       mm: move kvmalloc-related functions to slab.h
> 
> Subsystem: rapidio
> 
>     Kees Cook <keescook@chromium.org>:
>       rapidio: avoid bogus __alloc_size warning
> 
> Subsystem: mm/debug
> 
>     Kees Cook <keescook@chromium.org>:
>     Patch series "Add __alloc_size() for better bounds checking", v2:
>       Compiler Attributes: add __alloc_size() for better bounds checking
>       checkpatch: add __alloc_size() to known $Attribute
>       slab: clean up function declarations
>       slab: add __alloc_size attributes for better bounds checking
>       mm/page_alloc: add __alloc_size attributes for better bounds checking
>       percpu: add __alloc_size attributes for better bounds checking
>       mm/vmalloc: add __alloc_size attributes for better bounds checking

Hi,

FYI, in overnight build testing I found yet another corner case in
GCC's handling of the __alloc_size attribute. It's the gift that keeps
on giving. The fix is here:

https://lore.kernel.org/lkml/20210910165851.3296624-1-keescook@chromium.org/

> 
>  Makefile                                 |   15 +++
>  drivers/of/kexec.c                       |    1 
>  drivers/rapidio/devices/rio_mport_cdev.c |    9 +-
>  include/linux/compiler_attributes.h      |    6 +
>  include/linux/gfp.h                      |    2 
>  include/linux/mm.h                       |   34 --------
>  include/linux/percpu.h                   |    3 
>  include/linux/slab.h                     |  122 ++++++++++++++++++++++---------
>  include/linux/vmalloc.h                  |   11 ++
>  scripts/checkpatch.pl                    |    3 
>  10 files changed, 132 insertions(+), 74 deletions(-)
> 

-- 
Kees Cook

^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-09-10  3:09 Andrew Morton
  2021-09-10 17:11 ` incoming Kees Cook
  0 siblings, 1 reply; 395+ messages in thread
From: Andrew Morton @ 2021-09-10  3:09 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits


More post linux-next material.

9 patches, based on f154c806676ad7153c6e161f30c53a44855329d6.

Subsystems affected by this patch series:

  mm/slab-generic
  rapidio
  mm/debug

Subsystem: mm/slab-generic

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm: move kvmalloc-related functions to slab.h

Subsystem: rapidio

    Kees Cook <keescook@chromium.org>:
      rapidio: avoid bogus __alloc_size warning

Subsystem: mm/debug

    Kees Cook <keescook@chromium.org>:
    Patch series "Add __alloc_size() for better bounds checking", v2:
      Compiler Attributes: add __alloc_size() for better bounds checking
      checkpatch: add __alloc_size() to known $Attribute
      slab: clean up function declarations
      slab: add __alloc_size attributes for better bounds checking
      mm/page_alloc: add __alloc_size attributes for better bounds checking
      percpu: add __alloc_size attributes for better bounds checking
      mm/vmalloc: add __alloc_size attributes for better bounds checking

 Makefile                                 |   15 +++
 drivers/of/kexec.c                       |    1 
 drivers/rapidio/devices/rio_mport_cdev.c |    9 +-
 include/linux/compiler_attributes.h      |    6 +
 include/linux/gfp.h                      |    2 
 include/linux/mm.h                       |   34 --------
 include/linux/percpu.h                   |    3 
 include/linux/slab.h                     |  122 ++++++++++++++++++++++---------
 include/linux/vmalloc.h                  |   11 ++
 scripts/checkpatch.pl                    |    3 
 10 files changed, 132 insertions(+), 74 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-09-09  1:08 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-09-09  1:08 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm


A bunch of hotfixes, mostly cc:stable.


8 patches, based on 2d338201d5311bcd79d42f66df4cecbcbc5f4f2c.

Subsystems affected by this patch series:

  mm/hmm
  mm/hugetlb
  mm/vmscan
  mm/pagealloc
  mm/pagemap
  mm/kmemleak
  mm/mempolicy
  mm/memblock

Subsystem: mm/hmm

    Li Zhijian <lizhijian@cn.fujitsu.com>:
      mm/hmm: bypass devmap pte when all pfn requested flags are fulfilled

Subsystem: mm/hugetlb

    Liu Zixian <liuzixian4@huawei.com>:
      mm/hugetlb: initialize hugetlb_usage in mm_init

Subsystem: mm/vmscan

    Rik van Riel <riel@surriel.com>:
      mm,vmscan: fix divide by zero in get_scan_count

Subsystem: mm/pagealloc

    Miaohe Lin <linmiaohe@huawei.com>:
      mm/page_alloc.c: avoid accessing uninitialized pcp page migratetype

Subsystem: mm/pagemap

    Liam Howlett <liam.howlett@oracle.com>:
      mmap_lock: change trace and locking order

Subsystem: mm/kmemleak

    Naohiro Aota <naohiro.aota@wdc.com>:
      mm/kmemleak: allow __GFP_NOLOCKDEP passed to kmemleak's gfp

Subsystem: mm/mempolicy

    yanghui <yanghui.def@bytedance.com>:
      mm/mempolicy: fix a race between offset_il_node and mpol_rebind_task

Subsystem: mm/memblock

    Mike Rapoport <rppt@linux.ibm.com>:
      nds32/setup: remove unused memblock_region variable in setup_memory()

 arch/nds32/kernel/setup.c |    1 -
 include/linux/hugetlb.h   |    9 +++++++++
 include/linux/mmap_lock.h |    8 ++++----
 kernel/fork.c             |    1 +
 mm/hmm.c                  |    5 ++++-
 mm/kmemleak.c             |    3 ++-
 mm/mempolicy.c            |   17 +++++++++++++----
 mm/page_alloc.c           |    4 +++-
 mm/vmscan.c               |    2 +-
 9 files changed, 37 insertions(+), 13 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-09-08 22:17 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-09-08 22:17 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits


This is the post-linux-next material, so it is based upon latest
upstream to catch the now-merged dependencies.

10 patches, based on 2d338201d5311bcd79d42f66df4cecbcbc5f4f2c.

Subsystems affected by this patch series:

  mm/vmstat
  mm/migration
  compat

Subsystem: mm/vmstat

    Ingo Molnar <mingo@elte.hu>:
      mm/vmstat: protect per cpu variables with preempt disable on RT

Subsystem: mm/migration

    Baolin Wang <baolin.wang@linux.alibaba.com>:
      mm: migrate: introduce a local variable to get the number of pages
      mm: migrate: fix the incorrect function name in comments
      mm: migrate: change to use bool type for 'page_was_mapped'

Subsystem: compat

    Arnd Bergmann <arnd@arndb.de>:
    Patch series "compat: remove compat_alloc_user_space", v5:
      kexec: move locking into do_kexec_load
      kexec: avoid compat_alloc_user_space
      mm: simplify compat_sys_move_pages
      mm: simplify compat numa syscalls
      compat: remove some compat entry points
      arch: remove compat_alloc_user_space

 arch/arm64/include/asm/compat.h           |    5 
 arch/arm64/include/asm/uaccess.h          |   11 -
 arch/arm64/include/asm/unistd32.h         |   10 -
 arch/arm64/lib/Makefile                   |    2 
 arch/arm64/lib/copy_in_user.S             |   77 ----------
 arch/mips/cavium-octeon/octeon-memcpy.S   |    2 
 arch/mips/include/asm/compat.h            |    8 -
 arch/mips/include/asm/uaccess.h           |   26 ---
 arch/mips/kernel/syscalls/syscall_n32.tbl |   10 -
 arch/mips/kernel/syscalls/syscall_o32.tbl |   10 -
 arch/mips/lib/memcpy.S                    |   11 -
 arch/parisc/include/asm/compat.h          |    6 
 arch/parisc/include/asm/uaccess.h         |    2 
 arch/parisc/kernel/syscalls/syscall.tbl   |    8 -
 arch/parisc/lib/memcpy.c                  |    9 -
 arch/powerpc/include/asm/compat.h         |   16 --
 arch/powerpc/kernel/syscalls/syscall.tbl  |   10 -
 arch/s390/include/asm/compat.h            |   10 -
 arch/s390/include/asm/uaccess.h           |    3 
 arch/s390/kernel/syscalls/syscall.tbl     |   10 -
 arch/s390/lib/uaccess.c                   |   63 --------
 arch/sparc/include/asm/compat.h           |   19 --
 arch/sparc/kernel/process_64.c            |    2 
 arch/sparc/kernel/signal32.c              |   12 -
 arch/sparc/kernel/signal_64.c             |    8 -
 arch/sparc/kernel/syscalls/syscall.tbl    |   10 -
 arch/x86/entry/syscalls/syscall_32.tbl    |    4 
 arch/x86/entry/syscalls/syscall_64.tbl    |    2 
 arch/x86/include/asm/compat.h             |   13 -
 arch/x86/include/asm/uaccess_64.h         |    7 
 include/linux/compat.h                    |   39 +----
 include/linux/uaccess.h                   |   10 -
 include/uapi/asm-generic/unistd.h         |   10 -
 kernel/compat.c                           |   21 --
 kernel/kexec.c                            |  105 +++++---------
 kernel/sys_ni.c                           |    5 
 mm/mempolicy.c                            |  213 +++++++-----------------------
 mm/migrate.c                              |   69 +++++----
 mm/vmstat.c                               |   48 ++++++
 39 files changed, 243 insertions(+), 663 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-09-08  2:52 incoming Andrew Morton
@ 2021-09-08  8:57 ` Vlastimil Babka
  0 siblings, 0 replies; 395+ messages in thread
From: Vlastimil Babka @ 2021-09-08  8:57 UTC (permalink / raw)
  To: Andrew Morton, Linus Torvalds
  Cc: linux-mm, mm-commits, Mike Galbraith, Mel Gorman

On 9/8/21 04:52, Andrew Morton wrote:
> Subsystem: mm/slub
> 
>     Vlastimil Babka <vbabka@suse.cz>:
>     Patch series "SLUB: reduce irq disabled scope and make it RT compatible", v6:
>       mm, slub: don't call flush_all() from slab_debug_trace_open()
>       mm, slub: allocate private object map for debugfs listings
>       mm, slub: allocate private object map for validate_slab_cache()
>       mm, slub: don't disable irq for debug_check_no_locks_freed()
>       mm, slub: remove redundant unfreeze_partials() from put_cpu_partial()
>       mm, slub: extract get_partial() from new_slab_objects()
>       mm, slub: dissolve new_slab_objects() into ___slab_alloc()
>       mm, slub: return slab page from get_partial() and set c->page afterwards
>       mm, slub: restructure new page checks in ___slab_alloc()
>       mm, slub: simplify kmem_cache_cpu and tid setup
>       mm, slub: move disabling/enabling irqs to ___slab_alloc()
>       mm, slub: do initial checks in ___slab_alloc() with irqs enabled
>       mm, slub: move disabling irqs closer to get_partial() in ___slab_alloc()
>       mm, slub: restore irqs around calling new_slab()
>       mm, slub: validate slab from partial list or page allocator before making it cpu slab
>       mm, slub: check new pages with restored irqs
>       mm, slub: stop disabling irqs around get_partial()
>       mm, slub: move reset of c->page and freelist out of deactivate_slab()
>       mm, slub: make locking in deactivate_slab() irq-safe
>       mm, slub: call deactivate_slab() without disabling irqs
>       mm, slub: move irq control into unfreeze_partials()
>       mm, slub: discard slabs in unfreeze_partials() without irqs disabled
>       mm, slub: detach whole partial list at once in unfreeze_partials()
>       mm, slub: separate detaching of partial list in unfreeze_partials() from unfreezing
>       mm, slub: only disable irq with spin_lock in __unfreeze_partials()
>       mm, slub: don't disable irqs in slub_cpu_dead()
>       mm, slab: split out the cpu offline variant of flush_slab()
> 
>     Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
>       mm: slub: move flush_cpu_slab() invocations __free_slab() invocations out of IRQ context
>       mm: slub: make object_map_lock a raw_spinlock_t
> 
>     Vlastimil Babka <vbabka@suse.cz>:
>       mm, slub: make slab_lock() disable irqs with PREEMPT_RT
>       mm, slub: protect put_cpu_partial() with disabled irqs instead of cmpxchg
>       mm, slub: use migrate_disable() on PREEMPT_RT
>       mm, slub: convert kmem_cpu_slab protection to local_lock

For my own piece of mind, I've checked that this part (patches 1 to 33)
are identical to the v6 posting [1] and git version [2] that Mel and
Mike tested (replies to [1]).

[1] https://lore.kernel.org/all/20210904105003.11688-1-vbabka@suse.cz/
[2] git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git
tags/mm-slub-5.15-rc1

^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-09-08  2:52 Andrew Morton
  2021-09-08  8:57 ` incoming Vlastimil Babka
  0 siblings, 1 reply; 395+ messages in thread
From: Andrew Morton @ 2021-09-08  2:52 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

147 patches, based on 7d2a07b769330c34b4deabeed939325c77a7ec2f.

Subsystems affected by this patch series:

  mm/slub
  mm/memory-hotplug
  mm/rmap
  mm/ioremap
  mm/highmem
  mm/cleanups
  mm/secretmem
  mm/kfence
  mm/damon
  alpha
  percpu
  procfs
  misc
  core-kernel
  MAINTAINERS
  lib
  bitops
  checkpatch
  epoll
  init
  nilfs2
  coredump
  fork
  pids
  criu
  kconfig
  selftests
  ipc
  mm/vmscan
  scripts

Subsystem: mm/slub

    Vlastimil Babka <vbabka@suse.cz>:
    Patch series "SLUB: reduce irq disabled scope and make it RT compatible", v6:
      mm, slub: don't call flush_all() from slab_debug_trace_open()
      mm, slub: allocate private object map for debugfs listings
      mm, slub: allocate private object map for validate_slab_cache()
      mm, slub: don't disable irq for debug_check_no_locks_freed()
      mm, slub: remove redundant unfreeze_partials() from put_cpu_partial()
      mm, slub: extract get_partial() from new_slab_objects()
      mm, slub: dissolve new_slab_objects() into ___slab_alloc()
      mm, slub: return slab page from get_partial() and set c->page afterwards
      mm, slub: restructure new page checks in ___slab_alloc()
      mm, slub: simplify kmem_cache_cpu and tid setup
      mm, slub: move disabling/enabling irqs to ___slab_alloc()
      mm, slub: do initial checks in ___slab_alloc() with irqs enabled
      mm, slub: move disabling irqs closer to get_partial() in ___slab_alloc()
      mm, slub: restore irqs around calling new_slab()
      mm, slub: validate slab from partial list or page allocator before making it cpu slab
      mm, slub: check new pages with restored irqs
      mm, slub: stop disabling irqs around get_partial()
      mm, slub: move reset of c->page and freelist out of deactivate_slab()
      mm, slub: make locking in deactivate_slab() irq-safe
      mm, slub: call deactivate_slab() without disabling irqs
      mm, slub: move irq control into unfreeze_partials()
      mm, slub: discard slabs in unfreeze_partials() without irqs disabled
      mm, slub: detach whole partial list at once in unfreeze_partials()
      mm, slub: separate detaching of partial list in unfreeze_partials() from unfreezing
      mm, slub: only disable irq with spin_lock in __unfreeze_partials()
      mm, slub: don't disable irqs in slub_cpu_dead()
      mm, slab: split out the cpu offline variant of flush_slab()

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
      mm: slub: move flush_cpu_slab() invocations __free_slab() invocations out of IRQ context
      mm: slub: make object_map_lock a raw_spinlock_t

    Vlastimil Babka <vbabka@suse.cz>:
      mm, slub: make slab_lock() disable irqs with PREEMPT_RT
      mm, slub: protect put_cpu_partial() with disabled irqs instead of cmpxchg
      mm, slub: use migrate_disable() on PREEMPT_RT
      mm, slub: convert kmem_cpu_slab protection to local_lock

Subsystem: mm/memory-hotplug

    David Hildenbrand <david@redhat.com>:
    Patch series "memory-hotplug.rst: complete admin-guide overhaul", v3:
      memory-hotplug.rst: remove locking details from admin-guide
      memory-hotplug.rst: complete admin-guide overhaul

    Mike Rapoport <rppt@linux.ibm.com>:
    Patch series "mm: remove pfn_valid_within() and CONFIG_HOLES_IN_ZONE":
      mm: remove pfn_valid_within() and CONFIG_HOLES_IN_ZONE
      mm: memory_hotplug: cleanup after removal of pfn_valid_within()

    David Hildenbrand <david@redhat.com>:
    Patch series "mm/memory_hotplug: preparatory patches for new online policy and memory":
      mm/memory_hotplug: use "unsigned long" for PFN in zone_for_pfn_range()
      mm/memory_hotplug: remove nid parameter from arch_remove_memory()
      mm/memory_hotplug: remove nid parameter from remove_memory() and friends
      ACPI: memhotplug: memory resources cannot be enabled yet
    Patch series "mm/memory_hotplug: "auto-movable" online policy and memory groups", v3:
      mm: track present early pages per zone
      mm/memory_hotplug: introduce "auto-movable" online policy
      drivers/base/memory: introduce "memory groups" to logically group memory blocks
      mm/memory_hotplug: track present pages in memory groups
      ACPI: memhotplug: use a single static memory group for a single memory device
      dax/kmem: use a single static memory group for a single probed unit
      virtio-mem: use a single dynamic memory group for a single virtio-mem device
      mm/memory_hotplug: memory group aware "auto-movable" online policy
      mm/memory_hotplug: improved dynamic memory group aware "auto-movable" online policy

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanup and fixups for memory hotplug":
      mm/memory_hotplug: use helper zone_is_zone_device() to simplify the code

Subsystem: mm/rmap

    Muchun Song <songmuchun@bytedance.com>:
      mm: remove redundant compound_head() calling

Subsystem: mm/ioremap

    Christoph Hellwig <hch@lst.de>:
      riscv: only select GENERIC_IOREMAP if MMU support is enabled
    Patch series "small ioremap cleanups":
      mm: move ioremap_page_range to vmalloc.c
      mm: don't allow executable ioremap mappings

    Weizhao Ouyang <o451686892@gmail.com>:
      mm/early_ioremap.c: remove redundant early_ioremap_shutdown()

Subsystem: mm/highmem

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
      highmem: don't disable preemption on RT in kmap_atomic()

Subsystem: mm/cleanups

    Changbin Du <changbin.du@gmail.com>:
      mm: in_irq() cleanup

    Muchun Song <songmuchun@bytedance.com>:
      mm: introduce PAGEFLAGS_MASK to replace ((1UL << NR_PAGEFLAGS) - 1)

Subsystem: mm/secretmem

    Jordy Zomer <jordy@jordyzomer.github.io>:
      mm/secretmem: use refcount_t instead of atomic_t

Subsystem: mm/kfence

    Marco Elver <elver@google.com>:
      kfence: show cpu and timestamp in alloc/free info
      kfence: test: fail fast if disabled at boot

Subsystem: mm/damon

    SeongJae Park <sjpark@amazon.de>:
    Patch series "Introduce Data Access MONitor (DAMON)", v34:
      mm: introduce Data Access MONitor (DAMON)
      mm/damon/core: implement region-based sampling
      mm/damon: adaptively adjust regions
      mm/idle_page_tracking: make PG_idle reusable
      mm/damon: implement primitives for the virtual memory address spaces
      mm/damon: add a tracepoint
      mm/damon: implement a debugfs-based user space interface
      mm/damon/dbgfs: export kdamond pid to the user space
      mm/damon/dbgfs: support multiple contexts
      Documentation: add documents for DAMON
      mm/damon: add kunit tests
      mm/damon: add user space selftests
      MAINTAINERS: update for DAMON

Subsystem: alpha

    Randy Dunlap <rdunlap@infradead.org>:
      alpha: agp: make empty macros use do-while-0 style
      alpha: pci-sysfs: fix all kernel-doc warnings

Subsystem: percpu

    Greg Kroah-Hartman <gregkh@linuxfoundation.org>:
      percpu: remove export of pcpu_base_addr

Subsystem: procfs

    Feng Zhou <zhoufeng.zf@bytedance.com>:
      fs/proc/kcore.c: add mmap interface

    Christoph Hellwig <hch@lst.de>:
      proc: stop using seq_get_buf in proc_task_name

    Ohhoon Kwon <ohoono.kwon@samsung.com>:
      connector: send event on write to /proc/[pid]/comm

Subsystem: misc

    Colin Ian King <colin.king@canonical.com>:
      arch: Kconfig: fix spelling mistake "seperate" -> "separate"

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
      include/linux/once.h: fix trivia typo Not -> Note

    Daniel Lezcano <daniel.lezcano@linaro.org>:
    Patch series "Add Hz macros", v3:
      units: change from 'L' to 'UL'
      units: add the HZ macros
      thermal/drivers/devfreq_cooling: use HZ macros
      devfreq: use HZ macros
      iio/drivers/as73211: use HZ macros
      hwmon/drivers/mr75203: use HZ macros
      iio/drivers/hid-sensor: use HZ macros
      i2c/drivers/ov02q10: use HZ macros
      mtd/drivers/nand: use HZ macros
      phy/drivers/stm32: use HZ macros

Subsystem: core-kernel

    Yang Yang <yang.yang29@zte.com.cn>:
      kernel/acct.c: use dedicated helper to access rlimit values

    Pavel Skripkin <paskripkin@gmail.com>:
      profiling: fix shift-out-of-bounds bugs

Subsystem: MAINTAINERS

    Nathan Chancellor <nathan@kernel.org>:
      MAINTAINERS: update ClangBuiltLinux mailing list
      Documentation/llvm: update mailing list
      Documentation/llvm: update IRC location

Subsystem: lib

    Geert Uytterhoeven <geert@linux-m68k.org>:
    Patch series "math: RATIONAL and RATIONAL_KUNIT_TEST improvements":
      math: make RATIONAL tristate
      math: RATIONAL_KUNIT_TEST should depend on RATIONAL instead of selecting it

    Matteo Croce <mcroce@microsoft.com>:
    Patch series "lib/string: optimized mem* functions", v2:
      lib/string: optimized memcpy
      lib/string: optimized memmove
      lib/string: optimized memset

    Daniel Latypov <dlatypov@google.com>:
      lib/test: convert test_sort.c to use KUnit

    Randy Dunlap <rdunlap@infradead.org>:
      lib/dump_stack: correct kernel-doc notation
      lib/iov_iter.c: fix kernel-doc warnings

Subsystem: bitops

    Yury Norov <yury.norov@gmail.com>:
    Patch series "Resend bitmap patches":
      bitops: protect find_first_{,zero}_bit properly
      bitops: move find_bit_*_le functions from le.h to find.h
      include: move find.h from asm_generic to linux
      arch: remove GENERIC_FIND_FIRST_BIT entirely
      lib: add find_first_and_bit()
      cpumask: use find_first_and_bit()
      all: replace find_next{,_zero}_bit with find_first{,_zero}_bit where appropriate
      tools: sync tools/bitmap with mother linux
      cpumask: replace cpumask_next_* with cpumask_first_* where appropriate
      include/linux: move for_each_bit() macros from bitops.h to find.h
      find: micro-optimize for_each_{set,clear}_bit()
      bitops: replace for_each_*_bit_from() with for_each_*_bit() where appropriate

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
      tools: rename bitmap_alloc() to bitmap_zalloc()

    Yury Norov <yury.norov@gmail.com>:
      mm/percpu: micro-optimize pcpu_is_populated()
      bitmap: unify find_bit operations
      lib: bitmap: add performance test for bitmap_print_to_pagebuf
      vsprintf: rework bitmap_list_string

Subsystem: checkpatch

    Joe Perches <joe@perches.com>:
      checkpatch: support wide strings

    Mimi Zohar <zohar@linux.ibm.com>:
      checkpatch: make email address check case insensitive

    Joe Perches <joe@perches.com>:
      checkpatch: improve GIT_COMMIT_ID test

Subsystem: epoll

    Nicholas Piggin <npiggin@gmail.com>:
      fs/epoll: use a per-cpu counter for user's watches count

Subsystem: init

    Rasmus Villemoes <linux@rasmusvillemoes.dk>:
      init: move usermodehelper_enable() to populate_rootfs()

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      trap: cleanup trap_init()

Subsystem: nilfs2

    Nanyong Sun <sunnanyong@huawei.com>:
    Patch series "nilfs2: fix incorrect usage of kobject":
      nilfs2: fix memory leak in nilfs_sysfs_create_device_group
      nilfs2: fix NULL pointer in nilfs_##name##_attr_release
      nilfs2: fix memory leak in nilfs_sysfs_create_##name##_group
      nilfs2: fix memory leak in nilfs_sysfs_delete_##name##_group
      nilfs2: fix memory leak in nilfs_sysfs_create_snapshot_group
      nilfs2: fix memory leak in nilfs_sysfs_delete_snapshot_group

    Zhen Lei <thunder.leizhen@huawei.com>:
      nilfs2: use refcount_dec_and_lock() to fix potential UAF

Subsystem: coredump

    David Oberhollenzer <david.oberhollenzer@sigma-star.at>:
      fs/coredump.c: log if a core dump is aborted due to changed file permissions

    QiuXi <qiuxi1@huawei.com>:
      coredump: fix memleak in dump_vma_snapshot()

Subsystem: fork

    Christoph Hellwig <hch@lst.de>:
      kernel/fork.c: unexport get_{mm,task}_exe_file

Subsystem: pids

    Takahiro Itazuri <itazur@amazon.com>:
      pid: cleanup the stale comment mentioning pidmap_init().

Subsystem: criu

    Cyrill Gorcunov <gorcunov@gmail.com>:
      prctl: allow to setup brk for et_dyn executables

Subsystem: kconfig

    Zenghui Yu <yuzenghui@huawei.com>:
      configs: remove the obsolete CONFIG_INPUT_POLLDEV

    Lukas Bulwahn <lukas.bulwahn@gmail.com>:
      Kconfig.debug: drop selecting non-existing HARDLOCKUP_DETECTOR_ARCH

Subsystem: selftests

    Greg Thelen <gthelen@google.com>:
      selftests/memfd: remove unused variable

Subsystem: ipc

    Rafael Aquini <aquini@redhat.com>:
      ipc: replace costly bailout check in sysvipc_find_ipc()

Subsystem: mm/vmscan

    Randy Dunlap <rdunlap@infradead.org>:
      mm/workingset: correct kernel-doc notations

Subsystem: scripts

    Randy Dunlap <rdunlap@infradead.org>:
      scripts: check_extable: fix typo in user error message

 a/Documentation/admin-guide/mm/damon/index.rst            |   15 
 a/Documentation/admin-guide/mm/damon/start.rst            |  114 +
 a/Documentation/admin-guide/mm/damon/usage.rst            |  112 +
 a/Documentation/admin-guide/mm/index.rst                  |    1 
 a/Documentation/admin-guide/mm/memory-hotplug.rst         |  842 ++++++-----
 a/Documentation/dev-tools/kfence.rst                      |   98 -
 a/Documentation/kbuild/llvm.rst                           |    5 
 a/Documentation/vm/damon/api.rst                          |   20 
 a/Documentation/vm/damon/design.rst                       |  166 ++
 a/Documentation/vm/damon/faq.rst                          |   51 
 a/Documentation/vm/damon/index.rst                        |   30 
 a/Documentation/vm/index.rst                              |    1 
 a/MAINTAINERS                                             |   17 
 a/arch/Kconfig                                            |    2 
 a/arch/alpha/include/asm/agp.h                            |    4 
 a/arch/alpha/include/asm/bitops.h                         |    2 
 a/arch/alpha/kernel/pci-sysfs.c                           |   12 
 a/arch/arc/Kconfig                                        |    1 
 a/arch/arc/include/asm/bitops.h                           |    1 
 a/arch/arc/kernel/traps.c                                 |    5 
 a/arch/arm/configs/dove_defconfig                         |    1 
 a/arch/arm/configs/pxa_defconfig                          |    1 
 a/arch/arm/include/asm/bitops.h                           |    1 
 a/arch/arm/kernel/traps.c                                 |    5 
 a/arch/arm64/Kconfig                                      |    1 
 a/arch/arm64/include/asm/bitops.h                         |    1 
 a/arch/arm64/mm/mmu.c                                     |    3 
 a/arch/csky/include/asm/bitops.h                          |    1 
 a/arch/h8300/include/asm/bitops.h                         |    1 
 a/arch/h8300/kernel/traps.c                               |    4 
 a/arch/hexagon/include/asm/bitops.h                       |    1 
 a/arch/hexagon/kernel/traps.c                             |    4 
 a/arch/ia64/include/asm/bitops.h                          |    2 
 a/arch/ia64/mm/init.c                                     |    3 
 a/arch/m68k/include/asm/bitops.h                          |    2 
 a/arch/mips/Kconfig                                       |    1 
 a/arch/mips/configs/lemote2f_defconfig                    |    1 
 a/arch/mips/configs/pic32mzda_defconfig                   |    1 
 a/arch/mips/configs/rt305x_defconfig                      |    1 
 a/arch/mips/configs/xway_defconfig                        |    1 
 a/arch/mips/include/asm/bitops.h                          |    1 
 a/arch/nds32/kernel/traps.c                               |    5 
 a/arch/nios2/kernel/traps.c                               |    5 
 a/arch/openrisc/include/asm/bitops.h                      |    1 
 a/arch/openrisc/kernel/traps.c                            |    5 
 a/arch/parisc/configs/generic-32bit_defconfig             |    1 
 a/arch/parisc/include/asm/bitops.h                        |    2 
 a/arch/parisc/kernel/traps.c                              |    4 
 a/arch/powerpc/include/asm/bitops.h                       |    2 
 a/arch/powerpc/include/asm/cputhreads.h                   |    2 
 a/arch/powerpc/kernel/traps.c                             |    5 
 a/arch/powerpc/mm/mem.c                                   |    3 
 a/arch/powerpc/platforms/pasemi/dma_lib.c                 |    4 
 a/arch/powerpc/platforms/pseries/hotplug-memory.c         |    9 
 a/arch/riscv/Kconfig                                      |    2 
 a/arch/riscv/include/asm/bitops.h                         |    1 
 a/arch/riscv/kernel/traps.c                               |    5 
 a/arch/s390/Kconfig                                       |    1 
 a/arch/s390/include/asm/bitops.h                          |    1 
 a/arch/s390/kvm/kvm-s390.c                                |    2 
 a/arch/s390/mm/init.c                                     |    3 
 a/arch/sh/include/asm/bitops.h                            |    1 
 a/arch/sh/mm/init.c                                       |    3 
 a/arch/sparc/include/asm/bitops_32.h                      |    1 
 a/arch/sparc/include/asm/bitops_64.h                      |    2 
 a/arch/um/kernel/trap.c                                   |    4 
 a/arch/x86/Kconfig                                        |    1 
 a/arch/x86/configs/i386_defconfig                         |    1 
 a/arch/x86/configs/x86_64_defconfig                       |    1 
 a/arch/x86/include/asm/bitops.h                           |    2 
 a/arch/x86/kernel/apic/vector.c                           |    4 
 a/arch/x86/mm/init_32.c                                   |    3 
 a/arch/x86/mm/init_64.c                                   |    3 
 a/arch/x86/um/Kconfig                                     |    1 
 a/arch/xtensa/include/asm/bitops.h                        |    1 
 a/block/blk-mq.c                                          |    2 
 a/drivers/acpi/acpi_memhotplug.c                          |   46 
 a/drivers/base/memory.c                                   |  231 ++-
 a/drivers/base/node.c                                     |    2 
 a/drivers/block/rnbd/rnbd-clt.c                           |    2 
 a/drivers/dax/kmem.c                                      |   43 
 a/drivers/devfreq/devfreq.c                               |    2 
 a/drivers/dma/ti/edma.c                                   |    2 
 a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c                   |    4 
 a/drivers/hwmon/ltc2992.c                                 |    3 
 a/drivers/hwmon/mr75203.c                                 |    2 
 a/drivers/iio/adc/ad7124.c                                |    2 
 a/drivers/iio/common/hid-sensors/hid-sensor-attributes.c  |    3 
 a/drivers/iio/light/as73211.c                             |    3 
 a/drivers/infiniband/hw/irdma/hw.c                        |   16 
 a/drivers/media/cec/core/cec-core.c                       |    2 
 a/drivers/media/i2c/ov02a10.c                             |    2 
 a/drivers/media/mc/mc-devnode.c                           |    2 
 a/drivers/mmc/host/renesas_sdhi_core.c                    |    2 
 a/drivers/mtd/nand/raw/intel-nand-controller.c            |    2 
 a/drivers/net/virtio_net.c                                |    2 
 a/drivers/pci/controller/dwc/pci-dra7xx.c                 |    2 
 a/drivers/phy/st/phy-stm32-usbphyc.c                      |    2 
 a/drivers/scsi/lpfc/lpfc_sli.c                            |   10 
 a/drivers/soc/fsl/qbman/bman_portal.c                     |    2 
 a/drivers/soc/fsl/qbman/qman_portal.c                     |    2 
 a/drivers/soc/ti/k3-ringacc.c                             |    4 
 a/drivers/thermal/devfreq_cooling.c                       |    2 
 a/drivers/tty/n_tty.c                                     |    2 
 a/drivers/virt/acrn/ioreq.c                               |    3 
 a/drivers/virtio/virtio_mem.c                             |   26 
 a/fs/coredump.c                                           |   15 
 a/fs/eventpoll.c                                          |   18 
 a/fs/f2fs/segment.c                                       |    8 
 a/fs/nilfs2/sysfs.c                                       |   26 
 a/fs/nilfs2/the_nilfs.c                                   |    9 
 a/fs/ocfs2/cluster/heartbeat.c                            |    2 
 a/fs/ocfs2/dlm/dlmdomain.c                                |    4 
 a/fs/ocfs2/dlm/dlmmaster.c                                |   18 
 a/fs/ocfs2/dlm/dlmrecovery.c                              |    2 
 a/fs/ocfs2/dlm/dlmthread.c                                |    2 
 a/fs/proc/array.c                                         |   18 
 a/fs/proc/base.c                                          |    5 
 a/fs/proc/kcore.c                                         |   73 
 a/include/asm-generic/bitops.h                            |    1 
 a/include/asm-generic/bitops/find.h                       |  198 --
 a/include/asm-generic/bitops/le.h                         |   64 
 a/include/asm-generic/early_ioremap.h                     |    6 
 a/include/linux/bitmap.h                                  |   34 
 a/include/linux/bitops.h                                  |   34 
 a/include/linux/cpumask.h                                 |   46 
 a/include/linux/damon.h                                   |  290 +++
 a/include/linux/find.h                                    |  134 +
 a/include/linux/highmem-internal.h                        |   27 
 a/include/linux/memory.h                                  |   55 
 a/include/linux/memory_hotplug.h                          |   40 
 a/include/linux/mmzone.h                                  |   19 
 a/include/linux/once.h                                    |    2 
 a/include/linux/page-flags.h                              |   17 
 a/include/linux/page_ext.h                                |    2 
 a/include/linux/page_idle.h                               |    6 
 a/include/linux/pagemap.h                                 |    7 
 a/include/linux/sched/user.h                              |    3 
 a/include/linux/slub_def.h                                |    6 
 a/include/linux/threads.h                                 |    2 
 a/include/linux/units.h                                   |   10 
 a/include/linux/vmalloc.h                                 |    3 
 a/include/trace/events/damon.h                            |   43 
 a/include/trace/events/mmflags.h                          |    2 
 a/include/trace/events/page_ref.h                         |    4 
 a/init/initramfs.c                                        |    2 
 a/init/main.c                                             |    3 
 a/init/noinitramfs.c                                      |    2 
 a/ipc/util.c                                              |   16 
 a/kernel/acct.c                                           |    2 
 a/kernel/fork.c                                           |    2 
 a/kernel/profile.c                                        |   21 
 a/kernel/sys.c                                            |    7 
 a/kernel/time/clocksource.c                               |    4 
 a/kernel/user.c                                           |   25 
 a/lib/Kconfig                                             |    3 
 a/lib/Kconfig.debug                                       |    9 
 a/lib/dump_stack.c                                        |    3 
 a/lib/find_bit.c                                          |   21 
 a/lib/find_bit_benchmark.c                                |   21 
 a/lib/genalloc.c                                          |    2 
 a/lib/iov_iter.c                                          |    8 
 a/lib/math/Kconfig                                        |    2 
 a/lib/math/rational.c                                     |    3 
 a/lib/string.c                                            |  130 +
 a/lib/test_bitmap.c                                       |   37 
 a/lib/test_printf.c                                       |    2 
 a/lib/test_sort.c                                         |   40 
 a/lib/vsprintf.c                                          |   26 
 a/mm/Kconfig                                              |   15 
 a/mm/Makefile                                             |    4 
 a/mm/compaction.c                                         |   20 
 a/mm/damon/Kconfig                                        |   68 
 a/mm/damon/Makefile                                       |    5 
 a/mm/damon/core-test.h                                    |  253 +++
 a/mm/damon/core.c                                         |  748 ++++++++++
 a/mm/damon/dbgfs-test.h                                   |  126 +
 a/mm/damon/dbgfs.c                                        |  631 ++++++++
 a/mm/damon/vaddr-test.h                                   |  329 ++++
 a/mm/damon/vaddr.c                                        |  672 +++++++++
 a/mm/early_ioremap.c                                      |    5 
 a/mm/highmem.c                                            |    2 
 a/mm/ioremap.c                                            |   25 
 a/mm/kfence/core.c                                        |    3 
 a/mm/kfence/kfence.h                                      |    2 
 a/mm/kfence/kfence_test.c                                 |    3 
 a/mm/kfence/report.c                                      |   19 
 a/mm/kmemleak.c                                           |    2 
 a/mm/memory_hotplug.c                                     |  396 ++++-
 a/mm/memremap.c                                           |    5 
 a/mm/page_alloc.c                                         |   27 
 a/mm/page_ext.c                                           |   12 
 a/mm/page_idle.c                                          |   10 
 a/mm/page_isolation.c                                     |    7 
 a/mm/page_owner.c                                         |   14 
 a/mm/percpu.c                                             |   36 
 a/mm/rmap.c                                               |    6 
 a/mm/secretmem.c                                          |    9 
 a/mm/slab_common.c                                        |    2 
 a/mm/slub.c                                               | 1023 +++++++++-----
 a/mm/vmalloc.c                                            |   24 
 a/mm/workingset.c                                         |    2 
 a/net/ncsi/ncsi-manage.c                                  |    4 
 a/scripts/check_extable.sh                                |    2 
 a/scripts/checkpatch.pl                                   |   93 -
 a/tools/include/linux/bitmap.h                            |    4 
 a/tools/perf/bench/find-bit-bench.c                       |    2 
 a/tools/perf/builtin-c2c.c                                |    6 
 a/tools/perf/builtin-record.c                             |    2 
 a/tools/perf/tests/bitmap.c                               |    2 
 a/tools/perf/tests/mem2node.c                             |    2 
 a/tools/perf/util/affinity.c                              |    4 
 a/tools/perf/util/header.c                                |    4 
 a/tools/perf/util/metricgroup.c                           |    2 
 a/tools/perf/util/mmap.c                                  |    4 
 a/tools/testing/selftests/damon/Makefile                  |    7 
 a/tools/testing/selftests/damon/_chk_dependency.sh        |   28 
 a/tools/testing/selftests/damon/debugfs_attrs.sh          |   75 +
 a/tools/testing/selftests/kvm/dirty_log_perf_test.c       |    2 
 a/tools/testing/selftests/kvm/dirty_log_test.c            |    4 
 a/tools/testing/selftests/kvm/x86_64/vmx_dirty_log_test.c |    2 
 a/tools/testing/selftests/memfd/memfd_test.c              |    2 
 b/MAINTAINERS                                             |    2 
 b/tools/include/asm-generic/bitops.h                      |    1 
 b/tools/include/linux/bitmap.h                            |    7 
 b/tools/include/linux/find.h                              |   81 +
 b/tools/lib/find_bit.c                                    |   20 
 227 files changed, 6695 insertions(+), 1875 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-09-02 21:48 incoming Andrew Morton
@ 2021-09-02 21:49 ` Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-09-02 21:49 UTC (permalink / raw)
  To: Linus Torvalds, linux-mm, mm-commits

On Thu, 2 Sep 2021 14:48:20 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> 212 patches, based on 4a3bb4200a5958d76cc26ebe4db4257efa56812b.

Make that "based on 7d2a07b769330c34b4deabeed939325c77a7ec2f".

^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-09-02 21:48 Andrew Morton
  2021-09-02 21:49 ` incoming Andrew Morton
  0 siblings, 1 reply; 395+ messages in thread
From: Andrew Morton @ 2021-09-02 21:48 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

212 patches, based on 4a3bb4200a5958d76cc26ebe4db4257efa56812b.

Subsystems affected by this patch series:

  ia64
  ocfs2
  block
  mm/slub
  mm/debug
  mm/pagecache
  mm/gup
  mm/swap
  mm/shmem
  mm/memcg
  mm/selftests
  mm/pagemap
  mm/mremap
  mm/bootmem
  mm/sparsemem
  mm/vmalloc
  mm/kasan
  mm/pagealloc
  mm/memory-failure
  mm/hugetlb
  mm/userfaultfd
  mm/vmscan
  mm/compaction
  mm/mempolicy
  mm/memblock
  mm/oom-kill
  mm/migration
  mm/ksm
  mm/percpu
  mm/vmstat
  mm/madvise

Subsystem: ia64

    Jason Wang <wangborong@cdjrlc.com>:
      ia64: fix typo in a comment

    Geert Uytterhoeven <geert+renesas@glider.be>:
    Patch series "ia64: Miscellaneous fixes and cleanups":
      ia64: fix #endif comment for reserve_elfcorehdr()
      ia64: make reserve_elfcorehdr() static
      ia64: make num_rsvd_regions static

Subsystem: ocfs2

    Dan Carpenter <dan.carpenter@oracle.com>:
      ocfs2: remove an unnecessary condition

    Tuo Li <islituo@gmail.com>:
      ocfs2: quota_local: fix possible uninitialized-variable access in ocfs2_local_read_info()

    Gang He <ghe@suse.com>:
      ocfs2: ocfs2_downconvert_lock failure results in deadlock

Subsystem: block

    kernel test robot <lkp@intel.com>:
      arch/csky/kernel/probes/kprobes.c: fix bugon.cocci warnings

Subsystem: mm/slub

    Vlastimil Babka <vbabka@suse.cz>:
    Patch series "SLUB: reduce irq disabled scope and make it RT compatible", v4:
      mm, slub: don't call flush_all() from slab_debug_trace_open()
      mm, slub: allocate private object map for debugfs listings
      mm, slub: allocate private object map for validate_slab_cache()
      mm, slub: don't disable irq for debug_check_no_locks_freed()
      mm, slub: remove redundant unfreeze_partials() from put_cpu_partial()
      mm, slub: unify cmpxchg_double_slab() and __cmpxchg_double_slab()
      mm, slub: extract get_partial() from new_slab_objects()
      mm, slub: dissolve new_slab_objects() into ___slab_alloc()
      mm, slub: return slab page from get_partial() and set c->page afterwards
      mm, slub: restructure new page checks in ___slab_alloc()
      mm, slub: simplify kmem_cache_cpu and tid setup
      mm, slub: move disabling/enabling irqs to ___slab_alloc()
      mm, slub: do initial checks in ___slab_alloc() with irqs enabled
      mm, slub: move disabling irqs closer to get_partial() in ___slab_alloc()
      mm, slub: restore irqs around calling new_slab()
      mm, slub: validate slab from partial list or page allocator before making it cpu slab
      mm, slub: check new pages with restored irqs
      mm, slub: stop disabling irqs around get_partial()
      mm, slub: move reset of c->page and freelist out of deactivate_slab()
      mm, slub: make locking in deactivate_slab() irq-safe
      mm, slub: call deactivate_slab() without disabling irqs
      mm, slub: move irq control into unfreeze_partials()
      mm, slub: discard slabs in unfreeze_partials() without irqs disabled
      mm, slub: detach whole partial list at once in unfreeze_partials()
      mm, slub: separate detaching of partial list in unfreeze_partials() from unfreezing
      mm, slub: only disable irq with spin_lock in __unfreeze_partials()
      mm, slub: don't disable irqs in slub_cpu_dead()
      mm, slab: make flush_slab() possible to call with irqs enabled

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
      mm: slub: move flush_cpu_slab() invocations __free_slab() invocations out of IRQ context
      mm: slub: make object_map_lock a raw_spinlock_t

    Vlastimil Babka <vbabka@suse.cz>:
      mm, slub: optionally save/restore irqs in slab_[un]lock()/
      mm, slub: make slab_lock() disable irqs with PREEMPT_RT
      mm, slub: protect put_cpu_partial() with disabled irqs instead of cmpxchg
      mm, slub: use migrate_disable() on PREEMPT_RT
      mm, slub: convert kmem_cpu_slab protection to local_lock

Subsystem: mm/debug

    Gavin Shan <gshan@redhat.com>:
    Patch series "mm/debug_vm_pgtable: Enhancements", v6:
      mm/debug_vm_pgtable: introduce struct pgtable_debug_args
      mm/debug_vm_pgtable: use struct pgtable_debug_args in basic tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in leaf and savewrite tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in protnone and devmap tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in soft_dirty and swap tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in migration and thp tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in PTE modifying tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in PMD modifying tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in PUD modifying tests
      mm/debug_vm_pgtable: use struct pgtable_debug_args in PGD and P4D modifying tests
      mm/debug_vm_pgtable: remove unused code
      mm/debug_vm_pgtable: fix corrupted page flag

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm: report a more useful address for reclaim acquisition

    liuhailong <liuhailong@oppo.com>:
      mm: add kernel_misc_reclaimable in show_free_areas

Subsystem: mm/pagecache

    Jan Kara <jack@suse.cz>:
    Patch series "writeback: Fix bandwidth estimates", v4:
      writeback: track number of inodes under writeback
      writeback: reliably update bandwidth estimation
      writeback: fix bandwidth estimate for spiky workload
      writeback: rename domain_update_bandwidth()
      writeback: use READ_ONCE for unlocked reads of writeback stats

    Johannes Weiner <hannes@cmpxchg.org>:
      mm: remove irqsave/restore locking from contexts with irqs enabled
      fs: drop_caches: fix skipping over shadow cache inodes
      fs: inode: count invalidated shadow pages in pginodesteal

    Shakeel Butt <shakeelb@google.com>:
      writeback: memcg: simplify cgroup_writeback_by_id

    Jing Yangyang <jing.yangyang@zte.com.cn>:
      include/linux/buffer_head.h: fix boolreturn.cocci warnings

Subsystem: mm/gup

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanups and fixup for gup":
      mm: gup: remove set but unused local variable major
      mm: gup: remove unneed local variable orig_refs
      mm: gup: remove useless BUG_ON in __get_user_pages()
      mm: gup: fix potential pgmap refcnt leak in __gup_device_huge()
      mm: gup: use helper PAGE_ALIGNED in populate_vma_page_range()

    John Hubbard <jhubbard@nvidia.com>:
    Patch series "A few gup refactorings and documentation updates", v3:
      mm/gup: documentation corrections for gup/pup
      mm/gup: small refactoring: simplify try_grab_page()
      mm/gup: remove try_get_page(), call try_get_compound_head() directly

Subsystem: mm/swap

    Hugh Dickins <hughd@google.com>:
      fs, mm: fix race in unlinking swapfile

    John Hubbard <jhubbard@nvidia.com>:
      mm: delete unused get_kernel_page()

Subsystem: mm/shmem

    Sebastian Andrzej Siewior <bigeasy@linutronix.de>:
      shmem: use raw_spinlock_t for ->stat_lock

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanups for shmem":
      shmem: remove unneeded variable ret
      shmem: remove unneeded header file
      shmem: remove unneeded function forward declaration
      shmem: include header file to declare swap_info

    Hugh Dickins <hughd@google.com>:
    Patch series "huge tmpfs: shmem_is_huge() fixes and cleanups":
      huge tmpfs: fix fallocate(vanilla) advance over huge pages
      huge tmpfs: fix split_huge_page() after FALLOC_FL_KEEP_SIZE
      huge tmpfs: remove shrinklist addition from shmem_setattr()
      huge tmpfs: revert shmem's use of transhuge_vma_enabled()
      huge tmpfs: move shmem_huge_enabled() upwards
      huge tmpfs: SGP_NOALLOC to stop collapse_file() on race
      huge tmpfs: shmem_is_huge(vma, inode, index)
      huge tmpfs: decide stat.st_blksize by shmem_is_huge()
      shmem: shmem_writepage() split unlikely i915 THP

Subsystem: mm/memcg

    Suren Baghdasaryan <surenb@google.com>:
      mm, memcg: add mem_cgroup_disabled checks in vmpressure and swap-related functions
      mm, memcg: inline mem_cgroup_{charge/uncharge} to improve disabled memcg config
      mm, memcg: inline swap-related functions to improve disabled memcg config

    Vasily Averin <vvs@virtuozzo.com>:
      memcg: enable accounting for pids in nested pid namespaces

    Shakeel Butt <shakeelb@google.com>:
      memcg: switch lruvec stats to rstat
      memcg: infrastructure to flush memcg stats

    Yutian Yang <nglaive@gmail.com>:
      memcg: charge fs_context and legacy_fs_context

    Vasily Averin <vvs@virtuozzo.com>:
    Patch series "memcg accounting from OpenVZ", v7:
      memcg: enable accounting for mnt_cache entries
      memcg: enable accounting for pollfd and select bits arrays
      memcg: enable accounting for file lock caches
      memcg: enable accounting for fasync_cache
      memcg: enable accounting for new namesapces and struct nsproxy
      memcg: enable accounting of ipc resources
      memcg: enable accounting for signals
      memcg: enable accounting for posix_timers_cache slab
      memcg: enable accounting for ldt_struct objects

    Shakeel Butt <shakeelb@google.com>:
      memcg: cleanup racy sum avoidance code

    Vasily Averin <vvs@virtuozzo.com>:
      memcg: replace in_interrupt() by !in_task() in active_memcg()

    Baolin Wang <baolin.wang@linux.alibaba.com>:
      mm: memcontrol: set the correct memcg swappiness restriction

    Miaohe Lin <linmiaohe@huawei.com>:
      mm, memcg: remove unused functions
      mm, memcg: save some atomic ops when flush is already true

    Michal Hocko <mhocko@suse.com>:
      memcg: fix up drain_local_stock comment

    Shakeel Butt <shakeelb@google.com>:
      memcg: make memcg->event_list_lock irqsafe

Subsystem: mm/selftests

    Po-Hsu Lin <po-hsu.lin@canonical.com>:
      selftests/vm: use kselftest skip code for skipped tests

    Colin Ian King <colin.king@canonical.com>:
      selftests: Fix spelling mistake "cann't" -> "cannot"

Subsystem: mm/pagemap

    Nicholas Piggin <npiggin@gmail.com>:
    Patch series "shoot lazy tlbs", v4:
      lazy tlb: introduce lazy mm refcount helper functions
      lazy tlb: allow lazy tlb mm refcounting to be configurable
      lazy tlb: shoot lazies, a non-refcounting lazy tlb option
      powerpc/64s: enable MMU_LAZY_TLB_SHOOTDOWN

    Christoph Hellwig <hch@lst.de>:
    Patch series "_kernel_dcache_page fixes and removal":
      mmc: JZ4740: remove the flush_kernel_dcache_page call in jz4740_mmc_read_data
      mmc: mmc_spi: replace flush_kernel_dcache_page with flush_dcache_page
      scatterlist: replace flush_kernel_dcache_page with flush_dcache_page
      mm: remove flush_kernel_dcache_page

    Huang Ying <ying.huang@intel.com>:
      mm,do_huge_pmd_numa_page: remove unnecessary TLB flushing code

    Greg Kroah-Hartman <gregkh@linuxfoundation.org>:
      mm: change fault_in_pages_* to have an unsigned size parameter

    Luigi Rizzo <lrizzo@google.com>:
      mm/pagemap: add mmap_assert_locked() annotations to find_vma*()

    "Liam R. Howlett" <Liam.Howlett@Oracle.com>:
      remap_file_pages: Use vma_lookup() instead of find_vma()

Subsystem: mm/mremap

    Chen Wandun <chenwandun@huawei.com>:
      mm/mremap: fix memory account on do_munmap() failure

Subsystem: mm/bootmem

    Muchun Song <songmuchun@bytedance.com>:
      mm/bootmem_info.c: mark __init on register_page_bootmem_info_section

Subsystem: mm/sparsemem

    Ohhoon Kwon <ohoono.kwon@samsung.com>:
    Patch series "mm: sparse: remove __section_nr() function", v4:
      mm: sparse: pass section_nr to section_mark_present
      mm: sparse: pass section_nr to find_memory_block
      mm: sparse: remove __section_nr() function

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm/sparse: set SECTION_NID_SHIFT to 6

    Matthew Wilcox <willy@infradead.org>:
      include/linux/mmzone.h: avoid a warning in sparse memory support

    Miles Chen <miles.chen@mediatek.com>:
      mm/sparse: clarify pgdat_to_phys

Subsystem: mm/vmalloc

    "Uladzislau Rezki (Sony)" <urezki@gmail.com>:
      mm/vmalloc: use batched page requests in bulk-allocator
      mm/vmalloc: remove gfpflags_allow_blocking() check
      lib/test_vmalloc.c: add a new 'nr_pages' parameter

    Chen Wandun <chenwandun@huawei.com>:
      mm/vmalloc: fix wrong behavior in vread

Subsystem: mm/kasan

    Woody Lin <woodylin@google.com>:
      mm/kasan: move kasan.fault to mm/kasan/report.c

    Andrey Konovalov <andreyknvl@gmail.com>:
    Patch series "kasan: test: avoid crashing the kernel with HW_TAGS", v2:
      kasan: test: rework kmalloc_oob_right
      kasan: test: avoid writing invalid memory
      kasan: test: avoid corrupting memory via memset
      kasan: test: disable kmalloc_memmove_invalid_size for HW_TAGS
      kasan: test: only do kmalloc_uaf_memset for generic mode
      kasan: test: clean up ksize_uaf
      kasan: test: avoid corrupting memory in copy_user_test
      kasan: test: avoid corrupting memory in kasan_rcu_uaf

Subsystem: mm/pagealloc

    Mike Rapoport <rppt@linux.ibm.com>:
    Patch series "mm: ensure consistency of memory map poisoning":
      mm/page_alloc: always initialize memory map for the holes
      microblaze: simplify pte_alloc_one_kernel()
      mm: introduce memmap_alloc() to unify memory map allocation
      memblock: stop poisoning raw allocations

    Nico Pache <npache@redhat.com>:
      mm/page_alloc.c: fix 'zone_id' may be used uninitialized in this function warning

    Mike Rapoport <rppt@linux.ibm.com>:
      mm/page_alloc: make alloc_node_mem_map() __init rather than __ref

    Vasily Averin <vvs@virtuozzo.com>:
      mm/page_alloc.c: use in_task()

    "George G. Davis" <davis.george@siemens.com>:
      mm/page_isolation: tracing: trace all test_pages_isolated failures

Subsystem: mm/memory-failure

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanups and fixup for hwpoison":
      mm/hwpoison: remove unneeded variable unmap_success
      mm/hwpoison: fix potential pte_unmap_unlock pte error
      mm/hwpoison: change argument struct page **hpagep to *hpage
      mm/hwpoison: fix some obsolete comments

    Yang Shi <shy828301@gmail.com>:
      mm: hwpoison: don't drop slab caches for offlining non-LRU page
      doc: hwpoison: correct the support for hugepage
      mm: hwpoison: dump page for unhandlable page

    Michael Wang <yun.wang@linux.alibaba.com>:
      mm: fix panic caused by __page_handle_poison()

Subsystem: mm/hugetlb

    Mike Kravetz <mike.kravetz@oracle.com>:
      hugetlb: simplify prep_compound_gigantic_page ref count racing code
      hugetlb: drop ref count earlier after page allocation
      hugetlb: before freeing hugetlb page set dtor to appropriate value
      hugetlb: fix hugetlb cgroup refcounting during vma split

Subsystem: mm/userfaultfd

    Nadav Amit <namit@vmware.com>:
    Patch series "userfaultfd: minor bug fixes":
      userfaultfd: change mmap_changing to atomic
      userfaultfd: prevent concurrent API initialization
      selftests/vm/userfaultfd: wake after copy failure

Subsystem: mm/vmscan

    Dave Hansen <dave.hansen@linux.intel.com>:
    Patch series "Migrate Pages in lieu of discard", v11:
      mm/numa: automatically generate node migration order
      mm/migrate: update node demotion order on hotplug events

    Yang Shi <yang.shi@linux.alibaba.com>:
      mm/migrate: enable returning precise migrate_pages() success count

    Dave Hansen <dave.hansen@linux.intel.com>:
      mm/migrate: demote pages during reclaim

    Yang Shi <yang.shi@linux.alibaba.com>:
      mm/vmscan: add page demotion counter

    Dave Hansen <dave.hansen@linux.intel.com>:
      mm/vmscan: add helper for querying ability to age anonymous pages

    Keith Busch <kbusch@kernel.org>:
      mm/vmscan: Consider anonymous pages without swap

    Dave Hansen <dave.hansen@linux.intel.com>:
      mm/vmscan: never demote for memcg reclaim

    Huang Ying <ying.huang@intel.com>:
      mm/migrate: add sysfs interface to enable reclaim migration

    Hui Su <suhui@zeku.com>:
      mm/vmpressure: replace vmpressure_to_css() with vmpressure_to_memcg()

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanups for vmscan", v2:
      mm/vmscan: remove the PageDirty check after MADV_FREE pages are page_ref_freezed
      mm/vmscan: remove misleading setting to sc->priority
      mm/vmscan: remove unneeded return value of kswapd_run()
      mm/vmscan: add 'else' to remove check_pending label

    Vlastimil Babka <vbabka@suse.cz>:
      mm, vmscan: guarantee drop_slab_node() termination

Subsystem: mm/compaction

    Charan Teja Reddy <charante@codeaurora.org>:
      mm: compaction: optimize proactive compaction deferrals
      mm: compaction: support triggering of proactive compaction by user

Subsystem: mm/mempolicy

    Baolin Wang <baolin.wang@linux.alibaba.com>:
      mm/mempolicy: use readable NUMA_NO_NODE macro instead of magic number

    Dave Hansen <dave.hansen@linux.intel.com>:
    Patch series "Introduce multi-preference mempolicy", v7:
      mm/mempolicy: add MPOL_PREFERRED_MANY for multiple preferred nodes

    Feng Tang <feng.tang@intel.com>:
      mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy

    Ben Widawsky <ben.widawsky@intel.com>:
      mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY
      mm/mempolicy: advertise new MPOL_PREFERRED_MANY

    Feng Tang <feng.tang@intel.com>:
      mm/mempolicy: unify the create() func for bind/interleave/prefer-many policies

    Vasily Averin <vvs@virtuozzo.com>:
      mm/mempolicy.c: use in_task() in mempolicy_slab_node()

Subsystem: mm/memblock

    Mike Rapoport <rppt@linux.ibm.com>:
      memblock: make memblock_find_in_range method private

Subsystem: mm/oom-kill

    Suren Baghdasaryan <surenb@google.com>:
      mm: introduce process_mrelease system call
      mm: wire up syscall process_mrelease

Subsystem: mm/migration

    Randy Dunlap <rdunlap@infradead.org>:
      mm/migrate: correct kernel-doc notation

Subsystem: mm/ksm

    Zhansaya Bagdauletkyzy <zhansayabagdaulet@gmail.com>:
    Patch series "add KSM selftests":
      selftests: vm: add KSM merge test
      selftests: vm: add KSM unmerge test
      selftests: vm: add KSM zero page merging test
      selftests: vm: add KSM merging across nodes test
      mm: KSM: fix data type
    Patch series "add KSM performance tests", v3:
      selftests: vm: add KSM merging time test
      selftests: vm: add COW time test for KSM pages

Subsystem: mm/percpu

    Jing Xiangfeng <jingxiangfeng@huawei.com>:
      mm/percpu,c: remove obsolete comments of pcpu_chunk_populated()

Subsystem: mm/vmstat

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanup for vmstat":
      mm/vmstat: correct some wrong comments
      mm/vmstat: simplify the array size calculation
      mm/vmstat: remove unneeded return value

Subsystem: mm/madvise

    zhangkui <zhangkui@oppo.com>:
      mm/madvise: add MADV_WILLNEED to process_madvise()

 Documentation/ABI/testing/sysfs-kernel-mm-numa         |   24 
 Documentation/admin-guide/mm/numa_memory_policy.rst    |   15 
 Documentation/admin-guide/sysctl/vm.rst                |    3 
 Documentation/core-api/cachetlb.rst                    |   86 -
 Documentation/dev-tools/kasan.rst                      |   13 
 Documentation/translations/zh_CN/core-api/cachetlb.rst |    9 
 Documentation/vm/hwpoison.rst                          |    1 
 arch/Kconfig                                           |   28 
 arch/alpha/kernel/syscalls/syscall.tbl                 |    2 
 arch/arm/include/asm/cacheflush.h                      |    4 
 arch/arm/kernel/setup.c                                |   20 
 arch/arm/mach-rpc/ecard.c                              |    2 
 arch/arm/mm/flush.c                                    |   33 
 arch/arm/mm/nommu.c                                    |    6 
 arch/arm/tools/syscall.tbl                             |    2 
 arch/arm64/include/asm/unistd.h                        |    2 
 arch/arm64/include/asm/unistd32.h                      |    2 
 arch/arm64/kvm/hyp/reserved_mem.c                      |    9 
 arch/arm64/mm/init.c                                   |   38 
 arch/csky/abiv1/cacheflush.c                           |   11 
 arch/csky/abiv1/inc/abi/cacheflush.h                   |    4 
 arch/csky/kernel/probes/kprobes.c                      |    3 
 arch/ia64/include/asm/meminit.h                        |    2 
 arch/ia64/kernel/acpi.c                                |    2 
 arch/ia64/kernel/setup.c                               |   55 
 arch/ia64/kernel/syscalls/syscall.tbl                  |    2 
 arch/m68k/kernel/syscalls/syscall.tbl                  |    2 
 arch/microblaze/include/asm/page.h                     |    3 
 arch/microblaze/include/asm/pgtable.h                  |    2 
 arch/microblaze/kernel/syscalls/syscall.tbl            |    2 
 arch/microblaze/mm/init.c                              |   12 
 arch/microblaze/mm/pgtable.c                           |   17 
 arch/mips/include/asm/cacheflush.h                     |    8 
 arch/mips/kernel/setup.c                               |   14 
 arch/mips/kernel/syscalls/syscall_n32.tbl              |    2 
 arch/mips/kernel/syscalls/syscall_n64.tbl              |    2 
 arch/mips/kernel/syscalls/syscall_o32.tbl              |    2 
 arch/nds32/include/asm/cacheflush.h                    |    3 
 arch/nds32/mm/cacheflush.c                             |    9 
 arch/parisc/include/asm/cacheflush.h                   |    8 
 arch/parisc/kernel/cache.c                             |    3 
 arch/parisc/kernel/syscalls/syscall.tbl                |    2 
 arch/powerpc/Kconfig                                   |    1 
 arch/powerpc/kernel/smp.c                              |    2 
 arch/powerpc/kernel/syscalls/syscall.tbl               |    2 
 arch/powerpc/mm/book3s64/radix_tlb.c                   |    4 
 arch/powerpc/platforms/pseries/hotplug-memory.c        |    4 
 arch/riscv/mm/init.c                                   |   44 
 arch/s390/kernel/setup.c                               |    9 
 arch/s390/kernel/syscalls/syscall.tbl                  |    2 
 arch/s390/mm/fault.c                                   |    2 
 arch/sh/include/asm/cacheflush.h                       |    8 
 arch/sh/kernel/syscalls/syscall.tbl                    |    2 
 arch/sparc/kernel/syscalls/syscall.tbl                 |    2 
 arch/x86/entry/syscalls/syscall_32.tbl                 |    1 
 arch/x86/entry/syscalls/syscall_64.tbl                 |    1 
 arch/x86/kernel/aperture_64.c                          |    5 
 arch/x86/kernel/ldt.c                                  |    6 
 arch/x86/mm/init.c                                     |   23 
 arch/x86/mm/numa.c                                     |    5 
 arch/x86/mm/numa_emulation.c                           |    5 
 arch/x86/realmode/init.c                               |    2 
 arch/xtensa/kernel/syscalls/syscall.tbl                |    2 
 block/blk-map.c                                        |    2 
 drivers/acpi/tables.c                                  |    5 
 drivers/base/arch_numa.c                               |    5 
 drivers/base/memory.c                                  |    4 
 drivers/mmc/host/jz4740_mmc.c                          |    4 
 drivers/mmc/host/mmc_spi.c                             |    2 
 drivers/of/of_reserved_mem.c                           |   12 
 fs/drop_caches.c                                       |    3 
 fs/exec.c                                              |   12 
 fs/fcntl.c                                             |    3 
 fs/fs-writeback.c                                      |   28 
 fs/fs_context.c                                        |    4 
 fs/inode.c                                             |    2 
 fs/locks.c                                             |    6 
 fs/namei.c                                             |    8 
 fs/namespace.c                                         |    7 
 fs/ocfs2/dlmglue.c                                     |   14 
 fs/ocfs2/quota_global.c                                |    1 
 fs/ocfs2/quota_local.c                                 |    2 
 fs/pipe.c                                              |    2 
 fs/select.c                                            |    4 
 fs/userfaultfd.c                                       |  116 -
 include/linux/backing-dev-defs.h                       |    2 
 include/linux/backing-dev.h                            |   19 
 include/linux/buffer_head.h                            |    2 
 include/linux/compaction.h                             |    2 
 include/linux/highmem.h                                |    5 
 include/linux/hugetlb_cgroup.h                         |   12 
 include/linux/memblock.h                               |    2 
 include/linux/memcontrol.h                             |  118 +
 include/linux/memory.h                                 |    2 
 include/linux/mempolicy.h                              |   16 
 include/linux/migrate.h                                |   14 
 include/linux/mm.h                                     |   17 
 include/linux/mmzone.h                                 |    4 
 include/linux/page-flags.h                             |    9 
 include/linux/pagemap.h                                |    4 
 include/linux/sched/mm.h                               |   35 
 include/linux/shmem_fs.h                               |   25 
 include/linux/slub_def.h                               |    6 
 include/linux/swap.h                                   |   28 
 include/linux/syscalls.h                               |    1 
 include/linux/userfaultfd_k.h                          |    8 
 include/linux/vm_event_item.h                          |    2 
 include/linux/vmpressure.h                             |    2 
 include/linux/writeback.h                              |    4 
 include/trace/events/migrate.h                         |    3 
 include/uapi/asm-generic/unistd.h                      |    4 
 include/uapi/linux/mempolicy.h                         |    1 
 ipc/msg.c                                              |    2 
 ipc/namespace.c                                        |    2 
 ipc/sem.c                                              |    9 
 ipc/shm.c                                              |    2 
 kernel/cgroup/namespace.c                              |    2 
 kernel/cpu.c                                           |    2 
 kernel/exit.c                                          |    2 
 kernel/fork.c                                          |   51 
 kernel/kthread.c                                       |   21 
 kernel/nsproxy.c                                       |    2 
 kernel/pid_namespace.c                                 |    5 
 kernel/sched/core.c                                    |   37 
 kernel/sched/sched.h                                   |    4 
 kernel/signal.c                                        |    2 
 kernel/sys_ni.c                                        |    1 
 kernel/sysctl.c                                        |    2 
 kernel/time/namespace.c                                |    4 
 kernel/time/posix-timers.c                             |    4 
 kernel/user_namespace.c                                |    2 
 lib/scatterlist.c                                      |    5 
 lib/test_kasan.c                                       |   80 -
 lib/test_kasan_module.c                                |   20 
 lib/test_vmalloc.c                                     |    5 
 mm/backing-dev.c                                       |   11 
 mm/bootmem_info.c                                      |    4 
 mm/compaction.c                                        |   69 -
 mm/debug_vm_pgtable.c                                  |  982 +++++++++------
 mm/filemap.c                                           |   15 
 mm/gup.c                                               |  109 -
 mm/huge_memory.c                                       |   32 
 mm/hugetlb.c                                           |  173 ++
 mm/hwpoison-inject.c                                   |    2 
 mm/internal.h                                          |    9 
 mm/kasan/hw_tags.c                                     |   43 
 mm/kasan/kasan.h                                       |    1 
 mm/kasan/report.c                                      |   29 
 mm/khugepaged.c                                        |    2 
 mm/ksm.c                                               |    8 
 mm/madvise.c                                           |    1 
 mm/memblock.c                                          |   22 
 mm/memcontrol.c                                        |  234 +--
 mm/memory-failure.c                                    |   53 
 mm/memory_hotplug.c                                    |    2 
 mm/mempolicy.c                                         |  207 ++-
 mm/migrate.c                                           |  319 ++++
 mm/mmap.c                                              |    7 
 mm/mremap.c                                            |    2 
 mm/oom_kill.c                                          |   70 +
 mm/page-writeback.c                                    |  133 +-
 mm/page_alloc.c                                        |   62 
 mm/page_isolation.c                                    |   13 
 mm/percpu.c                                            |    3 
 mm/shmem.c                                             |  309 ++--
 mm/slab_common.c                                       |    2 
 mm/slub.c                                              | 1085 ++++++++++-------
 mm/sparse.c                                            |   46 
 mm/swap.c                                              |   22 
 mm/swapfile.c                                          |   14 
 mm/truncate.c                                          |   28 
 mm/userfaultfd.c                                       |   15 
 mm/vmalloc.c                                           |   79 -
 mm/vmpressure.c                                        |   10 
 mm/vmscan.c                                            |  220 ++-
 mm/vmstat.c                                            |   25 
 security/tomoyo/domain.c                               |   13 
 tools/testing/scatterlist/linux/mm.h                   |    1 
 tools/testing/selftests/vm/.gitignore                  |    1 
 tools/testing/selftests/vm/Makefile                    |    3 
 tools/testing/selftests/vm/charge_reserved_hugetlb.sh  |    5 
 tools/testing/selftests/vm/hugetlb_reparenting_test.sh |    5 
 tools/testing/selftests/vm/ksm_tests.c                 |  696 ++++++++++
 tools/testing/selftests/vm/mlock-random-test.c         |    2 
 tools/testing/selftests/vm/run_vmtests.sh              |   98 +
 tools/testing/selftests/vm/userfaultfd.c               |   13 
 186 files changed, 4488 insertions(+), 2281 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-08-25 19:17 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-08-25 19:17 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

2 patches, based on 6e764bcd1cf72a2846c0e53d3975a09b242c04c9.

Subsystems affected by this patch series:

  mm/memory-hotplug
  MAINTAINERS

Subsystem: mm/memory-hotplug

    Miaohe Lin <linmiaohe@huawei.com>:
      mm/memory_hotplug: fix potential permanent lru cache disable

Subsystem: MAINTAINERS

    Namjae Jeon <namjae.jeon@samsung.com>:
      MAINTAINERS: exfat: update my email address

 MAINTAINERS         |    2 +-
 mm/memory_hotplug.c |    1 +
 2 files changed, 2 insertions(+), 1 deletion(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-08-20  2:03 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-08-20  2:03 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

10 patches, based on 614cb2751d3150850d459bee596c397f344a7936.

Subsystems affected by this patch series:

  mm/shmem
  mm/pagealloc
  mm/tracing
  MAINTAINERS
  mm/memcg
  mm/memory-failure
  mm/vmscan
  mm/kfence
  mm/hugetlb

Subsystem: mm/shmem

    Yang Shi <shy828301@gmail.com>:
      Revert "mm/shmem: fix shmem_swapin() race with swapoff"
      Revert "mm: swap: check if swap backing device is congested or not"

Subsystem: mm/pagealloc

    Doug Berger <opendmb@gmail.com>:
      mm/page_alloc: don't corrupt pcppage_migratetype

Subsystem: mm/tracing

    Mike Rapoport <rppt@linux.ibm.com>:
      mmflags.h: add missing __GFP_ZEROTAGS and __GFP_SKIP_KASAN_POISON names

Subsystem: MAINTAINERS

    Nathan Chancellor <nathan@kernel.org>:
      MAINTAINERS: update ClangBuiltLinux IRC chat

Subsystem: mm/memcg

    Johannes Weiner <hannes@cmpxchg.org>:
      mm: memcontrol: fix occasional OOMs due to proportional memory.low reclaim

Subsystem: mm/memory-failure

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm/hwpoison: retry with shake_page() for unhandlable pages

Subsystem: mm/vmscan

    Johannes Weiner <hannes@cmpxchg.org>:
      mm: vmscan: fix missing psi annotation for node_reclaim()

Subsystem: mm/kfence

    Marco Elver <elver@google.com>:
      kfence: fix is_kfence_address() for addresses below KFENCE_POOL_SIZE

Subsystem: mm/hugetlb

    Mike Kravetz <mike.kravetz@oracle.com>:
      hugetlb: don't pass page cache pages to restore_reserve_on_error

 MAINTAINERS                    |    2 +-
 include/linux/kfence.h         |    7 ++++---
 include/linux/memcontrol.h     |   29 +++++++++++++++--------------
 include/trace/events/mmflags.h |    4 +++-
 mm/hugetlb.c                   |   19 ++++++++++++++-----
 mm/memory-failure.c            |   12 +++++++++---
 mm/page_alloc.c                |   25 ++++++++++++-------------
 mm/shmem.c                     |   14 +-------------
 mm/swap_state.c                |    7 -------
 mm/vmscan.c                    |   30 ++++++++++++++++++++++--------
 10 files changed, 81 insertions(+), 68 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-08-13 23:53 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-08-13 23:53 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

7 patches, based on f8e6dfc64f6135d1b6c5215c14cd30b9b60a0008.

Subsystems affected by this patch series:

  mm/kasan
  mm/slub
  mm/madvise
  mm/memcg
  lib

Subsystem: mm/kasan

    Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>:
    Patch series "kasan, slub: reset tag when printing address", v3:
      kasan, kmemleak: reset tags when scanning block
      kasan, slub: reset tag when printing address

Subsystem: mm/slub

    Shakeel Butt <shakeelb@google.com>:
      slub: fix kmalloc_pagealloc_invalid_free unit test

    Vlastimil Babka <vbabka@suse.cz>:
      mm: slub: fix slub_debug disabling for list of slabs

Subsystem: mm/madvise

    David Hildenbrand <david@redhat.com>:
      mm/madvise: report SIGBUS as -EFAULT for MADV_POPULATE_(READ|WRITE)

Subsystem: mm/memcg

    Waiman Long <longman@redhat.com>:
      mm/memcg: fix incorrect flushing of lruvec data in obj_stock

Subsystem: lib

    Liang Wang <wangliang101@huawei.com>:
      lib: use PFN_PHYS() in devmem_is_allowed()

 lib/devmem_is_allowed.c |    2 +-
 mm/gup.c                |    7 +++++--
 mm/kmemleak.c           |    6 +++---
 mm/madvise.c            |    4 +++-
 mm/memcontrol.c         |    6 ++++--
 mm/slub.c               |   25 ++++++++++++++-----------
 6 files changed, 30 insertions(+), 20 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-07-29 21:52 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-07-29 21:52 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

7 patches, based on 7e96bf476270aecea66740a083e51b38c1371cd2.

Subsystems affected by this patch series:

  lib
  ocfs2
  mm/memcg
  mm/migration
  mm/slub
  mm/memcg

Subsystem: lib

    Matteo Croce <mcroce@microsoft.com>:
      lib/test_string.c: move string selftest in the Runtime Testing menu

Subsystem: ocfs2

    Junxiao Bi <junxiao.bi@oracle.com>:
      ocfs2: fix zero out valid data
      ocfs2: issue zeroout to EOF blocks

Subsystem: mm/memcg

    Johannes Weiner <hannes@cmpxchg.org>:
      mm: memcontrol: fix blocking rstat function called from atomic cgroup1 thresholding code

Subsystem: mm/migration

    "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>:
      mm/migrate: fix NR_ISOLATED corruption on 64-bit

Subsystem: mm/slub

    Shakeel Butt <shakeelb@google.com>:
      slub: fix unreclaimable slab stat for bulk free

Subsystem: mm/memcg

    Wang Hai <wanghai38@huawei.com>:
      mm/memcg: fix NULL pointer dereference in memcg_slab_free_hook()

 fs/ocfs2/file.c   |  103 ++++++++++++++++++++++++++++++++----------------------
 lib/Kconfig       |    3 -
 lib/Kconfig.debug |    3 +
 mm/memcontrol.c   |    3 +
 mm/migrate.c      |    2 -
 mm/slab.h         |    2 -
 mm/slub.c         |   22 ++++++-----
 7 files changed, 81 insertions(+), 57 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-07-23 22:49 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-07-23 22:49 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

15 patches, based on 704f4cba43d4ed31ef4beb422313f1263d87bc55.

Subsystems affected by this patch series:

  mm/userfaultfd
  mm/kfence
  mm/highmem
  mm/pagealloc
  mm/memblock
  mm/pagecache
  mm/secretmem
  mm/pagemap
  mm/hugetlbfs

Subsystem: mm/userfaultfd

    Peter Collingbourne <pcc@google.com>:
    Patch series "userfaultfd: do not untag user pointers", v5:
      userfaultfd: do not untag user pointers
      selftest: use mmap instead of posix_memalign to allocate memory

Subsystem: mm/kfence

    Weizhao Ouyang <o451686892@gmail.com>:
      kfence: defer kfence_test_init to ensure that kunit debugfs is created

    Alexander Potapenko <glider@google.com>:
      kfence: move the size check to the beginning of __kfence_alloc()
      kfence: skip all GFP_ZONEMASK allocations

Subsystem: mm/highmem

    Christoph Hellwig <hch@lst.de>:
      mm: call flush_dcache_page() in memcpy_to_page() and memzero_page()
      mm: use kmap_local_page in memzero_page

Subsystem: mm/pagealloc

    Sergei Trofimovich <slyfox@gentoo.org>:
      mm: page_alloc: fix page_poison=1 / INIT_ON_ALLOC_DEFAULT_ON interaction

Subsystem: mm/memblock

    Mike Rapoport <rppt@linux.ibm.com>:
      memblock: make for_each_mem_range() traverse MEMBLOCK_HOTPLUG regions

Subsystem: mm/pagecache

    Roman Gushchin <guro@fb.com>:
      writeback, cgroup: remove wb from offline list before releasing refcnt
      writeback, cgroup: do not reparent dax inodes

Subsystem: mm/secretmem

    Mike Rapoport <rppt@linux.ibm.com>:
      mm/secretmem: wire up ->set_page_dirty

Subsystem: mm/pagemap

    Muchun Song <songmuchun@bytedance.com>:
      mm: mmap_lock: fix disabling preemption directly

    Qi Zheng <zhengqi.arch@bytedance.com>:
      mm: fix the deadlock in finish_fault()

Subsystem: mm/hugetlbfs

    Mike Kravetz <mike.kravetz@oracle.com>:
      hugetlbfs: fix mount mode command line processing

 Documentation/arm64/tagged-address-abi.rst |   26 ++++++++++++++++++--------
 fs/fs-writeback.c                          |    3 +++
 fs/hugetlbfs/inode.c                       |    2 +-
 fs/userfaultfd.c                           |   26 ++++++++++++--------------
 include/linux/highmem.h                    |    6 ++++--
 include/linux/memblock.h                   |    4 ++--
 mm/backing-dev.c                           |    2 +-
 mm/kfence/core.c                           |   19 ++++++++++++++++---
 mm/kfence/kfence_test.c                    |    2 +-
 mm/memblock.c                              |    3 ++-
 mm/memory.c                                |   11 ++++++++++-
 mm/mmap_lock.c                             |    4 ++--
 mm/page_alloc.c                            |   29 ++++++++++++++++-------------
 mm/secretmem.c                             |    1 +
 tools/testing/selftests/vm/userfaultfd.c   |    6 ++++--
 15 files changed, 93 insertions(+), 51 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-07-15  4:26 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-07-15  4:26 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

13 patches, based on 40226a3d96ef8ab8980f032681c8bfd46d63874e.

Subsystems affected by this patch series:

  mm/kasan
  mm/pagealloc
  mm/rmap
  mm/hmm
  hfs
  mm/hugetlb

Subsystem: mm/kasan

    Marco Elver <elver@google.com>:
      mm: move helper to check slub_debug_enabled

    Yee Lee <yee.lee@mediatek.com>:
      kasan: add memzero init for unaligned size at DEBUG

    Marco Elver <elver@google.com>:
      kasan: fix build by including kernel.h

Subsystem: mm/pagealloc

    Matteo Croce <mcroce@microsoft.com>:
      Revert "mm/page_alloc: make should_fail_alloc_page() static"

    Mel Gorman <mgorman@techsingularity.net>:
      mm/page_alloc: avoid page allocator recursion with pagesets.lock held

    Yanfei Xu <yanfei.xu@windriver.com>:
      mm/page_alloc: correct return value when failing at preparing

    Chuck Lever <chuck.lever@oracle.com>:
      mm/page_alloc: further fix __alloc_pages_bulk() return value

Subsystem: mm/rmap

    Christoph Hellwig <hch@lst.de>:
      mm: fix the try_to_unmap prototype for !CONFIG_MMU

Subsystem: mm/hmm

    Alistair Popple <apopple@nvidia.com>:
      lib/test_hmm: remove set but unused page variable

Subsystem: hfs

    Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>:
    Patch series "hfs: fix various errors", v2:
      hfs: add missing clean-up in hfs_fill_super
      hfs: fix high memory mapping in hfs_bnode_read
      hfs: add lock nesting notation to hfs_find_init

Subsystem: mm/hugetlb

    Joao Martins <joao.m.martins@oracle.com>:
      mm/hugetlb: fix refs calculation from unaligned @vaddr

 fs/hfs/bfind.c        |   14 +++++++++++++-
 fs/hfs/bnode.c        |   25 ++++++++++++++++++++-----
 fs/hfs/btree.h        |    7 +++++++
 fs/hfs/super.c        |   10 +++++-----
 include/linux/kasan.h |    1 +
 include/linux/rmap.h  |    4 +++-
 lib/test_hmm.c        |    2 --
 mm/hugetlb.c          |    5 +++--
 mm/kasan/kasan.h      |   12 ++++++++++++
 mm/page_alloc.c       |   30 ++++++++++++++++++++++--------
 mm/slab.h             |   15 +++++++++++----
 mm/slub.c             |   14 --------------
 12 files changed, 97 insertions(+), 42 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-07-08  0:59 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-07-08  0:59 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

54 patches, based on a931dd33d370896a683236bba67c0d6f3d01144d.

Subsystems affected by this patch series:

  lib
  mm/slub
  mm/secretmem
  mm/cleanups
  mm/init
  debug
  mm/pagemap
  mm/mremap

Subsystem: lib

    Zhen Lei <thunder.leizhen@huawei.com>:
      lib/test: fix spelling mistakes
      lib: fix spelling mistakes
      lib: fix spelling mistakes in header files

Subsystem: mm/slub

    Nathan Chancellor <nathan@kernel.org>:
    Patch series "hexagon: Fix build error with CONFIG_STACKDEPOT and select CONFIG_ARCH_WANT_LD_ORPHAN_WARN":
      hexagon: handle {,SOFT}IRQENTRY_TEXT in linker script
      hexagon: use common DISCARDS macro
      hexagon: select ARCH_WANT_LD_ORPHAN_WARN

    Oliver Glitta <glittao@gmail.com>:
      mm/slub: use stackdepot to save stack trace in objects

Subsystem: mm/secretmem

    Mike Rapoport <rppt@linux.ibm.com>:
    Patch series "mm: introduce memfd_secret system call to create "secret" memory areas", v20:
      mmap: make mlock_future_check() global
      riscv/Kconfig: make direct map manipulation options depend on MMU
      set_memory: allow querying whether set_direct_map_*() is actually enabled
      mm: introduce memfd_secret system call to create "secret" memory areas
      PM: hibernate: disable when there are active secretmem users
      arch, mm: wire up memfd_secret system call where relevant
      secretmem: test: add basic selftest for memfd_secret(2)

Subsystem: mm/cleanups

    Zhen Lei <thunder.leizhen@huawei.com>:
      mm: fix spelling mistakes in header files

Subsystem: mm/init

    Kefeng Wang <wangkefeng.wang@huawei.com>:
    Patch series "init_mm: cleanup ARCH's text/data/brk setup code", v3:
      mm: add setup_initial_init_mm() helper
      arc: convert to setup_initial_init_mm()
      arm: convert to setup_initial_init_mm()
      arm64: convert to setup_initial_init_mm()
      csky: convert to setup_initial_init_mm()
      h8300: convert to setup_initial_init_mm()
      m68k: convert to setup_initial_init_mm()
      nds32: convert to setup_initial_init_mm()
      nios2: convert to setup_initial_init_mm()
      openrisc: convert to setup_initial_init_mm()
      powerpc: convert to setup_initial_init_mm()
      riscv: convert to setup_initial_init_mm()
      s390: convert to setup_initial_init_mm()
      sh: convert to setup_initial_init_mm()
      x86: convert to setup_initial_init_mm()

Subsystem: debug

    Stephen Boyd <swboyd@chromium.org>:
    Patch series "Add build ID to stacktraces", v6:
      buildid: only consider GNU notes for build ID parsing
      buildid: add API to parse build ID out of buffer
      buildid: stash away kernels build ID on init
      dump_stack: add vmlinux build ID to stack traces
      module: add printk formats to add module build ID to stacktraces
      arm64: stacktrace: use %pSb for backtrace printing
      x86/dumpstack: use %pSb/%pBb for backtrace printing
      scripts/decode_stacktrace.sh: support debuginfod
      scripts/decode_stacktrace.sh: silence stderr messages from addr2line/nm
      scripts/decode_stacktrace.sh: indicate 'auto' can be used for base path
      buildid: mark some arguments const
      buildid: fix kernel-doc notation
      kdump: use vmlinux_build_id to simplify

Subsystem: mm/pagemap

    "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>:
      mm: rename pud_page_vaddr to pud_pgtable and make it return pmd_t *
      mm: rename p4d_page_vaddr to p4d_pgtable and make it return pud_t *

Subsystem: mm/mremap

    "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>:
    Patch series "mrermap fixes", v2:
      selftest/mremap_test: update the test to handle pagesize other than 4K
      selftest/mremap_test: avoid crash with static build
      mm/mremap: convert huge PUD move to separate helper
      mm/mremap: don't enable optimized PUD move if page table levels is 2
      mm/mremap: use pmd/pud_poplulate to update page table entries
      mm/mremap: hold the rmap lock in write mode when moving page table entries.
    Patch series "Speedup mremap on ppc64", v8:
      mm/mremap: allow arch runtime override
      powerpc/book3s64/mm: update flush_tlb_range to flush page walk cache
      powerpc/mm: enable HAVE_MOVE_PMD support

 Documentation/core-api/printk-formats.rst           |   11 
 arch/alpha/include/asm/pgtable.h                    |    8 
 arch/arc/mm/init.c                                  |    5 
 arch/arm/include/asm/pgtable-3level.h               |    2 
 arch/arm/kernel/setup.c                             |    5 
 arch/arm64/include/asm/Kbuild                       |    1 
 arch/arm64/include/asm/cacheflush.h                 |    6 
 arch/arm64/include/asm/kfence.h                     |    2 
 arch/arm64/include/asm/pgtable.h                    |    8 
 arch/arm64/include/asm/set_memory.h                 |   17 +
 arch/arm64/include/uapi/asm/unistd.h                |    1 
 arch/arm64/kernel/machine_kexec.c                   |    1 
 arch/arm64/kernel/setup.c                           |    5 
 arch/arm64/kernel/stacktrace.c                      |    2 
 arch/arm64/mm/mmu.c                                 |    7 
 arch/arm64/mm/pageattr.c                            |   13 
 arch/csky/kernel/setup.c                            |    5 
 arch/h8300/kernel/setup.c                           |    5 
 arch/hexagon/Kconfig                                |    1 
 arch/hexagon/kernel/vmlinux.lds.S                   |    9 
 arch/ia64/include/asm/pgtable.h                     |    4 
 arch/m68k/include/asm/motorola_pgtable.h            |    2 
 arch/m68k/kernel/setup_mm.c                         |    5 
 arch/m68k/kernel/setup_no.c                         |    5 
 arch/mips/include/asm/pgtable-64.h                  |    8 
 arch/nds32/kernel/setup.c                           |    5 
 arch/nios2/kernel/setup.c                           |    5 
 arch/openrisc/kernel/setup.c                        |    5 
 arch/parisc/include/asm/pgtable.h                   |    4 
 arch/powerpc/include/asm/book3s/64/pgtable.h        |   11 
 arch/powerpc/include/asm/book3s/64/tlbflush-radix.h |    2 
 arch/powerpc/include/asm/nohash/64/pgtable-4k.h     |    6 
 arch/powerpc/include/asm/nohash/64/pgtable.h        |    6 
 arch/powerpc/include/asm/tlb.h                      |    6 
 arch/powerpc/kernel/setup-common.c                  |    5 
 arch/powerpc/mm/book3s64/radix_hugetlbpage.c        |    8 
 arch/powerpc/mm/book3s64/radix_pgtable.c            |    6 
 arch/powerpc/mm/book3s64/radix_tlb.c                |   44 +-
 arch/powerpc/mm/pgtable_64.c                        |    4 
 arch/powerpc/platforms/Kconfig.cputype              |    2 
 arch/riscv/Kconfig                                  |    4 
 arch/riscv/include/asm/pgtable-64.h                 |    4 
 arch/riscv/include/asm/unistd.h                     |    1 
 arch/riscv/kernel/setup.c                           |    5 
 arch/s390/kernel/setup.c                            |    5 
 arch/sh/include/asm/pgtable-3level.h                |    4 
 arch/sh/kernel/setup.c                              |    5 
 arch/sparc/include/asm/pgtable_32.h                 |    6 
 arch/sparc/include/asm/pgtable_64.h                 |   10 
 arch/um/include/asm/pgtable-3level.h                |    2 
 arch/x86/entry/syscalls/syscall_32.tbl              |    1 
 arch/x86/entry/syscalls/syscall_64.tbl              |    1 
 arch/x86/include/asm/pgtable.h                      |    8 
 arch/x86/kernel/dumpstack.c                         |    2 
 arch/x86/kernel/setup.c                             |    5 
 arch/x86/mm/init_64.c                               |    4 
 arch/x86/mm/pat/set_memory.c                        |    4 
 arch/x86/mm/pgtable.c                               |    2 
 include/asm-generic/pgtable-nop4d.h                 |    2 
 include/asm-generic/pgtable-nopmd.h                 |    2 
 include/asm-generic/pgtable-nopud.h                 |    4 
 include/linux/bootconfig.h                          |    4 
 include/linux/buildid.h                             |   10 
 include/linux/compaction.h                          |    4 
 include/linux/cpumask.h                             |    2 
 include/linux/crash_core.h                          |   12 
 include/linux/debugobjects.h                        |    2 
 include/linux/hmm.h                                 |    2 
 include/linux/hugetlb.h                             |    6 
 include/linux/kallsyms.h                            |   21 +
 include/linux/list_lru.h                            |    4 
 include/linux/lru_cache.h                           |    8 
 include/linux/mm.h                                  |    3 
 include/linux/mmu_notifier.h                        |    8 
 include/linux/module.h                              |    9 
 include/linux/nodemask.h                            |    6 
 include/linux/percpu-defs.h                         |    2 
 include/linux/percpu-refcount.h                     |    2 
 include/linux/pgtable.h                             |    4 
 include/linux/scatterlist.h                         |    2 
 include/linux/secretmem.h                           |   54 +++
 include/linux/set_memory.h                          |   12 
 include/linux/shrinker.h                            |    2 
 include/linux/syscalls.h                            |    1 
 include/linux/vmalloc.h                             |    4 
 include/uapi/asm-generic/unistd.h                   |    7 
 include/uapi/linux/magic.h                          |    1 
 init/Kconfig                                        |    1 
 init/main.c                                         |    2 
 kernel/crash_core.c                                 |   50 ---
 kernel/kallsyms.c                                   |  104 +++++--
 kernel/module.c                                     |   42 ++
 kernel/power/hibernate.c                            |    5 
 kernel/sys_ni.c                                     |    2 
 lib/Kconfig.debug                                   |   17 -
 lib/asn1_encoder.c                                  |    2 
 lib/buildid.c                                       |   80 ++++-
 lib/devres.c                                        |    2 
 lib/dump_stack.c                                    |   13 
 lib/dynamic_debug.c                                 |    2 
 lib/fonts/font_pearl_8x8.c                          |    2 
 lib/kfifo.c                                         |    2 
 lib/list_sort.c                                     |    2 
 lib/nlattr.c                                        |    4 
 lib/oid_registry.c                                  |    2 
 lib/pldmfw/pldmfw.c                                 |    2 
 lib/reed_solomon/test_rslib.c                       |    2 
 lib/refcount.c                                      |    2 
 lib/rhashtable.c                                    |    2 
 lib/sbitmap.c                                       |    2 
 lib/scatterlist.c                                   |    4 
 lib/seq_buf.c                                       |    2 
 lib/sort.c                                          |    2 
 lib/stackdepot.c                                    |    2 
 lib/test_bitops.c                                   |    2 
 lib/test_bpf.c                                      |    2 
 lib/test_kasan.c                                    |    2 
 lib/test_kmod.c                                     |    6 
 lib/test_scanf.c                                    |    2 
 lib/vsprintf.c                                      |   10 
 mm/Kconfig                                          |    4 
 mm/Makefile                                         |    1 
 mm/gup.c                                            |   12 
 mm/init-mm.c                                        |    9 
 mm/internal.h                                       |    3 
 mm/mlock.c                                          |    3 
 mm/mmap.c                                           |    5 
 mm/mremap.c                                         |  108 ++++++-
 mm/secretmem.c                                      |  254 +++++++++++++++++
 mm/slub.c                                           |   79 +++--
 scripts/checksyscalls.sh                            |    4 
 scripts/decode_stacktrace.sh                        |   89 +++++-
 tools/testing/selftests/vm/.gitignore               |    1 
 tools/testing/selftests/vm/Makefile                 |    3 
 tools/testing/selftests/vm/memfd_secret.c           |  296 ++++++++++++++++++++
 tools/testing/selftests/vm/mremap_test.c            |  116 ++++---
 tools/testing/selftests/vm/run_vmtests.sh           |   17 +
 137 files changed, 1470 insertions(+), 442 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-07-03  0:28 ` incoming Linus Torvalds
@ 2021-07-03  1:06   ` Linus Torvalds
  0 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2021-07-03  1:06 UTC (permalink / raw)
  To: Andrew Morton; +Cc: Linux-MM, mm-commits

On Fri, Jul 2, 2021 at 5:28 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Commit e058a84bfddc42ba356a2316f2cf1141974625c9 is good, and looking
> at the pulls and merges I've done since, this -mm series looks like
> the obvious culprit.

No, unless my bisection is wrong, the -mm branch is innocent, and was
discarded from the suspects on the very first bisection trial.

So never mind.

             Linus

^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-07-01  1:46 incoming Andrew Morton
@ 2021-07-03  0:28 ` Linus Torvalds
  2021-07-03  1:06   ` incoming Linus Torvalds
  0 siblings, 1 reply; 395+ messages in thread
From: Linus Torvalds @ 2021-07-03  0:28 UTC (permalink / raw)
  To: Andrew Morton; +Cc: Linux-MM, mm-commits

On Wed, Jun 30, 2021 at 6:46 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> This is the rest of the -mm tree, less 66 patches which are dependent on
> things which are (or were recently) in linux-next.  I'll trickle that
> material over next week.

I haven't bisected this yet, but with the current -git I'm getting

   watchdog: BUG: soft lockup - CPU#41 stuck for 49s!

and the common call chain seems to be in flush_tlb_mm_range ->
on_each_cpu_cond_mask.

Commit e058a84bfddc42ba356a2316f2cf1141974625c9 is good, and looking
at the pulls and merges I've done since, this -mm series looks like
the obvious culprit.

I'll go start bisection, but I thought I'd give a heads-up in case
somebody else has seen TLB-flush-related lockups and already figured
out the guilty party..

                 Linus

^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-07-01  1:46 Andrew Morton
  2021-07-03  0:28 ` incoming Linus Torvalds
  0 siblings, 1 reply; 395+ messages in thread
From: Andrew Morton @ 2021-07-01  1:46 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits


This is the rest of the -mm tree, less 66 patches which are dependent on
things which are (or were recently) in linux-next.  I'll trickle that
material over next week.


192 patches, based on 7cf3dead1ad70c72edb03e2d98e1f3dcd332cdb2 plus the
June 28 sendings.

Subsystems affected by this patch series:

  mm/hugetlb
  mm/userfaultfd
  mm/vmscan
  mm/kconfig
  mm/proc
  mm/z3fold
  mm/zbud
  mm/ras
  mm/mempolicy
  mm/memblock
  mm/migration
  mm/thp
  mm/nommu
  mm/kconfig
  mm/madvise
  mm/memory-hotplug
  mm/zswap
  mm/zsmalloc
  mm/zram
  mm/cleanups
  mm/kfence
  mm/hmm
  procfs
  sysctl
  misc
  core-kernel
  lib
  lz4
  checkpatch
  init
  kprobes
  nilfs2
  hfs
  signals
  exec
  kcov
  selftests
  compress/decompress
  ipc

Subsystem: mm/hugetlb

    Muchun Song <songmuchun@bytedance.com>:
    Patch series "Free some vmemmap pages of HugeTLB page", v23:
      mm: memory_hotplug: factor out bootmem core functions to bootmem_info.c
      mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP
      mm: hugetlb: gather discrete indexes of tail page
      mm: hugetlb: free the vmemmap pages associated with each HugeTLB page
      mm: hugetlb: defer freeing of HugeTLB pages
      mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page
      mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap
      mm: memory_hotplug: disable memmap_on_memory when hugetlb_free_vmemmap enabled
      mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate

    Shixin Liu <liushixin2@huawei.com>:
      mm/debug_vm_pgtable: move {pmd/pud}_huge_tests out of CONFIG_TRANSPARENT_HUGEPAGE
      mm/debug_vm_pgtable: remove redundant pfn_{pmd/pte}() and fix one comment mistake

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanup and fixup for huge_memory:, v3:
      mm/huge_memory.c: remove dedicated macro HPAGE_CACHE_INDEX_MASK
      mm/huge_memory.c: use page->deferred_list
      mm/huge_memory.c: add missing read-only THP checking in transparent_hugepage_enabled()
      mm/huge_memory.c: remove unnecessary tlb_remove_page_size() for huge zero pmd
      mm/huge_memory.c: don't discard hugepage if other processes are mapping it

    Christophe Leroy <christophe.leroy@csgroup.eu>:
    Patch series "Subject: [PATCH v2 0/5] Implement huge VMAP and VMALLOC on powerpc 8xx", v2:
      mm/hugetlb: change parameters of arch_make_huge_pte()
      mm/pgtable: add stubs for {pmd/pub}_{set/clear}_huge
      mm/vmalloc: enable mapping of huge pages at pte level in vmap
      mm/vmalloc: enable mapping of huge pages at pte level in vmalloc
      powerpc/8xx: add support for huge pages on VMAP and VMALLOC

    Nanyong Sun <sunnanyong@huawei.com>:
      khugepaged: selftests: remove debug_cow

    Mina Almasry <almasrymina@google.com>:
      mm, hugetlb: fix racy resv_huge_pages underflow on UFFDIO_COPY

    Muchun Song <songmuchun@bytedance.com>:
    Patch series "Split huge PMD mapping of vmemmap pages", v4:
      mm: sparsemem: split the huge PMD mapping of vmemmap pages
      mm: sparsemem: use huge PMD mapping for vmemmap pages
      mm: hugetlb: introduce CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON

    Mike Kravetz <mike.kravetz@oracle.com>:
    Patch series "Fix prep_compound_gigantic_page ref count adjustment":
      hugetlb: remove prep_compound_huge_page cleanup
      hugetlb: address ref count racing in prep_compound_gigantic_page

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm/hwpoison: disable pcp for page_handle_poison()

Subsystem: mm/userfaultfd

    Peter Xu <peterx@redhat.com>:
    Patch series "userfaultfd/selftests: A few cleanups", v2:
      userfaultfd/selftests: use user mode only
      userfaultfd/selftests: remove the time() check on delayed uffd
      userfaultfd/selftests: dropping VERIFY check in locking_thread
      userfaultfd/selftests: only dump counts if mode enabled
      userfaultfd/selftests: unify error handling
    Patch series "mm/uffd: Misc fix for uffd-wp and one more test":
      mm/thp: simplify copying of huge zero page pmd when fork
      mm/userfaultfd: fix uffd-wp special cases for fork()
      mm/userfaultfd: fail uffd-wp registration if not supported
      mm/pagemap: export uffd-wp protection information
      userfaultfd/selftests: add pagemap uffd-wp test

    Axel Rasmussen <axelrasmussen@google.com>:
    Patch series "userfaultfd: add minor fault handling for shmem", v6:
      userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte
      userfaultfd/shmem: support minor fault registration for shmem
      userfaultfd/shmem: support UFFDIO_CONTINUE for shmem
      userfaultfd/shmem: advertise shmem minor fault support
      userfaultfd/shmem: modify shmem_mfill_atomic_pte to use install_pte()
      userfaultfd/selftests: use memfd_create for shmem test type
      userfaultfd/selftests: create alias mappings in the shmem test
      userfaultfd/selftests: reinitialize test context in each test
      userfaultfd/selftests: exercise minor fault handling shmem support

Subsystem: mm/vmscan

    Yu Zhao <yuzhao@google.com>:
      mm/vmscan.c: fix potential deadlock in reclaim_pages()
      include/trace/events/vmscan.h: remove mm_vmscan_inactive_list_is_low

    Miaohe Lin <linmiaohe@huawei.com>:
      mm: workingset: define macro WORKINGSET_SHIFT

Subsystem: mm/kconfig

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      mm/kconfig: move HOLES_IN_ZONE into mm

Subsystem: mm/proc

    Mike Rapoport <rppt@linux.ibm.com>:
      docs: proc.rst: meminfo: briefly describe gaps in memory accounting

    David Hildenbrand <david@redhat.com>:
    Patch series "fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages", v3:
      fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER
      fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM
      fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages
      mm: introduce page_offline_(begin|end|freeze|thaw) to synchronize setting PageOffline()
      virtio-mem: use page_offline_(start|end) when setting PageOffline()
      fs/proc/kcore: use page_offline_(freeze|thaw)

Subsystem: mm/z3fold

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanup and fixup for z3fold":
      mm/z3fold: define macro NCHUNKS as TOTAL_CHUNKS - ZHDR_CHUNKS
      mm/z3fold: avoid possible underflow in z3fold_alloc()
      mm/z3fold: remove magic number in z3fold_create_pool()
      mm/z3fold: remove unused function handle_to_z3fold_header()
      mm/z3fold: fix potential memory leak in z3fold_destroy_pool()
      mm/z3fold: use release_z3fold_page_locked() to release locked z3fold page

Subsystem: mm/zbud

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanups for zbud", v2:
      mm/zbud: reuse unbuddied[0] as buddied in zbud_pool
      mm/zbud: don't export any zbud API

Subsystem: mm/ras

    YueHaibing <yuehaibing@huawei.com>:
      mm/compaction: use DEVICE_ATTR_WO macro

    Liu Xiang <liu.xiang@zlingsmart.com>:
      mm: compaction: remove duplicate !list_empty(&sublist) check

    Wonhyuk Yang <vvghjk1234@gmail.com>:
      mm/compaction: fix 'limit' in fast_isolate_freepages

Subsystem: mm/mempolicy

    Feng Tang <feng.tang@intel.com>:
    Patch series "mm/mempolicy: some fix and semantics cleanup", v4:
      mm/mempolicy: cleanup nodemask intersection check for oom
      mm/mempolicy: don't handle MPOL_LOCAL like a fake MPOL_PREFERRED policy
      mm/mempolicy: unify the parameter sanity check for mbind and set_mempolicy

    Yang Shi <shy828301@gmail.com>:
      mm: mempolicy: don't have to split pmd for huge zero page

    Ben Widawsky <ben.widawsky@intel.com>:
      mm/mempolicy: use unified 'nodes' for bind/interleave/prefer policies

Subsystem: mm/memblock

    Mike Rapoport <rppt@linux.ibm.com>:
    Patch series "arm64: drop pfn_valid_within() and simplify pfn_valid()", v4:
      include/linux/mmzone.h: add documentation for pfn_valid()
      memblock: update initialization of reserved pages
      arm64: decouple check whether pfn is in linear map from pfn_valid()
      arm64: drop pfn_valid_within() and simplify pfn_valid()

    Anshuman Khandual <anshuman.khandual@arm.com>:
      arm64/mm: drop HAVE_ARCH_PFN_VALID

Subsystem: mm/migration

    Muchun Song <songmuchun@bytedance.com>:
      mm: migrate: fix missing update page_private to hugetlb_page_subpool

Subsystem: mm/thp

    Collin Fijalkovich <cfijalkovich@google.com>:
      mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs

    Yang Shi <shy828301@gmail.com>:
      mm: memory: add orig_pmd to struct vm_fault
      mm: memory: make numa_migrate_prep() non-static
      mm: thp: refactor NUMA fault handling
      mm: migrate: account THP NUMA migration counters correctly
      mm: migrate: don't split THP for misplaced NUMA page
      mm: migrate: check mapcount for THP instead of refcount
      mm: thp: skip make PMD PROT_NONE if THP migration is not supported

    Anshuman Khandual <anshuman.khandual@arm.com>:
      mm/thp: make ARCH_ENABLE_SPLIT_PMD_PTLOCK dependent on PGTABLE_LEVELS > 2

    Yang Shi <shy828301@gmail.com>:
      mm: rmap: make try_to_unmap() void function

    Hugh Dickins <hughd@google.com>:
      mm/thp: remap_page() is only needed on anonymous THP
      mm: hwpoison_user_mappings() try_to_unmap() with TTU_SYNC

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm/thp: fix strncpy warning

Subsystem: mm/nommu

    Chen Li <chenli@uniontech.com>:
      nommu: remove __GFP_HIGHMEM in vmalloc/vzalloc

    Liam Howlett <liam.howlett@oracle.com>:
      mm/nommu: unexport do_munmap()

Subsystem: mm/kconfig

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      mm: generalize ZONE_[DMA|DMA32]

Subsystem: mm/madvise

    David Hildenbrand <david@redhat.com>:
    Patch series "mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables", v2:
      mm: make variable names for populate_vma_page_range() consistent
      mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables
      MAINTAINERS: add tools/testing/selftests/vm/ to MEMORY MANAGEMENT
      selftests/vm: add protection_keys_32 / protection_keys_64 to gitignore
      selftests/vm: add test for MADV_POPULATE_(READ|WRITE)

Subsystem: mm/memory-hotplug

    Liam Mark <lmark@codeaurora.org>:
      mm/memory_hotplug: rate limit page migration warnings

    Oscar Salvador <osalvador@suse.de>:
      mm,memory_hotplug: drop unneeded locking

Subsystem: mm/zswap

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanup and fixup for zswap":
      mm/zswap.c: remove unused function zswap_debugfs_exit()
      mm/zswap.c: avoid unnecessary copy-in at map time
      mm/zswap.c: fix two bugs in zswap_writeback_entry()

Subsystem: mm/zsmalloc

    Zhaoyang Huang <zhaoyang.huang@unisoc.com>:
      mm: zram: amend SLAB_RECLAIM_ACCOUNT on zspage_cachep

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "Cleanup for zsmalloc":
      mm/zsmalloc.c: remove confusing code in obj_free()
      mm/zsmalloc.c: improve readability for async_free_zspage()

Subsystem: mm/zram

    Yue Hu <huyue2@yulong.com>:
      zram: move backing_dev under macro CONFIG_ZRAM_WRITEBACK

Subsystem: mm/cleanups

    Hyeonggon Yoo <42.hyeyoo@gmail.com>:
      mm: fix typos and grammar error in comments

    Anshuman Khandual <anshuman.khandual@arm.com>:
      mm: define default value for FIRST_USER_ADDRESS

    Zhen Lei <thunder.leizhen@huawei.com>:
      mm: fix spelling mistakes

    Mel Gorman <mgorman@techsingularity.net>:
    Patch series "Clean W=1 build warnings for mm/":
      mm/vmscan: remove kerneldoc-like comment from isolate_lru_pages
      mm/vmalloc: include header for prototype of set_iounmap_nonlazy
      mm/page_alloc: make should_fail_alloc_page() static
      mm/mapping_dirty_helpers: remove double Note in kerneldoc
      mm/memcontrol.c: fix kerneldoc comment for mem_cgroup_calculate_protection
      mm/memory_hotplug: fix kerneldoc comment for __try_online_node
      mm/memory_hotplug: fix kerneldoc comment for __remove_memory
      mm/zbud: add kerneldoc fields for zbud_pool
      mm/z3fold: add kerneldoc fields for z3fold_pool
      mm/swap: make swap_address_space an inline function
      mm/mmap_lock: remove dead code for !CONFIG_TRACING configurations
      mm/page_alloc: move prototype for find_suitable_fallback
      mm/swap: make NODE_DATA an inline function on CONFIG_FLATMEM

    Anshuman Khandual <anshuman.khandual@arm.com>:
      mm/thp: define default pmd_pgtable()

Subsystem: mm/kfence

    Marco Elver <elver@google.com>:
      kfence: unconditionally use unbound work queue

Subsystem: mm/hmm

    Alistair Popple <apopple@nvidia.com>:
    Patch series "Add support for SVM atomics in Nouveau", v11:
      mm: remove special swap entry functions
      mm/swapops: rework swap entry manipulation code
      mm/rmap: split try_to_munlock from try_to_unmap
      mm/rmap: split migration into its own function
      mm: rename migrate_pgmap_owner
      mm/memory.c: allow different return codes for copy_nonpresent_pte()
      mm: device exclusive memory access
      mm: selftests for exclusive device memory
      nouveau/svm: refactor nouveau_range_fault
      nouveau/svm: implement atomic SVM access

Subsystem: procfs

    Marcelo Henrique Cerri <marcelo.cerri@canonical.com>:
      proc: Avoid mixing integer types in mem_rw()

    ZHOUFENG <zhoufeng.zf@bytedance.com>:
      fs/proc/kcore.c: add mmap interface

    Kalesh Singh <kaleshsingh@google.com>:
      procfs: allow reading fdinfo with PTRACE_MODE_READ
      procfs/dmabuf: add inode number to /proc/*/fdinfo

Subsystem: sysctl

    Jiapeng Chong <jiapeng.chong@linux.alibaba.com>:
      sysctl: remove redundant assignment to first

Subsystem: misc

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
      drm: include only needed headers in ascii85.h

Subsystem: core-kernel

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
      kernel.h: split out panic and oops helpers

Subsystem: lib

    Zhen Lei <thunder.leizhen@huawei.com>:
      lib: decompress_bunzip2: remove an unneeded semicolon

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
    Patch series "lib/string_helpers: get rid of ugly *_escape_mem_ascii()", v3:
      lib/string_helpers: switch to use BIT() macro
      lib/string_helpers: move ESCAPE_NP check inside 'else' branch in a loop
      lib/string_helpers: drop indentation level in string_escape_mem()
      lib/string_helpers: introduce ESCAPE_NA for escaping non-ASCII
      lib/string_helpers: introduce ESCAPE_NAP to escape non-ASCII and non-printable
      lib/string_helpers: allow to append additional characters to be escaped
      lib/test-string_helpers: print flags in hexadecimal format
      lib/test-string_helpers: get rid of trailing comma in terminators
      lib/test-string_helpers: add test cases for new features
      MAINTAINERS: add myself as designated reviewer for generic string library
      seq_file: introduce seq_escape_mem()
      seq_file: add seq_escape_str() as replica of string_escape_str()
      seq_file: convert seq_escape() to use seq_escape_str()
      nfsd: avoid non-flexible API in seq_quote_mem()
      seq_file: drop unused *_escape_mem_ascii()

    Trent Piepho <tpiepho@gmail.com>:
      lib/math/rational.c: fix divide by zero
      lib/math/rational: add Kunit test cases

    Zhen Lei <thunder.leizhen@huawei.com>:
      lib/decompressors: fix spelling mistakes
      lib/mpi: fix spelling mistakes

    Alexey Dobriyan <adobriyan@gmail.com>:
      lib: memscan() fixlet
      lib: uninline simple_strtoull()

    Matteo Croce <mcroce@microsoft.com>:
      lib/test_string.c: allow module removal

    Andy Shevchenko <andriy.shevchenko@linux.intel.com>:
      kernel.h: split out kstrtox() and simple_strtox() to a separate header

Subsystem: lz4

    Rajat Asthana <thisisrast7@gmail.com>:
      lz4_decompress: declare LZ4_decompress_safe_withPrefix64k static

    Dimitri John Ledkov <dimitri.ledkov@canonical.com>:
      lib/decompress_unlz4.c: correctly handle zero-padding around initrds.

Subsystem: checkpatch

    Guenter Roeck <linux@roeck-us.net>:
      checkpatch: scripts/spdxcheck.py now requires python3

    Joe Perches <joe@perches.com>:
      checkpatch: improve the indented label test

    Guenter Roeck <linux@roeck-us.net>:
      checkpatch: do not complain about positive return values starting with EPOLL

Subsystem: init

    Andrew Halaney <ahalaney@redhat.com>:
      init: print out unknown kernel parameters

Subsystem: kprobes

    Barry Song <song.bao.hua@hisilicon.com>:
      kprobes: remove duplicated strong free_insn_page in x86 and s390

Subsystem: nilfs2

    Colin Ian King <colin.king@canonical.com>:
      nilfs2: remove redundant continue statement in a while-loop

Subsystem: hfs

    Zhen Lei <thunder.leizhen@huawei.com>:
      hfsplus: remove unnecessary oom message

    Chung-Chiang Cheng <shepjeng@gmail.com>:
      hfsplus: report create_date to kstat.btime

Subsystem: signals

    Al Viro <viro@zeniv.linux.org.uk>:
      x86: signal: don't do sas_ss_reset() until we are certain that sigframe won't be abandoned

Subsystem: exec

    Alexey Dobriyan <adobriyan@gmail.com>:
      exec: remove checks in __register_bimfmt()

Subsystem: kcov

    Marco Elver <elver@google.com>:
      kcov: add __no_sanitize_coverage to fix noinstr for all architectures

Subsystem: selftests

    Dave Hansen <dave.hansen@linux.intel.com>:
    Patch series "selftests/vm/pkeys: Bug fixes and a new test":
      selftests/vm/pkeys: fix alloc_random_pkey() to make it really, really random
      selftests/vm/pkeys: handle negative sys_pkey_alloc() return code
      selftests/vm/pkeys: refill shadow register after implicit kernel write
      selftests/vm/pkeys: exercise x86 XSAVE init state

Subsystem: compress/decompress

    Yu Kuai <yukuai3@huawei.com>:
      lib/decompressors: remove set but not used variabled 'level'

Subsystem: ipc

    Vasily Averin <vvs@virtuozzo.com>:
    Patch series "ipc: allocations cleanup", v2:
      ipc sem: use kvmalloc for sem_undo allocation
      ipc: use kmalloc for msg_queue and shmid_kernel

    Manfred Spraul <manfred@colorfullife.com>:
      ipc/sem.c: use READ_ONCE()/WRITE_ONCE() for use_global_lock
      ipc/util.c: use binary search for max_idx

 Documentation/admin-guide/kernel-parameters.txt    |   35 
 Documentation/admin-guide/mm/hugetlbpage.rst       |   11 
 Documentation/admin-guide/mm/memory-hotplug.rst    |   13 
 Documentation/admin-guide/mm/pagemap.rst           |    2 
 Documentation/admin-guide/mm/userfaultfd.rst       |    3 
 Documentation/core-api/kernel-api.rst              |    7 
 Documentation/filesystems/proc.rst                 |   48 
 Documentation/vm/hmm.rst                           |   19 
 Documentation/vm/unevictable-lru.rst               |   33 
 MAINTAINERS                                        |   10 
 arch/alpha/Kconfig                                 |    5 
 arch/alpha/include/asm/pgalloc.h                   |    1 
 arch/alpha/include/asm/pgtable.h                   |    1 
 arch/alpha/include/uapi/asm/mman.h                 |    3 
 arch/alpha/kernel/setup.c                          |    2 
 arch/arc/include/asm/pgalloc.h                     |    2 
 arch/arc/include/asm/pgtable.h                     |    8 
 arch/arm/Kconfig                                   |    3 
 arch/arm/include/asm/pgalloc.h                     |    1 
 arch/arm64/Kconfig                                 |   15 
 arch/arm64/include/asm/hugetlb.h                   |    3 
 arch/arm64/include/asm/memory.h                    |    2 
 arch/arm64/include/asm/page.h                      |    4 
 arch/arm64/include/asm/pgalloc.h                   |    1 
 arch/arm64/include/asm/pgtable.h                   |    2 
 arch/arm64/kernel/setup.c                          |    1 
 arch/arm64/kvm/mmu.c                               |    2 
 arch/arm64/mm/hugetlbpage.c                        |    5 
 arch/arm64/mm/init.c                               |   51 
 arch/arm64/mm/ioremap.c                            |    4 
 arch/arm64/mm/mmu.c                                |   22 
 arch/csky/include/asm/pgalloc.h                    |    2 
 arch/csky/include/asm/pgtable.h                    |    1 
 arch/hexagon/include/asm/pgtable.h                 |    4 
 arch/ia64/Kconfig                                  |    7 
 arch/ia64/include/asm/pal.h                        |    1 
 arch/ia64/include/asm/pgalloc.h                    |    1 
 arch/ia64/include/asm/pgtable.h                    |    1 
 arch/m68k/Kconfig                                  |    5 
 arch/m68k/include/asm/mcf_pgalloc.h                |    2 
 arch/m68k/include/asm/mcf_pgtable.h                |    2 
 arch/m68k/include/asm/motorola_pgalloc.h           |    1 
 arch/m68k/include/asm/motorola_pgtable.h           |    2 
 arch/m68k/include/asm/pgtable_mm.h                 |    1 
 arch/m68k/include/asm/sun3_pgalloc.h               |    1 
 arch/microblaze/Kconfig                            |    4 
 arch/microblaze/include/asm/pgalloc.h              |    2 
 arch/microblaze/include/asm/pgtable.h              |    2 
 arch/mips/Kconfig                                  |   10 
 arch/mips/include/asm/pgalloc.h                    |    1 
 arch/mips/include/asm/pgtable-32.h                 |    1 
 arch/mips/include/asm/pgtable-64.h                 |    1 
 arch/mips/include/uapi/asm/mman.h                  |    3 
 arch/mips/kernel/relocate.c                        |    1 
 arch/mips/sgi-ip22/ip22-reset.c                    |    1 
 arch/mips/sgi-ip32/ip32-reset.c                    |    1 
 arch/nds32/include/asm/pgalloc.h                   |    5 
 arch/nios2/include/asm/pgalloc.h                   |    1 
 arch/nios2/include/asm/pgtable.h                   |    2 
 arch/openrisc/include/asm/pgalloc.h                |    2 
 arch/openrisc/include/asm/pgtable.h                |    1 
 arch/parisc/include/asm/pgalloc.h                  |    1 
 arch/parisc/include/asm/pgtable.h                  |    2 
 arch/parisc/include/uapi/asm/mman.h                |    3 
 arch/parisc/kernel/pdc_chassis.c                   |    1 
 arch/powerpc/Kconfig                               |    6 
 arch/powerpc/include/asm/book3s/pgtable.h          |    1 
 arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h   |    5 
 arch/powerpc/include/asm/nohash/32/mmu-8xx.h       |   43 
 arch/powerpc/include/asm/nohash/32/pgtable.h       |    1 
 arch/powerpc/include/asm/nohash/64/pgtable.h       |    2 
 arch/powerpc/include/asm/pgalloc.h                 |    5 
 arch/powerpc/include/asm/pgtable.h                 |    6 
 arch/powerpc/kernel/setup-common.c                 |    1 
 arch/powerpc/platforms/Kconfig.cputype             |    1 
 arch/riscv/Kconfig                                 |    5 
 arch/riscv/include/asm/pgalloc.h                   |    2 
 arch/riscv/include/asm/pgtable.h                   |    2 
 arch/s390/Kconfig                                  |    6 
 arch/s390/include/asm/pgalloc.h                    |    3 
 arch/s390/include/asm/pgtable.h                    |    5 
 arch/s390/kernel/ipl.c                             |    1 
 arch/s390/kernel/kprobes.c                         |    5 
 arch/s390/mm/pgtable.c                             |    2 
 arch/sh/include/asm/pgalloc.h                      |    1 
 arch/sh/include/asm/pgtable.h                      |    2 
 arch/sparc/Kconfig                                 |    5 
 arch/sparc/include/asm/pgalloc_32.h                |    1 
 arch/sparc/include/asm/pgalloc_64.h                |    1 
 arch/sparc/include/asm/pgtable_32.h                |    3 
 arch/sparc/include/asm/pgtable_64.h                |    8 
 arch/sparc/kernel/sstate.c                         |    1 
 arch/sparc/mm/hugetlbpage.c                        |    6 
 arch/sparc/mm/init_64.c                            |    1 
 arch/um/drivers/mconsole_kern.c                    |    1 
 arch/um/include/asm/pgalloc.h                      |    1 
 arch/um/include/asm/pgtable-2level.h               |    1 
 arch/um/include/asm/pgtable-3level.h               |    1 
 arch/um/kernel/um_arch.c                           |    1 
 arch/x86/Kconfig                                   |   17 
 arch/x86/include/asm/desc.h                        |    1 
 arch/x86/include/asm/pgalloc.h                     |    2 
 arch/x86/include/asm/pgtable_types.h               |    2 
 arch/x86/kernel/cpu/mshyperv.c                     |    1 
 arch/x86/kernel/kprobes/core.c                     |    6 
 arch/x86/kernel/setup.c                            |    1 
 arch/x86/mm/init_64.c                              |   21 
 arch/x86/mm/pgtable.c                              |   34 
 arch/x86/purgatory/purgatory.c                     |    2 
 arch/x86/xen/enlighten.c                           |    1 
 arch/xtensa/include/asm/pgalloc.h                  |    2 
 arch/xtensa/include/asm/pgtable.h                  |    1 
 arch/xtensa/include/uapi/asm/mman.h                |    3 
 arch/xtensa/platforms/iss/setup.c                  |    1 
 drivers/block/zram/zram_drv.h                      |    2 
 drivers/bus/brcmstb_gisb.c                         |    1 
 drivers/char/ipmi/ipmi_msghandler.c                |    1 
 drivers/clk/analogbits/wrpll-cln28hpc.c            |    4 
 drivers/edac/altera_edac.c                         |    1 
 drivers/firmware/google/gsmi.c                     |    1 
 drivers/gpu/drm/nouveau/include/nvif/if000c.h      |    1 
 drivers/gpu/drm/nouveau/nouveau_svm.c              |  162 ++-
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h      |    1 
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c |    6 
 drivers/hv/vmbus_drv.c                             |    1 
 drivers/hwtracing/coresight/coresight-cpu-debug.c  |    1 
 drivers/leds/trigger/ledtrig-activity.c            |    1 
 drivers/leds/trigger/ledtrig-heartbeat.c           |    1 
 drivers/leds/trigger/ledtrig-panic.c               |    1 
 drivers/misc/bcm-vk/bcm_vk_dev.c                   |    1 
 drivers/misc/ibmasm/heartbeat.c                    |    1 
 drivers/misc/pvpanic/pvpanic.c                     |    1 
 drivers/net/ipa/ipa_smp2p.c                        |    1 
 drivers/parisc/power.c                             |    1 
 drivers/power/reset/ltc2952-poweroff.c             |    1 
 drivers/remoteproc/remoteproc_core.c               |    1 
 drivers/s390/char/con3215.c                        |    1 
 drivers/s390/char/con3270.c                        |    1 
 drivers/s390/char/sclp.c                           |    1 
 drivers/s390/char/sclp_con.c                       |    1 
 drivers/s390/char/sclp_vt220.c                     |    1 
 drivers/s390/char/zcore.c                          |    1 
 drivers/soc/bcm/brcmstb/pm/pm-arm.c                |    1 
 drivers/staging/olpc_dcon/olpc_dcon.c              |    1 
 drivers/video/fbdev/hyperv_fb.c                    |    1 
 drivers/virtio/virtio_mem.c                        |    2 
 fs/Kconfig                                         |   15 
 fs/exec.c                                          |    3 
 fs/hfsplus/inode.c                                 |    5 
 fs/hfsplus/xattr.c                                 |    1 
 fs/nfsd/nfs4state.c                                |    2 
 fs/nilfs2/btree.c                                  |    1 
 fs/open.c                                          |   13 
 fs/proc/base.c                                     |    6 
 fs/proc/fd.c                                       |   20 
 fs/proc/kcore.c                                    |  136 ++
 fs/proc/task_mmu.c                                 |   34 
 fs/seq_file.c                                      |   43 
 fs/userfaultfd.c                                   |   15 
 include/asm-generic/bug.h                          |    3 
 include/linux/ascii85.h                            |    3 
 include/linux/bootmem_info.h                       |   68 +
 include/linux/compat.h                             |    2 
 include/linux/compiler-clang.h                     |   17 
 include/linux/compiler-gcc.h                       |    6 
 include/linux/compiler_types.h                     |    2 
 include/linux/huge_mm.h                            |   74 -
 include/linux/hugetlb.h                            |   80 +
 include/linux/hugetlb_cgroup.h                     |   19 
 include/linux/kcore.h                              |    3 
 include/linux/kernel.h                             |  227 ----
 include/linux/kprobes.h                            |    1 
 include/linux/kstrtox.h                            |  155 ++
 include/linux/memblock.h                           |    4 
 include/linux/memory_hotplug.h                     |   27 
 include/linux/mempolicy.h                          |    9 
 include/linux/memremap.h                           |    2 
 include/linux/migrate.h                            |   27 
 include/linux/mm.h                                 |   18 
 include/linux/mm_types.h                           |    2 
 include/linux/mmu_notifier.h                       |   26 
 include/linux/mmzone.h                             |   27 
 include/linux/mpi.h                                |    4 
 include/linux/page-flags.h                         |   22 
 include/linux/panic.h                              |   98 +
 include/linux/panic_notifier.h                     |   12 
 include/linux/pgtable.h                            |   44 
 include/linux/rmap.h                               |   13 
 include/linux/seq_file.h                           |   10 
 include/linux/shmem_fs.h                           |   19 
 include/linux/signal.h                             |    2 
 include/linux/string.h                             |    7 
 include/linux/string_helpers.h                     |   31 
 include/linux/sunrpc/cache.h                       |    1 
 include/linux/swap.h                               |   19 
 include/linux/swapops.h                            |  171 +--
 include/linux/thread_info.h                        |    1 
 include/linux/userfaultfd_k.h                      |    5 
 include/linux/vmalloc.h                            |   15 
 include/linux/zbud.h                               |   23 
 include/trace/events/vmscan.h                      |   41 
 include/uapi/asm-generic/mman-common.h             |    3 
 include/uapi/linux/mempolicy.h                     |    1 
 include/uapi/linux/userfaultfd.h                   |    7 
 init/main.c                                        |   42 
 ipc/msg.c                                          |    6 
 ipc/sem.c                                          |   25 
 ipc/shm.c                                          |    6 
 ipc/util.c                                         |   44 
 ipc/util.h                                         |    3 
 kernel/hung_task.c                                 |    1 
 kernel/kexec_core.c                                |    1 
 kernel/kprobes.c                                   |    2 
 kernel/panic.c                                     |    1 
 kernel/rcu/tree.c                                  |    2 
 kernel/signal.c                                    |   14 
 kernel/sysctl.c                                    |    4 
 kernel/trace/trace.c                               |    1 
 lib/Kconfig.debug                                  |   12 
 lib/decompress_bunzip2.c                           |    6 
 lib/decompress_unlz4.c                             |    8 
 lib/decompress_unlzo.c                             |    3 
 lib/decompress_unxz.c                              |    2 
 lib/decompress_unzstd.c                            |    4 
 lib/kstrtox.c                                      |    5 
 lib/lz4/lz4_decompress.c                           |    2 
 lib/math/Makefile                                  |    1 
 lib/math/rational-test.c                           |   56 +
 lib/math/rational.c                                |   16 
 lib/mpi/longlong.h                                 |    4 
 lib/mpi/mpicoder.c                                 |    6 
 lib/mpi/mpiutil.c                                  |    2 
 lib/parser.c                                       |    1 
 lib/string.c                                       |    2 
 lib/string_helpers.c                               |  142 +-
 lib/test-string_helpers.c                          |  157 ++-
 lib/test_hmm.c                                     |  127 ++
 lib/test_hmm_uapi.h                                |    2 
 lib/test_string.c                                  |    5 
 lib/vsprintf.c                                     |    1 
 lib/xz/xz_dec_bcj.c                                |    2 
 lib/xz/xz_dec_lzma2.c                              |    8 
 lib/zlib_inflate/inffast.c                         |    2 
 lib/zstd/huf.h                                     |    2 
 mm/Kconfig                                         |   16 
 mm/Makefile                                        |    2 
 mm/bootmem_info.c                                  |  127 ++
 mm/compaction.c                                    |   20 
 mm/debug_vm_pgtable.c                              |  109 --
 mm/gup.c                                           |   58 +
 mm/hmm.c                                           |   12 
 mm/huge_memory.c                                   |  269 ++---
 mm/hugetlb.c                                       |  369 +++++--
 mm/hugetlb_vmemmap.c                               |  332 ++++++
 mm/hugetlb_vmemmap.h                               |   53 -
 mm/internal.h                                      |   29 
 mm/kfence/core.c                                   |    4 
 mm/khugepaged.c                                    |   20 
 mm/madvise.c                                       |   66 +
 mm/mapping_dirty_helpers.c                         |    2 
 mm/memblock.c                                      |   28 
 mm/memcontrol.c                                    |    4 
 mm/memory-failure.c                                |   38 
 mm/memory.c                                        |  239 +++-
 mm/memory_hotplug.c                                |  161 ---
 mm/mempolicy.c                                     |  323 ++----
 mm/migrate.c                                       |  268 +----
 mm/mlock.c                                         |   12 
 mm/mmap_lock.c                                     |   59 -
 mm/mprotect.c                                      |   18 
 mm/nommu.c                                         |    5 
 mm/oom_kill.c                                      |    2 
 mm/page_alloc.c                                    |    5 
 mm/page_vma_mapped.c                               |   15 
 mm/rmap.c                                          |  644 +++++++++---
 mm/shmem.c                                         |  125 --
 mm/sparse-vmemmap.c                                |  432 +++++++-
 mm/sparse.c                                        |    1 
 mm/swap.c                                          |    2 
 mm/swapfile.c                                      |    2 
 mm/userfaultfd.c                                   |  249 ++--
 mm/util.c                                          |   40 
 mm/vmalloc.c                                       |   37 
 mm/vmscan.c                                        |   20 
 mm/workingset.c                                    |   10 
 mm/z3fold.c                                        |   39 
 mm/zbud.c                                          |  235 ++--
 mm/zsmalloc.c                                      |    5 
 mm/zswap.c                                         |   26 
 scripts/checkpatch.pl                              |   16 
 tools/testing/selftests/vm/.gitignore              |    3 
 tools/testing/selftests/vm/Makefile                |    5 
 tools/testing/selftests/vm/hmm-tests.c             |  158 +++
 tools/testing/selftests/vm/khugepaged.c            |    4 
 tools/testing/selftests/vm/madv_populate.c         |  342 ++++++
 tools/testing/selftests/vm/pkey-x86.h              |    1 
 tools/testing/selftests/vm/protection_keys.c       |   85 +
 tools/testing/selftests/vm/run_vmtests.sh          |   16 
 tools/testing/selftests/vm/userfaultfd.c           | 1094 ++++++++++-----------
 299 files changed, 6277 insertions(+), 3183 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-06-29  2:32 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-06-29  2:32 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

192 patches, based on 7cf3dead1ad70c72edb03e2d98e1f3dcd332cdb2.

Subsystems affected by this patch series:

  mm/gup
  mm/pagealloc
  kthread
  ia64
  scripts
  ntfs
  squashfs
  ocfs2
  z
  kernel/watchdog
  mm/slab
  mm/slub
  mm/kmemleak
  mm/dax
  mm/debug
  mm/pagecache
  mm/gup
  mm/swap
  mm/memcg
  mm/pagemap
  mm/mprotect
  mm/bootmem
  mm/dma
  mm/tracing
  mm/vmalloc
  mm/kasan
  mm/initialization
  mm/pagealloc
  mm/memory-failure

Subsystem: mm/gup

    Jann Horn <jannh@google.com>:
      mm/gup: fix try_grab_compound_head() race with split_huge_page()

Subsystem: mm/pagealloc

    Mike Rapoport <rppt@linux.ibm.com>:
      mm/page_alloc: fix memory map initialization for descending nodes

    Mel Gorman <mgorman@techsingularity.net>:
      mm/page_alloc: correct return value of populated elements if bulk array is populated

Subsystem: kthread

    Jonathan Neuschäfer <j.neuschaefer@gmx.net>:
      kthread: switch to new kerneldoc syntax for named variable macro argument

    Petr Mladek <pmladek@suse.com>:
      kthread_worker: fix return value when kthread_mod_delayed_work() races with kthread_cancel_delayed_work_sync()

Subsystem: ia64

    Randy Dunlap <rdunlap@infradead.org>:
      ia64: headers: drop duplicated words

    Arnd Bergmann <arnd@arndb.de>:
      ia64: mca_drv: fix incorrect array size calculation

Subsystem: scripts

    "Steven Rostedt (VMware)" <rostedt@goodmis.org>:
    Patch series "streamline_config.pl: Fix Perl spacing":
      streamline_config.pl: make spacing consistent
      streamline_config.pl: add softtabstop=4 for vim users

    Colin Ian King <colin.king@canonical.com>:
      scripts/spelling.txt: add more spellings to spelling.txt

Subsystem: ntfs

    Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>:
      ntfs: fix validity check for file name attribute

Subsystem: squashfs

    Vincent Whitchurch <vincent.whitchurch@axis.com>:
      squashfs: add option to panic on errors

Subsystem: ocfs2

    Yang Yingliang <yangyingliang@huawei.com>:
      ocfs2: remove unnecessary INIT_LIST_HEAD()

Subsystem: z

    Dan Carpenter <dan.carpenter@oracle.com>:
      ocfs2: fix snprintf() checking

    Colin Ian King <colin.king@canonical.com>:
      ocfs2: remove redundant assignment to pointer queue

    Wan Jiabing <wanjiabing@vivo.com>:
      ocfs2: remove repeated uptodate check for buffer

    Chen Huang <chenhuang5@huawei.com>:
      ocfs2: replace simple_strtoull() with kstrtoull()

    Colin Ian King <colin.king@canonical.com>:
      ocfs2: remove redundant initialization of variable ret

Subsystem: kernel/watchdog

    Wang Qing <wangqing@vivo.com>:
      kernel: watchdog: modify the explanation related to watchdog thread
      doc: watchdog: modify the explanation related to watchdog thread
      doc: watchdog: modify the doc related to "watchdog/%u"

Subsystem: mm/slab

    gumingtao <gumingtao1225@gmail.com>:
      slab: use __func__ to trace function name

Subsystem: mm/slub

    Vlastimil Babka <vbabka@suse.cz>:
      kunit: make test->lock irq safe

    Oliver Glitta <glittao@gmail.com>:
      mm/slub, kunit: add a KUnit test for SLUB debugging functionality
      slub: remove resiliency_test() function

    Hyeonggon Yoo <42.hyeyoo@gmail.com>:
      mm, slub: change run-time assertion in kmalloc_index() to compile-time

    Stephen Boyd <swboyd@chromium.org>:
      slub: restore slub_debug=- behavior
      slub: actually use 'message' in restore_bytes()

    Joe Perches <joe@perches.com>:
      slub: indicate slab_fix() uses printf formats

    Stephen Boyd <swboyd@chromium.org>:
      slub: force on no_hash_pointers when slub_debug is enabled

    Faiyaz Mohammed <faiyazm@codeaurora.org>:
      mm: slub: move sysfs slab alloc/free interfaces to debugfs

    Georgi Djakov <quic_c_gdjako@quicinc.com>:
      mm/slub: add taint after the errors are printed

Subsystem: mm/kmemleak

    Yanfei Xu <yanfei.xu@windriver.com>:
      mm/kmemleak: fix possible wrong memory scanning period

Subsystem: mm/dax

    Jan Kara <jack@suse.cz>:
      dax: fix ENOMEM handling in grab_mapping_entry()

Subsystem: mm/debug

    Tang Bin <tangbin@cmss.chinamobile.com>:
      tools/vm/page_owner_sort.c: check malloc() return

    Anshuman Khandual <anshuman.khandual@arm.com>:
      mm/debug_vm_pgtable: ensure THP availability via has_transparent_hugepage()

    Nicolas Saenz Julienne <nsaenzju@redhat.com>:
      mm: mmap_lock: use local locks instead of disabling preemption

    Gavin Shan <gshan@redhat.com>:
    Patch series "mm/page_reporting: Make page reporting work on arm64 with 64KB page size", v4:
      mm/page_reporting: fix code style in __page_reporting_request()
      mm/page_reporting: export reporting order as module parameter
      mm/page_reporting: allow driver to specify reporting order
      virtio_balloon: specify page reporting order if needed

Subsystem: mm/pagecache

    Kefeng Wang <wangkefeng.wang@huawei.com>:
      mm: page-writeback: kill get_writeback_state() comments

    Chi Wu <wuchi.zero@gmail.com>:
      mm/page-writeback: Fix performance when BDI's share of ratio is 0.
      mm/page-writeback: update the comment of Dirty position control
      mm/page-writeback: use __this_cpu_inc() in account_page_dirtied()

    Roman Gushchin <guro@fb.com>:
    Patch series "cgroup, blkcg: prevent dirty inodes to pin dying memory cgroups", v9:
      writeback, cgroup: do not switch inodes with I_WILL_FREE flag
      writeback, cgroup: add smp_mb() to cgroup_writeback_umount()
      writeback, cgroup: increment isw_nr_in_flight before grabbing an inode
      writeback, cgroup: switch to rcu_work API in inode_switch_wbs()
      writeback, cgroup: keep list of inodes attached to bdi_writeback
      writeback, cgroup: split out the functional part of inode_switch_wbs_work_fn()
      writeback, cgroup: support switching multiple inodes at once
      writeback, cgroup: release dying cgwbs by switching attached inodes

    Christoph Hellwig <hch@lst.de>:
    Patch series "remove the implicit .set_page_dirty default":
      fs: unexport __set_page_dirty
      fs: move ramfs_aops to libfs
      mm: require ->set_page_dirty to be explicitly wired up

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
    Patch series "Further set_page_dirty cleanups":
      mm/writeback: move __set_page_dirty() to core mm
      mm/writeback: use __set_page_dirty in __set_page_dirty_nobuffers
      iomap: use __set_page_dirty_nobuffers
      fs: remove anon_set_page_dirty()
      fs: remove noop_set_page_dirty()
      mm: move page dirtying prototypes from mm.h

Subsystem: mm/gup

    Peter Xu <peterx@redhat.com>:
    Patch series "mm/gup: Fix pin page write cache bouncing on has_pinned", v2:
      mm/gup_benchmark: support threading

    Andrea Arcangeli <aarcange@redhat.com>:
      mm: gup: allow FOLL_PIN to scale in SMP
      mm: gup: pack has_pinned in MMF_HAS_PINNED

    Christophe Leroy <christophe.leroy@csgroup.eu>:
      mm: pagewalk: fix walk for hugepage tables

Subsystem: mm/swap

    Miaohe Lin <linmiaohe@huawei.com>:
    Patch series "close various race windows for swap", v6:
      mm/swapfile: use percpu_ref to serialize against concurrent swapoff
      swap: fix do_swap_page() race with swapoff
      mm/swap: remove confusing checking for non_swap_entry() in swap_ra_info()
      mm/shmem: fix shmem_swapin() race with swapoff
    Patch series "Cleanups for swap", v2:
      mm/swapfile: move get_swap_page_of_type() under CONFIG_HIBERNATION
      mm/swap: remove unused local variable nr_shadows
      mm/swap_slots.c: delete meaningless forward declarations

    Huang Ying <ying.huang@intel.com>:
      mm, swap: remove unnecessary smp_rmb() in swap_type_to_swap_info()
      mm: free idle swap cache page after COW
      swap: check mapping_empty() for swap cache before being freed

Subsystem: mm/memcg

    Waiman Long <longman@redhat.com>:
    Patch series "mm/memcg: Reduce kmemcache memory accounting overhead", v6:
      mm/memcg: move mod_objcg_state() to memcontrol.c
      mm/memcg: cache vmstat data in percpu memcg_stock_pcp
      mm/memcg: improve refill_obj_stock() performance
      mm/memcg: optimize user context object stock access
    Patch series "mm: memcg/slab: Fix objcg pointer array handling problem", v4:
      mm: memcg/slab: properly set up gfp flags for objcg pointer array
      mm: memcg/slab: create a new set of kmalloc-cg-<n> caches
      mm: memcg/slab: disable cache merging for KMALLOC_NORMAL caches

    Muchun Song <songmuchun@bytedance.com>:
      mm: memcontrol: fix root_mem_cgroup charging
    Patch series "memcontrol code cleanup and simplification", v3:
      mm: memcontrol: fix page charging in page replacement
      mm: memcontrol: bail out early when !mm in get_mem_cgroup_from_mm
      mm: memcontrol: remove the pgdata parameter of mem_cgroup_page_lruvec
      mm: memcontrol: simplify lruvec_holds_page_lru_lock
      mm: memcontrol: rename lruvec_holds_page_lru_lock to page_matches_lruvec
      mm: memcontrol: simplify the logic of objcg pinning memcg
      mm: memcontrol: move obj_cgroup_uncharge_pages() out of css_set_lock
      mm: vmscan: remove noinline_for_stack

    wenhuizhang <wenhui@gwmail.gwu.edu>:
      memcontrol: use flexible-array member

    Dan Schatzberg <schatzberg.dan@gmail.com>:
    Patch series "Charge loop device i/o to issuing cgroup", v14:
      loop: use worker per cgroup instead of kworker
      mm: charge active memcg when no mm is set
      loop: charge i/o to mem and blk cg

    Huilong Deng <denghuilong@cdjrlc.com>:
      mm: memcontrol: remove trailing semicolon in macros

Subsystem: mm/pagemap

    David Hildenbrand <david@redhat.com>:
    Patch series "perf/binfmt/mm: remove in-tree usage of MAP_EXECUTABLE":
      perf: MAP_EXECUTABLE does not indicate VM_MAYEXEC
      binfmt: remove in-tree usage of MAP_EXECUTABLE
      mm: ignore MAP_EXECUTABLE in ksys_mmap_pgoff()

    Gonzalo Matias Juarez Tello <gmjuareztello@gmail.com>:
      mm/mmap.c: logic of find_vma_intersection repeated in __do_munmap

    Liam Howlett <liam.howlett@oracle.com>:
      mm/mmap: introduce unlock_range() for code cleanup
      mm/mmap: use find_vma_intersection() in do_mmap() for overlap

    Liu Xiang <liu.xiang@zlingsmart.com>:
      mm/memory.c: fix comment of finish_mkwrite_fault()

    Liam Howlett <liam.howlett@oracle.com>:
    Patch series "mm: Add vma_lookup()", v2:
      mm: add vma_lookup(), update find_vma_intersection() comments
      drm/i915/selftests: use vma_lookup() in __igt_mmap()
      arch/arc/kernel/troubleshoot: use vma_lookup() instead of find_vma()
      arch/arm64/kvm: use vma_lookup() instead of find_vma_intersection()
      arch/powerpc/kvm/book3s_hv_uvmem: use vma_lookup() instead of find_vma_intersection()
      arch/powerpc/kvm/book3s: use vma_lookup() in kvmppc_hv_setup_htab_rma()
      arch/mips/kernel/traps: use vma_lookup() instead of find_vma()
      arch/m68k/kernel/sys_m68k: use vma_lookup() in sys_cacheflush()
      x86/sgx: use vma_lookup() in sgx_encl_find()
      virt/kvm: use vma_lookup() instead of find_vma_intersection()
      vfio: use vma_lookup() instead of find_vma_intersection()
      net/ipv5/tcp: use vma_lookup() in tcp_zerocopy_receive()
      drm/amdgpu: use vma_lookup() in amdgpu_ttm_tt_get_user_pages()
      media: videobuf2: use vma_lookup() in get_vaddr_frames()
      misc/sgi-gru/grufault: use vma_lookup() in gru_find_vma()
      kernel/events/uprobes: use vma_lookup() in find_active_uprobe()
      lib/test_hmm: use vma_lookup() in dmirror_migrate()
      mm/ksm: use vma_lookup() in find_mergeable_vma()
      mm/migrate: use vma_lookup() in do_pages_stat_array()
      mm/mremap: use vma_lookup() in vma_to_resize()
      mm/memory.c: use vma_lookup() in __access_remote_vm()
      mm/mempolicy: use vma_lookup() in __access_remote_vm()

    Chen Li <chenli@uniontech.com>:
      mm: update legacy flush_tlb_* to use vma

Subsystem: mm/mprotect

    Peter Collingbourne <pcc@google.com>:
      mm: improve mprotect(R|W) efficiency on pages referenced once

Subsystem: mm/bootmem

    Souptick Joarder <jrdr.linux@gmail.com>:
      h8300: remove unused variable

Subsystem: mm/dma

    YueHaibing <yuehaibing@huawei.com>:
      mm/dmapool: use DEVICE_ATTR_RO macro

Subsystem: mm/tracing

    Vincent Whitchurch <vincent.whitchurch@axis.com>:
      mm, tracing: unify PFN format strings

Subsystem: mm/vmalloc

    "Uladzislau Rezki (Sony)" <urezki@gmail.com>:
    Patch series "vmalloc() vs bulk allocator", v2:
      mm/page_alloc: add an alloc_pages_bulk_array_node() helper
      mm/vmalloc: switch to bulk allocator in __vmalloc_area_node()
      mm/vmalloc: print a warning message first on failure
      mm/vmalloc: remove quoted strings split across lines

    Uladzislau Rezki <urezki@gmail.com>:
      mm/vmalloc: fallback to a single page allocator

    Rafael Aquini <aquini@redhat.com>:
      mm: vmalloc: add cond_resched() in __vunmap()

Subsystem: mm/kasan

    Alexander Potapenko <glider@google.com>:
      printk: introduce dump_stack_lvl()
      kasan: use dump_stack_lvl(KERN_ERR) to print stacks

    David Gow <davidgow@google.com>:
      kasan: test: improve failure message in KUNIT_EXPECT_KASAN_FAIL()

    Daniel Axtens <dja@axtens.net>:
    Patch series "KASAN core changes for ppc64 radix KASAN", v16:
      kasan: allow an architecture to disable inline instrumentation
      kasan: allow architectures to provide an outline readiness check
      mm: define default MAX_PTRS_PER_* in include/pgtable.h
      kasan: use MAX_PTRS_PER_* for early shadow tables

    Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>:
    Patch series "kasan: add memory corruption identification support for hw tag-based kasan", v4:
      kasan: rename CONFIG_KASAN_SW_TAGS_IDENTIFY to CONFIG_KASAN_TAGS_IDENTIFY
      kasan: integrate the common part of two KASAN tag-based modes
      kasan: add memory corruption identification support for hardware tag-based mode

Subsystem: mm/initialization

    Jungseung Lee <js07.lee@samsung.com>:
      mm: report which part of mem is being freed on initmem case

Subsystem: mm/pagealloc

    Mike Rapoport <rppt@linux.ibm.com>:
      mm/mmzone.h: simplify is_highmem_idx()

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
    Patch series "Constify struct page arguments":
      mm: make __dump_page static

    Aaron Tomlin <atomlin@redhat.com>:
      mm/page_alloc: bail out on fatal signal during reclaim/compaction retry attempt

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm/debug: factor PagePoisoned out of __dump_page
      mm/page_owner: constify dump_page_owner
      mm: make compound_head const-preserving
      mm: constify get_pfnblock_flags_mask and get_pfnblock_migratetype
      mm: constify page_count and page_ref_count
      mm: optimise nth_page for contiguous memmap

    Heiner Kallweit <hkallweit1@gmail.com>:
      mm/page_alloc: switch to pr_debug

    Andrii Nakryiko <andrii@kernel.org>:
      kbuild: skip per-CPU BTF generation for pahole v1.18-v1.21

    Mel Gorman <mgorman@techsingularity.net>:
      mm/page_alloc: split per cpu page lists and zone stats
      mm/page_alloc: convert per-cpu list protection to local_lock
      mm/vmstat: convert NUMA statistics to basic NUMA counters
      mm/vmstat: inline NUMA event counter updates
      mm/page_alloc: batch the accounting updates in the bulk allocator
      mm/page_alloc: reduce duration that IRQs are disabled for VM counters
      mm/page_alloc: explicitly acquire the zone lock in __free_pages_ok
      mm/page_alloc: avoid conflating IRQs disabled with zone->lock
      mm/page_alloc: update PGFREE outside the zone lock in __free_pages_ok

    Minchan Kim <minchan@kernel.org>:
      mm: page_alloc: dump migrate-failed pages only at -EBUSY

    Mel Gorman <mgorman@techsingularity.net>:
    Patch series "Calculate pcp->high based on zone sizes and active CPUs", v2:
      mm/page_alloc: delete vm.percpu_pagelist_fraction
      mm/page_alloc: disassociate the pcp->high from pcp->batch
      mm/page_alloc: adjust pcp->high after CPU hotplug events
      mm/page_alloc: scale the number of pages that are batch freed
      mm/page_alloc: limit the number of pages on PCP lists when reclaim is active
      mm/page_alloc: introduce vm.percpu_pagelist_high_fraction

    Dong Aisheng <aisheng.dong@nxp.com>:
      mm: drop SECTION_SHIFT in code comments
      mm/page_alloc: improve memmap_pages dbg msg

    Liu Shixin <liushixin2@huawei.com>:
      mm/page_alloc: fix counting of managed_pages

    Mel Gorman <mgorman@techsingularity.net>:
    Patch series "Allow high order pages to be stored on PCP", v2:
      mm/page_alloc: move free_the_page

    Mike Rapoport <rppt@linux.ibm.com>:
    Patch series "Remove DISCONTIGMEM memory model", v3:
      alpha: remove DISCONTIGMEM and NUMA
      arc: update comment about HIGHMEM implementation
      arc: remove support for DISCONTIGMEM
      m68k: remove support for DISCONTIGMEM
      mm: remove CONFIG_DISCONTIGMEM
      arch, mm: remove stale mentions of DISCONIGMEM
      docs: remove description of DISCONTIGMEM
      mm: replace CONFIG_NEED_MULTIPLE_NODES with CONFIG_NUMA
      mm: replace CONFIG_FLAT_NODE_MEM_MAP with CONFIG_FLATMEM

    Mel Gorman <mgorman@techsingularity.net>:
      mm/page_alloc: allow high-order pages to be stored on the per-cpu lists
      mm/page_alloc: split pcp->high across all online CPUs for cpuless nodes

Subsystem: mm/memory-failure

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm,hwpoison: send SIGBUS with error virutal address
      mm,hwpoison: make get_hwpoison_page() call get_any_page()

 Documentation/admin-guide/kernel-parameters.txt    |    6 
 Documentation/admin-guide/lockup-watchdogs.rst     |    4 
 Documentation/admin-guide/sysctl/kernel.rst        |   10 
 Documentation/admin-guide/sysctl/vm.rst            |   52 -
 Documentation/dev-tools/kasan.rst                  |    9 
 Documentation/vm/memory-model.rst                  |   45 
 arch/alpha/Kconfig                                 |   22 
 arch/alpha/include/asm/machvec.h                   |    6 
 arch/alpha/include/asm/mmzone.h                    |  100 --
 arch/alpha/include/asm/pgtable.h                   |    4 
 arch/alpha/include/asm/topology.h                  |   39 
 arch/alpha/kernel/core_marvel.c                    |   53 -
 arch/alpha/kernel/core_wildfire.c                  |   29 
 arch/alpha/kernel/pci_iommu.c                      |   29 
 arch/alpha/kernel/proto.h                          |    8 
 arch/alpha/kernel/setup.c                          |   16 
 arch/alpha/kernel/sys_marvel.c                     |    5 
 arch/alpha/kernel/sys_wildfire.c                   |    5 
 arch/alpha/mm/Makefile                             |    2 
 arch/alpha/mm/init.c                               |    3 
 arch/alpha/mm/numa.c                               |  223 ----
 arch/arc/Kconfig                                   |   13 
 arch/arc/include/asm/mmzone.h                      |   40 
 arch/arc/kernel/troubleshoot.c                     |    8 
 arch/arc/mm/init.c                                 |   21 
 arch/arm/include/asm/tlbflush.h                    |   13 
 arch/arm/mm/tlb-v6.S                               |    2 
 arch/arm/mm/tlb-v7.S                               |    2 
 arch/arm64/Kconfig                                 |    2 
 arch/arm64/kvm/mmu.c                               |    2 
 arch/h8300/kernel/setup.c                          |    2 
 arch/ia64/Kconfig                                  |    2 
 arch/ia64/include/asm/pal.h                        |    2 
 arch/ia64/include/asm/spinlock.h                   |    2 
 arch/ia64/include/asm/uv/uv_hub.h                  |    2 
 arch/ia64/kernel/efi_stub.S                        |    2 
 arch/ia64/kernel/mca_drv.c                         |    2 
 arch/ia64/kernel/topology.c                        |    5 
 arch/ia64/mm/numa.c                                |    5 
 arch/m68k/Kconfig.cpu                              |   10 
 arch/m68k/include/asm/mmzone.h                     |   10 
 arch/m68k/include/asm/page.h                       |    2 
 arch/m68k/include/asm/page_mm.h                    |   35 
 arch/m68k/include/asm/tlbflush.h                   |    2 
 arch/m68k/kernel/sys_m68k.c                        |    4 
 arch/m68k/mm/init.c                                |   20 
 arch/mips/Kconfig                                  |    2 
 arch/mips/include/asm/mmzone.h                     |    8 
 arch/mips/include/asm/page.h                       |    2 
 arch/mips/kernel/traps.c                           |    4 
 arch/mips/mm/init.c                                |    7 
 arch/nds32/include/asm/memory.h                    |    6 
 arch/openrisc/include/asm/tlbflush.h               |    2 
 arch/powerpc/Kconfig                               |    2 
 arch/powerpc/include/asm/mmzone.h                  |    4 
 arch/powerpc/kernel/setup_64.c                     |    2 
 arch/powerpc/kernel/smp.c                          |    2 
 arch/powerpc/kexec/core.c                          |    4 
 arch/powerpc/kvm/book3s_hv.c                       |    4 
 arch/powerpc/kvm/book3s_hv_uvmem.c                 |    2 
 arch/powerpc/mm/Makefile                           |    2 
 arch/powerpc/mm/mem.c                              |    4 
 arch/riscv/Kconfig                                 |    2 
 arch/s390/Kconfig                                  |    2 
 arch/s390/include/asm/pgtable.h                    |    2 
 arch/sh/include/asm/mmzone.h                       |    4 
 arch/sh/kernel/topology.c                          |    2 
 arch/sh/mm/Kconfig                                 |    2 
 arch/sh/mm/init.c                                  |    2 
 arch/sparc/Kconfig                                 |    2 
 arch/sparc/include/asm/mmzone.h                    |    4 
 arch/sparc/kernel/smp_64.c                         |    2 
 arch/sparc/mm/init_64.c                            |   12 
 arch/x86/Kconfig                                   |    2 
 arch/x86/ia32/ia32_aout.c                          |    4 
 arch/x86/kernel/cpu/mce/core.c                     |   13 
 arch/x86/kernel/cpu/sgx/encl.h                     |    4 
 arch/x86/kernel/setup_percpu.c                     |    6 
 arch/x86/mm/init_32.c                              |    4 
 arch/xtensa/include/asm/page.h                     |    4 
 arch/xtensa/include/asm/tlbflush.h                 |    4 
 drivers/base/node.c                                |   18 
 drivers/block/loop.c                               |  270 ++++-
 drivers/block/loop.h                               |   15 
 drivers/dax/device.c                               |    2 
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c            |    4 
 drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c |    2 
 drivers/media/common/videobuf2/frame_vector.c      |    2 
 drivers/misc/sgi-gru/grufault.c                    |    4 
 drivers/vfio/vfio_iommu_type1.c                    |    2 
 drivers/virtio/virtio_balloon.c                    |   17 
 fs/adfs/inode.c                                    |    1 
 fs/affs/file.c                                     |    2 
 fs/bfs/file.c                                      |    1 
 fs/binfmt_aout.c                                   |    4 
 fs/binfmt_elf.c                                    |    2 
 fs/binfmt_elf_fdpic.c                              |   11 
 fs/binfmt_flat.c                                   |    2 
 fs/block_dev.c                                     |    1 
 fs/buffer.c                                        |   25 
 fs/configfs/inode.c                                |    8 
 fs/dax.c                                           |    3 
 fs/ecryptfs/mmap.c                                 |   13 
 fs/exfat/inode.c                                   |    1 
 fs/ext2/inode.c                                    |    4 
 fs/ext4/inode.c                                    |    2 
 fs/fat/inode.c                                     |    1 
 fs/fs-writeback.c                                  |  366 +++++---
 fs/fuse/dax.c                                      |    3 
 fs/gfs2/aops.c                                     |    2 
 fs/gfs2/meta_io.c                                  |    2 
 fs/hfs/inode.c                                     |    2 
 fs/hfsplus/inode.c                                 |    2 
 fs/hpfs/file.c                                     |    1 
 fs/iomap/buffered-io.c                             |   27 
 fs/jfs/inode.c                                     |    1 
 fs/kernfs/inode.c                                  |    8 
 fs/libfs.c                                         |   44 
 fs/minix/inode.c                                   |    1 
 fs/nilfs2/mdt.c                                    |    1 
 fs/ntfs/inode.c                                    |    2 
 fs/ocfs2/aops.c                                    |    4 
 fs/ocfs2/cluster/heartbeat.c                       |    7 
 fs/ocfs2/cluster/nodemanager.c                     |    2 
 fs/ocfs2/dlm/dlmmaster.c                           |    2 
 fs/ocfs2/filecheck.c                               |    6 
 fs/ocfs2/stackglue.c                               |    8 
 fs/omfs/file.c                                     |    1 
 fs/proc/task_mmu.c                                 |    2 
 fs/ramfs/inode.c                                   |    9 
 fs/squashfs/block.c                                |    5 
 fs/squashfs/squashfs_fs_sb.h                       |    1 
 fs/squashfs/super.c                                |   86 +
 fs/sysv/itree.c                                    |    1 
 fs/udf/file.c                                      |    1 
 fs/udf/inode.c                                     |    1 
 fs/ufs/inode.c                                     |    1 
 fs/xfs/xfs_aops.c                                  |    4 
 fs/zonefs/super.c                                  |    4 
 include/asm-generic/memory_model.h                 |   37 
 include/asm-generic/pgtable-nop4d.h                |    1 
 include/asm-generic/topology.h                     |    2 
 include/kunit/test.h                               |    5 
 include/linux/backing-dev-defs.h                   |   20 
 include/linux/cpuhotplug.h                         |    2 
 include/linux/fs.h                                 |    6 
 include/linux/gfp.h                                |   13 
 include/linux/iomap.h                              |    1 
 include/linux/kasan.h                              |    7 
 include/linux/kernel.h                             |    2 
 include/linux/kthread.h                            |    2 
 include/linux/memblock.h                           |    6 
 include/linux/memcontrol.h                         |   60 -
 include/linux/mm.h                                 |   53 -
 include/linux/mm_types.h                           |   10 
 include/linux/mman.h                               |    2 
 include/linux/mmdebug.h                            |    3 
 include/linux/mmzone.h                             |   96 +-
 include/linux/page-flags.h                         |   10 
 include/linux/page_owner.h                         |    6 
 include/linux/page_ref.h                           |    4 
 include/linux/page_reporting.h                     |    3 
 include/linux/pageblock-flags.h                    |    2 
 include/linux/pagemap.h                            |    4 
 include/linux/pgtable.h                            |   22 
 include/linux/printk.h                             |    5 
 include/linux/sched/coredump.h                     |    8 
 include/linux/slab.h                               |   59 +
 include/linux/swap.h                               |   19 
 include/linux/swapops.h                            |    5 
 include/linux/vmstat.h                             |   69 -
 include/linux/writeback.h                          |    1 
 include/trace/events/cma.h                         |    4 
 include/trace/events/filemap.h                     |    2 
 include/trace/events/kmem.h                        |   12 
 include/trace/events/page_pool.h                   |    4 
 include/trace/events/pagemap.h                     |    4 
 include/trace/events/vmscan.h                      |    2 
 kernel/cgroup/cgroup.c                             |    1 
 kernel/crash_core.c                                |    4 
 kernel/events/core.c                               |    2 
 kernel/events/uprobes.c                            |    4 
 kernel/fork.c                                      |    1 
 kernel/kthread.c                                   |   19 
 kernel/sysctl.c                                    |   16 
 kernel/watchdog.c                                  |   12 
 lib/Kconfig.debug                                  |   15 
 lib/Kconfig.kasan                                  |   16 
 lib/Makefile                                       |    1 
 lib/dump_stack.c                                   |   20 
 lib/kunit/test.c                                   |   18 
 lib/slub_kunit.c                                   |  152 +++
 lib/test_hmm.c                                     |    5 
 lib/test_kasan.c                                   |   11 
 lib/vsprintf.c                                     |    2 
 mm/Kconfig                                         |   38 
 mm/backing-dev.c                                   |   66 +
 mm/compaction.c                                    |    2 
 mm/debug.c                                         |   27 
 mm/debug_vm_pgtable.c                              |   63 +
 mm/dmapool.c                                       |    5 
 mm/filemap.c                                       |    2 
 mm/gup.c                                           |   81 +
 mm/hugetlb.c                                       |    2 
 mm/internal.h                                      |    9 
 mm/kasan/Makefile                                  |    4 
 mm/kasan/common.c                                  |    6 
 mm/kasan/generic.c                                 |    3 
 mm/kasan/hw_tags.c                                 |   22 
 mm/kasan/init.c                                    |    6 
 mm/kasan/kasan.h                                   |   12 
 mm/kasan/report.c                                  |    6 
 mm/kasan/report_hw_tags.c                          |    5 
 mm/kasan/report_sw_tags.c                          |   45 
 mm/kasan/report_tags.c                             |   51 +
 mm/kasan/shadow.c                                  |    6 
 mm/kasan/sw_tags.c                                 |   45 
 mm/kasan/tags.c                                    |   59 +
 mm/kfence/kfence_test.c                            |    5 
 mm/kmemleak.c                                      |   18 
 mm/ksm.c                                           |    6 
 mm/memblock.c                                      |    8 
 mm/memcontrol.c                                    |  385 ++++++--
 mm/memory-failure.c                                |  344 +++++--
 mm/memory.c                                        |   22 
 mm/memory_hotplug.c                                |    6 
 mm/mempolicy.c                                     |    4 
 mm/migrate.c                                       |    4 
 mm/mmap.c                                          |   54 -
 mm/mmap_lock.c                                     |   33 
 mm/mprotect.c                                      |   52 +
 mm/mremap.c                                        |    5 
 mm/nommu.c                                         |    2 
 mm/page-writeback.c                                |   89 +
 mm/page_alloc.c                                    |  950 +++++++++++++--------
 mm/page_ext.c                                      |    2 
 mm/page_owner.c                                    |    2 
 mm/page_reporting.c                                |   19 
 mm/page_reporting.h                                |    5 
 mm/pagewalk.c                                      |   58 +
 mm/shmem.c                                         |   18 
 mm/slab.h                                          |   24 
 mm/slab_common.c                                   |   60 -
 mm/slub.c                                          |  420 +++++----
 mm/sparse.c                                        |    2 
 mm/swap.c                                          |    4 
 mm/swap_slots.c                                    |    2 
 mm/swap_state.c                                    |   20 
 mm/swapfile.c                                      |  177 +--
 mm/vmalloc.c                                       |  181 ++--
 mm/vmscan.c                                        |   43 
 mm/vmstat.c                                        |  282 ++----
 mm/workingset.c                                    |    2 
 net/ipv4/tcp.c                                     |    4 
 scripts/kconfig/streamline_config.pl               |   76 -
 scripts/link-vmlinux.sh                            |    4 
 scripts/spelling.txt                               |   16 
 tools/testing/selftests/vm/gup_test.c              |   96 +-
 tools/vm/page_owner_sort.c                         |    4 
 virt/kvm/kvm_main.c                                |    2 
 260 files changed, 3989 insertions(+), 2996 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-06-25  1:38 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-06-25  1:38 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

24 patches, based on 4a09d388f2ab382f217a764e6a152b3f614246f6.

Subsystems affected by this patch series:

  mm/thp
  nilfs2
  mm/vmalloc
  kthread
  mm/hugetlb
  mm/memory-failure
  mm/pagealloc
  MAINTAINERS
  mailmap

Subsystem: mm/thp

    Hugh Dickins <hughd@google.com>:
    Patch series "mm: page_vma_mapped_walk() cleanup and THP fixes":
      mm: page_vma_mapped_walk(): use page for pvmw->page
      mm: page_vma_mapped_walk(): settle PageHuge on entry
      mm: page_vma_mapped_walk(): use pmde for *pvmw->pmd
      mm: page_vma_mapped_walk(): prettify PVMW_MIGRATION block
      mm: page_vma_mapped_walk(): crossing page table boundary
      mm: page_vma_mapped_walk(): add a level of indentation
      mm: page_vma_mapped_walk(): use goto instead of while (1)
      mm: page_vma_mapped_walk(): get vma_address_end() earlier
      mm/thp: fix page_vma_mapped_walk() if THP mapped by ptes
      mm/thp: another PVMW_SYNC fix in page_vma_mapped_walk()

Subsystem: nilfs2

    Pavel Skripkin <paskripkin@gmail.com>:
      nilfs2: fix memory leak in nilfs_sysfs_delete_device_group

Subsystem: mm/vmalloc

    Claudio Imbrenda <imbrenda@linux.ibm.com>:
    Patch series "mm: add vmalloc_no_huge and use it", v4:
      mm/vmalloc: add vmalloc_no_huge
      KVM: s390: prepare for hugepage vmalloc

    Daniel Axtens <dja@axtens.net>:
      mm/vmalloc: unbreak kasan vmalloc support

Subsystem: kthread

    Petr Mladek <pmladek@suse.com>:
    Patch series "kthread_worker: Fix race between kthread_mod_delayed_work():
      kthread_worker: split code for canceling the delayed work timer
      kthread: prevent deadlock when kthread_mod_delayed_work() races with kthread_cancel_delayed_work_sync()

Subsystem: mm/hugetlb

    Hugh Dickins <hughd@google.com>:
      mm, futex: fix shared futex pgoff on shmem huge page

Subsystem: mm/memory-failure

    Tony Luck <tony.luck@intel.com>:
    Patch series "mm,hwpoison: fix sending SIGBUS for Action Required MCE", v5:
      mm/memory-failure: use a mutex to avoid memory_failure() races

    Aili Yao <yaoaili@kingsoft.com>:
      mm,hwpoison: return -EHWPOISON to denote that the page has already been poisoned

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm/hwpoison: do not lock page again when me_huge_page() successfully recovers

Subsystem: mm/pagealloc

    Rasmus Villemoes <linux@rasmusvillemoes.dk>:
      mm/page_alloc: __alloc_pages_bulk(): do bounds check before accessing array

    Mel Gorman <mgorman@techsingularity.net>:
      mm/page_alloc: do bulk array bounds check after checking populated elements

Subsystem: MAINTAINERS

    Marek Behún <kabel@kernel.org>:
      MAINTAINERS: fix Marek's identity again

Subsystem: mailmap

    Marek Behún <kabel@kernel.org>:
      mailmap: add Marek's other e-mail address and identity without diacritics

 .mailmap                |    2 
 MAINTAINERS             |    4 
 arch/s390/kvm/pv.c      |    7 +
 fs/nilfs2/sysfs.c       |    1 
 include/linux/hugetlb.h |   16 ---
 include/linux/pagemap.h |   13 +-
 include/linux/vmalloc.h |    1 
 kernel/futex.c          |    3 
 kernel/kthread.c        |   81 ++++++++++------
 mm/hugetlb.c            |    5 -
 mm/memory-failure.c     |   83 +++++++++++------
 mm/page_alloc.c         |    6 +
 mm/page_vma_mapped.c    |  233 +++++++++++++++++++++++++++---------------------
 mm/vmalloc.c            |   41 ++++++--
 14 files changed, 297 insertions(+), 199 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-06-16  1:22 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-06-16  1:22 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits


18 patches, based on 94f0b2d4a1d0c52035aef425da5e022bd2cb1c71.

Subsystems affected by this patch series:

  mm/memory-failure
  mm/swap
  mm/slub
  mm/hugetlb
  mm/memory-failure
  coredump
  mm/slub
  mm/thp
  mm/sparsemem

Subsystem: mm/memory-failure

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      mm,hwpoison: fix race with hugetlb page allocation

Subsystem: mm/swap

    Peter Xu <peterx@redhat.com>:
      mm/swap: fix pte_same_as_swp() not removing uffd-wp bit when compare

Subsystem: mm/slub

    Kees Cook <keescook@chromium.org>:
    Patch series "Actually fix freelist pointer vs redzoning", v4:
      mm/slub: clarify verification reporting
      mm/slub: fix redzoning for small allocations
      mm/slub: actually fix freelist pointer vs redzoning

Subsystem: mm/hugetlb

    Mike Kravetz <mike.kravetz@oracle.com>:
      mm/hugetlb: expand restore_reserve_on_error functionality

Subsystem: mm/memory-failure

    yangerkun <yangerkun@huawei.com>:
      mm/memory-failure: make sure wait for page writeback in memory_failure

Subsystem: coredump

    Pingfan Liu <kernelfans@gmail.com>:
      crash_core, vmcoreinfo: append 'SECTION_SIZE_BITS' to vmcoreinfo

Subsystem: mm/slub

    Andrew Morton <akpm@linux-foundation.org>:
      mm/slub.c: include swab.h

Subsystem: mm/thp

    Xu Yu <xuyu@linux.alibaba.com>:
      mm, thp: use head page in __migration_entry_wait()

    Hugh Dickins <hughd@google.com>:
    Patch series "mm/thp: fix THP splitting unmap BUGs and related", v10:
      mm/thp: fix __split_huge_pmd_locked() on shmem migration entry
      mm/thp: make is_huge_zero_pmd() safe and quicker
      mm/thp: try_to_unmap() use TTU_SYNC for safe splitting
      mm/thp: fix vma_address() if virtual address below file offset

    Jue Wang <juew@google.com>:
      mm/thp: fix page_address_in_vma() on file THP tails

    Hugh Dickins <hughd@google.com>:
      mm/thp: unmap_mapping_page() to fix THP truncate_cleanup_page()

    Yang Shi <shy828301@gmail.com>:
      mm: thp: replace DEBUG_VM BUG with VM_WARN when unmap fails for split

Subsystem: mm/sparsemem

    Miles Chen <miles.chen@mediatek.com>:
      mm/sparse: fix check_usemap_section_nr warnings

 Documentation/vm/slub.rst |   10 +--
 fs/hugetlbfs/inode.c      |    1 
 include/linux/huge_mm.h   |    8 ++
 include/linux/hugetlb.h   |    8 ++
 include/linux/mm.h        |    3 +
 include/linux/rmap.h      |    1 
 include/linux/swapops.h   |   15 +++--
 kernel/crash_core.c       |    1 
 mm/huge_memory.c          |   58 ++++++++++---------
 mm/hugetlb.c              |  137 +++++++++++++++++++++++++++++++++++++---------
 mm/internal.h             |   51 ++++++++++++-----
 mm/memory-failure.c       |   36 +++++++++++-
 mm/memory.c               |   41 +++++++++++++
 mm/migrate.c              |    1 
 mm/page_vma_mapped.c      |   27 +++++----
 mm/pgtable-generic.c      |    5 -
 mm/rmap.c                 |   41 +++++++++----
 mm/slab_common.c          |    3 -
 mm/slub.c                 |   37 +++++-------
 mm/sparse.c               |   13 +++-
 mm/swapfile.c             |    2 
 mm/truncate.c             |   43 ++++++--------
 22 files changed, 388 insertions(+), 154 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-06-05  3:00 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-06-05  3:00 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-mm, mm-commits

13 patches, based on 16f0596fc1d78a1f3ae4628cff962bb297dc908c.

Subsystems affected by this patch series:

  mips
  mm/kfence
  init
  mm/debug
  mm/pagealloc
  mm/memory-hotplug
  mm/hugetlb
  proc
  mm/kasan
  mm/hugetlb
  lib
  ocfs2
  mailmap

Subsystem: mips

    Thomas Bogendoerfer <tsbogend@alpha.franken.de>:
      Revert "MIPS: make userspace mapping young by default"

Subsystem: mm/kfence

    Marco Elver <elver@google.com>:
      kfence: use TASK_IDLE when awaiting allocation

Subsystem: init

    Mark Rutland <mark.rutland@arm.com>:
      pid: take a reference when initializing `cad_pid`

Subsystem: mm/debug

    Gerald Schaefer <gerald.schaefer@linux.ibm.com>:
      mm/debug_vm_pgtable: fix alignment for pmd/pud_advanced_tests()

Subsystem: mm/pagealloc

    Ding Hui <dinghui@sangfor.com.cn>:
      mm/page_alloc: fix counting of free pages after take off from buddy

Subsystem: mm/memory-hotplug

    David Hildenbrand <david@redhat.com>:
      drivers/base/memory: fix trying offlining memory blocks with memory holes on aarch64

Subsystem: mm/hugetlb

    Naoya Horiguchi <naoya.horiguchi@nec.com>:
      hugetlb: pass head page to remove_hugetlb_page()

Subsystem: proc

    David Matlack <dmatlack@google.com>:
      proc: add .gitignore for proc-subset-pid selftest

Subsystem: mm/kasan

    Yu Kuai <yukuai3@huawei.com>:
      mm/kasan/init.c: fix doc warning

Subsystem: mm/hugetlb

    Mina Almasry <almasrymina@google.com>:
      mm, hugetlb: fix simple resv_huge_pages underflow on UFFDIO_COPY

Subsystem: lib

    YueHaibing <yuehaibing@huawei.com>:
      lib: crc64: fix kernel-doc warning

Subsystem: ocfs2

    Junxiao Bi <junxiao.bi@oracle.com>:
      ocfs2: fix data corruption by fallocate

Subsystem: mailmap

    Michel Lespinasse <michel@lespinasse.org>:
      mailmap: use private address for Michel Lespinasse

 .mailmap                                |    3 +
 arch/mips/mm/cache.c                    |   30 ++++++++---------
 drivers/base/memory.c                   |    6 +--
 fs/ocfs2/file.c                         |   55 +++++++++++++++++++++++++++++---
 include/linux/pgtable.h                 |    8 ++++
 init/main.c                             |    2 -
 lib/crc64.c                             |    2 -
 mm/debug_vm_pgtable.c                   |    4 +-
 mm/hugetlb.c                            |   16 +++++++--
 mm/kasan/init.c                         |    4 +-
 mm/kfence/core.c                        |    6 +--
 mm/memory.c                             |    4 ++
 mm/page_alloc.c                         |    2 +
 tools/testing/selftests/proc/.gitignore |    1 
 14 files changed, 107 insertions(+), 36 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-05-23  0:41 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-05-23  0:41 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

10 patches, based on 4ff2473bdb4cf2bb7d208ccf4418d3d7e6b1652c.

Subsystems affected by this patch series:

  mm/pagealloc
  mm/gup
  ipc
  selftests
  mm/kasan
  kernel/watchdog
  bitmap
  procfs
  lib
  mm/userfaultfd

Subsystem: mm/pagealloc

    Arnd Bergmann <arnd@arndb.de>:
      mm/shuffle: fix section mismatch warning

Subsystem: mm/gup

    Michal Hocko <mhocko@suse.com>:
      Revert "mm/gup: check page posion status for coredump."

Subsystem: ipc

    Varad Gautam <varad.gautam@suse.com>:
      ipc/mqueue, msg, sem: avoid relying on a stack reference past its expiry

Subsystem: selftests

    Yang Yingliang <yangyingliang@huawei.com>:
      tools/testing/selftests/exec: fix link error

Subsystem: mm/kasan

    Alexander Potapenko <glider@google.com>:
      kasan: slab: always reset the tag in get_freepointer_safe()

Subsystem: kernel/watchdog

    Petr Mladek <pmladek@suse.com>:
      watchdog: reliable handling of timestamps

Subsystem: bitmap

    Rikard Falkeborn <rikard.falkeborn@gmail.com>:
      linux/bits.h: fix compilation error with GENMASK

Subsystem: procfs

    Alexey Dobriyan <adobriyan@gmail.com>:
      proc: remove Alexey from MAINTAINERS

Subsystem: lib

    Zhen Lei <thunder.leizhen@huawei.com>:
      lib: kunit: suppress a compilation warning of frame size

Subsystem: mm/userfaultfd

    Mike Kravetz <mike.kravetz@oracle.com>:
      userfaultfd: hugetlbfs: fix new flag usage in error path

 MAINTAINERS                           |    1 -
 fs/hugetlbfs/inode.c                  |    2 +-
 include/linux/bits.h                  |    2 +-
 include/linux/const.h                 |    8 ++++++++
 include/linux/minmax.h                |   10 ++--------
 ipc/mqueue.c                          |    6 ++++--
 ipc/msg.c                             |    6 ++++--
 ipc/sem.c                             |    6 ++++--
 kernel/watchdog.c                     |   34 ++++++++++++++++++++--------------
 lib/Makefile                          |    1 +
 mm/gup.c                              |    4 ----
 mm/internal.h                         |   20 --------------------
 mm/shuffle.h                          |    4 ++--
 mm/slub.c                             |    1 +
 mm/userfaultfd.c                      |   28 ++++++++++++++--------------
 tools/include/linux/bits.h            |    2 +-
 tools/include/linux/const.h           |    8 ++++++++
 tools/testing/selftests/exec/Makefile |    6 +++---
 18 files changed, 74 insertions(+), 75 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-05-15  0:26 Andrew Morton
  0 siblings, 0 replies; 395+ messages in thread
From: Andrew Morton @ 2021-05-15  0:26 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm

13 patches, based on bd3c9cdb21a2674dd0db70199df884828e37abd4.

Subsystems affected by this patch series:

  mm/hugetlb
  mm/slub
  resource
  squashfs
  mm/userfaultfd
  mm/ksm
  mm/pagealloc
  mm/kasan
  mm/pagemap
  hfsplus
  modprobe
  mm/ioremap

Subsystem: mm/hugetlb

    Peter Xu <peterx@redhat.com>:
    Patch series "mm/hugetlb: Fix issues on file sealing and fork", v2:
      mm/hugetlb: fix F_SEAL_FUTURE_WRITE
      mm/hugetlb: fix cow where page writtable in child

Subsystem: mm/slub

    Vlastimil Babka <vbabka@suse.cz>:
      mm, slub: move slub_debug static key enabling outside slab_mutex

Subsystem: resource

    Alistair Popple <apopple@nvidia.com>:
      kernel/resource: fix return code check in __request_free_mem_region

Subsystem: squashfs

    Phillip Lougher <phillip@squashfs.org.uk>:
      squashfs: fix divide error in calculate_skip()

Subsystem: mm/userfaultfd

    Axel Rasmussen <axelrasmussen@google.com>:
      userfaultfd: release page in error path to avoid BUG_ON

Subsystem: mm/ksm

    Hugh Dickins <hughd@google.com>:
      ksm: revert "use GET_KSM_PAGE_NOLOCK to get ksm page in remove_rmap_item_from_tree()"

Subsystem: mm/pagealloc

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm: fix struct page layout on 32-bit systems

Subsystem: mm/kasan

    Peter Collingbourne <pcc@google.com>:
      kasan: fix unit tests with CONFIG_UBSAN_LOCAL_BOUNDS enabled

Subsystem: mm/pagemap

    "Matthew Wilcox (Oracle)" <willy@infradead.org>:
      mm/filemap: fix readahead return types

Subsystem: hfsplus

    Jouni Roivas <jouni.roivas@tuxera.com>:
      hfsplus: prevent corruption in shrinking truncate

Subsystem: modprobe

    Rasmus Villemoes <linux@rasmusvillemoes.dk>:
      docs: admin-guide: update description for kernel.modprobe sysctl

Subsystem: mm/ioremap

    Christophe Leroy <christophe.leroy@csgroup.eu>:
      mm/ioremap: fix iomap_max_page_shift

 Documentation/admin-guide/sysctl/kernel.rst |    9 ++++---
 fs/hfsplus/extents.c                        |    7 +++--
 fs/hugetlbfs/inode.c                        |    5 ++++
 fs/iomap/buffered-io.c                      |    4 +--
 fs/squashfs/file.c                          |    6 ++--
 include/linux/mm.h                          |   32 ++++++++++++++++++++++++++
 include/linux/mm_types.h                    |    4 +--
 include/linux/pagemap.h                     |    6 ++--
 include/net/page_pool.h                     |   12 +++++++++
 kernel/resource.c                           |    2 -
 lib/test_kasan.c                            |   29 ++++++++++++++++++-----
 mm/hugetlb.c                                |    1 
 mm/ioremap.c                                |    6 ++--
 mm/ksm.c                                    |    3 +-
 mm/shmem.c                                  |   34 ++++++++++++----------------
 mm/slab_common.c                            |   10 ++++++++
 mm/slub.c                                   |    9 -------
 net/core/page_pool.c                        |   12 +++++----
 18 files changed, 129 insertions(+), 62 deletions(-)


^ permalink raw reply	[flat|nested] 395+ messages in thread

* Re: incoming
  2021-05-07  1:01 incoming Andrew Morton
@ 2021-05-07  7:12 ` Linus Torvalds
  0 siblings, 0 replies; 395+ messages in thread
From: Linus Torvalds @ 2021-05-07  7:12 UTC (permalink / raw)
  To: Andrew Morton; +Cc: mm-commits, Linux-MM

On Thu, May 6, 2021 at 6:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> I've been wobbly about the secretmem patches due to doubts about
> whether the feature is sufficiently useful to justify inclusion, but
> developers are now weighing in with helpful information and I've asked Mike
> for an extensively updated [0/n] changelog.  This will take a few days
> to play out so it is possible that I will prevail upon you for a post-rc1
> merge.

Oh, much too late for this release by now.

> If that's a problem, there's always 5.13-rc1.

5.13-rc1 is two days from now, it would be for 5.14-rc1.. How time -
and version numbers - fly.

             Linus

^ permalink raw reply	[flat|nested] 395+ messages in thread

* incoming
@ 2021-05-07  1:01 Andrew Morton
  2021-05-07  7:12 ` incoming Linus Torvalds
  0 siblings, 1 reply; 395+ messages in thread
From: Andrew Morton @ 2021-05-07  1:01 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: mm-commits, linux-mm


This is everything else from -mm for this merge window, with the
possible exception of Mike Rapoport's "secretmem" syscall patch series
(https://lkml.kernel.org/r/20210303162209.8609-1-rppt@kernel.org).

I've been wobbly about the secretmem patches due to doubts about
whether the feature is sufficiently useful to justify inclusion, but
developers are now weighing in with helpful information and I've asked Mike
for an extensively updated [0/n] changelog.  This will take a few days
to play out so it is possible that I will prevail upon you for a post-rc1
merge.  If that's a problem, there's always 5.13-rc1.

91 patches, based on 8ca5297e7e38f2dc8c753d33a5092e7be181fff0, plus
previously sent patches.

Thanks.



Subsystems affected by this patch series:

  alpha
  procfs
  sysctl
  misc
  core-kernel
  bitmap
  lib
  compat
  checkpatch
  epoll
  isofs
  nilfs2
  hpfs
  exit
  fork
  kexec
  gcov
  panic
  delayacct
  gdb
  resource
  selftests
  async
  initramfs
  ipc
  mm/cleanups
  drivers/char
  mm/slub
  spelling

Subsystem: alpha

    Randy Dunlap <rdunlap@infradead.org>: