linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64
@ 2020-11-10 22:09 Andrey Konovalov
  2020-11-10 22:09 ` [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers Andrey Konovalov
                   ` (44 more replies)
  0 siblings, 45 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:09 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This patchset adds a new hardware tag-based mode to KASAN [1]. The new mode
is similar to the existing software tag-based KASAN, but relies on arm64
Memory Tagging Extension (MTE) [2] to perform memory and pointer tagging
(instead of shadow memory and compiler instrumentation).

This patchset is co-developed by
Vincenzo Frascino <vincenzo.frascino@arm.com>.

This patchset is available here:

https://github.com/xairy/linux/tree/up-kasan-mte-v9

and has also been uploaded to the Linux kernel Gerrit instance:

https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/3319

For testing in QEMU hardware tag-based KASAN requires:

1. QEMU built from master [4] (use "-machine virt,mte=on -cpu max" arguments
   to run).
2. GCC version 10.

[1] https://www.kernel.org/doc/html/latest/dev-tools/kasan.html
[2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety
[3] git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/mte
[4] https://github.com/qemu/qemu

====== Overview

The underlying ideas of the approach used by hardware tag-based KASAN are:

1. By relying on the Top Byte Ignore (TBI) arm64 CPU feature, pointer tags
   are stored in the top byte of each kernel pointer.

2. With the Memory Tagging Extension (MTE) arm64 CPU feature, memory tags
   for kernel memory allocations are stored in a dedicated memory not
   accessible via normal instuctions.

3. On each memory allocation, a random tag is generated, embedded it into
   the returned pointer, and the corresponding memory is tagged with the
   same tag value.

4. With MTE the CPU performs a check on each memory access to make sure
   that the pointer tag matches the memory tag.

5. On a tag mismatch the CPU generates a tag fault, and a KASAN report is
   printed.

Same as other KASAN modes, hardware tag-based KASAN is intended as a
debugging feature at this point.

====== Rationale

There are two main reasons for this new hardware tag-based mode:

1. Previously implemented software tag-based KASAN is being successfully
   used on dogfood testing devices due to its low memory overhead (as
   initially planned). The new hardware mode keeps the same low memory
   overhead, and is expected to have significantly lower performance
   impact, due to the tag checks being performed by the hardware.
   Therefore the new mode can be used as a better alternative in dogfood
   testing for hardware that supports MTE.

2. The new mode lays the groundwork for the planned in-kernel MTE-based
   memory corruption mitigation to be used in production.

====== Technical details

From the implementation perspective, hardware tag-based KASAN is almost
identical to the software mode. The key difference is using MTE for
assigning and checking tags.

Compared to the software mode, the hardware mode uses 4 bits per tag, as
dictated by MTE. Pointer tags are stored in bits [56:60), the top 4 bits
have the normal value 0xF. Having less distict tags increases the
probablity of false negatives (from ~1/256 to ~1/16) in certain cases.

Only synchronous exceptions are set up and used by hardware tag-based KASAN.

====== Benchmarks

Note: all measurements have been performed with software emulation of Memory
Tagging Extension, performance numbers for hardware tag-based KASAN on the
actual hardware are expected to be better.

Boot time [1]:
* 2.8 sec for clean kernel
* 5.7 sec for hardware tag-based KASAN
* 11.8 sec for software tag-based KASAN
* 11.6 sec for generic KASAN

Slab memory usage after boot [2]:
* 7.0 kb for clean kernel
* 9.7 kb for hardware tag-based KASAN
* 9.7 kb for software tag-based KASAN
* 41.3 kb for generic KASAN

Measurements have been performed with:
* defconfig-based configs
* Manually built QEMU master
* QEMU arguments: -machine virt,mte=on -cpu max
* CONFIG_KASAN_STACK_ENABLE disabled
* CONFIG_KASAN_INLINE enabled
* clang-10 as the compiler and gcc-10 as the assembler
    
[1] Time before the ext4 driver is initialized.
[2] Measured as `cat /proc/meminfo | grep Slab`.

====== Notes

The cover letter for software tag-based KASAN patchset can be found here:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0116523cfffa62aeb5aa3b85ce7419f3dae0c1b8

====== History

Changes v8->v9:
(Andrey:)
- Drop __init for kasan_init_hw_tags.
- Split out mte_enable() from mte_init_tags().
- Fix incorrect commit author.
- Revert addr_has_metadata() change done in a previous version, otherwise
  KASAN fails to print metadata for page_alloc allocations.
- Split kasan_init_hw_tags() into kasan_init_hw_tags_cpu() that is called
  for each CPU in cpu_enable_mte(), and kasan_init_hw_tags() that is called
  for boot CPU in smp_prepare_boot_cpu().
- Move kunit_kasan_expectation struct definition under CONFIG_KASAN.
(Vincenzo:)
- Address Catalin's comments for "arm64: mte: Reset the page tag in
  page->flags".
- New patch "kasan, mm: untag page address in free_reserved_area".

Changes v7->v8:
(Andrey:)
- Rebased onto 5.10-rc2+.
- Enable in-kernel MTE via kasan_init_hw_tags() instead of doing it
  directly from cpu_enable_mte(). This changes mte_init_tags() to both
  init RRND and enable in-kernel MTE in sync mode.
- Put the patches back into a logical order.
- Rename KASAN_SHADOW_GRANULE_SIZE to KASAN_GRANULE_SIZE in
  lib/test_kasan_module.c.
- Rename kasan_init_tags() to kasan_init_sw_tags() and mark as __init.

Changes v6->v7:
(Andrey:)
- Rebase onto 5.10-rc2.
- Fix pgd_t not defined build warning on s390.
- Mark kasan_init_depth() as static.
- Don't use mte_*() functions directly in report_hw_tags.c
- Add hw_ prefix to KASAN-level definitions of arch_*() tag helpers.
- Add missing <sched/task_stack.h> include to report_generic.h.

Changes v5->v6:
(Vincenzo:)
- Re-based on 5.10-rc1.
- Modified the flow of the mte fault handler in order to address an issue
  with word at a time routines that would affect Android init process.
- Dropped Reviewed-by from the involved patches.
(Andrey:)
- Properly use #if CONFIG_KASAN_STACK instead of #ifdef
  CONFIG_KASAN_STACK_ENABLE.
- Expand CONFIG_KASAN checks in arch/arm64/kernel/kaslr.c and
  arch/arm64/kernel/module.c.
- Don't select CONFIG_CONSTRUCTORS for HW_TAGS mode.
- Check PageSlab() in addr_has_metadata().

Changes v4->v5:
(Vincenzo:)
- Reset the tag associated by the kernel to a page when this is changed by
  the user.
- Add a kselftest to verify that GCR_EL1 is preserved during context
  switch.
- Squashed the TBI patch.
- Addressed some review comments.
- Verified the series with LTP.
(Andrey:)
- Put core arm64 patches first as requested by Catalin.

Changes v3->v4:
(Vincenzo:)
- Introduced mte-def.h that contains MTE definitions.
- Moved __MTE_PREAMBLE in mte.h.
- Clarified which API is part of mte-kasan.h.
- Removed tsk argument from mte_set_kernel_gcr().
- Addressed several nitpicks pointed out during review.
(Andrey:)
- Move include <asm/kasan.h> in include/linux/kasan.h to avoid build
  failures.
- Don't move "select SLUB_DEBUG if SLUB" back and forth between patches
  in KASAN Kconfig.
- Add arm64 prefix to "kasan: don't allow SW_TAGS with ARM64_MTE" commit.
- Don't add braces when defining KASAN_GRANULE_SIZE.
- Make KASAN_HW_TAGS compatible with init_on_alloc/free.

Changes v2->v3:
(Vincenzo:)
- Use ARM64_ASM_PREAMBLE for asm macros.
- Rename mte-helper.h to mte-kasan.h. The new header is meant to contain
  only macros and prototypes directly used in KASAN. The rest is defined
  in mte.h.
- Update mte_get_mem_tag()/mte_get_random_tag() to used directly asm
  volatile() macros instead of calling library functions.
- Harden mte_assign_mem_tag_range() to prevent an infinite loop in case of
  unaligned size.
- Made sure that report_tag_fault() is executed only once.
- Simplify the mte code in __cpu_setup.
- Remove kprobes.h from mte.c includes.
- General cleanup of the code.
(Andrey:)
- Use READ/WRITE_ONCE when accessing reported in do_tag_recovery().
- Move .unreq mte_tcr under CONFIG_ARM64_MTE to avoid build errors when MTE
  is not enabled.
- Improve mm/kasan/shadow.c comment header.
- Clarify what is a memory granule in "kasan: rename KASAN_SHADOW_* to
  KASAN_GRANULE_" commit description.
- Rename (report_)tags_sw/hw.c to to (report_)sw/hw_tags.c and drop
  unnecessary rename commit.
- Adopt 100 lines limit for some mm/kasan/ changes.
- Align arguments for stack_trace_save() call in mm/slub.c.
- Restore comment before kasan_init_tags().
- Remove GNU headers from all mm/kasan/ files.
- Simplify check_invalid_free() implementation tag-based modes.
- Drop subsequently removed report_tag_fault() implementation.
- Add KASAN_GRANULE_PAGE and use instead of PAGE_SIZE * KASAN_GRANULE_SIZE.
- Move kasan_enable/disable_current() declarations to simplify
  include/linux/kasan.h.
- Drop dependency on CONFIG_SLUB_DEBUG.
- Clarify the purpose of CONFIG_STACKTRACE in KASAN Kconfig.

Changes v1->v2:
- Rebase onto v10 of the user MTE patchset.
- Only enable in-kernel MTE when KASAN_HW_TAGS is enabled.
- Add a layer of arch-level indirection, so KASAN doesn't call MTE helpers
  directly (this will be useful in case more architectures will add support
  for HW_TAGS).
- Don't do arm64_skip_faulting_instruction() on MTE fault, disable MTE
  instead.
- Don't allow software tags with MTE via arch/arm64/Kconfig instead of
  lib/Kconfig.kasan.
- Rename mm/kasan/tags.c to tags_sw.c and mte.c to tags_hw.c, and do the
  same for report_*.c files.
- Reword HW_TAGS Kconfig help text to make it less MTE specific.
- Reword and clarify Documentation.
- Drop unnecessary is_el1_mte_sync_tag_check_fault().
- Change report_tag_fault() to only call kasan_report() once HW_TAGS is
  introduced.
- Rename arch/arm64/include/asm/mte_asm.h to mte-helpers.h and move all
  MTE-related defines and some helper functions there.
- Change mm/kasan/kasan.h to include mte-def.h instead of mte.h.
- Add WARN_ON() on unaligned size to mte_set_mem_tag_range().
- Implement ldg/irg MTE routines as inline assembly.
- Remove smp_wmb() from mte_set_mem_tag_range().
- Drop __must_check from mte_set_mem_tag_range() as KASAN has no use for
  the return value.
- Drop zero size check from mte_assign_mem_tag_range().
- Drop unnecessary include <asm/kasan.h> from low-level arm64 code.
- Move enabling TBI1 into __cpu_setup().
- Drop stale comment about callee-saved register from
  arch/arm64/kernel/entry.S.
- Mark gcr_kernel_excl as __ro_after_init.
- Use GENMASK() in mte_init_tags().

Andrey Konovalov (35):
  kasan: drop unnecessary GPL text from comment headers
  kasan: KASAN_VMALLOC depends on KASAN_GENERIC
  kasan: group vmalloc code
  s390/kasan: include asm/page.h from asm/kasan.h
  kasan: shadow declarations only for software modes
  kasan: rename (un)poison_shadow to (un)poison_memory
  kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_*
  kasan: only build init.c for software modes
  kasan: split out shadow.c from common.c
  kasan: define KASAN_GRANULE_PAGE
  kasan: rename report and tags files
  kasan: don't duplicate config dependencies
  kasan: hide invalid free check implementation
  kasan: decode stack frame only with KASAN_STACK_ENABLE
  kasan, arm64: only init shadow for software modes
  kasan, arm64: only use kasan_depth for software modes
  kasan, arm64: move initialization message
  kasan, arm64: rename kasan_init_tags and mark as __init
  kasan: rename addr_has_shadow to addr_has_metadata
  kasan: rename print_shadow_for_address to print_memory_metadata
  kasan: kasan_non_canonical_hook only for software modes
  kasan: rename SHADOW layout macros to META
  kasan: separate metadata_fetch_row for each mode
  kasan, arm64: don't allow SW_TAGS with ARM64_MTE
  kasan: introduce CONFIG_KASAN_HW_TAGS
  arm64: kasan: Align allocations for HW_TAGS
  arm64: kasan: Add arch layer for memory tagging helpers
  kasan: define KASAN_GRANULE_SIZE for HW_TAGS
  kasan, x86, s390: update undef CONFIG_KASAN
  kasan, arm64: expand CONFIG_KASAN checks
  kasan, arm64: implement HW_TAGS runtime
  kasan, arm64: print report from tag fault handler
  kasan, mm: reset tags when accessing metadata
  kasan, arm64: enable CONFIG_KASAN_HW_TAGS
  kasan: add documentation for hardware tag-based mode

Vincenzo Frascino (9):
  arm64: Enable armv8.5-a asm-arch option
  arm64: mte: Add in-kernel MTE helpers
  arm64: mte: Reset the page tag in page->flags
  arm64: mte: Add in-kernel tag fault handler
  arm64: kasan: Allow enabling in-kernel MTE
  arm64: mte: Convert gcr_user into an exclude mask
  arm64: mte: Switch GCR_EL1 in kernel entry and exit
  kasan, mm: untag page address in free_reserved_area
  kselftest/arm64: Check GCR_EL1 after context switch

 Documentation/dev-tools/kasan.rst             |  80 ++-
 arch/arm64/Kconfig                            |   9 +-
 arch/arm64/Makefile                           |   7 +-
 arch/arm64/include/asm/assembler.h            |   2 +-
 arch/arm64/include/asm/cache.h                |   3 +
 arch/arm64/include/asm/esr.h                  |   1 +
 arch/arm64/include/asm/kasan.h                |   5 +-
 arch/arm64/include/asm/memory.h               |  15 +-
 arch/arm64/include/asm/mte-def.h              |  14 +
 arch/arm64/include/asm/mte-kasan.h            |  67 +++
 arch/arm64/include/asm/mte.h                  |  22 +-
 arch/arm64/include/asm/processor.h            |   2 +-
 arch/arm64/include/asm/string.h               |   5 +-
 arch/arm64/include/asm/uaccess.h              |  23 +
 arch/arm64/kernel/asm-offsets.c               |   3 +
 arch/arm64/kernel/cpufeature.c                |   3 +
 arch/arm64/kernel/entry.S                     |  41 ++
 arch/arm64/kernel/head.S                      |   2 +-
 arch/arm64/kernel/hibernate.c                 |   5 +
 arch/arm64/kernel/image-vars.h                |   2 +-
 arch/arm64/kernel/kaslr.c                     |   3 +-
 arch/arm64/kernel/module.c                    |   6 +-
 arch/arm64/kernel/mte.c                       | 118 +++-
 arch/arm64/kernel/setup.c                     |   2 +-
 arch/arm64/kernel/smp.c                       |   2 +
 arch/arm64/lib/mte.S                          |  16 +
 arch/arm64/mm/copypage.c                      |   1 +
 arch/arm64/mm/fault.c                         |  59 ++
 arch/arm64/mm/kasan_init.c                    |  19 +-
 arch/arm64/mm/mteswap.c                       |   9 +
 arch/arm64/mm/proc.S                          |  23 +-
 arch/arm64/mm/ptdump.c                        |   6 +-
 arch/s390/boot/string.c                       |   1 +
 arch/s390/include/asm/kasan.h                 |   2 +
 arch/x86/boot/compressed/misc.h               |   1 +
 include/linux/kasan-checks.h                  |   2 +-
 include/linux/kasan.h                         | 125 ++--
 include/linux/mm.h                            |   2 +-
 include/linux/moduleloader.h                  |   3 +-
 include/linux/page-flags-layout.h             |   2 +-
 include/linux/sched.h                         |   2 +-
 include/linux/string.h                        |   2 +-
 init/init_task.c                              |   2 +-
 kernel/fork.c                                 |   4 +-
 lib/Kconfig.kasan                             |  62 +-
 lib/test_kasan.c                              |   2 +-
 lib/test_kasan_module.c                       |   2 +-
 mm/kasan/Makefile                             |  25 +-
 mm/kasan/common.c                             | 560 +-----------------
 mm/kasan/generic.c                            |  40 +-
 mm/kasan/generic_report.c                     | 165 ------
 mm/kasan/hw_tags.c                            |  80 +++
 mm/kasan/init.c                               |  17 +-
 mm/kasan/kasan.h                              |  72 ++-
 mm/kasan/quarantine.c                         |  10 -
 mm/kasan/report.c                             | 259 ++------
 mm/kasan/report_generic.c                     | 327 ++++++++++
 mm/kasan/report_hw_tags.c                     |  42 ++
 mm/kasan/{tags_report.c => report_sw_tags.c}  |  14 +-
 mm/kasan/shadow.c                             | 503 ++++++++++++++++
 mm/kasan/{tags.c => sw_tags.c}                |  24 +-
 mm/page_alloc.c                               |   9 +-
 mm/page_poison.c                              |   2 +-
 mm/ptdump.c                                   |  13 +-
 mm/slab_common.c                              |   2 +-
 mm/slub.c                                     |  29 +-
 scripts/Makefile.lib                          |   2 +
 tools/testing/selftests/arm64/mte/Makefile    |   2 +-
 .../arm64/mte/check_gcr_el1_cswitch.c         | 152 +++++
 69 files changed, 1990 insertions(+), 1148 deletions(-)
 create mode 100644 arch/arm64/include/asm/mte-def.h
 create mode 100644 arch/arm64/include/asm/mte-kasan.h
 delete mode 100644 mm/kasan/generic_report.c
 create mode 100644 mm/kasan/hw_tags.c
 create mode 100644 mm/kasan/report_generic.c
 create mode 100644 mm/kasan/report_hw_tags.c
 rename mm/kasan/{tags_report.c => report_sw_tags.c} (87%)
 create mode 100644 mm/kasan/shadow.c
 rename mm/kasan/{tags.c => sw_tags.c} (92%)
 create mode 100644 tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c

-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply	[flat|nested] 107+ messages in thread

* [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
@ 2020-11-10 22:09 ` Andrey Konovalov
  2020-11-11 13:16   ` Alexander Potapenko
  2020-11-10 22:09 ` [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrey Konovalov
                   ` (43 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:09 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Don't mention "GNU General Public License version 2" text explicitly,
as it's already covered by the SPDX-License-Identifier.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: If0a2690042a2aa0fca70cea601ae9aabe72fa233
---
 mm/kasan/common.c         |  5 -----
 mm/kasan/generic.c        |  5 -----
 mm/kasan/generic_report.c |  5 -----
 mm/kasan/init.c           |  5 -----
 mm/kasan/quarantine.c     | 10 ----------
 mm/kasan/report.c         |  5 -----
 mm/kasan/tags.c           |  5 -----
 mm/kasan/tags_report.c    |  5 -----
 8 files changed, 45 deletions(-)

diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 950fd372a07e..33d863f55db1 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/export.h>
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index 248264b9cb76..37ccfadd3263 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c
index a38c7a9e192a..6bb3f66992df 100644
--- a/mm/kasan/generic_report.c
+++ b/mm/kasan/generic_report.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/bitops.h>
diff --git a/mm/kasan/init.c b/mm/kasan/init.c
index fe6be0be1f76..9ce8cc5b8621 100644
--- a/mm/kasan/init.c
+++ b/mm/kasan/init.c
@@ -4,11 +4,6 @@
  *
  * Copyright (c) 2015 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/memblock.h>
diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c
index 4c5375810449..580ff5610fc1 100644
--- a/mm/kasan/quarantine.c
+++ b/mm/kasan/quarantine.c
@@ -6,16 +6,6 @@
  * Copyright (C) 2016 Google, Inc.
  *
  * Based on code by Dmitry Chernenkov.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License for more details.
- *
  */
 
 #include <linux/gfp.h>
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 00a53f1355ae..d500923abc8b 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/bitops.h>
diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c
index e02a36a51f42..5c8b08a25715 100644
--- a/mm/kasan/tags.c
+++ b/mm/kasan/tags.c
@@ -4,11 +4,6 @@
  *
  * Copyright (c) 2018 Google, Inc.
  * Author: Andrey Konovalov <andreyknvl@google.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c
index bee43717d6f0..5f183501b871 100644
--- a/mm/kasan/tags_report.c
+++ b/mm/kasan/tags_report.c
@@ -7,11 +7,6 @@
  *
  * Some code borrowed from https://github.com/xairy/kasan-prototype by
  *        Andrey Konovalov <andreyknvl@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
  */
 
 #include <linux/bitops.h>
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
  2020-11-10 22:09 ` [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers Andrey Konovalov
@ 2020-11-10 22:09 ` Andrey Konovalov
  2020-11-11 13:19   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 03/44] kasan: group vmalloc code Andrey Konovalov
                   ` (42 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:09 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Currently only generic KASAN mode supports vmalloc, reflect that
in the config.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I1889e5b3bed28cc5d607802fb6ae43ba461c0dc1
---
 lib/Kconfig.kasan | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
index 542a9c18398e..8f0742a0f23e 100644
--- a/lib/Kconfig.kasan
+++ b/lib/Kconfig.kasan
@@ -155,7 +155,7 @@ config KASAN_SW_TAGS_IDENTIFY
 
 config KASAN_VMALLOC
 	bool "Back mappings in vmalloc space with real shadow memory"
-	depends on HAVE_ARCH_KASAN_VMALLOC
+	depends on KASAN_GENERIC && HAVE_ARCH_KASAN_VMALLOC
 	help
 	  By default, the shadow region for vmalloc space is the read-only
 	  zero page. This means that KASAN cannot detect errors involving
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 03/44] kasan: group vmalloc code
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
  2020-11-10 22:09 ` [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers Andrey Konovalov
  2020-11-10 22:09 ` [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 13:21   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 04/44] s390/kasan: include asm/page.h from asm/kasan.h Andrey Konovalov
                   ` (41 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Group all vmalloc-related function declarations in include/linux/kasan.h,
and their implementations in mm/kasan/common.c.

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Ie20b6c689203cd6de4fd7f2c465ec081c00c5f15
---
 include/linux/kasan.h | 41 +++++++++++++----------
 mm/kasan/common.c     | 78 ++++++++++++++++++++++---------------------
 2 files changed, 63 insertions(+), 56 deletions(-)

diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 30d343b4a40a..59538e795df4 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -75,19 +75,6 @@ struct kasan_cache {
 	int free_meta_offset;
 };
 
-/*
- * These functions provide a special case to support backing module
- * allocations with real shadow memory. With KASAN vmalloc, the special
- * case is unnecessary, as the work is handled in the generic case.
- */
-#ifndef CONFIG_KASAN_VMALLOC
-int kasan_module_alloc(void *addr, size_t size);
-void kasan_free_shadow(const struct vm_struct *vm);
-#else
-static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
-static inline void kasan_free_shadow(const struct vm_struct *vm) {}
-#endif
-
 int kasan_add_zero_shadow(void *start, unsigned long size);
 void kasan_remove_zero_shadow(void *start, unsigned long size);
 
@@ -156,9 +143,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object,
 	return false;
 }
 
-static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
-static inline void kasan_free_shadow(const struct vm_struct *vm) {}
-
 static inline int kasan_add_zero_shadow(void *start, unsigned long size)
 {
 	return 0;
@@ -211,13 +195,16 @@ static inline void *kasan_reset_tag(const void *addr)
 #endif /* CONFIG_KASAN_SW_TAGS */
 
 #ifdef CONFIG_KASAN_VMALLOC
+
 int kasan_populate_vmalloc(unsigned long addr, unsigned long size);
 void kasan_poison_vmalloc(const void *start, unsigned long size);
 void kasan_unpoison_vmalloc(const void *start, unsigned long size);
 void kasan_release_vmalloc(unsigned long start, unsigned long end,
 			   unsigned long free_region_start,
 			   unsigned long free_region_end);
-#else
+
+#else /* CONFIG_KASAN_VMALLOC */
+
 static inline int kasan_populate_vmalloc(unsigned long start,
 					unsigned long size)
 {
@@ -232,7 +219,25 @@ static inline void kasan_release_vmalloc(unsigned long start,
 					 unsigned long end,
 					 unsigned long free_region_start,
 					 unsigned long free_region_end) {}
-#endif
+
+#endif /* CONFIG_KASAN_VMALLOC */
+
+#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
+
+/*
+ * These functions provide a special case to support backing module
+ * allocations with real shadow memory. With KASAN vmalloc, the special
+ * case is unnecessary, as the work is handled in the generic case.
+ */
+int kasan_module_alloc(void *addr, size_t size);
+void kasan_free_shadow(const struct vm_struct *vm);
+
+#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
+
+static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
+static inline void kasan_free_shadow(const struct vm_struct *vm) {}
+
+#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
 
 #ifdef CONFIG_KASAN_INLINE
 void kasan_non_canonical_hook(unsigned long addr);
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 33d863f55db1..89e5ef9417a7 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -536,44 +536,6 @@ void kasan_kfree_large(void *ptr, unsigned long ip)
 	/* The object will be poisoned by page_alloc. */
 }
 
-#ifndef CONFIG_KASAN_VMALLOC
-int kasan_module_alloc(void *addr, size_t size)
-{
-	void *ret;
-	size_t scaled_size;
-	size_t shadow_size;
-	unsigned long shadow_start;
-
-	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
-	scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
-	shadow_size = round_up(scaled_size, PAGE_SIZE);
-
-	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
-		return -EINVAL;
-
-	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
-			shadow_start + shadow_size,
-			GFP_KERNEL,
-			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
-			__builtin_return_address(0));
-
-	if (ret) {
-		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
-		find_vm_area(addr)->flags |= VM_KASAN;
-		kmemleak_ignore(ret);
-		return 0;
-	}
-
-	return -ENOMEM;
-}
-
-void kasan_free_shadow(const struct vm_struct *vm)
-{
-	if (vm->flags & VM_KASAN)
-		vfree(kasan_mem_to_shadow(vm->addr));
-}
-#endif
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 static bool shadow_mapped(unsigned long addr)
 {
@@ -685,6 +647,7 @@ core_initcall(kasan_memhotplug_init);
 #endif
 
 #ifdef CONFIG_KASAN_VMALLOC
+
 static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
 				      void *unused)
 {
@@ -923,4 +886,43 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
 				       (unsigned long)shadow_end);
 	}
 }
+
+#else /* CONFIG_KASAN_VMALLOC */
+
+int kasan_module_alloc(void *addr, size_t size)
+{
+	void *ret;
+	size_t scaled_size;
+	size_t shadow_size;
+	unsigned long shadow_start;
+
+	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
+	scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
+	shadow_size = round_up(scaled_size, PAGE_SIZE);
+
+	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
+		return -EINVAL;
+
+	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
+			shadow_start + shadow_size,
+			GFP_KERNEL,
+			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
+			__builtin_return_address(0));
+
+	if (ret) {
+		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
+		find_vm_area(addr)->flags |= VM_KASAN;
+		kmemleak_ignore(ret);
+		return 0;
+	}
+
+	return -ENOMEM;
+}
+
+void kasan_free_shadow(const struct vm_struct *vm)
+{
+	if (vm->flags & VM_KASAN)
+		vfree(kasan_mem_to_shadow(vm->addr));
+}
+
 #endif
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 04/44] s390/kasan: include asm/page.h from asm/kasan.h
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (2 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 03/44] kasan: group vmalloc code Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 05/44] kasan: shadow declarations only for software modes Andrey Konovalov
                   ` (40 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov,
	Vasily Gorbik

asm/kasan.h relies on pgd_t and _REGION1_SHIFT definitions and therefore
requires asm/pgtable.h include. Include asm/pgtable.h from asm/kasan.h.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
---
Change-Id: I369a8f9beb442b9d05733892232345c3f4120e0a
---
 arch/s390/include/asm/kasan.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/s390/include/asm/kasan.h b/arch/s390/include/asm/kasan.h
index e9bf486de136..4753ad0c3cba 100644
--- a/arch/s390/include/asm/kasan.h
+++ b/arch/s390/include/asm/kasan.h
@@ -2,6 +2,8 @@
 #ifndef __ASM_KASAN_H
 #define __ASM_KASAN_H
 
+#include <asm/pgtable.h>
+
 #ifdef CONFIG_KASAN
 
 #define KASAN_SHADOW_SCALE_SHIFT 3
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 05/44] kasan: shadow declarations only for software modes
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (3 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 04/44] s390/kasan: include asm/page.h from asm/kasan.h Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 13:25   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory Andrey Konovalov
                   ` (39 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Group shadow-related KASAN function declarations and only define them
for the two existing software modes.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I864be75a88b91b443c55e9c2042865e15703e164
---
 include/linux/kasan.h | 47 ++++++++++++++++++++++++++++---------------
 1 file changed, 31 insertions(+), 16 deletions(-)

diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 59538e795df4..26f2ab92e7ca 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -11,7 +11,6 @@ struct task_struct;
 
 #ifdef CONFIG_KASAN
 
-#include <linux/pgtable.h>
 #include <asm/kasan.h>
 
 /* kasan_data struct is used in KUnit tests for KASAN expected failures */
@@ -20,6 +19,20 @@ struct kunit_kasan_expectation {
 	bool report_found;
 };
 
+#endif
+
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
+
+#include <linux/pgtable.h>
+
+/* Software KASAN implementations use shadow memory. */
+
+#ifdef CONFIG_KASAN_SW_TAGS
+#define KASAN_SHADOW_INIT 0xFF
+#else
+#define KASAN_SHADOW_INIT 0
+#endif
+
 extern unsigned char kasan_early_shadow_page[PAGE_SIZE];
 extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE];
 extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD];
@@ -35,6 +48,23 @@ static inline void *kasan_mem_to_shadow(const void *addr)
 		+ KASAN_SHADOW_OFFSET;
 }
 
+int kasan_add_zero_shadow(void *start, unsigned long size);
+void kasan_remove_zero_shadow(void *start, unsigned long size);
+
+#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+static inline int kasan_add_zero_shadow(void *start, unsigned long size)
+{
+	return 0;
+}
+static inline void kasan_remove_zero_shadow(void *start,
+					unsigned long size)
+{}
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+#ifdef CONFIG_KASAN
+
 /* Enable reporting bugs after kasan_disable_current() */
 extern void kasan_enable_current(void);
 
@@ -75,9 +105,6 @@ struct kasan_cache {
 	int free_meta_offset;
 };
 
-int kasan_add_zero_shadow(void *start, unsigned long size);
-void kasan_remove_zero_shadow(void *start, unsigned long size);
-
 size_t __ksize(const void *);
 static inline void kasan_unpoison_slab(const void *ptr)
 {
@@ -143,14 +170,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object,
 	return false;
 }
 
-static inline int kasan_add_zero_shadow(void *start, unsigned long size)
-{
-	return 0;
-}
-static inline void kasan_remove_zero_shadow(void *start,
-					unsigned long size)
-{}
-
 static inline void kasan_unpoison_slab(const void *ptr) { }
 static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
 
@@ -158,8 +177,6 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
 
 #ifdef CONFIG_KASAN_GENERIC
 
-#define KASAN_SHADOW_INIT 0
-
 void kasan_cache_shrink(struct kmem_cache *cache);
 void kasan_cache_shutdown(struct kmem_cache *cache);
 void kasan_record_aux_stack(void *ptr);
@@ -174,8 +191,6 @@ static inline void kasan_record_aux_stack(void *ptr) {}
 
 #ifdef CONFIG_KASAN_SW_TAGS
 
-#define KASAN_SHADOW_INIT 0xFF
-
 void kasan_init_tags(void);
 
 void *kasan_reset_tag(const void *addr);
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (4 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 05/44] kasan: shadow declarations only for software modes Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 13:26   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrey Konovalov
                   ` (38 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory, but will reuse the same
functions. Rename kasan_unpoison_shadow to kasan_unpoison_memory,
and kasan_poison_shadow to kasan_poison_memory.

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Ia359f32815242c4704e49a5f1639ca2d2f8cba69
---
 include/linux/kasan.h |  6 +++---
 kernel/fork.c         |  4 ++--
 mm/kasan/common.c     | 38 +++++++++++++++++++-------------------
 mm/kasan/generic.c    | 12 ++++++------
 mm/kasan/kasan.h      |  2 +-
 mm/kasan/tags.c       |  2 +-
 mm/slab_common.c      |  2 +-
 7 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 26f2ab92e7ca..f6435b9f889c 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -71,7 +71,7 @@ extern void kasan_enable_current(void);
 /* Disable reporting bugs for current task */
 extern void kasan_disable_current(void);
 
-void kasan_unpoison_shadow(const void *address, size_t size);
+void kasan_unpoison_memory(const void *address, size_t size);
 
 void kasan_unpoison_task_stack(struct task_struct *task);
 
@@ -108,7 +108,7 @@ struct kasan_cache {
 size_t __ksize(const void *);
 static inline void kasan_unpoison_slab(const void *ptr)
 {
-	kasan_unpoison_shadow(ptr, __ksize(ptr));
+	kasan_unpoison_memory(ptr, __ksize(ptr));
 }
 size_t kasan_metadata_size(struct kmem_cache *cache);
 
@@ -117,7 +117,7 @@ void kasan_restore_multi_shot(bool enabled);
 
 #else /* CONFIG_KASAN */
 
-static inline void kasan_unpoison_shadow(const void *address, size_t size) {}
+static inline void kasan_unpoison_memory(const void *address, size_t size) {}
 
 static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
 
diff --git a/kernel/fork.c b/kernel/fork.c
index 6d266388d380..1c905e4290ab 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -225,8 +225,8 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node)
 		if (!s)
 			continue;
 
-		/* Clear the KASAN shadow of the stack. */
-		kasan_unpoison_shadow(s->addr, THREAD_SIZE);
+		/* Mark stack accessible for KASAN. */
+		kasan_unpoison_memory(s->addr, THREAD_SIZE);
 
 		/* Clear stale pointers from reused stack. */
 		memset(s->addr, 0, THREAD_SIZE);
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 89e5ef9417a7..a4b73fa0dd7e 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -108,7 +108,7 @@ void *memcpy(void *dest, const void *src, size_t len)
  * Poisons the shadow memory for 'size' bytes starting from 'addr'.
  * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
  */
-void kasan_poison_shadow(const void *address, size_t size, u8 value)
+void kasan_poison_memory(const void *address, size_t size, u8 value)
 {
 	void *shadow_start, *shadow_end;
 
@@ -125,7 +125,7 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value)
 	__memset(shadow_start, value, shadow_end - shadow_start);
 }
 
-void kasan_unpoison_shadow(const void *address, size_t size)
+void kasan_unpoison_memory(const void *address, size_t size)
 {
 	u8 tag = get_tag(address);
 
@@ -136,7 +136,7 @@ void kasan_unpoison_shadow(const void *address, size_t size)
 	 */
 	address = reset_tag(address);
 
-	kasan_poison_shadow(address, size, tag);
+	kasan_poison_memory(address, size, tag);
 
 	if (size & KASAN_SHADOW_MASK) {
 		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
@@ -153,7 +153,7 @@ static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
 	void *base = task_stack_page(task);
 	size_t size = sp - base;
 
-	kasan_unpoison_shadow(base, size);
+	kasan_unpoison_memory(base, size);
 }
 
 /* Unpoison the entire stack for a task. */
@@ -172,7 +172,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark)
 	 */
 	void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1));
 
-	kasan_unpoison_shadow(base, watermark - base);
+	kasan_unpoison_memory(base, watermark - base);
 }
 
 void kasan_alloc_pages(struct page *page, unsigned int order)
@@ -186,13 +186,13 @@ void kasan_alloc_pages(struct page *page, unsigned int order)
 	tag = random_tag();
 	for (i = 0; i < (1 << order); i++)
 		page_kasan_tag_set(page + i, tag);
-	kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
+	kasan_unpoison_memory(page_address(page), PAGE_SIZE << order);
 }
 
 void kasan_free_pages(struct page *page, unsigned int order)
 {
 	if (likely(!PageHighMem(page)))
-		kasan_poison_shadow(page_address(page),
+		kasan_poison_memory(page_address(page),
 				PAGE_SIZE << order,
 				KASAN_FREE_PAGE);
 }
@@ -284,18 +284,18 @@ void kasan_poison_slab(struct page *page)
 
 	for (i = 0; i < compound_nr(page); i++)
 		page_kasan_tag_reset(page + i);
-	kasan_poison_shadow(page_address(page), page_size(page),
+	kasan_poison_memory(page_address(page), page_size(page),
 			KASAN_KMALLOC_REDZONE);
 }
 
 void kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
 {
-	kasan_unpoison_shadow(object, cache->object_size);
+	kasan_unpoison_memory(object, cache->object_size);
 }
 
 void kasan_poison_object_data(struct kmem_cache *cache, void *object)
 {
-	kasan_poison_shadow(object,
+	kasan_poison_memory(object,
 			round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
 			KASAN_KMALLOC_REDZONE);
 }
@@ -408,7 +408,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
 	}
 
 	rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
-	kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
+	kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE);
 
 	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
 			unlikely(!(cache->flags & SLAB_KASAN)))
@@ -448,8 +448,8 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
 		tag = assign_tag(cache, object, false, keep_tag);
 
 	/* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
-	kasan_unpoison_shadow(set_tag(object, tag), size);
-	kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
+	kasan_unpoison_memory(set_tag(object, tag), size);
+	kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start,
 		KASAN_KMALLOC_REDZONE);
 
 	if (cache->flags & SLAB_KASAN)
@@ -489,8 +489,8 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size,
 				KASAN_SHADOW_SCALE_SIZE);
 	redzone_end = (unsigned long)ptr + page_size(page);
 
-	kasan_unpoison_shadow(ptr, size);
-	kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
+	kasan_unpoison_memory(ptr, size);
+	kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start,
 		KASAN_PAGE_REDZONE);
 
 	return (void *)ptr;
@@ -523,7 +523,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip)
 			kasan_report_invalid_free(ptr, ip);
 			return;
 		}
-		kasan_poison_shadow(ptr, page_size(page), KASAN_FREE_PAGE);
+		kasan_poison_memory(ptr, page_size(page), KASAN_FREE_PAGE);
 	} else {
 		__kasan_slab_free(page->slab_cache, ptr, ip, false);
 	}
@@ -709,7 +709,7 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
 	 * // vmalloc() allocates memory
 	 * // let a = area->addr
 	 * // we reach kasan_populate_vmalloc
-	 * // and call kasan_unpoison_shadow:
+	 * // and call kasan_unpoison_memory:
 	 * STORE shadow(a), unpoison_val
 	 * ...
 	 * STORE shadow(a+99), unpoison_val	x = LOAD p
@@ -744,7 +744,7 @@ void kasan_poison_vmalloc(const void *start, unsigned long size)
 		return;
 
 	size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
-	kasan_poison_shadow(start, size, KASAN_VMALLOC_INVALID);
+	kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
 }
 
 void kasan_unpoison_vmalloc(const void *start, unsigned long size)
@@ -752,7 +752,7 @@ void kasan_unpoison_vmalloc(const void *start, unsigned long size)
 	if (!is_vmalloc_or_module_addr(start))
 		return;
 
-	kasan_unpoison_shadow(start, size);
+	kasan_unpoison_memory(start, size);
 }
 
 static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index 37ccfadd3263..7006157c674b 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -202,9 +202,9 @@ static void register_global(struct kasan_global *global)
 {
 	size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
 
-	kasan_unpoison_shadow(global->beg, global->size);
+	kasan_unpoison_memory(global->beg, global->size);
 
-	kasan_poison_shadow(global->beg + aligned_size,
+	kasan_poison_memory(global->beg + aligned_size,
 		global->size_with_redzone - aligned_size,
 		KASAN_GLOBAL_REDZONE);
 }
@@ -285,11 +285,11 @@ void __asan_alloca_poison(unsigned long addr, size_t size)
 
 	WARN_ON(!IS_ALIGNED(addr, KASAN_ALLOCA_REDZONE_SIZE));
 
-	kasan_unpoison_shadow((const void *)(addr + rounded_down_size),
+	kasan_unpoison_memory((const void *)(addr + rounded_down_size),
 			      size - rounded_down_size);
-	kasan_poison_shadow(left_redzone, KASAN_ALLOCA_REDZONE_SIZE,
+	kasan_poison_memory(left_redzone, KASAN_ALLOCA_REDZONE_SIZE,
 			KASAN_ALLOCA_LEFT);
-	kasan_poison_shadow(right_redzone,
+	kasan_poison_memory(right_redzone,
 			padding_size + KASAN_ALLOCA_REDZONE_SIZE,
 			KASAN_ALLOCA_RIGHT);
 }
@@ -301,7 +301,7 @@ void __asan_allocas_unpoison(const void *stack_top, const void *stack_bottom)
 	if (unlikely(!stack_top || stack_top > stack_bottom))
 		return;
 
-	kasan_unpoison_shadow(stack_top, stack_bottom - stack_top);
+	kasan_unpoison_memory(stack_top, stack_bottom - stack_top);
 }
 EXPORT_SYMBOL(__asan_allocas_unpoison);
 
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index ac499456740f..03450d3b31f7 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -150,7 +150,7 @@ static inline bool addr_has_shadow(const void *addr)
 	return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
 }
 
-void kasan_poison_shadow(const void *address, size_t size, u8 value);
+void kasan_poison_memory(const void *address, size_t size, u8 value);
 
 /**
  * check_memory_region - Check memory region, and report if invalid access.
diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c
index 5c8b08a25715..4bdd7dbd6647 100644
--- a/mm/kasan/tags.c
+++ b/mm/kasan/tags.c
@@ -153,7 +153,7 @@ EXPORT_SYMBOL(__hwasan_storeN_noabort);
 
 void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size)
 {
-	kasan_poison_shadow((void *)addr, size, tag);
+	kasan_poison_memory((void *)addr, size, tag);
 }
 EXPORT_SYMBOL(__hwasan_tag_memory);
 
diff --git a/mm/slab_common.c b/mm/slab_common.c
index f9ccd5dc13f3..53d0f8bb57ea 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1176,7 +1176,7 @@ size_t ksize(const void *objp)
 	 * We assume that ksize callers could use whole allocated area,
 	 * so we need to unpoison this area.
 	 */
-	kasan_unpoison_shadow(objp, size);
+	kasan_unpoison_memory(objp, size);
 	return size;
 }
 EXPORT_SYMBOL(ksize);
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_*
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (5 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 13:45   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 08/44] kasan: only build init.c for software modes Andrey Konovalov
                   ` (37 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory, but will still use the concept
of memory granules. Each memory granule maps to a single metadata entry:
8 bytes per one shadow byte for generic mode, 16 bytes per one shadow byte
for software tag-based mode, and 16 bytes per one allocation tag for
hardware tag-based mode.

Rename KASAN_SHADOW_SCALE_SIZE to KASAN_GRANULE_SIZE, and KASAN_SHADOW_MASK
to KASAN_GRANULE_MASK.

Also use MASK when used as a mask, otherwise use SIZE.

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Iac733e2248aa9d29f6fc425d8946ba07cca73ecf
---
 Documentation/dev-tools/kasan.rst |  2 +-
 lib/test_kasan.c                  |  2 +-
 lib/test_kasan_module.c           |  2 +-
 mm/kasan/common.c                 | 39 ++++++++++++++++---------------
 mm/kasan/generic.c                | 14 +++++------
 mm/kasan/generic_report.c         |  8 +++----
 mm/kasan/init.c                   |  8 +++----
 mm/kasan/kasan.h                  |  4 ++--
 mm/kasan/report.c                 | 10 ++++----
 mm/kasan/tags_report.c            |  2 +-
 10 files changed, 46 insertions(+), 45 deletions(-)

diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst
index 2b68addaadcd..edca4be5e405 100644
--- a/Documentation/dev-tools/kasan.rst
+++ b/Documentation/dev-tools/kasan.rst
@@ -264,7 +264,7 @@ Most mappings in vmalloc space are small, requiring less than a full
 page of shadow space. Allocating a full shadow page per mapping would
 therefore be wasteful. Furthermore, to ensure that different mappings
 use different shadow pages, mappings would have to be aligned to
-``KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE``.
+``KASAN_GRANULE_SIZE * PAGE_SIZE``.
 
 Instead, we share backing space across multiple mappings. We allocate
 a backing page when a mapping in vmalloc space uses a particular page
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 662f862702fc..2947274cc2d3 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -25,7 +25,7 @@
 
 #include "../mm/kasan/kasan.h"
 
-#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE)
+#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE)
 
 /*
  * We assign some test results to these globals to make sure the tests
diff --git a/lib/test_kasan_module.c b/lib/test_kasan_module.c
index 2d68db6ae67b..fcb991c3aaf8 100644
--- a/lib/test_kasan_module.c
+++ b/lib/test_kasan_module.c
@@ -15,7 +15,7 @@
 
 #include "../mm/kasan/kasan.h"
 
-#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE)
+#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE)
 
 static noinline void __init copy_user_test(void)
 {
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index a4b73fa0dd7e..f65c9f792f8f 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -106,7 +106,7 @@ void *memcpy(void *dest, const void *src, size_t len)
 
 /*
  * Poisons the shadow memory for 'size' bytes starting from 'addr'.
- * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
+ * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
  */
 void kasan_poison_memory(const void *address, size_t size, u8 value)
 {
@@ -138,13 +138,13 @@ void kasan_unpoison_memory(const void *address, size_t size)
 
 	kasan_poison_memory(address, size, tag);
 
-	if (size & KASAN_SHADOW_MASK) {
+	if (size & KASAN_GRANULE_MASK) {
 		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
 
 		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 			*shadow = tag;
 		else
-			*shadow = size & KASAN_SHADOW_MASK;
+			*shadow = size & KASAN_GRANULE_MASK;
 	}
 }
 
@@ -296,7 +296,7 @@ void kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
 void kasan_poison_object_data(struct kmem_cache *cache, void *object)
 {
 	kasan_poison_memory(object,
-			round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
+			round_up(cache->object_size, KASAN_GRANULE_SIZE),
 			KASAN_KMALLOC_REDZONE);
 }
 
@@ -368,7 +368,7 @@ static inline bool shadow_invalid(u8 tag, s8 shadow_byte)
 {
 	if (IS_ENABLED(CONFIG_KASAN_GENERIC))
 		return shadow_byte < 0 ||
-			shadow_byte >= KASAN_SHADOW_SCALE_SIZE;
+			shadow_byte >= KASAN_GRANULE_SIZE;
 
 	/* else CONFIG_KASAN_SW_TAGS: */
 	if ((u8)shadow_byte == KASAN_TAG_INVALID)
@@ -407,7 +407,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
 		return true;
 	}
 
-	rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
+	rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE);
 	kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE);
 
 	if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
@@ -440,9 +440,9 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
 		return NULL;
 
 	redzone_start = round_up((unsigned long)(object + size),
-				KASAN_SHADOW_SCALE_SIZE);
+				KASAN_GRANULE_SIZE);
 	redzone_end = round_up((unsigned long)object + cache->object_size,
-				KASAN_SHADOW_SCALE_SIZE);
+				KASAN_GRANULE_SIZE);
 
 	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		tag = assign_tag(cache, object, false, keep_tag);
@@ -486,7 +486,7 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size,
 
 	page = virt_to_page(ptr);
 	redzone_start = round_up((unsigned long)(ptr + size),
-				KASAN_SHADOW_SCALE_SIZE);
+				KASAN_GRANULE_SIZE);
 	redzone_end = (unsigned long)ptr + page_size(page);
 
 	kasan_unpoison_memory(ptr, size);
@@ -584,8 +584,8 @@ static int __meminit kasan_mem_notifier(struct notifier_block *nb,
 	shadow_size = nr_shadow_pages << PAGE_SHIFT;
 	shadow_end = shadow_start + shadow_size;
 
-	if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) ||
-		WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT)))
+	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
+		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
 		return NOTIFY_BAD;
 
 	switch (action) {
@@ -743,7 +743,7 @@ void kasan_poison_vmalloc(const void *start, unsigned long size)
 	if (!is_vmalloc_or_module_addr(start))
 		return;
 
-	size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
+	size = round_up(size, KASAN_GRANULE_SIZE);
 	kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
 }
 
@@ -856,22 +856,22 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
 	unsigned long region_start, region_end;
 	unsigned long size;
 
-	region_start = ALIGN(start, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
-	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
+	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
+	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
 
 	free_region_start = ALIGN(free_region_start,
-				  PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
+				  PAGE_SIZE * KASAN_GRANULE_SIZE);
 
 	if (start != region_start &&
 	    free_region_start < region_start)
-		region_start -= PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE;
+		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
 
 	free_region_end = ALIGN_DOWN(free_region_end,
-				     PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
+				     PAGE_SIZE * KASAN_GRANULE_SIZE);
 
 	if (end != region_end &&
 	    free_region_end > region_end)
-		region_end += PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE;
+		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
 
 	shadow_start = kasan_mem_to_shadow((void *)region_start);
 	shadow_end = kasan_mem_to_shadow((void *)region_end);
@@ -897,7 +897,8 @@ int kasan_module_alloc(void *addr, size_t size)
 	unsigned long shadow_start;
 
 	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
-	scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
+	scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
+				KASAN_SHADOW_SCALE_SHIFT;
 	shadow_size = round_up(scaled_size, PAGE_SIZE);
 
 	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index 7006157c674b..ec4417156943 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -46,7 +46,7 @@ static __always_inline bool memory_is_poisoned_1(unsigned long addr)
 	s8 shadow_value = *(s8 *)kasan_mem_to_shadow((void *)addr);
 
 	if (unlikely(shadow_value)) {
-		s8 last_accessible_byte = addr & KASAN_SHADOW_MASK;
+		s8 last_accessible_byte = addr & KASAN_GRANULE_MASK;
 		return unlikely(last_accessible_byte >= shadow_value);
 	}
 
@@ -62,7 +62,7 @@ static __always_inline bool memory_is_poisoned_2_4_8(unsigned long addr,
 	 * Access crosses 8(shadow size)-byte boundary. Such access maps
 	 * into 2 shadow bytes, so we need to check them both.
 	 */
-	if (unlikely(((addr + size - 1) & KASAN_SHADOW_MASK) < size - 1))
+	if (unlikely(((addr + size - 1) & KASAN_GRANULE_MASK) < size - 1))
 		return *shadow_addr || memory_is_poisoned_1(addr + size - 1);
 
 	return memory_is_poisoned_1(addr + size - 1);
@@ -73,7 +73,7 @@ static __always_inline bool memory_is_poisoned_16(unsigned long addr)
 	u16 *shadow_addr = (u16 *)kasan_mem_to_shadow((void *)addr);
 
 	/* Unaligned 16-bytes access maps into 3 shadow bytes. */
-	if (unlikely(!IS_ALIGNED(addr, KASAN_SHADOW_SCALE_SIZE)))
+	if (unlikely(!IS_ALIGNED(addr, KASAN_GRANULE_SIZE)))
 		return *shadow_addr || memory_is_poisoned_1(addr + 15);
 
 	return *shadow_addr;
@@ -134,7 +134,7 @@ static __always_inline bool memory_is_poisoned_n(unsigned long addr,
 		s8 *last_shadow = (s8 *)kasan_mem_to_shadow((void *)last_byte);
 
 		if (unlikely(ret != (unsigned long)last_shadow ||
-			((long)(last_byte & KASAN_SHADOW_MASK) >= *last_shadow)))
+			((long)(last_byte & KASAN_GRANULE_MASK) >= *last_shadow)))
 			return true;
 	}
 	return false;
@@ -200,7 +200,7 @@ void kasan_cache_shutdown(struct kmem_cache *cache)
 
 static void register_global(struct kasan_global *global)
 {
-	size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
+	size_t aligned_size = round_up(global->size, KASAN_GRANULE_SIZE);
 
 	kasan_unpoison_memory(global->beg, global->size);
 
@@ -274,10 +274,10 @@ EXPORT_SYMBOL(__asan_handle_no_return);
 /* Emitted by compiler to poison alloca()ed objects. */
 void __asan_alloca_poison(unsigned long addr, size_t size)
 {
-	size_t rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
+	size_t rounded_up_size = round_up(size, KASAN_GRANULE_SIZE);
 	size_t padding_size = round_up(size, KASAN_ALLOCA_REDZONE_SIZE) -
 			rounded_up_size;
-	size_t rounded_down_size = round_down(size, KASAN_SHADOW_SCALE_SIZE);
+	size_t rounded_down_size = round_down(size, KASAN_GRANULE_SIZE);
 
 	const void *left_redzone = (const void *)(addr -
 			KASAN_ALLOCA_REDZONE_SIZE);
diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c
index 6bb3f66992df..7d5b9e5c7cfe 100644
--- a/mm/kasan/generic_report.c
+++ b/mm/kasan/generic_report.c
@@ -34,7 +34,7 @@ void *find_first_bad_addr(void *addr, size_t size)
 	void *p = addr;
 
 	while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p)))
-		p += KASAN_SHADOW_SCALE_SIZE;
+		p += KASAN_GRANULE_SIZE;
 	return p;
 }
 
@@ -46,14 +46,14 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info)
 	shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr);
 
 	/*
-	 * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look
+	 * If shadow byte value is in [0, KASAN_GRANULE_SIZE) we can look
 	 * at the next shadow byte to determine the type of the bad access.
 	 */
-	if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1)
+	if (*shadow_addr > 0 && *shadow_addr <= KASAN_GRANULE_SIZE - 1)
 		shadow_addr++;
 
 	switch (*shadow_addr) {
-	case 0 ... KASAN_SHADOW_SCALE_SIZE - 1:
+	case 0 ... KASAN_GRANULE_SIZE - 1:
 		/*
 		 * In theory it's still possible to see these shadow values
 		 * due to a data race in the kernel code.
diff --git a/mm/kasan/init.c b/mm/kasan/init.c
index 9ce8cc5b8621..dfddd6c39fe6 100644
--- a/mm/kasan/init.c
+++ b/mm/kasan/init.c
@@ -442,8 +442,8 @@ void kasan_remove_zero_shadow(void *start, unsigned long size)
 	end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
 	if (WARN_ON((unsigned long)start %
-			(KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)))
+			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
+	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
 		return;
 
 	for (; addr < end; addr = next) {
@@ -477,8 +477,8 @@ int kasan_add_zero_shadow(void *start, unsigned long size)
 	shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
 	if (WARN_ON((unsigned long)start %
-			(KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)))
+			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
+	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
 		return -EINVAL;
 
 	ret = kasan_populate_early_shadow(shadow_start, shadow_end);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 03450d3b31f7..c31e2c739301 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -5,8 +5,8 @@
 #include <linux/kasan.h>
 #include <linux/stackdepot.h>
 
-#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT)
-#define KASAN_SHADOW_MASK       (KASAN_SHADOW_SCALE_SIZE - 1)
+#define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
+#define KASAN_GRANULE_MASK	(KASAN_GRANULE_SIZE - 1)
 
 #define KASAN_TAG_KERNEL	0xFF /* native kernel pointers tag */
 #define KASAN_TAG_INVALID	0xFE /* inaccessible memory tag */
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index d500923abc8b..7b8dcb799a78 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -314,24 +314,24 @@ static bool __must_check get_address_stack_frame_info(const void *addr,
 		return false;
 
 	aligned_addr = round_down((unsigned long)addr, sizeof(long));
-	mem_ptr = round_down(aligned_addr, KASAN_SHADOW_SCALE_SIZE);
+	mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
 	shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
 	shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
 
 	while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
 		shadow_ptr--;
-		mem_ptr -= KASAN_SHADOW_SCALE_SIZE;
+		mem_ptr -= KASAN_GRANULE_SIZE;
 	}
 
 	while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
 		shadow_ptr--;
-		mem_ptr -= KASAN_SHADOW_SCALE_SIZE;
+		mem_ptr -= KASAN_GRANULE_SIZE;
 	}
 
 	if (shadow_ptr < shadow_bottom)
 		return false;
 
-	frame = (const unsigned long *)(mem_ptr + KASAN_SHADOW_SCALE_SIZE);
+	frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
 	if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
 		pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
 		       frame[0]);
@@ -599,6 +599,6 @@ void kasan_non_canonical_hook(unsigned long addr)
 	else
 		bug_type = "maybe wild-memory-access";
 	pr_alert("KASAN: %s in range [0x%016lx-0x%016lx]\n", bug_type,
-		 orig_addr, orig_addr + KASAN_SHADOW_MASK);
+		 orig_addr, orig_addr + KASAN_GRANULE_SIZE - 1);
 }
 #endif
diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c
index 5f183501b871..c87d5a343b4e 100644
--- a/mm/kasan/tags_report.c
+++ b/mm/kasan/tags_report.c
@@ -76,7 +76,7 @@ void *find_first_bad_addr(void *addr, size_t size)
 	void *end = p + size;
 
 	while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p))
-		p += KASAN_SHADOW_SCALE_SIZE;
+		p += KASAN_GRANULE_SIZE;
 	return p;
 }
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 08/44] kasan: only build init.c for software modes
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (6 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 13:46   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 09/44] kasan: split out shadow.c from common.c Andrey Konovalov
                   ` (36 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory, so only build init.c that
contains shadow initialization code for software modes.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I8d68c47345afc1dbedadde738f34a874dcae5080
---
 mm/kasan/Makefile | 6 +++---
 mm/kasan/init.c   | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index 370d970e5ab5..7cf685bb51bd 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -29,6 +29,6 @@ CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 
-obj-$(CONFIG_KASAN) := common.o init.o report.o
-obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o
-obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o
+obj-$(CONFIG_KASAN) := common.o report.o
+obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o
+obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o
diff --git a/mm/kasan/init.c b/mm/kasan/init.c
index dfddd6c39fe6..1a71eaa8c5f9 100644
--- a/mm/kasan/init.c
+++ b/mm/kasan/init.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains some kasan initialization code.
+ * This file contains KASAN shadow initialization code.
  *
  * Copyright (c) 2015 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 09/44] kasan: split out shadow.c from common.c
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (7 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 08/44] kasan: only build init.c for software modes Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:07   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE Andrey Konovalov
                   ` (35 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

The new mode won't be using shadow memory. Move all shadow-related code
to shadow.c, which is only enabled for software KASAN modes that use
shadow memory.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Ic1c32ce72d4649848e9e6a1f2c8dd269c77673f2
---
 mm/kasan/Makefile |   6 +-
 mm/kasan/common.c | 486 +-------------------------------------------
 mm/kasan/shadow.c | 505 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 510 insertions(+), 487 deletions(-)
 create mode 100644 mm/kasan/shadow.c

diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index 7cf685bb51bd..7cc1031e1ef8 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -10,6 +10,7 @@ CFLAGS_REMOVE_generic_report.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE)
 
@@ -26,9 +27,10 @@ CFLAGS_generic_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 
 obj-$(CONFIG_KASAN) := common.o report.o
-obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o
-obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o
+obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o
+obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index f65c9f792f8f..123abfb760d4 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains common generic and tag-based KASAN code.
+ * This file contains common KASAN code.
  *
  * Copyright (c) 2014 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
@@ -13,7 +13,6 @@
 #include <linux/init.h>
 #include <linux/kasan.h>
 #include <linux/kernel.h>
-#include <linux/kmemleak.h>
 #include <linux/linkage.h>
 #include <linux/memblock.h>
 #include <linux/memory.h>
@@ -26,12 +25,8 @@
 #include <linux/stacktrace.h>
 #include <linux/string.h>
 #include <linux/types.h>
-#include <linux/vmalloc.h>
 #include <linux/bug.h>
 
-#include <asm/cacheflush.h>
-#include <asm/tlbflush.h>
-
 #include "kasan.h"
 #include "../slab.h"
 
@@ -61,93 +56,6 @@ void kasan_disable_current(void)
 	current->kasan_depth--;
 }
 
-bool __kasan_check_read(const volatile void *p, unsigned int size)
-{
-	return check_memory_region((unsigned long)p, size, false, _RET_IP_);
-}
-EXPORT_SYMBOL(__kasan_check_read);
-
-bool __kasan_check_write(const volatile void *p, unsigned int size)
-{
-	return check_memory_region((unsigned long)p, size, true, _RET_IP_);
-}
-EXPORT_SYMBOL(__kasan_check_write);
-
-#undef memset
-void *memset(void *addr, int c, size_t len)
-{
-	if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_))
-		return NULL;
-
-	return __memset(addr, c, len);
-}
-
-#ifdef __HAVE_ARCH_MEMMOVE
-#undef memmove
-void *memmove(void *dest, const void *src, size_t len)
-{
-	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
-	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
-		return NULL;
-
-	return __memmove(dest, src, len);
-}
-#endif
-
-#undef memcpy
-void *memcpy(void *dest, const void *src, size_t len)
-{
-	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
-	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
-		return NULL;
-
-	return __memcpy(dest, src, len);
-}
-
-/*
- * Poisons the shadow memory for 'size' bytes starting from 'addr'.
- * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
- */
-void kasan_poison_memory(const void *address, size_t size, u8 value)
-{
-	void *shadow_start, *shadow_end;
-
-	/*
-	 * Perform shadow offset calculation based on untagged address, as
-	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
-	 * addresses to this function.
-	 */
-	address = reset_tag(address);
-
-	shadow_start = kasan_mem_to_shadow(address);
-	shadow_end = kasan_mem_to_shadow(address + size);
-
-	__memset(shadow_start, value, shadow_end - shadow_start);
-}
-
-void kasan_unpoison_memory(const void *address, size_t size)
-{
-	u8 tag = get_tag(address);
-
-	/*
-	 * Perform shadow offset calculation based on untagged address, as
-	 * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
-	 * addresses to this function.
-	 */
-	address = reset_tag(address);
-
-	kasan_poison_memory(address, size, tag);
-
-	if (size & KASAN_GRANULE_MASK) {
-		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
-
-		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
-			*shadow = tag;
-		else
-			*shadow = size & KASAN_GRANULE_MASK;
-	}
-}
-
 static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
 {
 	void *base = task_stack_page(task);
@@ -535,395 +443,3 @@ void kasan_kfree_large(void *ptr, unsigned long ip)
 		kasan_report_invalid_free(ptr, ip);
 	/* The object will be poisoned by page_alloc. */
 }
-
-#ifdef CONFIG_MEMORY_HOTPLUG
-static bool shadow_mapped(unsigned long addr)
-{
-	pgd_t *pgd = pgd_offset_k(addr);
-	p4d_t *p4d;
-	pud_t *pud;
-	pmd_t *pmd;
-	pte_t *pte;
-
-	if (pgd_none(*pgd))
-		return false;
-	p4d = p4d_offset(pgd, addr);
-	if (p4d_none(*p4d))
-		return false;
-	pud = pud_offset(p4d, addr);
-	if (pud_none(*pud))
-		return false;
-
-	/*
-	 * We can't use pud_large() or pud_huge(), the first one is
-	 * arch-specific, the last one depends on HUGETLB_PAGE.  So let's abuse
-	 * pud_bad(), if pud is bad then it's bad because it's huge.
-	 */
-	if (pud_bad(*pud))
-		return true;
-	pmd = pmd_offset(pud, addr);
-	if (pmd_none(*pmd))
-		return false;
-
-	if (pmd_bad(*pmd))
-		return true;
-	pte = pte_offset_kernel(pmd, addr);
-	return !pte_none(*pte);
-}
-
-static int __meminit kasan_mem_notifier(struct notifier_block *nb,
-			unsigned long action, void *data)
-{
-	struct memory_notify *mem_data = data;
-	unsigned long nr_shadow_pages, start_kaddr, shadow_start;
-	unsigned long shadow_end, shadow_size;
-
-	nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT;
-	start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn);
-	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr);
-	shadow_size = nr_shadow_pages << PAGE_SHIFT;
-	shadow_end = shadow_start + shadow_size;
-
-	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
-		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
-		return NOTIFY_BAD;
-
-	switch (action) {
-	case MEM_GOING_ONLINE: {
-		void *ret;
-
-		/*
-		 * If shadow is mapped already than it must have been mapped
-		 * during the boot. This could happen if we onlining previously
-		 * offlined memory.
-		 */
-		if (shadow_mapped(shadow_start))
-			return NOTIFY_OK;
-
-		ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start,
-					shadow_end, GFP_KERNEL,
-					PAGE_KERNEL, VM_NO_GUARD,
-					pfn_to_nid(mem_data->start_pfn),
-					__builtin_return_address(0));
-		if (!ret)
-			return NOTIFY_BAD;
-
-		kmemleak_ignore(ret);
-		return NOTIFY_OK;
-	}
-	case MEM_CANCEL_ONLINE:
-	case MEM_OFFLINE: {
-		struct vm_struct *vm;
-
-		/*
-		 * shadow_start was either mapped during boot by kasan_init()
-		 * or during memory online by __vmalloc_node_range().
-		 * In the latter case we can use vfree() to free shadow.
-		 * Non-NULL result of the find_vm_area() will tell us if
-		 * that was the second case.
-		 *
-		 * Currently it's not possible to free shadow mapped
-		 * during boot by kasan_init(). It's because the code
-		 * to do that hasn't been written yet. So we'll just
-		 * leak the memory.
-		 */
-		vm = find_vm_area((void *)shadow_start);
-		if (vm)
-			vfree((void *)shadow_start);
-	}
-	}
-
-	return NOTIFY_OK;
-}
-
-static int __init kasan_memhotplug_init(void)
-{
-	hotplug_memory_notifier(kasan_mem_notifier, 0);
-
-	return 0;
-}
-
-core_initcall(kasan_memhotplug_init);
-#endif
-
-#ifdef CONFIG_KASAN_VMALLOC
-
-static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
-				      void *unused)
-{
-	unsigned long page;
-	pte_t pte;
-
-	if (likely(!pte_none(*ptep)))
-		return 0;
-
-	page = __get_free_page(GFP_KERNEL);
-	if (!page)
-		return -ENOMEM;
-
-	memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
-	pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
-
-	spin_lock(&init_mm.page_table_lock);
-	if (likely(pte_none(*ptep))) {
-		set_pte_at(&init_mm, addr, ptep, pte);
-		page = 0;
-	}
-	spin_unlock(&init_mm.page_table_lock);
-	if (page)
-		free_page(page);
-	return 0;
-}
-
-int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
-{
-	unsigned long shadow_start, shadow_end;
-	int ret;
-
-	if (!is_vmalloc_or_module_addr((void *)addr))
-		return 0;
-
-	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
-	shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
-	shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
-	shadow_end = ALIGN(shadow_end, PAGE_SIZE);
-
-	ret = apply_to_page_range(&init_mm, shadow_start,
-				  shadow_end - shadow_start,
-				  kasan_populate_vmalloc_pte, NULL);
-	if (ret)
-		return ret;
-
-	flush_cache_vmap(shadow_start, shadow_end);
-
-	/*
-	 * We need to be careful about inter-cpu effects here. Consider:
-	 *
-	 *   CPU#0				  CPU#1
-	 * WRITE_ONCE(p, vmalloc(100));		while (x = READ_ONCE(p)) ;
-	 *					p[99] = 1;
-	 *
-	 * With compiler instrumentation, that ends up looking like this:
-	 *
-	 *   CPU#0				  CPU#1
-	 * // vmalloc() allocates memory
-	 * // let a = area->addr
-	 * // we reach kasan_populate_vmalloc
-	 * // and call kasan_unpoison_memory:
-	 * STORE shadow(a), unpoison_val
-	 * ...
-	 * STORE shadow(a+99), unpoison_val	x = LOAD p
-	 * // rest of vmalloc process		<data dependency>
-	 * STORE p, a				LOAD shadow(x+99)
-	 *
-	 * If there is no barrier between the end of unpoisioning the shadow
-	 * and the store of the result to p, the stores could be committed
-	 * in a different order by CPU#0, and CPU#1 could erroneously observe
-	 * poison in the shadow.
-	 *
-	 * We need some sort of barrier between the stores.
-	 *
-	 * In the vmalloc() case, this is provided by a smp_wmb() in
-	 * clear_vm_uninitialized_flag(). In the per-cpu allocator and in
-	 * get_vm_area() and friends, the caller gets shadow allocated but
-	 * doesn't have any pages mapped into the virtual address space that
-	 * has been reserved. Mapping those pages in will involve taking and
-	 * releasing a page-table lock, which will provide the barrier.
-	 */
-
-	return 0;
-}
-
-/*
- * Poison the shadow for a vmalloc region. Called as part of the
- * freeing process at the time the region is freed.
- */
-void kasan_poison_vmalloc(const void *start, unsigned long size)
-{
-	if (!is_vmalloc_or_module_addr(start))
-		return;
-
-	size = round_up(size, KASAN_GRANULE_SIZE);
-	kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
-}
-
-void kasan_unpoison_vmalloc(const void *start, unsigned long size)
-{
-	if (!is_vmalloc_or_module_addr(start))
-		return;
-
-	kasan_unpoison_memory(start, size);
-}
-
-static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
-					void *unused)
-{
-	unsigned long page;
-
-	page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT);
-
-	spin_lock(&init_mm.page_table_lock);
-
-	if (likely(!pte_none(*ptep))) {
-		pte_clear(&init_mm, addr, ptep);
-		free_page(page);
-	}
-	spin_unlock(&init_mm.page_table_lock);
-
-	return 0;
-}
-
-/*
- * Release the backing for the vmalloc region [start, end), which
- * lies within the free region [free_region_start, free_region_end).
- *
- * This can be run lazily, long after the region was freed. It runs
- * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap
- * infrastructure.
- *
- * How does this work?
- * -------------------
- *
- * We have a region that is page aligned, labelled as A.
- * That might not map onto the shadow in a way that is page-aligned:
- *
- *                    start                     end
- *                    v                         v
- * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc
- *  -------- -------- --------          -------- --------
- *      |        |       |                 |        |
- *      |        |       |         /-------/        |
- *      \-------\|/------/         |/---------------/
- *              |||                ||
- *             |??AAAAAA|AAAAAAAA|AA??????|                < shadow
- *                 (1)      (2)      (3)
- *
- * First we align the start upwards and the end downwards, so that the
- * shadow of the region aligns with shadow page boundaries. In the
- * example, this gives us the shadow page (2). This is the shadow entirely
- * covered by this allocation.
- *
- * Then we have the tricky bits. We want to know if we can free the
- * partially covered shadow pages - (1) and (3) in the example. For this,
- * we are given the start and end of the free region that contains this
- * allocation. Extending our previous example, we could have:
- *
- *  free_region_start                                    free_region_end
- *  |                 start                     end      |
- *  v                 v                         v        v
- * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc
- *  -------- -------- --------          -------- --------
- *      |        |       |                 |        |
- *      |        |       |         /-------/        |
- *      \-------\|/------/         |/---------------/
- *              |||                ||
- *             |FFAAAAAA|AAAAAAAA|AAF?????|                < shadow
- *                 (1)      (2)      (3)
- *
- * Once again, we align the start of the free region up, and the end of
- * the free region down so that the shadow is page aligned. So we can free
- * page (1) - we know no allocation currently uses anything in that page,
- * because all of it is in the vmalloc free region. But we cannot free
- * page (3), because we can't be sure that the rest of it is unused.
- *
- * We only consider pages that contain part of the original region for
- * freeing: we don't try to free other pages from the free region or we'd
- * end up trying to free huge chunks of virtual address space.
- *
- * Concurrency
- * -----------
- *
- * How do we know that we're not freeing a page that is simultaneously
- * being used for a fresh allocation in kasan_populate_vmalloc(_pte)?
- *
- * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running
- * at the same time. While we run under free_vmap_area_lock, the population
- * code does not.
- *
- * free_vmap_area_lock instead operates to ensure that the larger range
- * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and
- * the per-cpu region-finding algorithm both run under free_vmap_area_lock,
- * no space identified as free will become used while we are running. This
- * means that so long as we are careful with alignment and only free shadow
- * pages entirely covered by the free region, we will not run in to any
- * trouble - any simultaneous allocations will be for disjoint regions.
- */
-void kasan_release_vmalloc(unsigned long start, unsigned long end,
-			   unsigned long free_region_start,
-			   unsigned long free_region_end)
-{
-	void *shadow_start, *shadow_end;
-	unsigned long region_start, region_end;
-	unsigned long size;
-
-	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
-	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
-
-	free_region_start = ALIGN(free_region_start,
-				  PAGE_SIZE * KASAN_GRANULE_SIZE);
-
-	if (start != region_start &&
-	    free_region_start < region_start)
-		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
-
-	free_region_end = ALIGN_DOWN(free_region_end,
-				     PAGE_SIZE * KASAN_GRANULE_SIZE);
-
-	if (end != region_end &&
-	    free_region_end > region_end)
-		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
-
-	shadow_start = kasan_mem_to_shadow((void *)region_start);
-	shadow_end = kasan_mem_to_shadow((void *)region_end);
-
-	if (shadow_end > shadow_start) {
-		size = shadow_end - shadow_start;
-		apply_to_existing_page_range(&init_mm,
-					     (unsigned long)shadow_start,
-					     size, kasan_depopulate_vmalloc_pte,
-					     NULL);
-		flush_tlb_kernel_range((unsigned long)shadow_start,
-				       (unsigned long)shadow_end);
-	}
-}
-
-#else /* CONFIG_KASAN_VMALLOC */
-
-int kasan_module_alloc(void *addr, size_t size)
-{
-	void *ret;
-	size_t scaled_size;
-	size_t shadow_size;
-	unsigned long shadow_start;
-
-	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
-	scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
-				KASAN_SHADOW_SCALE_SHIFT;
-	shadow_size = round_up(scaled_size, PAGE_SIZE);
-
-	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
-		return -EINVAL;
-
-	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
-			shadow_start + shadow_size,
-			GFP_KERNEL,
-			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
-			__builtin_return_address(0));
-
-	if (ret) {
-		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
-		find_vm_area(addr)->flags |= VM_KASAN;
-		kmemleak_ignore(ret);
-		return 0;
-	}
-
-	return -ENOMEM;
-}
-
-void kasan_free_shadow(const struct vm_struct *vm)
-{
-	if (vm->flags & VM_KASAN)
-		vfree(kasan_mem_to_shadow(vm->addr));
-}
-
-#endif
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
new file mode 100644
index 000000000000..ca0cc4c31454
--- /dev/null
+++ b/mm/kasan/shadow.c
@@ -0,0 +1,505 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This file contains KASAN runtime code that manages shadow memory for
+ * generic and software tag-based KASAN modes.
+ *
+ * Copyright (c) 2014 Samsung Electronics Co., Ltd.
+ * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
+ *
+ * Some code borrowed from https://github.com/xairy/kasan-prototype by
+ *        Andrey Konovalov <andreyknvl@gmail.com>
+ */
+
+#include <linux/init.h>
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/kmemleak.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include <asm/cacheflush.h>
+#include <asm/tlbflush.h>
+
+#include "kasan.h"
+
+bool __kasan_check_read(const volatile void *p, unsigned int size)
+{
+	return check_memory_region((unsigned long)p, size, false, _RET_IP_);
+}
+EXPORT_SYMBOL(__kasan_check_read);
+
+bool __kasan_check_write(const volatile void *p, unsigned int size)
+{
+	return check_memory_region((unsigned long)p, size, true, _RET_IP_);
+}
+EXPORT_SYMBOL(__kasan_check_write);
+
+#undef memset
+void *memset(void *addr, int c, size_t len)
+{
+	if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_))
+		return NULL;
+
+	return __memset(addr, c, len);
+}
+
+#ifdef __HAVE_ARCH_MEMMOVE
+#undef memmove
+void *memmove(void *dest, const void *src, size_t len)
+{
+	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
+	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
+		return NULL;
+
+	return __memmove(dest, src, len);
+}
+#endif
+
+#undef memcpy
+void *memcpy(void *dest, const void *src, size_t len)
+{
+	if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
+	    !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
+		return NULL;
+
+	return __memcpy(dest, src, len);
+}
+
+/*
+ * Poisons the shadow memory for 'size' bytes starting from 'addr'.
+ * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
+ */
+void kasan_poison_memory(const void *address, size_t size, u8 value)
+{
+	void *shadow_start, *shadow_end;
+
+	/*
+	 * Perform shadow offset calculation based on untagged address, as
+	 * some of the callers (e.g. kasan_poison_object_data) pass tagged
+	 * addresses to this function.
+	 */
+	address = reset_tag(address);
+
+	shadow_start = kasan_mem_to_shadow(address);
+	shadow_end = kasan_mem_to_shadow(address + size);
+
+	__memset(shadow_start, value, shadow_end - shadow_start);
+}
+
+void kasan_unpoison_memory(const void *address, size_t size)
+{
+	u8 tag = get_tag(address);
+
+	/*
+	 * Perform shadow offset calculation based on untagged address, as
+	 * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
+	 * addresses to this function.
+	 */
+	address = reset_tag(address);
+
+	kasan_poison_memory(address, size, tag);
+
+	if (size & KASAN_GRANULE_MASK) {
+		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
+
+		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
+			*shadow = tag;
+		else
+			*shadow = size & KASAN_GRANULE_MASK;
+	}
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static bool shadow_mapped(unsigned long addr)
+{
+	pgd_t *pgd = pgd_offset_k(addr);
+	p4d_t *p4d;
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+
+	if (pgd_none(*pgd))
+		return false;
+	p4d = p4d_offset(pgd, addr);
+	if (p4d_none(*p4d))
+		return false;
+	pud = pud_offset(p4d, addr);
+	if (pud_none(*pud))
+		return false;
+
+	/*
+	 * We can't use pud_large() or pud_huge(), the first one is
+	 * arch-specific, the last one depends on HUGETLB_PAGE.  So let's abuse
+	 * pud_bad(), if pud is bad then it's bad because it's huge.
+	 */
+	if (pud_bad(*pud))
+		return true;
+	pmd = pmd_offset(pud, addr);
+	if (pmd_none(*pmd))
+		return false;
+
+	if (pmd_bad(*pmd))
+		return true;
+	pte = pte_offset_kernel(pmd, addr);
+	return !pte_none(*pte);
+}
+
+static int __meminit kasan_mem_notifier(struct notifier_block *nb,
+			unsigned long action, void *data)
+{
+	struct memory_notify *mem_data = data;
+	unsigned long nr_shadow_pages, start_kaddr, shadow_start;
+	unsigned long shadow_end, shadow_size;
+
+	nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT;
+	start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn);
+	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr);
+	shadow_size = nr_shadow_pages << PAGE_SHIFT;
+	shadow_end = shadow_start + shadow_size;
+
+	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
+		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
+		return NOTIFY_BAD;
+
+	switch (action) {
+	case MEM_GOING_ONLINE: {
+		void *ret;
+
+		/*
+		 * If shadow is mapped already than it must have been mapped
+		 * during the boot. This could happen if we onlining previously
+		 * offlined memory.
+		 */
+		if (shadow_mapped(shadow_start))
+			return NOTIFY_OK;
+
+		ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start,
+					shadow_end, GFP_KERNEL,
+					PAGE_KERNEL, VM_NO_GUARD,
+					pfn_to_nid(mem_data->start_pfn),
+					__builtin_return_address(0));
+		if (!ret)
+			return NOTIFY_BAD;
+
+		kmemleak_ignore(ret);
+		return NOTIFY_OK;
+	}
+	case MEM_CANCEL_ONLINE:
+	case MEM_OFFLINE: {
+		struct vm_struct *vm;
+
+		/*
+		 * shadow_start was either mapped during boot by kasan_init()
+		 * or during memory online by __vmalloc_node_range().
+		 * In the latter case we can use vfree() to free shadow.
+		 * Non-NULL result of the find_vm_area() will tell us if
+		 * that was the second case.
+		 *
+		 * Currently it's not possible to free shadow mapped
+		 * during boot by kasan_init(). It's because the code
+		 * to do that hasn't been written yet. So we'll just
+		 * leak the memory.
+		 */
+		vm = find_vm_area((void *)shadow_start);
+		if (vm)
+			vfree((void *)shadow_start);
+	}
+	}
+
+	return NOTIFY_OK;
+}
+
+static int __init kasan_memhotplug_init(void)
+{
+	hotplug_memory_notifier(kasan_mem_notifier, 0);
+
+	return 0;
+}
+
+core_initcall(kasan_memhotplug_init);
+#endif
+
+#ifdef CONFIG_KASAN_VMALLOC
+
+static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+				      void *unused)
+{
+	unsigned long page;
+	pte_t pte;
+
+	if (likely(!pte_none(*ptep)))
+		return 0;
+
+	page = __get_free_page(GFP_KERNEL);
+	if (!page)
+		return -ENOMEM;
+
+	memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
+	pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
+
+	spin_lock(&init_mm.page_table_lock);
+	if (likely(pte_none(*ptep))) {
+		set_pte_at(&init_mm, addr, ptep, pte);
+		page = 0;
+	}
+	spin_unlock(&init_mm.page_table_lock);
+	if (page)
+		free_page(page);
+	return 0;
+}
+
+int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
+{
+	unsigned long shadow_start, shadow_end;
+	int ret;
+
+	if (!is_vmalloc_or_module_addr((void *)addr))
+		return 0;
+
+	shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
+	shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
+	shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
+	shadow_end = ALIGN(shadow_end, PAGE_SIZE);
+
+	ret = apply_to_page_range(&init_mm, shadow_start,
+				  shadow_end - shadow_start,
+				  kasan_populate_vmalloc_pte, NULL);
+	if (ret)
+		return ret;
+
+	flush_cache_vmap(shadow_start, shadow_end);
+
+	/*
+	 * We need to be careful about inter-cpu effects here. Consider:
+	 *
+	 *   CPU#0				  CPU#1
+	 * WRITE_ONCE(p, vmalloc(100));		while (x = READ_ONCE(p)) ;
+	 *					p[99] = 1;
+	 *
+	 * With compiler instrumentation, that ends up looking like this:
+	 *
+	 *   CPU#0				  CPU#1
+	 * // vmalloc() allocates memory
+	 * // let a = area->addr
+	 * // we reach kasan_populate_vmalloc
+	 * // and call kasan_unpoison_memory:
+	 * STORE shadow(a), unpoison_val
+	 * ...
+	 * STORE shadow(a+99), unpoison_val	x = LOAD p
+	 * // rest of vmalloc process		<data dependency>
+	 * STORE p, a				LOAD shadow(x+99)
+	 *
+	 * If there is no barrier between the end of unpoisioning the shadow
+	 * and the store of the result to p, the stores could be committed
+	 * in a different order by CPU#0, and CPU#1 could erroneously observe
+	 * poison in the shadow.
+	 *
+	 * We need some sort of barrier between the stores.
+	 *
+	 * In the vmalloc() case, this is provided by a smp_wmb() in
+	 * clear_vm_uninitialized_flag(). In the per-cpu allocator and in
+	 * get_vm_area() and friends, the caller gets shadow allocated but
+	 * doesn't have any pages mapped into the virtual address space that
+	 * has been reserved. Mapping those pages in will involve taking and
+	 * releasing a page-table lock, which will provide the barrier.
+	 */
+
+	return 0;
+}
+
+/*
+ * Poison the shadow for a vmalloc region. Called as part of the
+ * freeing process at the time the region is freed.
+ */
+void kasan_poison_vmalloc(const void *start, unsigned long size)
+{
+	if (!is_vmalloc_or_module_addr(start))
+		return;
+
+	size = round_up(size, KASAN_GRANULE_SIZE);
+	kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
+}
+
+void kasan_unpoison_vmalloc(const void *start, unsigned long size)
+{
+	if (!is_vmalloc_or_module_addr(start))
+		return;
+
+	kasan_unpoison_memory(start, size);
+}
+
+static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+					void *unused)
+{
+	unsigned long page;
+
+	page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT);
+
+	spin_lock(&init_mm.page_table_lock);
+
+	if (likely(!pte_none(*ptep))) {
+		pte_clear(&init_mm, addr, ptep);
+		free_page(page);
+	}
+	spin_unlock(&init_mm.page_table_lock);
+
+	return 0;
+}
+
+/*
+ * Release the backing for the vmalloc region [start, end), which
+ * lies within the free region [free_region_start, free_region_end).
+ *
+ * This can be run lazily, long after the region was freed. It runs
+ * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap
+ * infrastructure.
+ *
+ * How does this work?
+ * -------------------
+ *
+ * We have a region that is page aligned, labelled as A.
+ * That might not map onto the shadow in a way that is page-aligned:
+ *
+ *                    start                     end
+ *                    v                         v
+ * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc
+ *  -------- -------- --------          -------- --------
+ *      |        |       |                 |        |
+ *      |        |       |         /-------/        |
+ *      \-------\|/------/         |/---------------/
+ *              |||                ||
+ *             |??AAAAAA|AAAAAAAA|AA??????|                < shadow
+ *                 (1)      (2)      (3)
+ *
+ * First we align the start upwards and the end downwards, so that the
+ * shadow of the region aligns with shadow page boundaries. In the
+ * example, this gives us the shadow page (2). This is the shadow entirely
+ * covered by this allocation.
+ *
+ * Then we have the tricky bits. We want to know if we can free the
+ * partially covered shadow pages - (1) and (3) in the example. For this,
+ * we are given the start and end of the free region that contains this
+ * allocation. Extending our previous example, we could have:
+ *
+ *  free_region_start                                    free_region_end
+ *  |                 start                     end      |
+ *  v                 v                         v        v
+ * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc
+ *  -------- -------- --------          -------- --------
+ *      |        |       |                 |        |
+ *      |        |       |         /-------/        |
+ *      \-------\|/------/         |/---------------/
+ *              |||                ||
+ *             |FFAAAAAA|AAAAAAAA|AAF?????|                < shadow
+ *                 (1)      (2)      (3)
+ *
+ * Once again, we align the start of the free region up, and the end of
+ * the free region down so that the shadow is page aligned. So we can free
+ * page (1) - we know no allocation currently uses anything in that page,
+ * because all of it is in the vmalloc free region. But we cannot free
+ * page (3), because we can't be sure that the rest of it is unused.
+ *
+ * We only consider pages that contain part of the original region for
+ * freeing: we don't try to free other pages from the free region or we'd
+ * end up trying to free huge chunks of virtual address space.
+ *
+ * Concurrency
+ * -----------
+ *
+ * How do we know that we're not freeing a page that is simultaneously
+ * being used for a fresh allocation in kasan_populate_vmalloc(_pte)?
+ *
+ * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running
+ * at the same time. While we run under free_vmap_area_lock, the population
+ * code does not.
+ *
+ * free_vmap_area_lock instead operates to ensure that the larger range
+ * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and
+ * the per-cpu region-finding algorithm both run under free_vmap_area_lock,
+ * no space identified as free will become used while we are running. This
+ * means that so long as we are careful with alignment and only free shadow
+ * pages entirely covered by the free region, we will not run in to any
+ * trouble - any simultaneous allocations will be for disjoint regions.
+ */
+void kasan_release_vmalloc(unsigned long start, unsigned long end,
+			   unsigned long free_region_start,
+			   unsigned long free_region_end)
+{
+	void *shadow_start, *shadow_end;
+	unsigned long region_start, region_end;
+	unsigned long size;
+
+	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
+	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
+
+	free_region_start = ALIGN(free_region_start,
+				  PAGE_SIZE * KASAN_GRANULE_SIZE);
+
+	if (start != region_start &&
+	    free_region_start < region_start)
+		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
+
+	free_region_end = ALIGN_DOWN(free_region_end,
+				     PAGE_SIZE * KASAN_GRANULE_SIZE);
+
+	if (end != region_end &&
+	    free_region_end > region_end)
+		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
+
+	shadow_start = kasan_mem_to_shadow((void *)region_start);
+	shadow_end = kasan_mem_to_shadow((void *)region_end);
+
+	if (shadow_end > shadow_start) {
+		size = shadow_end - shadow_start;
+		apply_to_existing_page_range(&init_mm,
+					     (unsigned long)shadow_start,
+					     size, kasan_depopulate_vmalloc_pte,
+					     NULL);
+		flush_tlb_kernel_range((unsigned long)shadow_start,
+				       (unsigned long)shadow_end);
+	}
+}
+
+#else /* CONFIG_KASAN_VMALLOC */
+
+int kasan_module_alloc(void *addr, size_t size)
+{
+	void *ret;
+	size_t scaled_size;
+	size_t shadow_size;
+	unsigned long shadow_start;
+
+	shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
+	scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
+				KASAN_SHADOW_SCALE_SHIFT;
+	shadow_size = round_up(scaled_size, PAGE_SIZE);
+
+	if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
+		return -EINVAL;
+
+	ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
+			shadow_start + shadow_size,
+			GFP_KERNEL,
+			PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
+			__builtin_return_address(0));
+
+	if (ret) {
+		__memset(ret, KASAN_SHADOW_INIT, shadow_size);
+		find_vm_area(addr)->flags |= VM_KASAN;
+		kmemleak_ignore(ret);
+		return 0;
+	}
+
+	return -ENOMEM;
+}
+
+void kasan_free_shadow(const struct vm_struct *vm)
+{
+	if (vm->flags & VM_KASAN)
+		vfree(kasan_mem_to_shadow(vm->addr));
+}
+
+#endif
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (8 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 09/44] kasan: split out shadow.c from common.c Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:13   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 11/44] kasan: rename report and tags files Andrey Konovalov
                   ` (34 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Define KASAN_GRANULE_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT), which is
the same as (KASAN_GRANULE_SIZE * PAGE_SIZE), and use it across KASAN code
to simplify it.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I0b627b24187d06c8b9bb2f1d04d94b3d06945e73
---
 mm/kasan/init.c   | 10 ++++------
 mm/kasan/kasan.h  |  1 +
 mm/kasan/shadow.c | 16 +++++++---------
 3 files changed, 12 insertions(+), 15 deletions(-)

diff --git a/mm/kasan/init.c b/mm/kasan/init.c
index 1a71eaa8c5f9..26b2663b3a42 100644
--- a/mm/kasan/init.c
+++ b/mm/kasan/init.c
@@ -441,9 +441,8 @@ void kasan_remove_zero_shadow(void *start, unsigned long size)
 	addr = (unsigned long)kasan_mem_to_shadow(start);
 	end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
-	if (WARN_ON((unsigned long)start %
-			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
+	if (WARN_ON((unsigned long)start % KASAN_GRANULE_PAGE) ||
+	    WARN_ON(size % KASAN_GRANULE_PAGE))
 		return;
 
 	for (; addr < end; addr = next) {
@@ -476,9 +475,8 @@ int kasan_add_zero_shadow(void *start, unsigned long size)
 	shadow_start = kasan_mem_to_shadow(start);
 	shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT);
 
-	if (WARN_ON((unsigned long)start %
-			(KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
-	    WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
+	if (WARN_ON((unsigned long)start % KASAN_GRANULE_PAGE) ||
+	    WARN_ON(size % KASAN_GRANULE_PAGE))
 		return -EINVAL;
 
 	ret = kasan_populate_early_shadow(shadow_start, shadow_end);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index c31e2c739301..1865bb92d47a 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -7,6 +7,7 @@
 
 #define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
 #define KASAN_GRANULE_MASK	(KASAN_GRANULE_SIZE - 1)
+#define KASAN_GRANULE_PAGE	(KASAN_GRANULE_SIZE << PAGE_SHIFT)
 
 #define KASAN_TAG_KERNEL	0xFF /* native kernel pointers tag */
 #define KASAN_TAG_INVALID	0xFE /* inaccessible memory tag */
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index ca0cc4c31454..1fadd4930d54 100644
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -161,7 +161,7 @@ static int __meminit kasan_mem_notifier(struct notifier_block *nb,
 	shadow_end = shadow_start + shadow_size;
 
 	if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
-		WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
+		WARN_ON(start_kaddr % KASAN_GRANULE_PAGE))
 		return NOTIFY_BAD;
 
 	switch (action) {
@@ -432,22 +432,20 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
 	unsigned long region_start, region_end;
 	unsigned long size;
 
-	region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
-	region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
+	region_start = ALIGN(start, KASAN_GRANULE_PAGE);
+	region_end = ALIGN_DOWN(end, KASAN_GRANULE_PAGE);
 
-	free_region_start = ALIGN(free_region_start,
-				  PAGE_SIZE * KASAN_GRANULE_SIZE);
+	free_region_start = ALIGN(free_region_start, KASAN_GRANULE_PAGE);
 
 	if (start != region_start &&
 	    free_region_start < region_start)
-		region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
+		region_start -= KASAN_GRANULE_PAGE;
 
-	free_region_end = ALIGN_DOWN(free_region_end,
-				     PAGE_SIZE * KASAN_GRANULE_SIZE);
+	free_region_end = ALIGN_DOWN(free_region_end, KASAN_GRANULE_PAGE);
 
 	if (end != region_end &&
 	    free_region_end > region_end)
-		region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
+		region_end += KASAN_GRANULE_PAGE;
 
 	shadow_start = kasan_mem_to_shadow((void *)region_start);
 	shadow_end = kasan_mem_to_shadow((void *)region_end);
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 11/44] kasan: rename report and tags files
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (9 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:15   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 12/44] kasan: don't duplicate config dependencies Andrey Konovalov
                   ` (33 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Rename generic_report.c to report_generic.c and tags_report.c to
report_sw_tags.c, as their content is more relevant to report.c file.
Also rename tags.c to sw_tags.c to better reflect that this file contains
code for software tag-based mode.

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: If77d21f655d52ef3e58c4c37fd6621a07f505f18
---
 mm/kasan/Makefile                               | 16 ++++++++--------
 mm/kasan/report.c                               |  2 +-
 mm/kasan/{generic_report.c => report_generic.c} |  0
 mm/kasan/{tags_report.c => report_sw_tags.c}    |  0
 mm/kasan/{tags.c => sw_tags.c}                  |  0
 5 files changed, 9 insertions(+), 9 deletions(-)
 rename mm/kasan/{generic_report.c => report_generic.c} (100%)
 rename mm/kasan/{tags_report.c => report_sw_tags.c} (100%)
 rename mm/kasan/{tags.c => sw_tags.c} (100%)

diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index 7cc1031e1ef8..f1d68a34f3c9 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -6,13 +6,13 @@ KCOV_INSTRUMENT := n
 # Disable ftrace to avoid recursion.
 CFLAGS_REMOVE_common.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_generic.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_generic_report.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE)
 
 # Function splitter causes unnecessary splits in __asan_load1/__asan_store1
 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533
@@ -23,14 +23,14 @@ CC_FLAGS_KASAN_RUNTIME += -DDISABLE_BRANCH_PROFILING
 
 CFLAGS_common.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
-CFLAGS_generic_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
-CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
-CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 
 obj-$(CONFIG_KASAN) := common.o report.o
-obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o
-obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o
+obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o
+obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 7b8dcb799a78..fff0c7befbfe 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains common generic and tag-based KASAN error reporting code.
+ * This file contains common KASAN error reporting code.
  *
  * Copyright (c) 2014 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
diff --git a/mm/kasan/generic_report.c b/mm/kasan/report_generic.c
similarity index 100%
rename from mm/kasan/generic_report.c
rename to mm/kasan/report_generic.c
diff --git a/mm/kasan/tags_report.c b/mm/kasan/report_sw_tags.c
similarity index 100%
rename from mm/kasan/tags_report.c
rename to mm/kasan/report_sw_tags.c
diff --git a/mm/kasan/tags.c b/mm/kasan/sw_tags.c
similarity index 100%
rename from mm/kasan/tags.c
rename to mm/kasan/sw_tags.c
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 12/44] kasan: don't duplicate config dependencies
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (10 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 11/44] kasan: rename report and tags files Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:20   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 13/44] kasan: hide invalid free check implementation Andrey Konovalov
                   ` (32 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Both KASAN_GENERIC and KASAN_SW_TAGS have common dependencies, move
those to KASAN.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I77e475802e8f1750b9154fe4a6e6da4456054fcd
---
 lib/Kconfig.kasan | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
index 8f0742a0f23e..ec59a0e26d09 100644
--- a/lib/Kconfig.kasan
+++ b/lib/Kconfig.kasan
@@ -24,6 +24,8 @@ menuconfig KASAN
 		   (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)
 	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
 	depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS
+	select CONSTRUCTORS
+	select STACKDEPOT
 	help
 	  Enables KASAN (KernelAddressSANitizer) - runtime memory debugger,
 	  designed to find out-of-bounds accesses and use-after-free bugs.
@@ -46,10 +48,7 @@ choice
 config KASAN_GENERIC
 	bool "Generic mode"
 	depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC
-	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
 	select SLUB_DEBUG if SLUB
-	select CONSTRUCTORS
-	select STACKDEPOT
 	help
 	  Enables generic KASAN mode.
 
@@ -70,10 +69,7 @@ config KASAN_GENERIC
 config KASAN_SW_TAGS
 	bool "Software tag-based mode"
 	depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS
-	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
 	select SLUB_DEBUG if SLUB
-	select CONSTRUCTORS
-	select STACKDEPOT
 	help
 	  Enables software tag-based KASAN mode.
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 13/44] kasan: hide invalid free check implementation
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (11 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 12/44] kasan: don't duplicate config dependencies Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:23   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrey Konovalov
                   ` (31 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

For software KASAN modes the check is based on the value in the shadow
memory. Hardware tag-based KASAN won't be using shadow, so hide the
implementation of the check in check_invalid_free().

Also simplify the code for software tag-based mode.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I5fae9531c9fc948eb4d4e0c589744032fc5a0789
---
 mm/kasan/common.c  | 19 +------------------
 mm/kasan/generic.c |  7 +++++++
 mm/kasan/kasan.h   |  2 ++
 mm/kasan/sw_tags.c |  9 +++++++++
 4 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 123abfb760d4..543e6bf2168f 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -272,25 +272,9 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
 	return (void *)object;
 }
 
-static inline bool shadow_invalid(u8 tag, s8 shadow_byte)
-{
-	if (IS_ENABLED(CONFIG_KASAN_GENERIC))
-		return shadow_byte < 0 ||
-			shadow_byte >= KASAN_GRANULE_SIZE;
-
-	/* else CONFIG_KASAN_SW_TAGS: */
-	if ((u8)shadow_byte == KASAN_TAG_INVALID)
-		return true;
-	if ((tag != KASAN_TAG_KERNEL) && (tag != (u8)shadow_byte))
-		return true;
-
-	return false;
-}
-
 static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
 			      unsigned long ip, bool quarantine)
 {
-	s8 shadow_byte;
 	u8 tag;
 	void *tagged_object;
 	unsigned long rounded_up_size;
@@ -309,8 +293,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
 	if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU))
 		return false;
 
-	shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object));
-	if (shadow_invalid(tag, shadow_byte)) {
+	if (check_invalid_free(tagged_object)) {
 		kasan_report_invalid_free(tagged_object, ip);
 		return true;
 	}
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index ec4417156943..e1af3b6c53b8 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -187,6 +187,13 @@ bool check_memory_region(unsigned long addr, size_t size, bool write,
 	return check_memory_region_inline(addr, size, write, ret_ip);
 }
 
+bool check_invalid_free(void *addr)
+{
+	s8 shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr));
+
+	return shadow_byte < 0 || shadow_byte >= KASAN_GRANULE_SIZE;
+}
+
 void kasan_cache_shrink(struct kmem_cache *cache)
 {
 	quarantine_remove_cache(cache);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 1865bb92d47a..3eff57e71ff5 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -164,6 +164,8 @@ void kasan_poison_memory(const void *address, size_t size, u8 value);
 bool check_memory_region(unsigned long addr, size_t size, bool write,
 				unsigned long ret_ip);
 
+bool check_invalid_free(void *addr);
+
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
 
diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
index 4bdd7dbd6647..b2638c2cd58a 100644
--- a/mm/kasan/sw_tags.c
+++ b/mm/kasan/sw_tags.c
@@ -121,6 +121,15 @@ bool check_memory_region(unsigned long addr, size_t size, bool write,
 	return true;
 }
 
+bool check_invalid_free(void *addr)
+{
+	u8 tag = get_tag(addr);
+	u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr)));
+
+	return (shadow_byte == KASAN_TAG_INVALID) ||
+		(tag != KASAN_TAG_KERNEL && tag != shadow_byte);
+}
+
 #define DEFINE_HWASAN_LOAD_STORE(size)					\
 	void __hwasan_load##size##_noabort(unsigned long addr)		\
 	{								\
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (12 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 13/44] kasan: hide invalid free check implementation Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:25   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 15/44] kasan, arm64: only init shadow for software modes Andrey Konovalov
                   ` (30 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Decoding routines aren't needed when CONFIG_KASAN_STACK_ENABLE is not
enabled. Currently only generic KASAN mode implements stack error
reporting.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I084e3214f2b40dc0bef7c5a9fafdc6f5c42b06a2
---
 mm/kasan/kasan.h          |   6 ++
 mm/kasan/report.c         | 162 --------------------------------------
 mm/kasan/report_generic.c | 162 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 168 insertions(+), 162 deletions(-)

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3eff57e71ff5..d0cf61d4d70d 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -169,6 +169,12 @@ bool check_invalid_free(void *addr);
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
 
+#if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK
+void print_address_stack_frame(const void *addr);
+#else
+static inline void print_address_stack_frame(const void *addr) { }
+#endif
+
 bool kasan_report(unsigned long addr, size_t size,
 		bool is_write, unsigned long ip);
 void kasan_report_invalid_free(void *object, unsigned long ip);
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index fff0c7befbfe..b18d193f7f58 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -211,168 +211,6 @@ static inline bool init_task_stack_addr(const void *addr)
 			sizeof(init_thread_union.stack));
 }
 
-static bool __must_check tokenize_frame_descr(const char **frame_descr,
-					      char *token, size_t max_tok_len,
-					      unsigned long *value)
-{
-	const char *sep = strchr(*frame_descr, ' ');
-
-	if (sep == NULL)
-		sep = *frame_descr + strlen(*frame_descr);
-
-	if (token != NULL) {
-		const size_t tok_len = sep - *frame_descr;
-
-		if (tok_len + 1 > max_tok_len) {
-			pr_err("KASAN internal error: frame description too long: %s\n",
-			       *frame_descr);
-			return false;
-		}
-
-		/* Copy token (+ 1 byte for '\0'). */
-		strlcpy(token, *frame_descr, tok_len + 1);
-	}
-
-	/* Advance frame_descr past separator. */
-	*frame_descr = sep + 1;
-
-	if (value != NULL && kstrtoul(token, 10, value)) {
-		pr_err("KASAN internal error: not a valid number: %s\n", token);
-		return false;
-	}
-
-	return true;
-}
-
-static void print_decoded_frame_descr(const char *frame_descr)
-{
-	/*
-	 * We need to parse the following string:
-	 *    "n alloc_1 alloc_2 ... alloc_n"
-	 * where alloc_i looks like
-	 *    "offset size len name"
-	 * or "offset size len name:line".
-	 */
-
-	char token[64];
-	unsigned long num_objects;
-
-	if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-				  &num_objects))
-		return;
-
-	pr_err("\n");
-	pr_err("this frame has %lu %s:\n", num_objects,
-	       num_objects == 1 ? "object" : "objects");
-
-	while (num_objects--) {
-		unsigned long offset;
-		unsigned long size;
-
-		/* access offset */
-		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-					  &offset))
-			return;
-		/* access size */
-		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-					  &size))
-			return;
-		/* name length (unused) */
-		if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL))
-			return;
-		/* object name */
-		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
-					  NULL))
-			return;
-
-		/* Strip line number; without filename it's not very helpful. */
-		strreplace(token, ':', '\0');
-
-		/* Finally, print object information. */
-		pr_err(" [%lu, %lu) '%s'", offset, offset + size, token);
-	}
-}
-
-static bool __must_check get_address_stack_frame_info(const void *addr,
-						      unsigned long *offset,
-						      const char **frame_descr,
-						      const void **frame_pc)
-{
-	unsigned long aligned_addr;
-	unsigned long mem_ptr;
-	const u8 *shadow_bottom;
-	const u8 *shadow_ptr;
-	const unsigned long *frame;
-
-	BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP));
-
-	/*
-	 * NOTE: We currently only support printing frame information for
-	 * accesses to the task's own stack.
-	 */
-	if (!object_is_on_stack(addr))
-		return false;
-
-	aligned_addr = round_down((unsigned long)addr, sizeof(long));
-	mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
-	shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
-	shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
-
-	while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
-		shadow_ptr--;
-		mem_ptr -= KASAN_GRANULE_SIZE;
-	}
-
-	while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
-		shadow_ptr--;
-		mem_ptr -= KASAN_GRANULE_SIZE;
-	}
-
-	if (shadow_ptr < shadow_bottom)
-		return false;
-
-	frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
-	if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
-		pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
-		       frame[0]);
-		return false;
-	}
-
-	*offset = (unsigned long)addr - (unsigned long)frame;
-	*frame_descr = (const char *)frame[1];
-	*frame_pc = (void *)frame[2];
-
-	return true;
-}
-
-static void print_address_stack_frame(const void *addr)
-{
-	unsigned long offset;
-	const char *frame_descr;
-	const void *frame_pc;
-
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
-		return;
-
-	if (!get_address_stack_frame_info(addr, &offset, &frame_descr,
-					  &frame_pc))
-		return;
-
-	/*
-	 * get_address_stack_frame_info only returns true if the given addr is
-	 * on the current task's stack.
-	 */
-	pr_err("\n");
-	pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n",
-	       addr, current->comm, task_pid_nr(current), offset);
-	pr_err(" %pS\n", frame_pc);
-
-	if (!frame_descr)
-		return;
-
-	print_decoded_frame_descr(frame_descr);
-}
-
 static void print_address_description(void *addr, u8 tag)
 {
 	struct page *page = kasan_addr_to_page(addr);
diff --git a/mm/kasan/report_generic.c b/mm/kasan/report_generic.c
index 7d5b9e5c7cfe..b543a1ed6078 100644
--- a/mm/kasan/report_generic.c
+++ b/mm/kasan/report_generic.c
@@ -16,6 +16,7 @@
 #include <linux/mm.h>
 #include <linux/printk.h>
 #include <linux/sched.h>
+#include <linux/sched/task_stack.h>
 #include <linux/slab.h>
 #include <linux/stackdepot.h>
 #include <linux/stacktrace.h>
@@ -122,6 +123,167 @@ const char *get_bug_type(struct kasan_access_info *info)
 	return get_wild_bug_type(info);
 }
 
+#if CONFIG_KASAN_STACK
+static bool __must_check tokenize_frame_descr(const char **frame_descr,
+					      char *token, size_t max_tok_len,
+					      unsigned long *value)
+{
+	const char *sep = strchr(*frame_descr, ' ');
+
+	if (sep == NULL)
+		sep = *frame_descr + strlen(*frame_descr);
+
+	if (token != NULL) {
+		const size_t tok_len = sep - *frame_descr;
+
+		if (tok_len + 1 > max_tok_len) {
+			pr_err("KASAN internal error: frame description too long: %s\n",
+			       *frame_descr);
+			return false;
+		}
+
+		/* Copy token (+ 1 byte for '\0'). */
+		strlcpy(token, *frame_descr, tok_len + 1);
+	}
+
+	/* Advance frame_descr past separator. */
+	*frame_descr = sep + 1;
+
+	if (value != NULL && kstrtoul(token, 10, value)) {
+		pr_err("KASAN internal error: not a valid number: %s\n", token);
+		return false;
+	}
+
+	return true;
+}
+
+static void print_decoded_frame_descr(const char *frame_descr)
+{
+	/*
+	 * We need to parse the following string:
+	 *    "n alloc_1 alloc_2 ... alloc_n"
+	 * where alloc_i looks like
+	 *    "offset size len name"
+	 * or "offset size len name:line".
+	 */
+
+	char token[64];
+	unsigned long num_objects;
+
+	if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+				  &num_objects))
+		return;
+
+	pr_err("\n");
+	pr_err("this frame has %lu %s:\n", num_objects,
+	       num_objects == 1 ? "object" : "objects");
+
+	while (num_objects--) {
+		unsigned long offset;
+		unsigned long size;
+
+		/* access offset */
+		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+					  &offset))
+			return;
+		/* access size */
+		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+					  &size))
+			return;
+		/* name length (unused) */
+		if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL))
+			return;
+		/* object name */
+		if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
+					  NULL))
+			return;
+
+		/* Strip line number; without filename it's not very helpful. */
+		strreplace(token, ':', '\0');
+
+		/* Finally, print object information. */
+		pr_err(" [%lu, %lu) '%s'", offset, offset + size, token);
+	}
+}
+
+static bool __must_check get_address_stack_frame_info(const void *addr,
+						      unsigned long *offset,
+						      const char **frame_descr,
+						      const void **frame_pc)
+{
+	unsigned long aligned_addr;
+	unsigned long mem_ptr;
+	const u8 *shadow_bottom;
+	const u8 *shadow_ptr;
+	const unsigned long *frame;
+
+	BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP));
+
+	/*
+	 * NOTE: We currently only support printing frame information for
+	 * accesses to the task's own stack.
+	 */
+	if (!object_is_on_stack(addr))
+		return false;
+
+	aligned_addr = round_down((unsigned long)addr, sizeof(long));
+	mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
+	shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
+	shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
+
+	while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
+		shadow_ptr--;
+		mem_ptr -= KASAN_GRANULE_SIZE;
+	}
+
+	while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
+		shadow_ptr--;
+		mem_ptr -= KASAN_GRANULE_SIZE;
+	}
+
+	if (shadow_ptr < shadow_bottom)
+		return false;
+
+	frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
+	if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
+		pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
+		       frame[0]);
+		return false;
+	}
+
+	*offset = (unsigned long)addr - (unsigned long)frame;
+	*frame_descr = (const char *)frame[1];
+	*frame_pc = (void *)frame[2];
+
+	return true;
+}
+
+void print_address_stack_frame(const void *addr)
+{
+	unsigned long offset;
+	const char *frame_descr;
+	const void *frame_pc;
+
+	if (!get_address_stack_frame_info(addr, &offset, &frame_descr,
+					  &frame_pc))
+		return;
+
+	/*
+	 * get_address_stack_frame_info only returns true if the given addr is
+	 * on the current task's stack.
+	 */
+	pr_err("\n");
+	pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n",
+	       addr, current->comm, task_pid_nr(current), offset);
+	pr_err(" %pS\n", frame_pc);
+
+	if (!frame_descr)
+		return;
+
+	print_decoded_frame_descr(frame_descr);
+}
+#endif /* CONFIG_KASAN_STACK */
+
 #define DEFINE_ASAN_REPORT_LOAD(size)                     \
 void __asan_report_load##size##_noabort(unsigned long addr) \
 {                                                         \
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 15/44] kasan, arm64: only init shadow for software modes
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (13 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:30   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 16/44] kasan, arm64: only use kasan_depth " Andrey Konovalov
                   ` (29 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory. Only initialize
it when one of the software KASAN modes are enabled.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I055e0651369b14d3e54cdaa8c48e6329b2e8952d
---
 arch/arm64/include/asm/kasan.h |  8 ++++++--
 arch/arm64/mm/kasan_init.c     | 15 ++++++++++++++-
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h
index b0dc4abc3589..f7ea70d02cab 100644
--- a/arch/arm64/include/asm/kasan.h
+++ b/arch/arm64/include/asm/kasan.h
@@ -13,6 +13,12 @@
 #define arch_kasan_get_tag(addr)	__tag_get(addr)
 
 #ifdef CONFIG_KASAN
+void kasan_init(void);
+#else
+static inline void kasan_init(void) { }
+#endif
+
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 
 /*
  * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
@@ -33,12 +39,10 @@
 #define _KASAN_SHADOW_START(va)	(KASAN_SHADOW_END - (1UL << ((va) - KASAN_SHADOW_SCALE_SHIFT)))
 #define KASAN_SHADOW_START      _KASAN_SHADOW_START(vabits_actual)
 
-void kasan_init(void);
 void kasan_copy_shadow(pgd_t *pgdir);
 asmlinkage void kasan_early_init(void);
 
 #else
-static inline void kasan_init(void) { }
 static inline void kasan_copy_shadow(pgd_t *pgdir) { }
 #endif
 
diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
index b24e43d20667..ffeb80d5aa8d 100644
--- a/arch/arm64/mm/kasan_init.c
+++ b/arch/arm64/mm/kasan_init.c
@@ -21,6 +21,8 @@
 #include <asm/sections.h>
 #include <asm/tlbflush.h>
 
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
+
 static pgd_t tmp_pg_dir[PTRS_PER_PGD] __initdata __aligned(PGD_SIZE);
 
 /*
@@ -208,7 +210,7 @@ static void __init clear_pgds(unsigned long start,
 		set_pgd(pgd_offset_k(start), __pgd(0));
 }
 
-void __init kasan_init(void)
+static void __init kasan_init_shadow(void)
 {
 	u64 kimg_shadow_start, kimg_shadow_end;
 	u64 mod_shadow_start, mod_shadow_end;
@@ -269,6 +271,17 @@ void __init kasan_init(void)
 
 	memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE);
 	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
+}
+
+#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
+
+static inline void __init kasan_init_shadow(void) { }
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+void __init kasan_init(void)
+{
+	kasan_init_shadow();
 
 	/* At this point kasan is fully initialized. Enable error messages */
 	init_task.kasan_depth = 0;
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 16/44] kasan, arm64: only use kasan_depth for software modes
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (14 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 15/44] kasan, arm64: only init shadow for software modes Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 14:59   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 17/44] kasan, arm64: move initialization message Andrey Konovalov
                   ` (28 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't use kasan_depth. Only define and use it
when one of the software KASAN modes are enabled.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I6109ea96c8df41ef6d75ad71bf22c1c8fa234a9a
---
 arch/arm64/mm/kasan_init.c | 11 ++++++++---
 include/linux/kasan.h      | 18 +++++++++---------
 include/linux/sched.h      |  2 +-
 init/init_task.c           |  2 +-
 mm/kasan/common.c          |  2 ++
 mm/kasan/report.c          |  2 ++
 6 files changed, 23 insertions(+), 14 deletions(-)

diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
index ffeb80d5aa8d..5172799f831f 100644
--- a/arch/arm64/mm/kasan_init.c
+++ b/arch/arm64/mm/kasan_init.c
@@ -273,17 +273,22 @@ static void __init kasan_init_shadow(void)
 	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
 }
 
+static void __init kasan_init_depth(void)
+{
+	init_task.kasan_depth = 0;
+}
+
 #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
 
 static inline void __init kasan_init_shadow(void) { }
 
+static inline void __init kasan_init_depth(void) { }
+
 #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 void __init kasan_init(void)
 {
 	kasan_init_shadow();
-
-	/* At this point kasan is fully initialized. Enable error messages */
-	init_task.kasan_depth = 0;
+	kasan_init_depth();
 	pr_info("KernelAddressSanitizer initialized\n");
 }
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index f6435b9f889c..979d598e1c30 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -51,6 +51,12 @@ static inline void *kasan_mem_to_shadow(const void *addr)
 int kasan_add_zero_shadow(void *start, unsigned long size);
 void kasan_remove_zero_shadow(void *start, unsigned long size);
 
+/* Enable reporting bugs after kasan_disable_current() */
+extern void kasan_enable_current(void);
+
+/* Disable reporting bugs for current task */
+extern void kasan_disable_current(void);
+
 #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 static inline int kasan_add_zero_shadow(void *start, unsigned long size)
@@ -61,16 +67,13 @@ static inline void kasan_remove_zero_shadow(void *start,
 					unsigned long size)
 {}
 
+static inline void kasan_enable_current(void) {}
+static inline void kasan_disable_current(void) {}
+
 #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 #ifdef CONFIG_KASAN
 
-/* Enable reporting bugs after kasan_disable_current() */
-extern void kasan_enable_current(void);
-
-/* Disable reporting bugs for current task */
-extern void kasan_disable_current(void);
-
 void kasan_unpoison_memory(const void *address, size_t size);
 
 void kasan_unpoison_task_stack(struct task_struct *task);
@@ -121,9 +124,6 @@ static inline void kasan_unpoison_memory(const void *address, size_t size) {}
 
 static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
 
-static inline void kasan_enable_current(void) {}
-static inline void kasan_disable_current(void) {}
-
 static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
 static inline void kasan_free_pages(struct page *page, unsigned int order) {}
 
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 063cd120b459..81b09bd31186 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1197,7 +1197,7 @@ struct task_struct {
 	u64				timer_slack_ns;
 	u64				default_timer_slack_ns;
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	unsigned int			kasan_depth;
 #endif
 
diff --git a/init/init_task.c b/init/init_task.c
index a56f0abb63e9..39703b4ef1f1 100644
--- a/init/init_task.c
+++ b/init/init_task.c
@@ -176,7 +176,7 @@ struct task_struct init_task
 	.numa_group	= NULL,
 	.numa_faults	= NULL,
 #endif
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	.kasan_depth	= 1,
 #endif
 #ifdef CONFIG_KCSAN
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 543e6bf2168f..d0b3ff410b0c 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -46,6 +46,7 @@ void kasan_set_track(struct kasan_track *track, gfp_t flags)
 	track->stack = kasan_save_stack(flags);
 }
 
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 void kasan_enable_current(void)
 {
 	current->kasan_depth++;
@@ -55,6 +56,7 @@ void kasan_disable_current(void)
 {
 	current->kasan_depth--;
 }
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
 
 static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
 {
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index b18d193f7f58..af9138ea54ad 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -292,8 +292,10 @@ static void print_shadow_for_address(const void *addr)
 
 static bool report_enabled(void)
 {
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	if (current->kasan_depth)
 		return false;
+#endif
 	if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags))
 		return true;
 	return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags);
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 17/44] kasan, arm64: move initialization message
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (15 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 16/44] kasan, arm64: only use kasan_depth " Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:03   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init Andrey Konovalov
                   ` (27 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Software tag-based KASAN mode is fully initialized with kasan_init_tags(),
while the generic mode only requires kasan_init(). Move the
initialization message for tag-based mode into kasan_init_tags().

Also fix pr_fmt() usage for KASAN code: generic.c doesn't need it as it
doesn't use any printing functions; tag-based mode should use "kasan:"
instead of KBUILD_MODNAME (which stands for file name).

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: Iddca9764b30ff0fab1922f26ca9d4f39b6f22673
---
 arch/arm64/include/asm/kasan.h |  9 +++------
 arch/arm64/mm/kasan_init.c     | 13 +++++--------
 mm/kasan/generic.c             |  2 --
 mm/kasan/sw_tags.c             |  4 +++-
 4 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h
index f7ea70d02cab..0aaf9044cd6a 100644
--- a/arch/arm64/include/asm/kasan.h
+++ b/arch/arm64/include/asm/kasan.h
@@ -12,14 +12,10 @@
 #define arch_kasan_reset_tag(addr)	__tag_reset(addr)
 #define arch_kasan_get_tag(addr)	__tag_get(addr)
 
-#ifdef CONFIG_KASAN
-void kasan_init(void);
-#else
-static inline void kasan_init(void) { }
-#endif
-
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 
+void kasan_init(void);
+
 /*
  * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
  * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses,
@@ -43,6 +39,7 @@ void kasan_copy_shadow(pgd_t *pgdir);
 asmlinkage void kasan_early_init(void);
 
 #else
+static inline void kasan_init(void) { }
 static inline void kasan_copy_shadow(pgd_t *pgdir) { }
 #endif
 
diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
index 5172799f831f..e35ce04beed1 100644
--- a/arch/arm64/mm/kasan_init.c
+++ b/arch/arm64/mm/kasan_init.c
@@ -278,17 +278,14 @@ static void __init kasan_init_depth(void)
 	init_task.kasan_depth = 0;
 }
 
-#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
-
-static inline void __init kasan_init_shadow(void) { }
-
-static inline void __init kasan_init_depth(void) { }
-
-#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
-
 void __init kasan_init(void)
 {
 	kasan_init_shadow();
 	kasan_init_depth();
+#if defined(CONFIG_KASAN_GENERIC)
+	/* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
 	pr_info("KernelAddressSanitizer initialized\n");
+#endif
 }
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
index e1af3b6c53b8..adb254df1b1d 100644
--- a/mm/kasan/generic.c
+++ b/mm/kasan/generic.c
@@ -9,8 +9,6 @@
  *        Andrey Konovalov <andreyknvl@gmail.com>
  */
 
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/init.h>
diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
index b2638c2cd58a..d25f8641b7cd 100644
--- a/mm/kasan/sw_tags.c
+++ b/mm/kasan/sw_tags.c
@@ -6,7 +6,7 @@
  * Author: Andrey Konovalov <andreyknvl@google.com>
  */
 
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#define pr_fmt(fmt) "kasan: " fmt
 
 #include <linux/export.h>
 #include <linux/interrupt.h>
@@ -41,6 +41,8 @@ void kasan_init_tags(void)
 
 	for_each_possible_cpu(cpu)
 		per_cpu(prng_state, cpu) = (u32)get_cycles();
+
+	pr_info("KernelAddressSanitizer initialized\n");
 }
 
 /*
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (16 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 17/44] kasan, arm64: move initialization message Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:05   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata Andrey Konovalov
                   ` (26 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Rename kasan_init_tags() to kasan_init_sw_tags() as the upcoming hardware
tag-based KASAN mode will have its own initialization routine.
Also similarly to kasan_init() mark kasan_init_tags() as __init.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I99aa2f7115d38a34ed85b329dadab6c7d6952416
---
 arch/arm64/kernel/setup.c  | 2 +-
 arch/arm64/mm/kasan_init.c | 2 +-
 include/linux/kasan.h      | 4 ++--
 mm/kasan/sw_tags.c         | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 133257ffd859..bb79b09f73c8 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -358,7 +358,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
 	smp_build_mpidr_hash();
 
 	/* Init percpu seeds for random tags after cpus are set up. */
-	kasan_init_tags();
+	kasan_init_sw_tags();
 
 #ifdef CONFIG_ARM64_SW_TTBR0_PAN
 	/*
diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
index e35ce04beed1..d8e66c78440e 100644
--- a/arch/arm64/mm/kasan_init.c
+++ b/arch/arm64/mm/kasan_init.c
@@ -283,7 +283,7 @@ void __init kasan_init(void)
 	kasan_init_shadow();
 	kasan_init_depth();
 #if defined(CONFIG_KASAN_GENERIC)
-	/* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
+	/* CONFIG_KASAN_SW_TAGS also requires kasan_init_sw_tags(). */
 	pr_info("KernelAddressSanitizer initialized\n");
 #endif
 }
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 979d598e1c30..1d6ec3325163 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -191,7 +191,7 @@ static inline void kasan_record_aux_stack(void *ptr) {}
 
 #ifdef CONFIG_KASAN_SW_TAGS
 
-void kasan_init_tags(void);
+void __init kasan_init_sw_tags(void);
 
 void *kasan_reset_tag(const void *addr);
 
@@ -200,7 +200,7 @@ bool kasan_report(unsigned long addr, size_t size,
 
 #else /* CONFIG_KASAN_SW_TAGS */
 
-static inline void kasan_init_tags(void) { }
+static inline void kasan_init_sw_tags(void) { }
 
 static inline void *kasan_reset_tag(const void *addr)
 {
diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
index d25f8641b7cd..b09a2c06abad 100644
--- a/mm/kasan/sw_tags.c
+++ b/mm/kasan/sw_tags.c
@@ -35,7 +35,7 @@
 
 static DEFINE_PER_CPU(u32, prng_state);
 
-void kasan_init_tags(void)
+void __init kasan_init_sw_tags(void)
 {
 	int cpu;
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (17 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:06   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata Andrey Konovalov
                   ` (25 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory, but will reuse
this function. Rename "shadow" to implementation-neutral "metadata".

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I03706fe34b38da7860c39aa0968e00001a7d1873
---
 mm/kasan/kasan.h          | 2 +-
 mm/kasan/report.c         | 6 +++---
 mm/kasan/report_generic.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index d0cf61d4d70d..f9366dfd94c9 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -146,7 +146,7 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
 		<< KASAN_SHADOW_SCALE_SHIFT);
 }
 
-static inline bool addr_has_shadow(const void *addr)
+static inline bool addr_has_metadata(const void *addr)
 {
 	return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
 }
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index af9138ea54ad..2990ca34abaf 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -361,7 +361,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write,
 	untagged_addr = reset_tag(tagged_addr);
 
 	info.access_addr = tagged_addr;
-	if (addr_has_shadow(untagged_addr))
+	if (addr_has_metadata(untagged_addr))
 		info.first_bad_addr = find_first_bad_addr(tagged_addr, size);
 	else
 		info.first_bad_addr = untagged_addr;
@@ -372,11 +372,11 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write,
 	start_report(&flags);
 
 	print_error_description(&info);
-	if (addr_has_shadow(untagged_addr))
+	if (addr_has_metadata(untagged_addr))
 		print_tags(get_tag(tagged_addr), info.first_bad_addr);
 	pr_err("\n");
 
-	if (addr_has_shadow(untagged_addr)) {
+	if (addr_has_metadata(untagged_addr)) {
 		print_address_description(untagged_addr, get_tag(tagged_addr));
 		pr_err("\n");
 		print_shadow_for_address(info.first_bad_addr);
diff --git a/mm/kasan/report_generic.c b/mm/kasan/report_generic.c
index b543a1ed6078..16ed550850e9 100644
--- a/mm/kasan/report_generic.c
+++ b/mm/kasan/report_generic.c
@@ -118,7 +118,7 @@ const char *get_bug_type(struct kasan_access_info *info)
 	if (info->access_addr + info->access_size < info->access_addr)
 		return "out-of-bounds";
 
-	if (addr_has_shadow(info->access_addr))
+	if (addr_has_metadata(info->access_addr))
 		return get_shadow_bug_type(info);
 	return get_wild_bug_type(info);
 }
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (18 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:07   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes Andrey Konovalov
                   ` (24 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory, but will reuse
this function. Rename "shadow" to implementation-neutral "metadata".

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I18397dddbed6bc6d365ddcaf063a83948e1150a5
---
 mm/kasan/report.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 2990ca34abaf..5d5733831ad7 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -252,7 +252,7 @@ static int shadow_pointer_offset(const void *row, const void *shadow)
 		(shadow - row) / SHADOW_BYTES_PER_BLOCK + 1;
 }
 
-static void print_shadow_for_address(const void *addr)
+static void print_memory_metadata(const void *addr)
 {
 	int i;
 	const void *shadow = kasan_mem_to_shadow(addr);
@@ -338,7 +338,7 @@ void kasan_report_invalid_free(void *object, unsigned long ip)
 	pr_err("\n");
 	print_address_description(object, tag);
 	pr_err("\n");
-	print_shadow_for_address(object);
+	print_memory_metadata(object);
 	end_report(&flags);
 }
 
@@ -379,7 +379,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write,
 	if (addr_has_metadata(untagged_addr)) {
 		print_address_description(untagged_addr, get_tag(tagged_addr));
 		pr_err("\n");
-		print_shadow_for_address(info.first_bad_addr);
+		print_memory_metadata(info.first_bad_addr);
 	} else {
 		dump_stack();
 	}
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (19 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:08   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 22/44] kasan: rename SHADOW layout macros to META Andrey Konovalov
                   ` (23 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

kasan_non_canonical_hook() is only applicable to KASAN modes that use
shadow memory, and won't be needed for hardware tag-based KASAN.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Icc9f5ef100a2e86f3a4214a0c3131a68266181b2
---
 mm/kasan/report.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 5d5733831ad7..594bad2a3a5e 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -403,7 +403,8 @@ bool kasan_report(unsigned long addr, size_t size, bool is_write,
 	return ret;
 }
 
-#ifdef CONFIG_KASAN_INLINE
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+	defined(CONFIG_KASAN_INLINE)
 /*
  * With CONFIG_KASAN_INLINE, accesses to bogus pointers (outside the high
  * canonical half of the address space) cause out-of-bounds shadow memory reads
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 22/44] kasan: rename SHADOW layout macros to META
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (20 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:18   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode Andrey Konovalov
                   ` (22 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Hardware tag-based KASAN won't be using shadow memory, but will reuse
these macros. Rename "SHADOW" to implementation-neutral "META".

No functional changes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Id2d836bf43b401bce1221cc06e745185f17b1cc
---
 mm/kasan/report.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 594bad2a3a5e..8c588588c88f 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -33,11 +33,11 @@
 #include "kasan.h"
 #include "../slab.h"
 
-/* Shadow layout customization. */
-#define SHADOW_BYTES_PER_BLOCK 1
-#define SHADOW_BLOCKS_PER_ROW 16
-#define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK)
-#define SHADOW_ROWS_AROUND_ADDR 2
+/* Metadata layout customization. */
+#define META_BYTES_PER_BLOCK 1
+#define META_BLOCKS_PER_ROW 16
+#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
+#define META_ROWS_AROUND_ADDR 2
 
 static unsigned long kasan_flags;
 
@@ -240,7 +240,7 @@ static void print_address_description(void *addr, u8 tag)
 
 static bool row_is_guilty(const void *row, const void *guilty)
 {
-	return (row <= guilty) && (guilty < row + SHADOW_BYTES_PER_ROW);
+	return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW);
 }
 
 static int shadow_pointer_offset(const void *row, const void *shadow)
@@ -249,7 +249,7 @@ static int shadow_pointer_offset(const void *row, const void *shadow)
 	 *    3 + (BITS_PER_LONG/8)*2 chars.
 	 */
 	return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 +
-		(shadow - row) / SHADOW_BYTES_PER_BLOCK + 1;
+		(shadow - row) / META_BYTES_PER_BLOCK + 1;
 }
 
 static void print_memory_metadata(const void *addr)
@@ -259,15 +259,15 @@ static void print_memory_metadata(const void *addr)
 	const void *shadow_row;
 
 	shadow_row = (void *)round_down((unsigned long)shadow,
-					SHADOW_BYTES_PER_ROW)
-		- SHADOW_ROWS_AROUND_ADDR * SHADOW_BYTES_PER_ROW;
+					META_BYTES_PER_ROW)
+		- META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW;
 
 	pr_err("Memory state around the buggy address:\n");
 
-	for (i = -SHADOW_ROWS_AROUND_ADDR; i <= SHADOW_ROWS_AROUND_ADDR; i++) {
+	for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) {
 		const void *kaddr = kasan_shadow_to_mem(shadow_row);
 		char buffer[4 + (BITS_PER_LONG/8)*2];
-		char shadow_buf[SHADOW_BYTES_PER_ROW];
+		char shadow_buf[META_BYTES_PER_ROW];
 
 		snprintf(buffer, sizeof(buffer),
 			(i == 0) ? ">%px: " : " %px: ", kaddr);
@@ -276,17 +276,17 @@ static void print_memory_metadata(const void *addr)
 		 * function, because generic functions may try to
 		 * access kasan mapping for the passed address.
 		 */
-		memcpy(shadow_buf, shadow_row, SHADOW_BYTES_PER_ROW);
+		memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW);
 		print_hex_dump(KERN_ERR, buffer,
-			DUMP_PREFIX_NONE, SHADOW_BYTES_PER_ROW, 1,
-			shadow_buf, SHADOW_BYTES_PER_ROW, 0);
+			DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1,
+			shadow_buf, META_BYTES_PER_ROW, 0);
 
 		if (row_is_guilty(shadow_row, shadow))
 			pr_err("%*c\n",
 				shadow_pointer_offset(shadow_row, shadow),
 				'^');
 
-		shadow_row += SHADOW_BYTES_PER_ROW;
+		shadow_row += META_BYTES_PER_ROW;
 	}
 }
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (21 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 22/44] kasan: rename SHADOW layout macros to META Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:22   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE Andrey Konovalov
                   ` (21 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This is a preparatory commit for the upcoming addition of a new hardware
tag-based (MTE-based) KASAN mode.

Rework print_memory_metadata() to make it agnostic with regard to the
way metadata is stored. Allow providing a separate metadata_fetch_row()
implementation for each KASAN mode. Hardware tag-based KASAN will provide
its own implementation that doesn't use shadow memory.

No functional changes for software modes.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I5b0ed1d079ea776e620beca6a529a861e7dced95
---
 mm/kasan/kasan.h          |  8 ++++++
 mm/kasan/report.c         | 56 +++++++++++++++++++--------------------
 mm/kasan/report_generic.c |  5 ++++
 mm/kasan/report_sw_tags.c |  5 ++++
 4 files changed, 45 insertions(+), 29 deletions(-)

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index f9366dfd94c9..b5b00bff358f 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -57,6 +57,13 @@
 #define KASAN_ABI_VERSION 1
 #endif
 
+/* Metadata layout customization. */
+#define META_BYTES_PER_BLOCK 1
+#define META_BLOCKS_PER_ROW 16
+#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
+#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_GRANULE_SIZE)
+#define META_ROWS_AROUND_ADDR 2
+
 struct kasan_access_info {
 	const void *access_addr;
 	const void *first_bad_addr;
@@ -168,6 +175,7 @@ bool check_invalid_free(void *addr);
 
 void *find_first_bad_addr(void *addr, size_t size);
 const char *get_bug_type(struct kasan_access_info *info);
+void metadata_fetch_row(char *buffer, void *row);
 
 #if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK
 void print_address_stack_frame(const void *addr);
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 8c588588c88f..8afc1a6ab202 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -33,12 +33,6 @@
 #include "kasan.h"
 #include "../slab.h"
 
-/* Metadata layout customization. */
-#define META_BYTES_PER_BLOCK 1
-#define META_BLOCKS_PER_ROW 16
-#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
-#define META_ROWS_AROUND_ADDR 2
-
 static unsigned long kasan_flags;
 
 #define KASAN_BIT_REPORTED	0
@@ -238,55 +232,59 @@ static void print_address_description(void *addr, u8 tag)
 	print_address_stack_frame(addr);
 }
 
-static bool row_is_guilty(const void *row, const void *guilty)
+static bool meta_row_is_guilty(const void *row, const void *addr)
 {
-	return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW);
+	return (row <= addr) && (addr < row + META_MEM_BYTES_PER_ROW);
 }
 
-static int shadow_pointer_offset(const void *row, const void *shadow)
+static int meta_pointer_offset(const void *row, const void *addr)
 {
-	/* The length of ">ff00ff00ff00ff00: " is
-	 *    3 + (BITS_PER_LONG/8)*2 chars.
+	/*
+	 * Memory state around the buggy address:
+	 *  ff00ff00ff00ff00: 00 00 00 05 fe fe fe fe fe fe fe fe fe fe fe fe
+	 *  ...
+	 *
+	 * The length of ">ff00ff00ff00ff00: " is
+	 *    3 + (BITS_PER_LONG / 8) * 2 chars.
+	 * The length of each granule metadata is 2 bytes
+	 *    plus 1 byte for space.
 	 */
-	return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 +
-		(shadow - row) / META_BYTES_PER_BLOCK + 1;
+	return 3 + (BITS_PER_LONG / 8) * 2 +
+		(addr - row) / KASAN_GRANULE_SIZE * 3 + 1;
 }
 
 static void print_memory_metadata(const void *addr)
 {
 	int i;
-	const void *shadow = kasan_mem_to_shadow(addr);
-	const void *shadow_row;
+	void *row;
 
-	shadow_row = (void *)round_down((unsigned long)shadow,
-					META_BYTES_PER_ROW)
-		- META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW;
+	row = (void *)round_down((unsigned long)addr, META_MEM_BYTES_PER_ROW)
+			- META_ROWS_AROUND_ADDR * META_MEM_BYTES_PER_ROW;
 
 	pr_err("Memory state around the buggy address:\n");
 
 	for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) {
-		const void *kaddr = kasan_shadow_to_mem(shadow_row);
-		char buffer[4 + (BITS_PER_LONG/8)*2];
-		char shadow_buf[META_BYTES_PER_ROW];
+		char buffer[4 + (BITS_PER_LONG / 8) * 2];
+		char metadata[META_BYTES_PER_ROW];
 
 		snprintf(buffer, sizeof(buffer),
-			(i == 0) ? ">%px: " : " %px: ", kaddr);
+				(i == 0) ? ">%px: " : " %px: ", row);
+
 		/*
 		 * We should not pass a shadow pointer to generic
 		 * function, because generic functions may try to
 		 * access kasan mapping for the passed address.
 		 */
-		memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW);
+		metadata_fetch_row(&metadata[0], row);
+
 		print_hex_dump(KERN_ERR, buffer,
 			DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1,
-			shadow_buf, META_BYTES_PER_ROW, 0);
+			metadata, META_BYTES_PER_ROW, 0);
 
-		if (row_is_guilty(shadow_row, shadow))
-			pr_err("%*c\n",
-				shadow_pointer_offset(shadow_row, shadow),
-				'^');
+		if (meta_row_is_guilty(row, addr))
+			pr_err("%*c\n", meta_pointer_offset(row, addr), '^');
 
-		shadow_row += META_BYTES_PER_ROW;
+		row += META_MEM_BYTES_PER_ROW;
 	}
 }
 
diff --git a/mm/kasan/report_generic.c b/mm/kasan/report_generic.c
index 16ed550850e9..8a9c889872da 100644
--- a/mm/kasan/report_generic.c
+++ b/mm/kasan/report_generic.c
@@ -123,6 +123,11 @@ const char *get_bug_type(struct kasan_access_info *info)
 	return get_wild_bug_type(info);
 }
 
+void metadata_fetch_row(char *buffer, void *row)
+{
+	memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);
+}
+
 #if CONFIG_KASAN_STACK
 static bool __must_check tokenize_frame_descr(const char **frame_descr,
 					      char *token, size_t max_tok_len,
diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c
index c87d5a343b4e..add2dfe6169c 100644
--- a/mm/kasan/report_sw_tags.c
+++ b/mm/kasan/report_sw_tags.c
@@ -80,6 +80,11 @@ void *find_first_bad_addr(void *addr, size_t size)
 	return p;
 }
 
+void metadata_fetch_row(char *buffer, void *row)
+{
+	memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);
+}
+
 void print_tags(u8 addr_tag, const void *addr)
 {
 	u8 *shadow = (u8 *)kasan_mem_to_shadow(addr);
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (22 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:52   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS Andrey Konovalov
                   ` (20 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Software tag-based KASAN provides its own tag checking machinery that
can conflict with MTE. Don't allow enabling software tag-based KASAN
when MTE is enabled.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: Icd29bd0c6b1d3d7a0ee3d50c20490f404d34fc97
---
 arch/arm64/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1515f6f153a0..25ead11074bf 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -134,7 +134,7 @@ config ARM64
 	select HAVE_ARCH_JUMP_LABEL
 	select HAVE_ARCH_JUMP_LABEL_RELATIVE
 	select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
-	select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
+	select HAVE_ARCH_KASAN_SW_TAGS if (HAVE_ARCH_KASAN && !ARM64_MTE)
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_MMAP_RND_BITS
 	select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (23 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 15:58   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 26/44] arm64: Enable armv8.5-a asm-arch option Andrey Konovalov
                   ` (19 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This patch adds a configuration option for a new KASAN mode called
hardware tag-based KASAN. This mode uses the memory tagging approach
like the software tag-based mode, but relies on arm64 Memory Tagging
Extension feature for tag management and access checking.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I246c2def9fffa6563278db1bddfbe742ca7bdefe
---
 lib/Kconfig.kasan | 58 +++++++++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 17 deletions(-)

diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
index ec59a0e26d09..e5f27ec8b254 100644
--- a/lib/Kconfig.kasan
+++ b/lib/Kconfig.kasan
@@ -6,7 +6,10 @@ config HAVE_ARCH_KASAN
 config HAVE_ARCH_KASAN_SW_TAGS
 	bool
 
-config	HAVE_ARCH_KASAN_VMALLOC
+config HAVE_ARCH_KASAN_HW_TAGS
+	bool
+
+config HAVE_ARCH_KASAN_VMALLOC
 	bool
 
 config CC_HAS_KASAN_GENERIC
@@ -20,11 +23,11 @@ config CC_HAS_WORKING_NOSANITIZE_ADDRESS
 
 menuconfig KASAN
 	bool "KASAN: runtime memory debugger"
-	depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \
-		   (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)
+	depends on (((HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \
+		     (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)) && \
+		    CC_HAS_WORKING_NOSANITIZE_ADDRESS) || \
+		   HAVE_ARCH_KASAN_HW_TAGS
 	depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
-	depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS
-	select CONSTRUCTORS
 	select STACKDEPOT
 	help
 	  Enables KASAN (KernelAddressSANitizer) - runtime memory debugger,
@@ -37,18 +40,24 @@ choice
 	prompt "KASAN mode"
 	default KASAN_GENERIC
 	help
-	  KASAN has two modes: generic KASAN (similar to userspace ASan,
-	  x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and
-	  software tag-based KASAN (a version based on software memory
-	  tagging, arm64 only, similar to userspace HWASan, enabled with
-	  CONFIG_KASAN_SW_TAGS).
+	  KASAN has three modes:
+	  1. generic KASAN (similar to userspace ASan,
+	     x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC),
+	  2. software tag-based KASAN (arm64 only, based on software
+	     memory tagging (similar to userspace HWASan), enabled with
+	     CONFIG_KASAN_SW_TAGS), and
+	  3. hardware tag-based KASAN (arm64 only, based on hardware
+	     memory tagging, enabled with CONFIG_KASAN_HW_TAGS).
+
+	  All KASAN modes are strictly debugging features.
 
-	  Both generic and tag-based KASAN are strictly debugging features.
+	  For better error reports enable CONFIG_STACKTRACE.
 
 config KASAN_GENERIC
 	bool "Generic mode"
 	depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC
 	select SLUB_DEBUG if SLUB
+	select CONSTRUCTORS
 	help
 	  Enables generic KASAN mode.
 
@@ -61,8 +70,6 @@ config KASAN_GENERIC
 	  and introduces an overhead of ~x1.5 for the rest of the allocations.
 	  The performance slowdown is ~x3.
 
-	  For better error detection enable CONFIG_STACKTRACE.
-
 	  Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB
 	  (the resulting kernel does not boot).
 
@@ -70,11 +77,15 @@ config KASAN_SW_TAGS
 	bool "Software tag-based mode"
 	depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS
 	select SLUB_DEBUG if SLUB
+	select CONSTRUCTORS
 	help
 	  Enables software tag-based KASAN mode.
 
-	  This mode requires Top Byte Ignore support by the CPU and therefore
-	  is only supported for arm64. This mode requires Clang.
+	  This mode require software memory tagging support in the form of
+	  HWASan-like compiler instrumentation.
+
+	  Currently this mode is only implemented for arm64 CPUs and relies on
+	  Top Byte Ignore. This mode requires Clang.
 
 	  This mode consumes about 1/16th of available memory at kernel start
 	  and introduces an overhead of ~20% for the rest of the allocations.
@@ -82,15 +93,27 @@ config KASAN_SW_TAGS
 	  casting and comparison, as it embeds tags into the top byte of each
 	  pointer.
 
-	  For better error detection enable CONFIG_STACKTRACE.
-
 	  Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB
 	  (the resulting kernel does not boot).
 
+config KASAN_HW_TAGS
+	bool "Hardware tag-based mode"
+	depends on HAVE_ARCH_KASAN_HW_TAGS
+	depends on SLUB
+	help
+	  Enables hardware tag-based KASAN mode.
+
+	  This mode requires hardware memory tagging support, and can be used
+	  by any architecture that provides it.
+
+	  Currently this mode is only implemented for arm64 CPUs starting from
+	  ARMv8.5 and relies on Memory Tagging Extension and Top Byte Ignore.
+
 endchoice
 
 choice
 	prompt "Instrumentation type"
+	depends on KASAN_GENERIC || KASAN_SW_TAGS
 	default KASAN_OUTLINE
 
 config KASAN_OUTLINE
@@ -114,6 +137,7 @@ endchoice
 
 config KASAN_STACK_ENABLE
 	bool "Enable stack instrumentation (unsafe)" if CC_IS_CLANG && !COMPILE_TEST
+	depends on KASAN_GENERIC || KASAN_SW_TAGS
 	help
 	  The LLVM stack address sanitizer has a know problem that
 	  causes excessive stack usage in a lot of functions, see
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 26/44] arm64: Enable armv8.5-a asm-arch option
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (24 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 27/44] arm64: mte: Add in-kernel MTE helpers Andrey Konovalov
                   ` (18 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) which
is an armv8.5-a architecture extension.

Enable the correct asm option when the compiler supports it in order to
allow the usage of ALTERNATIVE()s with MTE instructions.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I172e15e4c189f073e4c14a10276b276092e76536
---
 arch/arm64/Kconfig  | 4 ++++
 arch/arm64/Makefile | 5 +++++
 2 files changed, 9 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 25ead11074bf..c84a0e6b4650 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1591,6 +1591,9 @@ endmenu
 
 menu "ARMv8.5 architectural features"
 
+config AS_HAS_ARMV8_5
+	def_bool $(cc-option,-Wa$(comma)-march=armv8.5-a)
+
 config ARM64_BTI
 	bool "Branch Target Identification support"
 	default y
@@ -1665,6 +1668,7 @@ config ARM64_MTE
 	bool "Memory Tagging Extension support"
 	default y
 	depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI
+	depends on AS_HAS_ARMV8_5
 	select ARCH_USES_HIGH_VMA_FLAGS
 	help
 	  Memory Tagging (part of the ARMv8.5 Extensions) provides
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 5789c2d18d43..50ad9cbccb51 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -100,6 +100,11 @@ ifeq ($(CONFIG_AS_HAS_ARMV8_4), y)
 asm-arch := armv8.4-a
 endif
 
+ifeq ($(CONFIG_AS_HAS_ARMV8_5), y)
+# make sure to pass the newest target architecture to -march.
+asm-arch := armv8.5-a
+endif
+
 ifdef asm-arch
 KBUILD_CFLAGS	+= -Wa,-march=$(asm-arch) \
 		   -DARM64_ASM_ARCH='"$(asm-arch)"'
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 27/44] arm64: mte: Add in-kernel MTE helpers
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (25 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 26/44] arm64: Enable armv8.5-a asm-arch option Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags Andrey Konovalov
                   ` (17 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

Provide helper functions to manipulate allocation and pointer tags for
kernel addresses.

Low-level helper functions (mte_assign_*, written in assembly) operate
tag values from the [0x0, 0xF] range. High-level helper functions
(mte_get/set_*) use the [0xF0, 0xFF] range to preserve compatibility
with normal kernel pointers that have 0xFF in their top byte.

MTE_GRANULE_SIZE and related definitions are moved to mte-def.h header
that doesn't have any dependencies and is safe to include into any
low-level header.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I1b5230254f90dc21a913447cb17f07fea7944ece
---
 arch/arm64/include/asm/esr.h       |  1 +
 arch/arm64/include/asm/mte-def.h   | 15 ++++++++
 arch/arm64/include/asm/mte-kasan.h | 56 ++++++++++++++++++++++++++++++
 arch/arm64/include/asm/mte.h       | 20 +++++++----
 arch/arm64/kernel/mte.c            | 48 +++++++++++++++++++++++++
 arch/arm64/lib/mte.S               | 16 +++++++++
 6 files changed, 150 insertions(+), 6 deletions(-)
 create mode 100644 arch/arm64/include/asm/mte-def.h
 create mode 100644 arch/arm64/include/asm/mte-kasan.h

diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index 22c81f1edda2..971c20ddfed4 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -105,6 +105,7 @@
 #define ESR_ELx_FSC		(0x3F)
 #define ESR_ELx_FSC_TYPE	(0x3C)
 #define ESR_ELx_FSC_EXTABT	(0x10)
+#define ESR_ELx_FSC_MTE		(0x11)
 #define ESR_ELx_FSC_SERROR	(0x11)
 #define ESR_ELx_FSC_ACCESS	(0x08)
 #define ESR_ELx_FSC_FAULT	(0x04)
diff --git a/arch/arm64/include/asm/mte-def.h b/arch/arm64/include/asm/mte-def.h
new file mode 100644
index 000000000000..8401ac5840c7
--- /dev/null
+++ b/arch/arm64/include/asm/mte-def.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 ARM Ltd.
+ */
+#ifndef __ASM_MTE_DEF_H
+#define __ASM_MTE_DEF_H
+
+#define MTE_GRANULE_SIZE	UL(16)
+#define MTE_GRANULE_MASK	(~(MTE_GRANULE_SIZE - 1))
+#define MTE_TAG_SHIFT		56
+#define MTE_TAG_SIZE		4
+#define MTE_TAG_MASK		GENMASK((MTE_TAG_SHIFT + (MTE_TAG_SIZE - 1)), MTE_TAG_SHIFT)
+#define MTE_TAG_MAX		(MTE_TAG_MASK >> MTE_TAG_SHIFT)
+
+#endif /* __ASM_MTE_DEF_H  */
diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h
new file mode 100644
index 000000000000..3a70fb1807fd
--- /dev/null
+++ b/arch/arm64/include/asm/mte-kasan.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 ARM Ltd.
+ */
+#ifndef __ASM_MTE_KASAN_H
+#define __ASM_MTE_KASAN_H
+
+#include <asm/mte-def.h>
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+/*
+ * The functions below are meant to be used only for the
+ * KASAN_HW_TAGS interface defined in asm/memory.h.
+ */
+#ifdef CONFIG_ARM64_MTE
+
+static inline u8 mte_get_ptr_tag(void *ptr)
+{
+	/* Note: The format of KASAN tags is 0xF<x> */
+	u8 tag = 0xF0 | (u8)(((u64)(ptr)) >> MTE_TAG_SHIFT);
+
+	return tag;
+}
+
+u8 mte_get_mem_tag(void *addr);
+u8 mte_get_random_tag(void);
+void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag);
+
+#else /* CONFIG_ARM64_MTE */
+
+static inline u8 mte_get_ptr_tag(void *ptr)
+{
+	return 0xFF;
+}
+
+static inline u8 mte_get_mem_tag(void *addr)
+{
+	return 0xFF;
+}
+static inline u8 mte_get_random_tag(void)
+{
+	return 0xFF;
+}
+static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
+{
+	return addr;
+}
+
+#endif /* CONFIG_ARM64_MTE */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_MTE_KASAN_H  */
diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h
index 1c99fcadb58c..cf1cd181dcb2 100644
--- a/arch/arm64/include/asm/mte.h
+++ b/arch/arm64/include/asm/mte.h
@@ -5,14 +5,16 @@
 #ifndef __ASM_MTE_H
 #define __ASM_MTE_H
 
-#define MTE_GRANULE_SIZE	UL(16)
-#define MTE_GRANULE_MASK	(~(MTE_GRANULE_SIZE - 1))
-#define MTE_TAG_SHIFT		56
-#define MTE_TAG_SIZE		4
+#include <asm/compiler.h>
+#include <asm/mte-def.h>
+
+#define __MTE_PREAMBLE		ARM64_ASM_PREAMBLE ".arch_extension memtag\n"
 
 #ifndef __ASSEMBLY__
 
+#include <linux/bitfield.h>
 #include <linux/page-flags.h>
+#include <linux/types.h>
 
 #include <asm/pgtable-types.h>
 
@@ -45,7 +47,9 @@ long get_mte_ctrl(struct task_struct *task);
 int mte_ptrace_copy_tags(struct task_struct *child, long request,
 			 unsigned long addr, unsigned long data);
 
-#else
+void mte_assign_mem_tag_range(void *addr, size_t size);
+
+#else /* CONFIG_ARM64_MTE */
 
 /* unused if !CONFIG_ARM64_MTE, silence the compiler */
 #define PG_mte_tagged	0
@@ -80,7 +84,11 @@ static inline int mte_ptrace_copy_tags(struct task_struct *child,
 	return -EIO;
 }
 
-#endif
+static inline void mte_assign_mem_tag_range(void *addr, size_t size)
+{
+}
+
+#endif /* CONFIG_ARM64_MTE */
 
 #endif /* __ASSEMBLY__ */
 #endif /* __ASM_MTE_H  */
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index 52a0638ed967..8f99c65837fd 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -13,10 +13,13 @@
 #include <linux/swap.h>
 #include <linux/swapops.h>
 #include <linux/thread_info.h>
+#include <linux/types.h>
 #include <linux/uio.h>
 
+#include <asm/barrier.h>
 #include <asm/cpufeature.h>
 #include <asm/mte.h>
+#include <asm/mte-kasan.h>
 #include <asm/ptrace.h>
 #include <asm/sysreg.h>
 
@@ -72,6 +75,51 @@ int memcmp_pages(struct page *page1, struct page *page2)
 	return ret;
 }
 
+u8 mte_get_mem_tag(void *addr)
+{
+	if (!system_supports_mte())
+		return 0xFF;
+
+	asm(__MTE_PREAMBLE "ldg %0, [%0]"
+	    : "+r" (addr));
+
+	return mte_get_ptr_tag(addr);
+}
+
+u8 mte_get_random_tag(void)
+{
+	void *addr;
+
+	if (!system_supports_mte())
+		return 0xFF;
+
+	asm(__MTE_PREAMBLE "irg %0, %0"
+	    : "+r" (addr));
+
+	return mte_get_ptr_tag(addr);
+}
+
+void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
+{
+	void *ptr = addr;
+
+	if ((!system_supports_mte()) || (size == 0))
+		return addr;
+
+	/* Make sure that size is MTE granule aligned. */
+	WARN_ON(size & (MTE_GRANULE_SIZE - 1));
+
+	/* Make sure that the address is MTE granule aligned. */
+	WARN_ON((u64)addr & (MTE_GRANULE_SIZE - 1));
+
+	tag = 0xF0 | tag;
+	ptr = (void *)__tag_set(ptr, tag);
+
+	mte_assign_mem_tag_range(ptr, size);
+
+	return ptr;
+}
+
 static void update_sctlr_el1_tcf0(u64 tcf0)
 {
 	/* ISB required for the kernel uaccess routines */
diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S
index 03ca6d8b8670..ede1ea65428c 100644
--- a/arch/arm64/lib/mte.S
+++ b/arch/arm64/lib/mte.S
@@ -149,3 +149,19 @@ SYM_FUNC_START(mte_restore_page_tags)
 
 	ret
 SYM_FUNC_END(mte_restore_page_tags)
+
+/*
+ * Assign allocation tags for a region of memory based on the pointer tag
+ *   x0 - source pointer
+ *   x1 - size
+ *
+ * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and
+ * size must be non-zero and MTE_GRANULE_SIZE aligned.
+ */
+SYM_FUNC_START(mte_assign_mem_tag_range)
+1:	stg	x0, [x0]
+	add	x0, x0, #MTE_GRANULE_SIZE
+	subs	x1, x1, #MTE_GRANULE_SIZE
+	b.gt	1b
+	ret
+SYM_FUNC_END(mte_assign_mem_tag_range)
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (26 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 27/44] arm64: mte: Add in-kernel MTE helpers Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-12  9:31   ` Catalin Marinas
  2020-11-10 22:10 ` [PATCH v9 29/44] arm64: mte: Add in-kernel tag fault handler Andrey Konovalov
                   ` (16 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

The hardware tag-based KASAN for compatibility with the other modes
stores the tag associated to a page in page->flags.
Due to this the kernel faults on access when it allocates a page with an
initial tag and the user changes the tags.

Reset the tag associated by the kernel to a page in all the meaningful
places to prevent kernel faults on access.

Note: An alternative to this approach could be to modify page_to_virt().
This though could end up being racy, in fact if a CPU checks the
PG_mte_tagged bit and decides that the page is not tagged but another
CPU maps the same with PROT_MTE and becomes tagged the subsequent kernel
access would fail.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
Change-Id: I8451d438bb63364de2a3e68041e3a27866921d4e
---
 arch/arm64/kernel/hibernate.c | 5 +++++
 arch/arm64/kernel/mte.c       | 9 +++++++++
 arch/arm64/mm/copypage.c      | 1 +
 arch/arm64/mm/mteswap.c       | 9 +++++++++
 4 files changed, 24 insertions(+)

diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 42003774d261..9c9f47e9f7f4 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -371,6 +371,11 @@ static void swsusp_mte_restore_tags(void)
 		unsigned long pfn = xa_state.xa_index;
 		struct page *page = pfn_to_online_page(pfn);
 
+		/*
+		 * It is not required to invoke page_kasan_tag_reset(page)
+		 * at this point since the tags stored in page->flags are
+		 * already restored.
+		 */
 		mte_restore_page_tags(page_address(page), tags);
 
 		mte_free_tag_storage(tags);
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index 8f99c65837fd..600b26d65b41 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -34,6 +34,15 @@ static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_swap)
 			return;
 	}
 
+	page_kasan_tag_reset(page);
+	/*
+	 * We need smp_wmb() in between setting the flags and clearing the
+	 * tags because if another thread reads page->flags and builds a
+	 * tagged address out of it, there is an actual dependency to the
+	 * memory access, but on the current thread we do not guarantee that
+	 * the new new page->flags are visible before the tags were updated.
+	 */
+	smp_wmb();
 	mte_clear_page_tags(page_address(page));
 }
 
diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c
index 70a71f38b6a9..f0efa4847e2f 100644
--- a/arch/arm64/mm/copypage.c
+++ b/arch/arm64/mm/copypage.c
@@ -23,6 +23,7 @@ void copy_highpage(struct page *to, struct page *from)
 
 	if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) {
 		set_bit(PG_mte_tagged, &to->flags);
+		page_kasan_tag_reset(to);
 		mte_copy_page_tags(kto, kfrom);
 	}
 }
diff --git a/arch/arm64/mm/mteswap.c b/arch/arm64/mm/mteswap.c
index c52c1847079c..9cc59696489c 100644
--- a/arch/arm64/mm/mteswap.c
+++ b/arch/arm64/mm/mteswap.c
@@ -53,6 +53,15 @@ bool mte_restore_tags(swp_entry_t entry, struct page *page)
 	if (!tags)
 		return false;
 
+	page_kasan_tag_reset(page);
+	/*
+	 * We need smp_wmb() in between setting the flags and clearing the
+	 * tags because if another thread reads page->flags and builds a
+	 * tagged address out of it, there is an actual dependency to the
+	 * memory access, but on the current thread we do not guarantee that
+	 * the new new page->flags are visible before the tags were updated.
+	 */
+	smp_wmb();
 	mte_restore_page_tags(page_address(page), tags);
 
 	return true;
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 29/44] arm64: mte: Add in-kernel tag fault handler
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (27 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE Andrey Konovalov
                   ` (15 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

Add the implementation of the in-kernel fault handler.

When a tag fault happens on a kernel address:
* MTE is disabled on the current CPU,
* the execution continues.

When a tag fault happens on a user address:
* the kernel executes do_bad_area() and panics.

The tag fault handler for kernel addresses is currently empty and will be
filled in by a future commit.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I9b8aa79567f7c45f4d6a1290efcf34567e620717
---
 arch/arm64/include/asm/uaccess.h | 23 ++++++++++++++++
 arch/arm64/mm/fault.c            | 45 ++++++++++++++++++++++++++++++++
 2 files changed, 68 insertions(+)

diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index 991dd5f031e4..c7fff8daf2a7 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -200,13 +200,36 @@ do {									\
 				CONFIG_ARM64_PAN));			\
 } while (0)
 
+/*
+ * The Tag Check Flag (TCF) mode for MTE is per EL, hence TCF0
+ * affects EL0 and TCF affects EL1 irrespective of which TTBR is
+ * used.
+ * The kernel accesses TTBR0 usually with LDTR/STTR instructions
+ * when UAO is available, so these would act as EL0 accesses using
+ * TCF0.
+ * However futex.h code uses exclusives which would be executed as
+ * EL1, this can potentially cause a tag check fault even if the
+ * user disables TCF0.
+ *
+ * To address the problem we set the PSTATE.TCO bit in uaccess_enable()
+ * and reset it in uaccess_disable().
+ *
+ * The Tag check override (TCO) bit disables temporarily the tag checking
+ * preventing the issue.
+ */
 static inline void uaccess_disable(void)
 {
+	asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(0),
+				 ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
 	__uaccess_disable(ARM64_HAS_PAN);
 }
 
 static inline void uaccess_enable(void)
 {
+	asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(1),
+				 ARM64_MTE, CONFIG_KASAN_HW_TAGS));
+
 	__uaccess_enable(ARM64_HAS_PAN);
 }
 
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 1ee94002801f..fbceb14d93b1 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -33,6 +33,7 @@
 #include <asm/debug-monitors.h>
 #include <asm/esr.h>
 #include <asm/kprobes.h>
+#include <asm/mte.h>
 #include <asm/processor.h>
 #include <asm/sysreg.h>
 #include <asm/system_misc.h>
@@ -296,6 +297,44 @@ static void die_kernel_fault(const char *msg, unsigned long addr,
 	do_exit(SIGKILL);
 }
 
+static void report_tag_fault(unsigned long addr, unsigned int esr,
+			     struct pt_regs *regs)
+{
+}
+
+static void do_tag_recovery(unsigned long addr, unsigned int esr,
+			   struct pt_regs *regs)
+{
+	static bool reported = false;
+
+	if (!READ_ONCE(reported)) {
+		report_tag_fault(addr, esr, regs);
+		WRITE_ONCE(reported, true);
+	}
+
+	/*
+	 * Disable MTE Tag Checking on the local CPU for the current EL.
+	 * It will be done lazily on the other CPUs when they will hit a
+	 * tag fault.
+	 */
+	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_NONE);
+	isb();
+}
+
+static bool is_el1_mte_sync_tag_check_fault(unsigned int esr)
+{
+	unsigned int ec = ESR_ELx_EC(esr);
+	unsigned int fsc = esr & ESR_ELx_FSC;
+
+	if (ec != ESR_ELx_EC_DABT_CUR)
+		return false;
+
+	if (fsc == ESR_ELx_FSC_MTE)
+		return true;
+
+	return false;
+}
+
 static void __do_kernel_fault(unsigned long addr, unsigned int esr,
 			      struct pt_regs *regs)
 {
@@ -312,6 +351,12 @@ static void __do_kernel_fault(unsigned long addr, unsigned int esr,
 	    "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr))
 		return;
 
+	if (is_el1_mte_sync_tag_check_fault(esr)) {
+		do_tag_recovery(addr, esr, regs);
+
+		return;
+	}
+
 	if (is_el1_permission_fault(addr, esr, regs)) {
 		if (esr & ESR_ELx_WNR)
 			msg = "write to read-only memory";
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (28 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 29/44] arm64: mte: Add in-kernel tag fault handler Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-12  9:43   ` Catalin Marinas
  2020-11-10 22:10 ` [PATCH v9 31/44] arm64: mte: Convert gcr_user into an exclude mask Andrey Konovalov
                   ` (14 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

Hardware tag-based KASAN relies on Memory Tagging Extension (MTE)
feature and requires it to be enabled. MTE supports

This patch adds a new mte_init_tags() helper, that enables MTE in
Synchronous mode in EL1 and is intended to be called from KASAN runtime
during initialization.

The Tag Checking operation causes a synchronous data abort as
a consequence of a tag check fault when MTE is configured in
synchronous mode.

As part of this change enable match-all tag for EL1 to allow the
kernel to access user pages without faulting. This is required because
the kernel does not have knowledge of the tags set by the user in a
page.

Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a
similar way as TCF0 affects EL0.

MTE that is built on top of the Top Byte Ignore (TBI) feature hence we
enable it as part of this patch as well.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
Change-Id: I4d67497268bb7f0c2fc5dcacefa1e273df4af71d
---
 arch/arm64/include/asm/mte-kasan.h |  6 ++++++
 arch/arm64/kernel/mte.c            |  7 +++++++
 arch/arm64/mm/proc.S               | 23 ++++++++++++++++++++---
 3 files changed, 33 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h
index 3a70fb1807fd..aa3ea2e0b3a8 100644
--- a/arch/arm64/include/asm/mte-kasan.h
+++ b/arch/arm64/include/asm/mte-kasan.h
@@ -29,6 +29,8 @@ u8 mte_get_mem_tag(void *addr);
 u8 mte_get_random_tag(void);
 void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag);
 
+void mte_enable(void);
+
 #else /* CONFIG_ARM64_MTE */
 
 static inline u8 mte_get_ptr_tag(void *ptr)
@@ -49,6 +51,10 @@ static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
 	return addr;
 }
 
+static inline void mte_enable(void)
+{
+}
+
 #endif /* CONFIG_ARM64_MTE */
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index 600b26d65b41..7f477991a6cf 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -129,6 +129,13 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
 	return ptr;
 }
 
+void mte_enable(void)
+{
+	/* Enable MTE Sync Mode for EL1. */
+	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC);
+	isb();
+}
+
 static void update_sctlr_el1_tcf0(u64 tcf0)
 {
 	/* ISB required for the kernel uaccess routines */
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 23c326a06b2d..7c3304fb15d9 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -40,9 +40,15 @@
 #define TCR_CACHE_FLAGS	TCR_IRGN_WBWA | TCR_ORGN_WBWA
 
 #ifdef CONFIG_KASAN_SW_TAGS
-#define TCR_KASAN_FLAGS TCR_TBI1
+#define TCR_KASAN_SW_FLAGS TCR_TBI1
 #else
-#define TCR_KASAN_FLAGS 0
+#define TCR_KASAN_SW_FLAGS 0
+#endif
+
+#ifdef CONFIG_KASAN_HW_TAGS
+#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1
+#else
+#define TCR_KASAN_HW_FLAGS 0
 #endif
 
 /*
@@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup)
 	 */
 	mov_q	x5, MAIR_EL1_SET
 #ifdef CONFIG_ARM64_MTE
+	mte_tcr	.req	x20
+
+	mov	mte_tcr, #0
+
 	/*
 	 * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported
 	 * (ID_AA64PFR1_EL1[11:8] > 1).
@@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup)
 	/* clear any pending tag check faults in TFSR*_EL1 */
 	msr_s	SYS_TFSR_EL1, xzr
 	msr_s	SYS_TFSRE0_EL1, xzr
+
+	/* set the TCR_EL1 bits */
+	mov_q	mte_tcr, TCR_KASAN_HW_FLAGS
 1:
 #endif
 	msr	mair_el1, x5
@@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup)
 	 */
 	mov_q	x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \
 			TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \
-			TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS
+			TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS
+#ifdef CONFIG_ARM64_MTE
+	orr	x10, x10, mte_tcr
+	.unreq	mte_tcr
+#endif
 	tcr_clear_errata_bits x10, x9, x5
 
 #ifdef CONFIG_ARM64_VA_BITS_52
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 31/44] arm64: mte: Convert gcr_user into an exclude mask
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (29 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit Andrey Konovalov
                   ` (13 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

The gcr_user mask is a per thread mask that represents the tags that are
excluded from random generation when the Memory Tagging Extension is
present and an 'irg' instruction is invoked.

gcr_user affects the behavior on EL0 only.

Currently that mask is an include mask and it is controlled by the user
via prctl() while GCR_EL1 accepts an exclude mask.

Convert the include mask into an exclude one to make it easier the
register setting.

Note: This change will affect gcr_kernel (for EL1) introduced with a
future patch.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: Id15c0b47582fb51594bb26fb8353d78c7d0953c1
---
 arch/arm64/include/asm/processor.h |  2 +-
 arch/arm64/kernel/mte.c            | 29 +++++++++++++++--------------
 2 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index fce8cbecd6bc..e8cfc41a92d4 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -154,7 +154,7 @@ struct thread_struct {
 #endif
 #ifdef CONFIG_ARM64_MTE
 	u64			sctlr_tcf0;
-	u64			gcr_user_incl;
+	u64			gcr_user_excl;
 #endif
 };
 
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index 7f477991a6cf..664c968dc43c 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -156,23 +156,22 @@ static void set_sctlr_el1_tcf0(u64 tcf0)
 	preempt_enable();
 }
 
-static void update_gcr_el1_excl(u64 incl)
+static void update_gcr_el1_excl(u64 excl)
 {
-	u64 excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
 
 	/*
-	 * Note that 'incl' is an include mask (controlled by the user via
-	 * prctl()) while GCR_EL1 accepts an exclude mask.
+	 * Note that the mask controlled by the user via prctl() is an
+	 * include while GCR_EL1 accepts an exclude mask.
 	 * No need for ISB since this only affects EL0 currently, implicit
 	 * with ERET.
 	 */
 	sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl);
 }
 
-static void set_gcr_el1_excl(u64 incl)
+static void set_gcr_el1_excl(u64 excl)
 {
-	current->thread.gcr_user_incl = incl;
-	update_gcr_el1_excl(incl);
+	current->thread.gcr_user_excl = excl;
+	update_gcr_el1_excl(excl);
 }
 
 void flush_mte_state(void)
@@ -187,7 +186,7 @@ void flush_mte_state(void)
 	/* disable tag checking */
 	set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE);
 	/* reset tag generation mask */
-	set_gcr_el1_excl(0);
+	set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK);
 }
 
 void mte_thread_switch(struct task_struct *next)
@@ -198,7 +197,7 @@ void mte_thread_switch(struct task_struct *next)
 	/* avoid expensive SCTLR_EL1 accesses if no change */
 	if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0)
 		update_sctlr_el1_tcf0(next->thread.sctlr_tcf0);
-	update_gcr_el1_excl(next->thread.gcr_user_incl);
+	update_gcr_el1_excl(next->thread.gcr_user_excl);
 }
 
 void mte_suspend_exit(void)
@@ -206,13 +205,14 @@ void mte_suspend_exit(void)
 	if (!system_supports_mte())
 		return;
 
-	update_gcr_el1_excl(current->thread.gcr_user_incl);
+	update_gcr_el1_excl(current->thread.gcr_user_excl);
 }
 
 long set_mte_ctrl(struct task_struct *task, unsigned long arg)
 {
 	u64 tcf0;
-	u64 gcr_incl = (arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT;
+	u64 gcr_excl = ~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) &
+		       SYS_GCR_EL1_EXCL_MASK;
 
 	if (!system_supports_mte())
 		return 0;
@@ -233,10 +233,10 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg)
 
 	if (task != current) {
 		task->thread.sctlr_tcf0 = tcf0;
-		task->thread.gcr_user_incl = gcr_incl;
+		task->thread.gcr_user_excl = gcr_excl;
 	} else {
 		set_sctlr_el1_tcf0(tcf0);
-		set_gcr_el1_excl(gcr_incl);
+		set_gcr_el1_excl(gcr_excl);
 	}
 
 	return 0;
@@ -245,11 +245,12 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg)
 long get_mte_ctrl(struct task_struct *task)
 {
 	unsigned long ret;
+	u64 incl = ~task->thread.gcr_user_excl & SYS_GCR_EL1_EXCL_MASK;
 
 	if (!system_supports_mte())
 		return 0;
 
-	ret = task->thread.gcr_user_incl << PR_MTE_TAG_SHIFT;
+	ret = incl << PR_MTE_TAG_SHIFT;
 
 	switch (task->thread.sctlr_tcf0) {
 	case SCTLR_EL1_TCF0_NONE:
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (30 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 31/44] arm64: mte: Convert gcr_user into an exclude mask Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-12  9:39   ` Catalin Marinas
  2020-11-10 22:10 ` [PATCH v9 33/44] kasan, mm: untag page address in free_reserved_area Andrey Konovalov
                   ` (12 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

When MTE is present, the GCR_EL1 register contains the tags mask that
allows to exclude tags from the random generation via the IRG instruction.

With the introduction of the new Tag-Based KASAN API that provides a
mechanism to reserve tags for special reasons, the MTE implementation
has to make sure that the GCR_EL1 setting for the kernel does not affect
the userspace processes and viceversa.

Save and restore the kernel/user mask in GCR_EL1 in kernel entry and exit.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Co-developed-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I0081cba5ace27a9111bebb239075c9a466af4c84
---
 arch/arm64/include/asm/mte-def.h   |  1 -
 arch/arm64/include/asm/mte-kasan.h |  5 ++++
 arch/arm64/include/asm/mte.h       |  2 ++
 arch/arm64/kernel/asm-offsets.c    |  3 +++
 arch/arm64/kernel/entry.S          | 41 ++++++++++++++++++++++++++++++
 arch/arm64/kernel/mte.c            | 31 +++++++++++++++++++---
 6 files changed, 79 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/mte-def.h b/arch/arm64/include/asm/mte-def.h
index 8401ac5840c7..2d73a1612f09 100644
--- a/arch/arm64/include/asm/mte-def.h
+++ b/arch/arm64/include/asm/mte-def.h
@@ -10,6 +10,5 @@
 #define MTE_TAG_SHIFT		56
 #define MTE_TAG_SIZE		4
 #define MTE_TAG_MASK		GENMASK((MTE_TAG_SHIFT + (MTE_TAG_SIZE - 1)), MTE_TAG_SHIFT)
-#define MTE_TAG_MAX		(MTE_TAG_MASK >> MTE_TAG_SHIFT)
 
 #endif /* __ASM_MTE_DEF_H  */
diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h
index aa3ea2e0b3a8..7c282758d78b 100644
--- a/arch/arm64/include/asm/mte-kasan.h
+++ b/arch/arm64/include/asm/mte-kasan.h
@@ -30,6 +30,7 @@ u8 mte_get_random_tag(void);
 void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag);
 
 void mte_enable(void);
+void mte_init_tags(u64 max_tag);
 
 #else /* CONFIG_ARM64_MTE */
 
@@ -55,6 +56,10 @@ static inline void mte_enable(void)
 {
 }
 
+static inline void mte_init_tags(u64 max_tag)
+{
+}
+
 #endif /* CONFIG_ARM64_MTE */
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h
index cf1cd181dcb2..d02aff9f493d 100644
--- a/arch/arm64/include/asm/mte.h
+++ b/arch/arm64/include/asm/mte.h
@@ -18,6 +18,8 @@
 
 #include <asm/pgtable-types.h>
 
+extern u64 gcr_kernel_excl;
+
 void mte_clear_page_tags(void *addr);
 unsigned long mte_copy_tags_from_user(void *to, const void __user *from,
 				      unsigned long n);
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 7d32fc959b1a..dfe6ed8446ac 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -47,6 +47,9 @@ int main(void)
 #ifdef CONFIG_ARM64_PTR_AUTH
   DEFINE(THREAD_KEYS_USER,	offsetof(struct task_struct, thread.keys_user));
   DEFINE(THREAD_KEYS_KERNEL,	offsetof(struct task_struct, thread.keys_kernel));
+#endif
+#ifdef CONFIG_ARM64_MTE
+  DEFINE(THREAD_GCR_EL1_USER,	offsetof(struct task_struct, thread.gcr_user_excl));
 #endif
   BLANK();
   DEFINE(S_X0,			offsetof(struct pt_regs, regs[0]));
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index b295fb912b12..07646ef4f184 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -173,6 +173,43 @@ alternative_else_nop_endif
 #endif
 	.endm
 
+	.macro mte_set_gcr, tmp, tmp2
+#ifdef CONFIG_ARM64_MTE
+	/*
+	 * Calculate and set the exclude mask preserving
+	 * the RRND (bit[16]) setting.
+	 */
+	mrs_s	\tmp2, SYS_GCR_EL1
+	bfi	\tmp2, \tmp, #0, #16
+	msr_s	SYS_GCR_EL1, \tmp2
+	isb
+#endif
+	.endm
+
+	.macro mte_set_kernel_gcr, tmp, tmp2
+#ifdef CONFIG_KASAN_HW_TAGS
+alternative_if_not ARM64_MTE
+	b	1f
+alternative_else_nop_endif
+	ldr_l	\tmp, gcr_kernel_excl
+
+	mte_set_gcr \tmp, \tmp2
+1:
+#endif
+	.endm
+
+	.macro mte_set_user_gcr, tsk, tmp, tmp2
+#ifdef CONFIG_ARM64_MTE
+alternative_if_not ARM64_MTE
+	b	1f
+alternative_else_nop_endif
+	ldr	\tmp, [\tsk, #THREAD_GCR_EL1_USER]
+
+	mte_set_gcr \tmp, \tmp2
+1:
+#endif
+	.endm
+
 	.macro	kernel_entry, el, regsize = 64
 	.if	\regsize == 32
 	mov	w0, w0				// zero upper 32 bits of x0
@@ -212,6 +249,8 @@ alternative_else_nop_endif
 
 	ptrauth_keys_install_kernel tsk, x20, x22, x23
 
+	mte_set_kernel_gcr x22, x23
+
 	scs_load tsk, x20
 	.else
 	add	x21, sp, #S_FRAME_SIZE
@@ -330,6 +369,8 @@ alternative_else_nop_endif
 	/* No kernel C function calls after this as user keys are set. */
 	ptrauth_keys_install_user tsk, x0, x1, x2
 
+	mte_set_user_gcr tsk, x0, x1
+
 	apply_ssbd 0, x0, x1
 	.endif
 
diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
index 664c968dc43c..dbda6598c19d 100644
--- a/arch/arm64/kernel/mte.c
+++ b/arch/arm64/kernel/mte.c
@@ -23,6 +23,8 @@
 #include <asm/ptrace.h>
 #include <asm/sysreg.h>
 
+u64 gcr_kernel_excl __ro_after_init;
+
 static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_swap)
 {
 	pte_t old_pte = READ_ONCE(*ptep);
@@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
 	return ptr;
 }
 
+void mte_init_tags(u64 max_tag)
+{
+	static bool gcr_kernel_excl_initialized = false;
+
+	if (!gcr_kernel_excl_initialized) {
+		/*
+		 * The format of the tags in KASAN is 0xFF and in MTE is 0xF.
+		 * This conversion extracts an MTE tag from a KASAN tag.
+		 */
+		u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT,
+					     max_tag), 0);
+
+		gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
+		gcr_kernel_excl_initialized = true;
+	}
+
+	/* Enable the kernel exclude mask for random tags generation. */
+	write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1);
+}
+
 void mte_enable(void)
 {
 	/* Enable MTE Sync Mode for EL1. */
@@ -171,7 +193,11 @@ static void update_gcr_el1_excl(u64 excl)
 static void set_gcr_el1_excl(u64 excl)
 {
 	current->thread.gcr_user_excl = excl;
-	update_gcr_el1_excl(excl);
+
+	/*
+	 * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value
+	 * by mte_set_user_gcr() in kernel_exit,
+	 */
 }
 
 void flush_mte_state(void)
@@ -197,7 +223,6 @@ void mte_thread_switch(struct task_struct *next)
 	/* avoid expensive SCTLR_EL1 accesses if no change */
 	if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0)
 		update_sctlr_el1_tcf0(next->thread.sctlr_tcf0);
-	update_gcr_el1_excl(next->thread.gcr_user_excl);
 }
 
 void mte_suspend_exit(void)
@@ -205,7 +230,7 @@ void mte_suspend_exit(void)
 	if (!system_supports_mte())
 		return;
 
-	update_gcr_el1_excl(current->thread.gcr_user_excl);
+	update_gcr_el1_excl(gcr_kernel_excl);
 }
 
 long set_mte_ctrl(struct task_struct *task, unsigned long arg)
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 33/44] kasan, mm: untag page address in free_reserved_area
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (31 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS Andrey Konovalov
                   ` (11 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

free_reserved_area() memsets the pages belonging to a given memory area.
As that memory hasn't been allocated via page_alloc, the KASAN tags that
those pages have are 0x00. As the result the memset might result in a tag
mismatch.

Untag the address to avoid spurious faults.

Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
Change-Id: If12b4944383575b8bbd7d971decbd7f04be6748b
---
 mm/page_alloc.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 23f5066bd4a5..24b45261e2bd 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -7593,6 +7593,11 @@ unsigned long free_reserved_area(void *start, void *end, int poison, const char
 		 * alias for the memset().
 		 */
 		direct_map_addr = page_address(page);
+		/*
+		 * Perform a kasan-unchecked memset() since this memory
+		 * has not been initialized.
+		 */
+		direct_map_addr = kasan_reset_tag(direct_map_addr);
 		if ((unsigned int)poison <= 0xFF)
 			memset(direct_map_addr, poison, PAGE_SIZE);
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (32 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 33/44] kasan, mm: untag page address in free_reserved_area Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:10   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 35/44] arm64: kasan: Add arch layer for memory tagging helpers Andrey Konovalov
                   ` (10 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Hardware tag-based KASAN uses the memory tagging approach, which requires
all allocations to be aligned to the memory granule size. Align the
allocations to MTE_GRANULE_SIZE via ARCH_SLAB_MINALIGN when
CONFIG_KASAN_HW_TAGS is enabled.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I51ebd3f9645e6330e5a92973bf7c86b62d632c2b
---
 arch/arm64/include/asm/cache.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h
index 63d43b5f82f6..77cbbe3625f2 100644
--- a/arch/arm64/include/asm/cache.h
+++ b/arch/arm64/include/asm/cache.h
@@ -6,6 +6,7 @@
 #define __ASM_CACHE_H
 
 #include <asm/cputype.h>
+#include <asm/mte-kasan.h>
 
 #define CTR_L1IP_SHIFT		14
 #define CTR_L1IP_MASK		3
@@ -51,6 +52,8 @@
 
 #ifdef CONFIG_KASAN_SW_TAGS
 #define ARCH_SLAB_MINALIGN	(1ULL << KASAN_SHADOW_SCALE_SHIFT)
+#elif defined(CONFIG_KASAN_HW_TAGS)
+#define ARCH_SLAB_MINALIGN	MTE_GRANULE_SIZE
 #endif
 
 #ifndef __ASSEMBLY__
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 35/44] arm64: kasan: Add arch layer for memory tagging helpers
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (33 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-10 22:10 ` [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrey Konovalov
                   ` (9 subsequent siblings)
  44 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

This patch add a set of arch_*() memory tagging helpers currently only
defined for arm64 when hardware tag-based KASAN is enabled. These helpers
will be used by KASAN runtime to implement the hardware tag-based mode.

The arch-level indirection level is introduced to simplify adding hardware
tag-based KASAN support for other architectures in the future by defining
the appropriate arch_*() macros.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I42b0795a28067872f8308e00c6f0195bca435c2a
---
 arch/arm64/include/asm/memory.h |  9 +++++++++
 mm/kasan/kasan.h                | 26 ++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)

diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index cd61239bae8c..419bbace29d5 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -230,6 +230,15 @@ static inline const void *__tag_set(const void *addr, u8 tag)
 	return (const void *)(__addr | __tag_shifted(tag));
 }
 
+#ifdef CONFIG_KASAN_HW_TAGS
+#define arch_enable_tagging()			mte_enable()
+#define arch_init_tags(max_tag)			mte_init_tags(max_tag)
+#define arch_get_random_tag()			mte_get_random_tag()
+#define arch_get_mem_tag(addr)			mte_get_mem_tag(addr)
+#define arch_set_mem_tag_range(addr, size, tag)	\
+			mte_set_mem_tag_range((addr), (size), (tag))
+#endif /* CONFIG_KASAN_HW_TAGS */
+
 /*
  * Physical vs virtual RAM address space conversion.  These are
  * private definitions which should NOT be used outside memory.h
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index b5b00bff358f..ae7def3b725b 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -241,6 +241,32 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag)
 #define reset_tag(addr)		((void *)arch_kasan_reset_tag(addr))
 #define get_tag(addr)		arch_kasan_get_tag(addr)
 
+#ifdef CONFIG_KASAN_HW_TAGS
+
+#ifndef arch_enable_tagging
+#define arch_enable_tagging()
+#endif
+#ifndef arch_init_tags
+#define arch_init_tags(max_tag)
+#endif
+#ifndef arch_get_random_tag
+#define arch_get_random_tag()	(0xFF)
+#endif
+#ifndef arch_get_mem_tag
+#define arch_get_mem_tag(addr)	(0xFF)
+#endif
+#ifndef arch_set_mem_tag_range
+#define arch_set_mem_tag_range(addr, size, tag) ((void *)(addr))
+#endif
+
+#define hw_enable_tagging()			arch_enable_tagging()
+#define hw_init_tags(max_tag)			arch_init_tags(max_tag)
+#define hw_get_random_tag()			arch_get_random_tag()
+#define hw_get_mem_tag(addr)			arch_get_mem_tag(addr)
+#define hw_set_mem_tag_range(addr, size, tag)	arch_set_mem_tag_range((addr), (size), (tag))
+
+#endif /* CONFIG_KASAN_HW_TAGS */
+
 /*
  * Exported functions for interfaces called from assembly or from generated
  * code. Declarations here to avoid warning about missing declarations.
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (34 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 35/44] arm64: kasan: Add arch layer for memory tagging helpers Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:13   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN Andrey Konovalov
                   ` (8 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Hardware tag-based KASAN has granules of MTE_GRANULE_SIZE. Define
KASAN_GRANULE_SIZE to MTE_GRANULE_SIZE for CONFIG_KASAN_HW_TAGS.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: I5d1117e6a991cbca00d2cfb4ba66e8ae2d8f513a
---
 mm/kasan/kasan.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index ae7def3b725b..d745a78745dd 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -5,7 +5,13 @@
 #include <linux/kasan.h>
 #include <linux/stackdepot.h>
 
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define KASAN_GRANULE_SIZE	(1UL << KASAN_SHADOW_SCALE_SHIFT)
+#else
+#include <asm/mte-kasan.h>
+#define KASAN_GRANULE_SIZE	MTE_GRANULE_SIZE
+#endif
+
 #define KASAN_GRANULE_MASK	(KASAN_GRANULE_SIZE - 1)
 #define KASAN_GRANULE_PAGE	(KASAN_GRANULE_SIZE << PAGE_SHIFT)
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (35 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:15   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks Andrey Konovalov
                   ` (7 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov,
	Vasily Gorbik

With the intoduction of hardware tag-based KASAN some kernel checks of
this kind:

  ifdef CONFIG_KASAN

will be updated to:

  if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)

x86 and s390 use a trick to #undef CONFIG_KASAN for some of the code
that isn't linked with KASAN runtime and shouldn't have any KASAN
annotations.

Also #undef CONFIG_KASAN_GENERIC with CONFIG_KASAN.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
---
Change-Id: I2a622db0cb86a8feb60c30d8cb09190075be2a90
---
 arch/s390/boot/string.c         | 1 +
 arch/x86/boot/compressed/misc.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/s390/boot/string.c b/arch/s390/boot/string.c
index b11e8108773a..faccb33b462c 100644
--- a/arch/s390/boot/string.c
+++ b/arch/s390/boot/string.c
@@ -3,6 +3,7 @@
 #include <linux/kernel.h>
 #include <linux/errno.h>
 #undef CONFIG_KASAN
+#undef CONFIG_KASAN_GENERIC
 #include "../lib/string.c"
 
 int strncmp(const char *cs, const char *ct, size_t count)
diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
index d9a631c5973c..901ea5ebec22 100644
--- a/arch/x86/boot/compressed/misc.h
+++ b/arch/x86/boot/compressed/misc.h
@@ -12,6 +12,7 @@
 #undef CONFIG_PARAVIRT_XXL
 #undef CONFIG_PARAVIRT_SPINLOCKS
 #undef CONFIG_KASAN
+#undef CONFIG_KASAN_GENERIC
 
 /* cpu_feature_enabled() cannot be used this early */
 #define USE_EARLY_PGTABLE_L5
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (36 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:22   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime Andrey Konovalov
                   ` (6 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN
modes (either related to shadow memory or compiler instrumentation).
Expand those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I91e661e2c1627783cb845d877c6371dfc8779505
---
 arch/arm64/Kconfig                 |  2 +-
 arch/arm64/Makefile                |  2 +-
 arch/arm64/include/asm/assembler.h |  2 +-
 arch/arm64/include/asm/memory.h    |  2 +-
 arch/arm64/include/asm/string.h    |  5 +++--
 arch/arm64/kernel/head.S           |  2 +-
 arch/arm64/kernel/image-vars.h     |  2 +-
 arch/arm64/kernel/kaslr.c          |  3 ++-
 arch/arm64/kernel/module.c         |  6 ++++--
 arch/arm64/mm/ptdump.c             |  6 +++---
 include/linux/kasan-checks.h       |  2 +-
 include/linux/kasan.h              |  7 ++++---
 include/linux/moduleloader.h       |  3 ++-
 include/linux/string.h             |  2 +-
 mm/ptdump.c                        | 13 ++++++++-----
 scripts/Makefile.lib               |  2 ++
 16 files changed, 36 insertions(+), 25 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index c84a0e6b4650..456741645f01 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -330,7 +330,7 @@ config BROKEN_GAS_INST
 
 config KASAN_SHADOW_OFFSET
 	hex
-	depends on KASAN
+	depends on KASAN_GENERIC || KASAN_SW_TAGS
 	default 0xdfffa00000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && !KASAN_SW_TAGS
 	default 0xdfffd00000000000 if ARM64_VA_BITS_47 && !KASAN_SW_TAGS
 	default 0xdffffe8000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 50ad9cbccb51..0b31a3f06f15 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -141,7 +141,7 @@ head-y		:= arch/arm64/kernel/head.o
 
 ifeq ($(CONFIG_KASAN_SW_TAGS), y)
 KASAN_SHADOW_SCALE_SHIFT := 4
-else
+else ifeq ($(CONFIG_KASAN_GENERIC), y)
 KASAN_SHADOW_SCALE_SHIFT := 3
 endif
 
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index ddbe6bf00e33..bf125c591116 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -473,7 +473,7 @@ USER(\label, ic	ivau, \tmp2)			// invalidate I line PoU
 #define NOKPROBE(x)
 #endif
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define EXPORT_SYMBOL_NOKASAN(name)
 #else
 #define EXPORT_SYMBOL_NOKASAN(name)	EXPORT_SYMBOL(name)
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 419bbace29d5..656aaddb7014 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -72,7 +72,7 @@
  * address space for the shadow region respectively. They can bloat the stack
  * significantly, so double the (minimum) stack size when they are in use.
  */
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 #define KASAN_SHADOW_OFFSET	_AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
 #define KASAN_SHADOW_END	((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) \
 					+ KASAN_SHADOW_OFFSET)
diff --git a/arch/arm64/include/asm/string.h b/arch/arm64/include/asm/string.h
index b31e8e87a0db..3a3264ff47b9 100644
--- a/arch/arm64/include/asm/string.h
+++ b/arch/arm64/include/asm/string.h
@@ -5,7 +5,7 @@
 #ifndef __ASM_STRING_H
 #define __ASM_STRING_H
 
-#ifndef CONFIG_KASAN
+#if !(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 #define __HAVE_ARCH_STRRCHR
 extern char *strrchr(const char *, int c);
 
@@ -48,7 +48,8 @@ extern void *__memset(void *, int, __kernel_size_t);
 void memcpy_flushcache(void *dst, const void *src, size_t cnt);
 #endif
 
-#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+	!defined(__SANITIZE_ADDRESS__)
 
 /*
  * For files that are not instrumented (e.g. mm/slub.c) we
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index d8d9caf02834..fdcb99d7ba23 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -448,7 +448,7 @@ SYM_FUNC_START_LOCAL(__primary_switched)
 	bl	__pi_memset
 	dsb	ishst				// Make zero page visible to PTW
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	bl	kasan_early_init
 #endif
 #ifdef CONFIG_RANDOMIZE_BASE
diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
index c615b285ff5b..4282edd2fe81 100644
--- a/arch/arm64/kernel/image-vars.h
+++ b/arch/arm64/kernel/image-vars.h
@@ -37,7 +37,7 @@ __efistub_strncmp		= __pi_strncmp;
 __efistub_strrchr		= __pi_strrchr;
 __efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc;
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 __efistub___memcpy		= __pi_memcpy;
 __efistub___memmove		= __pi_memmove;
 __efistub___memset		= __pi_memset;
diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c
index b181e0544b79..e8e17e91aa02 100644
--- a/arch/arm64/kernel/kaslr.c
+++ b/arch/arm64/kernel/kaslr.c
@@ -151,7 +151,8 @@ u64 __init kaslr_early_init(u64 dt_phys)
 	/* use the top 16 bits to randomize the linear region */
 	memstart_offset_seed = seed >> 48;
 
-	if (IS_ENABLED(CONFIG_KASAN))
+	if (IS_ENABLED(CONFIG_KASAN_GENERIC) ||
+	    IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		/*
 		 * KASAN does not expect the module region to intersect the
 		 * vmalloc region, since shadow memory is allocated for each
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 2a1ad95d9b2c..fe21e0f06492 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -30,7 +30,8 @@ void *module_alloc(unsigned long size)
 	if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS))
 		gfp_mask |= __GFP_NOWARN;
 
-	if (IS_ENABLED(CONFIG_KASAN))
+	if (IS_ENABLED(CONFIG_KASAN_GENERIC) ||
+	    IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		/* don't exceed the static module region - see below */
 		module_alloc_end = MODULES_END;
 
@@ -39,7 +40,8 @@ void *module_alloc(unsigned long size)
 				NUMA_NO_NODE, __builtin_return_address(0));
 
 	if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) &&
-	    !IS_ENABLED(CONFIG_KASAN))
+	    !IS_ENABLED(CONFIG_KASAN_GENERIC) &&
+	    !IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 		/*
 		 * KASAN can only deal with module allocations being served
 		 * from the reserved module region, since the remainder of
diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
index 807dc634bbd2..04137a8f3d2d 100644
--- a/arch/arm64/mm/ptdump.c
+++ b/arch/arm64/mm/ptdump.c
@@ -29,7 +29,7 @@
 enum address_markers_idx {
 	PAGE_OFFSET_NR = 0,
 	PAGE_END_NR,
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	KASAN_START_NR,
 #endif
 };
@@ -37,7 +37,7 @@ enum address_markers_idx {
 static struct addr_marker address_markers[] = {
 	{ PAGE_OFFSET,			"Linear Mapping start" },
 	{ 0 /* PAGE_END */,		"Linear Mapping end" },
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	{ 0 /* KASAN_SHADOW_START */,	"Kasan shadow start" },
 	{ KASAN_SHADOW_END,		"Kasan shadow end" },
 #endif
@@ -383,7 +383,7 @@ void ptdump_check_wx(void)
 static int ptdump_init(void)
 {
 	address_markers[PAGE_END_NR].start_address = PAGE_END;
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	address_markers[KASAN_START_NR].start_address = KASAN_SHADOW_START;
 #endif
 	ptdump_initialize();
diff --git a/include/linux/kasan-checks.h b/include/linux/kasan-checks.h
index ac6aba632f2d..ca5e89fb10d3 100644
--- a/include/linux/kasan-checks.h
+++ b/include/linux/kasan-checks.h
@@ -9,7 +9,7 @@
  * even in compilation units that selectively disable KASAN, but must use KASAN
  * to validate access to an address.   Never use these in header files!
  */
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 bool __kasan_check_read(const volatile void *p, unsigned int size);
 bool __kasan_check_write(const volatile void *p, unsigned int size);
 #else
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 1d6ec3325163..b6fc14b3da53 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -237,7 +237,8 @@ static inline void kasan_release_vmalloc(unsigned long start,
 
 #endif /* CONFIG_KASAN_VMALLOC */
 
-#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+		!defined(CONFIG_KASAN_VMALLOC)
 
 /*
  * These functions provide a special case to support backing module
@@ -247,12 +248,12 @@ static inline void kasan_release_vmalloc(unsigned long start,
 int kasan_module_alloc(void *addr, size_t size);
 void kasan_free_shadow(const struct vm_struct *vm);
 
-#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
+#else /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */
 
 static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
 static inline void kasan_free_shadow(const struct vm_struct *vm) {}
 
-#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
+#endif /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */
 
 #ifdef CONFIG_KASAN_INLINE
 void kasan_non_canonical_hook(unsigned long addr);
diff --git a/include/linux/moduleloader.h b/include/linux/moduleloader.h
index 4fa67a8b2265..9e09d11ffe5b 100644
--- a/include/linux/moduleloader.h
+++ b/include/linux/moduleloader.h
@@ -96,7 +96,8 @@ void module_arch_cleanup(struct module *mod);
 /* Any cleanup before freeing mod->module_init */
 void module_arch_freeing_init(struct module *mod);
 
-#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
+#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
+		!defined(CONFIG_KASAN_VMALLOC)
 #include <linux/kasan.h>
 #define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT)
 #else
diff --git a/include/linux/string.h b/include/linux/string.h
index b1f3894a0a3e..016a157e2251 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -266,7 +266,7 @@ void __write_overflow(void) __compiletime_error("detected write beyond size of o
 
 #if !defined(__NO_FORTIFY) && defined(__OPTIMIZE__) && defined(CONFIG_FORTIFY_SOURCE)
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 extern void *__underlying_memchr(const void *p, int c, __kernel_size_t size) __RENAME(memchr);
 extern int __underlying_memcmp(const void *p, const void *q, __kernel_size_t size) __RENAME(memcmp);
 extern void *__underlying_memcpy(void *p, const void *q, __kernel_size_t size) __RENAME(memcpy);
diff --git a/mm/ptdump.c b/mm/ptdump.c
index ba88ec43ff21..4354c1422d57 100644
--- a/mm/ptdump.c
+++ b/mm/ptdump.c
@@ -4,7 +4,7 @@
 #include <linux/ptdump.h>
 #include <linux/kasan.h>
 
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 /*
  * This is an optimization for KASAN=y case. Since all kasan page tables
  * eventually point to the kasan_early_shadow_page we could call note_page()
@@ -31,7 +31,8 @@ static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr,
 	struct ptdump_state *st = walk->private;
 	pgd_t val = READ_ONCE(*pgd);
 
-#if CONFIG_PGTABLE_LEVELS > 4 && defined(CONFIG_KASAN)
+#if CONFIG_PGTABLE_LEVELS > 4 && \
+		(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 	if (pgd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_p4d)))
 		return note_kasan_page_table(walk, addr);
 #endif
@@ -51,7 +52,8 @@ static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr,
 	struct ptdump_state *st = walk->private;
 	p4d_t val = READ_ONCE(*p4d);
 
-#if CONFIG_PGTABLE_LEVELS > 3 && defined(CONFIG_KASAN)
+#if CONFIG_PGTABLE_LEVELS > 3 && \
+		(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 	if (p4d_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pud)))
 		return note_kasan_page_table(walk, addr);
 #endif
@@ -71,7 +73,8 @@ static int ptdump_pud_entry(pud_t *pud, unsigned long addr,
 	struct ptdump_state *st = walk->private;
 	pud_t val = READ_ONCE(*pud);
 
-#if CONFIG_PGTABLE_LEVELS > 2 && defined(CONFIG_KASAN)
+#if CONFIG_PGTABLE_LEVELS > 2 && \
+		(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
 	if (pud_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pmd)))
 		return note_kasan_page_table(walk, addr);
 #endif
@@ -91,7 +94,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr,
 	struct ptdump_state *st = walk->private;
 	pmd_t val = READ_ONCE(*pmd);
 
-#if defined(CONFIG_KASAN)
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	if (pmd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pte)))
 		return note_kasan_page_table(walk, addr);
 #endif
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 94133708889d..213677a5ed33 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -148,10 +148,12 @@ endif
 # we don't want to check (depends on variables KASAN_SANITIZE_obj.o, KASAN_SANITIZE)
 #
 ifeq ($(CONFIG_KASAN),y)
+ifneq ($(CONFIG_KASAN_HW_TAGS),y)
 _c_flags += $(if $(patsubst n%,, \
 		$(KASAN_SANITIZE_$(basetarget).o)$(KASAN_SANITIZE)y), \
 		$(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE))
 endif
+endif
 
 ifeq ($(CONFIG_UBSAN),y)
 _c_flags += $(if $(patsubst n%,, \
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (37 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-12 17:37   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 40/44] kasan, arm64: print report from tag fault handler Andrey Konovalov
                   ` (5 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Provide implementation of KASAN functions required for the hardware
tag-based mode. Those include core functions for memory and pointer
tagging (tags_hw.c) and bug reporting (report_tags_hw.c). Also adapt
common KASAN code to support the new mode.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I8a8689ba098174a4d0ef3f1d008178387c80ee1c
---
 arch/arm64/include/asm/memory.h   |  4 +-
 arch/arm64/kernel/cpufeature.c    |  3 ++
 arch/arm64/kernel/smp.c           |  2 +
 include/linux/kasan.h             | 24 +++++++---
 include/linux/mm.h                |  2 +-
 include/linux/page-flags-layout.h |  2 +-
 mm/kasan/Makefile                 |  5 ++
 mm/kasan/common.c                 | 15 +++---
 mm/kasan/hw_tags.c                | 80 +++++++++++++++++++++++++++++++
 mm/kasan/kasan.h                  | 17 +++++--
 mm/kasan/report_hw_tags.c         | 42 ++++++++++++++++
 mm/kasan/report_sw_tags.c         |  2 +-
 mm/kasan/shadow.c                 |  2 +-
 mm/kasan/sw_tags.c                |  2 +-
 14 files changed, 177 insertions(+), 25 deletions(-)
 create mode 100644 mm/kasan/hw_tags.c
 create mode 100644 mm/kasan/report_hw_tags.c

diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 656aaddb7014..5042eef5b111 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -214,7 +214,7 @@ static inline unsigned long kaslr_offset(void)
 	(__force __typeof__(addr))__addr;				\
 })
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 #define __tag_shifted(tag)	((u64)(tag) << 56)
 #define __tag_reset(addr)	__untagged_addr(addr)
 #define __tag_get(addr)		(__u8)((u64)(addr) >> 56)
@@ -222,7 +222,7 @@ static inline unsigned long kaslr_offset(void)
 #define __tag_shifted(tag)	0UL
 #define __tag_reset(addr)	(addr)
 #define __tag_get(addr)		0
-#endif /* CONFIG_KASAN_SW_TAGS */
+#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
 
 static inline const void *__tag_set(const void *addr, u8 tag)
 {
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index dcc165b3fc04..6a1f2e3558c5 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -70,6 +70,7 @@
 #include <linux/types.h>
 #include <linux/mm.h>
 #include <linux/cpu.h>
+#include <linux/kasan.h>
 #include <asm/cpu.h>
 #include <asm/cpufeature.h>
 #include <asm/cpu_ops.h>
@@ -1704,6 +1705,8 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap)
 		cleared_zero_page = true;
 		mte_clear_page_tags(lm_alias(empty_zero_page));
 	}
+
+	kasan_init_hw_tags_cpu();
 }
 #endif /* CONFIG_ARM64_MTE */
 
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 09c96f57818c..7235b9478413 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -461,6 +461,8 @@ void __init smp_prepare_boot_cpu(void)
 	/* Conditionally switch to GIC PMR for interrupt masking */
 	if (system_uses_irq_prio_masking())
 		init_gic_priority_masking();
+
+	kasan_init_hw_tags();
 }
 
 static u64 __init of_get_cpu_mpidr(struct device_node *dn)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index b6fc14b3da53..f22bdef82111 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -189,25 +189,35 @@ static inline void kasan_record_aux_stack(void *ptr) {}
 
 #endif /* CONFIG_KASAN_GENERIC */
 
-#ifdef CONFIG_KASAN_SW_TAGS
-
-void __init kasan_init_sw_tags(void);
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 
 void *kasan_reset_tag(const void *addr);
 
 bool kasan_report(unsigned long addr, size_t size,
 		bool is_write, unsigned long ip);
 
-#else /* CONFIG_KASAN_SW_TAGS */
-
-static inline void kasan_init_sw_tags(void) { }
+#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
 
 static inline void *kasan_reset_tag(const void *addr)
 {
 	return (void *)addr;
 }
 
-#endif /* CONFIG_KASAN_SW_TAGS */
+#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS*/
+
+#ifdef CONFIG_KASAN_SW_TAGS
+void __init kasan_init_sw_tags(void);
+#else
+static inline void kasan_init_sw_tags(void) { }
+#endif
+
+#ifdef CONFIG_KASAN_HW_TAGS
+void kasan_init_hw_tags_cpu(void);
+void kasan_init_hw_tags(void);
+#else
+static inline void kasan_init_hw_tags_cpu(void) { }
+static inline void kasan_init_hw_tags(void) { }
+#endif
 
 #ifdef CONFIG_KASAN_VMALLOC
 
diff --git a/include/linux/mm.h b/include/linux/mm.h
index db6ae4d3fb4e..0793d03a4183 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1413,7 +1413,7 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid)
 }
 #endif /* CONFIG_NUMA_BALANCING */
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 static inline u8 page_kasan_tag(const struct page *page)
 {
 	return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h
index e200eef6a7fd..7d4ec26d8a3e 100644
--- a/include/linux/page-flags-layout.h
+++ b/include/linux/page-flags-layout.h
@@ -77,7 +77,7 @@
 #define LAST_CPUPID_SHIFT 0
 #endif
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 #define KASAN_TAG_WIDTH 8
 #else
 #define KASAN_TAG_WIDTH 0
diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index f1d68a34f3c9..9fe39a66388a 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -10,8 +10,10 @@ CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report_hw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_hw_tags.o = $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE)
 
 # Function splitter causes unnecessary splits in __asan_load1/__asan_store1
@@ -27,10 +29,13 @@ CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_report_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
+CFLAGS_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
 
 obj-$(CONFIG_KASAN) := common.o report.o
 obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o
+obj-$(CONFIG_KASAN_HW_TAGS) += hw_tags.o report_hw_tags.o
 obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index d0b3ff410b0c..2bb0ef6da6bd 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -113,7 +113,7 @@ void kasan_free_pages(struct page *page, unsigned int order)
  */
 static inline unsigned int optimal_redzone(unsigned int object_size)
 {
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
+	if (!IS_ENABLED(CONFIG_KASAN_GENERIC))
 		return 0;
 
 	return
@@ -178,14 +178,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache)
 struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
 					const void *object)
 {
-	return (void *)object + cache->kasan_info.alloc_meta_offset;
+	return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset;
 }
 
 struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
 				      const void *object)
 {
 	BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
-	return (void *)object + cache->kasan_info.free_meta_offset;
+	return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset;
 }
 
 void kasan_poison_slab(struct page *page)
@@ -267,9 +267,8 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
 	alloc_info = get_alloc_info(cache, object);
 	__memset(alloc_info, 0, sizeof(*alloc_info));
 
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
-		object = set_tag(object,
-				assign_tag(cache, object, true, false));
+	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
+		object = set_tag(object, assign_tag(cache, object, true, false));
 
 	return (void *)object;
 }
@@ -337,10 +336,10 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
 	redzone_end = round_up((unsigned long)object + cache->object_size,
 				KASAN_GRANULE_SIZE);
 
-	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
+	if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
 		tag = assign_tag(cache, object, false, keep_tag);
 
-	/* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
+	/* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */
 	kasan_unpoison_memory(set_tag(object, tag), size);
 	kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start,
 		KASAN_KMALLOC_REDZONE);
diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c
new file mode 100644
index 000000000000..0080b78ec843
--- /dev/null
+++ b/mm/kasan/hw_tags.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This file contains core hardware tag-based KASAN code.
+ *
+ * Copyright (c) 2020 Google, Inc.
+ * Author: Andrey Konovalov <andreyknvl@google.com>
+ */
+
+#define pr_fmt(fmt) "kasan: " fmt
+
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "kasan.h"
+
+/* kasan_init_hw_tags_cpu() is called for each CPU. */
+void kasan_init_hw_tags_cpu(void)
+{
+	hw_init_tags(KASAN_TAG_MAX);
+	hw_enable_tagging();
+}
+
+/* kasan_init_hw_tags() is called once on boot CPU. */
+void kasan_init_hw_tags(void)
+{
+	pr_info("KernelAddressSanitizer initialized\n");
+}
+
+void *kasan_reset_tag(const void *addr)
+{
+	return reset_tag(addr);
+}
+
+void kasan_poison_memory(const void *address, size_t size, u8 value)
+{
+	hw_set_mem_tag_range(reset_tag(address),
+			round_up(size, KASAN_GRANULE_SIZE), value);
+}
+
+void kasan_unpoison_memory(const void *address, size_t size)
+{
+	hw_set_mem_tag_range(reset_tag(address),
+			round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
+}
+
+u8 random_tag(void)
+{
+	return hw_get_random_tag();
+}
+
+bool check_invalid_free(void *addr)
+{
+	u8 ptr_tag = get_tag(addr);
+	u8 mem_tag = hw_get_mem_tag(addr);
+
+	return (mem_tag == KASAN_TAG_INVALID) ||
+		(ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
+}
+
+void kasan_set_free_info(struct kmem_cache *cache,
+				void *object, u8 tag)
+{
+	struct kasan_alloc_meta *alloc_meta;
+
+	alloc_meta = get_alloc_info(cache, object);
+	kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT);
+}
+
+struct kasan_track *kasan_get_free_track(struct kmem_cache *cache,
+				void *object, u8 tag)
+{
+	struct kasan_alloc_meta *alloc_meta;
+
+	alloc_meta = get_alloc_info(cache, object);
+	return &alloc_meta->free_track[0];
+}
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index d745a78745dd..21fe75c66f26 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -153,6 +153,10 @@ struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
 struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
 					const void *object);
 
+void kasan_poison_memory(const void *address, size_t size, u8 value);
+
+#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
+
 static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
 {
 	return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET)
@@ -164,8 +168,6 @@ static inline bool addr_has_metadata(const void *addr)
 	return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
 }
 
-void kasan_poison_memory(const void *address, size_t size, u8 value);
-
 /**
  * check_memory_region - Check memory region, and report if invalid access.
  * @addr: the accessed address
@@ -177,6 +179,15 @@ void kasan_poison_memory(const void *address, size_t size, u8 value);
 bool check_memory_region(unsigned long addr, size_t size, bool write,
 				unsigned long ret_ip);
 
+#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
+static inline bool addr_has_metadata(const void *addr)
+{
+	return true;
+}
+
+#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
+
 bool check_invalid_free(void *addr);
 
 void *find_first_bad_addr(void *addr, size_t size);
@@ -213,7 +224,7 @@ static inline void quarantine_reduce(void) { }
 static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
 #endif
 
-#ifdef CONFIG_KASAN_SW_TAGS
+#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
 
 void print_tags(u8 addr_tag, const void *addr);
 
diff --git a/mm/kasan/report_hw_tags.c b/mm/kasan/report_hw_tags.c
new file mode 100644
index 000000000000..da543eb832cd
--- /dev/null
+++ b/mm/kasan/report_hw_tags.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * This file contains hardware tag-based KASAN specific error reporting code.
+ *
+ * Copyright (c) 2020 Google, Inc.
+ * Author: Andrey Konovalov <andreyknvl@google.com>
+ */
+
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "kasan.h"
+
+const char *get_bug_type(struct kasan_access_info *info)
+{
+	return "invalid-access";
+}
+
+void *find_first_bad_addr(void *addr, size_t size)
+{
+	return reset_tag(addr);
+}
+
+void metadata_fetch_row(char *buffer, void *row)
+{
+	int i;
+
+	for (i = 0; i < META_BYTES_PER_ROW; i++)
+		buffer[i] = hw_get_mem_tag(row + i * KASAN_GRANULE_SIZE);
+}
+
+void print_tags(u8 addr_tag, const void *addr)
+{
+	u8 memory_tag = hw_get_mem_tag((void *)addr);
+
+	pr_err("Pointer tag: [%02x], memory tag: [%02x]\n",
+		addr_tag, memory_tag);
+}
diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c
index add2dfe6169c..aebc44a29e83 100644
--- a/mm/kasan/report_sw_tags.c
+++ b/mm/kasan/report_sw_tags.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains tag-based KASAN specific error reporting code.
+ * This file contains software tag-based KASAN specific error reporting code.
  *
  * Copyright (c) 2014 Samsung Electronics Co., Ltd.
  * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index 1fadd4930d54..616ac64c4a21 100644
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -107,7 +107,7 @@ void kasan_unpoison_memory(const void *address, size_t size)
 
 		if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
 			*shadow = tag;
-		else
+		else /* CONFIG_KASAN_GENERIC */
 			*shadow = size & KASAN_GRANULE_MASK;
 	}
 }
diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
index b09a2c06abad..dfe707dd8d0d 100644
--- a/mm/kasan/sw_tags.c
+++ b/mm/kasan/sw_tags.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * This file contains core tag-based KASAN code.
+ * This file contains core software tag-based KASAN code.
  *
  * Copyright (c) 2018 Google, Inc.
  * Author: Andrey Konovalov <andreyknvl@google.com>
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 40/44] kasan, arm64: print report from tag fault handler
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (38 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:43   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata Andrey Konovalov
                   ` (4 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Add error reporting for hardware tag-based KASAN. When CONFIG_KASAN_HW_TAGS
is enabled, print KASAN report from the arm64 tag fault handler.

SAS bits aren't set in ESR for all faults reported in EL1, so it's
impossible to find out the size of the access the caused the fault.
Adapt KASAN reporting code to handle this case.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I3780fe7db6e075dff2937d3d8508f55c9322b095
---
 arch/arm64/mm/fault.c | 14 ++++++++++++++
 mm/kasan/report.c     | 11 ++++++++---
 2 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index fbceb14d93b1..7370e822e588 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -14,6 +14,7 @@
 #include <linux/mm.h>
 #include <linux/hardirq.h>
 #include <linux/init.h>
+#include <linux/kasan.h>
 #include <linux/kprobes.h>
 #include <linux/uaccess.h>
 #include <linux/page-flags.h>
@@ -297,10 +298,23 @@ static void die_kernel_fault(const char *msg, unsigned long addr,
 	do_exit(SIGKILL);
 }
 
+#ifdef CONFIG_KASAN_HW_TAGS
 static void report_tag_fault(unsigned long addr, unsigned int esr,
 			     struct pt_regs *regs)
 {
+	bool is_write  = ((esr & ESR_ELx_WNR) >> ESR_ELx_WNR_SHIFT) != 0;
+
+	/*
+	 * SAS bits aren't set for all faults reported in EL1, so we can't
+	 * find out access size.
+	 */
+	kasan_report(addr, 0, is_write, regs->pc);
 }
+#else
+/* Tag faults aren't enabled without CONFIG_KASAN_HW_TAGS. */
+static inline void report_tag_fault(unsigned long addr, unsigned int esr,
+				    struct pt_regs *regs) { }
+#endif
 
 static void do_tag_recovery(unsigned long addr, unsigned int esr,
 			   struct pt_regs *regs)
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 8afc1a6ab202..ce06005d4052 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -62,9 +62,14 @@ static void print_error_description(struct kasan_access_info *info)
 {
 	pr_err("BUG: KASAN: %s in %pS\n",
 		get_bug_type(info), (void *)info->ip);
-	pr_err("%s of size %zu at addr %px by task %s/%d\n",
-		info->is_write ? "Write" : "Read", info->access_size,
-		info->access_addr, current->comm, task_pid_nr(current));
+	if (info->access_size)
+		pr_err("%s of size %zu at addr %px by task %s/%d\n",
+			info->is_write ? "Write" : "Read", info->access_size,
+			info->access_addr, current->comm, task_pid_nr(current));
+	else
+		pr_err("%s at addr %px by task %s/%d\n",
+			info->is_write ? "Write" : "Read",
+			info->access_addr, current->comm, task_pid_nr(current));
 }
 
 static DEFINE_SPINLOCK(report_lock);
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (39 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 40/44] kasan, arm64: print report from tag fault handler Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-12 16:10   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrey Konovalov
                   ` (3 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Kernel allocator code accesses metadata for slab objects, that may lie
out-of-bounds of the object itself, or be accessed when an object is freed.
Such accesses trigger tag faults and lead to false-positive reports with
hardware tag-based KASAN.

Software KASAN modes disable instrumentation for allocator code via
KASAN_SANITIZE Makefile macro, and rely on kasan_enable/disable_current()
annotations which are used to ignore KASAN reports.

With hardware tag-based KASAN neither of those options are available, as
it doesn't use compiler instrumetation, no tag faults are ignored, and MTE
is disabled after the first one.

Instead, reset tags when accessing metadata (currently only for SLUB).

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Acked-by: Marco Elver <elver@google.com>
---
Change-Id: I39f3c4d4f29299d4fbbda039bedf230db1c746fb
---
 mm/page_alloc.c  |  4 +++-
 mm/page_poison.c |  2 +-
 mm/slub.c        | 29 ++++++++++++++++-------------
 3 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 24b45261e2bd..f1648aee8d88 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1195,8 +1195,10 @@ static void kernel_init_free_pages(struct page *page, int numpages)
 
 	/* s390's use of memset() could override KASAN redzones. */
 	kasan_disable_current();
-	for (i = 0; i < numpages; i++)
+	for (i = 0; i < numpages; i++) {
+		page_kasan_tag_reset(page + i);
 		clear_highpage(page + i);
+	}
 	kasan_enable_current();
 }
 
diff --git a/mm/page_poison.c b/mm/page_poison.c
index ae0482cded87..e6c994af7518 100644
--- a/mm/page_poison.c
+++ b/mm/page_poison.c
@@ -53,7 +53,7 @@ static void poison_page(struct page *page)
 
 	/* KASAN still think the page is in-use, so skip it. */
 	kasan_disable_current();
-	memset(addr, PAGE_POISON, PAGE_SIZE);
+	memset(kasan_reset_tag(addr), PAGE_POISON, PAGE_SIZE);
 	kasan_enable_current();
 	kunmap_atomic(addr);
 }
diff --git a/mm/slub.c b/mm/slub.c
index b30be2385d1c..df2fd5b57df1 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -249,7 +249,7 @@ static inline void *freelist_ptr(const struct kmem_cache *s, void *ptr,
 {
 #ifdef CONFIG_SLAB_FREELIST_HARDENED
 	/*
-	 * When CONFIG_KASAN_SW_TAGS is enabled, ptr_addr might be tagged.
+	 * When CONFIG_KASAN_SW/HW_TAGS is enabled, ptr_addr might be tagged.
 	 * Normally, this doesn't cause any issues, as both set_freepointer()
 	 * and get_freepointer() are called with a pointer with the same tag.
 	 * However, there are some issues with CONFIG_SLUB_DEBUG code. For
@@ -275,6 +275,7 @@ static inline void *freelist_dereference(const struct kmem_cache *s,
 
 static inline void *get_freepointer(struct kmem_cache *s, void *object)
 {
+	object = kasan_reset_tag(object);
 	return freelist_dereference(s, object + s->offset);
 }
 
@@ -304,6 +305,7 @@ static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp)
 	BUG_ON(object == fp); /* naive detection of double free or corruption */
 #endif
 
+	freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr);
 	*(void **)freeptr_addr = freelist_ptr(s, fp, freeptr_addr);
 }
 
@@ -538,8 +540,8 @@ static void print_section(char *level, char *text, u8 *addr,
 			  unsigned int length)
 {
 	metadata_access_enable();
-	print_hex_dump(level, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
-			length, 1);
+	print_hex_dump(level, kasan_reset_tag(text), DUMP_PREFIX_ADDRESS,
+			16, 1, addr, length, 1);
 	metadata_access_disable();
 }
 
@@ -570,7 +572,7 @@ static struct track *get_track(struct kmem_cache *s, void *object,
 
 	p = object + get_info_end(s);
 
-	return p + alloc;
+	return kasan_reset_tag(p + alloc);
 }
 
 static void set_track(struct kmem_cache *s, void *object,
@@ -583,7 +585,8 @@ static void set_track(struct kmem_cache *s, void *object,
 		unsigned int nr_entries;
 
 		metadata_access_enable();
-		nr_entries = stack_trace_save(p->addrs, TRACK_ADDRS_COUNT, 3);
+		nr_entries = stack_trace_save(kasan_reset_tag(p->addrs),
+					      TRACK_ADDRS_COUNT, 3);
 		metadata_access_disable();
 
 		if (nr_entries < TRACK_ADDRS_COUNT)
@@ -747,7 +750,7 @@ static __printf(3, 4) void slab_err(struct kmem_cache *s, struct page *page,
 
 static void init_object(struct kmem_cache *s, void *object, u8 val)
 {
-	u8 *p = object;
+	u8 *p = kasan_reset_tag(object);
 
 	if (s->flags & SLAB_RED_ZONE)
 		memset(p - s->red_left_pad, val, s->red_left_pad);
@@ -777,7 +780,7 @@ static int check_bytes_and_report(struct kmem_cache *s, struct page *page,
 	u8 *addr = page_address(page);
 
 	metadata_access_enable();
-	fault = memchr_inv(start, value, bytes);
+	fault = memchr_inv(kasan_reset_tag(start), value, bytes);
 	metadata_access_disable();
 	if (!fault)
 		return 1;
@@ -873,7 +876,7 @@ static int slab_pad_check(struct kmem_cache *s, struct page *page)
 
 	pad = end - remainder;
 	metadata_access_enable();
-	fault = memchr_inv(pad, POISON_INUSE, remainder);
+	fault = memchr_inv(kasan_reset_tag(pad), POISON_INUSE, remainder);
 	metadata_access_disable();
 	if (!fault)
 		return 1;
@@ -1118,7 +1121,7 @@ void setup_page_debug(struct kmem_cache *s, struct page *page, void *addr)
 		return;
 
 	metadata_access_enable();
-	memset(addr, POISON_INUSE, page_size(page));
+	memset(kasan_reset_tag(addr), POISON_INUSE, page_size(page));
 	metadata_access_disable();
 }
 
@@ -1566,10 +1569,10 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s,
 			 * Clear the object and the metadata, but don't touch
 			 * the redzone.
 			 */
-			memset(object, 0, s->object_size);
+			memset(kasan_reset_tag(object), 0, s->object_size);
 			rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad
 							   : 0;
-			memset((char *)object + s->inuse, 0,
+			memset((char *)kasan_reset_tag(object) + s->inuse, 0,
 			       s->size - s->inuse - rsize);
 
 		}
@@ -2883,10 +2886,10 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
 		stat(s, ALLOC_FASTPATH);
 	}
 
-	maybe_wipe_obj_freeptr(s, object);
+	maybe_wipe_obj_freeptr(s, kasan_reset_tag(object));
 
 	if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object)
-		memset(object, 0, s->object_size);
+		memset(kasan_reset_tag(object), 0, s->object_size);
 
 	slab_post_alloc_hook(s, objcg, gfpflags, 1, &object);
 
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (40 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:49   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode Andrey Konovalov
                   ` (2 subsequent siblings)
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Hardware tag-based KASAN is now ready, enable the configuration option.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
---
Change-Id: I6eb1eea770e6b61ad71c701231b8d815a7ccc853
---
 arch/arm64/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 456741645f01..c35e73efd407 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -135,6 +135,7 @@ config ARM64
 	select HAVE_ARCH_JUMP_LABEL_RELATIVE
 	select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
 	select HAVE_ARCH_KASAN_SW_TAGS if (HAVE_ARCH_KASAN && !ARM64_MTE)
+	select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_MMAP_RND_BITS
 	select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (41 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-11 16:57   ` Alexander Potapenko
  2020-11-10 22:10 ` [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch Andrey Konovalov
  2020-11-10 22:54 ` [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrew Morton
  44 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

Add documentation for hardware tag-based KASAN mode and also add some
clarifications for software tag-based mode.

Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Marco Elver <elver@google.com>
---
Change-Id: Ib46cb444cfdee44054628940a82f5139e10d0258
---
 Documentation/dev-tools/kasan.rst | 78 ++++++++++++++++++++++---------
 1 file changed, 57 insertions(+), 21 deletions(-)

diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst
index edca4be5e405..422f8ee1bb17 100644
--- a/Documentation/dev-tools/kasan.rst
+++ b/Documentation/dev-tools/kasan.rst
@@ -5,12 +5,14 @@ Overview
 --------
 
 KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to
-find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN
-(similar to userspace ASan) and software tag-based KASAN (similar to userspace
-HWASan).
+find out-of-bound and use-after-free bugs. KASAN has three modes:
+1. generic KASAN (similar to userspace ASan),
+2. software tag-based KASAN (similar to userspace HWASan),
+3. hardware tag-based KASAN (based on hardware memory tagging).
 
-KASAN uses compile-time instrumentation to insert validity checks before every
-memory access, and therefore requires a compiler version that supports that.
+Software KASAN modes (1 and 2) use compile-time instrumentation to insert
+validity checks before every memory access, and therefore require a compiler
+version that supports that.
 
 Generic KASAN is supported in both GCC and Clang. With GCC it requires version
 8.3.0 or later. Any supported Clang version is compatible, but detection of
@@ -19,7 +21,7 @@ out-of-bounds accesses for global variables is only supported since Clang 11.
 Tag-based KASAN is only supported in Clang.
 
 Currently generic KASAN is supported for the x86_64, arm64, xtensa, s390 and
-riscv architectures, and tag-based KASAN is supported only for arm64.
+riscv architectures, and tag-based KASAN modes are supported only for arm64.
 
 Usage
 -----
@@ -28,14 +30,16 @@ To enable KASAN configure kernel with::
 
 	  CONFIG_KASAN = y
 
-and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and
-CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN).
+and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN),
+CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN), and
+CONFIG_KASAN_HW_TAGS (to enable hardware tag-based KASAN).
 
-You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE.
-Outline and inline are compiler instrumentation types. The former produces
-smaller binary while the latter is 1.1 - 2 times faster.
+For software modes, you also need to choose between CONFIG_KASAN_OUTLINE and
+CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types.
+The former produces smaller binary while the latter is 1.1 - 2 times faster.
 
-Both KASAN modes work with both SLUB and SLAB memory allocators.
+Both software KASAN modes work with both SLUB and SLAB memory allocators,
+hardware tag-based KASAN currently only support SLUB.
 For better bug detection and nicer reporting, enable CONFIG_STACKTRACE.
 
 To augment reports with last allocation and freeing stack of the physical page,
@@ -196,17 +200,24 @@ and the second to last.
 Software tag-based KASAN
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
-Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to
-store a pointer tag in the top byte of kernel pointers. Like generic KASAN it
-uses shadow memory to store memory tags associated with each 16-byte memory
+Software tag-based KASAN requires software memory tagging support in the form
+of HWASan-like compiler instrumentation (see HWASan documentation for details).
+
+Software tag-based KASAN is currently only implemented for arm64 architecture.
+
+Software tag-based KASAN uses the Top Byte Ignore (TBI) feature of arm64 CPUs
+to store a pointer tag in the top byte of kernel pointers. Like generic KASAN
+it uses shadow memory to store memory tags associated with each 16-byte memory
 cell (therefore it dedicates 1/16th of the kernel memory for shadow memory).
 
-On each memory allocation tag-based KASAN generates a random tag, tags the
-allocated memory with this tag, and embeds this tag into the returned pointer.
+On each memory allocation software tag-based KASAN generates a random tag, tags
+the allocated memory with this tag, and embeds this tag into the returned
+pointer.
+
 Software tag-based KASAN uses compile-time instrumentation to insert checks
 before each memory access. These checks make sure that tag of the memory that
 is being accessed is equal to tag of the pointer that is used to access this
-memory. In case of a tag mismatch tag-based KASAN prints a bug report.
+memory. In case of a tag mismatch software tag-based KASAN prints a bug report.
 
 Software tag-based KASAN also has two instrumentation modes (outline, that
 emits callbacks to check memory accesses; and inline, that performs the shadow
@@ -215,9 +226,34 @@ simply printed from the function that performs the access check. With inline
 instrumentation a brk instruction is emitted by the compiler, and a dedicated
 brk handler is used to print bug reports.
 
-A potential expansion of this mode is a hardware tag-based mode, which would
-use hardware memory tagging support instead of compiler instrumentation and
-manual shadow memory manipulation.
+Software tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through
+pointers with 0xFF pointer tag aren't checked). The value 0xFE is currently
+reserved to tag freed memory regions.
+
+Software tag-based KASAN currently only supports tagging of slab memory.
+
+Hardware tag-based KASAN
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Hardware tag-based KASAN is similar to the software mode in concept, but uses
+hardware memory tagging support instead of compiler instrumentation and
+shadow memory.
+
+Hardware tag-based KASAN is currently only implemented for arm64 architecture
+and based on both arm64 Memory Tagging Extension (MTE) introduced in ARMv8.5
+Instruction Set Architecture, and Top Byte Ignore (TBI).
+
+Special arm64 instructions are used to assign memory tags for each allocation.
+Same tags are assigned to pointers to those allocations. On every memory
+access, hardware makes sure that tag of the memory that is being accessed is
+equal to tag of the pointer that is used to access this memory. In case of a
+tag mismatch a fault is generated and a report is printed.
+
+Hardware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through
+pointers with 0xFF pointer tag aren't checked). The value 0xFE is currently
+reserved to tag freed memory regions.
+
+Hardware tag-based KASAN currently only supports tagging of slab memory.
 
 What memory accesses are sanitised by KASAN?
 --------------------------------------------
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (42 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode Andrey Konovalov
@ 2020-11-10 22:10 ` Andrey Konovalov
  2020-11-12  9:46   ` Catalin Marinas
  2020-11-12 15:59   ` Alexander Potapenko
  2020-11-10 22:54 ` [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrew Morton
  44 siblings, 2 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-10 22:10 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel, Andrey Konovalov

From: Vincenzo Frascino <vincenzo.frascino@arm.com>

This test is specific to MTE and verifies that the GCR_EL1 register
is context switched correctly.

It spawn 1024 processes and each process spawns 5 threads. Each thread
writes a random setting of GCR_EL1 through the prctl() system call and
reads it back verifying that it is the same. If the values are not the
same it reports a failure.

Note: The test has been extended to verify that even SYNC and ASYNC mode
setting is preserved correctly over context switching.

Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
Change-Id: Ia917684a2b8e5f29e705ca5cbf360b010df6f61e
---
 tools/testing/selftests/arm64/mte/Makefile    |   2 +-
 .../arm64/mte/check_gcr_el1_cswitch.c         | 152 ++++++++++++++++++
 2 files changed, 153 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c

diff --git a/tools/testing/selftests/arm64/mte/Makefile b/tools/testing/selftests/arm64/mte/Makefile
index 2480226dfe57..0b3af552632a 100644
--- a/tools/testing/selftests/arm64/mte/Makefile
+++ b/tools/testing/selftests/arm64/mte/Makefile
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 # Copyright (C) 2020 ARM Limited
 
-CFLAGS += -std=gnu99 -I.
+CFLAGS += -std=gnu99 -I. -lpthread
 SRCS := $(filter-out mte_common_util.c,$(wildcard *.c))
 PROGS := $(patsubst %.c,%,$(SRCS))
 
diff --git a/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c b/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c
new file mode 100644
index 000000000000..55e33d96794c
--- /dev/null
+++ b/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2020 ARM Limited
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <unistd.h>
+#include <sys/auxv.h>
+#include <sys/mman.h>
+#include <sys/prctl.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "kselftest.h"
+#include "mte_common_util.h"
+
+#define PR_SET_TAGGED_ADDR_CTRL 55
+#define PR_GET_TAGGED_ADDR_CTRL 56
+# define PR_TAGGED_ADDR_ENABLE  (1UL << 0)
+# define PR_MTE_TCF_SHIFT	1
+# define PR_MTE_TCF_NONE	(0UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_SYNC	(1UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_ASYNC	(2UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_MASK	(3UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TAG_SHIFT	3
+# define PR_MTE_TAG_MASK	(0xffffUL << PR_MTE_TAG_SHIFT)
+
+#include "mte_def.h"
+
+#define NUM_ITERATIONS		1024
+#define MAX_THREADS		5
+#define THREAD_ITERATIONS	1000
+
+void *execute_thread(void *x)
+{
+	pid_t pid = *((pid_t *)x);
+	pid_t tid = gettid();
+	uint64_t prctl_tag_mask;
+	uint64_t prctl_set;
+	uint64_t prctl_get;
+	uint64_t prctl_tcf;
+
+	srand(time(NULL) ^ (pid << 16) ^ (tid << 16));
+
+	prctl_tag_mask = rand() % 0xffff;
+
+	if (prctl_tag_mask % 2)
+		prctl_tcf = PR_MTE_TCF_SYNC;
+	else
+		prctl_tcf = PR_MTE_TCF_ASYNC;
+
+	prctl_set = PR_TAGGED_ADDR_ENABLE | prctl_tcf | (prctl_tag_mask << PR_MTE_TAG_SHIFT);
+
+	for (int j = 0; j < THREAD_ITERATIONS; j++) {
+		if (prctl(PR_SET_TAGGED_ADDR_CTRL, prctl_set, 0, 0, 0)) {
+			perror("prctl() failed");
+			goto fail;
+		}
+
+		prctl_get = prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0);
+
+		if (prctl_set != prctl_get) {
+			ksft_print_msg("Error: prctl_set: 0x%lx != prctl_get: 0x%lx\n",
+						prctl_set, prctl_get);
+			goto fail;
+		}
+	}
+
+	return (void *)KSFT_PASS;
+
+fail:
+	return (void *)KSFT_FAIL;
+}
+
+int execute_test(pid_t pid)
+{
+	pthread_t thread_id[MAX_THREADS];
+	int thread_data[MAX_THREADS];
+
+	for (int i = 0; i < MAX_THREADS; i++)
+		pthread_create(&thread_id[i], NULL,
+			       execute_thread, (void *)&pid);
+
+	for (int i = 0; i < MAX_THREADS; i++)
+		pthread_join(thread_id[i], (void *)&thread_data[i]);
+
+	for (int i = 0; i < MAX_THREADS; i++)
+		if (thread_data[i] == KSFT_FAIL)
+			return KSFT_FAIL;
+
+	return KSFT_PASS;
+}
+
+int mte_gcr_fork_test()
+{
+	pid_t pid[NUM_ITERATIONS];
+	int results[NUM_ITERATIONS];
+	pid_t cpid;
+	int res;
+
+	for (int i = 0; i < NUM_ITERATIONS; i++) {
+		pid[i] = fork();
+
+		if (pid[i] == 0) {
+			cpid = getpid();
+
+			res = execute_test(cpid);
+
+			exit(res);
+		}
+	}
+
+	for (int i = 0; i < NUM_ITERATIONS; i++) {
+		wait(&res);
+
+		if(WIFEXITED(res))
+			results[i] = WEXITSTATUS(res);
+		else
+			--i;
+	}
+
+	for (int i = 0; i < NUM_ITERATIONS; i++)
+		if (results[i] == KSFT_FAIL)
+			return KSFT_FAIL;
+
+	return KSFT_PASS;
+}
+
+int main(int argc, char *argv[])
+{
+	int err;
+
+	err = mte_default_setup();
+	if (err)
+		return err;
+
+	ksft_set_plan(1);
+
+	evaluate_test(mte_gcr_fork_test(),
+		"Verify that GCR_EL1 is set correctly on context switch\n");
+
+	mte_restore_setup();
+	ksft_print_cnts();
+
+	return ksft_get_fail_cnt() == 0 ? KSFT_PASS : KSFT_FAIL;
+}
+
-- 
2.29.2.222.g5d2a92d10f8-goog



^ permalink raw reply related	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64
  2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
                   ` (43 preceding siblings ...)
  2020-11-10 22:10 ` [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch Andrey Konovalov
@ 2020-11-10 22:54 ` Andrew Morton
  2020-11-11 14:35   ` Andrey Konovalov
  44 siblings, 1 reply; 107+ messages in thread
From: Andrew Morton @ 2020-11-10 22:54 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Alexander Potapenko, Marco Elver,
	Evgenii Stepanov, Branislav Rankov, Kevin Brodsky, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Tue, 10 Nov 2020 23:09:57 +0100 Andrey Konovalov <andreyknvl@google.com> wrote:

> This patchset adds a new hardware tag-based mode to KASAN [1]. The new mode
> is similar to the existing software tag-based KASAN, but relies on arm64
> Memory Tagging Extension (MTE) [2] to perform memory and pointer tagging
> (instead of shadow memory and compiler instrumentation).

I have that all merged up on top of linux-next.  Numerous minor
conflicts, mainly in arch/arm/Kconfig.  Also the changes in
https://lkml.kernel.org/r/20201103175841.3495947-7-elver@google.com had
to be fed into "kasan: split out shadow.c from common.c".

I staged it after linux-next to provide visibility into potentially
conflicting changes in the arm tree as things move forward.



^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers
  2020-11-10 22:09 ` [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers Andrey Konovalov
@ 2020-11-11 13:16   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:16 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Don't mention "GNU General Public License version 2" text explicitly,
> as it's already covered by the SPDX-License-Identifier.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: If0a2690042a2aa0fca70cea601ae9aabe72fa233
> ---
>  mm/kasan/common.c         |  5 -----
>  mm/kasan/generic.c        |  5 -----
>  mm/kasan/generic_report.c |  5 -----
>  mm/kasan/init.c           |  5 -----
>  mm/kasan/quarantine.c     | 10 ----------
>  mm/kasan/report.c         |  5 -----
>  mm/kasan/tags.c           |  5 -----
>  mm/kasan/tags_report.c    |  5 -----
>  8 files changed, 45 deletions(-)
>
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index 950fd372a07e..33d863f55db1 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -7,11 +7,6 @@
>   *
>   * Some code borrowed from https://github.com/xairy/kasan-prototype by
>   *        Andrey Konovalov <andreyknvl@gmail.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #include <linux/export.h>
> diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
> index 248264b9cb76..37ccfadd3263 100644
> --- a/mm/kasan/generic.c
> +++ b/mm/kasan/generic.c
> @@ -7,11 +7,6 @@
>   *
>   * Some code borrowed from https://github.com/xairy/kasan-prototype by
>   *        Andrey Konovalov <andreyknvl@gmail.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c
> index a38c7a9e192a..6bb3f66992df 100644
> --- a/mm/kasan/generic_report.c
> +++ b/mm/kasan/generic_report.c
> @@ -7,11 +7,6 @@
>   *
>   * Some code borrowed from https://github.com/xairy/kasan-prototype by
>   *        Andrey Konovalov <andreyknvl@gmail.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #include <linux/bitops.h>
> diff --git a/mm/kasan/init.c b/mm/kasan/init.c
> index fe6be0be1f76..9ce8cc5b8621 100644
> --- a/mm/kasan/init.c
> +++ b/mm/kasan/init.c
> @@ -4,11 +4,6 @@
>   *
>   * Copyright (c) 2015 Samsung Electronics Co., Ltd.
>   * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #include <linux/memblock.h>
> diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c
> index 4c5375810449..580ff5610fc1 100644
> --- a/mm/kasan/quarantine.c
> +++ b/mm/kasan/quarantine.c
> @@ -6,16 +6,6 @@
>   * Copyright (C) 2016 Google, Inc.
>   *
>   * Based on code by Dmitry Chernenkov.
> - *
> - * This program is free software; you can redistribute it and/or
> - * modify it under the terms of the GNU General Public License
> - * version 2 as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License for more details.
> - *
>   */
>
>  #include <linux/gfp.h>
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 00a53f1355ae..d500923abc8b 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -7,11 +7,6 @@
>   *
>   * Some code borrowed from https://github.com/xairy/kasan-prototype by
>   *        Andrey Konovalov <andreyknvl@gmail.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #include <linux/bitops.h>
> diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c
> index e02a36a51f42..5c8b08a25715 100644
> --- a/mm/kasan/tags.c
> +++ b/mm/kasan/tags.c
> @@ -4,11 +4,6 @@
>   *
>   * Copyright (c) 2018 Google, Inc.
>   * Author: Andrey Konovalov <andreyknvl@google.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c
> index bee43717d6f0..5f183501b871 100644
> --- a/mm/kasan/tags_report.c
> +++ b/mm/kasan/tags_report.c
> @@ -7,11 +7,6 @@
>   *
>   * Some code borrowed from https://github.com/xairy/kasan-prototype by
>   *        Andrey Konovalov <andreyknvl@gmail.com>
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - *
>   */
>
>  #include <linux/bitops.h>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC
  2020-11-10 22:09 ` [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrey Konovalov
@ 2020-11-11 13:19   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:19 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Currently only generic KASAN mode supports vmalloc, reflect that
> in the config.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I1889e5b3bed28cc5d607802fb6ae43ba461c0dc1
> ---
>  lib/Kconfig.kasan | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
> index 542a9c18398e..8f0742a0f23e 100644
> --- a/lib/Kconfig.kasan
> +++ b/lib/Kconfig.kasan
> @@ -155,7 +155,7 @@ config KASAN_SW_TAGS_IDENTIFY
>
>  config KASAN_VMALLOC
>         bool "Back mappings in vmalloc space with real shadow memory"
> -       depends on HAVE_ARCH_KASAN_VMALLOC
> +       depends on KASAN_GENERIC && HAVE_ARCH_KASAN_VMALLOC
>         help
>           By default, the shadow region for vmalloc space is the read-only
>           zero page. This means that KASAN cannot detect errors involving
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 03/44] kasan: group vmalloc code
  2020-11-10 22:10 ` [PATCH v9 03/44] kasan: group vmalloc code Andrey Konovalov
@ 2020-11-11 13:21   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:21 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Group all vmalloc-related function declarations in include/linux/kasan.h,
> and their implementations in mm/kasan/common.c.
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
> ---
> Change-Id: Ie20b6c689203cd6de4fd7f2c465ec081c00c5f15
> ---
>  include/linux/kasan.h | 41 +++++++++++++----------
>  mm/kasan/common.c     | 78 ++++++++++++++++++++++---------------------
>  2 files changed, 63 insertions(+), 56 deletions(-)
>
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index 30d343b4a40a..59538e795df4 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -75,19 +75,6 @@ struct kasan_cache {
>         int free_meta_offset;
>  };
>
> -/*
> - * These functions provide a special case to support backing module
> - * allocations with real shadow memory. With KASAN vmalloc, the special
> - * case is unnecessary, as the work is handled in the generic case.
> - */
> -#ifndef CONFIG_KASAN_VMALLOC
> -int kasan_module_alloc(void *addr, size_t size);
> -void kasan_free_shadow(const struct vm_struct *vm);
> -#else
> -static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
> -static inline void kasan_free_shadow(const struct vm_struct *vm) {}
> -#endif
> -
>  int kasan_add_zero_shadow(void *start, unsigned long size);
>  void kasan_remove_zero_shadow(void *start, unsigned long size);
>
> @@ -156,9 +143,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object,
>         return false;
>  }
>
> -static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
> -static inline void kasan_free_shadow(const struct vm_struct *vm) {}
> -
>  static inline int kasan_add_zero_shadow(void *start, unsigned long size)
>  {
>         return 0;
> @@ -211,13 +195,16 @@ static inline void *kasan_reset_tag(const void *addr)
>  #endif /* CONFIG_KASAN_SW_TAGS */
>
>  #ifdef CONFIG_KASAN_VMALLOC
> +
>  int kasan_populate_vmalloc(unsigned long addr, unsigned long size);
>  void kasan_poison_vmalloc(const void *start, unsigned long size);
>  void kasan_unpoison_vmalloc(const void *start, unsigned long size);
>  void kasan_release_vmalloc(unsigned long start, unsigned long end,
>                            unsigned long free_region_start,
>                            unsigned long free_region_end);
> -#else
> +
> +#else /* CONFIG_KASAN_VMALLOC */
> +
>  static inline int kasan_populate_vmalloc(unsigned long start,
>                                         unsigned long size)
>  {
> @@ -232,7 +219,25 @@ static inline void kasan_release_vmalloc(unsigned long start,
>                                          unsigned long end,
>                                          unsigned long free_region_start,
>                                          unsigned long free_region_end) {}
> -#endif
> +
> +#endif /* CONFIG_KASAN_VMALLOC */
> +
> +#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
> +
> +/*
> + * These functions provide a special case to support backing module
> + * allocations with real shadow memory. With KASAN vmalloc, the special
> + * case is unnecessary, as the work is handled in the generic case.
> + */
> +int kasan_module_alloc(void *addr, size_t size);
> +void kasan_free_shadow(const struct vm_struct *vm);
> +
> +#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
> +
> +static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
> +static inline void kasan_free_shadow(const struct vm_struct *vm) {}
> +
> +#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
>
>  #ifdef CONFIG_KASAN_INLINE
>  void kasan_non_canonical_hook(unsigned long addr);
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index 33d863f55db1..89e5ef9417a7 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -536,44 +536,6 @@ void kasan_kfree_large(void *ptr, unsigned long ip)
>         /* The object will be poisoned by page_alloc. */
>  }
>
> -#ifndef CONFIG_KASAN_VMALLOC
> -int kasan_module_alloc(void *addr, size_t size)
> -{
> -       void *ret;
> -       size_t scaled_size;
> -       size_t shadow_size;
> -       unsigned long shadow_start;
> -
> -       shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
> -       scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
> -       shadow_size = round_up(scaled_size, PAGE_SIZE);
> -
> -       if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
> -               return -EINVAL;
> -
> -       ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
> -                       shadow_start + shadow_size,
> -                       GFP_KERNEL,
> -                       PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
> -                       __builtin_return_address(0));
> -
> -       if (ret) {
> -               __memset(ret, KASAN_SHADOW_INIT, shadow_size);
> -               find_vm_area(addr)->flags |= VM_KASAN;
> -               kmemleak_ignore(ret);
> -               return 0;
> -       }
> -
> -       return -ENOMEM;
> -}
> -
> -void kasan_free_shadow(const struct vm_struct *vm)
> -{
> -       if (vm->flags & VM_KASAN)
> -               vfree(kasan_mem_to_shadow(vm->addr));
> -}
> -#endif
> -
>  #ifdef CONFIG_MEMORY_HOTPLUG
>  static bool shadow_mapped(unsigned long addr)
>  {
> @@ -685,6 +647,7 @@ core_initcall(kasan_memhotplug_init);
>  #endif
>
>  #ifdef CONFIG_KASAN_VMALLOC
> +
>  static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
>                                       void *unused)
>  {
> @@ -923,4 +886,43 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
>                                        (unsigned long)shadow_end);
>         }
>  }
> +
> +#else /* CONFIG_KASAN_VMALLOC */
> +
> +int kasan_module_alloc(void *addr, size_t size)
> +{
> +       void *ret;
> +       size_t scaled_size;
> +       size_t shadow_size;
> +       unsigned long shadow_start;
> +
> +       shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
> +       scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
> +       shadow_size = round_up(scaled_size, PAGE_SIZE);
> +
> +       if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
> +               return -EINVAL;
> +
> +       ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
> +                       shadow_start + shadow_size,
> +                       GFP_KERNEL,
> +                       PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
> +                       __builtin_return_address(0));
> +
> +       if (ret) {
> +               __memset(ret, KASAN_SHADOW_INIT, shadow_size);
> +               find_vm_area(addr)->flags |= VM_KASAN;
> +               kmemleak_ignore(ret);
> +               return 0;
> +       }
> +
> +       return -ENOMEM;
> +}
> +
> +void kasan_free_shadow(const struct vm_struct *vm)
> +{
> +       if (vm->flags & VM_KASAN)
> +               vfree(kasan_mem_to_shadow(vm->addr));
> +}
> +
>  #endif
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 05/44] kasan: shadow declarations only for software modes
  2020-11-10 22:10 ` [PATCH v9 05/44] kasan: shadow declarations only for software modes Andrey Konovalov
@ 2020-11-11 13:25   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:25 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Group shadow-related KASAN function declarations and only define them
> for the two existing software modes.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I864be75a88b91b443c55e9c2042865e15703e164
> ---
>  include/linux/kasan.h | 47 ++++++++++++++++++++++++++++---------------
>  1 file changed, 31 insertions(+), 16 deletions(-)
>
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index 59538e795df4..26f2ab92e7ca 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -11,7 +11,6 @@ struct task_struct;
>
>  #ifdef CONFIG_KASAN
>
> -#include <linux/pgtable.h>
>  #include <asm/kasan.h>
>
>  /* kasan_data struct is used in KUnit tests for KASAN expected failures */
> @@ -20,6 +19,20 @@ struct kunit_kasan_expectation {
>         bool report_found;
>  };
>
> +#endif
> +
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
> +
> +#include <linux/pgtable.h>
> +
> +/* Software KASAN implementations use shadow memory. */
> +
> +#ifdef CONFIG_KASAN_SW_TAGS
> +#define KASAN_SHADOW_INIT 0xFF
> +#else
> +#define KASAN_SHADOW_INIT 0
> +#endif
> +
>  extern unsigned char kasan_early_shadow_page[PAGE_SIZE];
>  extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE];
>  extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD];
> @@ -35,6 +48,23 @@ static inline void *kasan_mem_to_shadow(const void *addr)
>                 + KASAN_SHADOW_OFFSET;
>  }
>
> +int kasan_add_zero_shadow(void *start, unsigned long size);
> +void kasan_remove_zero_shadow(void *start, unsigned long size);
> +
> +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> +
> +static inline int kasan_add_zero_shadow(void *start, unsigned long size)
> +{
> +       return 0;
> +}
> +static inline void kasan_remove_zero_shadow(void *start,
> +                                       unsigned long size)
> +{}
> +
> +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> +
> +#ifdef CONFIG_KASAN
> +
>  /* Enable reporting bugs after kasan_disable_current() */
>  extern void kasan_enable_current(void);
>
> @@ -75,9 +105,6 @@ struct kasan_cache {
>         int free_meta_offset;
>  };
>
> -int kasan_add_zero_shadow(void *start, unsigned long size);
> -void kasan_remove_zero_shadow(void *start, unsigned long size);
> -
>  size_t __ksize(const void *);
>  static inline void kasan_unpoison_slab(const void *ptr)
>  {
> @@ -143,14 +170,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object,
>         return false;
>  }
>
> -static inline int kasan_add_zero_shadow(void *start, unsigned long size)
> -{
> -       return 0;
> -}
> -static inline void kasan_remove_zero_shadow(void *start,
> -                                       unsigned long size)
> -{}
> -
>  static inline void kasan_unpoison_slab(const void *ptr) { }
>  static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
>
> @@ -158,8 +177,6 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }
>
>  #ifdef CONFIG_KASAN_GENERIC
>
> -#define KASAN_SHADOW_INIT 0
> -
>  void kasan_cache_shrink(struct kmem_cache *cache);
>  void kasan_cache_shutdown(struct kmem_cache *cache);
>  void kasan_record_aux_stack(void *ptr);
> @@ -174,8 +191,6 @@ static inline void kasan_record_aux_stack(void *ptr) {}
>
>  #ifdef CONFIG_KASAN_SW_TAGS
>
> -#define KASAN_SHADOW_INIT 0xFF
> -
>  void kasan_init_tags(void);
>
>  void *kasan_reset_tag(const void *addr);
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory
  2020-11-10 22:10 ` [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory Andrey Konovalov
@ 2020-11-11 13:26   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:26 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> The new mode won't be using shadow memory, but will reuse the same
> functions. Rename kasan_unpoison_shadow to kasan_unpoison_memory,
> and kasan_poison_shadow to kasan_poison_memory.
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: Ia359f32815242c4704e49a5f1639ca2d2f8cba69
> ---
>  include/linux/kasan.h |  6 +++---
>  kernel/fork.c         |  4 ++--
>  mm/kasan/common.c     | 38 +++++++++++++++++++-------------------
>  mm/kasan/generic.c    | 12 ++++++------
>  mm/kasan/kasan.h      |  2 +-
>  mm/kasan/tags.c       |  2 +-
>  mm/slab_common.c      |  2 +-
>  7 files changed, 33 insertions(+), 33 deletions(-)
>
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index 26f2ab92e7ca..f6435b9f889c 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -71,7 +71,7 @@ extern void kasan_enable_current(void);
>  /* Disable reporting bugs for current task */
>  extern void kasan_disable_current(void);
>
> -void kasan_unpoison_shadow(const void *address, size_t size);
> +void kasan_unpoison_memory(const void *address, size_t size);
>
>  void kasan_unpoison_task_stack(struct task_struct *task);
>
> @@ -108,7 +108,7 @@ struct kasan_cache {
>  size_t __ksize(const void *);
>  static inline void kasan_unpoison_slab(const void *ptr)
>  {
> -       kasan_unpoison_shadow(ptr, __ksize(ptr));
> +       kasan_unpoison_memory(ptr, __ksize(ptr));
>  }
>  size_t kasan_metadata_size(struct kmem_cache *cache);
>
> @@ -117,7 +117,7 @@ void kasan_restore_multi_shot(bool enabled);
>
>  #else /* CONFIG_KASAN */
>
> -static inline void kasan_unpoison_shadow(const void *address, size_t size) {}
> +static inline void kasan_unpoison_memory(const void *address, size_t size) {}
>
>  static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
>
> diff --git a/kernel/fork.c b/kernel/fork.c
> index 6d266388d380..1c905e4290ab 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -225,8 +225,8 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node)
>                 if (!s)
>                         continue;
>
> -               /* Clear the KASAN shadow of the stack. */
> -               kasan_unpoison_shadow(s->addr, THREAD_SIZE);
> +               /* Mark stack accessible for KASAN. */
> +               kasan_unpoison_memory(s->addr, THREAD_SIZE);
>
>                 /* Clear stale pointers from reused stack. */
>                 memset(s->addr, 0, THREAD_SIZE);
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index 89e5ef9417a7..a4b73fa0dd7e 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -108,7 +108,7 @@ void *memcpy(void *dest, const void *src, size_t len)
>   * Poisons the shadow memory for 'size' bytes starting from 'addr'.
>   * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
>   */
> -void kasan_poison_shadow(const void *address, size_t size, u8 value)
> +void kasan_poison_memory(const void *address, size_t size, u8 value)
>  {
>         void *shadow_start, *shadow_end;
>
> @@ -125,7 +125,7 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value)
>         __memset(shadow_start, value, shadow_end - shadow_start);
>  }
>
> -void kasan_unpoison_shadow(const void *address, size_t size)
> +void kasan_unpoison_memory(const void *address, size_t size)
>  {
>         u8 tag = get_tag(address);
>
> @@ -136,7 +136,7 @@ void kasan_unpoison_shadow(const void *address, size_t size)
>          */
>         address = reset_tag(address);
>
> -       kasan_poison_shadow(address, size, tag);
> +       kasan_poison_memory(address, size, tag);
>
>         if (size & KASAN_SHADOW_MASK) {
>                 u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
> @@ -153,7 +153,7 @@ static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
>         void *base = task_stack_page(task);
>         size_t size = sp - base;
>
> -       kasan_unpoison_shadow(base, size);
> +       kasan_unpoison_memory(base, size);
>  }
>
>  /* Unpoison the entire stack for a task. */
> @@ -172,7 +172,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark)
>          */
>         void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1));
>
> -       kasan_unpoison_shadow(base, watermark - base);
> +       kasan_unpoison_memory(base, watermark - base);
>  }
>
>  void kasan_alloc_pages(struct page *page, unsigned int order)
> @@ -186,13 +186,13 @@ void kasan_alloc_pages(struct page *page, unsigned int order)
>         tag = random_tag();
>         for (i = 0; i < (1 << order); i++)
>                 page_kasan_tag_set(page + i, tag);
> -       kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
> +       kasan_unpoison_memory(page_address(page), PAGE_SIZE << order);
>  }
>
>  void kasan_free_pages(struct page *page, unsigned int order)
>  {
>         if (likely(!PageHighMem(page)))
> -               kasan_poison_shadow(page_address(page),
> +               kasan_poison_memory(page_address(page),
>                                 PAGE_SIZE << order,
>                                 KASAN_FREE_PAGE);
>  }
> @@ -284,18 +284,18 @@ void kasan_poison_slab(struct page *page)
>
>         for (i = 0; i < compound_nr(page); i++)
>                 page_kasan_tag_reset(page + i);
> -       kasan_poison_shadow(page_address(page), page_size(page),
> +       kasan_poison_memory(page_address(page), page_size(page),
>                         KASAN_KMALLOC_REDZONE);
>  }
>
>  void kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
>  {
> -       kasan_unpoison_shadow(object, cache->object_size);
> +       kasan_unpoison_memory(object, cache->object_size);
>  }
>
>  void kasan_poison_object_data(struct kmem_cache *cache, void *object)
>  {
> -       kasan_poison_shadow(object,
> +       kasan_poison_memory(object,
>                         round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
>                         KASAN_KMALLOC_REDZONE);
>  }
> @@ -408,7 +408,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
>         }
>
>         rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
> -       kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
> +       kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE);
>
>         if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
>                         unlikely(!(cache->flags & SLAB_KASAN)))
> @@ -448,8 +448,8 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
>                 tag = assign_tag(cache, object, false, keep_tag);
>
>         /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
> -       kasan_unpoison_shadow(set_tag(object, tag), size);
> -       kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
> +       kasan_unpoison_memory(set_tag(object, tag), size);
> +       kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start,
>                 KASAN_KMALLOC_REDZONE);
>
>         if (cache->flags & SLAB_KASAN)
> @@ -489,8 +489,8 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size,
>                                 KASAN_SHADOW_SCALE_SIZE);
>         redzone_end = (unsigned long)ptr + page_size(page);
>
> -       kasan_unpoison_shadow(ptr, size);
> -       kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
> +       kasan_unpoison_memory(ptr, size);
> +       kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start,
>                 KASAN_PAGE_REDZONE);
>
>         return (void *)ptr;
> @@ -523,7 +523,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip)
>                         kasan_report_invalid_free(ptr, ip);
>                         return;
>                 }
> -               kasan_poison_shadow(ptr, page_size(page), KASAN_FREE_PAGE);
> +               kasan_poison_memory(ptr, page_size(page), KASAN_FREE_PAGE);
>         } else {
>                 __kasan_slab_free(page->slab_cache, ptr, ip, false);
>         }
> @@ -709,7 +709,7 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
>          * // vmalloc() allocates memory
>          * // let a = area->addr
>          * // we reach kasan_populate_vmalloc
> -        * // and call kasan_unpoison_shadow:
> +        * // and call kasan_unpoison_memory:
>          * STORE shadow(a), unpoison_val
>          * ...
>          * STORE shadow(a+99), unpoison_val     x = LOAD p
> @@ -744,7 +744,7 @@ void kasan_poison_vmalloc(const void *start, unsigned long size)
>                 return;
>
>         size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
> -       kasan_poison_shadow(start, size, KASAN_VMALLOC_INVALID);
> +       kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
>  }
>
>  void kasan_unpoison_vmalloc(const void *start, unsigned long size)
> @@ -752,7 +752,7 @@ void kasan_unpoison_vmalloc(const void *start, unsigned long size)
>         if (!is_vmalloc_or_module_addr(start))
>                 return;
>
> -       kasan_unpoison_shadow(start, size);
> +       kasan_unpoison_memory(start, size);
>  }
>
>  static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
> diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
> index 37ccfadd3263..7006157c674b 100644
> --- a/mm/kasan/generic.c
> +++ b/mm/kasan/generic.c
> @@ -202,9 +202,9 @@ static void register_global(struct kasan_global *global)
>  {
>         size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
>
> -       kasan_unpoison_shadow(global->beg, global->size);
> +       kasan_unpoison_memory(global->beg, global->size);
>
> -       kasan_poison_shadow(global->beg + aligned_size,
> +       kasan_poison_memory(global->beg + aligned_size,
>                 global->size_with_redzone - aligned_size,
>                 KASAN_GLOBAL_REDZONE);
>  }
> @@ -285,11 +285,11 @@ void __asan_alloca_poison(unsigned long addr, size_t size)
>
>         WARN_ON(!IS_ALIGNED(addr, KASAN_ALLOCA_REDZONE_SIZE));
>
> -       kasan_unpoison_shadow((const void *)(addr + rounded_down_size),
> +       kasan_unpoison_memory((const void *)(addr + rounded_down_size),
>                               size - rounded_down_size);
> -       kasan_poison_shadow(left_redzone, KASAN_ALLOCA_REDZONE_SIZE,
> +       kasan_poison_memory(left_redzone, KASAN_ALLOCA_REDZONE_SIZE,
>                         KASAN_ALLOCA_LEFT);
> -       kasan_poison_shadow(right_redzone,
> +       kasan_poison_memory(right_redzone,
>                         padding_size + KASAN_ALLOCA_REDZONE_SIZE,
>                         KASAN_ALLOCA_RIGHT);
>  }
> @@ -301,7 +301,7 @@ void __asan_allocas_unpoison(const void *stack_top, const void *stack_bottom)
>         if (unlikely(!stack_top || stack_top > stack_bottom))
>                 return;
>
> -       kasan_unpoison_shadow(stack_top, stack_bottom - stack_top);
> +       kasan_unpoison_memory(stack_top, stack_bottom - stack_top);
>  }
>  EXPORT_SYMBOL(__asan_allocas_unpoison);
>
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index ac499456740f..03450d3b31f7 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -150,7 +150,7 @@ static inline bool addr_has_shadow(const void *addr)
>         return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
>  }
>
> -void kasan_poison_shadow(const void *address, size_t size, u8 value);
> +void kasan_poison_memory(const void *address, size_t size, u8 value);
>
>  /**
>   * check_memory_region - Check memory region, and report if invalid access.
> diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c
> index 5c8b08a25715..4bdd7dbd6647 100644
> --- a/mm/kasan/tags.c
> +++ b/mm/kasan/tags.c
> @@ -153,7 +153,7 @@ EXPORT_SYMBOL(__hwasan_storeN_noabort);
>
>  void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size)
>  {
> -       kasan_poison_shadow((void *)addr, size, tag);
> +       kasan_poison_memory((void *)addr, size, tag);
>  }
>  EXPORT_SYMBOL(__hwasan_tag_memory);
>
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index f9ccd5dc13f3..53d0f8bb57ea 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -1176,7 +1176,7 @@ size_t ksize(const void *objp)
>          * We assume that ksize callers could use whole allocated area,
>          * so we need to unpoison this area.
>          */
> -       kasan_unpoison_shadow(objp, size);
> +       kasan_unpoison_memory(objp, size);
>         return size;
>  }
>  EXPORT_SYMBOL(ksize);
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_*
  2020-11-10 22:10 ` [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrey Konovalov
@ 2020-11-11 13:45   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:45 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> The new mode won't be using shadow memory, but will still use the concept
> of memory granules. Each memory granule maps to a single metadata entry:
> 8 bytes per one shadow byte for generic mode, 16 bytes per one shadow byte
> for software tag-based mode, and 16 bytes per one allocation tag for
> hardware tag-based mode.
>
> Rename KASAN_SHADOW_SCALE_SIZE to KASAN_GRANULE_SIZE, and KASAN_SHADOW_MASK
> to KASAN_GRANULE_MASK.
>
> Also use MASK when used as a mask, otherwise use SIZE.
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: Iac733e2248aa9d29f6fc425d8946ba07cca73ecf
> ---
>  Documentation/dev-tools/kasan.rst |  2 +-
>  lib/test_kasan.c                  |  2 +-
>  lib/test_kasan_module.c           |  2 +-
>  mm/kasan/common.c                 | 39 ++++++++++++++++---------------
>  mm/kasan/generic.c                | 14 +++++------
>  mm/kasan/generic_report.c         |  8 +++----
>  mm/kasan/init.c                   |  8 +++----
>  mm/kasan/kasan.h                  |  4 ++--
>  mm/kasan/report.c                 | 10 ++++----
>  mm/kasan/tags_report.c            |  2 +-
>  10 files changed, 46 insertions(+), 45 deletions(-)
>
> diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst
> index 2b68addaadcd..edca4be5e405 100644
> --- a/Documentation/dev-tools/kasan.rst
> +++ b/Documentation/dev-tools/kasan.rst
> @@ -264,7 +264,7 @@ Most mappings in vmalloc space are small, requiring less than a full
>  page of shadow space. Allocating a full shadow page per mapping would
>  therefore be wasteful. Furthermore, to ensure that different mappings
>  use different shadow pages, mappings would have to be aligned to
> -``KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE``.
> +``KASAN_GRANULE_SIZE * PAGE_SIZE``.
>
>  Instead, we share backing space across multiple mappings. We allocate
>  a backing page when a mapping in vmalloc space uses a particular page
> diff --git a/lib/test_kasan.c b/lib/test_kasan.c
> index 662f862702fc..2947274cc2d3 100644
> --- a/lib/test_kasan.c
> +++ b/lib/test_kasan.c
> @@ -25,7 +25,7 @@
>
>  #include "../mm/kasan/kasan.h"
>
> -#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE)
> +#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE)
>
>  /*
>   * We assign some test results to these globals to make sure the tests
> diff --git a/lib/test_kasan_module.c b/lib/test_kasan_module.c
> index 2d68db6ae67b..fcb991c3aaf8 100644
> --- a/lib/test_kasan_module.c
> +++ b/lib/test_kasan_module.c
> @@ -15,7 +15,7 @@
>
>  #include "../mm/kasan/kasan.h"
>
> -#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE)
> +#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE)
>
>  static noinline void __init copy_user_test(void)
>  {
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index a4b73fa0dd7e..f65c9f792f8f 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -106,7 +106,7 @@ void *memcpy(void *dest, const void *src, size_t len)
>
>  /*
>   * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
> + * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
>   */
>  void kasan_poison_memory(const void *address, size_t size, u8 value)
>  {
> @@ -138,13 +138,13 @@ void kasan_unpoison_memory(const void *address, size_t size)
>
>         kasan_poison_memory(address, size, tag);
>
> -       if (size & KASAN_SHADOW_MASK) {
> +       if (size & KASAN_GRANULE_MASK) {
>                 u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
>
>                 if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
>                         *shadow = tag;
>                 else
> -                       *shadow = size & KASAN_SHADOW_MASK;
> +                       *shadow = size & KASAN_GRANULE_MASK;
>         }
>  }
>
> @@ -296,7 +296,7 @@ void kasan_unpoison_object_data(struct kmem_cache *cache, void *object)
>  void kasan_poison_object_data(struct kmem_cache *cache, void *object)
>  {
>         kasan_poison_memory(object,
> -                       round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
> +                       round_up(cache->object_size, KASAN_GRANULE_SIZE),
>                         KASAN_KMALLOC_REDZONE);
>  }
>
> @@ -368,7 +368,7 @@ static inline bool shadow_invalid(u8 tag, s8 shadow_byte)
>  {
>         if (IS_ENABLED(CONFIG_KASAN_GENERIC))
>                 return shadow_byte < 0 ||
> -                       shadow_byte >= KASAN_SHADOW_SCALE_SIZE;
> +                       shadow_byte >= KASAN_GRANULE_SIZE;
>
>         /* else CONFIG_KASAN_SW_TAGS: */
>         if ((u8)shadow_byte == KASAN_TAG_INVALID)
> @@ -407,7 +407,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
>                 return true;
>         }
>
> -       rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE);
> +       rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE);
>         kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE);
>
>         if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) ||
> @@ -440,9 +440,9 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
>                 return NULL;
>
>         redzone_start = round_up((unsigned long)(object + size),
> -                               KASAN_SHADOW_SCALE_SIZE);
> +                               KASAN_GRANULE_SIZE);
>         redzone_end = round_up((unsigned long)object + cache->object_size,
> -                               KASAN_SHADOW_SCALE_SIZE);
> +                               KASAN_GRANULE_SIZE);
>
>         if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
>                 tag = assign_tag(cache, object, false, keep_tag);
> @@ -486,7 +486,7 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size,
>
>         page = virt_to_page(ptr);
>         redzone_start = round_up((unsigned long)(ptr + size),
> -                               KASAN_SHADOW_SCALE_SIZE);
> +                               KASAN_GRANULE_SIZE);
>         redzone_end = (unsigned long)ptr + page_size(page);
>
>         kasan_unpoison_memory(ptr, size);
> @@ -584,8 +584,8 @@ static int __meminit kasan_mem_notifier(struct notifier_block *nb,
>         shadow_size = nr_shadow_pages << PAGE_SHIFT;
>         shadow_end = shadow_start + shadow_size;
>
> -       if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) ||
> -               WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT)))
> +       if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
> +               WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
>                 return NOTIFY_BAD;
>
>         switch (action) {
> @@ -743,7 +743,7 @@ void kasan_poison_vmalloc(const void *start, unsigned long size)
>         if (!is_vmalloc_or_module_addr(start))
>                 return;
>
> -       size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
> +       size = round_up(size, KASAN_GRANULE_SIZE);
>         kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
>  }
>
> @@ -856,22 +856,22 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
>         unsigned long region_start, region_end;
>         unsigned long size;
>
> -       region_start = ALIGN(start, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
> -       region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
> +       region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
> +       region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
>
>         free_region_start = ALIGN(free_region_start,
> -                                 PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
> +                                 PAGE_SIZE * KASAN_GRANULE_SIZE);
>
>         if (start != region_start &&
>             free_region_start < region_start)
> -               region_start -= PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE;
> +               region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
>
>         free_region_end = ALIGN_DOWN(free_region_end,
> -                                    PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE);
> +                                    PAGE_SIZE * KASAN_GRANULE_SIZE);
>
>         if (end != region_end &&
>             free_region_end > region_end)
> -               region_end += PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE;
> +               region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
>
>         shadow_start = kasan_mem_to_shadow((void *)region_start);
>         shadow_end = kasan_mem_to_shadow((void *)region_end);
> @@ -897,7 +897,8 @@ int kasan_module_alloc(void *addr, size_t size)
>         unsigned long shadow_start;
>
>         shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
> -       scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT;
> +       scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
> +                               KASAN_SHADOW_SCALE_SHIFT;
>         shadow_size = round_up(scaled_size, PAGE_SIZE);
>
>         if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
> diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
> index 7006157c674b..ec4417156943 100644
> --- a/mm/kasan/generic.c
> +++ b/mm/kasan/generic.c
> @@ -46,7 +46,7 @@ static __always_inline bool memory_is_poisoned_1(unsigned long addr)
>         s8 shadow_value = *(s8 *)kasan_mem_to_shadow((void *)addr);
>
>         if (unlikely(shadow_value)) {
> -               s8 last_accessible_byte = addr & KASAN_SHADOW_MASK;
> +               s8 last_accessible_byte = addr & KASAN_GRANULE_MASK;
>                 return unlikely(last_accessible_byte >= shadow_value);
>         }
>
> @@ -62,7 +62,7 @@ static __always_inline bool memory_is_poisoned_2_4_8(unsigned long addr,
>          * Access crosses 8(shadow size)-byte boundary. Such access maps
>          * into 2 shadow bytes, so we need to check them both.
>          */
> -       if (unlikely(((addr + size - 1) & KASAN_SHADOW_MASK) < size - 1))
> +       if (unlikely(((addr + size - 1) & KASAN_GRANULE_MASK) < size - 1))
>                 return *shadow_addr || memory_is_poisoned_1(addr + size - 1);
>
>         return memory_is_poisoned_1(addr + size - 1);
> @@ -73,7 +73,7 @@ static __always_inline bool memory_is_poisoned_16(unsigned long addr)
>         u16 *shadow_addr = (u16 *)kasan_mem_to_shadow((void *)addr);
>
>         /* Unaligned 16-bytes access maps into 3 shadow bytes. */
> -       if (unlikely(!IS_ALIGNED(addr, KASAN_SHADOW_SCALE_SIZE)))
> +       if (unlikely(!IS_ALIGNED(addr, KASAN_GRANULE_SIZE)))
>                 return *shadow_addr || memory_is_poisoned_1(addr + 15);
>
>         return *shadow_addr;
> @@ -134,7 +134,7 @@ static __always_inline bool memory_is_poisoned_n(unsigned long addr,
>                 s8 *last_shadow = (s8 *)kasan_mem_to_shadow((void *)last_byte);
>
>                 if (unlikely(ret != (unsigned long)last_shadow ||
> -                       ((long)(last_byte & KASAN_SHADOW_MASK) >= *last_shadow)))
> +                       ((long)(last_byte & KASAN_GRANULE_MASK) >= *last_shadow)))
>                         return true;
>         }
>         return false;
> @@ -200,7 +200,7 @@ void kasan_cache_shutdown(struct kmem_cache *cache)
>
>  static void register_global(struct kasan_global *global)
>  {
> -       size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);
> +       size_t aligned_size = round_up(global->size, KASAN_GRANULE_SIZE);
>
>         kasan_unpoison_memory(global->beg, global->size);
>
> @@ -274,10 +274,10 @@ EXPORT_SYMBOL(__asan_handle_no_return);
>  /* Emitted by compiler to poison alloca()ed objects. */
>  void __asan_alloca_poison(unsigned long addr, size_t size)
>  {
> -       size_t rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
> +       size_t rounded_up_size = round_up(size, KASAN_GRANULE_SIZE);
>         size_t padding_size = round_up(size, KASAN_ALLOCA_REDZONE_SIZE) -
>                         rounded_up_size;
> -       size_t rounded_down_size = round_down(size, KASAN_SHADOW_SCALE_SIZE);
> +       size_t rounded_down_size = round_down(size, KASAN_GRANULE_SIZE);
>
>         const void *left_redzone = (const void *)(addr -
>                         KASAN_ALLOCA_REDZONE_SIZE);
> diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c
> index 6bb3f66992df..7d5b9e5c7cfe 100644
> --- a/mm/kasan/generic_report.c
> +++ b/mm/kasan/generic_report.c
> @@ -34,7 +34,7 @@ void *find_first_bad_addr(void *addr, size_t size)
>         void *p = addr;
>
>         while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p)))
> -               p += KASAN_SHADOW_SCALE_SIZE;
> +               p += KASAN_GRANULE_SIZE;
>         return p;
>  }
>
> @@ -46,14 +46,14 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info)
>         shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr);
>
>         /*
> -        * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look
> +        * If shadow byte value is in [0, KASAN_GRANULE_SIZE) we can look
>          * at the next shadow byte to determine the type of the bad access.
>          */
> -       if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1)
> +       if (*shadow_addr > 0 && *shadow_addr <= KASAN_GRANULE_SIZE - 1)
>                 shadow_addr++;
>
>         switch (*shadow_addr) {
> -       case 0 ... KASAN_SHADOW_SCALE_SIZE - 1:
> +       case 0 ... KASAN_GRANULE_SIZE - 1:
>                 /*
>                  * In theory it's still possible to see these shadow values
>                  * due to a data race in the kernel code.
> diff --git a/mm/kasan/init.c b/mm/kasan/init.c
> index 9ce8cc5b8621..dfddd6c39fe6 100644
> --- a/mm/kasan/init.c
> +++ b/mm/kasan/init.c
> @@ -442,8 +442,8 @@ void kasan_remove_zero_shadow(void *start, unsigned long size)
>         end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT);
>
>         if (WARN_ON((unsigned long)start %
> -                       (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) ||
> -           WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)))
> +                       (KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
> +           WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
>                 return;
>
>         for (; addr < end; addr = next) {
> @@ -477,8 +477,8 @@ int kasan_add_zero_shadow(void *start, unsigned long size)
>         shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT);
>
>         if (WARN_ON((unsigned long)start %
> -                       (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) ||
> -           WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)))
> +                       (KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
> +           WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
>                 return -EINVAL;
>
>         ret = kasan_populate_early_shadow(shadow_start, shadow_end);
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index 03450d3b31f7..c31e2c739301 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -5,8 +5,8 @@
>  #include <linux/kasan.h>
>  #include <linux/stackdepot.h>
>
> -#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT)
> -#define KASAN_SHADOW_MASK       (KASAN_SHADOW_SCALE_SIZE - 1)
> +#define KASAN_GRANULE_SIZE     (1UL << KASAN_SHADOW_SCALE_SHIFT)
> +#define KASAN_GRANULE_MASK     (KASAN_GRANULE_SIZE - 1)
>
>  #define KASAN_TAG_KERNEL       0xFF /* native kernel pointers tag */
>  #define KASAN_TAG_INVALID      0xFE /* inaccessible memory tag */
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index d500923abc8b..7b8dcb799a78 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -314,24 +314,24 @@ static bool __must_check get_address_stack_frame_info(const void *addr,
>                 return false;
>
>         aligned_addr = round_down((unsigned long)addr, sizeof(long));
> -       mem_ptr = round_down(aligned_addr, KASAN_SHADOW_SCALE_SIZE);
> +       mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
>         shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
>         shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
>
>         while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
>                 shadow_ptr--;
> -               mem_ptr -= KASAN_SHADOW_SCALE_SIZE;
> +               mem_ptr -= KASAN_GRANULE_SIZE;
>         }
>
>         while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
>                 shadow_ptr--;
> -               mem_ptr -= KASAN_SHADOW_SCALE_SIZE;
> +               mem_ptr -= KASAN_GRANULE_SIZE;
>         }
>
>         if (shadow_ptr < shadow_bottom)
>                 return false;
>
> -       frame = (const unsigned long *)(mem_ptr + KASAN_SHADOW_SCALE_SIZE);
> +       frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
>         if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
>                 pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
>                        frame[0]);
> @@ -599,6 +599,6 @@ void kasan_non_canonical_hook(unsigned long addr)
>         else
>                 bug_type = "maybe wild-memory-access";
>         pr_alert("KASAN: %s in range [0x%016lx-0x%016lx]\n", bug_type,
> -                orig_addr, orig_addr + KASAN_SHADOW_MASK);
> +                orig_addr, orig_addr + KASAN_GRANULE_SIZE - 1);
>  }
>  #endif
> diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c
> index 5f183501b871..c87d5a343b4e 100644
> --- a/mm/kasan/tags_report.c
> +++ b/mm/kasan/tags_report.c
> @@ -76,7 +76,7 @@ void *find_first_bad_addr(void *addr, size_t size)
>         void *end = p + size;
>
>         while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p))
> -               p += KASAN_SHADOW_SCALE_SIZE;
> +               p += KASAN_GRANULE_SIZE;
>         return p;
>  }
>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 08/44] kasan: only build init.c for software modes
  2020-11-10 22:10 ` [PATCH v9 08/44] kasan: only build init.c for software modes Andrey Konovalov
@ 2020-11-11 13:46   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 13:46 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> The new mode won't be using shadow memory, so only build init.c that
> contains shadow initialization code for software modes.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I8d68c47345afc1dbedadde738f34a874dcae5080
> ---
>  mm/kasan/Makefile | 6 +++---
>  mm/kasan/init.c   | 2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
> index 370d970e5ab5..7cf685bb51bd 100644
> --- a/mm/kasan/Makefile
> +++ b/mm/kasan/Makefile
> @@ -29,6 +29,6 @@ CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
>
> -obj-$(CONFIG_KASAN) := common.o init.o report.o
> -obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o
> -obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o
> +obj-$(CONFIG_KASAN) := common.o report.o
> +obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o
> +obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o
> diff --git a/mm/kasan/init.c b/mm/kasan/init.c
> index dfddd6c39fe6..1a71eaa8c5f9 100644
> --- a/mm/kasan/init.c
> +++ b/mm/kasan/init.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * This file contains some kasan initialization code.
> + * This file contains KASAN shadow initialization code.
>   *
>   * Copyright (c) 2015 Samsung Electronics Co., Ltd.
>   * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 09/44] kasan: split out shadow.c from common.c
  2020-11-10 22:10 ` [PATCH v9 09/44] kasan: split out shadow.c from common.c Andrey Konovalov
@ 2020-11-11 14:07   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:07 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> The new mode won't be using shadow memory. Move all shadow-related code
> to shadow.c, which is only enabled for software KASAN modes that use
> shadow memory.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: Ic1c32ce72d4649848e9e6a1f2c8dd269c77673f2
> ---
>  mm/kasan/Makefile |   6 +-
>  mm/kasan/common.c | 486 +-------------------------------------------
>  mm/kasan/shadow.c | 505 ++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 510 insertions(+), 487 deletions(-)
>  create mode 100644 mm/kasan/shadow.c
>
> diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
> index 7cf685bb51bd..7cc1031e1ef8 100644
> --- a/mm/kasan/Makefile
> +++ b/mm/kasan/Makefile
> @@ -10,6 +10,7 @@ CFLAGS_REMOVE_generic_report.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE)
>
> @@ -26,9 +27,10 @@ CFLAGS_generic_report.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
> +CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
>
>  obj-$(CONFIG_KASAN) := common.o report.o
> -obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o
> -obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o
> +obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o
> +obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index f65c9f792f8f..123abfb760d4 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * This file contains common generic and tag-based KASAN code.
> + * This file contains common KASAN code.
>   *
>   * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>   * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> @@ -13,7 +13,6 @@
>  #include <linux/init.h>
>  #include <linux/kasan.h>
>  #include <linux/kernel.h>
> -#include <linux/kmemleak.h>
>  #include <linux/linkage.h>
>  #include <linux/memblock.h>
>  #include <linux/memory.h>
> @@ -26,12 +25,8 @@
>  #include <linux/stacktrace.h>
>  #include <linux/string.h>
>  #include <linux/types.h>
> -#include <linux/vmalloc.h>
>  #include <linux/bug.h>
>
> -#include <asm/cacheflush.h>
> -#include <asm/tlbflush.h>
> -
>  #include "kasan.h"
>  #include "../slab.h"
>
> @@ -61,93 +56,6 @@ void kasan_disable_current(void)
>         current->kasan_depth--;
>  }
>
> -bool __kasan_check_read(const volatile void *p, unsigned int size)
> -{
> -       return check_memory_region((unsigned long)p, size, false, _RET_IP_);
> -}
> -EXPORT_SYMBOL(__kasan_check_read);
> -
> -bool __kasan_check_write(const volatile void *p, unsigned int size)
> -{
> -       return check_memory_region((unsigned long)p, size, true, _RET_IP_);
> -}
> -EXPORT_SYMBOL(__kasan_check_write);
> -
> -#undef memset
> -void *memset(void *addr, int c, size_t len)
> -{
> -       if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_))
> -               return NULL;
> -
> -       return __memset(addr, c, len);
> -}
> -
> -#ifdef __HAVE_ARCH_MEMMOVE
> -#undef memmove
> -void *memmove(void *dest, const void *src, size_t len)
> -{
> -       if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
> -           !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
> -               return NULL;
> -
> -       return __memmove(dest, src, len);
> -}
> -#endif
> -
> -#undef memcpy
> -void *memcpy(void *dest, const void *src, size_t len)
> -{
> -       if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
> -           !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
> -               return NULL;
> -
> -       return __memcpy(dest, src, len);
> -}
> -
> -/*
> - * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> - * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> - */
> -void kasan_poison_memory(const void *address, size_t size, u8 value)
> -{
> -       void *shadow_start, *shadow_end;
> -
> -       /*
> -        * Perform shadow offset calculation based on untagged address, as
> -        * some of the callers (e.g. kasan_poison_object_data) pass tagged
> -        * addresses to this function.
> -        */
> -       address = reset_tag(address);
> -
> -       shadow_start = kasan_mem_to_shadow(address);
> -       shadow_end = kasan_mem_to_shadow(address + size);
> -
> -       __memset(shadow_start, value, shadow_end - shadow_start);
> -}
> -
> -void kasan_unpoison_memory(const void *address, size_t size)
> -{
> -       u8 tag = get_tag(address);
> -
> -       /*
> -        * Perform shadow offset calculation based on untagged address, as
> -        * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
> -        * addresses to this function.
> -        */
> -       address = reset_tag(address);
> -
> -       kasan_poison_memory(address, size, tag);
> -
> -       if (size & KASAN_GRANULE_MASK) {
> -               u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
> -
> -               if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
> -                       *shadow = tag;
> -               else
> -                       *shadow = size & KASAN_GRANULE_MASK;
> -       }
> -}
> -
>  static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
>  {
>         void *base = task_stack_page(task);
> @@ -535,395 +443,3 @@ void kasan_kfree_large(void *ptr, unsigned long ip)
>                 kasan_report_invalid_free(ptr, ip);
>         /* The object will be poisoned by page_alloc. */
>  }
> -
> -#ifdef CONFIG_MEMORY_HOTPLUG
> -static bool shadow_mapped(unsigned long addr)
> -{
> -       pgd_t *pgd = pgd_offset_k(addr);
> -       p4d_t *p4d;
> -       pud_t *pud;
> -       pmd_t *pmd;
> -       pte_t *pte;
> -
> -       if (pgd_none(*pgd))
> -               return false;
> -       p4d = p4d_offset(pgd, addr);
> -       if (p4d_none(*p4d))
> -               return false;
> -       pud = pud_offset(p4d, addr);
> -       if (pud_none(*pud))
> -               return false;
> -
> -       /*
> -        * We can't use pud_large() or pud_huge(), the first one is
> -        * arch-specific, the last one depends on HUGETLB_PAGE.  So let's abuse
> -        * pud_bad(), if pud is bad then it's bad because it's huge.
> -        */
> -       if (pud_bad(*pud))
> -               return true;
> -       pmd = pmd_offset(pud, addr);
> -       if (pmd_none(*pmd))
> -               return false;
> -
> -       if (pmd_bad(*pmd))
> -               return true;
> -       pte = pte_offset_kernel(pmd, addr);
> -       return !pte_none(*pte);
> -}
> -
> -static int __meminit kasan_mem_notifier(struct notifier_block *nb,
> -                       unsigned long action, void *data)
> -{
> -       struct memory_notify *mem_data = data;
> -       unsigned long nr_shadow_pages, start_kaddr, shadow_start;
> -       unsigned long shadow_end, shadow_size;
> -
> -       nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT;
> -       start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn);
> -       shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr);
> -       shadow_size = nr_shadow_pages << PAGE_SHIFT;
> -       shadow_end = shadow_start + shadow_size;
> -
> -       if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
> -               WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
> -               return NOTIFY_BAD;
> -
> -       switch (action) {
> -       case MEM_GOING_ONLINE: {
> -               void *ret;
> -
> -               /*
> -                * If shadow is mapped already than it must have been mapped
> -                * during the boot. This could happen if we onlining previously
> -                * offlined memory.
> -                */
> -               if (shadow_mapped(shadow_start))
> -                       return NOTIFY_OK;
> -
> -               ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start,
> -                                       shadow_end, GFP_KERNEL,
> -                                       PAGE_KERNEL, VM_NO_GUARD,
> -                                       pfn_to_nid(mem_data->start_pfn),
> -                                       __builtin_return_address(0));
> -               if (!ret)
> -                       return NOTIFY_BAD;
> -
> -               kmemleak_ignore(ret);
> -               return NOTIFY_OK;
> -       }
> -       case MEM_CANCEL_ONLINE:
> -       case MEM_OFFLINE: {
> -               struct vm_struct *vm;
> -
> -               /*
> -                * shadow_start was either mapped during boot by kasan_init()
> -                * or during memory online by __vmalloc_node_range().
> -                * In the latter case we can use vfree() to free shadow.
> -                * Non-NULL result of the find_vm_area() will tell us if
> -                * that was the second case.
> -                *
> -                * Currently it's not possible to free shadow mapped
> -                * during boot by kasan_init(). It's because the code
> -                * to do that hasn't been written yet. So we'll just
> -                * leak the memory.
> -                */
> -               vm = find_vm_area((void *)shadow_start);
> -               if (vm)
> -                       vfree((void *)shadow_start);
> -       }
> -       }
> -
> -       return NOTIFY_OK;
> -}
> -
> -static int __init kasan_memhotplug_init(void)
> -{
> -       hotplug_memory_notifier(kasan_mem_notifier, 0);
> -
> -       return 0;
> -}
> -
> -core_initcall(kasan_memhotplug_init);
> -#endif
> -
> -#ifdef CONFIG_KASAN_VMALLOC
> -
> -static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
> -                                     void *unused)
> -{
> -       unsigned long page;
> -       pte_t pte;
> -
> -       if (likely(!pte_none(*ptep)))
> -               return 0;
> -
> -       page = __get_free_page(GFP_KERNEL);
> -       if (!page)
> -               return -ENOMEM;
> -
> -       memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
> -       pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
> -
> -       spin_lock(&init_mm.page_table_lock);
> -       if (likely(pte_none(*ptep))) {
> -               set_pte_at(&init_mm, addr, ptep, pte);
> -               page = 0;
> -       }
> -       spin_unlock(&init_mm.page_table_lock);
> -       if (page)
> -               free_page(page);
> -       return 0;
> -}
> -
> -int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
> -{
> -       unsigned long shadow_start, shadow_end;
> -       int ret;
> -
> -       if (!is_vmalloc_or_module_addr((void *)addr))
> -               return 0;
> -
> -       shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
> -       shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
> -       shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
> -       shadow_end = ALIGN(shadow_end, PAGE_SIZE);
> -
> -       ret = apply_to_page_range(&init_mm, shadow_start,
> -                                 shadow_end - shadow_start,
> -                                 kasan_populate_vmalloc_pte, NULL);
> -       if (ret)
> -               return ret;
> -
> -       flush_cache_vmap(shadow_start, shadow_end);
> -
> -       /*
> -        * We need to be careful about inter-cpu effects here. Consider:
> -        *
> -        *   CPU#0                                CPU#1
> -        * WRITE_ONCE(p, vmalloc(100));         while (x = READ_ONCE(p)) ;
> -        *                                      p[99] = 1;
> -        *
> -        * With compiler instrumentation, that ends up looking like this:
> -        *
> -        *   CPU#0                                CPU#1
> -        * // vmalloc() allocates memory
> -        * // let a = area->addr
> -        * // we reach kasan_populate_vmalloc
> -        * // and call kasan_unpoison_memory:
> -        * STORE shadow(a), unpoison_val
> -        * ...
> -        * STORE shadow(a+99), unpoison_val     x = LOAD p
> -        * // rest of vmalloc process           <data dependency>
> -        * STORE p, a                           LOAD shadow(x+99)
> -        *
> -        * If there is no barrier between the end of unpoisioning the shadow
> -        * and the store of the result to p, the stores could be committed
> -        * in a different order by CPU#0, and CPU#1 could erroneously observe
> -        * poison in the shadow.
> -        *
> -        * We need some sort of barrier between the stores.
> -        *
> -        * In the vmalloc() case, this is provided by a smp_wmb() in
> -        * clear_vm_uninitialized_flag(). In the per-cpu allocator and in
> -        * get_vm_area() and friends, the caller gets shadow allocated but
> -        * doesn't have any pages mapped into the virtual address space that
> -        * has been reserved. Mapping those pages in will involve taking and
> -        * releasing a page-table lock, which will provide the barrier.
> -        */
> -
> -       return 0;
> -}
> -
> -/*
> - * Poison the shadow for a vmalloc region. Called as part of the
> - * freeing process at the time the region is freed.
> - */
> -void kasan_poison_vmalloc(const void *start, unsigned long size)
> -{
> -       if (!is_vmalloc_or_module_addr(start))
> -               return;
> -
> -       size = round_up(size, KASAN_GRANULE_SIZE);
> -       kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
> -}
> -
> -void kasan_unpoison_vmalloc(const void *start, unsigned long size)
> -{
> -       if (!is_vmalloc_or_module_addr(start))
> -               return;
> -
> -       kasan_unpoison_memory(start, size);
> -}
> -
> -static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
> -                                       void *unused)
> -{
> -       unsigned long page;
> -
> -       page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT);
> -
> -       spin_lock(&init_mm.page_table_lock);
> -
> -       if (likely(!pte_none(*ptep))) {
> -               pte_clear(&init_mm, addr, ptep);
> -               free_page(page);
> -       }
> -       spin_unlock(&init_mm.page_table_lock);
> -
> -       return 0;
> -}
> -
> -/*
> - * Release the backing for the vmalloc region [start, end), which
> - * lies within the free region [free_region_start, free_region_end).
> - *
> - * This can be run lazily, long after the region was freed. It runs
> - * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap
> - * infrastructure.
> - *
> - * How does this work?
> - * -------------------
> - *
> - * We have a region that is page aligned, labelled as A.
> - * That might not map onto the shadow in a way that is page-aligned:
> - *
> - *                    start                     end
> - *                    v                         v
> - * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc
> - *  -------- -------- --------          -------- --------
> - *      |        |       |                 |        |
> - *      |        |       |         /-------/        |
> - *      \-------\|/------/         |/---------------/
> - *              |||                ||
> - *             |??AAAAAA|AAAAAAAA|AA??????|                < shadow
> - *                 (1)      (2)      (3)
> - *
> - * First we align the start upwards and the end downwards, so that the
> - * shadow of the region aligns with shadow page boundaries. In the
> - * example, this gives us the shadow page (2). This is the shadow entirely
> - * covered by this allocation.
> - *
> - * Then we have the tricky bits. We want to know if we can free the
> - * partially covered shadow pages - (1) and (3) in the example. For this,
> - * we are given the start and end of the free region that contains this
> - * allocation. Extending our previous example, we could have:
> - *
> - *  free_region_start                                    free_region_end
> - *  |                 start                     end      |
> - *  v                 v                         v        v
> - * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc
> - *  -------- -------- --------          -------- --------
> - *      |        |       |                 |        |
> - *      |        |       |         /-------/        |
> - *      \-------\|/------/         |/---------------/
> - *              |||                ||
> - *             |FFAAAAAA|AAAAAAAA|AAF?????|                < shadow
> - *                 (1)      (2)      (3)
> - *
> - * Once again, we align the start of the free region up, and the end of
> - * the free region down so that the shadow is page aligned. So we can free
> - * page (1) - we know no allocation currently uses anything in that page,
> - * because all of it is in the vmalloc free region. But we cannot free
> - * page (3), because we can't be sure that the rest of it is unused.
> - *
> - * We only consider pages that contain part of the original region for
> - * freeing: we don't try to free other pages from the free region or we'd
> - * end up trying to free huge chunks of virtual address space.
> - *
> - * Concurrency
> - * -----------
> - *
> - * How do we know that we're not freeing a page that is simultaneously
> - * being used for a fresh allocation in kasan_populate_vmalloc(_pte)?
> - *
> - * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running
> - * at the same time. While we run under free_vmap_area_lock, the population
> - * code does not.
> - *
> - * free_vmap_area_lock instead operates to ensure that the larger range
> - * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and
> - * the per-cpu region-finding algorithm both run under free_vmap_area_lock,
> - * no space identified as free will become used while we are running. This
> - * means that so long as we are careful with alignment and only free shadow
> - * pages entirely covered by the free region, we will not run in to any
> - * trouble - any simultaneous allocations will be for disjoint regions.
> - */
> -void kasan_release_vmalloc(unsigned long start, unsigned long end,
> -                          unsigned long free_region_start,
> -                          unsigned long free_region_end)
> -{
> -       void *shadow_start, *shadow_end;
> -       unsigned long region_start, region_end;
> -       unsigned long size;
> -
> -       region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
> -       region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
> -
> -       free_region_start = ALIGN(free_region_start,
> -                                 PAGE_SIZE * KASAN_GRANULE_SIZE);
> -
> -       if (start != region_start &&
> -           free_region_start < region_start)
> -               region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
> -
> -       free_region_end = ALIGN_DOWN(free_region_end,
> -                                    PAGE_SIZE * KASAN_GRANULE_SIZE);
> -
> -       if (end != region_end &&
> -           free_region_end > region_end)
> -               region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
> -
> -       shadow_start = kasan_mem_to_shadow((void *)region_start);
> -       shadow_end = kasan_mem_to_shadow((void *)region_end);
> -
> -       if (shadow_end > shadow_start) {
> -               size = shadow_end - shadow_start;
> -               apply_to_existing_page_range(&init_mm,
> -                                            (unsigned long)shadow_start,
> -                                            size, kasan_depopulate_vmalloc_pte,
> -                                            NULL);
> -               flush_tlb_kernel_range((unsigned long)shadow_start,
> -                                      (unsigned long)shadow_end);
> -       }
> -}
> -
> -#else /* CONFIG_KASAN_VMALLOC */
> -
> -int kasan_module_alloc(void *addr, size_t size)
> -{
> -       void *ret;
> -       size_t scaled_size;
> -       size_t shadow_size;
> -       unsigned long shadow_start;
> -
> -       shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
> -       scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
> -                               KASAN_SHADOW_SCALE_SHIFT;
> -       shadow_size = round_up(scaled_size, PAGE_SIZE);
> -
> -       if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
> -               return -EINVAL;
> -
> -       ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
> -                       shadow_start + shadow_size,
> -                       GFP_KERNEL,
> -                       PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
> -                       __builtin_return_address(0));
> -
> -       if (ret) {
> -               __memset(ret, KASAN_SHADOW_INIT, shadow_size);
> -               find_vm_area(addr)->flags |= VM_KASAN;
> -               kmemleak_ignore(ret);
> -               return 0;
> -       }
> -
> -       return -ENOMEM;
> -}
> -
> -void kasan_free_shadow(const struct vm_struct *vm)
> -{
> -       if (vm->flags & VM_KASAN)
> -               vfree(kasan_mem_to_shadow(vm->addr));
> -}
> -
> -#endif
> diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
> new file mode 100644
> index 000000000000..ca0cc4c31454
> --- /dev/null
> +++ b/mm/kasan/shadow.c
> @@ -0,0 +1,505 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * This file contains KASAN runtime code that manages shadow memory for
> + * generic and software tag-based KASAN modes.
> + *
> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> + * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> + *
> + * Some code borrowed from https://github.com/xairy/kasan-prototype by
> + *        Andrey Konovalov <andreyknvl@gmail.com>
> + */
> +
> +#include <linux/init.h>
> +#include <linux/kasan.h>
> +#include <linux/kernel.h>
> +#include <linux/kmemleak.h>
> +#include <linux/memory.h>
> +#include <linux/mm.h>
> +#include <linux/string.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +#include <asm/cacheflush.h>
> +#include <asm/tlbflush.h>
> +
> +#include "kasan.h"
> +
> +bool __kasan_check_read(const volatile void *p, unsigned int size)
> +{
> +       return check_memory_region((unsigned long)p, size, false, _RET_IP_);
> +}
> +EXPORT_SYMBOL(__kasan_check_read);
> +
> +bool __kasan_check_write(const volatile void *p, unsigned int size)
> +{
> +       return check_memory_region((unsigned long)p, size, true, _RET_IP_);
> +}
> +EXPORT_SYMBOL(__kasan_check_write);
> +
> +#undef memset
> +void *memset(void *addr, int c, size_t len)
> +{
> +       if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_))
> +               return NULL;
> +
> +       return __memset(addr, c, len);
> +}
> +
> +#ifdef __HAVE_ARCH_MEMMOVE
> +#undef memmove
> +void *memmove(void *dest, const void *src, size_t len)
> +{
> +       if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
> +           !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
> +               return NULL;
> +
> +       return __memmove(dest, src, len);
> +}
> +#endif
> +
> +#undef memcpy
> +void *memcpy(void *dest, const void *src, size_t len)
> +{
> +       if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) ||
> +           !check_memory_region((unsigned long)dest, len, true, _RET_IP_))
> +               return NULL;
> +
> +       return __memcpy(dest, src, len);
> +}
> +
> +/*
> + * Poisons the shadow memory for 'size' bytes starting from 'addr'.
> + * Memory addresses should be aligned to KASAN_GRANULE_SIZE.
> + */
> +void kasan_poison_memory(const void *address, size_t size, u8 value)
> +{
> +       void *shadow_start, *shadow_end;
> +
> +       /*
> +        * Perform shadow offset calculation based on untagged address, as
> +        * some of the callers (e.g. kasan_poison_object_data) pass tagged
> +        * addresses to this function.
> +        */
> +       address = reset_tag(address);
> +
> +       shadow_start = kasan_mem_to_shadow(address);
> +       shadow_end = kasan_mem_to_shadow(address + size);
> +
> +       __memset(shadow_start, value, shadow_end - shadow_start);
> +}
> +
> +void kasan_unpoison_memory(const void *address, size_t size)
> +{
> +       u8 tag = get_tag(address);
> +
> +       /*
> +        * Perform shadow offset calculation based on untagged address, as
> +        * some of the callers (e.g. kasan_unpoison_object_data) pass tagged
> +        * addresses to this function.
> +        */
> +       address = reset_tag(address);
> +
> +       kasan_poison_memory(address, size, tag);
> +
> +       if (size & KASAN_GRANULE_MASK) {
> +               u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
> +
> +               if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
> +                       *shadow = tag;
> +               else
> +                       *shadow = size & KASAN_GRANULE_MASK;
> +       }
> +}
> +
> +#ifdef CONFIG_MEMORY_HOTPLUG
> +static bool shadow_mapped(unsigned long addr)
> +{
> +       pgd_t *pgd = pgd_offset_k(addr);
> +       p4d_t *p4d;
> +       pud_t *pud;
> +       pmd_t *pmd;
> +       pte_t *pte;
> +
> +       if (pgd_none(*pgd))
> +               return false;
> +       p4d = p4d_offset(pgd, addr);
> +       if (p4d_none(*p4d))
> +               return false;
> +       pud = pud_offset(p4d, addr);
> +       if (pud_none(*pud))
> +               return false;
> +
> +       /*
> +        * We can't use pud_large() or pud_huge(), the first one is
> +        * arch-specific, the last one depends on HUGETLB_PAGE.  So let's abuse
> +        * pud_bad(), if pud is bad then it's bad because it's huge.
> +        */
> +       if (pud_bad(*pud))
> +               return true;
> +       pmd = pmd_offset(pud, addr);
> +       if (pmd_none(*pmd))
> +               return false;
> +
> +       if (pmd_bad(*pmd))
> +               return true;
> +       pte = pte_offset_kernel(pmd, addr);
> +       return !pte_none(*pte);
> +}
> +
> +static int __meminit kasan_mem_notifier(struct notifier_block *nb,
> +                       unsigned long action, void *data)
> +{
> +       struct memory_notify *mem_data = data;
> +       unsigned long nr_shadow_pages, start_kaddr, shadow_start;
> +       unsigned long shadow_end, shadow_size;
> +
> +       nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT;
> +       start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn);
> +       shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr);
> +       shadow_size = nr_shadow_pages << PAGE_SHIFT;
> +       shadow_end = shadow_start + shadow_size;
> +
> +       if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
> +               WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
> +               return NOTIFY_BAD;
> +
> +       switch (action) {
> +       case MEM_GOING_ONLINE: {
> +               void *ret;
> +
> +               /*
> +                * If shadow is mapped already than it must have been mapped
> +                * during the boot. This could happen if we onlining previously
> +                * offlined memory.
> +                */
> +               if (shadow_mapped(shadow_start))
> +                       return NOTIFY_OK;
> +
> +               ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start,
> +                                       shadow_end, GFP_KERNEL,
> +                                       PAGE_KERNEL, VM_NO_GUARD,
> +                                       pfn_to_nid(mem_data->start_pfn),
> +                                       __builtin_return_address(0));
> +               if (!ret)
> +                       return NOTIFY_BAD;
> +
> +               kmemleak_ignore(ret);
> +               return NOTIFY_OK;
> +       }
> +       case MEM_CANCEL_ONLINE:
> +       case MEM_OFFLINE: {
> +               struct vm_struct *vm;
> +
> +               /*
> +                * shadow_start was either mapped during boot by kasan_init()
> +                * or during memory online by __vmalloc_node_range().
> +                * In the latter case we can use vfree() to free shadow.
> +                * Non-NULL result of the find_vm_area() will tell us if
> +                * that was the second case.
> +                *
> +                * Currently it's not possible to free shadow mapped
> +                * during boot by kasan_init(). It's because the code
> +                * to do that hasn't been written yet. So we'll just
> +                * leak the memory.
> +                */
> +               vm = find_vm_area((void *)shadow_start);
> +               if (vm)
> +                       vfree((void *)shadow_start);
> +       }
> +       }
> +
> +       return NOTIFY_OK;
> +}
> +
> +static int __init kasan_memhotplug_init(void)
> +{
> +       hotplug_memory_notifier(kasan_mem_notifier, 0);
> +
> +       return 0;
> +}
> +
> +core_initcall(kasan_memhotplug_init);
> +#endif
> +
> +#ifdef CONFIG_KASAN_VMALLOC
> +
> +static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
> +                                     void *unused)
> +{
> +       unsigned long page;
> +       pte_t pte;
> +
> +       if (likely(!pte_none(*ptep)))
> +               return 0;
> +
> +       page = __get_free_page(GFP_KERNEL);
> +       if (!page)
> +               return -ENOMEM;
> +
> +       memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
> +       pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
> +
> +       spin_lock(&init_mm.page_table_lock);
> +       if (likely(pte_none(*ptep))) {
> +               set_pte_at(&init_mm, addr, ptep, pte);
> +               page = 0;
> +       }
> +       spin_unlock(&init_mm.page_table_lock);
> +       if (page)
> +               free_page(page);
> +       return 0;
> +}
> +
> +int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
> +{
> +       unsigned long shadow_start, shadow_end;
> +       int ret;
> +
> +       if (!is_vmalloc_or_module_addr((void *)addr))
> +               return 0;
> +
> +       shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
> +       shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
> +       shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
> +       shadow_end = ALIGN(shadow_end, PAGE_SIZE);
> +
> +       ret = apply_to_page_range(&init_mm, shadow_start,
> +                                 shadow_end - shadow_start,
> +                                 kasan_populate_vmalloc_pte, NULL);
> +       if (ret)
> +               return ret;
> +
> +       flush_cache_vmap(shadow_start, shadow_end);
> +
> +       /*
> +        * We need to be careful about inter-cpu effects here. Consider:
> +        *
> +        *   CPU#0                                CPU#1
> +        * WRITE_ONCE(p, vmalloc(100));         while (x = READ_ONCE(p)) ;
> +        *                                      p[99] = 1;
> +        *
> +        * With compiler instrumentation, that ends up looking like this:
> +        *
> +        *   CPU#0                                CPU#1
> +        * // vmalloc() allocates memory
> +        * // let a = area->addr
> +        * // we reach kasan_populate_vmalloc
> +        * // and call kasan_unpoison_memory:
> +        * STORE shadow(a), unpoison_val
> +        * ...
> +        * STORE shadow(a+99), unpoison_val     x = LOAD p
> +        * // rest of vmalloc process           <data dependency>
> +        * STORE p, a                           LOAD shadow(x+99)
> +        *
> +        * If there is no barrier between the end of unpoisioning the shadow
> +        * and the store of the result to p, the stores could be committed
> +        * in a different order by CPU#0, and CPU#1 could erroneously observe
> +        * poison in the shadow.
> +        *
> +        * We need some sort of barrier between the stores.
> +        *
> +        * In the vmalloc() case, this is provided by a smp_wmb() in
> +        * clear_vm_uninitialized_flag(). In the per-cpu allocator and in
> +        * get_vm_area() and friends, the caller gets shadow allocated but
> +        * doesn't have any pages mapped into the virtual address space that
> +        * has been reserved. Mapping those pages in will involve taking and
> +        * releasing a page-table lock, which will provide the barrier.
> +        */
> +
> +       return 0;
> +}
> +
> +/*
> + * Poison the shadow for a vmalloc region. Called as part of the
> + * freeing process at the time the region is freed.
> + */
> +void kasan_poison_vmalloc(const void *start, unsigned long size)
> +{
> +       if (!is_vmalloc_or_module_addr(start))
> +               return;
> +
> +       size = round_up(size, KASAN_GRANULE_SIZE);
> +       kasan_poison_memory(start, size, KASAN_VMALLOC_INVALID);
> +}
> +
> +void kasan_unpoison_vmalloc(const void *start, unsigned long size)
> +{
> +       if (!is_vmalloc_or_module_addr(start))
> +               return;
> +
> +       kasan_unpoison_memory(start, size);
> +}
> +
> +static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
> +                                       void *unused)
> +{
> +       unsigned long page;
> +
> +       page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT);
> +
> +       spin_lock(&init_mm.page_table_lock);
> +
> +       if (likely(!pte_none(*ptep))) {
> +               pte_clear(&init_mm, addr, ptep);
> +               free_page(page);
> +       }
> +       spin_unlock(&init_mm.page_table_lock);
> +
> +       return 0;
> +}
> +
> +/*
> + * Release the backing for the vmalloc region [start, end), which
> + * lies within the free region [free_region_start, free_region_end).
> + *
> + * This can be run lazily, long after the region was freed. It runs
> + * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap
> + * infrastructure.
> + *
> + * How does this work?
> + * -------------------
> + *
> + * We have a region that is page aligned, labelled as A.
> + * That might not map onto the shadow in a way that is page-aligned:
> + *
> + *                    start                     end
> + *                    v                         v
> + * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc
> + *  -------- -------- --------          -------- --------
> + *      |        |       |                 |        |
> + *      |        |       |         /-------/        |
> + *      \-------\|/------/         |/---------------/
> + *              |||                ||
> + *             |??AAAAAA|AAAAAAAA|AA??????|                < shadow
> + *                 (1)      (2)      (3)
> + *
> + * First we align the start upwards and the end downwards, so that the
> + * shadow of the region aligns with shadow page boundaries. In the
> + * example, this gives us the shadow page (2). This is the shadow entirely
> + * covered by this allocation.
> + *
> + * Then we have the tricky bits. We want to know if we can free the
> + * partially covered shadow pages - (1) and (3) in the example. For this,
> + * we are given the start and end of the free region that contains this
> + * allocation. Extending our previous example, we could have:
> + *
> + *  free_region_start                                    free_region_end
> + *  |                 start                     end      |
> + *  v                 v                         v        v
> + * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc
> + *  -------- -------- --------          -------- --------
> + *      |        |       |                 |        |
> + *      |        |       |         /-------/        |
> + *      \-------\|/------/         |/---------------/
> + *              |||                ||
> + *             |FFAAAAAA|AAAAAAAA|AAF?????|                < shadow
> + *                 (1)      (2)      (3)
> + *
> + * Once again, we align the start of the free region up, and the end of
> + * the free region down so that the shadow is page aligned. So we can free
> + * page (1) - we know no allocation currently uses anything in that page,
> + * because all of it is in the vmalloc free region. But we cannot free
> + * page (3), because we can't be sure that the rest of it is unused.
> + *
> + * We only consider pages that contain part of the original region for
> + * freeing: we don't try to free other pages from the free region or we'd
> + * end up trying to free huge chunks of virtual address space.
> + *
> + * Concurrency
> + * -----------
> + *
> + * How do we know that we're not freeing a page that is simultaneously
> + * being used for a fresh allocation in kasan_populate_vmalloc(_pte)?
> + *
> + * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running
> + * at the same time. While we run under free_vmap_area_lock, the population
> + * code does not.
> + *
> + * free_vmap_area_lock instead operates to ensure that the larger range
> + * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and
> + * the per-cpu region-finding algorithm both run under free_vmap_area_lock,
> + * no space identified as free will become used while we are running. This
> + * means that so long as we are careful with alignment and only free shadow
> + * pages entirely covered by the free region, we will not run in to any
> + * trouble - any simultaneous allocations will be for disjoint regions.
> + */
> +void kasan_release_vmalloc(unsigned long start, unsigned long end,
> +                          unsigned long free_region_start,
> +                          unsigned long free_region_end)
> +{
> +       void *shadow_start, *shadow_end;
> +       unsigned long region_start, region_end;
> +       unsigned long size;
> +
> +       region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
> +       region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
> +
> +       free_region_start = ALIGN(free_region_start,
> +                                 PAGE_SIZE * KASAN_GRANULE_SIZE);
> +
> +       if (start != region_start &&
> +           free_region_start < region_start)
> +               region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
> +
> +       free_region_end = ALIGN_DOWN(free_region_end,
> +                                    PAGE_SIZE * KASAN_GRANULE_SIZE);
> +
> +       if (end != region_end &&
> +           free_region_end > region_end)
> +               region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
> +
> +       shadow_start = kasan_mem_to_shadow((void *)region_start);
> +       shadow_end = kasan_mem_to_shadow((void *)region_end);
> +
> +       if (shadow_end > shadow_start) {
> +               size = shadow_end - shadow_start;
> +               apply_to_existing_page_range(&init_mm,
> +                                            (unsigned long)shadow_start,
> +                                            size, kasan_depopulate_vmalloc_pte,
> +                                            NULL);
> +               flush_tlb_kernel_range((unsigned long)shadow_start,
> +                                      (unsigned long)shadow_end);
> +       }
> +}
> +
> +#else /* CONFIG_KASAN_VMALLOC */
> +
> +int kasan_module_alloc(void *addr, size_t size)
> +{
> +       void *ret;
> +       size_t scaled_size;
> +       size_t shadow_size;
> +       unsigned long shadow_start;
> +
> +       shadow_start = (unsigned long)kasan_mem_to_shadow(addr);
> +       scaled_size = (size + KASAN_GRANULE_SIZE - 1) >>
> +                               KASAN_SHADOW_SCALE_SHIFT;
> +       shadow_size = round_up(scaled_size, PAGE_SIZE);
> +
> +       if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
> +               return -EINVAL;
> +
> +       ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
> +                       shadow_start + shadow_size,
> +                       GFP_KERNEL,
> +                       PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
> +                       __builtin_return_address(0));
> +
> +       if (ret) {
> +               __memset(ret, KASAN_SHADOW_INIT, shadow_size);
> +               find_vm_area(addr)->flags |= VM_KASAN;
> +               kmemleak_ignore(ret);
> +               return 0;
> +       }
> +
> +       return -ENOMEM;
> +}
> +
> +void kasan_free_shadow(const struct vm_struct *vm)
> +{
> +       if (vm->flags & VM_KASAN)
> +               vfree(kasan_mem_to_shadow(vm->addr));
> +}
> +
> +#endif
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE
  2020-11-10 22:10 ` [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE Andrey Konovalov
@ 2020-11-11 14:13   ` Alexander Potapenko
  2020-11-11 18:47     ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:13 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Define KASAN_GRANULE_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT), which is
> the same as (KASAN_GRANULE_SIZE * PAGE_SIZE), and use it across KASAN code
> to simplify it.

What's the physical sense behind KASAN_GRANULE_PAGE? Is it something
more than just a product of two constants?
The name suggests it might be something page-sized, but in reality it is not.

>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> ---
> Change-Id: I0b627b24187d06c8b9bb2f1d04d94b3d06945e73
> ---
>  mm/kasan/init.c   | 10 ++++------
>  mm/kasan/kasan.h  |  1 +
>  mm/kasan/shadow.c | 16 +++++++---------
>  3 files changed, 12 insertions(+), 15 deletions(-)
>
> diff --git a/mm/kasan/init.c b/mm/kasan/init.c
> index 1a71eaa8c5f9..26b2663b3a42 100644
> --- a/mm/kasan/init.c
> +++ b/mm/kasan/init.c
> @@ -441,9 +441,8 @@ void kasan_remove_zero_shadow(void *start, unsigned long size)
>         addr = (unsigned long)kasan_mem_to_shadow(start);
>         end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT);
>
> -       if (WARN_ON((unsigned long)start %
> -                       (KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
> -           WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
> +       if (WARN_ON((unsigned long)start % KASAN_GRANULE_PAGE) ||
> +           WARN_ON(size % KASAN_GRANULE_PAGE))
>                 return;
>
>         for (; addr < end; addr = next) {
> @@ -476,9 +475,8 @@ int kasan_add_zero_shadow(void *start, unsigned long size)
>         shadow_start = kasan_mem_to_shadow(start);
>         shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT);
>
> -       if (WARN_ON((unsigned long)start %
> -                       (KASAN_GRANULE_SIZE * PAGE_SIZE)) ||
> -           WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE)))
> +       if (WARN_ON((unsigned long)start % KASAN_GRANULE_PAGE) ||
> +           WARN_ON(size % KASAN_GRANULE_PAGE))
>                 return -EINVAL;
>
>         ret = kasan_populate_early_shadow(shadow_start, shadow_end);
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index c31e2c739301..1865bb92d47a 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -7,6 +7,7 @@
>
>  #define KASAN_GRANULE_SIZE     (1UL << KASAN_SHADOW_SCALE_SHIFT)
>  #define KASAN_GRANULE_MASK     (KASAN_GRANULE_SIZE - 1)
> +#define KASAN_GRANULE_PAGE     (KASAN_GRANULE_SIZE << PAGE_SHIFT)
>
>  #define KASAN_TAG_KERNEL       0xFF /* native kernel pointers tag */
>  #define KASAN_TAG_INVALID      0xFE /* inaccessible memory tag */
> diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
> index ca0cc4c31454..1fadd4930d54 100644
> --- a/mm/kasan/shadow.c
> +++ b/mm/kasan/shadow.c
> @@ -161,7 +161,7 @@ static int __meminit kasan_mem_notifier(struct notifier_block *nb,
>         shadow_end = shadow_start + shadow_size;
>
>         if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) ||
> -               WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT)))
> +               WARN_ON(start_kaddr % KASAN_GRANULE_PAGE))
>                 return NOTIFY_BAD;
>
>         switch (action) {
> @@ -432,22 +432,20 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
>         unsigned long region_start, region_end;
>         unsigned long size;
>
> -       region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE);
> -       region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE);
> +       region_start = ALIGN(start, KASAN_GRANULE_PAGE);
> +       region_end = ALIGN_DOWN(end, KASAN_GRANULE_PAGE);
>
> -       free_region_start = ALIGN(free_region_start,
> -                                 PAGE_SIZE * KASAN_GRANULE_SIZE);
> +       free_region_start = ALIGN(free_region_start, KASAN_GRANULE_PAGE);
>
>         if (start != region_start &&
>             free_region_start < region_start)
> -               region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE;
> +               region_start -= KASAN_GRANULE_PAGE;
>
> -       free_region_end = ALIGN_DOWN(free_region_end,
> -                                    PAGE_SIZE * KASAN_GRANULE_SIZE);
> +       free_region_end = ALIGN_DOWN(free_region_end, KASAN_GRANULE_PAGE);
>
>         if (end != region_end &&
>             free_region_end > region_end)
> -               region_end += PAGE_SIZE * KASAN_GRANULE_SIZE;
> +               region_end += KASAN_GRANULE_PAGE;
>
>         shadow_start = kasan_mem_to_shadow((void *)region_start);
>         shadow_end = kasan_mem_to_shadow((void *)region_end);
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 11/44] kasan: rename report and tags files
  2020-11-10 22:10 ` [PATCH v9 11/44] kasan: rename report and tags files Andrey Konovalov
@ 2020-11-11 14:15   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:15 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Rename generic_report.c to report_generic.c and tags_report.c to
> report_sw_tags.c, as their content is more relevant to report.c file.
> Also rename tags.c to sw_tags.c to better reflect that this file contains
> code for software tag-based mode.
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: If77d21f655d52ef3e58c4c37fd6621a07f505f18
> ---
>  mm/kasan/Makefile                               | 16 ++++++++--------
>  mm/kasan/report.c                               |  2 +-
>  mm/kasan/{generic_report.c => report_generic.c} |  0
>  mm/kasan/{tags_report.c => report_sw_tags.c}    |  0
>  mm/kasan/{tags.c => sw_tags.c}                  |  0
>  5 files changed, 9 insertions(+), 9 deletions(-)
>  rename mm/kasan/{generic_report.c => report_generic.c} (100%)
>  rename mm/kasan/{tags_report.c => report_sw_tags.c} (100%)
>  rename mm/kasan/{tags.c => sw_tags.c} (100%)
>
> diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
> index 7cc1031e1ef8..f1d68a34f3c9 100644
> --- a/mm/kasan/Makefile
> +++ b/mm/kasan/Makefile
> @@ -6,13 +6,13 @@ KCOV_INSTRUMENT := n
>  # Disable ftrace to avoid recursion.
>  CFLAGS_REMOVE_common.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_generic.o = $(CC_FLAGS_FTRACE)
> -CFLAGS_REMOVE_generic_report.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
> -CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE)
> -CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE)
>
>  # Function splitter causes unnecessary splits in __asan_load1/__asan_store1
>  # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533
> @@ -23,14 +23,14 @@ CC_FLAGS_KASAN_RUNTIME += -DDISABLE_BRANCH_PROFILING
>
>  CFLAGS_common.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
> -CFLAGS_generic_report.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
> +CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
> +CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
> -CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
> -CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME)
> +CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>
>  obj-$(CONFIG_KASAN) := common.o report.o
> -obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o
> -obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o
> +obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o
> +obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 7b8dcb799a78..fff0c7befbfe 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * This file contains common generic and tag-based KASAN error reporting code.
> + * This file contains common KASAN error reporting code.
>   *
>   * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>   * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> diff --git a/mm/kasan/generic_report.c b/mm/kasan/report_generic.c
> similarity index 100%
> rename from mm/kasan/generic_report.c
> rename to mm/kasan/report_generic.c
> diff --git a/mm/kasan/tags_report.c b/mm/kasan/report_sw_tags.c
> similarity index 100%
> rename from mm/kasan/tags_report.c
> rename to mm/kasan/report_sw_tags.c
> diff --git a/mm/kasan/tags.c b/mm/kasan/sw_tags.c
> similarity index 100%
> rename from mm/kasan/tags.c
> rename to mm/kasan/sw_tags.c
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 12/44] kasan: don't duplicate config dependencies
  2020-11-10 22:10 ` [PATCH v9 12/44] kasan: don't duplicate config dependencies Andrey Konovalov
@ 2020-11-11 14:20   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:20 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Both KASAN_GENERIC and KASAN_SW_TAGS have common dependencies, move
> those to KASAN.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I77e475802e8f1750b9154fe4a6e6da4456054fcd
> ---
>  lib/Kconfig.kasan | 8 ++------
>  1 file changed, 2 insertions(+), 6 deletions(-)
>
> diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
> index 8f0742a0f23e..ec59a0e26d09 100644
> --- a/lib/Kconfig.kasan
> +++ b/lib/Kconfig.kasan
> @@ -24,6 +24,8 @@ menuconfig KASAN
>                    (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)
>         depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
>         depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS
> +       select CONSTRUCTORS
> +       select STACKDEPOT
>         help
>           Enables KASAN (KernelAddressSANitizer) - runtime memory debugger,
>           designed to find out-of-bounds accesses and use-after-free bugs.
> @@ -46,10 +48,7 @@ choice
>  config KASAN_GENERIC
>         bool "Generic mode"
>         depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC
> -       depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
>         select SLUB_DEBUG if SLUB
> -       select CONSTRUCTORS
> -       select STACKDEPOT
>         help
>           Enables generic KASAN mode.
>
> @@ -70,10 +69,7 @@ config KASAN_GENERIC
>  config KASAN_SW_TAGS
>         bool "Software tag-based mode"
>         depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS
> -       depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB)
>         select SLUB_DEBUG if SLUB
> -       select CONSTRUCTORS
> -       select STACKDEPOT
>         help
>           Enables software tag-based KASAN mode.
>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 13/44] kasan: hide invalid free check implementation
  2020-11-10 22:10 ` [PATCH v9 13/44] kasan: hide invalid free check implementation Andrey Konovalov
@ 2020-11-11 14:23   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:23 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> For software KASAN modes the check is based on the value in the shadow
> memory. Hardware tag-based KASAN won't be using shadow, so hide the
> implementation of the check in check_invalid_free().
>
> Also simplify the code for software tag-based mode.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I5fae9531c9fc948eb4d4e0c589744032fc5a0789
> ---
>  mm/kasan/common.c  | 19 +------------------
>  mm/kasan/generic.c |  7 +++++++
>  mm/kasan/kasan.h   |  2 ++
>  mm/kasan/sw_tags.c |  9 +++++++++
>  4 files changed, 19 insertions(+), 18 deletions(-)
>
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index 123abfb760d4..543e6bf2168f 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -272,25 +272,9 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
>         return (void *)object;
>  }
>
> -static inline bool shadow_invalid(u8 tag, s8 shadow_byte)
> -{
> -       if (IS_ENABLED(CONFIG_KASAN_GENERIC))
> -               return shadow_byte < 0 ||
> -                       shadow_byte >= KASAN_GRANULE_SIZE;
> -
> -       /* else CONFIG_KASAN_SW_TAGS: */
> -       if ((u8)shadow_byte == KASAN_TAG_INVALID)
> -               return true;
> -       if ((tag != KASAN_TAG_KERNEL) && (tag != (u8)shadow_byte))
> -               return true;
> -
> -       return false;
> -}
> -
>  static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
>                               unsigned long ip, bool quarantine)
>  {
> -       s8 shadow_byte;
>         u8 tag;
>         void *tagged_object;
>         unsigned long rounded_up_size;
> @@ -309,8 +293,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
>         if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU))
>                 return false;
>
> -       shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object));
> -       if (shadow_invalid(tag, shadow_byte)) {
> +       if (check_invalid_free(tagged_object)) {
>                 kasan_report_invalid_free(tagged_object, ip);
>                 return true;
>         }
> diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
> index ec4417156943..e1af3b6c53b8 100644
> --- a/mm/kasan/generic.c
> +++ b/mm/kasan/generic.c
> @@ -187,6 +187,13 @@ bool check_memory_region(unsigned long addr, size_t size, bool write,
>         return check_memory_region_inline(addr, size, write, ret_ip);
>  }
>
> +bool check_invalid_free(void *addr)
> +{
> +       s8 shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr));
> +
> +       return shadow_byte < 0 || shadow_byte >= KASAN_GRANULE_SIZE;
> +}
> +
>  void kasan_cache_shrink(struct kmem_cache *cache)
>  {
>         quarantine_remove_cache(cache);
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index 1865bb92d47a..3eff57e71ff5 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -164,6 +164,8 @@ void kasan_poison_memory(const void *address, size_t size, u8 value);
>  bool check_memory_region(unsigned long addr, size_t size, bool write,
>                                 unsigned long ret_ip);
>
> +bool check_invalid_free(void *addr);
> +
>  void *find_first_bad_addr(void *addr, size_t size);
>  const char *get_bug_type(struct kasan_access_info *info);
>
> diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
> index 4bdd7dbd6647..b2638c2cd58a 100644
> --- a/mm/kasan/sw_tags.c
> +++ b/mm/kasan/sw_tags.c
> @@ -121,6 +121,15 @@ bool check_memory_region(unsigned long addr, size_t size, bool write,
>         return true;
>  }
>
> +bool check_invalid_free(void *addr)
> +{
> +       u8 tag = get_tag(addr);
> +       u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr)));
> +
> +       return (shadow_byte == KASAN_TAG_INVALID) ||
> +               (tag != KASAN_TAG_KERNEL && tag != shadow_byte);
> +}
> +
>  #define DEFINE_HWASAN_LOAD_STORE(size)                                 \
>         void __hwasan_load##size##_noabort(unsigned long addr)          \
>         {                                                               \
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE
  2020-11-10 22:10 ` [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrey Konovalov
@ 2020-11-11 14:25   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:25 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Decoding routines aren't needed when CONFIG_KASAN_STACK_ENABLE is not
> enabled. Currently only generic KASAN mode implements stack error
> reporting.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I084e3214f2b40dc0bef7c5a9fafdc6f5c42b06a2
> ---
>  mm/kasan/kasan.h          |   6 ++
>  mm/kasan/report.c         | 162 --------------------------------------
>  mm/kasan/report_generic.c | 162 ++++++++++++++++++++++++++++++++++++++
>  3 files changed, 168 insertions(+), 162 deletions(-)
>
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index 3eff57e71ff5..d0cf61d4d70d 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -169,6 +169,12 @@ bool check_invalid_free(void *addr);
>  void *find_first_bad_addr(void *addr, size_t size);
>  const char *get_bug_type(struct kasan_access_info *info);
>
> +#if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK
> +void print_address_stack_frame(const void *addr);
> +#else
> +static inline void print_address_stack_frame(const void *addr) { }
> +#endif
> +
>  bool kasan_report(unsigned long addr, size_t size,
>                 bool is_write, unsigned long ip);
>  void kasan_report_invalid_free(void *object, unsigned long ip);
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index fff0c7befbfe..b18d193f7f58 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -211,168 +211,6 @@ static inline bool init_task_stack_addr(const void *addr)
>                         sizeof(init_thread_union.stack));
>  }
>
> -static bool __must_check tokenize_frame_descr(const char **frame_descr,
> -                                             char *token, size_t max_tok_len,
> -                                             unsigned long *value)
> -{
> -       const char *sep = strchr(*frame_descr, ' ');
> -
> -       if (sep == NULL)
> -               sep = *frame_descr + strlen(*frame_descr);
> -
> -       if (token != NULL) {
> -               const size_t tok_len = sep - *frame_descr;
> -
> -               if (tok_len + 1 > max_tok_len) {
> -                       pr_err("KASAN internal error: frame description too long: %s\n",
> -                              *frame_descr);
> -                       return false;
> -               }
> -
> -               /* Copy token (+ 1 byte for '\0'). */
> -               strlcpy(token, *frame_descr, tok_len + 1);
> -       }
> -
> -       /* Advance frame_descr past separator. */
> -       *frame_descr = sep + 1;
> -
> -       if (value != NULL && kstrtoul(token, 10, value)) {
> -               pr_err("KASAN internal error: not a valid number: %s\n", token);
> -               return false;
> -       }
> -
> -       return true;
> -}
> -
> -static void print_decoded_frame_descr(const char *frame_descr)
> -{
> -       /*
> -        * We need to parse the following string:
> -        *    "n alloc_1 alloc_2 ... alloc_n"
> -        * where alloc_i looks like
> -        *    "offset size len name"
> -        * or "offset size len name:line".
> -        */
> -
> -       char token[64];
> -       unsigned long num_objects;
> -
> -       if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> -                                 &num_objects))
> -               return;
> -
> -       pr_err("\n");
> -       pr_err("this frame has %lu %s:\n", num_objects,
> -              num_objects == 1 ? "object" : "objects");
> -
> -       while (num_objects--) {
> -               unsigned long offset;
> -               unsigned long size;
> -
> -               /* access offset */
> -               if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> -                                         &offset))
> -                       return;
> -               /* access size */
> -               if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> -                                         &size))
> -                       return;
> -               /* name length (unused) */
> -               if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL))
> -                       return;
> -               /* object name */
> -               if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> -                                         NULL))
> -                       return;
> -
> -               /* Strip line number; without filename it's not very helpful. */
> -               strreplace(token, ':', '\0');
> -
> -               /* Finally, print object information. */
> -               pr_err(" [%lu, %lu) '%s'", offset, offset + size, token);
> -       }
> -}
> -
> -static bool __must_check get_address_stack_frame_info(const void *addr,
> -                                                     unsigned long *offset,
> -                                                     const char **frame_descr,
> -                                                     const void **frame_pc)
> -{
> -       unsigned long aligned_addr;
> -       unsigned long mem_ptr;
> -       const u8 *shadow_bottom;
> -       const u8 *shadow_ptr;
> -       const unsigned long *frame;
> -
> -       BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP));
> -
> -       /*
> -        * NOTE: We currently only support printing frame information for
> -        * accesses to the task's own stack.
> -        */
> -       if (!object_is_on_stack(addr))
> -               return false;
> -
> -       aligned_addr = round_down((unsigned long)addr, sizeof(long));
> -       mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
> -       shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
> -       shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
> -
> -       while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
> -               shadow_ptr--;
> -               mem_ptr -= KASAN_GRANULE_SIZE;
> -       }
> -
> -       while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
> -               shadow_ptr--;
> -               mem_ptr -= KASAN_GRANULE_SIZE;
> -       }
> -
> -       if (shadow_ptr < shadow_bottom)
> -               return false;
> -
> -       frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
> -       if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
> -               pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
> -                      frame[0]);
> -               return false;
> -       }
> -
> -       *offset = (unsigned long)addr - (unsigned long)frame;
> -       *frame_descr = (const char *)frame[1];
> -       *frame_pc = (void *)frame[2];
> -
> -       return true;
> -}
> -
> -static void print_address_stack_frame(const void *addr)
> -{
> -       unsigned long offset;
> -       const char *frame_descr;
> -       const void *frame_pc;
> -
> -       if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
> -               return;
> -
> -       if (!get_address_stack_frame_info(addr, &offset, &frame_descr,
> -                                         &frame_pc))
> -               return;
> -
> -       /*
> -        * get_address_stack_frame_info only returns true if the given addr is
> -        * on the current task's stack.
> -        */
> -       pr_err("\n");
> -       pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n",
> -              addr, current->comm, task_pid_nr(current), offset);
> -       pr_err(" %pS\n", frame_pc);
> -
> -       if (!frame_descr)
> -               return;
> -
> -       print_decoded_frame_descr(frame_descr);
> -}
> -
>  static void print_address_description(void *addr, u8 tag)
>  {
>         struct page *page = kasan_addr_to_page(addr);
> diff --git a/mm/kasan/report_generic.c b/mm/kasan/report_generic.c
> index 7d5b9e5c7cfe..b543a1ed6078 100644
> --- a/mm/kasan/report_generic.c
> +++ b/mm/kasan/report_generic.c
> @@ -16,6 +16,7 @@
>  #include <linux/mm.h>
>  #include <linux/printk.h>
>  #include <linux/sched.h>
> +#include <linux/sched/task_stack.h>
>  #include <linux/slab.h>
>  #include <linux/stackdepot.h>
>  #include <linux/stacktrace.h>
> @@ -122,6 +123,167 @@ const char *get_bug_type(struct kasan_access_info *info)
>         return get_wild_bug_type(info);
>  }
>
> +#if CONFIG_KASAN_STACK
> +static bool __must_check tokenize_frame_descr(const char **frame_descr,
> +                                             char *token, size_t max_tok_len,
> +                                             unsigned long *value)
> +{
> +       const char *sep = strchr(*frame_descr, ' ');
> +
> +       if (sep == NULL)
> +               sep = *frame_descr + strlen(*frame_descr);
> +
> +       if (token != NULL) {
> +               const size_t tok_len = sep - *frame_descr;
> +
> +               if (tok_len + 1 > max_tok_len) {
> +                       pr_err("KASAN internal error: frame description too long: %s\n",
> +                              *frame_descr);
> +                       return false;
> +               }
> +
> +               /* Copy token (+ 1 byte for '\0'). */
> +               strlcpy(token, *frame_descr, tok_len + 1);
> +       }
> +
> +       /* Advance frame_descr past separator. */
> +       *frame_descr = sep + 1;
> +
> +       if (value != NULL && kstrtoul(token, 10, value)) {
> +               pr_err("KASAN internal error: not a valid number: %s\n", token);
> +               return false;
> +       }
> +
> +       return true;
> +}
> +
> +static void print_decoded_frame_descr(const char *frame_descr)
> +{
> +       /*
> +        * We need to parse the following string:
> +        *    "n alloc_1 alloc_2 ... alloc_n"
> +        * where alloc_i looks like
> +        *    "offset size len name"
> +        * or "offset size len name:line".
> +        */
> +
> +       char token[64];
> +       unsigned long num_objects;
> +
> +       if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> +                                 &num_objects))
> +               return;
> +
> +       pr_err("\n");
> +       pr_err("this frame has %lu %s:\n", num_objects,
> +              num_objects == 1 ? "object" : "objects");
> +
> +       while (num_objects--) {
> +               unsigned long offset;
> +               unsigned long size;
> +
> +               /* access offset */
> +               if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> +                                         &offset))
> +                       return;
> +               /* access size */
> +               if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> +                                         &size))
> +                       return;
> +               /* name length (unused) */
> +               if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL))
> +                       return;
> +               /* object name */
> +               if (!tokenize_frame_descr(&frame_descr, token, sizeof(token),
> +                                         NULL))
> +                       return;
> +
> +               /* Strip line number; without filename it's not very helpful. */
> +               strreplace(token, ':', '\0');
> +
> +               /* Finally, print object information. */
> +               pr_err(" [%lu, %lu) '%s'", offset, offset + size, token);
> +       }
> +}
> +
> +static bool __must_check get_address_stack_frame_info(const void *addr,
> +                                                     unsigned long *offset,
> +                                                     const char **frame_descr,
> +                                                     const void **frame_pc)
> +{
> +       unsigned long aligned_addr;
> +       unsigned long mem_ptr;
> +       const u8 *shadow_bottom;
> +       const u8 *shadow_ptr;
> +       const unsigned long *frame;
> +
> +       BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP));
> +
> +       /*
> +        * NOTE: We currently only support printing frame information for
> +        * accesses to the task's own stack.
> +        */
> +       if (!object_is_on_stack(addr))
> +               return false;
> +
> +       aligned_addr = round_down((unsigned long)addr, sizeof(long));
> +       mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE);
> +       shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr);
> +       shadow_bottom = kasan_mem_to_shadow(end_of_stack(current));
> +
> +       while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) {
> +               shadow_ptr--;
> +               mem_ptr -= KASAN_GRANULE_SIZE;
> +       }
> +
> +       while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) {
> +               shadow_ptr--;
> +               mem_ptr -= KASAN_GRANULE_SIZE;
> +       }
> +
> +       if (shadow_ptr < shadow_bottom)
> +               return false;
> +
> +       frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE);
> +       if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) {
> +               pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n",
> +                      frame[0]);
> +               return false;
> +       }
> +
> +       *offset = (unsigned long)addr - (unsigned long)frame;
> +       *frame_descr = (const char *)frame[1];
> +       *frame_pc = (void *)frame[2];
> +
> +       return true;
> +}
> +
> +void print_address_stack_frame(const void *addr)
> +{
> +       unsigned long offset;
> +       const char *frame_descr;
> +       const void *frame_pc;
> +
> +       if (!get_address_stack_frame_info(addr, &offset, &frame_descr,
> +                                         &frame_pc))
> +               return;
> +
> +       /*
> +        * get_address_stack_frame_info only returns true if the given addr is
> +        * on the current task's stack.
> +        */
> +       pr_err("\n");
> +       pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n",
> +              addr, current->comm, task_pid_nr(current), offset);
> +       pr_err(" %pS\n", frame_pc);
> +
> +       if (!frame_descr)
> +               return;
> +
> +       print_decoded_frame_descr(frame_descr);
> +}
> +#endif /* CONFIG_KASAN_STACK */
> +
>  #define DEFINE_ASAN_REPORT_LOAD(size)                     \
>  void __asan_report_load##size##_noabort(unsigned long addr) \
>  {                                                         \
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 15/44] kasan, arm64: only init shadow for software modes
  2020-11-10 22:10 ` [PATCH v9 15/44] kasan, arm64: only init shadow for software modes Andrey Konovalov
@ 2020-11-11 14:30   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:30 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Hardware tag-based KASAN won't be using shadow memory. Only initialize
> it when one of the software KASAN modes are enabled.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
> ---
> Change-Id: I055e0651369b14d3e54cdaa8c48e6329b2e8952d
> ---
>  arch/arm64/include/asm/kasan.h |  8 ++++++--
>  arch/arm64/mm/kasan_init.c     | 15 ++++++++++++++-
>  2 files changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h
> index b0dc4abc3589..f7ea70d02cab 100644
> --- a/arch/arm64/include/asm/kasan.h
> +++ b/arch/arm64/include/asm/kasan.h
> @@ -13,6 +13,12 @@
>  #define arch_kasan_get_tag(addr)       __tag_get(addr)
>
>  #ifdef CONFIG_KASAN
> +void kasan_init(void);
> +#else
> +static inline void kasan_init(void) { }
> +#endif
> +
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>
>  /*
>   * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
> @@ -33,12 +39,10 @@
>  #define _KASAN_SHADOW_START(va)        (KASAN_SHADOW_END - (1UL << ((va) - KASAN_SHADOW_SCALE_SHIFT)))
>  #define KASAN_SHADOW_START      _KASAN_SHADOW_START(vabits_actual)
>
> -void kasan_init(void);
>  void kasan_copy_shadow(pgd_t *pgdir);
>  asmlinkage void kasan_early_init(void);
>
>  #else
> -static inline void kasan_init(void) { }
>  static inline void kasan_copy_shadow(pgd_t *pgdir) { }
>  #endif
>
> diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
> index b24e43d20667..ffeb80d5aa8d 100644
> --- a/arch/arm64/mm/kasan_init.c
> +++ b/arch/arm64/mm/kasan_init.c
> @@ -21,6 +21,8 @@
>  #include <asm/sections.h>
>  #include <asm/tlbflush.h>
>
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
> +
>  static pgd_t tmp_pg_dir[PTRS_PER_PGD] __initdata __aligned(PGD_SIZE);
>
>  /*
> @@ -208,7 +210,7 @@ static void __init clear_pgds(unsigned long start,
>                 set_pgd(pgd_offset_k(start), __pgd(0));
>  }
>
> -void __init kasan_init(void)
> +static void __init kasan_init_shadow(void)
>  {
>         u64 kimg_shadow_start, kimg_shadow_end;
>         u64 mod_shadow_start, mod_shadow_end;
> @@ -269,6 +271,17 @@ void __init kasan_init(void)
>
>         memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE);
>         cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
> +}
> +
> +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
> +
> +static inline void __init kasan_init_shadow(void) { }
> +
> +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> +
> +void __init kasan_init(void)
> +{
> +       kasan_init_shadow();
>
>         /* At this point kasan is fully initialized. Enable error messages */
>         init_task.kasan_depth = 0;
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64
  2020-11-10 22:54 ` [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrew Morton
@ 2020-11-11 14:35   ` Andrey Konovalov
  0 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-11 14:35 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Alexander Potapenko, Marco Elver,
	Evgenii Stepanov, Branislav Rankov, Kevin Brodsky, kasan-dev,
	Linux ARM, Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:54 PM Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> On Tue, 10 Nov 2020 23:09:57 +0100 Andrey Konovalov <andreyknvl@google.com> wrote:
>
> > This patchset adds a new hardware tag-based mode to KASAN [1]. The new mode
> > is similar to the existing software tag-based KASAN, but relies on arm64
> > Memory Tagging Extension (MTE) [2] to perform memory and pointer tagging
> > (instead of shadow memory and compiler instrumentation).
>
> I have that all merged up on top of linux-next.  Numerous minor
> conflicts, mainly in arch/arm/Kconfig.  Also the changes in
> https://lkml.kernel.org/r/20201103175841.3495947-7-elver@google.com had
> to be fed into "kasan: split out shadow.c from common.c".
>
> I staged it after linux-next to provide visibility into potentially
> conflicting changes in the arm tree as things move forward.

Thank you very much, Andrew!

In case we need a v10, I'll do the rebase on top of mm and include the
kasan_enabled declaration fix as well.


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 16/44] kasan, arm64: only use kasan_depth for software modes
  2020-11-10 22:10 ` [PATCH v9 16/44] kasan, arm64: only use kasan_depth " Andrey Konovalov
@ 2020-11-11 14:59   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 14:59 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Hardware tag-based KASAN won't use kasan_depth. Only define and use it
> when one of the software KASAN modes are enabled.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I6109ea96c8df41ef6d75ad71bf22c1c8fa234a9a
> ---
>  arch/arm64/mm/kasan_init.c | 11 ++++++++---
>  include/linux/kasan.h      | 18 +++++++++---------
>  include/linux/sched.h      |  2 +-
>  init/init_task.c           |  2 +-
>  mm/kasan/common.c          |  2 ++
>  mm/kasan/report.c          |  2 ++
>  6 files changed, 23 insertions(+), 14 deletions(-)
>
> diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
> index ffeb80d5aa8d..5172799f831f 100644
> --- a/arch/arm64/mm/kasan_init.c
> +++ b/arch/arm64/mm/kasan_init.c
> @@ -273,17 +273,22 @@ static void __init kasan_init_shadow(void)
>         cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
>  }
>
> +static void __init kasan_init_depth(void)
> +{
> +       init_task.kasan_depth = 0;
> +}
> +
>  #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
>
>  static inline void __init kasan_init_shadow(void) { }
>
> +static inline void __init kasan_init_depth(void) { }
> +
>  #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
>
>  void __init kasan_init(void)
>  {
>         kasan_init_shadow();
> -
> -       /* At this point kasan is fully initialized. Enable error messages */
> -       init_task.kasan_depth = 0;
> +       kasan_init_depth();
>         pr_info("KernelAddressSanitizer initialized\n");
>  }
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index f6435b9f889c..979d598e1c30 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -51,6 +51,12 @@ static inline void *kasan_mem_to_shadow(const void *addr)
>  int kasan_add_zero_shadow(void *start, unsigned long size);
>  void kasan_remove_zero_shadow(void *start, unsigned long size);
>
> +/* Enable reporting bugs after kasan_disable_current() */
> +extern void kasan_enable_current(void);
> +
> +/* Disable reporting bugs for current task */
> +extern void kasan_disable_current(void);
> +
>  #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
>
>  static inline int kasan_add_zero_shadow(void *start, unsigned long size)
> @@ -61,16 +67,13 @@ static inline void kasan_remove_zero_shadow(void *start,
>                                         unsigned long size)
>  {}
>
> +static inline void kasan_enable_current(void) {}
> +static inline void kasan_disable_current(void) {}
> +
>  #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
>
>  #ifdef CONFIG_KASAN
>
> -/* Enable reporting bugs after kasan_disable_current() */
> -extern void kasan_enable_current(void);
> -
> -/* Disable reporting bugs for current task */
> -extern void kasan_disable_current(void);
> -
>  void kasan_unpoison_memory(const void *address, size_t size);
>
>  void kasan_unpoison_task_stack(struct task_struct *task);
> @@ -121,9 +124,6 @@ static inline void kasan_unpoison_memory(const void *address, size_t size) {}
>
>  static inline void kasan_unpoison_task_stack(struct task_struct *task) {}
>
> -static inline void kasan_enable_current(void) {}
> -static inline void kasan_disable_current(void) {}
> -
>  static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
>  static inline void kasan_free_pages(struct page *page, unsigned int order) {}
>
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index 063cd120b459..81b09bd31186 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -1197,7 +1197,7 @@ struct task_struct {
>         u64                             timer_slack_ns;
>         u64                             default_timer_slack_ns;
>
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         unsigned int                    kasan_depth;
>  #endif
>
> diff --git a/init/init_task.c b/init/init_task.c
> index a56f0abb63e9..39703b4ef1f1 100644
> --- a/init/init_task.c
> +++ b/init/init_task.c
> @@ -176,7 +176,7 @@ struct task_struct init_task
>         .numa_group     = NULL,
>         .numa_faults    = NULL,
>  #endif
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         .kasan_depth    = 1,
>  #endif
>  #ifdef CONFIG_KCSAN
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index 543e6bf2168f..d0b3ff410b0c 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -46,6 +46,7 @@ void kasan_set_track(struct kasan_track *track, gfp_t flags)
>         track->stack = kasan_save_stack(flags);
>  }
>
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  void kasan_enable_current(void)
>  {
>         current->kasan_depth++;
> @@ -55,6 +56,7 @@ void kasan_disable_current(void)
>  {
>         current->kasan_depth--;
>  }
> +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
>
>  static void __kasan_unpoison_stack(struct task_struct *task, const void *sp)
>  {
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index b18d193f7f58..af9138ea54ad 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -292,8 +292,10 @@ static void print_shadow_for_address(const void *addr)
>
>  static bool report_enabled(void)
>  {
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         if (current->kasan_depth)
>                 return false;
> +#endif
>         if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags))
>                 return true;
>         return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags);
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 17/44] kasan, arm64: move initialization message
  2020-11-10 22:10 ` [PATCH v9 17/44] kasan, arm64: move initialization message Andrey Konovalov
@ 2020-11-11 15:03   ` Alexander Potapenko
  2020-11-11 18:50     ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:03 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Software tag-based KASAN mode is fully initialized with kasan_init_tags(),
> while the generic mode only requires kasan_init(). Move the
> initialization message for tag-based mode into kasan_init_tags().
>
> Also fix pr_fmt() usage for KASAN code: generic.c doesn't need it as it
> doesn't use any printing functions; tag-based mode should use "kasan:"
> instead of KBUILD_MODNAME (which stands for file name).
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
> ---
> Change-Id: Iddca9764b30ff0fab1922f26ca9d4f39b6f22673
> ---
>  arch/arm64/include/asm/kasan.h |  9 +++------
>  arch/arm64/mm/kasan_init.c     | 13 +++++--------
>  mm/kasan/generic.c             |  2 --
>  mm/kasan/sw_tags.c             |  4 +++-
>  4 files changed, 11 insertions(+), 17 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h
> index f7ea70d02cab..0aaf9044cd6a 100644
> --- a/arch/arm64/include/asm/kasan.h
> +++ b/arch/arm64/include/asm/kasan.h
> @@ -12,14 +12,10 @@
>  #define arch_kasan_reset_tag(addr)     __tag_reset(addr)
>  #define arch_kasan_get_tag(addr)       __tag_get(addr)
>
> -#ifdef CONFIG_KASAN
> -void kasan_init(void);
> -#else
> -static inline void kasan_init(void) { }
> -#endif
> -
>  #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>
> +void kasan_init(void);
> +
>  /*
>   * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
>   * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses,
> @@ -43,6 +39,7 @@ void kasan_copy_shadow(pgd_t *pgdir);
>  asmlinkage void kasan_early_init(void);
>
>  #else
> +static inline void kasan_init(void) { }
>  static inline void kasan_copy_shadow(pgd_t *pgdir) { }
>  #endif
>
> diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
> index 5172799f831f..e35ce04beed1 100644
> --- a/arch/arm64/mm/kasan_init.c
> +++ b/arch/arm64/mm/kasan_init.c
> @@ -278,17 +278,14 @@ static void __init kasan_init_depth(void)
>         init_task.kasan_depth = 0;
>  }
>
> -#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
> -
> -static inline void __init kasan_init_shadow(void) { }
> -
> -static inline void __init kasan_init_depth(void) { }
> -
> -#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> -
>  void __init kasan_init(void)
>  {
>         kasan_init_shadow();
>         kasan_init_depth();
> +#if defined(CONFIG_KASAN_GENERIC)
> +       /* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
>         pr_info("KernelAddressSanitizer initialized\n");
> +#endif
>  }

Cannot we have a single kasan_init() function that will call
tool-specific initialization functions and print the message at the
end?

> +
> +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c
> index e1af3b6c53b8..adb254df1b1d 100644
> --- a/mm/kasan/generic.c
> +++ b/mm/kasan/generic.c
> @@ -9,8 +9,6 @@
>   *        Andrey Konovalov <andreyknvl@gmail.com>
>   */
>
> -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> -
>  #include <linux/export.h>
>  #include <linux/interrupt.h>
>  #include <linux/init.h>
> diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
> index b2638c2cd58a..d25f8641b7cd 100644
> --- a/mm/kasan/sw_tags.c
> +++ b/mm/kasan/sw_tags.c
> @@ -6,7 +6,7 @@
>   * Author: Andrey Konovalov <andreyknvl@google.com>
>   */
>
> -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +#define pr_fmt(fmt) "kasan: " fmt
>
>  #include <linux/export.h>
>  #include <linux/interrupt.h>
> @@ -41,6 +41,8 @@ void kasan_init_tags(void)
>
>         for_each_possible_cpu(cpu)
>                 per_cpu(prng_state, cpu) = (u32)get_cycles();
> +
> +       pr_info("KernelAddressSanitizer initialized\n");
>  }
>
>  /*
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init
  2020-11-10 22:10 ` [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init Andrey Konovalov
@ 2020-11-11 15:05   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:05 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Rename kasan_init_tags() to kasan_init_sw_tags() as the upcoming hardware
> tag-based KASAN mode will have its own initialization routine.
> Also similarly to kasan_init() mark kasan_init_tags() as __init.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I99aa2f7115d38a34ed85b329dadab6c7d6952416
> ---
>  arch/arm64/kernel/setup.c  | 2 +-
>  arch/arm64/mm/kasan_init.c | 2 +-
>  include/linux/kasan.h      | 4 ++--
>  mm/kasan/sw_tags.c         | 2 +-
>  4 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index 133257ffd859..bb79b09f73c8 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -358,7 +358,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>         smp_build_mpidr_hash();
>
>         /* Init percpu seeds for random tags after cpus are set up. */
> -       kasan_init_tags();
> +       kasan_init_sw_tags();
>
>  #ifdef CONFIG_ARM64_SW_TTBR0_PAN
>         /*
> diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
> index e35ce04beed1..d8e66c78440e 100644
> --- a/arch/arm64/mm/kasan_init.c
> +++ b/arch/arm64/mm/kasan_init.c
> @@ -283,7 +283,7 @@ void __init kasan_init(void)
>         kasan_init_shadow();
>         kasan_init_depth();
>  #if defined(CONFIG_KASAN_GENERIC)
> -       /* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
> +       /* CONFIG_KASAN_SW_TAGS also requires kasan_init_sw_tags(). */
>         pr_info("KernelAddressSanitizer initialized\n");
>  #endif
>  }
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index 979d598e1c30..1d6ec3325163 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -191,7 +191,7 @@ static inline void kasan_record_aux_stack(void *ptr) {}
>
>  #ifdef CONFIG_KASAN_SW_TAGS
>
> -void kasan_init_tags(void);
> +void __init kasan_init_sw_tags(void);
>
>  void *kasan_reset_tag(const void *addr);
>
> @@ -200,7 +200,7 @@ bool kasan_report(unsigned long addr, size_t size,
>
>  #else /* CONFIG_KASAN_SW_TAGS */
>
> -static inline void kasan_init_tags(void) { }
> +static inline void kasan_init_sw_tags(void) { }
>
>  static inline void *kasan_reset_tag(const void *addr)
>  {
> diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
> index d25f8641b7cd..b09a2c06abad 100644
> --- a/mm/kasan/sw_tags.c
> +++ b/mm/kasan/sw_tags.c
> @@ -35,7 +35,7 @@
>
>  static DEFINE_PER_CPU(u32, prng_state);
>
> -void kasan_init_tags(void)
> +void __init kasan_init_sw_tags(void)
>  {
>         int cpu;
>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata
  2020-11-10 22:10 ` [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata Andrey Konovalov
@ 2020-11-11 15:06   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:06 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Hardware tag-based KASAN won't be using shadow memory, but will reuse
> this function. Rename "shadow" to implementation-neutral "metadata".
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
> ---
> Change-Id: I03706fe34b38da7860c39aa0968e00001a7d1873
> ---
>  mm/kasan/kasan.h          | 2 +-
>  mm/kasan/report.c         | 6 +++---
>  mm/kasan/report_generic.c | 2 +-
>  3 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index d0cf61d4d70d..f9366dfd94c9 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -146,7 +146,7 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
>                 << KASAN_SHADOW_SCALE_SHIFT);
>  }
>
> -static inline bool addr_has_shadow(const void *addr)
> +static inline bool addr_has_metadata(const void *addr)
>  {
>         return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
>  }
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index af9138ea54ad..2990ca34abaf 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -361,7 +361,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write,
>         untagged_addr = reset_tag(tagged_addr);
>
>         info.access_addr = tagged_addr;
> -       if (addr_has_shadow(untagged_addr))
> +       if (addr_has_metadata(untagged_addr))
>                 info.first_bad_addr = find_first_bad_addr(tagged_addr, size);
>         else
>                 info.first_bad_addr = untagged_addr;
> @@ -372,11 +372,11 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write,
>         start_report(&flags);
>
>         print_error_description(&info);
> -       if (addr_has_shadow(untagged_addr))
> +       if (addr_has_metadata(untagged_addr))
>                 print_tags(get_tag(tagged_addr), info.first_bad_addr);
>         pr_err("\n");
>
> -       if (addr_has_shadow(untagged_addr)) {
> +       if (addr_has_metadata(untagged_addr)) {
>                 print_address_description(untagged_addr, get_tag(tagged_addr));
>                 pr_err("\n");
>                 print_shadow_for_address(info.first_bad_addr);
> diff --git a/mm/kasan/report_generic.c b/mm/kasan/report_generic.c
> index b543a1ed6078..16ed550850e9 100644
> --- a/mm/kasan/report_generic.c
> +++ b/mm/kasan/report_generic.c
> @@ -118,7 +118,7 @@ const char *get_bug_type(struct kasan_access_info *info)
>         if (info->access_addr + info->access_size < info->access_addr)
>                 return "out-of-bounds";
>
> -       if (addr_has_shadow(info->access_addr))
> +       if (addr_has_metadata(info->access_addr))
>                 return get_shadow_bug_type(info);
>         return get_wild_bug_type(info);
>  }
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata
  2020-11-10 22:10 ` [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata Andrey Konovalov
@ 2020-11-11 15:07   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:07 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Hardware tag-based KASAN won't be using shadow memory, but will reuse
> this function. Rename "shadow" to implementation-neutral "metadata".
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
> ---
> Change-Id: I18397dddbed6bc6d365ddcaf063a83948e1150a5
> ---
>  mm/kasan/report.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 2990ca34abaf..5d5733831ad7 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -252,7 +252,7 @@ static int shadow_pointer_offset(const void *row, const void *shadow)
>                 (shadow - row) / SHADOW_BYTES_PER_BLOCK + 1;
>  }
>
> -static void print_shadow_for_address(const void *addr)
> +static void print_memory_metadata(const void *addr)
>  {
>         int i;
>         const void *shadow = kasan_mem_to_shadow(addr);
> @@ -338,7 +338,7 @@ void kasan_report_invalid_free(void *object, unsigned long ip)
>         pr_err("\n");
>         print_address_description(object, tag);
>         pr_err("\n");
> -       print_shadow_for_address(object);
> +       print_memory_metadata(object);
>         end_report(&flags);
>  }
>
> @@ -379,7 +379,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write,
>         if (addr_has_metadata(untagged_addr)) {
>                 print_address_description(untagged_addr, get_tag(tagged_addr));
>                 pr_err("\n");
> -               print_shadow_for_address(info.first_bad_addr);
> +               print_memory_metadata(info.first_bad_addr);
>         } else {
>                 dump_stack();
>         }
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes
  2020-11-10 22:10 ` [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes Andrey Konovalov
@ 2020-11-11 15:08   ` Alexander Potapenko
  2020-11-11 18:52     ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:08 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> kasan_non_canonical_hook() is only applicable to KASAN modes that use
> shadow memory, and won't be needed for hardware tag-based KASAN.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> ---
> Change-Id: Icc9f5ef100a2e86f3a4214a0c3131a68266181b2
> ---
>  mm/kasan/report.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 5d5733831ad7..594bad2a3a5e 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -403,7 +403,8 @@ bool kasan_report(unsigned long addr, size_t size, bool is_write,
>         return ret;
>  }
>
> -#ifdef CONFIG_KASAN_INLINE
> +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> +       defined(CONFIG_KASAN_INLINE)
>  /*
>   * With CONFIG_KASAN_INLINE, accesses to bogus pointers (outside the high
>   * canonical half of the address space) cause out-of-bounds shadow memory reads

Perhaps this comment also needs to be updated.

> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 22/44] kasan: rename SHADOW layout macros to META
  2020-11-10 22:10 ` [PATCH v9 22/44] kasan: rename SHADOW layout macros to META Andrey Konovalov
@ 2020-11-11 15:18   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:18 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Hardware tag-based KASAN won't be using shadow memory, but will reuse
> these macros. Rename "SHADOW" to implementation-neutral "META".
>
> No functional changes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: Id2d836bf43b401bce1221cc06e745185f17b1cc
> ---
>  mm/kasan/report.c | 30 +++++++++++++++---------------
>  1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 594bad2a3a5e..8c588588c88f 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -33,11 +33,11 @@
>  #include "kasan.h"
>  #include "../slab.h"
>
> -/* Shadow layout customization. */
> -#define SHADOW_BYTES_PER_BLOCK 1
> -#define SHADOW_BLOCKS_PER_ROW 16
> -#define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK)
> -#define SHADOW_ROWS_AROUND_ADDR 2
> +/* Metadata layout customization. */
> +#define META_BYTES_PER_BLOCK 1
> +#define META_BLOCKS_PER_ROW 16
> +#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
> +#define META_ROWS_AROUND_ADDR 2
>
>  static unsigned long kasan_flags;
>
> @@ -240,7 +240,7 @@ static void print_address_description(void *addr, u8 tag)
>
>  static bool row_is_guilty(const void *row, const void *guilty)
>  {
> -       return (row <= guilty) && (guilty < row + SHADOW_BYTES_PER_ROW);
> +       return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW);
>  }
>
>  static int shadow_pointer_offset(const void *row, const void *shadow)
> @@ -249,7 +249,7 @@ static int shadow_pointer_offset(const void *row, const void *shadow)
>          *    3 + (BITS_PER_LONG/8)*2 chars.
>          */
>         return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 +
> -               (shadow - row) / SHADOW_BYTES_PER_BLOCK + 1;
> +               (shadow - row) / META_BYTES_PER_BLOCK + 1;
>  }
>
>  static void print_memory_metadata(const void *addr)
> @@ -259,15 +259,15 @@ static void print_memory_metadata(const void *addr)
>         const void *shadow_row;
>
>         shadow_row = (void *)round_down((unsigned long)shadow,
> -                                       SHADOW_BYTES_PER_ROW)
> -               - SHADOW_ROWS_AROUND_ADDR * SHADOW_BYTES_PER_ROW;
> +                                       META_BYTES_PER_ROW)
> +               - META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW;
>
>         pr_err("Memory state around the buggy address:\n");
>
> -       for (i = -SHADOW_ROWS_AROUND_ADDR; i <= SHADOW_ROWS_AROUND_ADDR; i++) {
> +       for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) {
>                 const void *kaddr = kasan_shadow_to_mem(shadow_row);
>                 char buffer[4 + (BITS_PER_LONG/8)*2];
> -               char shadow_buf[SHADOW_BYTES_PER_ROW];
> +               char shadow_buf[META_BYTES_PER_ROW];
>
>                 snprintf(buffer, sizeof(buffer),
>                         (i == 0) ? ">%px: " : " %px: ", kaddr);
> @@ -276,17 +276,17 @@ static void print_memory_metadata(const void *addr)
>                  * function, because generic functions may try to
>                  * access kasan mapping for the passed address.
>                  */
> -               memcpy(shadow_buf, shadow_row, SHADOW_BYTES_PER_ROW);
> +               memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW);
>                 print_hex_dump(KERN_ERR, buffer,
> -                       DUMP_PREFIX_NONE, SHADOW_BYTES_PER_ROW, 1,
> -                       shadow_buf, SHADOW_BYTES_PER_ROW, 0);
> +                       DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1,
> +                       shadow_buf, META_BYTES_PER_ROW, 0);
>
>                 if (row_is_guilty(shadow_row, shadow))
>                         pr_err("%*c\n",
>                                 shadow_pointer_offset(shadow_row, shadow),
>                                 '^');
>
> -               shadow_row += SHADOW_BYTES_PER_ROW;
> +               shadow_row += META_BYTES_PER_ROW;
>         }
>  }
>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode
  2020-11-10 22:10 ` [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode Andrey Konovalov
@ 2020-11-11 15:22   ` Alexander Potapenko
  2020-11-11 16:19     ` Alexander Potapenko
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:22 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This is a preparatory commit for the upcoming addition of a new hardware
> tag-based (MTE-based) KASAN mode.
>
> Rework print_memory_metadata() to make it agnostic with regard to the
> way metadata is stored. Allow providing a separate metadata_fetch_row()
> implementation for each KASAN mode. Hardware tag-based KASAN will provide
> its own implementation that doesn't use shadow memory.
>
> No functional changes for software modes.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> ---
> Change-Id: I5b0ed1d079ea776e620beca6a529a861e7dced95
> ---
>  mm/kasan/kasan.h          |  8 ++++++
>  mm/kasan/report.c         | 56 +++++++++++++++++++--------------------
>  mm/kasan/report_generic.c |  5 ++++
>  mm/kasan/report_sw_tags.c |  5 ++++
>  4 files changed, 45 insertions(+), 29 deletions(-)
>
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index f9366dfd94c9..b5b00bff358f 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -57,6 +57,13 @@
>  #define KASAN_ABI_VERSION 1
>  #endif
>
> +/* Metadata layout customization. */
> +#define META_BYTES_PER_BLOCK 1
> +#define META_BLOCKS_PER_ROW 16
> +#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
> +#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_GRANULE_SIZE)
> +#define META_ROWS_AROUND_ADDR 2
> +
>  struct kasan_access_info {
>         const void *access_addr;
>         const void *first_bad_addr;
> @@ -168,6 +175,7 @@ bool check_invalid_free(void *addr);
>
>  void *find_first_bad_addr(void *addr, size_t size);
>  const char *get_bug_type(struct kasan_access_info *info);
> +void metadata_fetch_row(char *buffer, void *row);
>
>  #if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK
>  void print_address_stack_frame(const void *addr);
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 8c588588c88f..8afc1a6ab202 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -33,12 +33,6 @@
>  #include "kasan.h"
>  #include "../slab.h"
>
> -/* Metadata layout customization. */
> -#define META_BYTES_PER_BLOCK 1
> -#define META_BLOCKS_PER_ROW 16
> -#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK)
> -#define META_ROWS_AROUND_ADDR 2
> -
>  static unsigned long kasan_flags;
>
>  #define KASAN_BIT_REPORTED     0
> @@ -238,55 +232,59 @@ static void print_address_description(void *addr, u8 tag)
>         print_address_stack_frame(addr);
>  }
>
> -static bool row_is_guilty(const void *row, const void *guilty)
> +static bool meta_row_is_guilty(const void *row, const void *addr)
>  {
> -       return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW);
> +       return (row <= addr) && (addr < row + META_MEM_BYTES_PER_ROW);
>  }
>
> -static int shadow_pointer_offset(const void *row, const void *shadow)
> +static int meta_pointer_offset(const void *row, const void *addr)
>  {
> -       /* The length of ">ff00ff00ff00ff00: " is
> -        *    3 + (BITS_PER_LONG/8)*2 chars.
> +       /*
> +        * Memory state around the buggy address:
> +        *  ff00ff00ff00ff00: 00 00 00 05 fe fe fe fe fe fe fe fe fe fe fe fe
> +        *  ...
> +        *
> +        * The length of ">ff00ff00ff00ff00: " is
> +        *    3 + (BITS_PER_LONG / 8) * 2 chars.
> +        * The length of each granule metadata is 2 bytes
> +        *    plus 1 byte for space.
>          */
> -       return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 +
> -               (shadow - row) / META_BYTES_PER_BLOCK + 1;
> +       return 3 + (BITS_PER_LONG / 8) * 2 +
> +               (addr - row) / KASAN_GRANULE_SIZE * 3 + 1;
>  }
>
>  static void print_memory_metadata(const void *addr)
>  {
>         int i;
> -       const void *shadow = kasan_mem_to_shadow(addr);
> -       const void *shadow_row;
> +       void *row;
>
> -       shadow_row = (void *)round_down((unsigned long)shadow,
> -                                       META_BYTES_PER_ROW)
> -               - META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW;
> +       row = (void *)round_down((unsigned long)addr, META_MEM_BYTES_PER_ROW)
> +                       - META_ROWS_AROUND_ADDR * META_MEM_BYTES_PER_ROW;
>
>         pr_err("Memory state around the buggy address:\n");
>
>         for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) {
> -               const void *kaddr = kasan_shadow_to_mem(shadow_row);
> -               char buffer[4 + (BITS_PER_LONG/8)*2];
> -               char shadow_buf[META_BYTES_PER_ROW];
> +               char buffer[4 + (BITS_PER_LONG / 8) * 2];
> +               char metadata[META_BYTES_PER_ROW];
>
>                 snprintf(buffer, sizeof(buffer),
> -                       (i == 0) ? ">%px: " : " %px: ", kaddr);
> +                               (i == 0) ? ">%px: " : " %px: ", row);
> +
>                 /*
>                  * We should not pass a shadow pointer to generic
>                  * function, because generic functions may try to
>                  * access kasan mapping for the passed address.
>                  */
> -               memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW);
> +               metadata_fetch_row(&metadata[0], row);
> +
>                 print_hex_dump(KERN_ERR, buffer,
>                         DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1,
> -                       shadow_buf, META_BYTES_PER_ROW, 0);
> +                       metadata, META_BYTES_PER_ROW, 0);
>
> -               if (row_is_guilty(shadow_row, shadow))
> -                       pr_err("%*c\n",
> -                               shadow_pointer_offset(shadow_row, shadow),
> -                               '^');
> +               if (meta_row_is_guilty(row, addr))
> +                       pr_err("%*c\n", meta_pointer_offset(row, addr), '^');
>
> -               shadow_row += META_BYTES_PER_ROW;
> +               row += META_MEM_BYTES_PER_ROW;
>         }
>  }
>
> diff --git a/mm/kasan/report_generic.c b/mm/kasan/report_generic.c
> index 16ed550850e9..8a9c889872da 100644
> --- a/mm/kasan/report_generic.c
> +++ b/mm/kasan/report_generic.c
> @@ -123,6 +123,11 @@ const char *get_bug_type(struct kasan_access_info *info)
>         return get_wild_bug_type(info);
>  }
>
> +void metadata_fetch_row(char *buffer, void *row)
> +{
> +       memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);

I think it is important to use __memcpy() instead of memcpy() in KASAN
runtime to avoid calling instrumented code.

> +}
> +
>  #if CONFIG_KASAN_STACK
>  static bool __must_check tokenize_frame_descr(const char **frame_descr,
>                                               char *token, size_t max_tok_len,
> diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c
> index c87d5a343b4e..add2dfe6169c 100644
> --- a/mm/kasan/report_sw_tags.c
> +++ b/mm/kasan/report_sw_tags.c
> @@ -80,6 +80,11 @@ void *find_first_bad_addr(void *addr, size_t size)
>         return p;
>  }
>
> +void metadata_fetch_row(char *buffer, void *row)
> +{
> +       memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);

Ditto.

> +}
> +
>  void print_tags(u8 addr_tag, const void *addr)
>  {
>         u8 *shadow = (u8 *)kasan_mem_to_shadow(addr);
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE
  2020-11-10 22:10 ` [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE Andrey Konovalov
@ 2020-11-11 15:52   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:52 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM 'Andrey Konovalov' via kasan-dev
<kasan-dev@googlegroups.com> wrote:
>
> Software tag-based KASAN provides its own tag checking machinery that
> can conflict with MTE. Don't allow enabling software tag-based KASAN
> when MTE is enabled.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: Icd29bd0c6b1d3d7a0ee3d50c20490f404d34fc97
> ---
>  arch/arm64/Kconfig | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 1515f6f153a0..25ead11074bf 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -134,7 +134,7 @@ config ARM64
>         select HAVE_ARCH_JUMP_LABEL
>         select HAVE_ARCH_JUMP_LABEL_RELATIVE
>         select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
> -       select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
> +       select HAVE_ARCH_KASAN_SW_TAGS if (HAVE_ARCH_KASAN && !ARM64_MTE)
>         select HAVE_ARCH_KGDB
>         select HAVE_ARCH_MMAP_RND_BITS
>         select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
> --
> 2.29.2.222.g5d2a92d10f8-goog
>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/67c17dafa28036b628234c8f1d88368af374449c.1605046192.git.andreyknvl%40google.com.



-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS
  2020-11-10 22:10 ` [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS Andrey Konovalov
@ 2020-11-11 15:58   ` Alexander Potapenko
  2020-11-11 18:53     ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 15:58 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> This patch adds a configuration option for a new KASAN mode called
> hardware tag-based KASAN. This mode uses the memory tagging approach
> like the software tag-based mode, but relies on arm64 Memory Tagging
> Extension feature for tag management and access checking.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> ---
> Change-Id: I246c2def9fffa6563278db1bddfbe742ca7bdefe
> ---
>  lib/Kconfig.kasan | 58 +++++++++++++++++++++++++++++++++--------------
>  1 file changed, 41 insertions(+), 17 deletions(-)
>
> diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
> index ec59a0e26d09..e5f27ec8b254 100644
> --- a/lib/Kconfig.kasan
> +++ b/lib/Kconfig.kasan
> @@ -6,7 +6,10 @@ config HAVE_ARCH_KASAN
>  config HAVE_ARCH_KASAN_SW_TAGS
>         bool
>
> -config HAVE_ARCH_KASAN_VMALLOC
> +config HAVE_ARCH_KASAN_HW_TAGS
> +       bool
> +
> +config HAVE_ARCH_KASAN_VMALLOC
>         bool
>
>  config CC_HAS_KASAN_GENERIC
> @@ -20,11 +23,11 @@ config CC_HAS_WORKING_NOSANITIZE_ADDRESS
It might make sense to add a comment to
CC_HAS_WORKING_NOSANITIZE_ADDRESS describing which modes need it (and
why).


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS
  2020-11-10 22:10 ` [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS Andrey Konovalov
@ 2020-11-11 16:10   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:10 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Hardware tag-based KASAN uses the memory tagging approach, which requires
> all allocations to be aligned to the memory granule size. Align the
> allocations to MTE_GRANULE_SIZE via ARCH_SLAB_MINALIGN when
> CONFIG_KASAN_HW_TAGS is enabled.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I51ebd3f9645e6330e5a92973bf7c86b62d632c2b
> ---
>  arch/arm64/include/asm/cache.h | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h
> index 63d43b5f82f6..77cbbe3625f2 100644
> --- a/arch/arm64/include/asm/cache.h
> +++ b/arch/arm64/include/asm/cache.h
> @@ -6,6 +6,7 @@
>  #define __ASM_CACHE_H
>
>  #include <asm/cputype.h>
> +#include <asm/mte-kasan.h>
>
>  #define CTR_L1IP_SHIFT         14
>  #define CTR_L1IP_MASK          3
> @@ -51,6 +52,8 @@
>
>  #ifdef CONFIG_KASAN_SW_TAGS
>  #define ARCH_SLAB_MINALIGN     (1ULL << KASAN_SHADOW_SCALE_SHIFT)
> +#elif defined(CONFIG_KASAN_HW_TAGS)
> +#define ARCH_SLAB_MINALIGN     MTE_GRANULE_SIZE
>  #endif
>
>  #ifndef __ASSEMBLY__
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS
  2020-11-10 22:10 ` [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrey Konovalov
@ 2020-11-11 16:13   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:13 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Hardware tag-based KASAN has granules of MTE_GRANULE_SIZE. Define
> KASAN_GRANULE_SIZE to MTE_GRANULE_SIZE for CONFIG_KASAN_HW_TAGS.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I5d1117e6a991cbca00d2cfb4ba66e8ae2d8f513a
> ---
>  mm/kasan/kasan.h | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index ae7def3b725b..d745a78745dd 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -5,7 +5,13 @@
>  #include <linux/kasan.h>
>  #include <linux/stackdepot.h>
>
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  #define KASAN_GRANULE_SIZE     (1UL << KASAN_SHADOW_SCALE_SHIFT)
> +#else
> +#include <asm/mte-kasan.h>
> +#define KASAN_GRANULE_SIZE     MTE_GRANULE_SIZE
> +#endif
> +
>  #define KASAN_GRANULE_MASK     (KASAN_GRANULE_SIZE - 1)
>  #define KASAN_GRANULE_PAGE     (KASAN_GRANULE_SIZE << PAGE_SHIFT)
>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN
  2020-11-10 22:10 ` [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN Andrey Konovalov
@ 2020-11-11 16:15   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:15 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML, Vasily Gorbik

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> With the intoduction of hardware tag-based KASAN some kernel checks of
> this kind:
>
>   ifdef CONFIG_KASAN
>
> will be updated to:
>
>   if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>
> x86 and s390 use a trick to #undef CONFIG_KASAN for some of the code
> that isn't linked with KASAN runtime and shouldn't have any KASAN
> annotations.
>
> Also #undef CONFIG_KASAN_GENERIC with CONFIG_KASAN.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I2a622db0cb86a8feb60c30d8cb09190075be2a90
> ---
>  arch/s390/boot/string.c         | 1 +
>  arch/x86/boot/compressed/misc.h | 1 +
>  2 files changed, 2 insertions(+)
>
> diff --git a/arch/s390/boot/string.c b/arch/s390/boot/string.c
> index b11e8108773a..faccb33b462c 100644
> --- a/arch/s390/boot/string.c
> +++ b/arch/s390/boot/string.c
> @@ -3,6 +3,7 @@
>  #include <linux/kernel.h>
>  #include <linux/errno.h>
>  #undef CONFIG_KASAN
> +#undef CONFIG_KASAN_GENERIC
>  #include "../lib/string.c"
>
>  int strncmp(const char *cs, const char *ct, size_t count)
> diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
> index d9a631c5973c..901ea5ebec22 100644
> --- a/arch/x86/boot/compressed/misc.h
> +++ b/arch/x86/boot/compressed/misc.h
> @@ -12,6 +12,7 @@
>  #undef CONFIG_PARAVIRT_XXL
>  #undef CONFIG_PARAVIRT_SPINLOCKS
>  #undef CONFIG_KASAN
> +#undef CONFIG_KASAN_GENERIC
>
>  /* cpu_feature_enabled() cannot be used this early */
>  #define USE_EARLY_PGTABLE_L5
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode
  2020-11-11 15:22   ` Alexander Potapenko
@ 2020-11-11 16:19     ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:19 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 4:22 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > This is a preparatory commit for the upcoming addition of a new hardware
> > tag-based (MTE-based) KASAN mode.
> >
> > Rework print_memory_metadata() to make it agnostic with regard to the
> > way metadata is stored. Allow providing a separate metadata_fetch_row()
> > implementation for each KASAN mode. Hardware tag-based KASAN will provide
> > its own implementation that doesn't use shadow memory.
> >
> > No functional changes for software modes.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> > +void metadata_fetch_row(char *buffer, void *row)
> > +{
> > +       memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW);
>
> I think it is important to use __memcpy() instead of memcpy() in KASAN
> runtime to avoid calling instrumented code.

Please disregard this. Turns out we define memcpy to __memcpy for
non-instrumented files.


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks
  2020-11-10 22:10 ` [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks Andrey Konovalov
@ 2020-11-11 16:22   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:22 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN
> modes (either related to shadow memory or compiler instrumentation).
> Expand those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I91e661e2c1627783cb845d877c6371dfc8779505
> ---
>  arch/arm64/Kconfig                 |  2 +-
>  arch/arm64/Makefile                |  2 +-
>  arch/arm64/include/asm/assembler.h |  2 +-
>  arch/arm64/include/asm/memory.h    |  2 +-
>  arch/arm64/include/asm/string.h    |  5 +++--
>  arch/arm64/kernel/head.S           |  2 +-
>  arch/arm64/kernel/image-vars.h     |  2 +-
>  arch/arm64/kernel/kaslr.c          |  3 ++-
>  arch/arm64/kernel/module.c         |  6 ++++--
>  arch/arm64/mm/ptdump.c             |  6 +++---
>  include/linux/kasan-checks.h       |  2 +-
>  include/linux/kasan.h              |  7 ++++---
>  include/linux/moduleloader.h       |  3 ++-
>  include/linux/string.h             |  2 +-
>  mm/ptdump.c                        | 13 ++++++++-----
>  scripts/Makefile.lib               |  2 ++
>  16 files changed, 36 insertions(+), 25 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index c84a0e6b4650..456741645f01 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -330,7 +330,7 @@ config BROKEN_GAS_INST
>
>  config KASAN_SHADOW_OFFSET
>         hex
> -       depends on KASAN
> +       depends on KASAN_GENERIC || KASAN_SW_TAGS
>         default 0xdfffa00000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && !KASAN_SW_TAGS
>         default 0xdfffd00000000000 if ARM64_VA_BITS_47 && !KASAN_SW_TAGS
>         default 0xdffffe8000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 50ad9cbccb51..0b31a3f06f15 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -141,7 +141,7 @@ head-y              := arch/arm64/kernel/head.o
>
>  ifeq ($(CONFIG_KASAN_SW_TAGS), y)
>  KASAN_SHADOW_SCALE_SHIFT := 4
> -else
> +else ifeq ($(CONFIG_KASAN_GENERIC), y)
>  KASAN_SHADOW_SCALE_SHIFT := 3
>  endif
>
> diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
> index ddbe6bf00e33..bf125c591116 100644
> --- a/arch/arm64/include/asm/assembler.h
> +++ b/arch/arm64/include/asm/assembler.h
> @@ -473,7 +473,7 @@ USER(\label, ic     ivau, \tmp2)                    // invalidate I line PoU
>  #define NOKPROBE(x)
>  #endif
>
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  #define EXPORT_SYMBOL_NOKASAN(name)
>  #else
>  #define EXPORT_SYMBOL_NOKASAN(name)    EXPORT_SYMBOL(name)
> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
> index 419bbace29d5..656aaddb7014 100644
> --- a/arch/arm64/include/asm/memory.h
> +++ b/arch/arm64/include/asm/memory.h
> @@ -72,7 +72,7 @@
>   * address space for the shadow region respectively. They can bloat the stack
>   * significantly, so double the (minimum) stack size when they are in use.
>   */
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  #define KASAN_SHADOW_OFFSET    _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
>  #define KASAN_SHADOW_END       ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) \
>                                         + KASAN_SHADOW_OFFSET)
> diff --git a/arch/arm64/include/asm/string.h b/arch/arm64/include/asm/string.h
> index b31e8e87a0db..3a3264ff47b9 100644
> --- a/arch/arm64/include/asm/string.h
> +++ b/arch/arm64/include/asm/string.h
> @@ -5,7 +5,7 @@
>  #ifndef __ASM_STRING_H
>  #define __ASM_STRING_H
>
> -#ifndef CONFIG_KASAN
> +#if !(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
>  #define __HAVE_ARCH_STRRCHR
>  extern char *strrchr(const char *, int c);
>
> @@ -48,7 +48,8 @@ extern void *__memset(void *, int, __kernel_size_t);
>  void memcpy_flushcache(void *dst, const void *src, size_t cnt);
>  #endif
>
> -#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
> +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> +       !defined(__SANITIZE_ADDRESS__)
>
>  /*
>   * For files that are not instrumented (e.g. mm/slub.c) we
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index d8d9caf02834..fdcb99d7ba23 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -448,7 +448,7 @@ SYM_FUNC_START_LOCAL(__primary_switched)
>         bl      __pi_memset
>         dsb     ishst                           // Make zero page visible to PTW
>
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         bl      kasan_early_init
>  #endif
>  #ifdef CONFIG_RANDOMIZE_BASE
> diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
> index c615b285ff5b..4282edd2fe81 100644
> --- a/arch/arm64/kernel/image-vars.h
> +++ b/arch/arm64/kernel/image-vars.h
> @@ -37,7 +37,7 @@ __efistub_strncmp             = __pi_strncmp;
>  __efistub_strrchr              = __pi_strrchr;
>  __efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc;
>
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  __efistub___memcpy             = __pi_memcpy;
>  __efistub___memmove            = __pi_memmove;
>  __efistub___memset             = __pi_memset;
> diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c
> index b181e0544b79..e8e17e91aa02 100644
> --- a/arch/arm64/kernel/kaslr.c
> +++ b/arch/arm64/kernel/kaslr.c
> @@ -151,7 +151,8 @@ u64 __init kaslr_early_init(u64 dt_phys)
>         /* use the top 16 bits to randomize the linear region */
>         memstart_offset_seed = seed >> 48;
>
> -       if (IS_ENABLED(CONFIG_KASAN))
> +       if (IS_ENABLED(CONFIG_KASAN_GENERIC) ||
> +           IS_ENABLED(CONFIG_KASAN_SW_TAGS))
>                 /*
>                  * KASAN does not expect the module region to intersect the
>                  * vmalloc region, since shadow memory is allocated for each
> diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
> index 2a1ad95d9b2c..fe21e0f06492 100644
> --- a/arch/arm64/kernel/module.c
> +++ b/arch/arm64/kernel/module.c
> @@ -30,7 +30,8 @@ void *module_alloc(unsigned long size)
>         if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS))
>                 gfp_mask |= __GFP_NOWARN;
>
> -       if (IS_ENABLED(CONFIG_KASAN))
> +       if (IS_ENABLED(CONFIG_KASAN_GENERIC) ||
> +           IS_ENABLED(CONFIG_KASAN_SW_TAGS))
>                 /* don't exceed the static module region - see below */
>                 module_alloc_end = MODULES_END;
>
> @@ -39,7 +40,8 @@ void *module_alloc(unsigned long size)
>                                 NUMA_NO_NODE, __builtin_return_address(0));
>
>         if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) &&
> -           !IS_ENABLED(CONFIG_KASAN))
> +           !IS_ENABLED(CONFIG_KASAN_GENERIC) &&
> +           !IS_ENABLED(CONFIG_KASAN_SW_TAGS))
>                 /*
>                  * KASAN can only deal with module allocations being served
>                  * from the reserved module region, since the remainder of
> diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
> index 807dc634bbd2..04137a8f3d2d 100644
> --- a/arch/arm64/mm/ptdump.c
> +++ b/arch/arm64/mm/ptdump.c
> @@ -29,7 +29,7 @@
>  enum address_markers_idx {
>         PAGE_OFFSET_NR = 0,
>         PAGE_END_NR,
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         KASAN_START_NR,
>  #endif
>  };
> @@ -37,7 +37,7 @@ enum address_markers_idx {
>  static struct addr_marker address_markers[] = {
>         { PAGE_OFFSET,                  "Linear Mapping start" },
>         { 0 /* PAGE_END */,             "Linear Mapping end" },
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         { 0 /* KASAN_SHADOW_START */,   "Kasan shadow start" },
>         { KASAN_SHADOW_END,             "Kasan shadow end" },
>  #endif
> @@ -383,7 +383,7 @@ void ptdump_check_wx(void)
>  static int ptdump_init(void)
>  {
>         address_markers[PAGE_END_NR].start_address = PAGE_END;
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         address_markers[KASAN_START_NR].start_address = KASAN_SHADOW_START;
>  #endif
>         ptdump_initialize();
> diff --git a/include/linux/kasan-checks.h b/include/linux/kasan-checks.h
> index ac6aba632f2d..ca5e89fb10d3 100644
> --- a/include/linux/kasan-checks.h
> +++ b/include/linux/kasan-checks.h
> @@ -9,7 +9,7 @@
>   * even in compilation units that selectively disable KASAN, but must use KASAN
>   * to validate access to an address.   Never use these in header files!
>   */
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  bool __kasan_check_read(const volatile void *p, unsigned int size);
>  bool __kasan_check_write(const volatile void *p, unsigned int size);
>  #else
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index 1d6ec3325163..b6fc14b3da53 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -237,7 +237,8 @@ static inline void kasan_release_vmalloc(unsigned long start,
>
>  #endif /* CONFIG_KASAN_VMALLOC */
>
> -#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
> +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> +               !defined(CONFIG_KASAN_VMALLOC)
>
>  /*
>   * These functions provide a special case to support backing module
> @@ -247,12 +248,12 @@ static inline void kasan_release_vmalloc(unsigned long start,
>  int kasan_module_alloc(void *addr, size_t size);
>  void kasan_free_shadow(const struct vm_struct *vm);
>
> -#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
> +#else /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */
>
>  static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
>  static inline void kasan_free_shadow(const struct vm_struct *vm) {}
>
> -#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */
> +#endif /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */
>
>  #ifdef CONFIG_KASAN_INLINE
>  void kasan_non_canonical_hook(unsigned long addr);
> diff --git a/include/linux/moduleloader.h b/include/linux/moduleloader.h
> index 4fa67a8b2265..9e09d11ffe5b 100644
> --- a/include/linux/moduleloader.h
> +++ b/include/linux/moduleloader.h
> @@ -96,7 +96,8 @@ void module_arch_cleanup(struct module *mod);
>  /* Any cleanup before freeing mod->module_init */
>  void module_arch_freeing_init(struct module *mod);
>
> -#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC)
> +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> +               !defined(CONFIG_KASAN_VMALLOC)
>  #include <linux/kasan.h>
>  #define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT)
>  #else
> diff --git a/include/linux/string.h b/include/linux/string.h
> index b1f3894a0a3e..016a157e2251 100644
> --- a/include/linux/string.h
> +++ b/include/linux/string.h
> @@ -266,7 +266,7 @@ void __write_overflow(void) __compiletime_error("detected write beyond size of o
>
>  #if !defined(__NO_FORTIFY) && defined(__OPTIMIZE__) && defined(CONFIG_FORTIFY_SOURCE)
>
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  extern void *__underlying_memchr(const void *p, int c, __kernel_size_t size) __RENAME(memchr);
>  extern int __underlying_memcmp(const void *p, const void *q, __kernel_size_t size) __RENAME(memcmp);
>  extern void *__underlying_memcpy(void *p, const void *q, __kernel_size_t size) __RENAME(memcpy);
> diff --git a/mm/ptdump.c b/mm/ptdump.c
> index ba88ec43ff21..4354c1422d57 100644
> --- a/mm/ptdump.c
> +++ b/mm/ptdump.c
> @@ -4,7 +4,7 @@
>  #include <linux/ptdump.h>
>  #include <linux/kasan.h>
>
> -#ifdef CONFIG_KASAN
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>  /*
>   * This is an optimization for KASAN=y case. Since all kasan page tables
>   * eventually point to the kasan_early_shadow_page we could call note_page()
> @@ -31,7 +31,8 @@ static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr,
>         struct ptdump_state *st = walk->private;
>         pgd_t val = READ_ONCE(*pgd);
>
> -#if CONFIG_PGTABLE_LEVELS > 4 && defined(CONFIG_KASAN)
> +#if CONFIG_PGTABLE_LEVELS > 4 && \
> +               (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
>         if (pgd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_p4d)))
>                 return note_kasan_page_table(walk, addr);
>  #endif
> @@ -51,7 +52,8 @@ static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr,
>         struct ptdump_state *st = walk->private;
>         p4d_t val = READ_ONCE(*p4d);
>
> -#if CONFIG_PGTABLE_LEVELS > 3 && defined(CONFIG_KASAN)
> +#if CONFIG_PGTABLE_LEVELS > 3 && \
> +               (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
>         if (p4d_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pud)))
>                 return note_kasan_page_table(walk, addr);
>  #endif
> @@ -71,7 +73,8 @@ static int ptdump_pud_entry(pud_t *pud, unsigned long addr,
>         struct ptdump_state *st = walk->private;
>         pud_t val = READ_ONCE(*pud);
>
> -#if CONFIG_PGTABLE_LEVELS > 2 && defined(CONFIG_KASAN)
> +#if CONFIG_PGTABLE_LEVELS > 2 && \
> +               (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS))
>         if (pud_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pmd)))
>                 return note_kasan_page_table(walk, addr);
>  #endif
> @@ -91,7 +94,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr,
>         struct ptdump_state *st = walk->private;
>         pmd_t val = READ_ONCE(*pmd);
>
> -#if defined(CONFIG_KASAN)
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
>         if (pmd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pte)))
>                 return note_kasan_page_table(walk, addr);
>  #endif
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 94133708889d..213677a5ed33 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -148,10 +148,12 @@ endif
>  # we don't want to check (depends on variables KASAN_SANITIZE_obj.o, KASAN_SANITIZE)
>  #
>  ifeq ($(CONFIG_KASAN),y)
> +ifneq ($(CONFIG_KASAN_HW_TAGS),y)
>  _c_flags += $(if $(patsubst n%,, \
>                 $(KASAN_SANITIZE_$(basetarget).o)$(KASAN_SANITIZE)y), \
>                 $(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE))
>  endif
> +endif
>
>  ifeq ($(CONFIG_UBSAN),y)
>  _c_flags += $(if $(patsubst n%,, \
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 40/44] kasan, arm64: print report from tag fault handler
  2020-11-10 22:10 ` [PATCH v9 40/44] kasan, arm64: print report from tag fault handler Andrey Konovalov
@ 2020-11-11 16:43   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:43 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM 'Andrey Konovalov' via kasan-dev
<kasan-dev@googlegroups.com> wrote:
>
> Add error reporting for hardware tag-based KASAN. When CONFIG_KASAN_HW_TAGS
> is enabled, print KASAN report from the arm64 tag fault handler.
>
> SAS bits aren't set in ESR for all faults reported in EL1, so it's
> impossible to find out the size of the access the caused the fault.
> Adapt KASAN reporting code to handle this case.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I3780fe7db6e075dff2937d3d8508f55c9322b095
> ---
>  arch/arm64/mm/fault.c | 14 ++++++++++++++
>  mm/kasan/report.c     | 11 ++++++++---
>  2 files changed, 22 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
> index fbceb14d93b1..7370e822e588 100644
> --- a/arch/arm64/mm/fault.c
> +++ b/arch/arm64/mm/fault.c
> @@ -14,6 +14,7 @@
>  #include <linux/mm.h>
>  #include <linux/hardirq.h>
>  #include <linux/init.h>
> +#include <linux/kasan.h>
>  #include <linux/kprobes.h>
>  #include <linux/uaccess.h>
>  #include <linux/page-flags.h>
> @@ -297,10 +298,23 @@ static void die_kernel_fault(const char *msg, unsigned long addr,
>         do_exit(SIGKILL);
>  }
>
> +#ifdef CONFIG_KASAN_HW_TAGS
>  static void report_tag_fault(unsigned long addr, unsigned int esr,
>                              struct pt_regs *regs)
>  {
> +       bool is_write  = ((esr & ESR_ELx_WNR) >> ESR_ELx_WNR_SHIFT) != 0;
> +
> +       /*
> +        * SAS bits aren't set for all faults reported in EL1, so we can't
> +        * find out access size.
> +        */
> +       kasan_report(addr, 0, is_write, regs->pc);
>  }
> +#else
> +/* Tag faults aren't enabled without CONFIG_KASAN_HW_TAGS. */
> +static inline void report_tag_fault(unsigned long addr, unsigned int esr,
> +                                   struct pt_regs *regs) { }
> +#endif
>
>  static void do_tag_recovery(unsigned long addr, unsigned int esr,
>                            struct pt_regs *regs)
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 8afc1a6ab202..ce06005d4052 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -62,9 +62,14 @@ static void print_error_description(struct kasan_access_info *info)
>  {
>         pr_err("BUG: KASAN: %s in %pS\n",
>                 get_bug_type(info), (void *)info->ip);
> -       pr_err("%s of size %zu at addr %px by task %s/%d\n",
> -               info->is_write ? "Write" : "Read", info->access_size,
> -               info->access_addr, current->comm, task_pid_nr(current));
> +       if (info->access_size)
> +               pr_err("%s of size %zu at addr %px by task %s/%d\n",
> +                       info->is_write ? "Write" : "Read", info->access_size,
> +                       info->access_addr, current->comm, task_pid_nr(current));
> +       else
> +               pr_err("%s at addr %px by task %s/%d\n",
> +                       info->is_write ? "Write" : "Read",
> +                       info->access_addr, current->comm, task_pid_nr(current));
>  }
>
>  static DEFINE_SPINLOCK(report_lock);
> --
> 2.29.2.222.g5d2a92d10f8-goog
>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/fe78d723ba64456d68754a944fa93fe4a25c730f.1605046192.git.andreyknvl%40google.com.



-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS
  2020-11-10 22:10 ` [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrey Konovalov
@ 2020-11-11 16:49   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:49 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Hardware tag-based KASAN is now ready, enable the configuration option.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I6eb1eea770e6b61ad71c701231b8d815a7ccc853
> ---
>  arch/arm64/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 456741645f01..c35e73efd407 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -135,6 +135,7 @@ config ARM64
>         select HAVE_ARCH_JUMP_LABEL_RELATIVE
>         select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
>         select HAVE_ARCH_KASAN_SW_TAGS if (HAVE_ARCH_KASAN && !ARM64_MTE)
> +       select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
>         select HAVE_ARCH_KGDB
>         select HAVE_ARCH_MMAP_RND_BITS
>         select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode
  2020-11-10 22:10 ` [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode Andrey Konovalov
@ 2020-11-11 16:57   ` Alexander Potapenko
  2020-11-11 18:54     ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-11 16:57 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Add documentation for hardware tag-based KASAN mode and also add some
> clarifications for software tag-based mode.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Reviewed-by: Marco Elver <elver@google.com>
> ---
> Change-Id: Ib46cb444cfdee44054628940a82f5139e10d0258

> +
> +Software tag-based KASAN currently only supports tagging of slab memory.

I think the reader may confuse "slab memory" here with "memory
returned by SLAB" (as opposed to SLUB).
Maybe "heap memory" is less ambiguous?


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE
  2020-11-11 14:13   ` Alexander Potapenko
@ 2020-11-11 18:47     ` Andrey Konovalov
  2020-11-11 19:04       ` Marco Elver
  0 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-11 18:47 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 3:13 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > Define KASAN_GRANULE_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT), which is
> > the same as (KASAN_GRANULE_SIZE * PAGE_SIZE), and use it across KASAN code
> > to simplify it.
>
> What's the physical sense behind KASAN_GRANULE_PAGE? Is it something
> more than just a product of two constants?

No, just a product.

> The name suggests it might be something page-sized, but in reality it is not.

What name would you prefer?


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 17/44] kasan, arm64: move initialization message
  2020-11-11 15:03   ` Alexander Potapenko
@ 2020-11-11 18:50     ` Andrey Konovalov
  2020-11-12 15:09       ` Alexander Potapenko
  0 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-11 18:50 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 4:04 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > Software tag-based KASAN mode is fully initialized with kasan_init_tags(),
> > while the generic mode only requires kasan_init(). Move the
> > initialization message for tag-based mode into kasan_init_tags().
> >
> > Also fix pr_fmt() usage for KASAN code: generic.c doesn't need it as it
> > doesn't use any printing functions; tag-based mode should use "kasan:"
> > instead of KBUILD_MODNAME (which stands for file name).
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
> > ---
> > Change-Id: Iddca9764b30ff0fab1922f26ca9d4f39b6f22673
> > ---
> >  arch/arm64/include/asm/kasan.h |  9 +++------
> >  arch/arm64/mm/kasan_init.c     | 13 +++++--------
> >  mm/kasan/generic.c             |  2 --
> >  mm/kasan/sw_tags.c             |  4 +++-
> >  4 files changed, 11 insertions(+), 17 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h
> > index f7ea70d02cab..0aaf9044cd6a 100644
> > --- a/arch/arm64/include/asm/kasan.h
> > +++ b/arch/arm64/include/asm/kasan.h
> > @@ -12,14 +12,10 @@
> >  #define arch_kasan_reset_tag(addr)     __tag_reset(addr)
> >  #define arch_kasan_get_tag(addr)       __tag_get(addr)
> >
> > -#ifdef CONFIG_KASAN
> > -void kasan_init(void);
> > -#else
> > -static inline void kasan_init(void) { }
> > -#endif
> > -
> >  #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
> >
> > +void kasan_init(void);
> > +
> >  /*
> >   * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
> >   * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses,
> > @@ -43,6 +39,7 @@ void kasan_copy_shadow(pgd_t *pgdir);
> >  asmlinkage void kasan_early_init(void);
> >
> >  #else
> > +static inline void kasan_init(void) { }
> >  static inline void kasan_copy_shadow(pgd_t *pgdir) { }
> >  #endif
> >
> > diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c
> > index 5172799f831f..e35ce04beed1 100644
> > --- a/arch/arm64/mm/kasan_init.c
> > +++ b/arch/arm64/mm/kasan_init.c
> > @@ -278,17 +278,14 @@ static void __init kasan_init_depth(void)
> >         init_task.kasan_depth = 0;
> >  }
> >
> > -#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */
> > -
> > -static inline void __init kasan_init_shadow(void) { }
> > -
> > -static inline void __init kasan_init_depth(void) { }
> > -
> > -#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> > -
> >  void __init kasan_init(void)
> >  {
> >         kasan_init_shadow();
> >         kasan_init_depth();
> > +#if defined(CONFIG_KASAN_GENERIC)
> > +       /* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */
> >         pr_info("KernelAddressSanitizer initialized\n");
> > +#endif
> >  }
>
> Cannot we have a single kasan_init() function that will call
> tool-specific initialization functions and print the message at the
> end?

Unfortunately no. For different modes we need different functions that
are called in different places in the kernel. E.g. for generic KASAN
we only need kasan_init() to setup shadow pages; for SW tags we also
need kasan_init_sw_tags() which initializes per-cpu state and
finilizes initialization process.


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes
  2020-11-11 15:08   ` Alexander Potapenko
@ 2020-11-11 18:52     ` Andrey Konovalov
  2020-11-12 15:16       ` Alexander Potapenko
  0 siblings, 1 reply; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-11 18:52 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 4:09 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > This is a preparatory commit for the upcoming addition of a new hardware
> > tag-based (MTE-based) KASAN mode.
> >
> > kasan_non_canonical_hook() is only applicable to KASAN modes that use
> > shadow memory, and won't be needed for hardware tag-based KASAN.
> >
> > No functional changes for software modes.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > Reviewed-by: Marco Elver <elver@google.com>
> > ---
> > Change-Id: Icc9f5ef100a2e86f3a4214a0c3131a68266181b2
> > ---
> >  mm/kasan/report.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> > index 5d5733831ad7..594bad2a3a5e 100644
> > --- a/mm/kasan/report.c
> > +++ b/mm/kasan/report.c
> > @@ -403,7 +403,8 @@ bool kasan_report(unsigned long addr, size_t size, bool is_write,
> >         return ret;
> >  }
> >
> > -#ifdef CONFIG_KASAN_INLINE
> > +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> > +       defined(CONFIG_KASAN_INLINE)
> >  /*
> >   * With CONFIG_KASAN_INLINE, accesses to bogus pointers (outside the high
> >   * canonical half of the address space) cause out-of-bounds shadow memory reads
>
> Perhaps this comment also needs to be updated.

In what way?


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS
  2020-11-11 15:58   ` Alexander Potapenko
@ 2020-11-11 18:53     ` Andrey Konovalov
  0 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-11 18:53 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 4:58 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > This patch adds a configuration option for a new KASAN mode called
> > hardware tag-based KASAN. This mode uses the memory tagging approach
> > like the software tag-based mode, but relies on arm64 Memory Tagging
> > Extension feature for tag management and access checking.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > Co-developed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > Reviewed-by: Marco Elver <elver@google.com>
> > ---
> > Change-Id: I246c2def9fffa6563278db1bddfbe742ca7bdefe
> > ---
> >  lib/Kconfig.kasan | 58 +++++++++++++++++++++++++++++++++--------------
> >  1 file changed, 41 insertions(+), 17 deletions(-)
> >
> > diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
> > index ec59a0e26d09..e5f27ec8b254 100644
> > --- a/lib/Kconfig.kasan
> > +++ b/lib/Kconfig.kasan
> > @@ -6,7 +6,10 @@ config HAVE_ARCH_KASAN
> >  config HAVE_ARCH_KASAN_SW_TAGS
> >         bool
> >
> > -config HAVE_ARCH_KASAN_VMALLOC
> > +config HAVE_ARCH_KASAN_HW_TAGS
> > +       bool
> > +
> > +config HAVE_ARCH_KASAN_VMALLOC
> >         bool
> >
> >  config CC_HAS_KASAN_GENERIC
> > @@ -20,11 +23,11 @@ config CC_HAS_WORKING_NOSANITIZE_ADDRESS
> It might make sense to add a comment to
> CC_HAS_WORKING_NOSANITIZE_ADDRESS describing which modes need it (and
> why).

OK, will do in v10, thanks!


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode
  2020-11-11 16:57   ` Alexander Potapenko
@ 2020-11-11 18:54     ` Andrey Konovalov
  0 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-11 18:54 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 5:57 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > Add documentation for hardware tag-based KASAN mode and also add some
> > clarifications for software tag-based mode.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > Reviewed-by: Marco Elver <elver@google.com>
> > ---
> > Change-Id: Ib46cb444cfdee44054628940a82f5139e10d0258
>
> > +
> > +Software tag-based KASAN currently only supports tagging of slab memory.
>
> I think the reader may confuse "slab memory" here with "memory
> returned by SLAB" (as opposed to SLUB).
> Maybe "heap memory" is less ambiguous?

I think heap memory isn't widely used in the kernel context. But I'll
clarify this part in v10, thanks!


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE
  2020-11-11 18:47     ` Andrey Konovalov
@ 2020-11-11 19:04       ` Marco Elver
  2020-11-12  0:05         ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Marco Elver @ 2020-11-11 19:04 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Alexander Potapenko, Catalin Marinas, Will Deacon,
	Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Evgenii Stepanov, Branislav Rankov, Kevin Brodsky, Andrew Morton,
	kasan-dev, Linux ARM, Linux Memory Management List, LKML

On Wed, 11 Nov 2020 at 19:48, Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Wed, Nov 11, 2020 at 3:13 PM Alexander Potapenko <glider@google.com> wrote:
> >
> > On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> > >
> > > Define KASAN_GRANULE_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT), which is
> > > the same as (KASAN_GRANULE_SIZE * PAGE_SIZE), and use it across KASAN code
> > > to simplify it.
> >
> > What's the physical sense behind KASAN_GRANULE_PAGE? Is it something
> > more than just a product of two constants?
>
> No, just a product.
>
> > The name suggests it might be something page-sized, but in reality it is not.
>
> What name would you prefer?

Is it actually KASAN_GRANULES_PER_SHADOW_PAGE ?   AFAIK we're trying
to calculate the granules that we can fit into a page of shadow
memory.


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE
  2020-11-11 19:04       ` Marco Elver
@ 2020-11-12  0:05         ` Andrey Konovalov
  0 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-12  0:05 UTC (permalink / raw)
  To: Marco Elver
  Cc: Alexander Potapenko, Catalin Marinas, Will Deacon,
	Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Evgenii Stepanov, Branislav Rankov, Kevin Brodsky, Andrew Morton,
	kasan-dev, Linux ARM, Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 8:05 PM Marco Elver <elver@google.com> wrote:
>
> On Wed, 11 Nov 2020 at 19:48, Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > On Wed, Nov 11, 2020 at 3:13 PM Alexander Potapenko <glider@google.com> wrote:
> > >
> > > On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> > > >
> > > > Define KASAN_GRANULE_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT), which is
> > > > the same as (KASAN_GRANULE_SIZE * PAGE_SIZE), and use it across KASAN code
> > > > to simplify it.
> > >
> > > What's the physical sense behind KASAN_GRANULE_PAGE? Is it something
> > > more than just a product of two constants?
> >
> > No, just a product.
> >
> > > The name suggests it might be something page-sized, but in reality it is not.
> >
> > What name would you prefer?
>
> Is it actually KASAN_GRANULES_PER_SHADOW_PAGE ?   AFAIK we're trying
> to calculate the granules that we can fit into a page of shadow
> memory.

Not exactly, it's the amount of memory, not the number of granules.

Will name it KASAN_MEMORY_PER_SHADOW_PAGE in v10.


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags
  2020-11-10 22:10 ` [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags Andrey Konovalov
@ 2020-11-12  9:31   ` Catalin Marinas
  2020-11-12  9:39     ` Vincenzo Frascino
  0 siblings, 1 reply; 107+ messages in thread
From: Catalin Marinas @ 2020-11-12  9:31 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Tue, Nov 10, 2020 at 11:10:25PM +0100, Andrey Konovalov wrote:
> diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c
> index 70a71f38b6a9..f0efa4847e2f 100644
> --- a/arch/arm64/mm/copypage.c
> +++ b/arch/arm64/mm/copypage.c
> @@ -23,6 +23,7 @@ void copy_highpage(struct page *to, struct page *from)
>  
>  	if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) {
>  		set_bit(PG_mte_tagged, &to->flags);
> +		page_kasan_tag_reset(to);
>  		mte_copy_page_tags(kto, kfrom);

Any reason why this doesn't have an smp_wmb() between resetting the tags
and copying them into kto?

-- 
Catalin


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit
  2020-11-10 22:10 ` [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit Andrey Konovalov
@ 2020-11-12  9:39   ` Catalin Marinas
  2020-11-12  9:45     ` Vincenzo Frascino
  0 siblings, 1 reply; 107+ messages in thread
From: Catalin Marinas @ 2020-11-12  9:39 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Tue, Nov 10, 2020 at 11:10:29PM +0100, Andrey Konovalov wrote:
> diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
> index 664c968dc43c..dbda6598c19d 100644
> --- a/arch/arm64/kernel/mte.c
> +++ b/arch/arm64/kernel/mte.c
> @@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
>  	return ptr;
>  }
>  
> +void mte_init_tags(u64 max_tag)
> +{
> +	static bool gcr_kernel_excl_initialized = false;
> +
> +	if (!gcr_kernel_excl_initialized) {
> +		/*
> +		 * The format of the tags in KASAN is 0xFF and in MTE is 0xF.
> +		 * This conversion extracts an MTE tag from a KASAN tag.
> +		 */
> +		u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT,
> +					     max_tag), 0);
> +
> +		gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
> +		gcr_kernel_excl_initialized = true;
> +	}
> +
> +	/* Enable the kernel exclude mask for random tags generation. */
> +	write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1);
> +}

I don't think this function belongs to this patch. There is an earlier
patch that talks about mte_init_tags() but no trace of it until this
patch.

-- 
Catalin


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags
  2020-11-12  9:31   ` Catalin Marinas
@ 2020-11-12  9:39     ` Vincenzo Frascino
  0 siblings, 0 replies; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-12  9:39 UTC (permalink / raw)
  To: Catalin Marinas, Andrey Konovalov
  Cc: Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko,
	Marco Elver, Evgenii Stepanov, Branislav Rankov, Kevin Brodsky,
	Andrew Morton, kasan-dev, linux-arm-kernel, linux-mm,
	linux-kernel

On 11/12/20 9:31 AM, Catalin Marinas wrote:
> On Tue, Nov 10, 2020 at 11:10:25PM +0100, Andrey Konovalov wrote:
>> diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c
>> index 70a71f38b6a9..f0efa4847e2f 100644
>> --- a/arch/arm64/mm/copypage.c
>> +++ b/arch/arm64/mm/copypage.c
>> @@ -23,6 +23,7 @@ void copy_highpage(struct page *to, struct page *from)
>>  
>>  	if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) {
>>  		set_bit(PG_mte_tagged, &to->flags);
>> +		page_kasan_tag_reset(to);
>>  		mte_copy_page_tags(kto, kfrom);
> 
> Any reason why this doesn't have an smp_wmb() between resetting the tags
> and copying them into kto?
> 

Yes, the reason is I am not sure why it disappeared from the submitted patch ;)
I am going to respin the patch.

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE
  2020-11-10 22:10 ` [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE Andrey Konovalov
@ 2020-11-12  9:43   ` Catalin Marinas
  2020-11-12 16:46     ` Vincenzo Frascino
  2020-11-13 11:17     ` Vincenzo Frascino
  0 siblings, 2 replies; 107+ messages in thread
From: Catalin Marinas @ 2020-11-12  9:43 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Tue, Nov 10, 2020 at 11:10:27PM +0100, Andrey Konovalov wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> 
> Hardware tag-based KASAN relies on Memory Tagging Extension (MTE)
> feature and requires it to be enabled. MTE supports
> 
> This patch adds a new mte_init_tags() helper, that enables MTE in
> Synchronous mode in EL1 and is intended to be called from KASAN runtime
> during initialization.

There's no mte_init_tags() in this function.

> diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
> index 600b26d65b41..7f477991a6cf 100644
> --- a/arch/arm64/kernel/mte.c
> +++ b/arch/arm64/kernel/mte.c
> @@ -129,6 +129,13 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
>  	return ptr;
>  }
>  
> +void mte_enable(void)
> +{
> +	/* Enable MTE Sync Mode for EL1. */
> +	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC);
> +	isb();
> +}

Nitpick: maybe rename this to mte_enable_kernel() since MTE is already
enabled for user apps.

-- 
Catalin


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit
  2020-11-12  9:39   ` Catalin Marinas
@ 2020-11-12  9:45     ` Vincenzo Frascino
  2020-11-12  9:45       ` Catalin Marinas
  0 siblings, 1 reply; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-12  9:45 UTC (permalink / raw)
  To: Catalin Marinas, Andrey Konovalov
  Cc: Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko,
	Marco Elver, Evgenii Stepanov, Branislav Rankov, Kevin Brodsky,
	Andrew Morton, kasan-dev, linux-arm-kernel, linux-mm,
	linux-kernel



On 11/12/20 9:39 AM, Catalin Marinas wrote:
> On Tue, Nov 10, 2020 at 11:10:29PM +0100, Andrey Konovalov wrote:
>> diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
>> index 664c968dc43c..dbda6598c19d 100644
>> --- a/arch/arm64/kernel/mte.c
>> +++ b/arch/arm64/kernel/mte.c
>> @@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
>>  	return ptr;
>>  }
>>  
>> +void mte_init_tags(u64 max_tag)
>> +{
>> +	static bool gcr_kernel_excl_initialized = false;
>> +
>> +	if (!gcr_kernel_excl_initialized) {
>> +		/*
>> +		 * The format of the tags in KASAN is 0xFF and in MTE is 0xF.
>> +		 * This conversion extracts an MTE tag from a KASAN tag.
>> +		 */
>> +		u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT,
>> +					     max_tag), 0);
>> +
>> +		gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
>> +		gcr_kernel_excl_initialized = true;
>> +	}
>> +
>> +	/* Enable the kernel exclude mask for random tags generation. */
>> +	write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1);
>> +}
> 
> I don't think this function belongs to this patch. There is an earlier
> patch that talks about mte_init_tags() but no trace of it until this
> patch.
> 

Could you please point out to which patch are you referring to?

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit
  2020-11-12  9:45     ` Vincenzo Frascino
@ 2020-11-12  9:45       ` Catalin Marinas
  2020-11-12  9:55         ` Vincenzo Frascino
  0 siblings, 1 reply; 107+ messages in thread
From: Catalin Marinas @ 2020-11-12  9:45 UTC (permalink / raw)
  To: Vincenzo Frascino
  Cc: Andrey Konovalov, Will Deacon, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Thu, Nov 12, 2020 at 09:45:45AM +0000, Vincenzo Frascino wrote:
> On 11/12/20 9:39 AM, Catalin Marinas wrote:
> > On Tue, Nov 10, 2020 at 11:10:29PM +0100, Andrey Konovalov wrote:
> >> diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
> >> index 664c968dc43c..dbda6598c19d 100644
> >> --- a/arch/arm64/kernel/mte.c
> >> +++ b/arch/arm64/kernel/mte.c
> >> @@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
> >>  	return ptr;
> >>  }
> >>  
> >> +void mte_init_tags(u64 max_tag)
> >> +{
> >> +	static bool gcr_kernel_excl_initialized = false;
> >> +
> >> +	if (!gcr_kernel_excl_initialized) {
> >> +		/*
> >> +		 * The format of the tags in KASAN is 0xFF and in MTE is 0xF.
> >> +		 * This conversion extracts an MTE tag from a KASAN tag.
> >> +		 */
> >> +		u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT,
> >> +					     max_tag), 0);
> >> +
> >> +		gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
> >> +		gcr_kernel_excl_initialized = true;
> >> +	}
> >> +
> >> +	/* Enable the kernel exclude mask for random tags generation. */
> >> +	write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1);
> >> +}
> > 
> > I don't think this function belongs to this patch. There is an earlier
> > patch that talks about mte_init_tags() but no trace of it until this
> > patch.
> 
> Could you please point out to which patch are you referring to?

I replied to it already (or you can search ;)). But this patch is about
switching GCR_EL1 on exception entry/exit rather than setting up the
initial kernel GCR_EL1 value.

-- 
Catalin


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch
  2020-11-10 22:10 ` [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch Andrey Konovalov
@ 2020-11-12  9:46   ` Catalin Marinas
  2020-11-12 15:59   ` Alexander Potapenko
  1 sibling, 0 replies; 107+ messages in thread
From: Catalin Marinas @ 2020-11-12  9:46 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Will Deacon, Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Tue, Nov 10, 2020 at 11:10:41PM +0100, Andrey Konovalov wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> 
> This test is specific to MTE and verifies that the GCR_EL1 register
> is context switched correctly.
> 
> It spawn 1024 processes and each process spawns 5 threads. Each thread
> writes a random setting of GCR_EL1 through the prctl() system call and
> reads it back verifying that it is the same. If the values are not the
> same it reports a failure.
> 
> Note: The test has been extended to verify that even SYNC and ASYNC mode
> setting is preserved correctly over context switching.
> 
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit
  2020-11-12  9:45       ` Catalin Marinas
@ 2020-11-12  9:55         ` Vincenzo Frascino
  0 siblings, 0 replies; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-12  9:55 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Andrey Konovalov, Will Deacon, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel



On 11/12/20 9:45 AM, Catalin Marinas wrote:
> On Thu, Nov 12, 2020 at 09:45:45AM +0000, Vincenzo Frascino wrote:
>> On 11/12/20 9:39 AM, Catalin Marinas wrote:
>>> On Tue, Nov 10, 2020 at 11:10:29PM +0100, Andrey Konovalov wrote:
>>>> diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
>>>> index 664c968dc43c..dbda6598c19d 100644
>>>> --- a/arch/arm64/kernel/mte.c
>>>> +++ b/arch/arm64/kernel/mte.c
>>>> @@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
>>>>  	return ptr;
>>>>  }
>>>>  
>>>> +void mte_init_tags(u64 max_tag)
>>>> +{
>>>> +	static bool gcr_kernel_excl_initialized = false;
>>>> +
>>>> +	if (!gcr_kernel_excl_initialized) {
>>>> +		/*
>>>> +		 * The format of the tags in KASAN is 0xFF and in MTE is 0xF.
>>>> +		 * This conversion extracts an MTE tag from a KASAN tag.
>>>> +		 */
>>>> +		u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT,
>>>> +					     max_tag), 0);
>>>> +
>>>> +		gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK;
>>>> +		gcr_kernel_excl_initialized = true;
>>>> +	}
>>>> +
>>>> +	/* Enable the kernel exclude mask for random tags generation. */
>>>> +	write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1);
>>>> +}
>>>
>>> I don't think this function belongs to this patch. There is an earlier
>>> patch that talks about mte_init_tags() but no trace of it until this
>>> patch.
>>
>> Could you please point out to which patch are you referring to?
> 
> I replied to it already (or you can search ;)). But this patch is about
> switching GCR_EL1 on exception entry/exit rather than setting up the
> initial kernel GCR_EL1 value.
> 

Temporally after I asked ;) (I give you the benefit of delay of the mail server
;) ). I think that during the development the logic changed a bit, but I agree
that the comments are outdated. I am fine to move the code.

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 17/44] kasan, arm64: move initialization message
  2020-11-11 18:50     ` Andrey Konovalov
@ 2020-11-12 15:09       ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-12 15:09 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 7:50 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Wed, Nov 11, 2020 at 4:04 PM Alexander Potapenko <glider@google.com> wrote:
> >
> > On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> > >
> > > Software tag-based KASAN mode is fully initialized with kasan_init_tags(),
> > > while the generic mode only requires kasan_init(). Move the
> > > initialization message for tag-based mode into kasan_init_tags().
> > >
> > > Also fix pr_fmt() usage for KASAN code: generic.c doesn't need it as it
> > > doesn't use any printing functions; tag-based mode should use "kasan:"
> > > instead of KBUILD_MODNAME (which stands for file name).
> > >
> > > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > > Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

Reviewed-by: Alexander Potapenko <glider@google.com>

> > Cannot we have a single kasan_init() function that will call
> > tool-specific initialization functions and print the message at the
> > end?
>
> Unfortunately no. For different modes we need different functions that
> are called in different places in the kernel. E.g. for generic KASAN
> we only need kasan_init() to setup shadow pages; for SW tags we also
> need kasan_init_sw_tags() which initializes per-cpu state and
> finilizes initialization process.

Ok, got it.

-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes
  2020-11-11 18:52     ` Andrey Konovalov
@ 2020-11-12 15:16       ` Alexander Potapenko
  2020-11-12 19:27         ` Andrey Konovalov
  0 siblings, 1 reply; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-12 15:16 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Wed, Nov 11, 2020 at 7:52 PM 'Andrey Konovalov' via kasan-dev
<kasan-dev@googlegroups.com> wrote:
>
> On Wed, Nov 11, 2020 at 4:09 PM Alexander Potapenko <glider@google.com> wrote:
> >
> > On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> > >
> > > This is a preparatory commit for the upcoming addition of a new hardware
> > > tag-based (MTE-based) KASAN mode.
> > >
> > > kasan_non_canonical_hook() is only applicable to KASAN modes that use
> > > shadow memory, and won't be needed for hardware tag-based KASAN.
> > >
> > > No functional changes for software modes.
> > >
> > > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > > Reviewed-by: Marco Elver <elver@google.com>
> > > ---
> > > Change-Id: Icc9f5ef100a2e86f3a4214a0c3131a68266181b2
> > > ---
> > >  mm/kasan/report.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> > > index 5d5733831ad7..594bad2a3a5e 100644
> > > --- a/mm/kasan/report.c
> > > +++ b/mm/kasan/report.c
> > > @@ -403,7 +403,8 @@ bool kasan_report(unsigned long addr, size_t size, bool is_write,
> > >         return ret;
> > >  }
> > >
> > > -#ifdef CONFIG_KASAN_INLINE
> > > +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> > > +       defined(CONFIG_KASAN_INLINE)
> > >  /*
> > >   * With CONFIG_KASAN_INLINE, accesses to bogus pointers (outside the high
> > >   * canonical half of the address space) cause out-of-bounds shadow memory reads
> >
> > Perhaps this comment also needs to be updated.
>
> In what way?

Ok, maybe not. I thought you were restricting the set of configs under
which this hook is used, so this should've been explained.
But as far as I understand, CONFIG_KASAN_INLINE already implies
"defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)",
doesn't it?
Maybe this change is not needed at all then?

>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/CAAeHK%2BwX%2BJPyZm2A5mDdGFCqnH6kdSBLyOZ2TnWfZnZuq_V0Bw%40mail.gmail.com.



-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch
  2020-11-10 22:10 ` [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch Andrey Konovalov
  2020-11-12  9:46   ` Catalin Marinas
@ 2020-11-12 15:59   ` Alexander Potapenko
  2020-11-12 16:09     ` Marco Elver
  2020-11-13 11:47     ` Vincenzo Frascino
  1 sibling, 2 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-12 15:59 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> This test is specific to MTE and verifies that the GCR_EL1 register
> is context switched correctly.
>
> It spawn 1024 processes and each process spawns 5 threads. Each thread

Nit: "spawns"


> +       srand(time(NULL) ^ (pid << 16) ^ (tid << 16));
> +
> +       prctl_tag_mask = rand() % 0xffff;

Nit: if you want values between 0 and 0xffff you probably want to use
bitwise AND.


> +
> +int execute_test(pid_t pid)
> +{
> +       pthread_t thread_id[MAX_THREADS];
> +       int thread_data[MAX_THREADS];
> +
> +       for (int i = 0; i < MAX_THREADS; i++)
> +               pthread_create(&thread_id[i], NULL,
> +                              execute_thread, (void *)&pid);

It might be simpler to call getpid() in execute_thread() instead.

> +int mte_gcr_fork_test()
> +{
> +       pid_t pid[NUM_ITERATIONS];
> +       int results[NUM_ITERATIONS];
> +       pid_t cpid;
> +       int res;
> +
> +       for (int i = 0; i < NUM_ITERATIONS; i++) {
> +               pid[i] = fork();
> +
> +               if (pid[i] == 0) {

pid[i] isn't used anywhere else. Did you want to keep the pids to
ensure that all children finished the work?
If not, we can probably go with a scalar here.


> +       for (int i = 0; i < NUM_ITERATIONS; i++) {
> +               wait(&res);
> +
> +               if(WIFEXITED(res))
> +                       results[i] = WEXITSTATUS(res);
> +               else
> +                       --i;

Won't we get stuck in this loop if fork() returns -1 for one of the processes?

> +       }
> +
> +       for (int i = 0; i < NUM_ITERATIONS; i++)
> +               if (results[i] == KSFT_FAIL)
> +                       return KSFT_FAIL;
> +
> +       return KSFT_PASS;
> +}
> +


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch
  2020-11-12 15:59   ` Alexander Potapenko
@ 2020-11-12 16:09     ` Marco Elver
  2020-11-12 16:41       ` Alexander Potapenko
  2020-11-13 11:47     ` Vincenzo Frascino
  1 sibling, 1 reply; 107+ messages in thread
From: Marco Elver @ 2020-11-12 16:09 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Andrey Konovalov, Catalin Marinas, Will Deacon,
	Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Evgenii Stepanov, Branislav Rankov, Kevin Brodsky, Andrew Morton,
	kasan-dev, Linux ARM, Linux Memory Management List, LKML

On Thu, 12 Nov 2020 at 16:59, Alexander Potapenko <glider@google.com> wrote:
>
> On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> >
> > This test is specific to MTE and verifies that the GCR_EL1 register
> > is context switched correctly.
> >
> > It spawn 1024 processes and each process spawns 5 threads. Each thread
>
> Nit: "spawns"
>
>
> > +       srand(time(NULL) ^ (pid << 16) ^ (tid << 16));
> > +
> > +       prctl_tag_mask = rand() % 0xffff;
>
> Nit: if you want values between 0 and 0xffff you probably want to use
> bitwise AND.

Another question would be, is the max here meant to be 0xffff or
0xffff-1. Because, as-is now, it's 0xffff-1. Only one of them has a
trivial conversion to bitwise AND ( x % 2^n == x & (2^n - 1) ).


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata
  2020-11-10 22:10 ` [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata Andrey Konovalov
@ 2020-11-12 16:10   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-12 16:10 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Kernel allocator code accesses metadata for slab objects, that may lie
> out-of-bounds of the object itself, or be accessed when an object is freed.
> Such accesses trigger tag faults and lead to false-positive reports with
> hardware tag-based KASAN.
>
> Software KASAN modes disable instrumentation for allocator code via
> KASAN_SANITIZE Makefile macro, and rely on kasan_enable/disable_current()
> annotations which are used to ignore KASAN reports.
>
> With hardware tag-based KASAN neither of those options are available, as
> it doesn't use compiler instrumetation, no tag faults are ignored, and MTE
> is disabled after the first one.
>
> Instead, reset tags when accessing metadata (currently only for SLUB).
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Acked-by: Marco Elver <elver@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>


> ---
> Change-Id: I39f3c4d4f29299d4fbbda039bedf230db1c746fb
> ---
>  mm/page_alloc.c  |  4 +++-
>  mm/page_poison.c |  2 +-
>  mm/slub.c        | 29 ++++++++++++++++-------------
>  3 files changed, 20 insertions(+), 15 deletions(-)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 24b45261e2bd..f1648aee8d88 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1195,8 +1195,10 @@ static void kernel_init_free_pages(struct page *page, int numpages)
>
>         /* s390's use of memset() could override KASAN redzones. */
>         kasan_disable_current();
> -       for (i = 0; i < numpages; i++)
> +       for (i = 0; i < numpages; i++) {
> +               page_kasan_tag_reset(page + i);
>                 clear_highpage(page + i);
> +       }
>         kasan_enable_current();
>  }
>
> diff --git a/mm/page_poison.c b/mm/page_poison.c
> index ae0482cded87..e6c994af7518 100644
> --- a/mm/page_poison.c
> +++ b/mm/page_poison.c
> @@ -53,7 +53,7 @@ static void poison_page(struct page *page)
>
>         /* KASAN still think the page is in-use, so skip it. */
>         kasan_disable_current();
> -       memset(addr, PAGE_POISON, PAGE_SIZE);
> +       memset(kasan_reset_tag(addr), PAGE_POISON, PAGE_SIZE);
>         kasan_enable_current();
>         kunmap_atomic(addr);
>  }
> diff --git a/mm/slub.c b/mm/slub.c
> index b30be2385d1c..df2fd5b57df1 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -249,7 +249,7 @@ static inline void *freelist_ptr(const struct kmem_cache *s, void *ptr,
>  {
>  #ifdef CONFIG_SLAB_FREELIST_HARDENED
>         /*
> -        * When CONFIG_KASAN_SW_TAGS is enabled, ptr_addr might be tagged.
> +        * When CONFIG_KASAN_SW/HW_TAGS is enabled, ptr_addr might be tagged.
>          * Normally, this doesn't cause any issues, as both set_freepointer()
>          * and get_freepointer() are called with a pointer with the same tag.
>          * However, there are some issues with CONFIG_SLUB_DEBUG code. For
> @@ -275,6 +275,7 @@ static inline void *freelist_dereference(const struct kmem_cache *s,
>
>  static inline void *get_freepointer(struct kmem_cache *s, void *object)
>  {
> +       object = kasan_reset_tag(object);
>         return freelist_dereference(s, object + s->offset);
>  }
>
> @@ -304,6 +305,7 @@ static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp)
>         BUG_ON(object == fp); /* naive detection of double free or corruption */
>  #endif
>
> +       freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr);
>         *(void **)freeptr_addr = freelist_ptr(s, fp, freeptr_addr);
>  }
>
> @@ -538,8 +540,8 @@ static void print_section(char *level, char *text, u8 *addr,
>                           unsigned int length)
>  {
>         metadata_access_enable();
> -       print_hex_dump(level, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
> -                       length, 1);
> +       print_hex_dump(level, kasan_reset_tag(text), DUMP_PREFIX_ADDRESS,
> +                       16, 1, addr, length, 1);
>         metadata_access_disable();
>  }
>
> @@ -570,7 +572,7 @@ static struct track *get_track(struct kmem_cache *s, void *object,
>
>         p = object + get_info_end(s);
>
> -       return p + alloc;
> +       return kasan_reset_tag(p + alloc);
>  }
>
>  static void set_track(struct kmem_cache *s, void *object,
> @@ -583,7 +585,8 @@ static void set_track(struct kmem_cache *s, void *object,
>                 unsigned int nr_entries;
>
>                 metadata_access_enable();
> -               nr_entries = stack_trace_save(p->addrs, TRACK_ADDRS_COUNT, 3);
> +               nr_entries = stack_trace_save(kasan_reset_tag(p->addrs),
> +                                             TRACK_ADDRS_COUNT, 3);
>                 metadata_access_disable();
>
>                 if (nr_entries < TRACK_ADDRS_COUNT)
> @@ -747,7 +750,7 @@ static __printf(3, 4) void slab_err(struct kmem_cache *s, struct page *page,
>
>  static void init_object(struct kmem_cache *s, void *object, u8 val)
>  {
> -       u8 *p = object;
> +       u8 *p = kasan_reset_tag(object);
>
>         if (s->flags & SLAB_RED_ZONE)
>                 memset(p - s->red_left_pad, val, s->red_left_pad);
> @@ -777,7 +780,7 @@ static int check_bytes_and_report(struct kmem_cache *s, struct page *page,
>         u8 *addr = page_address(page);
>
>         metadata_access_enable();
> -       fault = memchr_inv(start, value, bytes);
> +       fault = memchr_inv(kasan_reset_tag(start), value, bytes);
>         metadata_access_disable();
>         if (!fault)
>                 return 1;
> @@ -873,7 +876,7 @@ static int slab_pad_check(struct kmem_cache *s, struct page *page)
>
>         pad = end - remainder;
>         metadata_access_enable();
> -       fault = memchr_inv(pad, POISON_INUSE, remainder);
> +       fault = memchr_inv(kasan_reset_tag(pad), POISON_INUSE, remainder);
>         metadata_access_disable();
>         if (!fault)
>                 return 1;
> @@ -1118,7 +1121,7 @@ void setup_page_debug(struct kmem_cache *s, struct page *page, void *addr)
>                 return;
>
>         metadata_access_enable();
> -       memset(addr, POISON_INUSE, page_size(page));
> +       memset(kasan_reset_tag(addr), POISON_INUSE, page_size(page));
>         metadata_access_disable();
>  }
>
> @@ -1566,10 +1569,10 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s,
>                          * Clear the object and the metadata, but don't touch
>                          * the redzone.
>                          */
> -                       memset(object, 0, s->object_size);
> +                       memset(kasan_reset_tag(object), 0, s->object_size);
>                         rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad
>                                                            : 0;
> -                       memset((char *)object + s->inuse, 0,
> +                       memset((char *)kasan_reset_tag(object) + s->inuse, 0,
>                                s->size - s->inuse - rsize);
>
>                 }
> @@ -2883,10 +2886,10 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
>                 stat(s, ALLOC_FASTPATH);
>         }
>
> -       maybe_wipe_obj_freeptr(s, object);
> +       maybe_wipe_obj_freeptr(s, kasan_reset_tag(object));
>
>         if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object)
> -               memset(object, 0, s->object_size);
> +               memset(kasan_reset_tag(object), 0, s->object_size);
>
>         slab_post_alloc_hook(s, objcg, gfpflags, 1, &object);
>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch
  2020-11-12 16:09     ` Marco Elver
@ 2020-11-12 16:41       ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-12 16:41 UTC (permalink / raw)
  To: Marco Elver
  Cc: Andrey Konovalov, Catalin Marinas, Will Deacon,
	Vincenzo Frascino, Dmitry Vyukov, Andrey Ryabinin,
	Evgenii Stepanov, Branislav Rankov, Kevin Brodsky, Andrew Morton,
	kasan-dev, Linux ARM, Linux Memory Management List, LKML

On Thu, Nov 12, 2020 at 5:09 PM Marco Elver <elver@google.com> wrote:
>
> On Thu, 12 Nov 2020 at 16:59, Alexander Potapenko <glider@google.com> wrote:
> >
> > On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> > >
> > > From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > >
> > > This test is specific to MTE and verifies that the GCR_EL1 register
> > > is context switched correctly.
> > >
> > > It spawn 1024 processes and each process spawns 5 threads. Each thread
> >
> > Nit: "spawns"
> >
> >
> > > +       srand(time(NULL) ^ (pid << 16) ^ (tid << 16));
> > > +
> > > +       prctl_tag_mask = rand() % 0xffff;
> >
> > Nit: if you want values between 0 and 0xffff you probably want to use
> > bitwise AND.
>
> Another question would be, is the max here meant to be 0xffff or
> 0xffff-1. Because, as-is now, it's 0xffff-1. Only one of them has a
> trivial conversion to bitwise AND ( x % 2^n == x & (2^n - 1) ).

Yes, that is basically what I meant, assuming that Vincenzo wanted the
max to be 0xffff

-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE
  2020-11-12  9:43   ` Catalin Marinas
@ 2020-11-12 16:46     ` Vincenzo Frascino
  2020-11-13 11:17     ` Vincenzo Frascino
  1 sibling, 0 replies; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-12 16:46 UTC (permalink / raw)
  To: Catalin Marinas, Andrey Konovalov
  Cc: Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko,
	Marco Elver, Evgenii Stepanov, Branislav Rankov, Kevin Brodsky,
	Andrew Morton, kasan-dev, linux-arm-kernel, linux-mm,
	linux-kernel

Hi Catalin,

missed this one.

On 11/12/20 9:43 AM, Catalin Marinas wrote:
> On Tue, Nov 10, 2020 at 11:10:27PM +0100, Andrey Konovalov wrote:
>> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
>>
>> Hardware tag-based KASAN relies on Memory Tagging Extension (MTE)
>> feature and requires it to be enabled. MTE supports
>>
>> This patch adds a new mte_init_tags() helper, that enables MTE in
>> Synchronous mode in EL1 and is intended to be called from KASAN runtime
>> during initialization.
> 
> There's no mte_init_tags() in this function.
> 
>> diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c
>> index 600b26d65b41..7f477991a6cf 100644
>> --- a/arch/arm64/kernel/mte.c
>> +++ b/arch/arm64/kernel/mte.c
>> @@ -129,6 +129,13 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag)
>>  	return ptr;
>>  }
>>  
>> +void mte_enable(void)
>> +{
>> +	/* Enable MTE Sync Mode for EL1. */
>> +	sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC);
>> +	isb();
>> +}
> 
> Nitpick: maybe rename this to mte_enable_kernel() since MTE is already
> enabled for user apps.
> 

I will fix this in the next iteration.

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime
  2020-11-10 22:10 ` [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime Andrey Konovalov
@ 2020-11-12 17:37   ` Alexander Potapenko
  0 siblings, 0 replies; 107+ messages in thread
From: Alexander Potapenko @ 2020-11-12 17:37 UTC (permalink / raw)
  To: Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> Provide implementation of KASAN functions required for the hardware
> tag-based mode. Those include core functions for memory and pointer
> tagging (tags_hw.c) and bug reporting (report_tags_hw.c). Also adapt
> common KASAN code to support the new mode.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alexander Potapenko <glider@google.com>

> ---
> Change-Id: I8a8689ba098174a4d0ef3f1d008178387c80ee1c
> ---
>  arch/arm64/include/asm/memory.h   |  4 +-
>  arch/arm64/kernel/cpufeature.c    |  3 ++
>  arch/arm64/kernel/smp.c           |  2 +
>  include/linux/kasan.h             | 24 +++++++---
>  include/linux/mm.h                |  2 +-
>  include/linux/page-flags-layout.h |  2 +-
>  mm/kasan/Makefile                 |  5 ++
>  mm/kasan/common.c                 | 15 +++---
>  mm/kasan/hw_tags.c                | 80 +++++++++++++++++++++++++++++++
>  mm/kasan/kasan.h                  | 17 +++++--
>  mm/kasan/report_hw_tags.c         | 42 ++++++++++++++++
>  mm/kasan/report_sw_tags.c         |  2 +-
>  mm/kasan/shadow.c                 |  2 +-
>  mm/kasan/sw_tags.c                |  2 +-
>  14 files changed, 177 insertions(+), 25 deletions(-)
>  create mode 100644 mm/kasan/hw_tags.c
>  create mode 100644 mm/kasan/report_hw_tags.c
>
> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
> index 656aaddb7014..5042eef5b111 100644
> --- a/arch/arm64/include/asm/memory.h
> +++ b/arch/arm64/include/asm/memory.h
> @@ -214,7 +214,7 @@ static inline unsigned long kaslr_offset(void)
>         (__force __typeof__(addr))__addr;                               \
>  })
>
> -#ifdef CONFIG_KASAN_SW_TAGS
> +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
>  #define __tag_shifted(tag)     ((u64)(tag) << 56)
>  #define __tag_reset(addr)      __untagged_addr(addr)
>  #define __tag_get(addr)                (__u8)((u64)(addr) >> 56)
> @@ -222,7 +222,7 @@ static inline unsigned long kaslr_offset(void)
>  #define __tag_shifted(tag)     0UL
>  #define __tag_reset(addr)      (addr)
>  #define __tag_get(addr)                0
> -#endif /* CONFIG_KASAN_SW_TAGS */
> +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
>
>  static inline const void *__tag_set(const void *addr, u8 tag)
>  {
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index dcc165b3fc04..6a1f2e3558c5 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -70,6 +70,7 @@
>  #include <linux/types.h>
>  #include <linux/mm.h>
>  #include <linux/cpu.h>
> +#include <linux/kasan.h>
>  #include <asm/cpu.h>
>  #include <asm/cpufeature.h>
>  #include <asm/cpu_ops.h>
> @@ -1704,6 +1705,8 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap)
>                 cleared_zero_page = true;
>                 mte_clear_page_tags(lm_alias(empty_zero_page));
>         }
> +
> +       kasan_init_hw_tags_cpu();
>  }
>  #endif /* CONFIG_ARM64_MTE */
>
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index 09c96f57818c..7235b9478413 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -461,6 +461,8 @@ void __init smp_prepare_boot_cpu(void)
>         /* Conditionally switch to GIC PMR for interrupt masking */
>         if (system_uses_irq_prio_masking())
>                 init_gic_priority_masking();
> +
> +       kasan_init_hw_tags();
>  }
>
>  static u64 __init of_get_cpu_mpidr(struct device_node *dn)
> diff --git a/include/linux/kasan.h b/include/linux/kasan.h
> index b6fc14b3da53..f22bdef82111 100644
> --- a/include/linux/kasan.h
> +++ b/include/linux/kasan.h
> @@ -189,25 +189,35 @@ static inline void kasan_record_aux_stack(void *ptr) {}
>
>  #endif /* CONFIG_KASAN_GENERIC */
>
> -#ifdef CONFIG_KASAN_SW_TAGS
> -
> -void __init kasan_init_sw_tags(void);
> +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
>
>  void *kasan_reset_tag(const void *addr);
>
>  bool kasan_report(unsigned long addr, size_t size,
>                 bool is_write, unsigned long ip);
>
> -#else /* CONFIG_KASAN_SW_TAGS */
> -
> -static inline void kasan_init_sw_tags(void) { }
> +#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */
>
>  static inline void *kasan_reset_tag(const void *addr)
>  {
>         return (void *)addr;
>  }
>
> -#endif /* CONFIG_KASAN_SW_TAGS */
> +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS*/
> +
> +#ifdef CONFIG_KASAN_SW_TAGS
> +void __init kasan_init_sw_tags(void);
> +#else
> +static inline void kasan_init_sw_tags(void) { }
> +#endif
> +
> +#ifdef CONFIG_KASAN_HW_TAGS
> +void kasan_init_hw_tags_cpu(void);
> +void kasan_init_hw_tags(void);
> +#else
> +static inline void kasan_init_hw_tags_cpu(void) { }
> +static inline void kasan_init_hw_tags(void) { }
> +#endif
>
>  #ifdef CONFIG_KASAN_VMALLOC
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index db6ae4d3fb4e..0793d03a4183 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -1413,7 +1413,7 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid)
>  }
>  #endif /* CONFIG_NUMA_BALANCING */
>
> -#ifdef CONFIG_KASAN_SW_TAGS
> +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
>  static inline u8 page_kasan_tag(const struct page *page)
>  {
>         return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK;
> diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h
> index e200eef6a7fd..7d4ec26d8a3e 100644
> --- a/include/linux/page-flags-layout.h
> +++ b/include/linux/page-flags-layout.h
> @@ -77,7 +77,7 @@
>  #define LAST_CPUPID_SHIFT 0
>  #endif
>
> -#ifdef CONFIG_KASAN_SW_TAGS
> +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
>  #define KASAN_TAG_WIDTH 8
>  #else
>  #define KASAN_TAG_WIDTH 0
> diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
> index f1d68a34f3c9..9fe39a66388a 100644
> --- a/mm/kasan/Makefile
> +++ b/mm/kasan/Makefile
> @@ -10,8 +10,10 @@ CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_report_hw_tags.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_hw_tags.o = $(CC_FLAGS_FTRACE)
>  CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE)
>
>  # Function splitter causes unnecessary splits in __asan_load1/__asan_store1
> @@ -27,10 +29,13 @@ CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME)
> +CFLAGS_report_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME)
> +CFLAGS_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>  CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME)
>
>  obj-$(CONFIG_KASAN) := common.o report.o
>  obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o
> +obj-$(CONFIG_KASAN_HW_TAGS) += hw_tags.o report_hw_tags.o
>  obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index d0b3ff410b0c..2bb0ef6da6bd 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -113,7 +113,7 @@ void kasan_free_pages(struct page *page, unsigned int order)
>   */
>  static inline unsigned int optimal_redzone(unsigned int object_size)
>  {
> -       if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
> +       if (!IS_ENABLED(CONFIG_KASAN_GENERIC))
>                 return 0;
>
>         return
> @@ -178,14 +178,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache)
>  struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>                                         const void *object)
>  {
> -       return (void *)object + cache->kasan_info.alloc_meta_offset;
> +       return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset;
>  }
>
>  struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>                                       const void *object)
>  {
>         BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
> -       return (void *)object + cache->kasan_info.free_meta_offset;
> +       return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset;
>  }
>
>  void kasan_poison_slab(struct page *page)
> @@ -267,9 +267,8 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache,
>         alloc_info = get_alloc_info(cache, object);
>         __memset(alloc_info, 0, sizeof(*alloc_info));
>
> -       if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
> -               object = set_tag(object,
> -                               assign_tag(cache, object, true, false));
> +       if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
> +               object = set_tag(object, assign_tag(cache, object, true, false));
>
>         return (void *)object;
>  }
> @@ -337,10 +336,10 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
>         redzone_end = round_up((unsigned long)object + cache->object_size,
>                                 KASAN_GRANULE_SIZE);
>
> -       if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
> +       if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS))
>                 tag = assign_tag(cache, object, false, keep_tag);
>
> -       /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */
> +       /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */
>         kasan_unpoison_memory(set_tag(object, tag), size);
>         kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start,
>                 KASAN_KMALLOC_REDZONE);
> diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c
> new file mode 100644
> index 000000000000..0080b78ec843
> --- /dev/null
> +++ b/mm/kasan/hw_tags.c
> @@ -0,0 +1,80 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * This file contains core hardware tag-based KASAN code.
> + *
> + * Copyright (c) 2020 Google, Inc.
> + * Author: Andrey Konovalov <andreyknvl@google.com>
> + */
> +
> +#define pr_fmt(fmt) "kasan: " fmt
> +
> +#include <linux/kasan.h>
> +#include <linux/kernel.h>
> +#include <linux/memory.h>
> +#include <linux/mm.h>
> +#include <linux/string.h>
> +#include <linux/types.h>
> +
> +#include "kasan.h"
> +
> +/* kasan_init_hw_tags_cpu() is called for each CPU. */
> +void kasan_init_hw_tags_cpu(void)
> +{
> +       hw_init_tags(KASAN_TAG_MAX);
> +       hw_enable_tagging();
> +}
> +
> +/* kasan_init_hw_tags() is called once on boot CPU. */
> +void kasan_init_hw_tags(void)
> +{
> +       pr_info("KernelAddressSanitizer initialized\n");
> +}
> +
> +void *kasan_reset_tag(const void *addr)
> +{
> +       return reset_tag(addr);
> +}
> +
> +void kasan_poison_memory(const void *address, size_t size, u8 value)
> +{
> +       hw_set_mem_tag_range(reset_tag(address),
> +                       round_up(size, KASAN_GRANULE_SIZE), value);
> +}
> +
> +void kasan_unpoison_memory(const void *address, size_t size)
> +{
> +       hw_set_mem_tag_range(reset_tag(address),
> +                       round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
> +}
> +
> +u8 random_tag(void)
> +{
> +       return hw_get_random_tag();
> +}
> +
> +bool check_invalid_free(void *addr)
> +{
> +       u8 ptr_tag = get_tag(addr);
> +       u8 mem_tag = hw_get_mem_tag(addr);
> +
> +       return (mem_tag == KASAN_TAG_INVALID) ||
> +               (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
> +}
> +
> +void kasan_set_free_info(struct kmem_cache *cache,
> +                               void *object, u8 tag)
> +{
> +       struct kasan_alloc_meta *alloc_meta;
> +
> +       alloc_meta = get_alloc_info(cache, object);
> +       kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT);
> +}
> +
> +struct kasan_track *kasan_get_free_track(struct kmem_cache *cache,
> +                               void *object, u8 tag)
> +{
> +       struct kasan_alloc_meta *alloc_meta;
> +
> +       alloc_meta = get_alloc_info(cache, object);
> +       return &alloc_meta->free_track[0];
> +}
> diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
> index d745a78745dd..21fe75c66f26 100644
> --- a/mm/kasan/kasan.h
> +++ b/mm/kasan/kasan.h
> @@ -153,6 +153,10 @@ struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>  struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>                                         const void *object);
>
> +void kasan_poison_memory(const void *address, size_t size, u8 value);
> +
> +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
> +
>  static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
>  {
>         return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET)
> @@ -164,8 +168,6 @@ static inline bool addr_has_metadata(const void *addr)
>         return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START));
>  }
>
> -void kasan_poison_memory(const void *address, size_t size, u8 value);
> -
>  /**
>   * check_memory_region - Check memory region, and report if invalid access.
>   * @addr: the accessed address
> @@ -177,6 +179,15 @@ void kasan_poison_memory(const void *address, size_t size, u8 value);
>  bool check_memory_region(unsigned long addr, size_t size, bool write,
>                                 unsigned long ret_ip);
>
> +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> +
> +static inline bool addr_has_metadata(const void *addr)
> +{
> +       return true;
> +}
> +
> +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */
> +
>  bool check_invalid_free(void *addr);
>
>  void *find_first_bad_addr(void *addr, size_t size);
> @@ -213,7 +224,7 @@ static inline void quarantine_reduce(void) { }
>  static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
>  #endif
>
> -#ifdef CONFIG_KASAN_SW_TAGS
> +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS)
>
>  void print_tags(u8 addr_tag, const void *addr);
>
> diff --git a/mm/kasan/report_hw_tags.c b/mm/kasan/report_hw_tags.c
> new file mode 100644
> index 000000000000..da543eb832cd
> --- /dev/null
> +++ b/mm/kasan/report_hw_tags.c
> @@ -0,0 +1,42 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * This file contains hardware tag-based KASAN specific error reporting code.
> + *
> + * Copyright (c) 2020 Google, Inc.
> + * Author: Andrey Konovalov <andreyknvl@google.com>
> + */
> +
> +#include <linux/kasan.h>
> +#include <linux/kernel.h>
> +#include <linux/memory.h>
> +#include <linux/mm.h>
> +#include <linux/string.h>
> +#include <linux/types.h>
> +
> +#include "kasan.h"
> +
> +const char *get_bug_type(struct kasan_access_info *info)
> +{
> +       return "invalid-access";
> +}
> +
> +void *find_first_bad_addr(void *addr, size_t size)
> +{
> +       return reset_tag(addr);
> +}
> +
> +void metadata_fetch_row(char *buffer, void *row)
> +{
> +       int i;
> +
> +       for (i = 0; i < META_BYTES_PER_ROW; i++)
> +               buffer[i] = hw_get_mem_tag(row + i * KASAN_GRANULE_SIZE);
> +}
> +
> +void print_tags(u8 addr_tag, const void *addr)
> +{
> +       u8 memory_tag = hw_get_mem_tag((void *)addr);
> +
> +       pr_err("Pointer tag: [%02x], memory tag: [%02x]\n",
> +               addr_tag, memory_tag);
> +}
> diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c
> index add2dfe6169c..aebc44a29e83 100644
> --- a/mm/kasan/report_sw_tags.c
> +++ b/mm/kasan/report_sw_tags.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * This file contains tag-based KASAN specific error reporting code.
> + * This file contains software tag-based KASAN specific error reporting code.
>   *
>   * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>   * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
> index 1fadd4930d54..616ac64c4a21 100644
> --- a/mm/kasan/shadow.c
> +++ b/mm/kasan/shadow.c
> @@ -107,7 +107,7 @@ void kasan_unpoison_memory(const void *address, size_t size)
>
>                 if (IS_ENABLED(CONFIG_KASAN_SW_TAGS))
>                         *shadow = tag;
> -               else
> +               else /* CONFIG_KASAN_GENERIC */
>                         *shadow = size & KASAN_GRANULE_MASK;
>         }
>  }
> diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c
> index b09a2c06abad..dfe707dd8d0d 100644
> --- a/mm/kasan/sw_tags.c
> +++ b/mm/kasan/sw_tags.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * This file contains core tag-based KASAN code.
> + * This file contains core software tag-based KASAN code.
>   *
>   * Copyright (c) 2018 Google, Inc.
>   * Author: Andrey Konovalov <andreyknvl@google.com>
> --
> 2.29.2.222.g5d2a92d10f8-goog
>


-- 
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes
  2020-11-12 15:16       ` Alexander Potapenko
@ 2020-11-12 19:27         ` Andrey Konovalov
  0 siblings, 0 replies; 107+ messages in thread
From: Andrey Konovalov @ 2020-11-12 19:27 UTC (permalink / raw)
  To: Alexander Potapenko
  Cc: Catalin Marinas, Will Deacon, Vincenzo Frascino, Dmitry Vyukov,
	Andrey Ryabinin, Marco Elver, Evgenii Stepanov, Branislav Rankov,
	Kevin Brodsky, Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

On Thu, Nov 12, 2020 at 4:16 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Wed, Nov 11, 2020 at 7:52 PM 'Andrey Konovalov' via kasan-dev
> <kasan-dev@googlegroups.com> wrote:
> >
> > On Wed, Nov 11, 2020 at 4:09 PM Alexander Potapenko <glider@google.com> wrote:
> > >
> > > On Tue, Nov 10, 2020 at 11:11 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> > > >
> > > > This is a preparatory commit for the upcoming addition of a new hardware
> > > > tag-based (MTE-based) KASAN mode.
> > > >
> > > > kasan_non_canonical_hook() is only applicable to KASAN modes that use
> > > > shadow memory, and won't be needed for hardware tag-based KASAN.
> > > >
> > > > No functional changes for software modes.
> > > >
> > > > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > > > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > > > Reviewed-by: Marco Elver <elver@google.com>
> > > > ---
> > > > Change-Id: Icc9f5ef100a2e86f3a4214a0c3131a68266181b2
> > > > ---
> > > >  mm/kasan/report.c | 3 ++-
> > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> > > > index 5d5733831ad7..594bad2a3a5e 100644
> > > > --- a/mm/kasan/report.c
> > > > +++ b/mm/kasan/report.c
> > > > @@ -403,7 +403,8 @@ bool kasan_report(unsigned long addr, size_t size, bool is_write,
> > > >         return ret;
> > > >  }
> > > >
> > > > -#ifdef CONFIG_KASAN_INLINE
> > > > +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
> > > > +       defined(CONFIG_KASAN_INLINE)
> > > >  /*
> > > >   * With CONFIG_KASAN_INLINE, accesses to bogus pointers (outside the high
> > > >   * canonical half of the address space) cause out-of-bounds shadow memory reads
> > >
> > > Perhaps this comment also needs to be updated.
> >
> > In what way?
>
> Ok, maybe not. I thought you were restricting the set of configs under
> which this hook is used, so this should've been explained.
> But as far as I understand, CONFIG_KASAN_INLINE already implies
> "defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)",
> doesn't it?
> Maybe this change is not needed at all then?

Ah, yes, you're right. Will drop this patch, thanks!


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE
  2020-11-12  9:43   ` Catalin Marinas
  2020-11-12 16:46     ` Vincenzo Frascino
@ 2020-11-13 11:17     ` Vincenzo Frascino
  2020-11-13 12:00       ` Catalin Marinas
  1 sibling, 1 reply; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-13 11:17 UTC (permalink / raw)
  To: Catalin Marinas, Andrey Konovalov
  Cc: Will Deacon, Dmitry Vyukov, Andrey Ryabinin, Alexander Potapenko,
	Marco Elver, Evgenii Stepanov, Branislav Rankov, Kevin Brodsky,
	Andrew Morton, kasan-dev, linux-arm-kernel, linux-mm,
	linux-kernel

Hi Catalin,

On 11/12/20 9:43 AM, Catalin Marinas wrote:
> On Tue, Nov 10, 2020 at 11:10:27PM +0100, Andrey Konovalov wrote:
>> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
>>
>> Hardware tag-based KASAN relies on Memory Tagging Extension (MTE)
>> feature and requires it to be enabled. MTE supports
>>
>> This patch adds a new mte_init_tags() helper, that enables MTE in
>> Synchronous mode in EL1 and is intended to be called from KASAN runtime
>> during initialization.
> 
> There's no mte_init_tags() in this function.
> 

During the rework, I realized that the description of mte_init_tags() in this
patch refers to mte_enable_kernel(). In fact the only thing that mte_init_tags()
does is to configure the GCR_EL1 register, hence my preference would be to keep
all the code that deals with such a register in one patch.

What is your preference?

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch
  2020-11-12 15:59   ` Alexander Potapenko
  2020-11-12 16:09     ` Marco Elver
@ 2020-11-13 11:47     ` Vincenzo Frascino
  1 sibling, 0 replies; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-13 11:47 UTC (permalink / raw)
  To: Alexander Potapenko, Andrey Konovalov
  Cc: Catalin Marinas, Will Deacon, Dmitry Vyukov, Andrey Ryabinin,
	Marco Elver, Evgenii Stepanov, Branislav Rankov, Kevin Brodsky,
	Andrew Morton, kasan-dev, Linux ARM,
	Linux Memory Management List, LKML

Hi Alexander,

thank you for the review.

On 11/12/20 3:59 PM, Alexander Potapenko wrote:
> On Tue, Nov 10, 2020 at 11:12 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>>
>> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
>>
>> This test is specific to MTE and verifies that the GCR_EL1 register
>> is context switched correctly.
>>
>> It spawn 1024 processes and each process spawns 5 threads. Each thread
> 
> Nit: "spawns"
> 

I will fix it in the next iteration.

> 
>> +       srand(time(NULL) ^ (pid << 16) ^ (tid << 16));
>> +
>> +       prctl_tag_mask = rand() % 0xffff;
> 
> Nit: if you want values between 0 and 0xffff you probably want to use
> bitwise AND.
> 

The main goal here is to have a good probability of having a different setting
to the GCR_EL1 register. Hence the difference in between 0xffff and 0xffff-1 is
negligible. Anyway I agree that we should aim to cover all the possible
combinations.

> 
>> +
>> +int execute_test(pid_t pid)
>> +{
>> +       pthread_t thread_id[MAX_THREADS];
>> +       int thread_data[MAX_THREADS];
>> +
>> +       for (int i = 0; i < MAX_THREADS; i++)
>> +               pthread_create(&thread_id[i], NULL,
>> +                              execute_thread, (void *)&pid);
> 
> It might be simpler to call getpid() in execute_thread() instead.
> 

Yes it might, but I would like to avoid another syscall if I can.

>> +int mte_gcr_fork_test()
>> +{
>> +       pid_t pid[NUM_ITERATIONS];
>> +       int results[NUM_ITERATIONS];
>> +       pid_t cpid;
>> +       int res;
>> +
>> +       for (int i = 0; i < NUM_ITERATIONS; i++) {
>> +               pid[i] = fork();
>> +
>> +               if (pid[i] == 0) {
> 
> pid[i] isn't used anywhere else. Did you want to keep the pids to
> ensure that all children finished the work?
> If not, we can probably go with a scalar here.
> 

Yes, I agree, I had some debug code making use of it, but I removed it in the end.

> 
>> +       for (int i = 0; i < NUM_ITERATIONS; i++) {
>> +               wait(&res);
>> +
>> +               if(WIFEXITED(res))
>> +                       results[i] = WEXITSTATUS(res);
>> +               else
>> +                       --i;
> 
> Won't we get stuck in this loop if fork() returns -1 for one of the processes?
> 

Yes I agree, I forgot to check a condition. We should abort the test in such a
case returning KSFT_FAIL directly.

>> +       }
>> +
>> +       for (int i = 0; i < NUM_ITERATIONS; i++)
>> +               if (results[i] == KSFT_FAIL)
>> +                       return KSFT_FAIL;
>> +
>> +       return KSFT_PASS;
>> +}
>> +
> 
> 

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE
  2020-11-13 11:17     ` Vincenzo Frascino
@ 2020-11-13 12:00       ` Catalin Marinas
  2020-11-13 12:04         ` Vincenzo Frascino
  0 siblings, 1 reply; 107+ messages in thread
From: Catalin Marinas @ 2020-11-13 12:00 UTC (permalink / raw)
  To: Vincenzo Frascino
  Cc: Andrey Konovalov, Will Deacon, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel

On Fri, Nov 13, 2020 at 11:17:15AM +0000, Vincenzo Frascino wrote:
> On 11/12/20 9:43 AM, Catalin Marinas wrote:
> > On Tue, Nov 10, 2020 at 11:10:27PM +0100, Andrey Konovalov wrote:
> >> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> >>
> >> Hardware tag-based KASAN relies on Memory Tagging Extension (MTE)
> >> feature and requires it to be enabled. MTE supports
> >>
> >> This patch adds a new mte_init_tags() helper, that enables MTE in
> >> Synchronous mode in EL1 and is intended to be called from KASAN runtime
> >> during initialization.
> > 
> > There's no mte_init_tags() in this function.
> 
> During the rework, I realized that the description of mte_init_tags() in this
> patch refers to mte_enable_kernel(). In fact the only thing that mte_init_tags()
> does is to configure the GCR_EL1 register, hence my preference would be to keep
> all the code that deals with such a register in one patch.

Fine by me as long as the commit text is consistent with the diff.

-- 
Catalin


^ permalink raw reply	[flat|nested] 107+ messages in thread

* Re: [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE
  2020-11-13 12:00       ` Catalin Marinas
@ 2020-11-13 12:04         ` Vincenzo Frascino
  0 siblings, 0 replies; 107+ messages in thread
From: Vincenzo Frascino @ 2020-11-13 12:04 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: Andrey Konovalov, Will Deacon, Dmitry Vyukov, Andrey Ryabinin,
	Alexander Potapenko, Marco Elver, Evgenii Stepanov,
	Branislav Rankov, Kevin Brodsky, Andrew Morton, kasan-dev,
	linux-arm-kernel, linux-mm, linux-kernel



On 11/13/20 12:00 PM, Catalin Marinas wrote:
> On Fri, Nov 13, 2020 at 11:17:15AM +0000, Vincenzo Frascino wrote:
>> On 11/12/20 9:43 AM, Catalin Marinas wrote:
>>> On Tue, Nov 10, 2020 at 11:10:27PM +0100, Andrey Konovalov wrote:
>>>> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
>>>>
>>>> Hardware tag-based KASAN relies on Memory Tagging Extension (MTE)
>>>> feature and requires it to be enabled. MTE supports
>>>>
>>>> This patch adds a new mte_init_tags() helper, that enables MTE in
>>>> Synchronous mode in EL1 and is intended to be called from KASAN runtime
>>>> during initialization.
>>>
>>> There's no mte_init_tags() in this function.
>>
>> During the rework, I realized that the description of mte_init_tags() in this
>> patch refers to mte_enable_kernel(). In fact the only thing that mte_init_tags()
>> does is to configure the GCR_EL1 register, hence my preference would be to keep
>> all the code that deals with such a register in one patch.
> 
> Fine by me as long as the commit text is consistent with the diff.
> 

Done already, it will be in the next series. Thank you for the quick turnaround.

-- 
Regards,
Vincenzo


^ permalink raw reply	[flat|nested] 107+ messages in thread

end of thread, other threads:[~2020-11-13 12:01 UTC | newest]

Thread overview: 107+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-10 22:09 [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrey Konovalov
2020-11-10 22:09 ` [PATCH v9 01/44] kasan: drop unnecessary GPL text from comment headers Andrey Konovalov
2020-11-11 13:16   ` Alexander Potapenko
2020-11-10 22:09 ` [PATCH v9 02/44] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Andrey Konovalov
2020-11-11 13:19   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 03/44] kasan: group vmalloc code Andrey Konovalov
2020-11-11 13:21   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 04/44] s390/kasan: include asm/page.h from asm/kasan.h Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 05/44] kasan: shadow declarations only for software modes Andrey Konovalov
2020-11-11 13:25   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 06/44] kasan: rename (un)poison_shadow to (un)poison_memory Andrey Konovalov
2020-11-11 13:26   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 07/44] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Andrey Konovalov
2020-11-11 13:45   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 08/44] kasan: only build init.c for software modes Andrey Konovalov
2020-11-11 13:46   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 09/44] kasan: split out shadow.c from common.c Andrey Konovalov
2020-11-11 14:07   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 10/44] kasan: define KASAN_GRANULE_PAGE Andrey Konovalov
2020-11-11 14:13   ` Alexander Potapenko
2020-11-11 18:47     ` Andrey Konovalov
2020-11-11 19:04       ` Marco Elver
2020-11-12  0:05         ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 11/44] kasan: rename report and tags files Andrey Konovalov
2020-11-11 14:15   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 12/44] kasan: don't duplicate config dependencies Andrey Konovalov
2020-11-11 14:20   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 13/44] kasan: hide invalid free check implementation Andrey Konovalov
2020-11-11 14:23   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 14/44] kasan: decode stack frame only with KASAN_STACK_ENABLE Andrey Konovalov
2020-11-11 14:25   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 15/44] kasan, arm64: only init shadow for software modes Andrey Konovalov
2020-11-11 14:30   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 16/44] kasan, arm64: only use kasan_depth " Andrey Konovalov
2020-11-11 14:59   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 17/44] kasan, arm64: move initialization message Andrey Konovalov
2020-11-11 15:03   ` Alexander Potapenko
2020-11-11 18:50     ` Andrey Konovalov
2020-11-12 15:09       ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 18/44] kasan, arm64: rename kasan_init_tags and mark as __init Andrey Konovalov
2020-11-11 15:05   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 19/44] kasan: rename addr_has_shadow to addr_has_metadata Andrey Konovalov
2020-11-11 15:06   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 20/44] kasan: rename print_shadow_for_address to print_memory_metadata Andrey Konovalov
2020-11-11 15:07   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 21/44] kasan: kasan_non_canonical_hook only for software modes Andrey Konovalov
2020-11-11 15:08   ` Alexander Potapenko
2020-11-11 18:52     ` Andrey Konovalov
2020-11-12 15:16       ` Alexander Potapenko
2020-11-12 19:27         ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 22/44] kasan: rename SHADOW layout macros to META Andrey Konovalov
2020-11-11 15:18   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 23/44] kasan: separate metadata_fetch_row for each mode Andrey Konovalov
2020-11-11 15:22   ` Alexander Potapenko
2020-11-11 16:19     ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 24/44] kasan, arm64: don't allow SW_TAGS with ARM64_MTE Andrey Konovalov
2020-11-11 15:52   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 25/44] kasan: introduce CONFIG_KASAN_HW_TAGS Andrey Konovalov
2020-11-11 15:58   ` Alexander Potapenko
2020-11-11 18:53     ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 26/44] arm64: Enable armv8.5-a asm-arch option Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 27/44] arm64: mte: Add in-kernel MTE helpers Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 28/44] arm64: mte: Reset the page tag in page->flags Andrey Konovalov
2020-11-12  9:31   ` Catalin Marinas
2020-11-12  9:39     ` Vincenzo Frascino
2020-11-10 22:10 ` [PATCH v9 29/44] arm64: mte: Add in-kernel tag fault handler Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 30/44] arm64: kasan: Allow enabling in-kernel MTE Andrey Konovalov
2020-11-12  9:43   ` Catalin Marinas
2020-11-12 16:46     ` Vincenzo Frascino
2020-11-13 11:17     ` Vincenzo Frascino
2020-11-13 12:00       ` Catalin Marinas
2020-11-13 12:04         ` Vincenzo Frascino
2020-11-10 22:10 ` [PATCH v9 31/44] arm64: mte: Convert gcr_user into an exclude mask Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 32/44] arm64: mte: Switch GCR_EL1 in kernel entry and exit Andrey Konovalov
2020-11-12  9:39   ` Catalin Marinas
2020-11-12  9:45     ` Vincenzo Frascino
2020-11-12  9:45       ` Catalin Marinas
2020-11-12  9:55         ` Vincenzo Frascino
2020-11-10 22:10 ` [PATCH v9 33/44] kasan, mm: untag page address in free_reserved_area Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 34/44] arm64: kasan: Align allocations for HW_TAGS Andrey Konovalov
2020-11-11 16:10   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 35/44] arm64: kasan: Add arch layer for memory tagging helpers Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 36/44] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Andrey Konovalov
2020-11-11 16:13   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 37/44] kasan, x86, s390: update undef CONFIG_KASAN Andrey Konovalov
2020-11-11 16:15   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 38/44] kasan, arm64: expand CONFIG_KASAN checks Andrey Konovalov
2020-11-11 16:22   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 39/44] kasan, arm64: implement HW_TAGS runtime Andrey Konovalov
2020-11-12 17:37   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 40/44] kasan, arm64: print report from tag fault handler Andrey Konovalov
2020-11-11 16:43   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 41/44] kasan, mm: reset tags when accessing metadata Andrey Konovalov
2020-11-12 16:10   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 42/44] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Andrey Konovalov
2020-11-11 16:49   ` Alexander Potapenko
2020-11-10 22:10 ` [PATCH v9 43/44] kasan: add documentation for hardware tag-based mode Andrey Konovalov
2020-11-11 16:57   ` Alexander Potapenko
2020-11-11 18:54     ` Andrey Konovalov
2020-11-10 22:10 ` [PATCH v9 44/44] kselftest/arm64: Check GCR_EL1 after context switch Andrey Konovalov
2020-11-12  9:46   ` Catalin Marinas
2020-11-12 15:59   ` Alexander Potapenko
2020-11-12 16:09     ` Marco Elver
2020-11-12 16:41       ` Alexander Potapenko
2020-11-13 11:47     ` Vincenzo Frascino
2020-11-10 22:54 ` [PATCH v9 00/44] kasan: add hardware tag-based mode for arm64 Andrew Morton
2020-11-11 14:35   ` Andrey Konovalov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).