[GIT PULL] Locking: Add the Kernel Concurrency Sanitizer (KCSAN) subsystem

[GIT PULL] Locking: Add the Kernel Concurrency Sanitizer (KCSAN) subsystem

[Ingo Molnar]

Linus,

Please pull the latest locking-kcsan-for-linus git tree from:

   git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking-kcsan-for-linus

   # HEAD: 5cbaefe9743bf14c9d3106db0cc19f8cb0a3ca22 kcsan: Improve various small stylistic details

This tree adds the Kernel Concurrency Sanitizer (KCSAN) subsystem 
authored by Marco Elver, which is a debugging facility that uses compiler 
instrumentation of data accesses to statistically detect and report data 
races on live kernels.

KCSAN uses the -fsanitize=thread build time instrumentation features of 
both GCC and Clang, which transforms all memory reads/writes into 
__tsan_*callbacks with addresses and access type flags passed in that 
KCSAN can process and turn into a global array of 'watchpoints' that 
denote ongoing accesses. If two CPUs happen upon each other via an unsafe 
(non-atomic) access then a warning is generated.

Most users probably don't want to enable CONFIG_KCSAN due to the 
significant runtime overhead, bloat and the intentional udelay()s added 
to widen data races:

   text     data    bss	
   19769486 5201776 1613896 vmlinux.defconfig
   30181608 5313912 1675336 vmlinux.defconfig.CONFIG_KCSAN=y

... but for kernel developers it's a powerful facility that has already 
found a number of concurrency bugs.

There's a significant set of false positives and early and low level code 
complications that require the whitelisting and blacklisting of various 
pieces of kernel code - but in practice the code is already functional 
enough to find races.

 Thanks,

	Ingo

------------------>
Ingo Molnar (1):
      kcsan: Improve various small stylistic details

Marco Elver (9):
      kcsan: Add Kernel Concurrency Sanitizer infrastructure
      include/linux/compiler.h: Introduce data_race(expr) macro
      kcsan: Add Documentation entry in dev-tools
      objtool, kcsan: Add KCSAN runtime functions to whitelist
      build, kcsan: Add KCSAN build exceptions
      seqlock, kcsan: Add annotations for KCSAN
      seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
      locking/atomics, kcsan: Add KCSAN instrumentation
      x86, kcsan: Enable KCSAN for x86


 Documentation/dev-tools/index.rst         |   1 +
 Documentation/dev-tools/kcsan.rst         | 256 ++++++++++++
 MAINTAINERS                               |  11 +
 Makefile                                  |   3 +-
 arch/x86/Kconfig                          |   1 +
 arch/x86/boot/Makefile                    |   2 +
 arch/x86/boot/compressed/Makefile         |   2 +
 arch/x86/entry/vdso/Makefile              |   3 +
 arch/x86/include/asm/bitops.h             |   6 +-
 arch/x86/kernel/Makefile                  |   4 +
 arch/x86/kernel/cpu/Makefile              |   3 +
 arch/x86/lib/Makefile                     |   4 +
 arch/x86/mm/Makefile                      |   4 +
 arch/x86/purgatory/Makefile               |   2 +
 arch/x86/realmode/Makefile                |   3 +
 arch/x86/realmode/rm/Makefile             |   3 +
 drivers/firmware/efi/libstub/Makefile     |   2 +
 include/asm-generic/atomic-instrumented.h | 393 ++++++++++---------
 include/linux/compiler-clang.h            |  11 +-
 include/linux/compiler-gcc.h              |   7 +
 include/linux/compiler.h                  |  57 ++-
 include/linux/kcsan-checks.h              |  93 +++++
 include/linux/kcsan.h                     | 108 ++++++
 include/linux/sched.h                     |   4 +
 include/linux/seqlock.h                   |  51 ++-
 init/init_task.c                          |   8 +
 init/main.c                               |   2 +
 kernel/Makefile                           |   6 +
 kernel/kcsan/Makefile                     |  11 +
 kernel/kcsan/atomic.h                     |  27 ++
 kernel/kcsan/core.c                       | 621 ++++++++++++++++++++++++++++++
 kernel/kcsan/debugfs.c                    | 271 +++++++++++++
 kernel/kcsan/encoding.h                   |  95 +++++
 kernel/kcsan/kcsan.h                      | 109 ++++++
 kernel/kcsan/report.c                     | 318 +++++++++++++++
 kernel/kcsan/test.c                       | 121 ++++++
 kernel/sched/Makefile                     |   6 +
 lib/Kconfig.debug                         |   2 +
 lib/Kconfig.kcsan                         | 116 ++++++
 lib/Makefile                              |   3 +
 mm/Makefile                               |   8 +
 scripts/Makefile.kcsan                    |   6 +
 scripts/Makefile.lib                      |  10 +
 scripts/atomic/gen-atomic-instrumented.sh |  17 +-
 tools/objtool/check.c                     |  18 +
 45 files changed, 2602 insertions(+), 207 deletions(-)
 create mode 100644 Documentation/dev-tools/kcsan.rst
 create mode 100644 include/linux/kcsan-checks.h
 create mode 100644 include/linux/kcsan.h
 create mode 100644 kernel/kcsan/Makefile
 create mode 100644 kernel/kcsan/atomic.h
 create mode 100644 kernel/kcsan/core.c
 create mode 100644 kernel/kcsan/debugfs.c
 create mode 100644 kernel/kcsan/encoding.h
 create mode 100644 kernel/kcsan/kcsan.h
 create mode 100644 kernel/kcsan/report.c
 create mode 100644 kernel/kcsan/test.c
 create mode 100644 lib/Kconfig.kcsan
 create mode 100644 scripts/Makefile.kcsan