* [PATCH RFC v3 01/36] stackdepot: check depot_index before accessing the stack slab
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
@ 2019-11-22 11:25 ` glider
2019-11-27 14:22 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 02/36] stackdepot: build with -fno-builtin glider
` (34 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
Avoid crashes on corrupted stack ids.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v3:
- fix the return statement
Change-Id: I0a0b38ed5057090696a2c6ff0be7cfcc24ae6738
---
lib/stackdepot.c | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index ed717dd08ff3..0bc6182bc7a6 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -198,9 +198,22 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle,
unsigned long **entries)
{
union handle_parts parts = { .handle = handle };
- void *slab = stack_slabs[parts.slabindex];
+ void *slab;
size_t offset = parts.offset << STACK_ALLOC_ALIGN;
- struct stack_record *stack = slab + offset;
+ struct stack_record *stack;
+
+ if (parts.slabindex > depot_index) {
+ WARN(1, "slab index %d out of bounds (%d) for stack id %08x\n",
+ parts.slabindex, depot_index, handle);
+ *entries = NULL;
+ return 0;
+ }
+ slab = stack_slabs[parts.slabindex];
+ stack = slab + offset;
+ if (!stack) {
+ entries = NULL;
+ return 0;
+ }
*entries = stack->entries;
return stack->size;
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 01/36] stackdepot: check depot_index before accessing the stack slab
2019-11-22 11:25 ` [PATCH RFC v3 01/36] stackdepot: check depot_index before accessing the stack slab glider
@ 2019-11-27 14:22 ` Marco Elver
0 siblings, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-11-27 14:22 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, adilger.kernel, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
hch, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
Eric Dumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, Mark Rutland,
martin.petersen, schwidefsky, Matthew Wilcox, mst, monstr,
pmladek, Qian Cai, Randy Dunlap, robin.murphy,
sergey.senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> Avoid crashes on corrupted stack ids.
Under what circumstances can they be corrupted?
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
> v3:
> - fix the return statement
>
> Change-Id: I0a0b38ed5057090696a2c6ff0be7cfcc24ae6738
> ---
> lib/stackdepot.c | 17 +++++++++++++++--
> 1 file changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> index ed717dd08ff3..0bc6182bc7a6 100644
> --- a/lib/stackdepot.c
> +++ b/lib/stackdepot.c
> @@ -198,9 +198,22 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle,
> unsigned long **entries)
> {
> union handle_parts parts = { .handle = handle };
> - void *slab = stack_slabs[parts.slabindex];
> + void *slab;
> size_t offset = parts.offset << STACK_ALLOC_ALIGN;
> - struct stack_record *stack = slab + offset;
> + struct stack_record *stack;
> +
> + if (parts.slabindex > depot_index) {
> + WARN(1, "slab index %d out of bounds (%d) for stack id %08x\n",
> + parts.slabindex, depot_index, handle);
On syzbot with panic_on_warn this will crash the kernel. Is this
desirable? Or is a pr_err here more appropriate?
> + *entries = NULL;
> + return 0;
> + }
> + slab = stack_slabs[parts.slabindex];
> + stack = slab + offset;
> + if (!stack) {
> + entries = NULL;
> + return 0;
> + }
>
> *entries = stack->entries;
> return stack->size;
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 02/36] stackdepot: build with -fno-builtin
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
2019-11-22 11:25 ` [PATCH RFC v3 01/36] stackdepot: check depot_index before accessing the stack slab glider
@ 2019-11-22 11:25 ` glider
2019-11-27 14:22 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 03/36] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c glider
` (33 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, Sergey Senozhatsky, Arnd Bergmann,
Andrey Ryabinin, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, luto,
ard.biesheuvel, hch, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, rostedt, tiwai, tytso, tglx, gor, wsa
Clang may replace stackdepot_memcmp() with a call to instrumented bcmp(),
which is exactly what we wanted to avoid creating stackdepot_memcmp().
Building the file with -fno-builtin prevents such optimizations.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: linux-mm@kvack.org
---
This patch was previously called "stackdepot: prevent Clang from optimizing
away stackdepot_memcmp()".
v3:
- use -fno-builtin instead of a barrier
Change-Id: I4495b617b15c0ab003a61c1f0d54d0026fa8b144
---
lib/Makefile | 4 ++++
lib/stackdepot.c | 2 +-
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/Makefile b/lib/Makefile
index c5892807e06f..58a3e1b1a868 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -215,6 +215,10 @@ obj-$(CONFIG_SG_POOL) += sg_pool.o
obj-$(CONFIG_STMP_DEVICE) += stmp_device.o
obj-$(CONFIG_IRQ_POLL) += irq_poll.o
+# stackdepot.c should not be instrumented or call instrumented functions.
+# Prevent the compiler from calling builtins like memcmp() or bcmp() from this
+# file.
+CFLAGS_stackdepot.o += -fno-builtin
obj-$(CONFIG_STACKDEPOT) += stackdepot.o
KASAN_SANITIZE_stackdepot.o := n
KCOV_INSTRUMENT_stackdepot.o := n
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index 0bc6182bc7a6..6d1123123e56 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -163,7 +163,7 @@ int stackdepot_memcmp(const unsigned long *u1, const unsigned long *u2,
unsigned int n)
{
for ( ; n-- ; u1++, u2++) {
- if (*u1 != *u2)
+ if ((*u1) != (*u2))
return 1;
}
return 0;
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 02/36] stackdepot: build with -fno-builtin
2019-11-22 11:25 ` [PATCH RFC v3 02/36] stackdepot: build with -fno-builtin glider
@ 2019-11-27 14:22 ` Marco Elver
0 siblings, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-11-27 14:22 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Sergey Senozhatsky, Arnd Bergmann,
Andrey Ryabinin, Linux Memory Management List, Al Viro,
adilger.kernel, Andrew Morton, Andrey Konovalov, Andy Lutomirski,
Ard Biesheuvel, hch, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, Eric Dumazet, ericvh, gregkh, harry.wentland, herbert,
iii, mingo, jasowang, axboe, m.szyprowski, Mark Rutland,
martin.petersen, schwidefsky, Matthew Wilcox, mst, monstr,
pmladek, Qian Cai, Randy Dunlap, robin.murphy, Steven Rostedt,
tiwai, tytso, Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> Clang may replace stackdepot_memcmp() with a call to instrumented bcmp(),
> which is exactly what we wanted to avoid creating stackdepot_memcmp().
> Building the file with -fno-builtin prevents such optimizations.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: linux-mm@kvack.org
> ---
> This patch was previously called "stackdepot: prevent Clang from optimizing
> away stackdepot_memcmp()".
>
> v3:
> - use -fno-builtin instead of a barrier
>
> Change-Id: I4495b617b15c0ab003a61c1f0d54d0026fa8b144
> ---
> lib/Makefile | 4 ++++
> lib/stackdepot.c | 2 +-
> 2 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/lib/Makefile b/lib/Makefile
> index c5892807e06f..58a3e1b1a868 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -215,6 +215,10 @@ obj-$(CONFIG_SG_POOL) += sg_pool.o
> obj-$(CONFIG_STMP_DEVICE) += stmp_device.o
> obj-$(CONFIG_IRQ_POLL) += irq_poll.o
>
> +# stackdepot.c should not be instrumented or call instrumented functions.
> +# Prevent the compiler from calling builtins like memcmp() or bcmp() from this
> +# file.
> +CFLAGS_stackdepot.o += -fno-builtin
> obj-$(CONFIG_STACKDEPOT) += stackdepot.o
> KASAN_SANITIZE_stackdepot.o := n
> KCOV_INSTRUMENT_stackdepot.o := n
> diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> index 0bc6182bc7a6..6d1123123e56 100644
> --- a/lib/stackdepot.c
> +++ b/lib/stackdepot.c
> @@ -163,7 +163,7 @@ int stackdepot_memcmp(const unsigned long *u1, const unsigned long *u2,
> unsigned int n)
> {
> for ( ; n-- ; u1++, u2++) {
> - if (*u1 != *u2)
> + if ((*u1) != (*u2))
Why this change? Can stackdepot.c be reverted?
> return 1;
> }
> return 0;
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 03/36] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
2019-11-22 11:25 ` [PATCH RFC v3 01/36] stackdepot: check depot_index before accessing the stack slab glider
2019-11-22 11:25 ` [PATCH RFC v3 02/36] stackdepot: build with -fno-builtin glider
@ 2019-11-22 11:25 ` glider
2019-11-27 14:22 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 04/36] stackdepot: reserve 5 extra bits in depot_stack_handle_t glider
` (32 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, Andrey Ryabinin, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
filter_irq_stacks() can be used by other tools (e.g. KMSAN), so it needs
to be moved to a common location.
lib/stackdepot.c seems a good place, as filter_irq_stacks() is usually
applied to the output of stack_trace_save().
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: linux-mm@kvack.org
---
Change-Id: I65acecf64930a3658e8c2aa7423801082ded8602
---
include/linux/stackdepot.h | 2 ++
lib/stackdepot.c | 23 +++++++++++++++++++++++
mm/kasan/common.c | 23 -----------------------
3 files changed, 25 insertions(+), 23 deletions(-)
diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h
index 3efa97d482cb..24d49c732341 100644
--- a/include/linux/stackdepot.h
+++ b/include/linux/stackdepot.h
@@ -19,4 +19,6 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
unsigned int stack_depot_fetch(depot_stack_handle_t handle,
unsigned long **entries);
+unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
+
#endif
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index 6d1123123e56..eb95197b8743 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -20,6 +20,7 @@
*/
#include <linux/gfp.h>
+#include <linux/interrupt.h>
#include <linux/jhash.h>
#include <linux/kernel.h>
#include <linux/mm.h>
@@ -314,3 +315,25 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
return retval;
}
EXPORT_SYMBOL_GPL(stack_depot_save);
+
+static inline int in_irqentry_text(unsigned long ptr)
+{
+ return (ptr >= (unsigned long)&__irqentry_text_start &&
+ ptr < (unsigned long)&__irqentry_text_end) ||
+ (ptr >= (unsigned long)&__softirqentry_text_start &&
+ ptr < (unsigned long)&__softirqentry_text_end);
+}
+
+unsigned int filter_irq_stacks(unsigned long *entries,
+ unsigned int nr_entries)
+{
+ unsigned int i;
+
+ for (i = 0; i < nr_entries; i++) {
+ if (in_irqentry_text(entries[i])) {
+ /* Include the irqentry function into the stack. */
+ return i + 1;
+ }
+ }
+ return nr_entries;
+}
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 6814d6d6a023..154eba5700d8 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -15,7 +15,6 @@
*/
#include <linux/export.h>
-#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/kasan.h>
#include <linux/kernel.h>
@@ -39,28 +38,6 @@
#include "kasan.h"
#include "../slab.h"
-static inline int in_irqentry_text(unsigned long ptr)
-{
- return (ptr >= (unsigned long)&__irqentry_text_start &&
- ptr < (unsigned long)&__irqentry_text_end) ||
- (ptr >= (unsigned long)&__softirqentry_text_start &&
- ptr < (unsigned long)&__softirqentry_text_end);
-}
-
-static inline unsigned int filter_irq_stacks(unsigned long *entries,
- unsigned int nr_entries)
-{
- unsigned int i;
-
- for (i = 0; i < nr_entries; i++) {
- if (in_irqentry_text(entries[i])) {
- /* Include the irqentry function into the stack. */
- return i + 1;
- }
- }
- return nr_entries;
-}
-
static inline depot_stack_handle_t save_stack(gfp_t flags)
{
unsigned long entries[KASAN_STACK_DEPTH];
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 03/36] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c
2019-11-22 11:25 ` [PATCH RFC v3 03/36] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c glider
@ 2019-11-27 14:22 ` Marco Elver
2019-11-27 14:56 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-11-27 14:22 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Andrey Ryabinin,
Linux Memory Management List, Al Viro, adilger.kernel,
Andrew Morton, Andrey Konovalov, Andy Lutomirski, Ard Biesheuvel,
Arnd Bergmann, hch, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, Eric Dumazet, ericvh, gregkh, harry.wentland, herbert,
iii, mingo, jasowang, axboe, m.szyprowski, Mark Rutland,
martin.petersen, schwidefsky, Matthew Wilcox, mst, monstr,
pmladek, Qian Cai, Randy Dunlap, robin.murphy,
Sergey Senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
[...]
> diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h
> index 3efa97d482cb..24d49c732341 100644
> --- a/include/linux/stackdepot.h
> +++ b/include/linux/stackdepot.h
> @@ -19,4 +19,6 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
> unsigned int stack_depot_fetch(depot_stack_handle_t handle,
> unsigned long **entries);
>
> +unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
> +
> #endif
> diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> index 6d1123123e56..eb95197b8743 100644
> --- a/lib/stackdepot.c
> +++ b/lib/stackdepot.c
> @@ -20,6 +20,7 @@
> */
>
> #include <linux/gfp.h>
> +#include <linux/interrupt.h>
> #include <linux/jhash.h>
> #include <linux/kernel.h>
> #include <linux/mm.h>
> @@ -314,3 +315,25 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
> return retval;
> }
> EXPORT_SYMBOL_GPL(stack_depot_save);
> +
> +static inline int in_irqentry_text(unsigned long ptr)
> +{
> + return (ptr >= (unsigned long)&__irqentry_text_start &&
> + ptr < (unsigned long)&__irqentry_text_end) ||
> + (ptr >= (unsigned long)&__softirqentry_text_start &&
> + ptr < (unsigned long)&__softirqentry_text_end);
> +}
> +
> +unsigned int filter_irq_stacks(unsigned long *entries,
> + unsigned int nr_entries)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < nr_entries; i++) {
> + if (in_irqentry_text(entries[i])) {
> + /* Include the irqentry function into the stack. */
> + return i + 1;
> + }
> + }
> + return nr_entries;
> +}
Does this need an EXPORT_SYMBOL_GPL ?
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 03/36] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c
2019-11-27 14:22 ` Marco Elver
@ 2019-11-27 14:56 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-11-27 14:56 UTC (permalink / raw)
To: Marco Elver
Cc: Vegard Nossum, Dmitry Vyukov, Andrey Ryabinin,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andy Lutomirski, Ard Biesheuvel,
Arnd Bergmann, hch, Christoph Hellwig, darrick.wong,
David Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, jasowang, Jens Axboe,
Marek Szyprowski, Mark Rutland, Martin K . Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Wed, Nov 27, 2019 at 3:23 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
> [...]
> > diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h
> > index 3efa97d482cb..24d49c732341 100644
> > --- a/include/linux/stackdepot.h
> > +++ b/include/linux/stackdepot.h
> > @@ -19,4 +19,6 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
> > unsigned int stack_depot_fetch(depot_stack_handle_t handle,
> > unsigned long **entries);
> >
> > +unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
> > +
> > #endif
> > diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> > index 6d1123123e56..eb95197b8743 100644
> > --- a/lib/stackdepot.c
> > +++ b/lib/stackdepot.c
> > @@ -20,6 +20,7 @@
> > */
> >
> > #include <linux/gfp.h>
> > +#include <linux/interrupt.h>
> > #include <linux/jhash.h>
> > #include <linux/kernel.h>
> > #include <linux/mm.h>
> > @@ -314,3 +315,25 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
> > return retval;
> > }
> > EXPORT_SYMBOL_GPL(stack_depot_save);
> > +
> > +static inline int in_irqentry_text(unsigned long ptr)
> > +{
> > + return (ptr >= (unsigned long)&__irqentry_text_start &&
> > + ptr < (unsigned long)&__irqentry_text_end) ||
> > + (ptr >= (unsigned long)&__softirqentry_text_start &&
> > + ptr < (unsigned long)&__softirqentry_text_end);
> > +}
> > +
> > +unsigned int filter_irq_stacks(unsigned long *entries,
> > + unsigned int nr_entries)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < nr_entries; i++) {
> > + if (in_irqentry_text(entries[i])) {
> > + /* Include the irqentry function into the stack. */
> > + return i + 1;
> > + }
> > + }
> > + return nr_entries;
> > +}
>
> Does this need an EXPORT_SYMBOL_GPL ?
Yes, makes sense.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 04/36] stackdepot: reserve 5 extra bits in depot_stack_handle_t
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (2 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 03/36] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c glider
@ 2019-11-22 11:25 ` glider
2019-11-27 14:23 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 05/36] kmsan: add ReST documentation glider
` (31 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
Some users (currently only KMSAN) may want to use spare bits in
depot_stack_handle_t. Let them do so and provide get_dsh_extra_bits()
and set_dsh_extra_bits() to access those bits.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I23580dbde85908eeda0bdd8f83a8c3882ab3e012
---
include/linux/stackdepot.h | 8 ++++++++
lib/stackdepot.c | 24 +++++++++++++++++++++++-
2 files changed, 31 insertions(+), 1 deletion(-)
diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h
index 24d49c732341..ac1b5a78d7f6 100644
--- a/include/linux/stackdepot.h
+++ b/include/linux/stackdepot.h
@@ -12,6 +12,11 @@
#define _LINUX_STACKDEPOT_H
typedef u32 depot_stack_handle_t;
+/*
+ * Number of bits in the handle that stack depot doesn't use. Users may store
+ * information in them.
+ */
+#define STACK_DEPOT_EXTRA_BITS 5
depot_stack_handle_t stack_depot_save(unsigned long *entries,
unsigned int nr_entries, gfp_t gfp_flags);
@@ -20,5 +25,8 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle,
unsigned long **entries);
unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
+depot_stack_handle_t set_dsh_extra_bits(depot_stack_handle_t handle,
+ unsigned int bits);
+unsigned int get_dsh_extra_bits(depot_stack_handle_t handle);
#endif
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index eb95197b8743..e2f000a9fad8 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -40,8 +40,10 @@
#define STACK_ALLOC_ALIGN 4
#define STACK_ALLOC_OFFSET_BITS (STACK_ALLOC_ORDER + PAGE_SHIFT - \
STACK_ALLOC_ALIGN)
+
#define STACK_ALLOC_INDEX_BITS (DEPOT_STACK_BITS - \
- STACK_ALLOC_NULL_PROTECTION_BITS - STACK_ALLOC_OFFSET_BITS)
+ STACK_ALLOC_NULL_PROTECTION_BITS - \
+ STACK_ALLOC_OFFSET_BITS - STACK_DEPOT_EXTRA_BITS)
#define STACK_ALLOC_SLABS_CAP 8192
#define STACK_ALLOC_MAX_SLABS \
(((1LL << (STACK_ALLOC_INDEX_BITS)) < STACK_ALLOC_SLABS_CAP) ? \
@@ -54,6 +56,7 @@ union handle_parts {
u32 slabindex : STACK_ALLOC_INDEX_BITS;
u32 offset : STACK_ALLOC_OFFSET_BITS;
u32 valid : STACK_ALLOC_NULL_PROTECTION_BITS;
+ u32 extra : STACK_DEPOT_EXTRA_BITS;
};
};
@@ -72,6 +75,24 @@ static int next_slab_inited;
static size_t depot_offset;
static DEFINE_SPINLOCK(depot_lock);
+depot_stack_handle_t set_dsh_extra_bits(depot_stack_handle_t handle,
+ u32 bits)
+{
+ union handle_parts parts = { .handle = handle };
+
+ parts.extra = bits & ((1U << STACK_DEPOT_EXTRA_BITS) - 1);
+ return parts.handle;
+}
+EXPORT_SYMBOL_GPL(set_dsh_extra_bits);
+
+u32 get_dsh_extra_bits(depot_stack_handle_t handle)
+{
+ union handle_parts parts = { .handle = handle };
+
+ return parts.extra;
+}
+EXPORT_SYMBOL_GPL(get_dsh_extra_bits);
+
static bool init_stack_slab(void **prealloc)
{
if (!*prealloc)
@@ -132,6 +153,7 @@ static struct stack_record *depot_alloc_stack(unsigned long *entries, int size,
stack->handle.slabindex = depot_index;
stack->handle.offset = depot_offset >> STACK_ALLOC_ALIGN;
stack->handle.valid = 1;
+ stack->handle.extra = 0;
memcpy(stack->entries, entries, size * sizeof(unsigned long));
depot_offset += required_size;
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 04/36] stackdepot: reserve 5 extra bits in depot_stack_handle_t
2019-11-22 11:25 ` [PATCH RFC v3 04/36] stackdepot: reserve 5 extra bits in depot_stack_handle_t glider
@ 2019-11-27 14:23 ` Marco Elver
0 siblings, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-11-27 14:23 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, adilger.kernel, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
hch, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
Eric Dumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, Mark Rutland,
martin.petersen, schwidefsky, Matthew Wilcox, mst, monstr,
pmladek, Qian Cai, Randy Dunlap, robin.murphy,
Sergey Senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
[...]
> diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h
> index 24d49c732341..ac1b5a78d7f6 100644
> --- a/include/linux/stackdepot.h
> +++ b/include/linux/stackdepot.h
> @@ -12,6 +12,11 @@
> #define _LINUX_STACKDEPOT_H
>
> typedef u32 depot_stack_handle_t;
> +/*
> + * Number of bits in the handle that stack depot doesn't use. Users may store
> + * information in them.
> + */
> +#define STACK_DEPOT_EXTRA_BITS 5
>
> depot_stack_handle_t stack_depot_save(unsigned long *entries,
> unsigned int nr_entries, gfp_t gfp_flags);
> @@ -20,5 +25,8 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle,
> unsigned long **entries);
>
> unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries);
> +depot_stack_handle_t set_dsh_extra_bits(depot_stack_handle_t handle,
> + unsigned int bits);
The function declaration mismatches its definition ('unsigned bits' vs 'u32').
> +unsigned int get_dsh_extra_bits(depot_stack_handle_t handle);
This also doesn't match the definition.
The abbreviation 'dsh' in these function names is not very readable.
Maybe just '{set,get}_stack_depot_extra' ?
> #endif
> diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> index eb95197b8743..e2f000a9fad8 100644
> --- a/lib/stackdepot.c
> +++ b/lib/stackdepot.c
> @@ -40,8 +40,10 @@
> #define STACK_ALLOC_ALIGN 4
> #define STACK_ALLOC_OFFSET_BITS (STACK_ALLOC_ORDER + PAGE_SHIFT - \
> STACK_ALLOC_ALIGN)
> +
> #define STACK_ALLOC_INDEX_BITS (DEPOT_STACK_BITS - \
> - STACK_ALLOC_NULL_PROTECTION_BITS - STACK_ALLOC_OFFSET_BITS)
> + STACK_ALLOC_NULL_PROTECTION_BITS - \
> + STACK_ALLOC_OFFSET_BITS - STACK_DEPOT_EXTRA_BITS)
> #define STACK_ALLOC_SLABS_CAP 8192
> #define STACK_ALLOC_MAX_SLABS \
> (((1LL << (STACK_ALLOC_INDEX_BITS)) < STACK_ALLOC_SLABS_CAP) ? \
> @@ -54,6 +56,7 @@ union handle_parts {
> u32 slabindex : STACK_ALLOC_INDEX_BITS;
> u32 offset : STACK_ALLOC_OFFSET_BITS;
> u32 valid : STACK_ALLOC_NULL_PROTECTION_BITS;
> + u32 extra : STACK_DEPOT_EXTRA_BITS;
Would a BUILD_BUG_ON somewhere to assert that the total bits do not
exceed DEPOT_STACK_BITS make sense?
> };
> };
>
> @@ -72,6 +75,24 @@ static int next_slab_inited;
> static size_t depot_offset;
> static DEFINE_SPINLOCK(depot_lock);
>
> +depot_stack_handle_t set_dsh_extra_bits(depot_stack_handle_t handle,
> + u32 bits)
> +{
> + union handle_parts parts = { .handle = handle };
> +
> + parts.extra = bits & ((1U << STACK_DEPOT_EXTRA_BITS) - 1);
> + return parts.handle;
> +}
> +EXPORT_SYMBOL_GPL(set_dsh_extra_bits);
> +
> +u32 get_dsh_extra_bits(depot_stack_handle_t handle)
> +{
> + union handle_parts parts = { .handle = handle };
> +
> + return parts.extra;
> +}
> +EXPORT_SYMBOL_GPL(get_dsh_extra_bits);
> +
> static bool init_stack_slab(void **prealloc)
> {
> if (!*prealloc)
> @@ -132,6 +153,7 @@ static struct stack_record *depot_alloc_stack(unsigned long *entries, int size,
> stack->handle.slabindex = depot_index;
> stack->handle.offset = depot_offset >> STACK_ALLOC_ALIGN;
> stack->handle.valid = 1;
> + stack->handle.extra = 0;
> memcpy(stack->entries, entries, size * sizeof(unsigned long));
> depot_offset += required_size;
>
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 05/36] kmsan: add ReST documentation
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (3 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 04/36] stackdepot: reserve 5 extra bits in depot_stack_handle_t glider
@ 2019-11-22 11:25 ` glider
2019-11-27 14:22 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 06/36] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW glider
` (30 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
Add Documentation/dev-tools/kmsan.rst and reference it in the dev-tools
index.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: Iac6345065e6804ef811f1124fdf779c67ff1530e
---
Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kmsan.rst | 418 ++++++++++++++++++++++++++++++
2 files changed, 419 insertions(+)
create mode 100644 Documentation/dev-tools/kmsan.rst
diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
index b0522a4dd107..bc5e3fd87efa 100644
--- a/Documentation/dev-tools/index.rst
+++ b/Documentation/dev-tools/index.rst
@@ -19,6 +19,7 @@ whole; patches welcome!
kcov
gcov
kasan
+ kmsan
ubsan
kmemleak
gdb-kernel-debugging
diff --git a/Documentation/dev-tools/kmsan.rst b/Documentation/dev-tools/kmsan.rst
new file mode 100644
index 000000000000..51f9c207cc2c
--- /dev/null
+++ b/Documentation/dev-tools/kmsan.rst
@@ -0,0 +1,418 @@
+=============================
+KernelMemorySanitizer (KMSAN)
+=============================
+
+KMSAN is a dynamic memory error detector aimed at finding uses of uninitialized
+memory.
+It is based on compiler instrumentation, and is quite similar to the userspace
+MemorySanitizer tool (http://clang.llvm.org/docs/MemorySanitizer.html).
+
+KMSAN and Clang
+===============
+
+In order for KMSAN to work the kernel must be
+built with Clang, which is so far the only compiler that has KMSAN support.
+The kernel instrumentation pass is based on the userspace MemorySanitizer tool
+(http://clang.llvm.org/docs/MemorySanitizer.html). Because of the
+instrumentation complexity it's unlikely that any other compiler will support
+KMSAN soon.
+
+Right now the instrumentation pass supports x86_64 only.
+
+How to build
+============
+
+In order to build a kernel with KMSAN you'll need a fresh Clang (10.0.0+, trunk
+version r365008 or greater). Please refer to
+https://llvm.org/docs/GettingStarted.html for the instructions on how to build
+Clang::
+
+ export KMSAN_CLANG_PATH=/path/to/clang
+ # Now configure and build the kernel with CONFIG_KMSAN enabled.
+ make CC=$KMSAN_CLANG_PATH -j64
+
+How KMSAN works
+===============
+
+KMSAN shadow memory
+-------------------
+
+KMSAN associates a so-called shadow byte with every byte of kernel memory.
+A bit in the shadow byte is set iff the corresponding bit of the kernel memory
+byte is uninitialized.
+Marking the memory uninitialized (i.e. setting its shadow bytes to 0xff) is
+called poisoning, marking it initialized (setting the shadow bytes to 0x00) is
+called unpoisoning.
+
+When a new variable is allocated on the stack, it's poisoned by default by
+instrumentation code inserted by the compiler (unless it's a stack variable that
+is immediately initialized). Any new heap allocation done without ``__GFP_ZERO``
+is also poisoned.
+
+Compiler instrumentation also tracks the shadow values with the help from the
+runtime library in ``mm/kmsan/``.
+
+The shadow value of a basic or compound type is an array of bytes of the same
+length.
+When a constant value is written into memory, that memory is unpoisoned.
+When a value is read from memory, its shadow memory is also obtained and
+propagated into all the operations which use that value. For every instruction
+that takes one or more values the compiler generates code that calculates the
+shadow of the result depending on those values and their shadows.
+
+Example::
+
+ int a = 0xff;
+ int b;
+ int c = a | b;
+
+In this case the shadow of ``a`` is ``0``, shadow of ``b`` is ``0xffffffff``,
+shadow of ``c`` is ``0xffffff00``. This means that the upper three bytes of
+``c`` are uninitialized, while the lower byte is initialized.
+
+
+Origin tracking
+---------------
+
+Every four bytes of kernel memory also have a so-called origin assigned to
+them.
+This origin describes the point in program execution at which the uninitialized
+value was created. Every origin is associated with a creation stack, which lets
+the user figure out what's going on.
+
+When an uninitialized variable is allocated on stack or heap, a new origin
+value is created, and that variable's origin is filled with that value.
+When a value is read from memory, its origin is also read and kept together
+with the shadow. For every instruction that takes one or more values the origin
+of the result is one of the origins corresponding to any of the uninitialized
+inputs.
+If a poisoned value is written into memory, its origin is written to the
+corresponding storage as well.
+
+Example 1::
+
+ int a = 0;
+ int b;
+ int c = a + b;
+
+In this case the origin of ``b`` is generated upon function entry, and is
+stored to the origin of ``c`` right before the addition result is written into
+memory.
+
+Several variables may share the same origin address, if they are stored in the
+same four-byte chunk.
+In this case every write to either variable updates the origin for all of them.
+
+Example 2::
+
+ int combine(short a, short b) {
+ union ret_t {
+ int i;
+ short s[2];
+ } ret;
+ ret.s[0] = a;
+ ret.s[1] = b;
+ return ret.i;
+ }
+
+If ``a`` is initialized and ``b`` is not, the shadow of the result would be
+0xffff0000, and the origin of the result would be the origin of ``b``.
+``ret.s[0]`` would have the same origin, but it will be never used, because
+that variable is initialized.
+
+If both function arguments are uninitialized, only the origin of the second
+argument is preserved.
+
+Origin chaining
+~~~~~~~~~~~~~~~
+To ease the debugging, KMSAN creates a new origin for every memory store.
+The new origin references both its creation stack and the previous origin the
+memory location had.
+This may cause increased memory consumption, so we limit the length of origin
+chains in the runtime.
+
+Clang instrumentation API
+-------------------------
+
+Clang instrumentation pass inserts calls to functions defined in
+``mm/kmsan/kmsan_instr.c`` into the kernel code.
+
+Shadow manipulation
+~~~~~~~~~~~~~~~~~~~
+For every memory access the compiler emits a call to a function that returns a
+pair of pointers to the shadow and origin addresses of the given memory::
+
+ typedef struct {
+ void *s, *o;
+ } shadow_origin_ptr_t
+
+ shadow_origin_ptr_t __msan_metadata_ptr_for_load_{1,2,4,8}(void *addr)
+ shadow_origin_ptr_t __msan_metadata_ptr_for_store_{1,2,4,8}(void *addr)
+ shadow_origin_ptr_t __msan_metadata_ptr_for_load_n(void *addr, u64 size)
+ shadow_origin_ptr_t __msan_metadata_ptr_for_store_n(void *addr, u64 size)
+
+The function name depends on the memory access size.
+Each such function also checks if the shadow of the memory in the range
+[``addr``, ``addr + n``) is contiguous and reports an error otherwise.
+
+The compiler makes sure that for every loaded value its shadow and origin
+values are read from memory.
+When a value is stored to memory, its shadow and origin are also stored using
+the metadata pointers.
+
+Origin tracking
+~~~~~~~~~~~~~~~
+A special function is used to create a new origin value for a local variable
+and set the origin of that variable to that value::
+
+ void __msan_poison_alloca(u64 address, u64 size, char *descr)
+
+Access to per-task data
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+At the beginning of every instrumented function KMSAN inserts a call to
+``__msan_get_context_state()``::
+
+ kmsan_context_state *__msan_get_context_state(void)
+
+``kmsan_context_state`` is declared in ``include/linux/kmsan.h``::
+
+ struct kmsan_context_s {
+ char param_tls[KMSAN_PARAM_SIZE];
+ char retval_tls[RETVAL_SIZE];
+ char va_arg_tls[KMSAN_PARAM_SIZE];
+ char va_arg_origin_tls[KMSAN_PARAM_SIZE];
+ u64 va_arg_overflow_size_tls;
+ depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE];
+ depot_stack_handle_t retval_origin_tls;
+ depot_stack_handle_t origin_tls;
+ };
+
+This structure is used by KMSAN to pass parameter shadows and origins between
+instrumented functions.
+
+String functions
+~~~~~~~~~~~~~~~~
+
+The compiler replaces calls to ``memcpy()``/``memmove()``/``memset()`` with the
+following functions. These functions are also called when data structures are
+initialized or copied, making sure shadow and origin values are copied alongside
+with the data::
+
+ void *__msan_memcpy(void *dst, void *src, u64 n)
+ void *__msan_memmove(void *dst, void *src, u64 n)
+ void *__msan_memset(void *dst, int c, size_t n)
+
+Error reporting
+~~~~~~~~~~~~~~~
+
+For each pointer dereference and each condition the compiler emits a shadow
+check that calls ``__msan_warning()`` in the case a poisoned value is being
+used::
+
+ void __msan_warning(u32 origin)
+
+``__msan_warning()`` causes KMSAN runtime to print an error report.
+
+Inline assembly instrumentation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+KMSAN instruments every inline assembly output with a call to::
+
+ void __msan_instrument_asm_store(u64 addr, u64 size)
+
+, which unpoisons the memory region.
+
+This approach may mask certain errors, but it also helps to avoid a lot of
+false positives in bitwise operations, atomics etc.
+
+Sometimes the pointers passed into inline assembly don't point to valid memory.
+In such cases they are ignored at runtime.
+
+Disabling the instrumentation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A function can be marked with ``__no_sanitize_memory``.
+Doing so doesn't remove KMSAN instrumentation from it, however it makes the
+compiler ignore the uninitialized values coming from the function's inputs,
+and initialize the function's outputs.
+The compiler won't inline functions marked with this attribute into functions
+not marked with it, and vice versa.
+
+It's also possible to disable KMSAN for a single file (e.g. main.o)::
+
+ KMSAN_SANITIZE_main.o := n
+
+or for the whole directory::
+
+ KMSAN_SANITIZE := n
+
+in the Makefile. This comes at a cost however: stack allocations from such files
+and parameters of instrumented functions called from them will have incorrect
+shadow/origin values. As a rule of thumb, avoid using KMSAN_SANITIZE.
+
+Runtime library
+---------------
+The code is located in ``mm/kmsan/``.
+
+Per-task KMSAN state
+~~~~~~~~~~~~~~~~~~~~
+
+Every task_struct has an associated KMSAN task state that holds the KMSAN
+context (see above) and a per-task flag disallowing KMSAN reports::
+
+ struct kmsan_task_state {
+ ...
+ bool allow_reporting;
+ struct kmsan_context_state cstate;
+ ...
+ }
+
+ struct task_struct {
+ ...
+ struct kmsan_task_state kmsan;
+ ...
+ }
+
+
+KMSAN contexts
+~~~~~~~~~~~~~~
+
+When running in a kernel task context, KMSAN uses ``current->kmsan.cstate`` to
+hold the metadata for function parameters and return values.
+
+But in the case the kernel is running in the interrupt, softirq or NMI context,
+where ``current`` is unavailable, KMSAN switches to per-cpu interrupt state::
+
+ DEFINE_PER_CPU(kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
+ kmsan_percpu_cstate);
+
+Metadata allocation
+~~~~~~~~~~~~~~~~~~~
+There are several places in the kernel for which the metadata is stored.
+
+1. Each ``struct page`` instance contains two pointers to its shadow and
+origin pages::
+
+ struct page {
+ ...
+ struct page *shadow, *origin;
+ ...
+ };
+
+Every time a ``struct page`` is allocated, the runtime library allocates two
+additional pages to hold its shadow and origins. This is done by adding hooks
+to ``alloc_pages()``/``free_pages()`` in ``mm/page_alloc.c``.
+To avoid allocating the metadata for non-interesting pages (right now only the
+shadow/origin page themselves and stackdepot storage) the
+``__GFP_NO_KMSAN_SHADOW`` flag is used.
+
+There is a problem related to this allocation algorithm: when two contiguous
+memory blocks are allocated with two different ``alloc_pages()`` calls, their
+shadow pages may not be contiguous. So, if a memory access crosses the boundary
+of a memory block, accesses to shadow/origin memory may potentially corrupt
+other pages or read incorrect values from them.
+
+As a workaround, we check the access size in
+``__msan_metadata_ptr_for_XXX_YYY()`` and return a pointer to a fake shadow
+region in the case of an error::
+
+ char dummy_load_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+ char dummy_store_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+
+``dummy_load_page`` is zero-initialized, so reads from it always yield zeroes.
+All stores to ``dummy_store_page`` are ignored.
+
+Unfortunately at boot time we need to allocate shadow and origin pages for the
+kernel data (``.data``, ``.bss`` etc.) and percpu memory regions, the size of
+which is not a power of 2. As a result, we have to allocate the metadata page by
+page, so that it is also non-contiguous, although it may be perfectly valid to
+access the corresponding kernel memory across page boundaries.
+This can be probably fixed by allocating 1<<N pages at once, splitting them and
+deallocating the rest.
+
+LSB of the ``shadow`` pointer in a ``struct page`` may be set to 1. In this case
+shadow and origin pages are allocated, but KMSAN ignores accesses to them by
+falling back to dummy pages. Allocating the metadata pages is still needed to
+support ``vmap()/vunmap()`` operations on this struct page.
+
+2. For vmalloc memory and modules, there's a direct mapping between the memory
+range, its shadow and origin. KMSAN lessens the vmalloc area by 3/4, making only
+the first quarter available to ``vmalloc()``. The second quarter of the vmalloc
+area contains shadow memory for the first quarter, the third one holds the
+origins. A small part of the fourth quarter contains shadow and origins for the
+kernel modules. Please refer to ``arch/x86/include/asm/pgtable_64_types.h`` for
+more details.
+
+When an array of pages is mapped into a contiguous virtual memory space, their
+shadow and origin pages are similarly mapped into contiguous regions.
+
+3. For CPU entry area there're separate per-CPU arrays that hold its metadata::
+
+ DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
+ DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
+
+When calculating shadow and origin addresses for a given memory address, the
+runtime checks whether the address belongs to the physical page range, the
+virtual page range or CPU entry area.
+
+Handling ``pt_regs``
+~~~~~~~~~~~~~~~~~~~
+
+Many functions receive a ``struct pt_regs`` holding the register state at a
+certain point. Registers don't have (easily calculatable) shadow or origin
+associated with them.
+We can assume that the registers are always initialized.
+
+Example report
+--------------
+Here's an example of a real KMSAN report in ``packet_bind_spkt()``::
+
+ ==================================================================
+ BUG: KMSAN: uninit-value in strlen
+ CPU: 0 PID: 1074 Comm: packet Not tainted 4.8.0-rc6+ #1891
+ Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
+ 0000000000000000 ffff88006b6dfc08 ffffffff82559ae8 ffff88006b6dfb48
+ ffffffff818a7c91 ffffffff85b9c870 0000000000000092 ffffffff85b9c550
+ 0000000000000000 0000000000000092 00000000ec400911 0000000000000002
+ Call Trace:
+ [< inline >] __dump_stack lib/dump_stack.c:15
+ [<ffffffff82559ae8>] dump_stack+0x238/0x290 lib/dump_stack.c:51
+ [<ffffffff818a6626>] kmsan_report+0x276/0x2e0 mm/kmsan/kmsan.c:1003
+ [<ffffffff818a783b>] __msan_warning+0x5b/0xb0 mm/kmsan/kmsan_instr.c:424
+ [< inline >] strlen lib/string.c:484
+ [<ffffffff8259b58d>] strlcpy+0x9d/0x200 lib/string.c:144
+ [<ffffffff84b2eca4>] packet_bind_spkt+0x144/0x230 net/packet/af_packet.c:3132
+ [<ffffffff84242e4d>] SYSC_bind+0x40d/0x5f0 net/socket.c:1370
+ [<ffffffff84242a22>] SyS_bind+0x82/0xa0 net/socket.c:1356
+ [<ffffffff8515991b>] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:?
+ chained origin:
+ [<ffffffff810bb787>] save_stack_trace+0x27/0x50 arch/x86/kernel/stacktrace.c:67
+ [< inline >] kmsan_save_stack_with_flags mm/kmsan/kmsan.c:322
+ [< inline >] kmsan_save_stack mm/kmsan/kmsan.c:334
+ [<ffffffff818a59f8>] kmsan_internal_chain_origin+0x118/0x1e0 mm/kmsan/kmsan.c:527
+ [<ffffffff818a7773>] __msan_set_alloca_origin4+0xc3/0x130 mm/kmsan/kmsan_instr.c:380
+ [<ffffffff84242b69>] SYSC_bind+0x129/0x5f0 net/socket.c:1356
+ [<ffffffff84242a22>] SyS_bind+0x82/0xa0 net/socket.c:1356
+ [<ffffffff8515991b>] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:?
+ origin description: ----address@SYSC_bind (origin=00000000eb400911)
+ ==================================================================
+
+The report tells that the local variable ``address`` was created uninitialized
+in ``SYSC_bind()`` (the ``bind`` system call implementation). The lower stack
+trace corresponds to the place where this variable was created.
+
+The upper stack shows where the uninit value was used - in ``strlen()``.
+It turned out that the contents of ``address`` were partially copied from the
+userspace, but the buffer wasn't zero-terminated and contained some trailing
+uninitialized bytes.
+``packet_bind_spkt()`` didn't check the length of the buffer, but called
+``strlcpy()`` on it, which called ``strlen()``, which started reading the
+buffer byte by byte till it hit the uninitialized memory.
+
+
+References
+==========
+
+E. Stepanov, K. Serebryany. MemorySanitizer: fast detector of uninitialized
+memory use in C++.
+In Proceedings of CGO 2015.
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 05/36] kmsan: add ReST documentation
2019-11-22 11:25 ` [PATCH RFC v3 05/36] kmsan: add ReST documentation glider
@ 2019-11-27 14:22 ` Marco Elver
2019-12-03 12:42 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-11-27 14:22 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, adilger.kernel, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
hch, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
Eric Dumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, Mark Rutland,
martin.petersen, schwidefsky, Matthew Wilcox, mst, monstr,
pmladek, Qian Cai, Randy Dunlap, robin.murphy,
sergey.senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
General comments:
* it's -> it is
* don't -> do not
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
[...]
> diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
> index b0522a4dd107..bc5e3fd87efa 100644
> --- a/Documentation/dev-tools/index.rst
> +++ b/Documentation/dev-tools/index.rst
> @@ -19,6 +19,7 @@ whole; patches welcome!
> kcov
> gcov
> kasan
> + kmsan
> ubsan
> kmemleak
> gdb-kernel-debugging
> diff --git a/Documentation/dev-tools/kmsan.rst b/Documentation/dev-tools/kmsan.rst
> new file mode 100644
> index 000000000000..51f9c207cc2c
> --- /dev/null
> +++ b/Documentation/dev-tools/kmsan.rst
> @@ -0,0 +1,418 @@
> +=============================
> +KernelMemorySanitizer (KMSAN)
> +=============================
> +
> +KMSAN is a dynamic memory error detector aimed at finding uses of uninitialized
> +memory.
> +It is based on compiler instrumentation, and is quite similar to the userspace
> +MemorySanitizer tool (http://clang.llvm.org/docs/MemorySanitizer.html).
These should be real links: `Memory sanitizer tool <...url...>`_.
> +KMSAN and Clang
> +===============
> +
> +In order for KMSAN to work the kernel must be
> +built with Clang, which is so far the only compiler that has KMSAN support.
"is so far" -> "so far is"
> +The kernel instrumentation pass is based on the userspace MemorySanitizer tool
> +(http://clang.llvm.org/docs/MemorySanitizer.html). Because of the
Should also be real link: `MemorySanitizer tool <..url..>`_
> +instrumentation complexity it's unlikely that any other compiler will support
> +KMSAN soon.
> +
> +Right now the instrumentation pass supports x86_64 only.
> +
> +How to build
> +============
> +
> +In order to build a kernel with KMSAN you'll need a fresh Clang (10.0.0+, trunk
> +version r365008 or greater). Please refer to
> +https://llvm.org/docs/GettingStarted.html for the instructions on how to build
> +Clang::
> +
> + export KMSAN_CLANG_PATH=/path/to/clang
>
> + # Now configure and build the kernel with CONFIG_KMSAN enabled.
> + make CC=$KMSAN_CLANG_PATH -j64
I don't think '-j64' is necessary to build. Also the 'export' is
technically not required AFAIK, but I don't think it bothers anyone.
> +How KMSAN works
> +===============
> +
> +KMSAN shadow memory
> +-------------------
> +
> +KMSAN associates a so-called shadow byte with every byte of kernel memory.
'shadow' memory may not be a well-defined term. More intuitive would
be saying that it's metadata associated with every byte of kernel
memory. From then on you can say it's shadow memory.
> +A bit in the shadow byte is set iff the corresponding bit of the kernel memory
> +byte is uninitialized.
> +Marking the memory uninitialized (i.e. setting its shadow bytes to 0xff) is
> +called poisoning, marking it initialized (setting the shadow bytes to 0x00) is
> +called unpoisoning.
> +
> +When a new variable is allocated on the stack, it's poisoned by default by
> +instrumentation code inserted by the compiler (unless it's a stack variable that
> +is immediately initialized). Any new heap allocation done without ``__GFP_ZERO``
> +is also poisoned.
> +
> +Compiler instrumentation also tracks the shadow values with the help from the
> +runtime library in ``mm/kmsan/``.
> +
> +The shadow value of a basic or compound type is an array of bytes of the same
> +length.
> +When a constant value is written into memory, that memory is unpoisoned.
> +When a value is read from memory, its shadow memory is also obtained and
> +propagated into all the operations which use that value. For every instruction
> +that takes one or more values the compiler generates code that calculates the
> +shadow of the result depending on those values and their shadows.
> +
> +Example::
> +
> + int a = 0xff;
> + int b;
> + int c = a | b;
> +
> +In this case the shadow of ``a`` is ``0``, shadow of ``b`` is ``0xffffffff``,
> +shadow of ``c`` is ``0xffffff00``. This means that the upper three bytes of
> +``c`` are uninitialized, while the lower byte is initialized.
> +
> +
> +Origin tracking
> +---------------
> +
> +Every four bytes of kernel memory also have a so-called origin assigned to
> +them.
> +This origin describes the point in program execution at which the uninitialized
> +value was created. Every origin is associated with a creation stack, which lets
> +the user figure out what's going on.
> +
> +When an uninitialized variable is allocated on stack or heap, a new origin
> +value is created, and that variable's origin is filled with that value.
> +When a value is read from memory, its origin is also read and kept together
> +with the shadow. For every instruction that takes one or more values the origin
> +of the result is one of the origins corresponding to any of the uninitialized
> +inputs.
> +If a poisoned value is written into memory, its origin is written to the
> +corresponding storage as well.
> +
> +Example 1::
> +
> + int a = 0;
> + int b;
> + int c = a + b;
> +
> +In this case the origin of ``b`` is generated upon function entry, and is
> +stored to the origin of ``c`` right before the addition result is written into
> +memory.
> +
> +Several variables may share the same origin address, if they are stored in the
> +same four-byte chunk.
> +In this case every write to either variable updates the origin for all of them.
> +
> +Example 2::
> +
> + int combine(short a, short b) {
> + union ret_t {
> + int i;
> + short s[2];
> + } ret;
> + ret.s[0] = a;
> + ret.s[1] = b;
> + return ret.i;
> + }
> +
> +If ``a`` is initialized and ``b`` is not, the shadow of the result would be
> +0xffff0000, and the origin of the result would be the origin of ``b``.
> +``ret.s[0]`` would have the same origin, but it will be never used, because
> +that variable is initialized.
> +
> +If both function arguments are uninitialized, only the origin of the second
> +argument is preserved.
> +
> +Origin chaining
> +~~~~~~~~~~~~~~~
> +To ease the debugging, KMSAN creates a new origin for every memory store.
"the debugging" -> "debugging"
> +The new origin references both its creation stack and the previous origin the
> +memory location had.
> +This may cause increased memory consumption, so we limit the length of origin
> +chains in the runtime.
> +
> +Clang instrumentation API
> +-------------------------
> +
> +Clang instrumentation pass inserts calls to functions defined in
> +``mm/kmsan/kmsan_instr.c`` into the kernel code.
> +Shadow manipulation
> +~~~~~~~~~~~~~~~~~~~
> +For every memory access the compiler emits a call to a function that returns a
> +pair of pointers to the shadow and origin addresses of the given memory::
> +
> + typedef struct {
> + void *s, *o;
> + } shadow_origin_ptr_t
> +
> + shadow_origin_ptr_t __msan_metadata_ptr_for_load_{1,2,4,8}(void *addr)
> + shadow_origin_ptr_t __msan_metadata_ptr_for_store_{1,2,4,8}(void *addr)
> + shadow_origin_ptr_t __msan_metadata_ptr_for_load_n(void *addr, u64 size)
> + shadow_origin_ptr_t __msan_metadata_ptr_for_store_n(void *addr, u64 size)
> +
> +The function name depends on the memory access size.
> +Each such function also checks if the shadow of the memory in the range
> +[``addr``, ``addr + n``) is contiguous and reports an error otherwise.
> +
> +The compiler makes sure that for every loaded value its shadow and origin
> +values are read from memory.
> +When a value is stored to memory, its shadow and origin are also stored using
> +the metadata pointers.
> +
> +Origin tracking
> +~~~~~~~~~~~~~~~
> +A special function is used to create a new origin value for a local variable
> +and set the origin of that variable to that value::
> +
> + void __msan_poison_alloca(u64 address, u64 size, char *descr)
> +
> +Access to per-task data
> +~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +At the beginning of every instrumented function KMSAN inserts a call to
> +``__msan_get_context_state()``::
> +
> + kmsan_context_state *__msan_get_context_state(void)
> +
> +``kmsan_context_state`` is declared in ``include/linux/kmsan.h``::
> +
> + struct kmsan_context_s {
> + char param_tls[KMSAN_PARAM_SIZE];
> + char retval_tls[RETVAL_SIZE];
> + char va_arg_tls[KMSAN_PARAM_SIZE];
> + char va_arg_origin_tls[KMSAN_PARAM_SIZE];
> + u64 va_arg_overflow_size_tls;
> + depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE];
> + depot_stack_handle_t retval_origin_tls;
> + depot_stack_handle_t origin_tls;
> + };
> +
> +This structure is used by KMSAN to pass parameter shadows and origins between
> +instrumented functions.
> +
> +String functions
> +~~~~~~~~~~~~~~~~
> +
> +The compiler replaces calls to ``memcpy()``/``memmove()``/``memset()`` with the
> +following functions. These functions are also called when data structures are
> +initialized or copied, making sure shadow and origin values are copied alongside
> +with the data::
> +
> + void *__msan_memcpy(void *dst, void *src, u64 n)
> + void *__msan_memmove(void *dst, void *src, u64 n)
> + void *__msan_memset(void *dst, int c, size_t n)
> +
> +Error reporting
> +~~~~~~~~~~~~~~~
> +
> +For each pointer dereference and each condition the compiler emits a shadow
> +check that calls ``__msan_warning()`` in the case a poisoned value is being
> +used::
> +
> + void __msan_warning(u32 origin)
> +
> +``__msan_warning()`` causes KMSAN runtime to print an error report.
> +
> +Inline assembly instrumentation
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +KMSAN instruments every inline assembly output with a call to::
> +
> + void __msan_instrument_asm_store(u64 addr, u64 size)
> +
> +, which unpoisons the memory region.
> +
> +This approach may mask certain errors, but it also helps to avoid a lot of
> +false positives in bitwise operations, atomics etc.
> +
> +Sometimes the pointers passed into inline assembly don't point to valid memory.
> +In such cases they are ignored at runtime.
> +
> +Disabling the instrumentation
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +A function can be marked with ``__no_sanitize_memory``.
> +Doing so doesn't remove KMSAN instrumentation from it, however it makes the
> +compiler ignore the uninitialized values coming from the function's inputs,
> +and initialize the function's outputs.
> +The compiler won't inline functions marked with this attribute into functions
> +not marked with it, and vice versa.
> +
> +It's also possible to disable KMSAN for a single file (e.g. main.o)::
> +
> + KMSAN_SANITIZE_main.o := n
> +
> +or for the whole directory::
> +
> + KMSAN_SANITIZE := n
> +
> +in the Makefile. This comes at a cost however: stack allocations from such files
> +and parameters of instrumented functions called from them will have incorrect
> +shadow/origin values. As a rule of thumb, avoid using KMSAN_SANITIZE.
> +
> +Runtime library
> +---------------
> +The code is located in ``mm/kmsan/``.
> +
> +Per-task KMSAN state
> +~~~~~~~~~~~~~~~~~~~~
> +
> +Every task_struct has an associated KMSAN task state that holds the KMSAN
> +context (see above) and a per-task flag disallowing KMSAN reports::
> +
> + struct kmsan_task_state {
> + ...
> + bool allow_reporting;
> + struct kmsan_context_state cstate;
> + ...
> + }
> +
> + struct task_struct {
> + ...
> + struct kmsan_task_state kmsan;
> + ...
> + }
> +
> +
> +KMSAN contexts
> +~~~~~~~~~~~~~~
> +
> +When running in a kernel task context, KMSAN uses ``current->kmsan.cstate`` to
> +hold the metadata for function parameters and return values.
> +
> +But in the case the kernel is running in the interrupt, softirq or NMI context,
> +where ``current`` is unavailable, KMSAN switches to per-cpu interrupt state::
> +
> + DEFINE_PER_CPU(kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
> + kmsan_percpu_cstate);
> +
> +Metadata allocation
> +~~~~~~~~~~~~~~~~~~~
> +There are several places in the kernel for which the metadata is stored.
> +
> +1. Each ``struct page`` instance contains two pointers to its shadow and
> +origin pages::
> +
> + struct page {
> + ...
> + struct page *shadow, *origin;
> + ...
> + };
> +
> +Every time a ``struct page`` is allocated, the runtime library allocates two
> +additional pages to hold its shadow and origins. This is done by adding hooks
> +to ``alloc_pages()``/``free_pages()`` in ``mm/page_alloc.c``.
> +To avoid allocating the metadata for non-interesting pages (right now only the
> +shadow/origin page themselves and stackdepot storage) the
> +``__GFP_NO_KMSAN_SHADOW`` flag is used.
> +
> +There is a problem related to this allocation algorithm: when two contiguous
> +memory blocks are allocated with two different ``alloc_pages()`` calls, their
> +shadow pages may not be contiguous. So, if a memory access crosses the boundary
> +of a memory block, accesses to shadow/origin memory may potentially corrupt
> +other pages or read incorrect values from them.
> +
> +As a workaround, we check the access size in
> +``__msan_metadata_ptr_for_XXX_YYY()`` and return a pointer to a fake shadow
> +region in the case of an error::
> +
> + char dummy_load_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
> + char dummy_store_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
> +
> +``dummy_load_page`` is zero-initialized, so reads from it always yield zeroes.
> +All stores to ``dummy_store_page`` are ignored.
> +
> +Unfortunately at boot time we need to allocate shadow and origin pages for the
> +kernel data (``.data``, ``.bss`` etc.) and percpu memory regions, the size of
> +which is not a power of 2. As a result, we have to allocate the metadata page by
> +page, so that it is also non-contiguous, although it may be perfectly valid to
> +access the corresponding kernel memory across page boundaries.
> +This can be probably fixed by allocating 1<<N pages at once, splitting them and
> +deallocating the rest.
> +
> +LSB of the ``shadow`` pointer in a ``struct page`` may be set to 1. In this case
> +shadow and origin pages are allocated, but KMSAN ignores accesses to them by
> +falling back to dummy pages. Allocating the metadata pages is still needed to
> +support ``vmap()/vunmap()`` operations on this struct page.
> +
> +2. For vmalloc memory and modules, there's a direct mapping between the memory
> +range, its shadow and origin. KMSAN lessens the vmalloc area by 3/4, making only
> +the first quarter available to ``vmalloc()``. The second quarter of the vmalloc
> +area contains shadow memory for the first quarter, the third one holds the
> +origins. A small part of the fourth quarter contains shadow and origins for the
> +kernel modules. Please refer to ``arch/x86/include/asm/pgtable_64_types.h`` for
> +more details.
> +
> +When an array of pages is mapped into a contiguous virtual memory space, their
> +shadow and origin pages are similarly mapped into contiguous regions.
> +
> +3. For CPU entry area there're separate per-CPU arrays that hold its metadata::
> +
> + DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
> + DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
For some reason rst2html complains here that this is not a literal block.
> +When calculating shadow and origin addresses for a given memory address, the
> +runtime checks whether the address belongs to the physical page range, the
> +virtual page range or CPU entry area.
> +
> +Handling ``pt_regs``
> +~~~~~~~~~~~~~~~~~~~
This is missing a '~' (I ran it through rst2html to find).
> +Many functions receive a ``struct pt_regs`` holding the register state at a
> +certain point. Registers don't have (easily calculatable) shadow or origin
> +associated with them.
> +We can assume that the registers are always initialized.
> +
> +Example report
> +--------------
> +Here's an example of a real KMSAN report in ``packet_bind_spkt()``::
Shouldn't this section be somewhere at the top in a section such as
"usage". A user of KMSAN doesn't really care how KMSAN works.
> + ==================================================================
> + BUG: KMSAN: uninit-value in strlen
> + CPU: 0 PID: 1074 Comm: packet Not tainted 4.8.0-rc6+ #1891
> + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> + 0000000000000000 ffff88006b6dfc08 ffffffff82559ae8 ffff88006b6dfb48
> + ffffffff818a7c91 ffffffff85b9c870 0000000000000092 ffffffff85b9c550
> + 0000000000000000 0000000000000092 00000000ec400911 0000000000000002
> + Call Trace:
> + [< inline >] __dump_stack lib/dump_stack.c:15
> + [<ffffffff82559ae8>] dump_stack+0x238/0x290 lib/dump_stack.c:51
> + [<ffffffff818a6626>] kmsan_report+0x276/0x2e0 mm/kmsan/kmsan.c:1003
> + [<ffffffff818a783b>] __msan_warning+0x5b/0xb0 mm/kmsan/kmsan_instr.c:424
> + [< inline >] strlen lib/string.c:484
> + [<ffffffff8259b58d>] strlcpy+0x9d/0x200 lib/string.c:144
> + [<ffffffff84b2eca4>] packet_bind_spkt+0x144/0x230 net/packet/af_packet.c:3132
> + [<ffffffff84242e4d>] SYSC_bind+0x40d/0x5f0 net/socket.c:1370
> + [<ffffffff84242a22>] SyS_bind+0x82/0xa0 net/socket.c:1356
> + [<ffffffff8515991b>] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:?
> + chained origin:
> + [<ffffffff810bb787>] save_stack_trace+0x27/0x50 arch/x86/kernel/stacktrace.c:67
> + [< inline >] kmsan_save_stack_with_flags mm/kmsan/kmsan.c:322
> + [< inline >] kmsan_save_stack mm/kmsan/kmsan.c:334
> + [<ffffffff818a59f8>] kmsan_internal_chain_origin+0x118/0x1e0 mm/kmsan/kmsan.c:527
> + [<ffffffff818a7773>] __msan_set_alloca_origin4+0xc3/0x130 mm/kmsan/kmsan_instr.c:380
> + [<ffffffff84242b69>] SYSC_bind+0x129/0x5f0 net/socket.c:1356
> + [<ffffffff84242a22>] SyS_bind+0x82/0xa0 net/socket.c:1356
> + [<ffffffff8515991b>] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:?
> + origin description: ----address@SYSC_bind (origin=00000000eb400911)
> + ==================================================================
> +
> +The report tells that the local variable ``address`` was created uninitialized
> +in ``SYSC_bind()`` (the ``bind`` system call implementation). The lower stack
> +trace corresponds to the place where this variable was created.
> +
> +The upper stack shows where the uninit value was used - in ``strlen()``.
> +It turned out that the contents of ``address`` were partially copied from the
> +userspace, but the buffer wasn't zero-terminated and contained some trailing
> +uninitialized bytes.
> +``packet_bind_spkt()`` didn't check the length of the buffer, but called
> +``strlcpy()`` on it, which called ``strlen()``, which started reading the
> +buffer byte by byte till it hit the uninitialized memory.
> +
> +
> +References
> +==========
> +
> +E. Stepanov, K. Serebryany. MemorySanitizer: fast detector of uninitialized
> +memory use in C++.
> +In Proceedings of CGO 2015.
This should be turned into a link.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 05/36] kmsan: add ReST documentation
2019-11-27 14:22 ` Marco Elver
@ 2019-12-03 12:42 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 12:42 UTC (permalink / raw)
To: Marco Elver
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K . Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Wed, Nov 27, 2019 at 3:23 PM Marco Elver <elver@google.com> wrote:
>
> General comments:
> * it's -> it is
> * don't -> do not
>
> On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
> [...]
> > diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
> > index b0522a4dd107..bc5e3fd87efa 100644
> > --- a/Documentation/dev-tools/index.rst
> > +++ b/Documentation/dev-tools/index.rst
> > @@ -19,6 +19,7 @@ whole; patches welcome!
> > kcov
> > gcov
> > kasan
> > + kmsan
> > ubsan
> > kmemleak
> > gdb-kernel-debugging
> > diff --git a/Documentation/dev-tools/kmsan.rst b/Documentation/dev-tools/kmsan.rst
> > new file mode 100644
> > index 000000000000..51f9c207cc2c
> > --- /dev/null
> > +++ b/Documentation/dev-tools/kmsan.rst
> > @@ -0,0 +1,418 @@
> > +=============================
> > +KernelMemorySanitizer (KMSAN)
> > +=============================
> > +
> > +KMSAN is a dynamic memory error detector aimed at finding uses of uninitialized
> > +memory.
> > +It is based on compiler instrumentation, and is quite similar to the userspace
> > +MemorySanitizer tool (http://clang.llvm.org/docs/MemorySanitizer.html).
>
> These should be real links: `Memory sanitizer tool <...url...>`_.
Changed these links to link targets.
> > +KMSAN and Clang
> > +===============
> > +
> > +In order for KMSAN to work the kernel must be
> > +built with Clang, which is so far the only compiler that has KMSAN support.
>
> "is so far" -> "so far is"
Ack.
> > +The kernel instrumentation pass is based on the userspace MemorySanitizer tool
> > +(http://clang.llvm.org/docs/MemorySanitizer.html). Because of the
>
> Should also be real link: `MemorySanitizer tool <..url..>`_
Ack.
> > +instrumentation complexity it's unlikely that any other compiler will support
> > +KMSAN soon.
> > +
> > +Right now the instrumentation pass supports x86_64 only.
> > +
> > +How to build
> > +============
> > +
> > +In order to build a kernel with KMSAN you'll need a fresh Clang (10.0.0+, trunk
> > +version r365008 or greater). Please refer to
> > +https://llvm.org/docs/GettingStarted.html for the instructions on how to build
> > +Clang::
> > +
> > + export KMSAN_CLANG_PATH=/path/to/clang
> >
> > + # Now configure and build the kernel with CONFIG_KMSAN enabled.
> > + make CC=$KMSAN_CLANG_PATH -j64
>
> I don't think '-j64' is necessary to build. Also the 'export' is
> technically not required AFAIK, but I don't think it bothers anyone.
Ack.
> > +How KMSAN works
> > +===============
> > +
> > +KMSAN shadow memory
> > +-------------------
> > +
> > +KMSAN associates a so-called shadow byte with every byte of kernel memory.
>
> 'shadow' memory may not be a well-defined term. More intuitive would
> be saying that it's metadata associated with every byte of kernel
> memory. From then on you can say it's shadow memory.
Changed this to be:
"KMSAN associates a metadata byte (also called shadow byte) with every byte of
kernel memory.
A bit in the shadow byte is set..."
> > +A bit in the shadow byte is set iff the corresponding bit of the kernel memory
> > +byte is uninitialized.
> > +Marking the memory uninitialized (i.e. setting its shadow bytes to 0xff) is
> > +called poisoning, marking it initialized (setting the shadow bytes to 0x00) is
> > +called unpoisoning.
> > +
> > +When a new variable is allocated on the stack, it's poisoned by default by
> > +instrumentation code inserted by the compiler (unless it's a stack variable that
> > +is immediately initialized). Any new heap allocation done without ``__GFP_ZERO``
> > +is also poisoned.
> > +
> > +Compiler instrumentation also tracks the shadow values with the help from the
> > +runtime library in ``mm/kmsan/``.
> > +
> > +The shadow value of a basic or compound type is an array of bytes of the same
> > +length.
> > +When a constant value is written into memory, that memory is unpoisoned.
> > +When a value is read from memory, its shadow memory is also obtained and
> > +propagated into all the operations which use that value. For every instruction
> > +that takes one or more values the compiler generates code that calculates the
> > +shadow of the result depending on those values and their shadows.
> > +
> > +Example::
> > +
> > + int a = 0xff;
> > + int b;
> > + int c = a | b;
> > +
> > +In this case the shadow of ``a`` is ``0``, shadow of ``b`` is ``0xffffffff``,
> > +shadow of ``c`` is ``0xffffff00``. This means that the upper three bytes of
> > +``c`` are uninitialized, while the lower byte is initialized.
> > +
> > +
> > +Origin tracking
> > +---------------
> > +
> > +Every four bytes of kernel memory also have a so-called origin assigned to
> > +them.
> > +This origin describes the point in program execution at which the uninitialized
> > +value was created. Every origin is associated with a creation stack, which lets
> > +the user figure out what's going on.
> > +
> > +When an uninitialized variable is allocated on stack or heap, a new origin
> > +value is created, and that variable's origin is filled with that value.
> > +When a value is read from memory, its origin is also read and kept together
> > +with the shadow. For every instruction that takes one or more values the origin
> > +of the result is one of the origins corresponding to any of the uninitialized
> > +inputs.
> > +If a poisoned value is written into memory, its origin is written to the
> > +corresponding storage as well.
> > +
> > +Example 1::
> > +
> > + int a = 0;
> > + int b;
> > + int c = a + b;
> > +
> > +In this case the origin of ``b`` is generated upon function entry, and is
> > +stored to the origin of ``c`` right before the addition result is written into
> > +memory.
> > +
> > +Several variables may share the same origin address, if they are stored in the
> > +same four-byte chunk.
> > +In this case every write to either variable updates the origin for all of them.
> > +
> > +Example 2::
> > +
> > + int combine(short a, short b) {
> > + union ret_t {
> > + int i;
> > + short s[2];
> > + } ret;
> > + ret.s[0] = a;
> > + ret.s[1] = b;
> > + return ret.i;
> > + }
> > +
> > +If ``a`` is initialized and ``b`` is not, the shadow of the result would be
> > +0xffff0000, and the origin of the result would be the origin of ``b``.
> > +``ret.s[0]`` would have the same origin, but it will be never used, because
> > +that variable is initialized.
> > +
> > +If both function arguments are uninitialized, only the origin of the second
> > +argument is preserved.
> > +
> > +Origin chaining
> > +~~~~~~~~~~~~~~~
> > +To ease the debugging, KMSAN creates a new origin for every memory store.
>
> "the debugging" -> "debugging"
Ack
> > +The new origin references both its creation stack and the previous origin the
> > +memory location had.
> > +This may cause increased memory consumption, so we limit the length of origin
> > +chains in the runtime.
> > +
> > +Clang instrumentation API
> > +-------------------------
> > +
> > +Clang instrumentation pass inserts calls to functions defined in
> > +``mm/kmsan/kmsan_instr.c`` into the kernel code.
>
> > +Shadow manipulation
> > +~~~~~~~~~~~~~~~~~~~
> > +For every memory access the compiler emits a call to a function that returns a
> > +pair of pointers to the shadow and origin addresses of the given memory::
> > +
> > + typedef struct {
> > + void *s, *o;
> > + } shadow_origin_ptr_t
> > +
> > + shadow_origin_ptr_t __msan_metadata_ptr_for_load_{1,2,4,8}(void *addr)
> > + shadow_origin_ptr_t __msan_metadata_ptr_for_store_{1,2,4,8}(void *addr)
> > + shadow_origin_ptr_t __msan_metadata_ptr_for_load_n(void *addr, u64 size)
> > + shadow_origin_ptr_t __msan_metadata_ptr_for_store_n(void *addr, u64 size)
> > +
> > +The function name depends on the memory access size.
> > +Each such function also checks if the shadow of the memory in the range
> > +[``addr``, ``addr + n``) is contiguous and reports an error otherwise.
> > +
> > +The compiler makes sure that for every loaded value its shadow and origin
> > +values are read from memory.
> > +When a value is stored to memory, its shadow and origin are also stored using
> > +the metadata pointers.
> > +
> > +Origin tracking
> > +~~~~~~~~~~~~~~~
> > +A special function is used to create a new origin value for a local variable
> > +and set the origin of that variable to that value::
> > +
> > + void __msan_poison_alloca(u64 address, u64 size, char *descr)
> > +
> > +Access to per-task data
> > +~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +At the beginning of every instrumented function KMSAN inserts a call to
> > +``__msan_get_context_state()``::
> > +
> > + kmsan_context_state *__msan_get_context_state(void)
> > +
> > +``kmsan_context_state`` is declared in ``include/linux/kmsan.h``::
> > +
> > + struct kmsan_context_s {
> > + char param_tls[KMSAN_PARAM_SIZE];
> > + char retval_tls[RETVAL_SIZE];
> > + char va_arg_tls[KMSAN_PARAM_SIZE];
> > + char va_arg_origin_tls[KMSAN_PARAM_SIZE];
> > + u64 va_arg_overflow_size_tls;
> > + depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE];
> > + depot_stack_handle_t retval_origin_tls;
> > + depot_stack_handle_t origin_tls;
> > + };
> > +
> > +This structure is used by KMSAN to pass parameter shadows and origins between
> > +instrumented functions.
> > +
> > +String functions
> > +~~~~~~~~~~~~~~~~
> > +
> > +The compiler replaces calls to ``memcpy()``/``memmove()``/``memset()`` with the
> > +following functions. These functions are also called when data structures are
> > +initialized or copied, making sure shadow and origin values are copied alongside
> > +with the data::
> > +
> > + void *__msan_memcpy(void *dst, void *src, u64 n)
> > + void *__msan_memmove(void *dst, void *src, u64 n)
> > + void *__msan_memset(void *dst, int c, size_t n)
> > +
> > +Error reporting
> > +~~~~~~~~~~~~~~~
> > +
> > +For each pointer dereference and each condition the compiler emits a shadow
> > +check that calls ``__msan_warning()`` in the case a poisoned value is being
> > +used::
> > +
> > + void __msan_warning(u32 origin)
> > +
> > +``__msan_warning()`` causes KMSAN runtime to print an error report.
> > +
> > +Inline assembly instrumentation
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +KMSAN instruments every inline assembly output with a call to::
> > +
> > + void __msan_instrument_asm_store(u64 addr, u64 size)
> > +
> > +, which unpoisons the memory region.
> > +
> > +This approach may mask certain errors, but it also helps to avoid a lot of
> > +false positives in bitwise operations, atomics etc.
> > +
> > +Sometimes the pointers passed into inline assembly don't point to valid memory.
> > +In such cases they are ignored at runtime.
> > +
> > +Disabling the instrumentation
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +A function can be marked with ``__no_sanitize_memory``.
> > +Doing so doesn't remove KMSAN instrumentation from it, however it makes the
> > +compiler ignore the uninitialized values coming from the function's inputs,
> > +and initialize the function's outputs.
> > +The compiler won't inline functions marked with this attribute into functions
> > +not marked with it, and vice versa.
> > +
> > +It's also possible to disable KMSAN for a single file (e.g. main.o)::
> > +
> > + KMSAN_SANITIZE_main.o := n
> > +
> > +or for the whole directory::
> > +
> > + KMSAN_SANITIZE := n
> > +
> > +in the Makefile. This comes at a cost however: stack allocations from such files
> > +and parameters of instrumented functions called from them will have incorrect
> > +shadow/origin values. As a rule of thumb, avoid using KMSAN_SANITIZE.
> > +
> > +Runtime library
> > +---------------
> > +The code is located in ``mm/kmsan/``.
> > +
> > +Per-task KMSAN state
> > +~~~~~~~~~~~~~~~~~~~~
> > +
> > +Every task_struct has an associated KMSAN task state that holds the KMSAN
> > +context (see above) and a per-task flag disallowing KMSAN reports::
> > +
> > + struct kmsan_task_state {
> > + ...
> > + bool allow_reporting;
> > + struct kmsan_context_state cstate;
> > + ...
> > + }
> > +
> > + struct task_struct {
> > + ...
> > + struct kmsan_task_state kmsan;
> > + ...
> > + }
> > +
> > +
> > +KMSAN contexts
> > +~~~~~~~~~~~~~~
> > +
> > +When running in a kernel task context, KMSAN uses ``current->kmsan.cstate`` to
> > +hold the metadata for function parameters and return values.
> > +
> > +But in the case the kernel is running in the interrupt, softirq or NMI context,
> > +where ``current`` is unavailable, KMSAN switches to per-cpu interrupt state::
> > +
> > + DEFINE_PER_CPU(kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
> > + kmsan_percpu_cstate);
> > +
> > +Metadata allocation
> > +~~~~~~~~~~~~~~~~~~~
> > +There are several places in the kernel for which the metadata is stored.
> > +
> > +1. Each ``struct page`` instance contains two pointers to its shadow and
> > +origin pages::
> > +
> > + struct page {
> > + ...
> > + struct page *shadow, *origin;
> > + ...
> > + };
> > +
> > +Every time a ``struct page`` is allocated, the runtime library allocates two
> > +additional pages to hold its shadow and origins. This is done by adding hooks
> > +to ``alloc_pages()``/``free_pages()`` in ``mm/page_alloc.c``.
> > +To avoid allocating the metadata for non-interesting pages (right now only the
> > +shadow/origin page themselves and stackdepot storage) the
> > +``__GFP_NO_KMSAN_SHADOW`` flag is used.
> > +
> > +There is a problem related to this allocation algorithm: when two contiguous
> > +memory blocks are allocated with two different ``alloc_pages()`` calls, their
> > +shadow pages may not be contiguous. So, if a memory access crosses the boundary
> > +of a memory block, accesses to shadow/origin memory may potentially corrupt
> > +other pages or read incorrect values from them.
> > +
> > +As a workaround, we check the access size in
> > +``__msan_metadata_ptr_for_XXX_YYY()`` and return a pointer to a fake shadow
> > +region in the case of an error::
> > +
> > + char dummy_load_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
> > + char dummy_store_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
> > +
> > +``dummy_load_page`` is zero-initialized, so reads from it always yield zeroes.
> > +All stores to ``dummy_store_page`` are ignored.
> > +
> > +Unfortunately at boot time we need to allocate shadow and origin pages for the
> > +kernel data (``.data``, ``.bss`` etc.) and percpu memory regions, the size of
> > +which is not a power of 2. As a result, we have to allocate the metadata page by
> > +page, so that it is also non-contiguous, although it may be perfectly valid to
> > +access the corresponding kernel memory across page boundaries.
> > +This can be probably fixed by allocating 1<<N pages at once, splitting them and
> > +deallocating the rest.
> > +
> > +LSB of the ``shadow`` pointer in a ``struct page`` may be set to 1. In this case
> > +shadow and origin pages are allocated, but KMSAN ignores accesses to them by
> > +falling back to dummy pages. Allocating the metadata pages is still needed to
> > +support ``vmap()/vunmap()`` operations on this struct page.
> > +
> > +2. For vmalloc memory and modules, there's a direct mapping between the memory
> > +range, its shadow and origin. KMSAN lessens the vmalloc area by 3/4, making only
> > +the first quarter available to ``vmalloc()``. The second quarter of the vmalloc
> > +area contains shadow memory for the first quarter, the third one holds the
> > +origins. A small part of the fourth quarter contains shadow and origins for the
> > +kernel modules. Please refer to ``arch/x86/include/asm/pgtable_64_types.h`` for
> > +more details.
> > +
> > +When an array of pages is mapped into a contiguous virtual memory space, their
> > +shadow and origin pages are similarly mapped into contiguous regions.
> > +
> > +3. For CPU entry area there're separate per-CPU arrays that hold its metadata::
> > +
> > + DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
> > + DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
>
> For some reason rst2html complains here that this is not a literal block.
Maybe that's because the preceding paragraph only contained a single
line. Adding a line break fixed the problem.
> > +When calculating shadow and origin addresses for a given memory address, the
> > +runtime checks whether the address belongs to the physical page range, the
> > +virtual page range or CPU entry area.
> > +
> > +Handling ``pt_regs``
> > +~~~~~~~~~~~~~~~~~~~
>
> This is missing a '~' (I ran it through rst2html to find).
Ack.
> > +Many functions receive a ``struct pt_regs`` holding the register state at a
> > +certain point. Registers don't have (easily calculatable) shadow or origin
> > +associated with them.
> > +We can assume that the registers are always initialized.
> > +
> > +Example report
> > +--------------
> > +Here's an example of a real KMSAN report in ``packet_bind_spkt()``::
>
> Shouldn't this section be somewhere at the top in a section such as
> "usage". A user of KMSAN doesn't really care how KMSAN works.
Good idea, thanks!
Moved this section to the very beginning.
> > + ==================================================================
> > + BUG: KMSAN: uninit-value in strlen
> > + CPU: 0 PID: 1074 Comm: packet Not tainted 4.8.0-rc6+ #1891
> > + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > + 0000000000000000 ffff88006b6dfc08 ffffffff82559ae8 ffff88006b6dfb48
> > + ffffffff818a7c91 ffffffff85b9c870 0000000000000092 ffffffff85b9c550
> > + 0000000000000000 0000000000000092 00000000ec400911 0000000000000002
> > + Call Trace:
> > + [< inline >] __dump_stack lib/dump_stack.c:15
> > + [<ffffffff82559ae8>] dump_stack+0x238/0x290 lib/dump_stack.c:51
> > + [<ffffffff818a6626>] kmsan_report+0x276/0x2e0 mm/kmsan/kmsan.c:1003
> > + [<ffffffff818a783b>] __msan_warning+0x5b/0xb0 mm/kmsan/kmsan_instr.c:424
> > + [< inline >] strlen lib/string.c:484
> > + [<ffffffff8259b58d>] strlcpy+0x9d/0x200 lib/string.c:144
> > + [<ffffffff84b2eca4>] packet_bind_spkt+0x144/0x230 net/packet/af_packet.c:3132
> > + [<ffffffff84242e4d>] SYSC_bind+0x40d/0x5f0 net/socket.c:1370
> > + [<ffffffff84242a22>] SyS_bind+0x82/0xa0 net/socket.c:1356
> > + [<ffffffff8515991b>] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:?
> > + chained origin:
> > + [<ffffffff810bb787>] save_stack_trace+0x27/0x50 arch/x86/kernel/stacktrace.c:67
> > + [< inline >] kmsan_save_stack_with_flags mm/kmsan/kmsan.c:322
> > + [< inline >] kmsan_save_stack mm/kmsan/kmsan.c:334
> > + [<ffffffff818a59f8>] kmsan_internal_chain_origin+0x118/0x1e0 mm/kmsan/kmsan.c:527
> > + [<ffffffff818a7773>] __msan_set_alloca_origin4+0xc3/0x130 mm/kmsan/kmsan_instr.c:380
> > + [<ffffffff84242b69>] SYSC_bind+0x129/0x5f0 net/socket.c:1356
> > + [<ffffffff84242a22>] SyS_bind+0x82/0xa0 net/socket.c:1356
> > + [<ffffffff8515991b>] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:?
> > + origin description: ----address@SYSC_bind (origin=00000000eb400911)
> > + ==================================================================
> > +
> > +The report tells that the local variable ``address`` was created uninitialized
> > +in ``SYSC_bind()`` (the ``bind`` system call implementation). The lower stack
> > +trace corresponds to the place where this variable was created.
> > +
> > +The upper stack shows where the uninit value was used - in ``strlen()``.
> > +It turned out that the contents of ``address`` were partially copied from the
> > +userspace, but the buffer wasn't zero-terminated and contained some trailing
> > +uninitialized bytes.
> > +``packet_bind_spkt()`` didn't check the length of the buffer, but called
> > +``strlcpy()`` on it, which called ``strlen()``, which started reading the
> > +buffer byte by byte till it hit the uninitialized memory.
> > +
> > +
> > +References
> > +==========
> > +
> > +E. Stepanov, K. Serebryany. MemorySanitizer: fast detector of uninitialized
> > +memory use in C++.
> > +In Proceedings of CGO 2015.
>
> This should be turned into a link.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 06/36] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (4 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 05/36] kmsan: add ReST documentation glider
@ 2019-11-22 11:25 ` glider
2019-11-27 14:48 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__ glider
` (29 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Andrew Morton, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
This flag is to be used by KMSAN runtime to mark that newly created
memory pages don't need KMSAN metadata backing them.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
We can't decide what to do here:
- do we need to conditionally define ___GFP_NO_KMSAN_SHADOW depending on
CONFIG_KMSAN like LOCKDEP does?
- if KMSAN is defined, and LOCKDEP is not, do we want to "compactify" the GFP
bits?
Change-Id: If5d0352fd5711ad103328e2c185eb885e826423a
---
include/linux/gfp.h | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index fb07b503dc45..b4e7963cd94b 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -44,6 +44,7 @@ struct vm_area_struct;
#else
#define ___GFP_NOLOCKDEP 0
#endif
+#define ___GFP_NO_KMSAN_SHADOW 0x1000000u
/* If the above are modified, __GFP_BITS_SHIFT may need updating */
/*
@@ -212,12 +213,13 @@ struct vm_area_struct;
#define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN)
#define __GFP_COMP ((__force gfp_t)___GFP_COMP)
#define __GFP_ZERO ((__force gfp_t)___GFP_ZERO)
+#define __GFP_NO_KMSAN_SHADOW ((__force gfp_t)___GFP_NO_KMSAN_SHADOW)
/* Disable lockdep for GFP context tracking */
#define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP)
/* Room for N __GFP_FOO bits */
-#define __GFP_BITS_SHIFT (23 + IS_ENABLED(CONFIG_LOCKDEP))
+#define __GFP_BITS_SHIFT (25)
#define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1))
/**
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 06/36] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW
2019-11-22 11:25 ` [PATCH RFC v3 06/36] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW glider
@ 2019-11-27 14:48 ` Marco Elver
2019-12-03 12:57 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-11-27 14:48 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Andrew Morton, Dmitry Vyukov,
Linux Memory Management List, Al Viro, adilger.kernel,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, Eric Dumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, Mark Rutland, martin.petersen, schwidefsky,
Matthew Wilcox, mst, monstr, pmladek, Qian Cai, Randy Dunlap,
robin.murphy, Sergey Senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> This flag is to be used by KMSAN runtime to mark that newly created
> memory pages don't need KMSAN metadata backing them.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
> We can't decide what to do here:
> - do we need to conditionally define ___GFP_NO_KMSAN_SHADOW depending on
> CONFIG_KMSAN like LOCKDEP does?
> - if KMSAN is defined, and LOCKDEP is not, do we want to "compactify" the GFP
> bits?
A maintainer would know the real answer, but this is my guess: making
the behaviour not change without KMSAN would probably be better. It
would require some ifdef trickery to deal with LOCKDEP on/off case
though.
> Change-Id: If5d0352fd5711ad103328e2c185eb885e826423a
> ---
> include/linux/gfp.h | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/gfp.h b/include/linux/gfp.h
> index fb07b503dc45..b4e7963cd94b 100644
> --- a/include/linux/gfp.h
> +++ b/include/linux/gfp.h
> @@ -44,6 +44,7 @@ struct vm_area_struct;
> #else
> #define ___GFP_NOLOCKDEP 0
> #endif
Since this change unconditionally changes GFP_BITS_SHIFT to 25, the
#ifdef for GFP_NOLOCKDEP could also go away -- but: see above.
> +#define ___GFP_NO_KMSAN_SHADOW 0x1000000u
> /* If the above are modified, __GFP_BITS_SHIFT may need updating */
>
> /*
> @@ -212,12 +213,13 @@ struct vm_area_struct;
> #define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN)
> #define __GFP_COMP ((__force gfp_t)___GFP_COMP)
> #define __GFP_ZERO ((__force gfp_t)___GFP_ZERO)
> +#define __GFP_NO_KMSAN_SHADOW ((__force gfp_t)___GFP_NO_KMSAN_SHADOW)
Should this be ordered after __GFP_NOLOCKDEP with a brief comment what
it does? All of these up to __GFP_ZERO have a doc comment above.
> /* Disable lockdep for GFP context tracking */
> #define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP)
>
> /* Room for N __GFP_FOO bits */
> -#define __GFP_BITS_SHIFT (23 + IS_ENABLED(CONFIG_LOCKDEP))
> +#define __GFP_BITS_SHIFT (25)
> #define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1))
>
> /**
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 06/36] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW
2019-11-27 14:48 ` Marco Elver
@ 2019-12-03 12:57 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 12:57 UTC (permalink / raw)
To: Marco Elver, Michal Hocko
Cc: Vegard Nossum, Andrew Morton, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K . Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Wed, Nov 27, 2019 at 3:48 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
> >
> > This flag is to be used by KMSAN runtime to mark that newly created
> > memory pages don't need KMSAN metadata backing them.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> >
> > ---
> > We can't decide what to do here:
> > - do we need to conditionally define ___GFP_NO_KMSAN_SHADOW depending on
> > CONFIG_KMSAN like LOCKDEP does?
> > - if KMSAN is defined, and LOCKDEP is not, do we want to "compactify" the GFP
> > bits?
>
> A maintainer would know the real answer, but this is my guess: making
> the behaviour not change without KMSAN would probably be better. It
> would require some ifdef trickery to deal with LOCKDEP on/off case
> though.
I actually find it quite unfortunate that LOCKDEP has this on/off case.
It's already inconvenient to add a new GFP flag past ___GFP_NOLOCKDEP,
as its value will be depending on this blinking LOCKDEP bit.
Now imagine someone wants to add a second GFP flag that can be
disabled similarly to LOCKDEP - there'll be even more hassle counting
which bits are present.
Michal, do you think it's possible to make __GFP_BITS_SHIFT
independent from CONFIG_LOCKDEP?
> > Change-Id: If5d0352fd5711ad103328e2c185eb885e826423a
> > ---
> > include/linux/gfp.h | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/linux/gfp.h b/include/linux/gfp.h
> > index fb07b503dc45..b4e7963cd94b 100644
> > --- a/include/linux/gfp.h
> > +++ b/include/linux/gfp.h
> > @@ -44,6 +44,7 @@ struct vm_area_struct;
> > #else
> > #define ___GFP_NOLOCKDEP 0
> > #endif
>
> Since this change unconditionally changes GFP_BITS_SHIFT to 25, the
> #ifdef for GFP_NOLOCKDEP could also go away -- but: see above.
>
> > +#define ___GFP_NO_KMSAN_SHADOW 0x1000000u
> > /* If the above are modified, __GFP_BITS_SHIFT may need updating */
> >
> > /*
> > @@ -212,12 +213,13 @@ struct vm_area_struct;
> > #define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN)
> > #define __GFP_COMP ((__force gfp_t)___GFP_COMP)
> > #define __GFP_ZERO ((__force gfp_t)___GFP_ZERO)
> > +#define __GFP_NO_KMSAN_SHADOW ((__force gfp_t)___GFP_NO_KMSAN_SHADOW)
>
> Should this be ordered after __GFP_NOLOCKDEP with a brief comment what
> it does? All of these up to __GFP_ZERO have a doc comment above.
>
> > /* Disable lockdep for GFP context tracking */
> > #define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP)
> >
> > /* Room for N __GFP_FOO bits */
> > -#define __GFP_BITS_SHIFT (23 + IS_ENABLED(CONFIG_LOCKDEP))
> > +#define __GFP_BITS_SHIFT (25)
> > #define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1))
> >
> > /**
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (5 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 06/36] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW glider
@ 2019-11-22 11:25 ` glider
2019-11-28 13:13 ` Marco Elver
2019-11-29 16:09 ` Andrey Konovalov
2019-11-22 11:25 ` [PATCH RFC v3 08/36] kmsan: reduce vmalloc space glider
` (28 subsequent siblings)
35 siblings, 2 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
__no_sanitize_memory is a function attribute that makes KMSAN
ignore the uninitialized values coming from the function's
inputs, and initialize the function's outputs.
Functions marked with this attribute can't be inlined into functions
not marked with it, and vice versa.
__SANITIZE_MEMORY__ is a macro that's defined iff the file is
instrumented with KMSAN. This is not the same as CONFIG_KMSAN, which is
defined for every file.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I1f1672652c8392f15f7ca8ac26cd4e71f9cc1e4b
---
include/linux/compiler-clang.h | 8 ++++++++
include/linux/compiler-gcc.h | 5 +++++
2 files changed, 13 insertions(+)
diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h
index 333a6695a918..edba13a069a6 100644
--- a/include/linux/compiler-clang.h
+++ b/include/linux/compiler-clang.h
@@ -24,6 +24,14 @@
#define __no_sanitize_address
#endif
+/* KMSAN is a Clang-only tool, thus putting the defines here */
+#if __has_feature(memory_sanitizer)
+# define __SANITIZE_MEMORY__
+# define __no_sanitize_memory __attribute__((no_sanitize("kernel-memory")))
+#else
+# define __no_sanitize_memory
+#endif
+
/*
* Not all versions of clang implement the the type-generic versions
* of the builtin overflow checkers. Fortunately, clang implements
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index d7ee4c6bad48..e5ebc788dde4 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -145,6 +145,11 @@
#define __no_sanitize_address
#endif
+/*
+ * GCC doesn't support KMSAN.
+ */
+#define __no_sanitize_memory
+
#if GCC_VERSION >= 50100
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__
2019-11-22 11:25 ` [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__ glider
@ 2019-11-28 13:13 ` Marco Elver
2019-11-29 16:09 ` Andrey Konovalov
1 sibling, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-11-28 13:13 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, adilger.kernel, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
hch, hch, darrick.wong, davem, dmitry.torokhov, Eric Biggers,
Eric Dumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, Mark Rutland,
martin.petersen, schwidefsky, Matthew Wilcox, mst, Michal Simek,
pmladek, Qian Cai, Randy Dunlap, robin.murphy,
Sergey Senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> __no_sanitize_memory is a function attribute that makes KMSAN
> ignore the uninitialized values coming from the function's
> inputs, and initialize the function's outputs.
>
> Functions marked with this attribute can't be inlined into functions
> not marked with it, and vice versa.
>
> __SANITIZE_MEMORY__ is a macro that's defined iff the file is
> instrumented with KMSAN. This is not the same as CONFIG_KMSAN, which is
> defined for every file.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
Acked-by: Marco Elver <elver@google.com>
> ---
>
> Change-Id: I1f1672652c8392f15f7ca8ac26cd4e71f9cc1e4b
> ---
> include/linux/compiler-clang.h | 8 ++++++++
> include/linux/compiler-gcc.h | 5 +++++
> 2 files changed, 13 insertions(+)
>
> diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h
> index 333a6695a918..edba13a069a6 100644
> --- a/include/linux/compiler-clang.h
> +++ b/include/linux/compiler-clang.h
> @@ -24,6 +24,14 @@
> #define __no_sanitize_address
> #endif
>
> +/* KMSAN is a Clang-only tool, thus putting the defines here */
The comment is a bit confusing, because compiler-gcc.h also has a
define. I assume that if GCC ever supports KMSAN, the definition here
wouldn't change?
> +#if __has_feature(memory_sanitizer)
> +# define __SANITIZE_MEMORY__
> +# define __no_sanitize_memory __attribute__((no_sanitize("kernel-memory")))
> +#else
> +# define __no_sanitize_memory
> +#endif
> +
> /*
> * Not all versions of clang implement the the type-generic versions
> * of the builtin overflow checkers. Fortunately, clang implements
> diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
> index d7ee4c6bad48..e5ebc788dde4 100644
> --- a/include/linux/compiler-gcc.h
> +++ b/include/linux/compiler-gcc.h
> @@ -145,6 +145,11 @@
> #define __no_sanitize_address
> #endif
>
> +/*
> + * GCC doesn't support KMSAN.
> + */
> +#define __no_sanitize_memory
> +
> #if GCC_VERSION >= 50100
> #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
> #endif
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__
2019-11-22 11:25 ` [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__ glider
2019-11-28 13:13 ` Marco Elver
@ 2019-11-29 16:09 ` Andrey Konovalov
2019-12-16 11:35 ` Alexander Potapenko
1 sibling, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 16:09 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, pmladek,
Qian Cai, Randy Dunlap, Robin Murphy, sergey.senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
gor, wsa
On Fri, Nov 22, 2019 at 12:26 PM <glider@google.com> wrote:
>
> __no_sanitize_memory is a function attribute that makes KMSAN
> ignore the uninitialized values coming from the function's
> inputs, and initialize the function's outputs.
>
> Functions marked with this attribute can't be inlined into functions
> not marked with it, and vice versa.
>
> __SANITIZE_MEMORY__ is a macro that's defined iff the file is
> instrumented with KMSAN. This is not the same as CONFIG_KMSAN, which is
> defined for every file.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
>
> Change-Id: I1f1672652c8392f15f7ca8ac26cd4e71f9cc1e4b
> ---
> include/linux/compiler-clang.h | 8 ++++++++
> include/linux/compiler-gcc.h | 5 +++++
> 2 files changed, 13 insertions(+)
>
> diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h
> index 333a6695a918..edba13a069a6 100644
> --- a/include/linux/compiler-clang.h
> +++ b/include/linux/compiler-clang.h
> @@ -24,6 +24,14 @@
> #define __no_sanitize_address
> #endif
>
> +/* KMSAN is a Clang-only tool, thus putting the defines here */
I'm not sure if this comment is useful here.
> +#if __has_feature(memory_sanitizer)
> +# define __SANITIZE_MEMORY__
> +# define __no_sanitize_memory __attribute__((no_sanitize("kernel-memory")))
> +#else
> +# define __no_sanitize_memory
> +#endif
> +
> /*
> * Not all versions of clang implement the the type-generic versions
> * of the builtin overflow checkers. Fortunately, clang implements
> diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
> index d7ee4c6bad48..e5ebc788dde4 100644
> --- a/include/linux/compiler-gcc.h
> +++ b/include/linux/compiler-gcc.h
> @@ -145,6 +145,11 @@
> #define __no_sanitize_address
> #endif
>
> +/*
> + * GCC doesn't support KMSAN.
> + */
> +#define __no_sanitize_memory
> +
> #if GCC_VERSION >= 50100
> #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
> #endif
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__
2019-11-29 16:09 ` Andrey Konovalov
@ 2019-12-16 11:35 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-16 11:35 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Fri, Nov 29, 2019 at 5:09 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:26 PM <glider@google.com> wrote:
> >
> > __no_sanitize_memory is a function attribute that makes KMSAN
> > ignore the uninitialized values coming from the function's
> > inputs, and initialize the function's outputs.
> >
> > Functions marked with this attribute can't be inlined into functions
> > not marked with it, and vice versa.
> >
> > __SANITIZE_MEMORY__ is a macro that's defined iff the file is
> > instrumented with KMSAN. This is not the same as CONFIG_KMSAN, which is
> > defined for every file.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> >
> > ---
> >
> > Change-Id: I1f1672652c8392f15f7ca8ac26cd4e71f9cc1e4b
> > ---
> > include/linux/compiler-clang.h | 8 ++++++++
> > include/linux/compiler-gcc.h | 5 +++++
> > 2 files changed, 13 insertions(+)
> >
> > diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h
> > index 333a6695a918..edba13a069a6 100644
> > --- a/include/linux/compiler-clang.h
> > +++ b/include/linux/compiler-clang.h
> > @@ -24,6 +24,14 @@
> > #define __no_sanitize_address
> > #endif
> >
> > +/* KMSAN is a Clang-only tool, thus putting the defines here */
>
> I'm not sure if this comment is useful here.
Dropped the comment in v4.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 08/36] kmsan: reduce vmalloc space
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (6 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 07/36] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__ glider
@ 2019-11-22 11:25 ` glider
2019-11-28 13:30 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 09/36] kmsan: add KMSAN bits to struct page and struct task_struct glider
` (27 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Andrew Morton, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
KMSAN is going to use 3/4 of existing vmalloc space to hold the
metadata, therefore we lower VMALLOC_END to make sure vmalloc() doesn't
allocate past the first 1/4.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: Iaa5e8e0fc2aa66c956f937f5a1de6e5ef40d57cc
---
arch/x86/include/asm/pgtable_64_types.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h
index 52e5f5f2240d..586629e20436 100644
--- a/arch/x86/include/asm/pgtable_64_types.h
+++ b/arch/x86/include/asm/pgtable_64_types.h
@@ -139,7 +139,22 @@ extern unsigned int ptrs_per_p4d;
# define VMEMMAP_START __VMEMMAP_BASE_L4
#endif /* CONFIG_DYNAMIC_MEMORY_LAYOUT */
+#ifndef CONFIG_KMSAN
#define VMALLOC_END (VMALLOC_START + (VMALLOC_SIZE_TB << 40) - 1)
+#else
+/*
+ * In KMSAN builds vmalloc area is four times smaller, and the remaining 3/4
+ * are used to keep the metadata for virtual pages.
+ */
+#define VMALLOC_QUARTER_SIZE ((VMALLOC_SIZE_TB << 40) >> 2)
+#define VMALLOC_END (VMALLOC_START + VMALLOC_QUARTER_SIZE - 1)
+#define VMALLOC_SHADOW_OFFSET VMALLOC_QUARTER_SIZE
+#define VMALLOC_ORIGIN_OFFSET (VMALLOC_QUARTER_SIZE * 2)
+#define VMALLOC_META_END (VMALLOC_END + VMALLOC_ORIGIN_OFFSET)
+#define MODULES_SHADOW_START (VMALLOC_META_END + 1)
+#define MODULES_ORIGIN_START (MODULES_SHADOW_START + MODULES_LEN)
+#define MODULES_ORIGIN_END (MODULES_ORIGIN_START + MODULES_LEN)
+#endif
#define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE)
/* The module sections ends with the start of the fixmap */
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 08/36] kmsan: reduce vmalloc space
2019-11-22 11:25 ` [PATCH RFC v3 08/36] kmsan: reduce vmalloc space glider
@ 2019-11-28 13:30 ` Marco Elver
0 siblings, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-11-28 13:30 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Andrew Morton, Dmitry Vyukov,
Linux Memory Management List, Al Viro, adilger.kernel,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, hch, hch, darrick.wong, davem,
dmitry.torokhov, Eric Biggers, Eric Dumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, Mark Rutland, martin.petersen, schwidefsky,
Matthew Wilcox, mst, Michal Simek, pmladek, Qian Cai,
Randy Dunlap, robin.murphy, Sergey Senozhatsky, Steven Rostedt,
tiwai, tytso, Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> KMSAN is going to use 3/4 of existing vmalloc space to hold the
> metadata, therefore we lower VMALLOC_END to make sure vmalloc() doesn't
> allocate past the first 1/4.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
>
> Change-Id: Iaa5e8e0fc2aa66c956f937f5a1de6e5ef40d57cc
> ---
> arch/x86/include/asm/pgtable_64_types.h | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h
> index 52e5f5f2240d..586629e20436 100644
> --- a/arch/x86/include/asm/pgtable_64_types.h
> +++ b/arch/x86/include/asm/pgtable_64_types.h
> @@ -139,7 +139,22 @@ extern unsigned int ptrs_per_p4d;
> # define VMEMMAP_START __VMEMMAP_BASE_L4
> #endif /* CONFIG_DYNAMIC_MEMORY_LAYOUT */
>
> +#ifndef CONFIG_KMSAN
I think it might be more readable if this was non-negative, i.e.
'#ifdef CONFIG_KMSAN'.
> #define VMALLOC_END (VMALLOC_START + (VMALLOC_SIZE_TB << 40) - 1)
> +#else
> +/*
> + * In KMSAN builds vmalloc area is four times smaller, and the remaining 3/4
> + * are used to keep the metadata for virtual pages.
> + */
> +#define VMALLOC_QUARTER_SIZE ((VMALLOC_SIZE_TB << 40) >> 2)
> +#define VMALLOC_END (VMALLOC_START + VMALLOC_QUARTER_SIZE - 1)
> +#define VMALLOC_SHADOW_OFFSET VMALLOC_QUARTER_SIZE
> +#define VMALLOC_ORIGIN_OFFSET (VMALLOC_QUARTER_SIZE * 2)
> +#define VMALLOC_META_END (VMALLOC_END + VMALLOC_ORIGIN_OFFSET)
> +#define MODULES_SHADOW_START (VMALLOC_META_END + 1)
> +#define MODULES_ORIGIN_START (MODULES_SHADOW_START + MODULES_LEN)
> +#define MODULES_ORIGIN_END (MODULES_ORIGIN_START + MODULES_LEN)
Are all these, except VMALLOC_END, KMSAN specific? For readability
and avoid further confusion, would it make sense to prefix these with
'KMSAN_' ?
This file is for x86 only -- would other architectures define these
similarly? If so, maybe some of this could be moved into a helper,
such as include/asm-generic/kmsan_pgtable.h?
Thanks,
-- Marco
> +#endif
>
> #define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE)
> /* The module sections ends with the start of the fixmap */
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 09/36] kmsan: add KMSAN bits to struct page and struct task_struct
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (7 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 08/36] kmsan: reduce vmalloc space glider
@ 2019-11-22 11:25 ` glider
2019-11-28 13:44 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 10/36] kmsan: add KMSAN runtime glider
` (26 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin,
ard.biesheuvel, arnd, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso,
tglx, gor, wsa
Each struct page now contains pointers to two struct pages holding KMSAN
metadata (shadow and origins) for the original struct page.
Each task_struct contains a struct kmsan_task_state used to track the
metadata of function parameters and return values for that task.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: linux-mm@kvack.org
---
Change-Id: Ie329527e558dd60307fb88b2da151f7f4db951ac
---
include/linux/mm_types.h | 9 +++++++++
include/linux/sched.h | 5 +++++
2 files changed, 14 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 2222fa795284..c87c5416a802 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -216,6 +216,15 @@ struct page {
not kmapped, ie. highmem) */
#endif /* WANT_PAGE_VIRTUAL */
+#ifdef CONFIG_KMSAN
+ /*
+ * Bits in struct page are scarce, so the LSB in *shadow is used to
+ * indicate whether the page should be ignored by KMSAN or not.
+ */
+ struct page *shadow;
+ struct page *origin;
+#endif
+
#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS
int _last_cpupid;
#endif
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 72b20f33c56e..ba705f66f78c 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -15,6 +15,7 @@
#include <linux/sem.h>
#include <linux/shm.h>
#include <linux/kcov.h>
+#include <linux/kmsan.h>
#include <linux/mutex.h>
#include <linux/plist.h>
#include <linux/hrtimer.h>
@@ -1173,6 +1174,10 @@ struct task_struct {
unsigned int kasan_depth;
#endif
+#ifdef CONFIG_KMSAN
+ struct kmsan_task_state kmsan;
+#endif
+
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
/* Index of current stored address in ret_stack: */
int curr_ret_stack;
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 09/36] kmsan: add KMSAN bits to struct page and struct task_struct
2019-11-22 11:25 ` [PATCH RFC v3 09/36] kmsan: add KMSAN bits to struct page and struct task_struct glider
@ 2019-11-28 13:44 ` Marco Elver
2019-11-28 14:05 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-11-28 13:44 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, Linux Memory Management List, Al Viro,
adilger.kernel, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Ard Biesheuvel, Arnd Bergmann, hch, darrick.wong, davem,
dmitry.torokhov, Eric Biggers, Eric Dumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, m.szyprowski,
Mark Rutland, martin.petersen, schwidefsky, Matthew Wilcox, mst,
Michal Simek, pmladek, Qian Cai, Randy Dunlap, robin.murphy,
Sergey Senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> Each struct page now contains pointers to two struct pages holding KMSAN
> metadata (shadow and origins) for the original struct page.
>
> Each task_struct contains a struct kmsan_task_state used to track the
> metadata of function parameters and return values for that task.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: Ie329527e558dd60307fb88b2da151f7f4db951ac
> ---
> include/linux/mm_types.h | 9 +++++++++
> include/linux/sched.h | 5 +++++
> 2 files changed, 14 insertions(+)
>
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 2222fa795284..c87c5416a802 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -216,6 +216,15 @@ struct page {
> not kmapped, ie. highmem) */
> #endif /* WANT_PAGE_VIRTUAL */
>
> +#ifdef CONFIG_KMSAN
> + /*
> + * Bits in struct page are scarce, so the LSB in *shadow is used to
> + * indicate whether the page should be ignored by KMSAN or not.
> + */
> + struct page *shadow;
> + struct page *origin;
> +#endif
> +
> #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS
> int _last_cpupid;
> #endif
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index 72b20f33c56e..ba705f66f78c 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -15,6 +15,7 @@
> #include <linux/sem.h>
> #include <linux/shm.h>
> #include <linux/kcov.h>
> +#include <linux/kmsan.h>
This file does not exist at this point in the series, and the build
would fail for all configs. This can cause problems for 'git bisect'
users etc.
I would just merge this patch with patch 10.
Thanks,
-- Marco
> #include <linux/mutex.h>
> #include <linux/plist.h>
> #include <linux/hrtimer.h>
> @@ -1173,6 +1174,10 @@ struct task_struct {
> unsigned int kasan_depth;
> #endif
>
> +#ifdef CONFIG_KMSAN
> + struct kmsan_task_state kmsan;
> +#endif
> +
> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> /* Index of current stored address in ret_stack: */
> int curr_ret_stack;
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 09/36] kmsan: add KMSAN bits to struct page and struct task_struct
2019-11-28 13:44 ` Marco Elver
@ 2019-11-28 14:05 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-11-28 14:05 UTC (permalink / raw)
To: Marco Elver
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Ard Biesheuvel, Arnd Bergmann, hch, darrick.wong, David Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, jasowang, Marek Szyprowski, Mark Rutland,
Martin K . Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S. Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Thu, Nov 28, 2019 at 2:44 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
> >
> > Each struct page now contains pointers to two struct pages holding KMSAN
> > metadata (shadow and origins) for the original struct page.
> >
> > Each task_struct contains a struct kmsan_task_state used to track the
> > metadata of function parameters and return values for that task.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > Cc: Jens Axboe <axboe@kernel.dk>
> > Cc: Andy Lutomirski <luto@kernel.org>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: Ie329527e558dd60307fb88b2da151f7f4db951ac
> > ---
> > include/linux/mm_types.h | 9 +++++++++
> > include/linux/sched.h | 5 +++++
> > 2 files changed, 14 insertions(+)
> >
> > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> > index 2222fa795284..c87c5416a802 100644
> > --- a/include/linux/mm_types.h
> > +++ b/include/linux/mm_types.h
> > @@ -216,6 +216,15 @@ struct page {
> > not kmapped, ie. highmem) */
> > #endif /* WANT_PAGE_VIRTUAL */
> >
> > +#ifdef CONFIG_KMSAN
> > + /*
> > + * Bits in struct page are scarce, so the LSB in *shadow is used to
> > + * indicate whether the page should be ignored by KMSAN or not.
> > + */
> > + struct page *shadow;
> > + struct page *origin;
> > +#endif
> > +
> > #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS
> > int _last_cpupid;
> > #endif
> > diff --git a/include/linux/sched.h b/include/linux/sched.h
> > index 72b20f33c56e..ba705f66f78c 100644
> > --- a/include/linux/sched.h
> > +++ b/include/linux/sched.h
> > @@ -15,6 +15,7 @@
> > #include <linux/sem.h>
> > #include <linux/shm.h>
> > #include <linux/kcov.h>
> > +#include <linux/kmsan.h>
>
> This file does not exist at this point in the series, and the build
> would fail for all configs. This can cause problems for 'git bisect'
> users etc.
I thought we can move this patch past the KMSAN runtime then, but then
the runtime won't be able to use the newly added fields.
> I would just merge this patch with patch 10.
Ack.
> Thanks,
> -- Marco
>
>
> > #include <linux/mutex.h>
> > #include <linux/plist.h>
> > #include <linux/hrtimer.h>
> > @@ -1173,6 +1174,10 @@ struct task_struct {
> > unsigned int kasan_depth;
> > #endif
> >
> > +#ifdef CONFIG_KMSAN
> > + struct kmsan_task_state kmsan;
> > +#endif
> > +
> > #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> > /* Index of current stored address in ret_stack: */
> > int curr_ret_stack;
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (8 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 09/36] kmsan: add KMSAN bits to struct page and struct task_struct glider
@ 2019-11-22 11:25 ` glider
2019-11-24 19:44 ` Wolfram Sang
` (3 more replies)
2019-11-22 11:25 ` [PATCH RFC v3 11/36] kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot glider
` (25 subsequent siblings)
35 siblings, 4 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Wolfram Sang, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor
This patch adds the KernelMemorySanitizer runtime and associated files:
- arch/x86/include/asm/kmsan.h: assembly definitions for hooking
interrupt handlers;
- include/linux/kmsan-checks.h: user API to enable/disable KMSAN,
poison/unpoison memory etc.
- include/linux/kmsan.h: declarations of KMSAN memory hooks to be
referenced outside KMSAN runtime
- lib/Kconfig.kmsan: declarations for CONFIG_KMSAN and
CONFIG_TEST_KMSAN
- mm/kmsan/Makefile: boilerplate Makefile
- mm/kmsan/kmsan.h: internal KMSAN declarations
- mm/kmsan/kmsan.c: core functions that operate with shadow and
origin memory and perform checks, utility functions
- mm/kmsan/kmsan_entry.c: KMSAN hooks for entry_64.S
- mm/kmsan/kmsan_hooks.c: KMSAN hooks for kernel subsystems
- mm/kmsan/kmsan_init.c: KMSAN initialization routines
- mm/kmsan/kmsan_instr.c: functions called by KMSAN instrumentation
- scripts/Makefile.kmsan: CFLAGS_KMSAN
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Wolfram Sang <wsa@the-dreams.de>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v2:
- dropped kmsan_handle_vprintk()
- use locking for single kmsan_pr_err() calls
- don't try to understand we're inside printk()
v3:
- fix an endless loop in __msan_poison_alloca()
- implement kmsan_handle_dma()
- dropped kmsan_handle_i2c_transfer()
- fixed compilation with UNWINDER_ORC
- dropped assembly hooks for system calls
Change-Id: I4b3a7aba6d5804afac4f5f7274cadf8675b6e119
---
arch/x86/Kconfig | 1 +
arch/x86/include/asm/kmsan.h | 117 ++++++++
include/linux/kmsan-checks.h | 122 ++++++++
include/linux/kmsan.h | 143 +++++++++
lib/Kconfig.debug | 2 +
lib/Kconfig.kmsan | 22 ++
mm/kmsan/Makefile | 4 +
mm/kmsan/kmsan.c | 563 +++++++++++++++++++++++++++++++++++
mm/kmsan/kmsan.h | 146 +++++++++
mm/kmsan/kmsan_entry.c | 118 ++++++++
mm/kmsan/kmsan_hooks.c | 422 ++++++++++++++++++++++++++
mm/kmsan/kmsan_init.c | 88 ++++++
mm/kmsan/kmsan_instr.c | 259 ++++++++++++++++
mm/kmsan/kmsan_report.c | 133 +++++++++
mm/kmsan/kmsan_shadow.c | 543 +++++++++++++++++++++++++++++++++
mm/kmsan/kmsan_shadow.h | 30 ++
scripts/Makefile.kmsan | 12 +
17 files changed, 2725 insertions(+)
create mode 100644 arch/x86/include/asm/kmsan.h
create mode 100644 include/linux/kmsan-checks.h
create mode 100644 include/linux/kmsan.h
create mode 100644 lib/Kconfig.kmsan
create mode 100644 mm/kmsan/Makefile
create mode 100644 mm/kmsan/kmsan.c
create mode 100644 mm/kmsan/kmsan.h
create mode 100644 mm/kmsan/kmsan_entry.c
create mode 100644 mm/kmsan/kmsan_hooks.c
create mode 100644 mm/kmsan/kmsan_init.c
create mode 100644 mm/kmsan/kmsan_instr.c
create mode 100644 mm/kmsan/kmsan_report.c
create mode 100644 mm/kmsan/kmsan_shadow.c
create mode 100644 mm/kmsan/kmsan_shadow.h
create mode 100644 scripts/Makefile.kmsan
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index d6e1faa28c58..3f83a5c53808 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -135,6 +135,7 @@ config X86
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_JUMP_LABEL_RELATIVE
select HAVE_ARCH_KASAN if X86_64
+ select HAVE_ARCH_KMSAN if X86_64
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS if MMU
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT
diff --git a/arch/x86/include/asm/kmsan.h b/arch/x86/include/asm/kmsan.h
new file mode 100644
index 000000000000..fc5f1224a059
--- /dev/null
+++ b/arch/x86/include/asm/kmsan.h
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Assembly bits to safely invoke KMSAN hooks from .S files.
+ *
+ * Adopted from KTSAN assembly hooks implementation by Dmitry Vyukov:
+ * https://github.com/google/ktsan/blob/ktsan/arch/x86/include/asm/ktsan.h
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+#ifndef _ASM_X86_KMSAN_H
+#define _ASM_X86_KMSAN_H
+
+#ifdef CONFIG_KMSAN
+
+#define KMSAN_PUSH_REGS \
+ pushq %rax; \
+ pushq %rcx; \
+ pushq %rdx; \
+ pushq %rdi; \
+ pushq %rsi; \
+ pushq %r8; \
+ pushq %r9; \
+ pushq %r10; \
+ pushq %r11; \
+/**/
+
+#define KMSAN_POP_REGS \
+ popq %r11; \
+ popq %r10; \
+ popq %r9; \
+ popq %r8; \
+ popq %rsi; \
+ popq %rdi; \
+ popq %rdx; \
+ popq %rcx; \
+ popq %rax; \
+/**/
+
+#define KMSAN_INTERRUPT_ENTER \
+ KMSAN_PUSH_REGS \
+ call kmsan_interrupt_enter; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_INTERRUPT_EXIT \
+ KMSAN_PUSH_REGS \
+ call kmsan_interrupt_exit; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_SOFTIRQ_ENTER \
+ KMSAN_PUSH_REGS \
+ call kmsan_softirq_enter; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_SOFTIRQ_EXIT \
+ KMSAN_PUSH_REGS \
+ call kmsan_softirq_exit; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_NMI_ENTER \
+ KMSAN_PUSH_REGS \
+ call kmsan_nmi_enter; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_NMI_EXIT \
+ KMSAN_PUSH_REGS \
+ call kmsan_nmi_exit; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_IST_ENTER(shift_ist) \
+ KMSAN_PUSH_REGS \
+ movq $shift_ist, %rdi; \
+ call kmsan_ist_enter; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_IST_EXIT(shift_ist) \
+ KMSAN_PUSH_REGS \
+ movq $shift_ist, %rdi; \
+ call kmsan_ist_exit; \
+ KMSAN_POP_REGS \
+/**/
+
+#define KMSAN_UNPOISON_PT_REGS \
+ KMSAN_PUSH_REGS \
+ call kmsan_unpoison_pt_regs; \
+ KMSAN_POP_REGS \
+/**/
+
+
+#else /* ifdef CONFIG_KMSAN */
+
+#define KMSAN_INTERRUPT_ENTER
+#define KMSAN_INTERRUPT_EXIT
+#define KMSAN_SOFTIRQ_ENTER
+#define KMSAN_SOFTIRQ_EXIT
+#define KMSAN_NMI_ENTER
+#define KMSAN_NMI_EXIT
+#define KMSAN_SYSCALL_ENTER
+#define KMSAN_SYSCALL_EXIT
+#define KMSAN_IST_ENTER(shift_ist)
+#define KMSAN_IST_EXIT(shift_ist)
+#define KMSAN_UNPOISON_PT_REGS
+
+#endif /* ifdef CONFIG_KMSAN */
+#endif /* ifndef _ASM_X86_KMSAN_H */
diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h
new file mode 100644
index 000000000000..623854e88d4b
--- /dev/null
+++ b/include/linux/kmsan-checks.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * KMSAN checks.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef _LINUX_KMSAN_CHECKS_H
+#define _LINUX_KMSAN_CHECKS_H
+
+#include <linux/bug.h>
+#include <linux/dma-direction.h>
+#include <linux/types.h>
+
+struct page;
+struct sk_buff;
+struct urb;
+
+#ifdef CONFIG_KMSAN
+
+/*
+ * Helper functions that mark the return value initialized.
+ * Note that Clang ignores the inline attribute in the cases when a no_sanitize
+ * function is called from an instrumented one.
+ */
+
+__no_sanitize_memory
+static inline unsigned char KMSAN_INIT_1(unsigned char value)
+{
+ return value;
+}
+
+__no_sanitize_memory
+static inline unsigned short KMSAN_INIT_2(unsigned short value)
+{
+ return value;
+}
+
+__no_sanitize_memory
+static inline unsigned int KMSAN_INIT_4(unsigned int value)
+{
+ return value;
+}
+
+__no_sanitize_memory
+static inline unsigned long KMSAN_INIT_8(unsigned long value)
+{
+ return value;
+}
+
+#define KMSAN_INIT_VALUE(val) \
+ ({ \
+ typeof(val) __ret; \
+ switch (sizeof(val)) { \
+ case 1: \
+ *(unsigned char *)&__ret = KMSAN_INIT_1( \
+ (unsigned char)val); \
+ break; \
+ case 2: \
+ *(unsigned short *)&__ret = KMSAN_INIT_2( \
+ (unsigned short)val); \
+ break; \
+ case 4: \
+ *(unsigned int *)&__ret = KMSAN_INIT_4( \
+ (unsigned int)val); \
+ break; \
+ case 8: \
+ *(unsigned long *)&__ret = KMSAN_INIT_8( \
+ (unsigned long)val); \
+ break; \
+ default: \
+ BUILD_BUG_ON(1); \
+ } \
+ __ret; \
+ }) /**/
+
+void kmsan_ignore_page(struct page *page, int order);
+void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags);
+void kmsan_unpoison_shadow(const void *address, size_t size);
+void kmsan_check_memory(const void *address, size_t size);
+void kmsan_check_skb(const struct sk_buff *skb);
+void kmsan_handle_dma(const void *address, size_t size,
+ enum dma_data_direction direction);
+void kmsan_handle_urb(const struct urb *urb, bool is_out);
+void kmsan_copy_to_user(const void *to, const void *from, size_t to_copy,
+ size_t left);
+void *__msan_memcpy(void *dst, const void *src, u64 n);
+void kmsan_enter_runtime(unsigned long *flags);
+void kmsan_leave_runtime(unsigned long *flags);
+
+#else
+
+#define KMSAN_INIT_VALUE(value) (value)
+
+static inline void kmsan_ignore_page(struct page *page, int order) {}
+static inline void kmsan_poison_shadow(const void *address, size_t size,
+ gfp_t flags) {}
+static inline void kmsan_unpoison_shadow(const void *address, size_t size) {}
+static inline void kmsan_check_memory(const void *address, size_t size) {}
+static inline void kmsan_check_skb(const struct sk_buff *skb) {}
+static inline void kmsan_handle_urb(const struct urb *urb, bool is_out) {}
+static inline void kmsan_handle_dma(const void *address, size_t size,
+ enum dma_data_direction direction) {}
+static inline void kmsan_copy_to_user(
+ const void *to, const void *from, size_t to_copy, size_t left) {}
+static inline void *__msan_memcpy(void *dst, const void *src, size_t n)
+{
+ return NULL;
+}
+
+static inline void kmsan_enter_runtime(unsigned long *flags) {}
+static inline void kmsan_leave_runtime(unsigned long *flags) {}
+
+#endif
+
+#endif /* _LINUX_KMSAN_CHECKS_H */
diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
new file mode 100644
index 000000000000..f5638bac368e
--- /dev/null
+++ b/include/linux/kmsan.h
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * KMSAN API for subsystems.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+#ifndef LINUX_KMSAN_H
+#define LINUX_KMSAN_H
+
+#include <linux/gfp.h>
+#include <linux/stackdepot.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+struct page;
+struct kmem_cache;
+struct task_struct;
+struct vm_struct;
+
+
+extern bool kmsan_ready;
+
+#ifdef CONFIG_KMSAN
+void __init kmsan_initialize_shadow(void);
+void __init kmsan_initialize(void);
+
+/* These constants are defined in the MSan LLVM instrumentation pass. */
+#define RETVAL_SIZE 800
+#define KMSAN_PARAM_SIZE 800
+
+#define PARAM_ARRAY_SIZE (KMSAN_PARAM_SIZE / sizeof(depot_stack_handle_t))
+
+struct kmsan_context_state {
+ char param_tls[KMSAN_PARAM_SIZE];
+ char retval_tls[RETVAL_SIZE];
+ char va_arg_tls[KMSAN_PARAM_SIZE];
+ char va_arg_origin_tls[KMSAN_PARAM_SIZE];
+ u64 va_arg_overflow_size_tls;
+ depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE];
+ depot_stack_handle_t retval_origin_tls;
+ depot_stack_handle_t origin_tls;
+};
+
+struct kmsan_task_state {
+ bool allow_reporting;
+ struct kmsan_context_state cstate;
+};
+
+void kmsan_task_create(struct task_struct *task);
+void kmsan_task_exit(struct task_struct *task);
+void kmsan_alloc_shadow_for_region(void *start, size_t size);
+int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags);
+void kmsan_gup_pgd_range(struct page **pages, int nr);
+void kmsan_free_page(struct page *page, unsigned int order);
+void kmsan_split_page(struct page *page, unsigned int order);
+void kmsan_copy_page_meta(struct page *dst, struct page *src);
+
+void kmsan_poison_slab(struct page *page, gfp_t flags);
+void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
+void kmsan_kfree_large(const void *ptr);
+void kmsan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
+ gfp_t flags);
+void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
+void kmsan_slab_free(struct kmem_cache *s, void *object);
+
+void kmsan_slab_setup_object(struct kmem_cache *s, void *object);
+void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
+ size_t size, void *object);
+
+/* vmap */
+void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages);
+void kmsan_vunmap_page_range(unsigned long addr, unsigned long end);
+
+/* ioremap */
+void kmsan_ioremap_page_range(unsigned long addr, unsigned long end,
+ phys_addr_t phys_addr, pgprot_t prot);
+void kmsan_iounmap_page_range(unsigned long start, unsigned long end);
+
+void kmsan_softirq_enter(void);
+void kmsan_softirq_exit(void);
+
+void kmsan_clear_page(void *page_addr);
+
+#else
+
+static inline void __init kmsan_initialize_shadow(void) { }
+static inline void __init kmsan_initialize(void) { }
+
+static inline void kmsan_task_create(struct task_struct *task) {}
+static inline void kmsan_task_exit(struct task_struct *task) {}
+static inline void kmsan_alloc_shadow_for_region(void *start, size_t size) {}
+static inline int kmsan_alloc_page(struct page *page, unsigned int order,
+ gfp_t flags)
+{
+ return 0;
+}
+static inline void kmsan_gup_pgd_range(struct page **pages, int nr) {}
+static inline void kmsan_free_page(struct page *page, unsigned int order) {}
+static inline void kmsan_split_page(struct page *page, unsigned int order) {}
+static inline void kmsan_copy_page_meta(struct page *dst, struct page *src) {}
+
+static inline void kmsan_poison_slab(struct page *page, gfp_t flags) {}
+static inline void kmsan_kmalloc_large(const void *ptr, size_t size,
+ gfp_t flags) {}
+static inline void kmsan_kfree_large(const void *ptr) {}
+static inline void kmsan_kmalloc(struct kmem_cache *s, const void *object,
+ size_t size, gfp_t flags) {}
+static inline void kmsan_slab_alloc(struct kmem_cache *s, void *object,
+ gfp_t flags) {}
+static inline void kmsan_slab_free(struct kmem_cache *s, void *object) {}
+
+static inline void kmsan_slab_setup_object(struct kmem_cache *s,
+ void *object) {}
+static inline void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
+ size_t size, void *object) {}
+
+static inline void kmsan_vmap_page_range_noflush(unsigned long start,
+ unsigned long end,
+ pgprot_t prot,
+ struct page **pages) {}
+static inline void kmsan_vunmap_page_range(unsigned long start,
+ unsigned long end) {}
+
+static inline void kmsan_ioremap_page_range(unsigned long start,
+ unsigned long end,
+ phys_addr_t phys_addr,
+ pgprot_t prot) {}
+static inline void kmsan_iounmap_page_range(unsigned long start,
+ unsigned long end) {}
+static inline void kmsan_softirq_enter(void) {}
+static inline void kmsan_softirq_exit(void) {}
+
+static inline void kmsan_clear_page(void *page_addr) {}
+#endif
+
+#endif /* LINUX_KMSAN_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 93d97f9b0157..75c36318943d 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -756,6 +756,8 @@ config DEBUG_STACKOVERFLOW
source "lib/Kconfig.kasan"
+source "lib/Kconfig.kmsan"
+
endmenu # "Memory Debugging"
config ARCH_HAS_KCOV
diff --git a/lib/Kconfig.kmsan b/lib/Kconfig.kmsan
new file mode 100644
index 000000000000..187dddfcf220
--- /dev/null
+++ b/lib/Kconfig.kmsan
@@ -0,0 +1,22 @@
+config HAVE_ARCH_KMSAN
+ bool
+
+if HAVE_ARCH_KMSAN
+
+config KMSAN
+ bool "KMSAN: detector of uninitialized memory use"
+ depends on SLUB && !KASAN
+ select STACKDEPOT
+ help
+ KMSAN is a dynamic detector of uses of uninitialized memory in the
+ kernel. It is based on compiler instrumentation provided by Clang
+ and thus requires Clang 10.0.0+ to build.
+
+config TEST_KMSAN
+ tristate "Module for testing KMSAN for bug detection"
+ depends on m && KMSAN
+ help
+ Test module that can trigger various uses of uninitialized memory
+ detectable by KMSAN.
+
+endif
diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile
new file mode 100644
index 000000000000..ccf6d2d00a7a
--- /dev/null
+++ b/mm/kmsan/Makefile
@@ -0,0 +1,4 @@
+obj-y := kmsan.o kmsan_instr.o kmsan_init.o kmsan_entry.o kmsan_hooks.o kmsan_report.o kmsan_shadow.o
+
+KMSAN_SANITIZE := n
+KCOV_INSTRUMENT := n
diff --git a/mm/kmsan/kmsan.c b/mm/kmsan/kmsan.c
new file mode 100644
index 000000000000..21e97d4b1a99
--- /dev/null
+++ b/mm/kmsan/kmsan.c
@@ -0,0 +1,563 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN runtime library.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/page.h>
+#include <linux/compiler.h>
+#include <linux/export.h>
+#include <linux/highmem.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/kmsan.h>
+#include <linux/memory.h>
+#include <linux/mm.h>
+#include <linux/preempt.h>
+#include <linux/percpu-defs.h>
+#include <linux/mm_types.h>
+#include <linux/slab.h>
+#include <linux/stackdepot.h>
+#include <linux/stacktrace.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include <linux/mmzone.h>
+
+#include "../slab.h"
+#include "kmsan.h"
+
+/*
+ * Some kernel asm() calls mention the non-existing |__force_order| variable
+ * in the asm constraints to preserve the order of accesses to control
+ * registers. KMSAN turns those mentions into actual memory accesses, therefore
+ * the variable is now required to link the kernel.
+ */
+unsigned long __force_order;
+
+bool kmsan_ready;
+#define KMSAN_STACK_DEPTH 64
+#define MAX_CHAIN_DEPTH 7
+
+/*
+ * According to Documentation/x86/kernel-stacks, kernel code can run on the
+ * following stacks:
+ * - regular task stack - when executing the task code
+ * - interrupt stack - when handling external hardware interrupts and softirqs
+ * - NMI stack
+ * 0 is for regular interrupts, 1 for softirqs, 2 for NMI.
+ * Because interrupts may nest, trying to use a new context for every new
+ * interrupt.
+ */
+/* [0] for dummy per-CPU context. */
+DEFINE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
+ kmsan_percpu_cstate);
+/* 0 for task context, |i>0| for kmsan_context_state[i]. */
+DEFINE_PER_CPU(int, kmsan_context_level);
+DEFINE_PER_CPU(int, kmsan_in_interrupt);
+DEFINE_PER_CPU(bool, kmsan_in_softirq);
+DEFINE_PER_CPU(bool, kmsan_in_nmi);
+DEFINE_PER_CPU(int, kmsan_in_runtime);
+
+struct kmsan_context_state *task_kmsan_context_state(void)
+{
+ int cpu = smp_processor_id();
+ int level = this_cpu_read(kmsan_context_level);
+ struct kmsan_context_state *ret;
+
+ if (!kmsan_ready || IN_RUNTIME()) {
+ ret = &per_cpu(kmsan_percpu_cstate[0], cpu);
+ __memset(ret, 0, sizeof(struct kmsan_context_state));
+ return ret;
+ }
+
+ if (!level)
+ ret = ¤t->kmsan.cstate;
+ else
+ ret = &per_cpu(kmsan_percpu_cstate[level], cpu);
+ return ret;
+}
+
+void kmsan_internal_task_create(struct task_struct *task)
+{
+ struct kmsan_task_state *state = &task->kmsan;
+
+ __memset(state, 0, sizeof(struct kmsan_task_state));
+ state->allow_reporting = true;
+}
+
+void kmsan_internal_memset_shadow(void *addr, int b, size_t size,
+ bool checked)
+{
+ void *shadow_start;
+ u64 page_offset, address = (u64)addr;
+ size_t to_fill;
+
+ BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
+ while (size) {
+ page_offset = address % PAGE_SIZE;
+ to_fill = min(PAGE_SIZE - page_offset, (u64)size);
+ shadow_start = kmsan_get_metadata((void *)address, to_fill,
+ META_SHADOW);
+ if (!shadow_start) {
+ if (checked) {
+ kmsan_pr_locked("WARNING: not memsetting %d bytes starting at %px, because the shadow is NULL\n", to_fill, address);
+ BUG();
+ }
+ /* Otherwise just move on. */
+ } else {
+ __memset(shadow_start, b, to_fill);
+ }
+ address += to_fill;
+ size -= to_fill;
+ }
+}
+
+void kmsan_internal_poison_shadow(void *address, size_t size,
+ gfp_t flags, unsigned int poison_flags)
+{
+ bool checked = poison_flags & KMSAN_POISON_CHECK;
+ depot_stack_handle_t handle;
+ u32 extra_bits = 0;
+
+ if (poison_flags & KMSAN_POISON_FREE)
+ extra_bits = 1;
+ kmsan_internal_memset_shadow(address, -1, size, checked);
+ handle = kmsan_save_stack_with_flags(flags, extra_bits);
+ kmsan_set_origin_checked(address, size, handle, checked);
+}
+
+void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked)
+{
+ kmsan_internal_memset_shadow(address, 0, size, checked);
+ kmsan_set_origin_checked(address, size, 0, checked);
+}
+
+depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags,
+ unsigned int reserved)
+{
+ depot_stack_handle_t handle;
+ unsigned long entries[KMSAN_STACK_DEPTH];
+ unsigned int nr_entries;
+
+ nr_entries = stack_trace_save(entries, KMSAN_STACK_DEPTH, 0);
+ filter_irq_stacks(entries, nr_entries);
+
+ /* Don't sleep (see might_sleep_if() in __alloc_pages_nodemask()). */
+ flags &= ~__GFP_DIRECT_RECLAIM;
+
+ handle = stack_depot_save(entries, nr_entries, flags);
+ return set_dsh_extra_bits(handle, reserved);
+}
+
+/*
+ * Depending on the value of is_memmove, this serves as both a memcpy and a
+ * memmove implementation.
+ *
+ * As with the regular memmove, do the following:
+ * - if src and dst don't overlap, use memcpy();
+ * - if src and dst overlap:
+ * - if src > dst, use memcpy();
+ * - if src < dst, use reverse-memcpy.
+ * Why this is correct:
+ * - problems may arise if for some part of the overlapping region we
+ * overwrite its shadow with a new value before copying it somewhere.
+ * But there's a 1:1 mapping between the kernel memory and its shadow,
+ * therefore if this doesn't happen with the kernel memory it can't happen
+ * with the shadow.
+ */
+void kmsan_memcpy_memmove_metadata(void *dst, void *src, size_t n,
+ bool is_memmove)
+{
+ void *shadow_src, *shadow_dst;
+ depot_stack_handle_t *origin_src, *origin_dst;
+ int src_slots, dst_slots, i, iter, step, skip_bits;
+ depot_stack_handle_t old_origin = 0, chain_origin, new_origin = 0;
+ u32 *align_shadow_src, shadow;
+ bool backwards;
+
+ BUG_ON(!metadata_is_contiguous(dst, n, META_SHADOW));
+ BUG_ON(!metadata_is_contiguous(src, n, META_SHADOW));
+
+ shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
+ if (!shadow_dst)
+ return;
+
+ shadow_src = kmsan_get_metadata(src, n, META_SHADOW);
+ if (!shadow_src) {
+ /*
+ * |src| is untracked: zero out destination shadow, ignore the
+ * origins, we're done.
+ */
+ __memset(shadow_dst, 0, n);
+ return;
+ }
+ if (is_memmove)
+ __memmove(shadow_dst, shadow_src, n);
+ else
+ __memcpy(shadow_dst, shadow_src, n);
+
+ origin_dst = kmsan_get_metadata(dst, n, META_ORIGIN);
+ origin_src = kmsan_get_metadata(src, n, META_ORIGIN);
+ BUG_ON(!origin_dst || !origin_src);
+ BUG_ON(!metadata_is_contiguous(dst, n, META_ORIGIN));
+ BUG_ON(!metadata_is_contiguous(src, n, META_ORIGIN));
+ src_slots = (ALIGN((u64)src + n, ORIGIN_SIZE) -
+ ALIGN_DOWN((u64)src, ORIGIN_SIZE)) / ORIGIN_SIZE;
+ dst_slots = (ALIGN((u64)dst + n, ORIGIN_SIZE) -
+ ALIGN_DOWN((u64)dst, ORIGIN_SIZE)) / ORIGIN_SIZE;
+ BUG_ON(!src_slots || !dst_slots);
+ BUG_ON((src_slots < 1) || (dst_slots < 1));
+ BUG_ON((src_slots - dst_slots > 1) || (dst_slots - src_slots < -1));
+
+ backwards = is_memmove && (dst > src);
+ i = backwards ? min(src_slots, dst_slots) - 1 : 0;
+ iter = backwards ? -1 : 1;
+
+ align_shadow_src = (u32 *)ALIGN_DOWN((u64)shadow_src, ORIGIN_SIZE);
+ for (step = 0; step < min(src_slots, dst_slots); step++, i += iter) {
+ BUG_ON(i < 0);
+ shadow = align_shadow_src[i];
+ if (i == 0) {
+ /*
+ * If |src| isn't aligned on ORIGIN_SIZE, don't
+ * look at the first |src % ORIGIN_SIZE| bytes
+ * of the first shadow slot.
+ */
+ skip_bits = ((u64)src % ORIGIN_SIZE) * 8;
+ shadow = (shadow << skip_bits) >> skip_bits;
+ }
+ if (i == src_slots - 1) {
+ /*
+ * If |src + n| isn't aligned on
+ * ORIGIN_SIZE, don't look at the last
+ * |(src + n) % ORIGIN_SIZE| bytes of the
+ * last shadow slot.
+ */
+ skip_bits = (((u64)src + n) % ORIGIN_SIZE) * 8;
+ shadow = (shadow >> skip_bits) << skip_bits;
+ }
+ /*
+ * Overwrite the origin only if the corresponding
+ * shadow is nonempty.
+ */
+ if (origin_src[i] && (origin_src[i] != old_origin) && shadow) {
+ old_origin = origin_src[i];
+ chain_origin = kmsan_internal_chain_origin(old_origin);
+ /*
+ * kmsan_internal_chain_origin() may return
+ * NULL, but we don't want to lose the previous
+ * origin value.
+ */
+ if (chain_origin)
+ new_origin = chain_origin;
+ else
+ new_origin = old_origin;
+ }
+ if (shadow)
+ origin_dst[i] = new_origin;
+ else
+ origin_dst[i] = 0;
+ }
+}
+
+void kmsan_memcpy_metadata(void *dst, void *src, size_t n)
+{
+ kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/false);
+}
+
+void kmsan_memmove_metadata(void *dst, void *src, size_t n)
+{
+ kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/true);
+}
+
+depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id)
+{
+ depot_stack_handle_t handle;
+ unsigned long entries[3];
+ u64 magic = KMSAN_CHAIN_MAGIC_ORIGIN_FULL;
+ int depth = 0;
+ static int skipped;
+ u32 extra_bits;
+
+ if (!kmsan_ready)
+ return 0;
+
+ if (!id)
+ return id;
+ /*
+ * Make sure we have enough spare bits in |id| to hold the UAF bit and
+ * the chain depth.
+ */
+ BUILD_BUG_ON((1 << STACK_DEPOT_EXTRA_BITS) <= (MAX_CHAIN_DEPTH << 1));
+
+ extra_bits = get_dsh_extra_bits(id);
+
+ depth = extra_bits >> 1;
+ if (depth >= MAX_CHAIN_DEPTH) {
+ skipped++;
+ if (skipped % 10000 == 0) {
+ kmsan_pr_locked("not chained %d origins\n", skipped);
+ dump_stack();
+ kmsan_print_origin(id);
+ }
+ return id;
+ }
+ depth++;
+ /* Lowest bit is the UAF flag, higher bits hold the depth. */
+ extra_bits = (depth << 1) | (extra_bits & 1);
+ /* TODO(glider): how do we figure out we've dropped some frames? */
+ entries[0] = magic + depth;
+ entries[1] = kmsan_save_stack_with_flags(GFP_ATOMIC, extra_bits);
+ entries[2] = id;
+ handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
+ return set_dsh_extra_bits(handle, extra_bits);
+}
+
+void kmsan_write_aligned_origin(void *var, size_t size, u32 origin)
+{
+ u32 *var_cast = (u32 *)var;
+ int i;
+
+ BUG_ON((u64)var_cast % ORIGIN_SIZE);
+ BUG_ON(size % ORIGIN_SIZE);
+ for (i = 0; i < size / ORIGIN_SIZE; i++)
+ var_cast[i] = origin;
+}
+
+/*
+ * TODO(glider): writing an initialized byte shouldn't zero out the origin, if
+ * the remaining three bytes are uninitialized.
+ */
+void kmsan_internal_set_origin(void *addr, int size, u32 origin)
+{
+ void *origin_start;
+ u64 address = (u64)addr, page_offset;
+ size_t to_fill, pad = 0;
+
+ if (!IS_ALIGNED(address, ORIGIN_SIZE)) {
+ pad = address % ORIGIN_SIZE;
+ address -= pad;
+ size += pad;
+ }
+
+ while (size > 0) {
+ page_offset = address % PAGE_SIZE;
+ to_fill = min(PAGE_SIZE - page_offset, (u64)size);
+ /* write at least ORIGIN_SIZE bytes */
+ to_fill = ALIGN(to_fill, ORIGIN_SIZE);
+ BUG_ON(!to_fill);
+ origin_start = kmsan_get_metadata((void *)address, to_fill,
+ META_ORIGIN);
+ address += to_fill;
+ size -= to_fill;
+ if (!origin_start)
+ /* Can happen e.g. if the memory is untracked. */
+ continue;
+ kmsan_write_aligned_origin(origin_start, to_fill, origin);
+ }
+}
+
+void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked)
+{
+ if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) {
+ kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr);
+ BUG();
+ }
+ kmsan_internal_set_origin(addr, size, origin);
+}
+
+struct page *vmalloc_to_page_or_null(void *vaddr)
+{
+ struct page *page;
+
+ if (!kmsan_internal_is_vmalloc_addr(vaddr) &&
+ !kmsan_internal_is_module_addr(vaddr))
+ return NULL;
+ page = vmalloc_to_page(vaddr);
+ if (pfn_valid(page_to_pfn(page)))
+ return page;
+ else
+ return NULL;
+}
+
+void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr,
+ int reason)
+{
+ unsigned long irq_flags;
+ unsigned long addr64 = (unsigned long)addr;
+ unsigned char *shadow = NULL;
+ depot_stack_handle_t *origin = NULL;
+ depot_stack_handle_t cur_origin = 0, new_origin = 0;
+ int cur_off_start = -1;
+ int i, chunk_size;
+ size_t pos = 0;
+
+ BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
+ if (size <= 0)
+ return;
+ while (pos < size) {
+ chunk_size = min(size - pos,
+ PAGE_SIZE - ((addr64 + pos) % PAGE_SIZE));
+ shadow = kmsan_get_metadata((void *)(addr64 + pos), chunk_size,
+ META_SHADOW);
+ if (!shadow) {
+ /*
+ * This page is untracked. If there were uninitialized
+ * bytes before, report them.
+ */
+ if (cur_origin) {
+ ENTER_RUNTIME(irq_flags);
+ kmsan_report(cur_origin, addr, size,
+ cur_off_start, pos - 1, user_addr,
+ reason);
+ LEAVE_RUNTIME(irq_flags);
+ }
+ cur_origin = 0;
+ cur_off_start = -1;
+ pos += chunk_size;
+ continue;
+ }
+ for (i = 0; i < chunk_size; i++) {
+ if (!shadow[i]) {
+ /*
+ * This byte is unpoisoned. If there were
+ * poisoned bytes before, report them.
+ */
+ if (cur_origin) {
+ ENTER_RUNTIME(irq_flags);
+ kmsan_report(cur_origin, addr, size,
+ cur_off_start, pos + i - 1,
+ user_addr, reason);
+ LEAVE_RUNTIME(irq_flags);
+ }
+ cur_origin = 0;
+ cur_off_start = -1;
+ continue;
+ }
+ origin = kmsan_get_metadata((void *)(addr64 + pos + i),
+ chunk_size - i, META_ORIGIN);
+ BUG_ON(!origin);
+ new_origin = *origin;
+ /*
+ * Encountered new origin - report the previous
+ * uninitialized range.
+ */
+ if (cur_origin != new_origin) {
+ if (cur_origin) {
+ ENTER_RUNTIME(irq_flags);
+ kmsan_report(cur_origin, addr, size,
+ cur_off_start, pos + i - 1,
+ user_addr, reason);
+ LEAVE_RUNTIME(irq_flags);
+ }
+ cur_origin = new_origin;
+ cur_off_start = pos + i;
+ }
+ }
+ pos += chunk_size;
+ }
+ BUG_ON(pos != size);
+ if (cur_origin) {
+ ENTER_RUNTIME(irq_flags);
+ kmsan_report(cur_origin, addr, size, cur_off_start, pos - 1,
+ user_addr, reason);
+ LEAVE_RUNTIME(irq_flags);
+ }
+}
+
+/*
+ * TODO(glider): this check shouldn't be performed for origin pages, because
+ * they're always accessed after the shadow pages.
+ */
+bool metadata_is_contiguous(void *addr, size_t size, bool is_origin)
+{
+ u64 cur_addr = (u64)addr, next_addr;
+ char *cur_meta = NULL, *next_meta = NULL;
+ depot_stack_handle_t *origin_p;
+ bool all_untracked = false;
+ const char *fname = is_origin ? "origin" : "shadow";
+
+ if (!size)
+ return true;
+
+ /* The whole range belongs to the same page. */
+ if (ALIGN_DOWN(cur_addr + size - 1, PAGE_SIZE) ==
+ ALIGN_DOWN(cur_addr, PAGE_SIZE))
+ return true;
+ cur_meta = kmsan_get_metadata((void *)cur_addr, 1, is_origin);
+ if (!cur_meta)
+ all_untracked = true;
+ for (next_addr = cur_addr + PAGE_SIZE; next_addr < (u64)addr + size;
+ cur_addr = next_addr,
+ cur_meta = next_meta,
+ next_addr += PAGE_SIZE) {
+ next_meta = kmsan_get_metadata((void *)next_addr, 1, is_origin);
+ if (!next_meta) {
+ if (!all_untracked)
+ goto report;
+ continue;
+ }
+ if ((u64)cur_meta == ((u64)next_meta - PAGE_SIZE))
+ continue;
+ goto report;
+ }
+ return true;
+
+report:
+ kmsan_pr_locked("BUG: attempting to access two shadow page ranges.\n");
+ dump_stack();
+ kmsan_pr_locked("\n");
+ kmsan_pr_locked("Access of size %d at %px.\n", size, addr);
+ kmsan_pr_locked("Addresses belonging to different ranges: %px and %px\n",
+ cur_addr, next_addr);
+ kmsan_pr_locked("page[0].%s: %px, page[1].%s: %px\n",
+ fname, cur_meta, fname, next_meta);
+ origin_p = kmsan_get_metadata(addr, 1, META_ORIGIN);
+ if (origin_p) {
+ kmsan_pr_locked("Origin: %08x\n", *origin_p);
+ kmsan_print_origin(*origin_p);
+ } else {
+ kmsan_pr_locked("Origin: unavailable\n");
+ }
+ return false;
+}
+
+/*
+ * Dummy replacement for __builtin_return_address() which may crash without
+ * frame pointers.
+ */
+void *kmsan_internal_return_address(int arg)
+{
+#ifdef CONFIG_UNWINDER_FRAME_POINTER
+ switch (arg) {
+ case 1:
+ return __builtin_return_address(1);
+ case 2:
+ return __builtin_return_address(2);
+ default:
+ BUG();
+ }
+#else
+ unsigned long entries[1];
+
+ stack_trace_save(entries, 1, arg);
+ return (void *)entries[0];
+#endif
+}
+
+bool kmsan_internal_is_module_addr(void *vaddr)
+{
+ return ((u64)vaddr >= MODULES_VADDR) && ((u64)vaddr < MODULES_END);
+}
+
+bool kmsan_internal_is_vmalloc_addr(void *addr)
+{
+ return ((u64)addr >= VMALLOC_START) && ((u64)addr < VMALLOC_END);
+}
diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h
new file mode 100644
index 000000000000..8760feef39bf
--- /dev/null
+++ b/mm/kmsan/kmsan.h
@@ -0,0 +1,146 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * KMSAN internal declarations.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __MM_KMSAN_KMSAN_H
+#define __MM_KMSAN_KMSAN_H
+
+#include <asm/pgtable_64_types.h>
+#include <linux/irqflags.h>
+#include <linux/sched.h>
+#include <linux/stackdepot.h>
+#include <linux/stacktrace.h>
+#include <linux/nmi.h>
+#include <linux/mm.h>
+#include <linux/printk.h>
+
+#include "kmsan_shadow.h"
+
+#define KMSAN_MAGIC_MASK 0xffffffffff00
+#define KMSAN_ALLOCA_MAGIC_ORIGIN 0x4110c4071900
+#define KMSAN_CHAIN_MAGIC_ORIGIN_FULL 0xd419170cba00
+
+#define KMSAN_POISON_NOCHECK 0x0
+#define KMSAN_POISON_CHECK 0x1
+#define KMSAN_POISON_FREE 0x2
+
+#define ORIGIN_SIZE 4
+
+#define META_SHADOW (false)
+#define META_ORIGIN (true)
+
+#define KMSAN_NESTED_CONTEXT_MAX (8)
+/* [0] for dummy per-CPU context */
+DECLARE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
+ kmsan_percpu_cstate);
+/* 0 for task context, |i>0| for kmsan_context_state[i]. */
+DECLARE_PER_CPU(int, kmsan_context_level);
+DECLARE_PER_CPU(int, kmsan_in_interrupt);
+DECLARE_PER_CPU(bool, kmsan_in_softirq);
+DECLARE_PER_CPU(bool, kmsan_in_nmi);
+
+extern spinlock_t report_lock;
+
+/* Stolen from kernel/printk/internal.h */
+#define PRINTK_SAFE_CONTEXT_MASK 0x3fffffff
+
+/* Called by kmsan_report.c under a lock. */
+#define kmsan_pr_err(...) pr_err(__VA_ARGS__)
+
+/* Used in other places - doesn't require a lock. */
+#define kmsan_pr_locked(...) \
+ do { \
+ unsigned long flags; \
+ spin_lock_irqsave(&report_lock, flags); \
+ pr_err(__VA_ARGS__); \
+ spin_unlock_irqrestore(&report_lock, flags); \
+ } while (0)
+
+void kmsan_print_origin(depot_stack_handle_t origin);
+void kmsan_report(depot_stack_handle_t origin,
+ void *address, int size, int off_first, int off_last,
+ const void *user_addr, int reason);
+
+
+enum KMSAN_BUG_REASON {
+ REASON_ANY = 0,
+ REASON_COPY_TO_USER = 1,
+ REASON_USE_AFTER_FREE = 2,
+ REASON_SUBMIT_URB = 3,
+};
+
+/*
+ * When a compiler hook is invoked, it may make a call to instrumented code
+ * and eventually call itself recursively. To avoid that, we protect the
+ * runtime entry points with ENTER_RUNTIME()/LEAVE_RUNTIME() macros and exit
+ * the hook if IN_RUNTIME() is true. But when an interrupt occurs inside the
+ * runtime, the hooks won’t run either, which may lead to errors.
+ * Therefore we have to disable interrupts inside the runtime.
+ */
+DECLARE_PER_CPU(int, kmsan_in_runtime);
+#define IN_RUNTIME() (this_cpu_read(kmsan_in_runtime))
+#define ENTER_RUNTIME(irq_flags) \
+ do { \
+ preempt_disable(); \
+ local_irq_save(irq_flags); \
+ stop_nmi(); \
+ this_cpu_inc(kmsan_in_runtime); \
+ BUG_ON(this_cpu_read(kmsan_in_runtime) > 1); \
+ } while (0)
+#define LEAVE_RUNTIME(irq_flags) \
+ do { \
+ this_cpu_dec(kmsan_in_runtime); \
+ if (this_cpu_read(kmsan_in_runtime)) { \
+ kmsan_pr_err("kmsan_in_runtime: %d\n", \
+ this_cpu_read(kmsan_in_runtime)); \
+ BUG(); \
+ } \
+ restart_nmi(); \
+ local_irq_restore(irq_flags); \
+ preempt_enable(); } while (0)
+
+void kmsan_memcpy_metadata(void *dst, void *src, size_t n);
+void kmsan_memmove_metadata(void *dst, void *src, size_t n);
+
+depot_stack_handle_t kmsan_save_stack(void);
+depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags,
+ unsigned int extra_bits);
+void kmsan_internal_poison_shadow(void *address, size_t size, gfp_t flags,
+ unsigned int poison_flags);
+void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked);
+void kmsan_internal_memset_shadow(void *address, int b, size_t size,
+ bool checked);
+depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id);
+void kmsan_write_aligned_origin(void *var, size_t size, u32 origin);
+
+void kmsan_internal_task_create(struct task_struct *task);
+void kmsan_internal_set_origin(void *addr, int size, u32 origin);
+void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked);
+
+struct kmsan_context_state *task_kmsan_context_state(void);
+
+bool metadata_is_contiguous(void *addr, size_t size, bool is_origin);
+void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr,
+ int reason);
+
+struct page *vmalloc_to_page_or_null(void *vaddr);
+
+/* Declared in mm/vmalloc.c */
+void __vunmap_page_range(unsigned long addr, unsigned long end);
+int __vmap_page_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages);
+
+void *kmsan_internal_return_address(int arg);
+bool kmsan_internal_is_module_addr(void *vaddr);
+bool kmsan_internal_is_vmalloc_addr(void *addr);
+
+#endif /* __MM_KMSAN_KMSAN_H */
diff --git a/mm/kmsan/kmsan_entry.c b/mm/kmsan/kmsan_entry.c
new file mode 100644
index 000000000000..47bc7736f1a9
--- /dev/null
+++ b/mm/kmsan/kmsan_entry.c
@@ -0,0 +1,118 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN hooks for entry_64.S
+ *
+ * Copyright (C) 2018-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include "kmsan.h"
+
+static void kmsan_context_enter(void)
+{
+ int level = this_cpu_read(kmsan_context_level) + 1;
+
+ BUG_ON(level >= KMSAN_NESTED_CONTEXT_MAX);
+ this_cpu_write(kmsan_context_level, level);
+}
+
+static void kmsan_context_exit(void)
+{
+ int level = this_cpu_read(kmsan_context_level) - 1;
+
+ BUG_ON(level < 0);
+ this_cpu_write(kmsan_context_level, level);
+}
+
+void kmsan_interrupt_enter(void)
+{
+ int in_interrupt = this_cpu_read(kmsan_in_interrupt);
+
+ /* Turns out it's possible for in_interrupt to be >0 here. */
+ kmsan_context_enter();
+ BUG_ON(in_interrupt > 1);
+ /* Can't check preempt_count() here, it may be zero. */
+ this_cpu_write(kmsan_in_interrupt, in_interrupt + 1);
+}
+EXPORT_SYMBOL(kmsan_interrupt_enter);
+
+void kmsan_interrupt_exit(void)
+{
+ int in_interrupt = this_cpu_read(kmsan_in_interrupt);
+
+ BUG_ON(!in_interrupt);
+ kmsan_context_exit();
+ /* Can't check preempt_count() here, it may be zero. */
+ this_cpu_write(kmsan_in_interrupt, in_interrupt - 1);
+}
+EXPORT_SYMBOL(kmsan_interrupt_exit);
+
+void kmsan_softirq_enter(void)
+{
+ bool in_softirq = this_cpu_read(kmsan_in_softirq);
+
+ BUG_ON(in_softirq);
+ kmsan_context_enter();
+ /* Can't check preempt_count() here, it may be zero. */
+ this_cpu_write(kmsan_in_softirq, true);
+}
+EXPORT_SYMBOL(kmsan_softirq_enter);
+
+void kmsan_softirq_exit(void)
+{
+ bool in_softirq = this_cpu_read(kmsan_in_softirq);
+
+ BUG_ON(!in_softirq);
+ kmsan_context_exit();
+ /* Can't check preempt_count() here, it may be zero. */
+ this_cpu_write(kmsan_in_softirq, false);
+}
+EXPORT_SYMBOL(kmsan_softirq_exit);
+
+void kmsan_nmi_enter(void)
+{
+ bool in_nmi = this_cpu_read(kmsan_in_nmi);
+
+ BUG_ON(in_nmi);
+ BUG_ON(preempt_count() & NMI_MASK);
+ kmsan_context_enter();
+ this_cpu_write(kmsan_in_nmi, true);
+}
+EXPORT_SYMBOL(kmsan_nmi_enter);
+
+void kmsan_nmi_exit(void)
+{
+ bool in_nmi = this_cpu_read(kmsan_in_nmi);
+
+ BUG_ON(!in_nmi);
+ BUG_ON(preempt_count() & NMI_MASK);
+ kmsan_context_exit();
+ this_cpu_write(kmsan_in_nmi, false);
+
+}
+EXPORT_SYMBOL(kmsan_nmi_exit);
+
+void kmsan_ist_enter(u64 shift_ist)
+{
+ kmsan_context_enter();
+}
+EXPORT_SYMBOL(kmsan_ist_enter);
+
+void kmsan_ist_exit(u64 shift_ist)
+{
+ kmsan_context_exit();
+}
+EXPORT_SYMBOL(kmsan_ist_exit);
+
+void kmsan_unpoison_pt_regs(struct pt_regs *regs)
+{
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ kmsan_internal_unpoison_shadow(regs, sizeof(*regs), /*checked*/true);
+}
+EXPORT_SYMBOL(kmsan_unpoison_pt_regs);
diff --git a/mm/kmsan/kmsan_hooks.c b/mm/kmsan/kmsan_hooks.c
new file mode 100644
index 000000000000..13a6ed809d81
--- /dev/null
+++ b/mm/kmsan/kmsan_hooks.c
@@ -0,0 +1,422 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN hooks for kernel subsystems.
+ *
+ * These functions handle creation of KMSAN metadata for memory allocations.
+ *
+ * Copyright (C) 2018-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <asm/cacheflush.h>
+#include <linux/dma-direction.h>
+#include <linux/gfp.h>
+#include <linux/mm.h>
+#include <linux/mm_types.h>
+#include <linux/skbuff.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include "../slab.h"
+#include "kmsan.h"
+
+/*
+ * The functions may call back to instrumented code, which, in turn, may call
+ * these hooks again. To avoid re-entrancy, we use __GFP_NO_KMSAN_SHADOW.
+ * Instrumented functions shouldn't be called under
+ * ENTER_RUNTIME()/LEAVE_RUNTIME(), because this will lead to skipping
+ * effects of functions like memset() inside instrumented code.
+ */
+/* Called from kernel/kthread.c, kernel/fork.c */
+void kmsan_task_create(struct task_struct *task)
+{
+ unsigned long irq_flags;
+
+ if (!task)
+ return;
+ ENTER_RUNTIME(irq_flags);
+ kmsan_internal_task_create(task);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_task_create);
+
+
+/* Called from kernel/exit.c */
+void kmsan_task_exit(struct task_struct *task)
+{
+ unsigned long irq_flags;
+ struct kmsan_task_state *state = &task->kmsan;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ ENTER_RUNTIME(irq_flags);
+ state->allow_reporting = false;
+
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_task_exit);
+
+/* Called from mm/slub.c */
+void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags)
+{
+ unsigned long irq_flags;
+
+ if (unlikely(object == NULL))
+ return;
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ /*
+ * There's a ctor or this is an RCU cache - do nothing. The memory
+ * status hasn't changed since last use.
+ */
+ if (s->ctor || (s->flags & SLAB_TYPESAFE_BY_RCU))
+ return;
+
+ ENTER_RUNTIME(irq_flags);
+ if (flags & __GFP_ZERO) {
+ kmsan_internal_unpoison_shadow(object, s->object_size,
+ KMSAN_POISON_CHECK);
+ } else {
+ kmsan_internal_poison_shadow(object, s->object_size, flags,
+ KMSAN_POISON_CHECK);
+ }
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_slab_alloc);
+
+/* Called from mm/slub.c */
+void kmsan_slab_free(struct kmem_cache *s, void *object)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ ENTER_RUNTIME(irq_flags);
+
+ /* RCU slabs could be legally used after free within the RCU period */
+ if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
+ goto leave;
+ if (s->ctor)
+ goto leave;
+ kmsan_internal_poison_shadow(object, s->object_size,
+ GFP_KERNEL,
+ KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
+leave:
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_slab_free);
+
+/* Called from mm/slub.c */
+void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
+{
+ unsigned long irq_flags;
+
+ if (unlikely(ptr == NULL))
+ return;
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ ENTER_RUNTIME(irq_flags);
+ if (flags & __GFP_ZERO) {
+ kmsan_internal_unpoison_shadow((void *)ptr, size,
+ /*checked*/true);
+ } else {
+ kmsan_internal_poison_shadow((void *)ptr, size, flags,
+ KMSAN_POISON_CHECK);
+ }
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_kmalloc_large);
+
+/* Called from mm/slub.c */
+void kmsan_kfree_large(const void *ptr)
+{
+ struct page *page;
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ ENTER_RUNTIME(irq_flags);
+ page = virt_to_head_page((void *)ptr);
+ BUG_ON(ptr != page_address(page));
+ kmsan_internal_poison_shadow(
+ (void *)ptr, PAGE_SIZE << compound_order(page), GFP_KERNEL,
+ KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_kfree_large);
+
+static unsigned long vmalloc_shadow(unsigned long addr)
+{
+ return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_SHADOW);
+}
+
+static unsigned long vmalloc_origin(unsigned long addr)
+{
+ return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_ORIGIN);
+}
+
+/* Called from mm/vmalloc.c */
+void kmsan_vunmap_page_range(unsigned long start, unsigned long end)
+{
+ __vunmap_page_range(vmalloc_shadow(start), vmalloc_shadow(end));
+ __vunmap_page_range(vmalloc_origin(start), vmalloc_origin(end));
+}
+EXPORT_SYMBOL(kmsan_vunmap_page_range);
+
+/* Called from lib/ioremap.c */
+/*
+ * This function creates new shadow/origin pages for the physical pages mapped
+ * into the virtual memory. If those physical pages already had shadow/origin,
+ * those are ignored.
+ */
+void kmsan_ioremap_page_range(unsigned long start, unsigned long end,
+ phys_addr_t phys_addr, pgprot_t prot)
+{
+ unsigned long irq_flags;
+ struct page *shadow, *origin;
+ int i, nr;
+ unsigned long off = 0;
+ gfp_t gfp_mask = GFP_KERNEL | __GFP_ZERO | __GFP_NO_KMSAN_SHADOW;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ nr = (end - start) / PAGE_SIZE;
+ ENTER_RUNTIME(irq_flags);
+ for (i = 0; i < nr; i++, off += PAGE_SIZE) {
+ shadow = alloc_pages(gfp_mask, 1);
+ origin = alloc_pages(gfp_mask, 1);
+ __vmap_page_range_noflush(vmalloc_shadow(start + off),
+ vmalloc_shadow(start + off + PAGE_SIZE),
+ prot, &shadow);
+ __vmap_page_range_noflush(vmalloc_origin(start + off),
+ vmalloc_origin(start + off + PAGE_SIZE),
+ prot, &origin);
+ }
+ flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
+ flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_ioremap_page_range);
+
+void kmsan_iounmap_page_range(unsigned long start, unsigned long end)
+{
+ int i, nr;
+ struct page *shadow, *origin;
+ unsigned long v_shadow, v_origin;
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ nr = (end - start) / PAGE_SIZE;
+ ENTER_RUNTIME(irq_flags);
+ v_shadow = (unsigned long)vmalloc_shadow(start);
+ v_origin = (unsigned long)vmalloc_origin(start);
+ for (i = 0; i < nr; i++, v_shadow += PAGE_SIZE, v_origin += PAGE_SIZE) {
+ shadow = vmalloc_to_page_or_null((void *)v_shadow);
+ origin = vmalloc_to_page_or_null((void *)v_origin);
+ __vunmap_page_range(v_shadow, v_shadow + PAGE_SIZE);
+ __vunmap_page_range(v_origin, v_origin + PAGE_SIZE);
+ if (shadow)
+ __free_pages(shadow, 1);
+ if (origin)
+ __free_pages(origin, 1);
+ }
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_iounmap_page_range);
+
+/* Called from include/linux/uaccess.h, include/linux/uaccess.h */
+void kmsan_copy_to_user(const void *to, const void *from,
+ size_t to_copy, size_t left)
+{
+ void *shadow;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ /*
+ * At this point we've copied the memory already. It's hard to check it
+ * before copying, as the size of actually copied buffer is unknown.
+ */
+
+ /* copy_to_user() may copy zero bytes. No need to check. */
+ if (!to_copy)
+ return;
+ /* Or maybe copy_to_user() failed to copy anything. */
+ if (to_copy == left)
+ return;
+ if ((u64)to < TASK_SIZE) {
+ /* This is a user memory access, check it. */
+ kmsan_internal_check_memory((void *)from, to_copy - left, to,
+ REASON_COPY_TO_USER);
+ return;
+ }
+ /* Otherwise this is a kernel memory access. This happens when a compat
+ * syscall passes an argument allocated on the kernel stack to a real
+ * syscall.
+ * Don't check anything, just copy the shadow of the copied bytes.
+ */
+ shadow = kmsan_get_metadata((void *)to, to_copy - left, META_SHADOW);
+ if (shadow)
+ kmsan_memcpy_metadata((void *)to, (void *)from, to_copy - left);
+}
+EXPORT_SYMBOL(kmsan_copy_to_user);
+
+void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ ENTER_RUNTIME(irq_flags);
+ /* The users may want to poison/unpoison random memory. */
+ kmsan_internal_poison_shadow((void *)address, size, flags,
+ KMSAN_POISON_NOCHECK);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_poison_shadow);
+
+void kmsan_unpoison_shadow(const void *address, size_t size)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ ENTER_RUNTIME(irq_flags);
+ /* The users may want to poison/unpoison random memory. */
+ kmsan_internal_unpoison_shadow((void *)address, size,
+ KMSAN_POISON_NOCHECK);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_unpoison_shadow);
+
+void kmsan_check_memory(const void *addr, size_t size)
+{
+ return kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0,
+ REASON_ANY);
+}
+EXPORT_SYMBOL(kmsan_check_memory);
+
+void kmsan_gup_pgd_range(struct page **pages, int nr)
+{
+ int i;
+ void *page_addr;
+
+ /*
+ * gup_pgd_range() has just created a number of new pages that KMSAN
+ * treats as uninitialized. In the case they belong to the userspace
+ * memory, unpoison the corresponding kernel pages.
+ */
+ for (i = 0; i < nr; i++) {
+ page_addr = page_address(pages[i]);
+ if (((u64)page_addr < TASK_SIZE) &&
+ ((u64)page_addr + PAGE_SIZE < TASK_SIZE))
+ kmsan_unpoison_shadow(page_addr, PAGE_SIZE);
+ }
+
+}
+EXPORT_SYMBOL(kmsan_gup_pgd_range);
+
+/* Helper function to check an SKB. */
+void kmsan_check_skb(const struct sk_buff *skb)
+{
+ int start = skb_headlen(skb);
+ struct sk_buff *frag_iter;
+ int i, copy = 0;
+ skb_frag_t *f;
+ u32 p_off, p_len, copied;
+ struct page *p;
+ u8 *vaddr;
+
+ if (!skb || !skb->len)
+ return;
+
+ kmsan_internal_check_memory(skb->data, skb_headlen(skb), 0, REASON_ANY);
+ if (skb_is_nonlinear(skb)) {
+ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+ f = &skb_shinfo(skb)->frags[i];
+
+ skb_frag_foreach_page(f,
+ skb_frag_off(f) - start,
+ copy, p, p_off, p_len, copied) {
+
+ vaddr = kmap_atomic(p);
+ kmsan_internal_check_memory(vaddr + p_off,
+ p_len, /*user_addr*/ 0,
+ REASON_ANY);
+ kunmap_atomic(vaddr);
+ }
+ }
+ }
+ skb_walk_frags(skb, frag_iter)
+ kmsan_check_skb(frag_iter);
+}
+EXPORT_SYMBOL(kmsan_check_skb);
+
+/* Helper function to check an URB. */
+void kmsan_handle_urb(const struct urb *urb, bool is_out)
+{
+ if (!urb)
+ return;
+ if (is_out)
+ kmsan_internal_check_memory(urb->transfer_buffer,
+ urb->transfer_buffer_length,
+ /*user_addr*/ 0, REASON_SUBMIT_URB);
+ else
+ kmsan_internal_unpoison_shadow(urb->transfer_buffer,
+ urb->transfer_buffer_length,
+ /*checked*/false);
+}
+EXPORT_SYMBOL(kmsan_handle_urb);
+
+static void kmsan_handle_dma_page(const void *addr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_BIDIRECTIONAL:
+ kmsan_internal_check_memory((void *)addr, size, /*user_addr*/0,
+ REASON_ANY);
+ kmsan_internal_unpoison_shadow((void *)addr, size,
+ /*checked*/false);
+ break;
+ case DMA_TO_DEVICE:
+ kmsan_internal_check_memory((void *)addr, size, /*user_addr*/0,
+ REASON_ANY);
+ break;
+ case DMA_FROM_DEVICE:
+ kmsan_internal_unpoison_shadow((void *)addr, size,
+ /*checked*/false);
+ break;
+ case DMA_NONE:
+ break;
+ }
+}
+
+/* Helper function to handle DMA data transfers. */
+void kmsan_handle_dma(const void *addr, size_t size,
+ enum dma_data_direction dir)
+{
+ u64 page_offset, to_go, uaddr = (u64)addr;
+
+ /*
+ * The kernel may occasionally give us adjacent DMA pages not belonging
+ * to the same allocation. Process them separately to avoid triggering
+ * internal KMSAN checks.
+ */
+ while (size > 0) {
+ page_offset = uaddr % PAGE_SIZE;
+ to_go = min(PAGE_SIZE - page_offset, (u64)size);
+ kmsan_handle_dma_page((void *)uaddr, to_go, dir);
+ uaddr += to_go;
+ size -= to_go;
+ }
+}
+EXPORT_SYMBOL(kmsan_handle_dma);
diff --git a/mm/kmsan/kmsan_init.c b/mm/kmsan/kmsan_init.c
new file mode 100644
index 000000000000..2816e7075a30
--- /dev/null
+++ b/mm/kmsan/kmsan_init.c
@@ -0,0 +1,88 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN initialization routines.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include "kmsan.h"
+
+#include <asm/cpu_entry_area.h>
+#include <linux/mm.h>
+#include <linux/memblock.h>
+
+#define NUM_FUTURE_RANGES 128
+struct start_end_pair {
+ void *start, *end;
+};
+
+static struct start_end_pair start_end_pairs[NUM_FUTURE_RANGES] __initdata;
+static int future_index __initdata;
+
+/*
+ * Record a range of memory for which the metadata pages will be created once
+ * the page allocator becomes available.
+ * TODO(glider): squash together ranges belonging to the same page.
+ */
+static void __init kmsan_record_future_shadow_range(void *start, void *end)
+{
+ BUG_ON(future_index == NUM_FUTURE_RANGES);
+ BUG_ON((start >= end) || !start || !end);
+ start_end_pairs[future_index].start = start;
+ start_end_pairs[future_index].end = end;
+ future_index++;
+}
+
+extern char _sdata[], _edata[];
+
+
+
+/*
+ * Initialize the shadow for existing mappings during kernel initialization.
+ * These include kernel text/data sections, NODE_DATA and future ranges
+ * registered while creating other data (e.g. percpu).
+ *
+ * Allocations via memblock can be only done before slab is initialized.
+ */
+void __init kmsan_initialize_shadow(void)
+{
+ int nid;
+ u64 i;
+ const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE);
+ phys_addr_t p_start, p_end;
+
+ for_each_reserved_mem_region(i, &p_start, &p_end) {
+ kmsan_record_future_shadow_range(phys_to_virt(p_start),
+ phys_to_virt(p_end+1));
+ }
+ /* Allocate shadow for .data */
+ kmsan_record_future_shadow_range(_sdata, _edata);
+
+ /*
+ * TODO(glider): alloc_node_data() in arch/x86/mm/numa.c uses
+ * sizeof(pg_data_t).
+ */
+ for_each_online_node(nid)
+ kmsan_record_future_shadow_range(
+ NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size);
+
+ for (i = 0; i < future_index; i++)
+ kmsan_init_alloc_meta_for_range(start_end_pairs[i].start,
+ start_end_pairs[i].end);
+}
+EXPORT_SYMBOL(kmsan_initialize_shadow);
+
+void __init kmsan_initialize(void)
+{
+ /* Assuming current is init_task */
+ kmsan_internal_task_create(current);
+ kmsan_pr_locked("Starting KernelMemorySanitizer\n");
+ kmsan_ready = true;
+}
+EXPORT_SYMBOL(kmsan_initialize);
diff --git a/mm/kmsan/kmsan_instr.c b/mm/kmsan/kmsan_instr.c
new file mode 100644
index 000000000000..7695daf2d88a
--- /dev/null
+++ b/mm/kmsan/kmsan_instr.c
@@ -0,0 +1,259 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN compiler API.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include "kmsan.h"
+#include <linux/gfp.h>
+#include <linux/mm.h>
+
+static bool is_bad_asm_addr(void *addr, u64 size, bool is_store)
+{
+ if ((u64)addr < TASK_SIZE)
+ return true;
+ if (!kmsan_get_metadata(addr, size, META_SHADOW))
+ return true;
+ return false;
+}
+
+struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, u64 size)
+{
+ return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false);
+}
+EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n);
+
+struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, u64 size)
+{
+ return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true);
+}
+EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n);
+
+#define DECLARE_METADATA_PTR_GETTER(size) \
+struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size(void *addr) \
+{ \
+ return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); \
+} \
+EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \
+ \
+struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size(void *addr) \
+{ \
+ return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); \
+} \
+EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size)
+
+DECLARE_METADATA_PTR_GETTER(1);
+DECLARE_METADATA_PTR_GETTER(2);
+DECLARE_METADATA_PTR_GETTER(4);
+DECLARE_METADATA_PTR_GETTER(8);
+
+void __msan_instrument_asm_store(void *addr, u64 size)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ /*
+ * Most of the accesses are below 32 bytes. The two exceptions so far
+ * are clwb() (64 bytes) and FPU state (512 bytes).
+ * It's unlikely that the assembly will touch more than 512 bytes.
+ */
+ if (size > 512)
+ size = 8;
+ if (is_bad_asm_addr(addr, size, /*is_store*/true))
+ return;
+ ENTER_RUNTIME(irq_flags);
+ /* Unpoisoning the memory on best effort. */
+ kmsan_internal_unpoison_shadow(addr, size, /*checked*/false);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(__msan_instrument_asm_store);
+
+void *__msan_memmove(void *dst, void *src, u64 n)
+{
+ void *result;
+ void *shadow_dst;
+
+ result = __memmove(dst, src, n);
+ if (!n)
+ /* Some people call memmove() with zero length. */
+ return result;
+ if (!kmsan_ready || IN_RUNTIME())
+ return result;
+
+ /* Ok to skip address check here, we'll do it later. */
+ shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
+
+ if (!shadow_dst)
+ /* Can happen e.g. if the memory is untracked. */
+ return result;
+
+ kmsan_memmove_metadata(dst, src, n);
+
+ return result;
+}
+EXPORT_SYMBOL(__msan_memmove);
+
+void *__msan_memmove_nosanitize(void *dst, void *src, u64 n)
+{
+ return __memmove(dst, src, n);
+}
+EXPORT_SYMBOL(__msan_memmove_nosanitize);
+
+void *__msan_memcpy(void *dst, const void *src, u64 n)
+{
+ void *result;
+ void *shadow_dst;
+
+ result = __memcpy(dst, src, n);
+ if (!n)
+ /* Some people call memcpy() with zero length. */
+ return result;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return result;
+
+ /* Ok to skip address check here, we'll do it later. */
+ shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
+ if (!shadow_dst)
+ /* Can happen e.g. if the memory is untracked. */
+ return result;
+
+ kmsan_memcpy_metadata(dst, (void *)src, n);
+
+ return result;
+}
+EXPORT_SYMBOL(__msan_memcpy);
+
+void *__msan_memcpy_nosanitize(void *dst, void *src, u64 n)
+{
+ return __memcpy(dst, src, n);
+}
+EXPORT_SYMBOL(__msan_memcpy_nosanitize);
+
+void *__msan_memset(void *dst, int c, size_t n)
+{
+ void *result;
+ unsigned long irq_flags;
+ depot_stack_handle_t new_origin;
+ unsigned int shadow;
+
+ result = __memset(dst, c, n);
+ if (!kmsan_ready || IN_RUNTIME())
+ return result;
+
+ ENTER_RUNTIME(irq_flags);
+ shadow = 0;
+ kmsan_internal_memset_shadow(dst, shadow, n, /*checked*/false);
+ new_origin = 0;
+ kmsan_internal_set_origin(dst, n, new_origin);
+ LEAVE_RUNTIME(irq_flags);
+
+ return result;
+}
+EXPORT_SYMBOL(__msan_memset);
+
+void *__msan_memset_nosanitize(void *dst, int c, size_t n)
+{
+ return __memset(dst, c, n);
+}
+EXPORT_SYMBOL(__msan_memset_nosanitize);
+
+depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin)
+{
+ depot_stack_handle_t ret = 0;
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return ret;
+
+ /* Creating new origins may allocate memory. */
+ ENTER_RUNTIME(irq_flags);
+ ret = kmsan_internal_chain_origin(origin);
+ LEAVE_RUNTIME(irq_flags);
+ return ret;
+}
+EXPORT_SYMBOL(__msan_chain_origin);
+
+void __msan_poison_alloca(void *address, u64 size, char *descr)
+{
+ depot_stack_handle_t handle;
+ unsigned long entries[4];
+ unsigned long irq_flags;
+ u64 size_copy = size, to_fill;
+ u64 addr_copy = (u64)address;
+ u64 page_offset;
+ void *shadow_start;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ while (size_copy) {
+ page_offset = addr_copy % PAGE_SIZE;
+ to_fill = min(PAGE_SIZE - page_offset, size_copy);
+ shadow_start = kmsan_get_metadata((void *)addr_copy, to_fill,
+ META_SHADOW);
+ addr_copy += to_fill;
+ size_copy -= to_fill;
+ if (!shadow_start)
+ /* Can happen e.g. if the memory is untracked. */
+ continue;
+ __memset(shadow_start, -1, to_fill);
+ }
+
+ entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN;
+ entries[1] = (u64)descr;
+ entries[2] = (u64)__builtin_return_address(0);
+ entries[3] = (u64)kmsan_internal_return_address(1);
+
+ /* stack_depot_save() may allocate memory. */
+ ENTER_RUNTIME(irq_flags);
+ handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
+ LEAVE_RUNTIME(irq_flags);
+ kmsan_internal_set_origin(address, size, handle);
+}
+EXPORT_SYMBOL(__msan_poison_alloca);
+
+void __msan_unpoison_alloca(void *address, u64 size)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ ENTER_RUNTIME(irq_flags);
+ /* Assuming the shadow exists. */
+ kmsan_internal_unpoison_shadow(address, size, /*checked*/true);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(__msan_unpoison_alloca);
+
+void __msan_warning(u32 origin)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ ENTER_RUNTIME(irq_flags);
+ kmsan_report(origin, /*address*/0, /*size*/0,
+ /*off_first*/0, /*off_last*/0, /*user_addr*/0, REASON_ANY);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(__msan_warning);
+
+struct kmsan_context_state *__msan_get_context_state(void)
+{
+ struct kmsan_context_state *ret;
+
+ ret = task_kmsan_context_state();
+ BUG_ON(!ret);
+ return ret;
+}
+EXPORT_SYMBOL(__msan_get_context_state);
diff --git a/mm/kmsan/kmsan_report.c b/mm/kmsan/kmsan_report.c
new file mode 100644
index 000000000000..443ab9c1e8bf
--- /dev/null
+++ b/mm/kmsan/kmsan_report.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN error reporting routines.
+ *
+ * Copyright (C) 2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/console.h>
+#include <linux/stackdepot.h>
+#include <linux/stacktrace.h>
+
+#include "kmsan.h"
+
+DEFINE_SPINLOCK(report_lock);
+
+void kmsan_print_origin(depot_stack_handle_t origin)
+{
+ unsigned long *entries = NULL, *chained_entries = NULL;
+ unsigned long nr_entries, chained_nr_entries, magic;
+ char *descr = NULL;
+ void *pc1 = NULL, *pc2 = NULL;
+ depot_stack_handle_t head;
+
+ if (!origin) {
+ kmsan_pr_err("Origin not found, presumably a false report.\n");
+ return;
+ }
+
+ while (true) {
+ nr_entries = stack_depot_fetch(origin, &entries);
+ magic = nr_entries ? (entries[0] & KMSAN_MAGIC_MASK) : 0;
+ if ((nr_entries == 4) && (magic == KMSAN_ALLOCA_MAGIC_ORIGIN)) {
+ descr = (char *)entries[1];
+ pc1 = (void *)entries[2];
+ pc2 = (void *)entries[3];
+ kmsan_pr_err("Local variable description: %s\n", descr);
+ kmsan_pr_err("Variable was created at:\n");
+ kmsan_pr_err(" %pS\n", pc1);
+ kmsan_pr_err(" %pS\n", pc2);
+ break;
+ }
+ if ((nr_entries == 3) &&
+ (magic == KMSAN_CHAIN_MAGIC_ORIGIN_FULL)) {
+ head = entries[1];
+ origin = entries[2];
+ kmsan_pr_err("Uninit was stored to memory at:\n");
+ chained_nr_entries =
+ stack_depot_fetch(head, &chained_entries);
+ stack_trace_print(chained_entries, chained_nr_entries,
+ 0);
+ kmsan_pr_err("\n");
+ continue;
+ }
+ kmsan_pr_err("Uninit was created at:\n");
+ if (entries)
+ stack_trace_print(entries, nr_entries, 0);
+ else
+ kmsan_pr_err("No stack\n");
+ break;
+ }
+}
+
+void kmsan_report(depot_stack_handle_t origin,
+ void *address, int size, int off_first, int off_last,
+ const void *user_addr, int reason)
+{
+ unsigned long flags;
+ unsigned long *entries;
+ unsigned int nr_entries;
+ bool is_uaf = false;
+ char *bug_type = NULL;
+
+ if (!kmsan_ready)
+ return;
+ if (!current->kmsan.allow_reporting)
+ return;
+ if (!origin)
+ return;
+
+ nr_entries = stack_depot_fetch(origin, &entries);
+
+ current->kmsan.allow_reporting = false;
+ spin_lock_irqsave(&report_lock, flags);
+ kmsan_pr_err("=====================================================\n");
+ if (get_dsh_extra_bits(origin) & 1)
+ is_uaf = true;
+ switch (reason) {
+ case REASON_ANY:
+ bug_type = is_uaf ? "use-after-free" : "uninit-value";
+ break;
+ case REASON_COPY_TO_USER:
+ bug_type = is_uaf ? "kernel-infoleak-after-free" :
+ "kernel-infoleak";
+ break;
+ case REASON_SUBMIT_URB:
+ bug_type = is_uaf ? "kernel-usb-infoleak-after-free" :
+ "kernel-usb-infoleak";
+ break;
+ }
+ kmsan_pr_err("BUG: KMSAN: %s in %pS\n",
+ bug_type, kmsan_internal_return_address(2));
+ dump_stack();
+ kmsan_pr_err("\n");
+
+ kmsan_print_origin(origin);
+
+ if (size) {
+ kmsan_pr_err("\n");
+ if (off_first == off_last)
+ kmsan_pr_err("Byte %d of %d is uninitialized\n",
+ off_first, size);
+ else
+ kmsan_pr_err("Bytes %d-%d of %d are uninitialized\n",
+ off_first, off_last, size);
+ }
+ if (address)
+ kmsan_pr_err("Memory access of size %d starts at %px\n",
+ size, address);
+ if (user_addr && reason == REASON_COPY_TO_USER)
+ kmsan_pr_err("Data copied to user address %px\n", user_addr);
+ kmsan_pr_err("=====================================================\n");
+ add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
+ spin_unlock_irqrestore(&report_lock, flags);
+ if (panic_on_warn)
+ panic("panic_on_warn set ...\n");
+ current->kmsan.allow_reporting = true;
+}
diff --git a/mm/kmsan/kmsan_shadow.c b/mm/kmsan/kmsan_shadow.c
new file mode 100644
index 000000000000..06801d76e6b8
--- /dev/null
+++ b/mm/kmsan/kmsan_shadow.c
@@ -0,0 +1,543 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KMSAN shadow implementation.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <asm/cpu_entry_area.h>
+#include <asm/page.h>
+#include <asm/pgtable_64_types.h>
+#include <asm/tlbflush.h>
+#include <linux/memblock.h>
+#include <linux/mm_types.h>
+#include <linux/percpu-defs.h>
+#include <linux/slab.h>
+#include <linux/smp.h>
+#include <linux/stddef.h>
+
+#include "kmsan.h"
+#include "kmsan_shadow.h"
+
+#define shadow_page_for(page) \
+ ((page)->shadow)
+
+#define origin_page_for(page) \
+ ((page)->origin)
+
+#define shadow_ptr_for(page) \
+ (page_address((page)->shadow))
+
+#define origin_ptr_for(page) \
+ (page_address((page)->origin))
+
+#define has_shadow_page(page) \
+ (!!((page)->shadow))
+
+#define has_origin_page(page) \
+ (!!((page)->origin))
+
+#define set_no_shadow_origin_page(page) \
+ do { \
+ (page)->shadow = NULL; \
+ (page)->origin = NULL; \
+ } while (0) /**/
+
+#define is_ignored_page(page) \
+ (!!(((u64)((page)->shadow)) % 2))
+
+#define ignore_page(pg) \
+ ((pg)->shadow = (struct page *)((u64)((pg)->shadow) | 1)) \
+
+DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
+DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
+
+/*
+ * Dummy load and store pages to be used when the real metadata is unavailable.
+ * There are separate pages for loads and stores, so that every load returns a
+ * zero, and every store doesn't affect other stores.
+ */
+char dummy_load_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+char dummy_store_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * Taken from arch/x86/mm/physaddr.h to avoid using an instrumented version.
+ */
+static int kmsan_phys_addr_valid(unsigned long addr)
+{
+#ifdef CONFIG_PHYS_ADDR_T_64BIT
+ return !(addr >> boot_cpu_data.x86_phys_bits);
+#else
+ return 1;
+#endif
+}
+
+/*
+ * Taken from arch/x86/mm/physaddr.c to avoid using an instrumented version.
+ */
+static bool kmsan_virt_addr_valid(void *addr)
+{
+ unsigned long x = (unsigned long)addr;
+ unsigned long y = x - __START_KERNEL_map;
+
+ /* use the carry flag to determine if x was < __START_KERNEL_map */
+ if (unlikely(x > y)) {
+ x = y + phys_base;
+
+ if (y >= KERNEL_IMAGE_SIZE)
+ return false;
+ } else {
+ x = y + (__START_KERNEL_map - PAGE_OFFSET);
+
+ /* carry flag will be set if starting x was >= PAGE_OFFSET */
+ if ((x > y) || !kmsan_phys_addr_valid(x))
+ return false;
+ }
+
+ return pfn_valid(x >> PAGE_SHIFT);
+}
+
+static unsigned long vmalloc_meta(void *addr, bool is_origin)
+{
+ unsigned long addr64 = (unsigned long)addr, off;
+
+ BUG_ON(is_origin && !IS_ALIGNED(addr64, ORIGIN_SIZE));
+ if (kmsan_internal_is_vmalloc_addr(addr)) {
+ return addr64 + (is_origin ? VMALLOC_ORIGIN_OFFSET
+ : VMALLOC_SHADOW_OFFSET);
+ }
+ if (kmsan_internal_is_module_addr(addr)) {
+ off = addr64 - MODULES_VADDR;
+ return off + (is_origin ? MODULES_ORIGIN_START
+ : MODULES_SHADOW_START);
+ }
+ return 0;
+}
+
+static void *get_cea_meta_or_null(void *addr, bool is_origin)
+{
+ int cpu = smp_processor_id();
+ int off;
+ char *metadata_array;
+
+ if (((u64)addr < CPU_ENTRY_AREA_BASE) ||
+ ((u64)addr >= (CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE)))
+ return NULL;
+ off = (char *)addr - (char *)get_cpu_entry_area(cpu);
+ if ((off < 0) || (off >= CPU_ENTRY_AREA_SIZE))
+ return NULL;
+ metadata_array = is_origin ? cpu_entry_area_origin :
+ cpu_entry_area_shadow;
+ return &per_cpu(metadata_array[off], cpu);
+}
+
+static struct page *virt_to_page_or_null(void *vaddr)
+{
+ if (kmsan_virt_addr_valid(vaddr))
+ return virt_to_page(vaddr);
+ else
+ return NULL;
+}
+
+struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *address, u64 size,
+ bool store)
+{
+ struct shadow_origin_ptr ret;
+ struct page *page;
+ u64 pad, offset, o_offset;
+ const u64 addr64 = (u64)address;
+ u64 o_addr64 = (u64)address;
+ void *shadow;
+
+ if (size > PAGE_SIZE) {
+ WARN(1, "size too big in %s(%px, %d, %d)\n",
+ __func__, address, size, store);
+ BUG();
+ }
+ if (store) {
+ ret.s = dummy_store_page;
+ ret.o = dummy_store_page;
+ } else {
+ ret.s = dummy_load_page;
+ ret.o = dummy_load_page;
+ }
+ if (!kmsan_ready || IN_RUNTIME())
+ return ret;
+ BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW));
+
+ if (!IS_ALIGNED(addr64, ORIGIN_SIZE)) {
+ pad = addr64 % ORIGIN_SIZE;
+ o_addr64 -= pad;
+ }
+
+ if (kmsan_internal_is_vmalloc_addr(address) ||
+ kmsan_internal_is_module_addr(address)) {
+ ret.s = (void *)vmalloc_meta(address, META_SHADOW);
+ ret.o = (void *)vmalloc_meta((void *)o_addr64, META_ORIGIN);
+ return ret;
+ }
+
+ if (!kmsan_virt_addr_valid(address)) {
+ page = vmalloc_to_page_or_null(address);
+ if (page)
+ goto next;
+ shadow = get_cea_meta_or_null(address, META_SHADOW);
+ if (shadow) {
+ ret.s = shadow;
+ ret.o = get_cea_meta_or_null((void *)o_addr64,
+ META_ORIGIN);
+ return ret;
+ }
+ }
+ page = virt_to_page_or_null(address);
+ if (!page)
+ return ret;
+next:
+ if (is_ignored_page(page))
+ return ret;
+
+ if (!has_shadow_page(page) || !has_origin_page(page))
+ return ret;
+ offset = addr64 % PAGE_SIZE;
+ o_offset = o_addr64 % PAGE_SIZE;
+
+ if (offset + size - 1 > PAGE_SIZE) {
+ /*
+ * The access overflows the current page and touches the
+ * subsequent ones. Make sure the shadow/origin pages are also
+ * consequent.
+ */
+ BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW));
+ }
+
+ ret.s = shadow_ptr_for(page) + offset;
+ ret.o = origin_ptr_for(page) + o_offset;
+ return ret;
+}
+
+/*
+ * Obtain the shadow or origin pointer for the given address, or NULL if there's
+ * none. The caller must check the return value for being non-NULL if needed.
+ * The return value of this function should not depend on whether we're in the
+ * runtime or not.
+ */
+void *kmsan_get_metadata(void *address, size_t size, bool is_origin)
+{
+ struct page *page;
+ void *ret;
+ u64 addr = (u64)address, pad, off;
+
+ if (is_origin && !IS_ALIGNED(addr, ORIGIN_SIZE)) {
+ pad = addr % ORIGIN_SIZE;
+ addr -= pad;
+ size += pad;
+ }
+ address = (void *)addr;
+ if (kmsan_internal_is_vmalloc_addr(address) ||
+ kmsan_internal_is_module_addr(address)) {
+ return (void *)vmalloc_meta(address, is_origin);
+ }
+
+ if (!kmsan_virt_addr_valid(address)) {
+ page = vmalloc_to_page_or_null(address);
+ if (page)
+ goto next;
+ ret = get_cea_meta_or_null(address, is_origin);
+ if (ret)
+ return ret;
+ }
+ page = virt_to_page_or_null(address);
+ if (!page)
+ return NULL;
+next:
+ if (is_ignored_page(page))
+ return NULL;
+ if (!has_shadow_page(page) || !has_origin_page(page))
+ return NULL;
+ off = addr % PAGE_SIZE;
+
+ ret = (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off;
+ return ret;
+}
+
+void __init kmsan_init_alloc_meta_for_range(void *start, void *end)
+{
+ u64 addr, size;
+ struct page *page;
+ void *shadow, *origin;
+ struct page *shadow_p, *origin_p;
+
+ start = (void *)ALIGN_DOWN((u64)start, PAGE_SIZE);
+ size = ALIGN((u64)end - (u64)start, PAGE_SIZE);
+ shadow = memblock_alloc(size, PAGE_SIZE);
+ origin = memblock_alloc(size, PAGE_SIZE);
+ for (addr = 0; addr < size; addr += PAGE_SIZE) {
+ page = virt_to_page_or_null((char *)start + addr);
+ shadow_p = virt_to_page_or_null((char *)shadow + addr);
+ set_no_shadow_origin_page(shadow_p);
+ shadow_page_for(page) = shadow_p;
+ origin_p = virt_to_page_or_null((char *)origin + addr);
+ set_no_shadow_origin_page(origin_p);
+ origin_page_for(page) = origin_p;
+ }
+}
+
+/* Called from mm/memory.c */
+void kmsan_copy_page_meta(struct page *dst, struct page *src)
+{
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ if (!has_shadow_page(src)) {
+ /* TODO(glider): are we leaking pages here? */
+ set_no_shadow_origin_page(dst);
+ return;
+ }
+ if (!has_shadow_page(dst))
+ return;
+ if (is_ignored_page(src)) {
+ ignore_page(dst);
+ return;
+ }
+
+ ENTER_RUNTIME(irq_flags);
+ __memcpy(shadow_ptr_for(dst), shadow_ptr_for(src),
+ PAGE_SIZE);
+ BUG_ON(!has_origin_page(src) || !has_origin_page(dst));
+ __memcpy(origin_ptr_for(dst), origin_ptr_for(src),
+ PAGE_SIZE);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_copy_page_meta);
+
+/* Helper function to allocate page metadata. */
+static int kmsan_internal_alloc_meta_for_pages(struct page *page,
+ unsigned int order,
+ gfp_t flags, int node)
+{
+ struct page *shadow, *origin;
+ int pages = 1 << order;
+ int i;
+ bool initialized = (flags & __GFP_ZERO) || !kmsan_ready;
+ depot_stack_handle_t handle;
+
+ if (flags & __GFP_NO_KMSAN_SHADOW) {
+ for (i = 0; i < pages; i++)
+ set_no_shadow_origin_page(&page[i]);
+ return 0;
+ }
+
+ /* TODO(glider): must we override the flags? */
+ flags = GFP_ATOMIC;
+ if (initialized)
+ flags |= __GFP_ZERO;
+ shadow = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order);
+ if (!shadow) {
+ for (i = 0; i < pages; i++) {
+ set_no_shadow_origin_page(&page[i]);
+ set_no_shadow_origin_page(&page[i]);
+ }
+ return -ENOMEM;
+ }
+ if (!initialized)
+ __memset(page_address(shadow), -1, PAGE_SIZE * pages);
+
+ origin = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order);
+ /* Assume we've allocated the origin. */
+ if (!origin) {
+ __free_pages(shadow, order);
+ for (i = 0; i < pages; i++)
+ set_no_shadow_origin_page(&page[i]);
+ return -ENOMEM;
+ }
+
+ if (!initialized) {
+ handle = kmsan_save_stack_with_flags(flags, /*extra_bits*/0);
+ /*
+ * Addresses are page-aligned, pages are contiguous, so it's ok
+ * to just fill the origin pages with |handle|.
+ */
+ for (i = 0; i < PAGE_SIZE * pages / sizeof(handle); i++) {
+ ((depot_stack_handle_t *)page_address(origin))[i] =
+ handle;
+ }
+ }
+
+ for (i = 0; i < pages; i++) {
+ shadow_page_for(&page[i]) = &shadow[i];
+ set_no_shadow_origin_page(shadow_page_for(&page[i]));
+ origin_page_for(&page[i]) = &origin[i];
+ set_no_shadow_origin_page(origin_page_for(&page[i]));
+ }
+ return 0;
+}
+
+/* Called from mm/page_alloc.c */
+int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags)
+{
+ unsigned long irq_flags;
+ int ret;
+
+ if (IN_RUNTIME())
+ return 0;
+ ENTER_RUNTIME(irq_flags);
+ ret = kmsan_internal_alloc_meta_for_pages(page, order, flags, -1);
+ LEAVE_RUNTIME(irq_flags);
+ return ret;
+}
+
+/* Called from mm/page_alloc.c */
+void kmsan_free_page(struct page *page, unsigned int order)
+{
+ struct page *shadow, *origin, *cur_page;
+ int pages = 1 << order;
+ int i;
+ unsigned long irq_flags;
+
+ if (!shadow_page_for(page)) {
+ for (i = 0; i < pages; i++) {
+ cur_page = &page[i];
+ BUG_ON(shadow_page_for(cur_page));
+ }
+ return;
+ }
+
+ if (!kmsan_ready) {
+ for (i = 0; i < pages; i++) {
+ cur_page = &page[i];
+ set_no_shadow_origin_page(cur_page);
+ }
+ return;
+ }
+
+ if (IN_RUNTIME()) {
+ /*
+ * TODO(glider): looks legit. depot_save_stack() may call
+ * free_pages().
+ */
+ return;
+ }
+
+ ENTER_RUNTIME(irq_flags);
+ shadow = shadow_page_for(&page[0]);
+ origin = origin_page_for(&page[0]);
+
+ /* TODO(glider): this is racy. */
+ for (i = 0; i < pages; i++) {
+ BUG_ON(has_shadow_page(shadow_page_for(&page[i])));
+ BUG_ON(has_shadow_page(origin_page_for(&page[i])));
+ set_no_shadow_origin_page(&page[i]);
+ }
+ BUG_ON(has_shadow_page(shadow));
+ __free_pages(shadow, order);
+
+ BUG_ON(has_shadow_page(origin));
+ __free_pages(origin, order);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_free_page);
+
+/* Called from mm/page_alloc.c */
+void kmsan_split_page(struct page *page, unsigned int order)
+{
+ struct page *shadow, *origin;
+ unsigned long irq_flags;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+
+ ENTER_RUNTIME(irq_flags);
+ if (!has_shadow_page(&page[0])) {
+ BUG_ON(has_origin_page(&page[0]));
+ LEAVE_RUNTIME(irq_flags);
+ return;
+ }
+ shadow = shadow_page_for(&page[0]);
+ split_page(shadow, order);
+
+ origin = origin_page_for(&page[0]);
+ split_page(origin, order);
+ LEAVE_RUNTIME(irq_flags);
+}
+EXPORT_SYMBOL(kmsan_split_page);
+
+/* Called from include/linux/highmem.h */
+void kmsan_clear_page(void *page_addr)
+{
+ struct page *page;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ BUG_ON(!IS_ALIGNED((u64)page_addr, PAGE_SIZE));
+ page = vmalloc_to_page_or_null(page_addr);
+ if (!page)
+ page = virt_to_page_or_null(page_addr);
+ if (!page || !has_shadow_page(page))
+ return;
+ __memset(shadow_ptr_for(page), 0, PAGE_SIZE);
+ BUG_ON(!has_origin_page(page));
+ __memset(origin_ptr_for(page), 0, PAGE_SIZE);
+}
+EXPORT_SYMBOL(kmsan_clear_page);
+
+/* Called from mm/vmalloc.c */
+void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages)
+{
+ int nr, i, mapped;
+ struct page **s_pages, **o_pages;
+ unsigned long shadow_start, shadow_end, origin_start, origin_end;
+
+ if (!kmsan_ready || IN_RUNTIME())
+ return;
+ shadow_start = vmalloc_meta((void *)start, META_SHADOW);
+ if (!shadow_start)
+ return;
+
+ BUG_ON(start >= end);
+ nr = (end - start) / PAGE_SIZE;
+ s_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL);
+ o_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL);
+ if (!s_pages || !o_pages)
+ goto ret;
+ for (i = 0; i < nr; i++) {
+ s_pages[i] = shadow_page_for(pages[i]);
+ o_pages[i] = origin_page_for(pages[i]);
+ }
+ prot = __pgprot(pgprot_val(prot) | _PAGE_NX);
+ prot = PAGE_KERNEL;
+
+ shadow_end = vmalloc_meta((void *)end, META_SHADOW);
+ origin_start = vmalloc_meta((void *)start, META_ORIGIN);
+ origin_end = vmalloc_meta((void *)end, META_ORIGIN);
+ mapped = __vmap_page_range_noflush(shadow_start, shadow_end,
+ prot, s_pages);
+ BUG_ON(mapped != nr);
+ flush_tlb_kernel_range(shadow_start, shadow_end);
+ mapped = __vmap_page_range_noflush(origin_start, origin_end,
+ prot, o_pages);
+ BUG_ON(mapped != nr);
+ flush_tlb_kernel_range(origin_start, origin_end);
+ret:
+ kfree(s_pages);
+ kfree(o_pages);
+}
+
+void kmsan_ignore_page(struct page *page, int order)
+{
+ int pages = 1 << order;
+ int i;
+ struct page *cp;
+
+ for (i = 0; i < pages; i++) {
+ cp = &page[i];
+ ignore_page(cp);
+ }
+}
diff --git a/mm/kmsan/kmsan_shadow.h b/mm/kmsan/kmsan_shadow.h
new file mode 100644
index 000000000000..eaa7f771b6a5
--- /dev/null
+++ b/mm/kmsan/kmsan_shadow.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * KMSAN shadow API.
+ *
+ * This should be agnostic to shadow implementation details.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __MM_KMSAN_KMSAN_SHADOW_H
+#define __MM_KMSAN_KMSAN_SHADOW_H
+
+#include <asm/cpu_entry_area.h> /* for CPU_ENTRY_AREA_MAP_SIZE */
+
+struct shadow_origin_ptr {
+ void *s, *o;
+};
+
+struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size,
+ bool store);
+void *kmsan_get_metadata(void *addr, size_t size, bool is_origin);
+void __init kmsan_init_alloc_meta_for_range(void *start, void *end);
+
+#endif /* __MM_KMSAN_KMSAN_SHADOW_H */
diff --git a/scripts/Makefile.kmsan b/scripts/Makefile.kmsan
new file mode 100644
index 000000000000..8b3844b66b22
--- /dev/null
+++ b/scripts/Makefile.kmsan
@@ -0,0 +1,12 @@
+ifdef CONFIG_KMSAN
+
+CFLAGS_KMSAN := -fsanitize=kernel-memory
+
+ifeq ($(call cc-option, $(CFLAGS_KMSAN) -Werror),)
+ ifneq ($(CONFIG_COMPILE_TEST),y)
+ $(warning Cannot use CONFIG_KMSAN: \
+ -fsanitize=kernel-memory is not supported by compiler)
+ endif
+endif
+
+endif
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-22 11:25 ` [PATCH RFC v3 10/36] kmsan: add KMSAN runtime glider
@ 2019-11-24 19:44 ` Wolfram Sang
2019-11-25 9:14 ` Alexander Potapenko
2019-11-29 16:07 ` Marco Elver
` (2 subsequent siblings)
3 siblings, 1 reply; 120+ messages in thread
From: Wolfram Sang @ 2019-11-24 19:44 UTC (permalink / raw)
To: glider
Cc: Vegard Nossum, Dmitry Vyukov, linux-mm, viro, adilger.kernel,
akpm, andreyknvl, aryabinin, luto, ard.biesheuvel, arnd, hch,
hch, darrick.wong, davem, dmitry.torokhov, ebiggers, edumazet,
ericvh, gregkh, harry.wentland, herbert, iii, mingo, jasowang,
axboe, m.szyprowski, elver, mark.rutland, martin.petersen,
schwidefsky, willy, mst, monstr, pmladek, cai, rdunlap,
robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso, tglx,
gor
[-- Attachment #1: Type: text/plain, Size: 156 bytes --]
Hi,
> - dropped kmsan_handle_i2c_transfer()
Could you elaborate a little why it was there in the beginning and why
it is gone now?
Thanks,
Wolfram
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-24 19:44 ` Wolfram Sang
@ 2019-11-25 9:14 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-11-25 9:14 UTC (permalink / raw)
To: Wolfram Sang
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, adilger.kernel, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
hch, Christoph Hellwig, darrick.wong, David Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, jasowang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K . Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik
On Sun, Nov 24, 2019 at 8:44 PM Wolfram Sang <wsa@the-dreams.de> wrote:
>
> Hi,
>
> > - dropped kmsan_handle_i2c_transfer()
>
> Could you elaborate a little why it was there in the beginning and why
> it is gone now?
Previous patchset version didn't handle DMA and iomap-based I/O
properly, so there was a bunch of KMSAN calls scattered across the
codebase that were initializing memory at the places where data was
copied from devices.
Now they are all gone, because v3 supports DMA/iomap.
> Thanks,
>
> Wolfram
>
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-22 11:25 ` [PATCH RFC v3 10/36] kmsan: add KMSAN runtime glider
2019-11-24 19:44 ` Wolfram Sang
@ 2019-11-29 16:07 ` Marco Elver
2019-12-19 14:16 ` Alexander Potapenko
2019-12-02 15:39 ` Andrey Konovalov
2019-12-03 14:34 ` Andrey Konovalov
3 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-11-29 16:07 UTC (permalink / raw)
To: glider
Cc: Wolfram Sang, Vegard Nossum, Dmitry Vyukov, linux-mm, viro,
adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, mark.rutland, martin.petersen, schwidefsky, willy,
mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor
On Fri, 22 Nov 2019, glider@google.com wrote:
[...]
> diff --git a/arch/x86/include/asm/kmsan.h b/arch/x86/include/asm/kmsan.h
> new file mode 100644
> index 000000000000..fc5f1224a059
> --- /dev/null
> +++ b/arch/x86/include/asm/kmsan.h
> @@ -0,0 +1,117 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Assembly bits to safely invoke KMSAN hooks from .S files.
> + *
> + * Adopted from KTSAN assembly hooks implementation by Dmitry Vyukov:
> + * https://github.com/google/ktsan/blob/ktsan/arch/x86/include/asm/ktsan.h
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +#ifndef _ASM_X86_KMSAN_H
> +#define _ASM_X86_KMSAN_H
> +
> +#ifdef CONFIG_KMSAN
> +
> +#define KMSAN_PUSH_REGS \
> + pushq %rax; \
> + pushq %rcx; \
> + pushq %rdx; \
> + pushq %rdi; \
> + pushq %rsi; \
> + pushq %r8; \
> + pushq %r9; \
> + pushq %r10; \
> + pushq %r11; \
> +/**/
> +
[...]
> +
> +#define KMSAN_IST_EXIT(shift_ist) \
> + KMSAN_PUSH_REGS \
> + movq $shift_ist, %rdi; \
> + call kmsan_ist_exit; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_UNPOISON_PT_REGS \
> + KMSAN_PUSH_REGS \
> + call kmsan_unpoison_pt_regs; \
> + KMSAN_POP_REGS \
> +/**/
Could all these just be using '.macro .endm'?
> +#else /* ifdef CONFIG_KMSAN */
> +
> +#define KMSAN_INTERRUPT_ENTER
> +#define KMSAN_INTERRUPT_EXIT
> +#define KMSAN_SOFTIRQ_ENTER
> +#define KMSAN_SOFTIRQ_EXIT
> +#define KMSAN_NMI_ENTER
> +#define KMSAN_NMI_EXIT
> +#define KMSAN_SYSCALL_ENTER
> +#define KMSAN_SYSCALL_EXIT
> +#define KMSAN_IST_ENTER(shift_ist)
> +#define KMSAN_IST_EXIT(shift_ist)
> +#define KMSAN_UNPOISON_PT_REGS
> +
> +#endif /* ifdef CONFIG_KMSAN */
> +#endif /* ifndef _ASM_X86_KMSAN_H */
> diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h
> new file mode 100644
> index 000000000000..623854e88d4b
> --- /dev/null
> +++ b/include/linux/kmsan-checks.h
> @@ -0,0 +1,122 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN checks.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#ifndef _LINUX_KMSAN_CHECKS_H
> +#define _LINUX_KMSAN_CHECKS_H
> +
> +#include <linux/bug.h>
> +#include <linux/dma-direction.h>
> +#include <linux/types.h>
> +
> +struct page;
> +struct sk_buff;
> +struct urb;
> +
> +#ifdef CONFIG_KMSAN
> +
> +/*
> + * Helper functions that mark the return value initialized.
> + * Note that Clang ignores the inline attribute in the cases when a no_sanitize
> + * function is called from an instrumented one.
> + */
> +
> +__no_sanitize_memory
> +static inline unsigned char KMSAN_INIT_1(unsigned char value)
> +{
> + return value;
> +}
> +
> +__no_sanitize_memory
> +static inline unsigned short KMSAN_INIT_2(unsigned short value)
> +{
> + return value;
> +}
> +
> +__no_sanitize_memory
> +static inline unsigned int KMSAN_INIT_4(unsigned int value)
> +{
> + return value;
> +}
> +
> +__no_sanitize_memory
> +static inline unsigned long KMSAN_INIT_8(unsigned long value)
> +{
> + return value;
> +}
Should the above be __always_inline?
Does it make sense to use u8, u16, u32, u64 here -- just in case it's
ported to other architectures in future?
> +
> +#define KMSAN_INIT_VALUE(val) \
> + ({ \
> + typeof(val) __ret; \
> + switch (sizeof(val)) { \
> + case 1: \
> + *(unsigned char *)&__ret = KMSAN_INIT_1( \
> + (unsigned char)val); \
> + break; \
> + case 2: \
> + *(unsigned short *)&__ret = KMSAN_INIT_2( \
> + (unsigned short)val); \
> + break; \
> + case 4: \
> + *(unsigned int *)&__ret = KMSAN_INIT_4( \
> + (unsigned int)val); \
> + break; \
> + case 8: \
> + *(unsigned long *)&__ret = KMSAN_INIT_8( \
> + (unsigned long)val); \
> + break; \
> + default: \
> + BUILD_BUG_ON(1); \
> + } \
> + __ret; \
> + }) /**/
Is the /**/ needed?
> +
> +void kmsan_ignore_page(struct page *page, int order);
> +void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags);
> +void kmsan_unpoison_shadow(const void *address, size_t size);
> +void kmsan_check_memory(const void *address, size_t size);
> +void kmsan_check_skb(const struct sk_buff *skb);
> +void kmsan_handle_dma(const void *address, size_t size,
> + enum dma_data_direction direction);
> +void kmsan_handle_urb(const struct urb *urb, bool is_out);
> +void kmsan_copy_to_user(const void *to, const void *from, size_t to_copy,
> + size_t left);
> +void *__msan_memcpy(void *dst, const void *src, u64 n);
> +void kmsan_enter_runtime(unsigned long *flags);
> +void kmsan_leave_runtime(unsigned long *flags);
It would be good to add doc comments to all API functions.
> +
> +#else
> +
> +#define KMSAN_INIT_VALUE(value) (value)
> +
> +static inline void kmsan_ignore_page(struct page *page, int order) {}
> +static inline void kmsan_poison_shadow(const void *address, size_t size,
> + gfp_t flags) {}
> +static inline void kmsan_unpoison_shadow(const void *address, size_t size) {}
> +static inline void kmsan_check_memory(const void *address, size_t size) {}
> +static inline void kmsan_check_skb(const struct sk_buff *skb) {}
> +static inline void kmsan_handle_urb(const struct urb *urb, bool is_out) {}
> +static inline void kmsan_handle_dma(const void *address, size_t size,
> + enum dma_data_direction direction) {}
> +static inline void kmsan_copy_to_user(
> + const void *to, const void *from, size_t to_copy, size_t left) {}
> +static inline void *__msan_memcpy(void *dst, const void *src, size_t n)
> +{
> + return NULL;
> +}
> +
> +static inline void kmsan_enter_runtime(unsigned long *flags) {}
> +static inline void kmsan_leave_runtime(unsigned long *flags) {}
> +
> +#endif
> +
> +#endif /* _LINUX_KMSAN_CHECKS_H */
> diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
> new file mode 100644
> index 000000000000..f5638bac368e
> --- /dev/null
> +++ b/include/linux/kmsan.h
> @@ -0,0 +1,143 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN API for subsystems.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +#ifndef LINUX_KMSAN_H
> +#define LINUX_KMSAN_H
> +
> +#include <linux/gfp.h>
> +#include <linux/stackdepot.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +struct page;
> +struct kmem_cache;
> +struct task_struct;
> +struct vm_struct;
> +
> +
> +extern bool kmsan_ready;
What does this variable mean. Would 'kmsan_enabled' be more accurate?
> +#ifdef CONFIG_KMSAN
> +void __init kmsan_initialize_shadow(void);
> +void __init kmsan_initialize(void);
> +
> +/* These constants are defined in the MSan LLVM instrumentation pass. */
> +#define RETVAL_SIZE 800
This is in include/linux -- do they need a KMSAN_ prefix to not clash
with other definitions?
> +#define KMSAN_PARAM_SIZE 800
> +
> +#define PARAM_ARRAY_SIZE (KMSAN_PARAM_SIZE / sizeof(depot_stack_handle_t))
Similar here -- does it need a KMSAN_ prefix?
> +struct kmsan_context_state {
> + char param_tls[KMSAN_PARAM_SIZE];
> + char retval_tls[RETVAL_SIZE];
> + char va_arg_tls[KMSAN_PARAM_SIZE];
> + char va_arg_origin_tls[KMSAN_PARAM_SIZE];
> + u64 va_arg_overflow_size_tls;
> + depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE];
> + depot_stack_handle_t retval_origin_tls;
> + depot_stack_handle_t origin_tls;
> +};
> +
> +struct kmsan_task_state {
> + bool allow_reporting;
> + struct kmsan_context_state cstate;
> +};
> +
> +void kmsan_task_create(struct task_struct *task);
> +void kmsan_task_exit(struct task_struct *task);
> +void kmsan_alloc_shadow_for_region(void *start, size_t size);
> +int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags);
> +void kmsan_gup_pgd_range(struct page **pages, int nr);
> +void kmsan_free_page(struct page *page, unsigned int order);
> +void kmsan_split_page(struct page *page, unsigned int order);
> +void kmsan_copy_page_meta(struct page *dst, struct page *src);
> +
> +void kmsan_poison_slab(struct page *page, gfp_t flags);
> +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
> +void kmsan_kfree_large(const void *ptr);
> +void kmsan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
> + gfp_t flags);
> +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
> +void kmsan_slab_free(struct kmem_cache *s, void *object);
> +
> +void kmsan_slab_setup_object(struct kmem_cache *s, void *object);
> +void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
> + size_t size, void *object);
> +
> +/* vmap */
> +void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages);
> +void kmsan_vunmap_page_range(unsigned long addr, unsigned long end);
> +
> +/* ioremap */
> +void kmsan_ioremap_page_range(unsigned long addr, unsigned long end,
> + phys_addr_t phys_addr, pgprot_t prot);
> +void kmsan_iounmap_page_range(unsigned long start, unsigned long end);
> +
> +void kmsan_softirq_enter(void);
> +void kmsan_softirq_exit(void);
> +
> +void kmsan_clear_page(void *page_addr);
It would be good to have doc comments for each of them.
> +#else
> +
> +static inline void __init kmsan_initialize_shadow(void) { }
> +static inline void __init kmsan_initialize(void) { }
> +
> +static inline void kmsan_task_create(struct task_struct *task) {}
> +static inline void kmsan_task_exit(struct task_struct *task) {}
> +static inline void kmsan_alloc_shadow_for_region(void *start, size_t size) {}
> +static inline int kmsan_alloc_page(struct page *page, unsigned int order,
> + gfp_t flags)
> +{
> + return 0;
> +}
> +static inline void kmsan_gup_pgd_range(struct page **pages, int nr) {}
> +static inline void kmsan_free_page(struct page *page, unsigned int order) {}
> +static inline void kmsan_split_page(struct page *page, unsigned int order) {}
> +static inline void kmsan_copy_page_meta(struct page *dst, struct page *src) {}
> +
> +static inline void kmsan_poison_slab(struct page *page, gfp_t flags) {}
> +static inline void kmsan_kmalloc_large(const void *ptr, size_t size,
> + gfp_t flags) {}
> +static inline void kmsan_kfree_large(const void *ptr) {}
> +static inline void kmsan_kmalloc(struct kmem_cache *s, const void *object,
> + size_t size, gfp_t flags) {}
> +static inline void kmsan_slab_alloc(struct kmem_cache *s, void *object,
> + gfp_t flags) {}
> +static inline void kmsan_slab_free(struct kmem_cache *s, void *object) {}
> +
> +static inline void kmsan_slab_setup_object(struct kmem_cache *s,
> + void *object) {}
> +static inline void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
> + size_t size, void *object) {}
> +
> +static inline void kmsan_vmap_page_range_noflush(unsigned long start,
> + unsigned long end,
> + pgprot_t prot,
> + struct page **pages) {}
> +static inline void kmsan_vunmap_page_range(unsigned long start,
> + unsigned long end) {}
> +
> +static inline void kmsan_ioremap_page_range(unsigned long start,
> + unsigned long end,
> + phys_addr_t phys_addr,
> + pgprot_t prot) {}
> +static inline void kmsan_iounmap_page_range(unsigned long start,
> + unsigned long end) {}
> +static inline void kmsan_softirq_enter(void) {}
> +static inline void kmsan_softirq_exit(void) {}
> +
> +static inline void kmsan_clear_page(void *page_addr) {}
> +#endif
> +
> +#endif /* LINUX_KMSAN_H */
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 93d97f9b0157..75c36318943d 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -756,6 +756,8 @@ config DEBUG_STACKOVERFLOW
>
> source "lib/Kconfig.kasan"
>
> +source "lib/Kconfig.kmsan"
> +
> endmenu # "Memory Debugging"
>
> config ARCH_HAS_KCOV
> diff --git a/lib/Kconfig.kmsan b/lib/Kconfig.kmsan
> new file mode 100644
> index 000000000000..187dddfcf220
> --- /dev/null
> +++ b/lib/Kconfig.kmsan
> @@ -0,0 +1,22 @@
> +config HAVE_ARCH_KMSAN
> + bool
> +
> +if HAVE_ARCH_KMSAN
> +
> +config KMSAN
> + bool "KMSAN: detector of uninitialized memory use"
> + depends on SLUB && !KASAN
> + select STACKDEPOT
> + help
> + KMSAN is a dynamic detector of uses of uninitialized memory in the
> + kernel. It is based on compiler instrumentation provided by Clang
> + and thus requires Clang 10.0.0+ to build.
> +
> +config TEST_KMSAN
> + tristate "Module for testing KMSAN for bug detection"
> + depends on m && KMSAN
> + help
> + Test module that can trigger various uses of uninitialized memory
> + detectable by KMSAN.
> +
> +endif
> diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile
> new file mode 100644
> index 000000000000..ccf6d2d00a7a
> --- /dev/null
> +++ b/mm/kmsan/Makefile
> @@ -0,0 +1,4 @@
> +obj-y := kmsan.o kmsan_instr.o kmsan_init.o kmsan_entry.o kmsan_hooks.o kmsan_report.o kmsan_shadow.o
> +
> +KMSAN_SANITIZE := n
> +KCOV_INSTRUMENT := n
Does KMSAN work together with UBSAN? In that case may this needs a
UBSAN_SANITIZE := n
> diff --git a/mm/kmsan/kmsan.c b/mm/kmsan/kmsan.c
> new file mode 100644
> index 000000000000..21e97d4b1a99
> --- /dev/null
> +++ b/mm/kmsan/kmsan.c
> @@ -0,0 +1,563 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN runtime library.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <asm/page.h>
> +#include <linux/compiler.h>
> +#include <linux/export.h>
> +#include <linux/highmem.h>
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/kmsan.h>
> +#include <linux/memory.h>
> +#include <linux/mm.h>
> +#include <linux/preempt.h>
> +#include <linux/percpu-defs.h>
> +#include <linux/mm_types.h>
> +#include <linux/slab.h>
> +#include <linux/stackdepot.h>
> +#include <linux/stacktrace.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +#include <linux/mmzone.h>
Why the space above the mmzone.h include?
> +
> +#include "../slab.h"
> +#include "kmsan.h"
> +
> +/*
> + * Some kernel asm() calls mention the non-existing |__force_order| variable
> + * in the asm constraints to preserve the order of accesses to control
> + * registers. KMSAN turns those mentions into actual memory accesses, therefore
> + * the variable is now required to link the kernel.
> + */
> +unsigned long __force_order;
Not sure if this is related, but when compiling with KMSAN I get
ERROR: "__force_order" [drivers/misc/lkdtm/lkdtm.ko] undefined!
with a default config with KMSAN selected.
> +bool kmsan_ready;
> +#define KMSAN_STACK_DEPTH 64
> +#define MAX_CHAIN_DEPTH 7
Should these defines be above the variable definitions?
> +/*
> + * According to Documentation/x86/kernel-stacks, kernel code can run on the
> + * following stacks:
> + * - regular task stack - when executing the task code
> + * - interrupt stack - when handling external hardware interrupts and softirqs
> + * - NMI stack
> + * 0 is for regular interrupts, 1 for softirqs, 2 for NMI.
> + * Because interrupts may nest, trying to use a new context for every new
> + * interrupt.
> + */
> +/* [0] for dummy per-CPU context. */
> +DEFINE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
> + kmsan_percpu_cstate);
> +/* 0 for task context, |i>0| for kmsan_context_state[i]. */
> +DEFINE_PER_CPU(int, kmsan_context_level);
> +DEFINE_PER_CPU(int, kmsan_in_interrupt);
> +DEFINE_PER_CPU(bool, kmsan_in_softirq);
> +DEFINE_PER_CPU(bool, kmsan_in_nmi);
> +DEFINE_PER_CPU(int, kmsan_in_runtime);
> +
> +struct kmsan_context_state *task_kmsan_context_state(void)
> +{
> + int cpu = smp_processor_id();
> + int level = this_cpu_read(kmsan_context_level);
> + struct kmsan_context_state *ret;
> +
> + if (!kmsan_ready || IN_RUNTIME()) {
> + ret = &per_cpu(kmsan_percpu_cstate[0], cpu);
> + __memset(ret, 0, sizeof(struct kmsan_context_state));
> + return ret;
> + }
> +
> + if (!level)
> + ret = ¤t->kmsan.cstate;
> + else
> + ret = &per_cpu(kmsan_percpu_cstate[level], cpu);
> + return ret;
> +}
> +
> +void kmsan_internal_task_create(struct task_struct *task)
> +{
> + struct kmsan_task_state *state = &task->kmsan;
> +
> + __memset(state, 0, sizeof(struct kmsan_task_state));
> + state->allow_reporting = true;
> +}
> +
> +void kmsan_internal_memset_shadow(void *addr, int b, size_t size,
> + bool checked)
> +{
> + void *shadow_start;
> + u64 page_offset, address = (u64)addr;
> + size_t to_fill;
> +
> + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
> + while (size) {
> + page_offset = address % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, (u64)size);
> + shadow_start = kmsan_get_metadata((void *)address, to_fill,
> + META_SHADOW);
> + if (!shadow_start) {
> + if (checked) {
> + kmsan_pr_locked("WARNING: not memsetting %d bytes starting at %px, because the shadow is NULL\n", to_fill, address);
> + BUG();
Why not just 'panic("%s: ...", __func__, ...)' ?
If the BUG() should not be here, then maybe just WARN_ON?
> + }
> + /* Otherwise just move on. */
> + } else {
> + __memset(shadow_start, b, to_fill);
> + }
> + address += to_fill;
> + size -= to_fill;
> + }
> +}
> +
> +void kmsan_internal_poison_shadow(void *address, size_t size,
> + gfp_t flags, unsigned int poison_flags)
> +{
> + bool checked = poison_flags & KMSAN_POISON_CHECK;
> + depot_stack_handle_t handle;
> + u32 extra_bits = 0;
> +
> + if (poison_flags & KMSAN_POISON_FREE)
> + extra_bits = 1;
> + kmsan_internal_memset_shadow(address, -1, size, checked);
> + handle = kmsan_save_stack_with_flags(flags, extra_bits);
> + kmsan_set_origin_checked(address, size, handle, checked);
> +}
> +
> +void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked)
> +{
> + kmsan_internal_memset_shadow(address, 0, size, checked);
> + kmsan_set_origin_checked(address, size, 0, checked);
> +}
> +
> +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags,
> + unsigned int reserved)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[KMSAN_STACK_DEPTH];
> + unsigned int nr_entries;
> +
> + nr_entries = stack_trace_save(entries, KMSAN_STACK_DEPTH, 0);
> + filter_irq_stacks(entries, nr_entries);
> +
> + /* Don't sleep (see might_sleep_if() in __alloc_pages_nodemask()). */
> + flags &= ~__GFP_DIRECT_RECLAIM;
> +
> + handle = stack_depot_save(entries, nr_entries, flags);
> + return set_dsh_extra_bits(handle, reserved);
> +}
> +
> +/*
> + * Depending on the value of is_memmove, this serves as both a memcpy and a
> + * memmove implementation.
> + *
> + * As with the regular memmove, do the following:
> + * - if src and dst don't overlap, use memcpy();
> + * - if src and dst overlap:
> + * - if src > dst, use memcpy();
> + * - if src < dst, use reverse-memcpy.
> + * Why this is correct:
> + * - problems may arise if for some part of the overlapping region we
> + * overwrite its shadow with a new value before copying it somewhere.
> + * But there's a 1:1 mapping between the kernel memory and its shadow,
> + * therefore if this doesn't happen with the kernel memory it can't happen
> + * with the shadow.
> + */
> +void kmsan_memcpy_memmove_metadata(void *dst, void *src, size_t n,
> + bool is_memmove)
> +{
> + void *shadow_src, *shadow_dst;
> + depot_stack_handle_t *origin_src, *origin_dst;
> + int src_slots, dst_slots, i, iter, step, skip_bits;
> + depot_stack_handle_t old_origin = 0, chain_origin, new_origin = 0;
> + u32 *align_shadow_src, shadow;
> + bool backwards;
> +
> + BUG_ON(!metadata_is_contiguous(dst, n, META_SHADOW));
> + BUG_ON(!metadata_is_contiguous(src, n, META_SHADOW));
> +
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
> + if (!shadow_dst)
> + return;
> +
> + shadow_src = kmsan_get_metadata(src, n, META_SHADOW);
> + if (!shadow_src) {
> + /*
> + * |src| is untracked: zero out destination shadow, ignore the
> + * origins, we're done.
> + */
> + __memset(shadow_dst, 0, n);
> + return;
> + }
> + if (is_memmove)
> + __memmove(shadow_dst, shadow_src, n);
> + else
> + __memcpy(shadow_dst, shadow_src, n);
> +
> + origin_dst = kmsan_get_metadata(dst, n, META_ORIGIN);
> + origin_src = kmsan_get_metadata(src, n, META_ORIGIN);
> + BUG_ON(!origin_dst || !origin_src);
> + BUG_ON(!metadata_is_contiguous(dst, n, META_ORIGIN));
> + BUG_ON(!metadata_is_contiguous(src, n, META_ORIGIN));
> + src_slots = (ALIGN((u64)src + n, ORIGIN_SIZE) -
> + ALIGN_DOWN((u64)src, ORIGIN_SIZE)) / ORIGIN_SIZE;
> + dst_slots = (ALIGN((u64)dst + n, ORIGIN_SIZE) -
> + ALIGN_DOWN((u64)dst, ORIGIN_SIZE)) / ORIGIN_SIZE;
> + BUG_ON(!src_slots || !dst_slots);
> + BUG_ON((src_slots < 1) || (dst_slots < 1));
> + BUG_ON((src_slots - dst_slots > 1) || (dst_slots - src_slots < -1));
> +
> + backwards = is_memmove && (dst > src);
> + i = backwards ? min(src_slots, dst_slots) - 1 : 0;
> + iter = backwards ? -1 : 1;
> +
> + align_shadow_src = (u32 *)ALIGN_DOWN((u64)shadow_src, ORIGIN_SIZE);
> + for (step = 0; step < min(src_slots, dst_slots); step++, i += iter) {
> + BUG_ON(i < 0);
> + shadow = align_shadow_src[i];
> + if (i == 0) {
> + /*
> + * If |src| isn't aligned on ORIGIN_SIZE, don't
> + * look at the first |src % ORIGIN_SIZE| bytes
> + * of the first shadow slot.
> + */
> + skip_bits = ((u64)src % ORIGIN_SIZE) * 8;
> + shadow = (shadow << skip_bits) >> skip_bits;
> + }
> + if (i == src_slots - 1) {
> + /*
> + * If |src + n| isn't aligned on
> + * ORIGIN_SIZE, don't look at the last
> + * |(src + n) % ORIGIN_SIZE| bytes of the
> + * last shadow slot.
> + */
> + skip_bits = (((u64)src + n) % ORIGIN_SIZE) * 8;
> + shadow = (shadow >> skip_bits) << skip_bits;
> + }
> + /*
> + * Overwrite the origin only if the corresponding
> + * shadow is nonempty.
> + */
> + if (origin_src[i] && (origin_src[i] != old_origin) && shadow) {
> + old_origin = origin_src[i];
> + chain_origin = kmsan_internal_chain_origin(old_origin);
> + /*
> + * kmsan_internal_chain_origin() may return
> + * NULL, but we don't want to lose the previous
> + * origin value.
> + */
> + if (chain_origin)
> + new_origin = chain_origin;
> + else
> + new_origin = old_origin;
> + }
> + if (shadow)
> + origin_dst[i] = new_origin;
> + else
> + origin_dst[i] = 0;
> + }
> +}
> +
> +void kmsan_memcpy_metadata(void *dst, void *src, size_t n)
> +{
> + kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/false);
> +}
> +
> +void kmsan_memmove_metadata(void *dst, void *src, size_t n)
> +{
> + kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/true);
> +}
> +
> +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[3];
> + u64 magic = KMSAN_CHAIN_MAGIC_ORIGIN_FULL;
> + int depth = 0;
> + static int skipped;
> + u32 extra_bits;
> +
> + if (!kmsan_ready)
> + return 0;
> +
> + if (!id)
> + return id;
> + /*
> + * Make sure we have enough spare bits in |id| to hold the UAF bit and
> + * the chain depth.
> + */
> + BUILD_BUG_ON((1 << STACK_DEPOT_EXTRA_BITS) <= (MAX_CHAIN_DEPTH << 1));
> +
> + extra_bits = get_dsh_extra_bits(id);
> +
> + depth = extra_bits >> 1;
> + if (depth >= MAX_CHAIN_DEPTH) {
> + skipped++;
> + if (skipped % 10000 == 0) {
> + kmsan_pr_locked("not chained %d origins\n", skipped);
> + dump_stack();
> + kmsan_print_origin(id);
> + }
> + return id;
> + }
> + depth++;
> + /* Lowest bit is the UAF flag, higher bits hold the depth. */
> + extra_bits = (depth << 1) | (extra_bits & 1);
> + /* TODO(glider): how do we figure out we've dropped some frames? */
> + entries[0] = magic + depth;
> + entries[1] = kmsan_save_stack_with_flags(GFP_ATOMIC, extra_bits);
> + entries[2] = id;
> + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
> + return set_dsh_extra_bits(handle, extra_bits);
> +}
> +
> +void kmsan_write_aligned_origin(void *var, size_t size, u32 origin)
> +{
> + u32 *var_cast = (u32 *)var;
> + int i;
> +
> + BUG_ON((u64)var_cast % ORIGIN_SIZE);
> + BUG_ON(size % ORIGIN_SIZE);
> + for (i = 0; i < size / ORIGIN_SIZE; i++)
> + var_cast[i] = origin;
> +}
> +
> +/*
> + * TODO(glider): writing an initialized byte shouldn't zero out the origin, if
> + * the remaining three bytes are uninitialized.
> + */
What needs to be done to address the TODO? Just adding a comment is
fine (or if the TODO can be resolved that's also fine).
> +void kmsan_internal_set_origin(void *addr, int size, u32 origin)
> +{
> + void *origin_start;
> + u64 address = (u64)addr, page_offset;
> + size_t to_fill, pad = 0;
> +
> + if (!IS_ALIGNED(address, ORIGIN_SIZE)) {
> + pad = address % ORIGIN_SIZE;
> + address -= pad;
> + size += pad;
> + }
> +
> + while (size > 0) {
> + page_offset = address % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, (u64)size);
> + /* write at least ORIGIN_SIZE bytes */
> + to_fill = ALIGN(to_fill, ORIGIN_SIZE);
> + BUG_ON(!to_fill);
> + origin_start = kmsan_get_metadata((void *)address, to_fill,
> + META_ORIGIN);
> + address += to_fill;
> + size -= to_fill;
> + if (!origin_start)
> + /* Can happen e.g. if the memory is untracked. */
> + continue;
> + kmsan_write_aligned_origin(origin_start, to_fill, origin);
> + }
> +}
> +
> +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked)
> +{
> + if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) {
> + kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr);
> + BUG();
Just panic?
> + }
> + kmsan_internal_set_origin(addr, size, origin);
> +}
> +
> +struct page *vmalloc_to_page_or_null(void *vaddr)
> +{
> + struct page *page;
> +
> + if (!kmsan_internal_is_vmalloc_addr(vaddr) &&
> + !kmsan_internal_is_module_addr(vaddr))
> + return NULL;
> + page = vmalloc_to_page(vaddr);
> + if (pfn_valid(page_to_pfn(page)))
> + return page;
> + else
> + return NULL;
> +}
> +
> +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr,
> + int reason)
> +{
> + unsigned long irq_flags;
> + unsigned long addr64 = (unsigned long)addr;
> + unsigned char *shadow = NULL;
> + depot_stack_handle_t *origin = NULL;
> + depot_stack_handle_t cur_origin = 0, new_origin = 0;
> + int cur_off_start = -1;
> + int i, chunk_size;
> + size_t pos = 0;
> +
> + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
> + if (size <= 0)
> + return;
> + while (pos < size) {
> + chunk_size = min(size - pos,
> + PAGE_SIZE - ((addr64 + pos) % PAGE_SIZE));
> + shadow = kmsan_get_metadata((void *)(addr64 + pos), chunk_size,
> + META_SHADOW);
> + if (!shadow) {
> + /*
> + * This page is untracked. If there were uninitialized
> + * bytes before, report them.
> + */
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size,
> + cur_off_start, pos - 1, user_addr,
> + reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> + cur_origin = 0;
> + cur_off_start = -1;
> + pos += chunk_size;
> + continue;
> + }
> + for (i = 0; i < chunk_size; i++) {
> + if (!shadow[i]) {
> + /*
> + * This byte is unpoisoned. If there were
> + * poisoned bytes before, report them.
> + */
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size,
> + cur_off_start, pos + i - 1,
> + user_addr, reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> + cur_origin = 0;
> + cur_off_start = -1;
> + continue;
> + }
> + origin = kmsan_get_metadata((void *)(addr64 + pos + i),
> + chunk_size - i, META_ORIGIN);
> + BUG_ON(!origin);
> + new_origin = *origin;
> + /*
> + * Encountered new origin - report the previous
> + * uninitialized range.
> + */
> + if (cur_origin != new_origin) {
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size,
> + cur_off_start, pos + i - 1,
> + user_addr, reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> + cur_origin = new_origin;
> + cur_off_start = pos + i;
> + }
> + }
> + pos += chunk_size;
> + }
> + BUG_ON(pos != size);
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size, cur_off_start, pos - 1,
> + user_addr, reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> +}
> +
> +/*
> + * TODO(glider): this check shouldn't be performed for origin pages, because
> + * they're always accessed after the shadow pages.
> + */
What needs to be done to address the TODO? Just adding a comment is
fine (or if the TODO can be resolved that's also fine).
> +bool metadata_is_contiguous(void *addr, size_t size, bool is_origin)
> +{
> + u64 cur_addr = (u64)addr, next_addr;
> + char *cur_meta = NULL, *next_meta = NULL;
> + depot_stack_handle_t *origin_p;
> + bool all_untracked = false;
> + const char *fname = is_origin ? "origin" : "shadow";
> +
> + if (!size)
> + return true;
> +
> + /* The whole range belongs to the same page. */
> + if (ALIGN_DOWN(cur_addr + size - 1, PAGE_SIZE) ==
> + ALIGN_DOWN(cur_addr, PAGE_SIZE))
> + return true;
> + cur_meta = kmsan_get_metadata((void *)cur_addr, 1, is_origin);
> + if (!cur_meta)
> + all_untracked = true;
> + for (next_addr = cur_addr + PAGE_SIZE; next_addr < (u64)addr + size;
> + cur_addr = next_addr,
> + cur_meta = next_meta,
> + next_addr += PAGE_SIZE) {
> + next_meta = kmsan_get_metadata((void *)next_addr, 1, is_origin);
> + if (!next_meta) {
> + if (!all_untracked)
> + goto report;
> + continue;
> + }
> + if ((u64)cur_meta == ((u64)next_meta - PAGE_SIZE))
> + continue;
> + goto report;
> + }
> + return true;
> +
> +report:
> + kmsan_pr_locked("BUG: attempting to access two shadow page ranges.\n");
> + dump_stack();
> + kmsan_pr_locked("\n");
> + kmsan_pr_locked("Access of size %d at %px.\n", size, addr);
> + kmsan_pr_locked("Addresses belonging to different ranges: %px and %px\n",
> + cur_addr, next_addr);
> + kmsan_pr_locked("page[0].%s: %px, page[1].%s: %px\n",
> + fname, cur_meta, fname, next_meta);
> + origin_p = kmsan_get_metadata(addr, 1, META_ORIGIN);
> + if (origin_p) {
> + kmsan_pr_locked("Origin: %08x\n", *origin_p);
> + kmsan_print_origin(*origin_p);
> + } else {
> + kmsan_pr_locked("Origin: unavailable\n");
> + }
These repeated calls to kmsan_pr_locked seem unnecessary. There is
nothing ensuring atomicity of all these print calls w.r.t. reporting.
> + return false;
> +}
> +
> +/*
> + * Dummy replacement for __builtin_return_address() which may crash without
> + * frame pointers.
> + */
> +void *kmsan_internal_return_address(int arg)
> +{
> +#ifdef CONFIG_UNWINDER_FRAME_POINTER
> + switch (arg) {
> + case 1:
> + return __builtin_return_address(1);
> + case 2:
> + return __builtin_return_address(2);
> + default:
> + BUG();
> + }
> +#else
> + unsigned long entries[1];
> +
> + stack_trace_save(entries, 1, arg);
> + return (void *)entries[0];
> +#endif
> +}
> +
> +bool kmsan_internal_is_module_addr(void *vaddr)
> +{
> + return ((u64)vaddr >= MODULES_VADDR) && ((u64)vaddr < MODULES_END);
> +}
> +
> +bool kmsan_internal_is_vmalloc_addr(void *addr)
> +{
> + return ((u64)addr >= VMALLOC_START) && ((u64)addr < VMALLOC_END);
> +}
> diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h
> new file mode 100644
> index 000000000000..8760feef39bf
> --- /dev/null
> +++ b/mm/kmsan/kmsan.h
> @@ -0,0 +1,146 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN internal declarations.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#ifndef __MM_KMSAN_KMSAN_H
> +#define __MM_KMSAN_KMSAN_H
> +
> +#include <asm/pgtable_64_types.h>
> +#include <linux/irqflags.h>
> +#include <linux/sched.h>
> +#include <linux/stackdepot.h>
> +#include <linux/stacktrace.h>
> +#include <linux/nmi.h>
> +#include <linux/mm.h>
> +#include <linux/printk.h>
> +
> +#include "kmsan_shadow.h"
> +
> +#define KMSAN_MAGIC_MASK 0xffffffffff00
> +#define KMSAN_ALLOCA_MAGIC_ORIGIN 0x4110c4071900
> +#define KMSAN_CHAIN_MAGIC_ORIGIN_FULL 0xd419170cba00
> +
> +#define KMSAN_POISON_NOCHECK 0x0
> +#define KMSAN_POISON_CHECK 0x1
> +#define KMSAN_POISON_FREE 0x2
> +
> +#define ORIGIN_SIZE 4
> +
> +#define META_SHADOW (false)
> +#define META_ORIGIN (true)
> +
> +#define KMSAN_NESTED_CONTEXT_MAX (8)
> +/* [0] for dummy per-CPU context */
> +DECLARE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
> + kmsan_percpu_cstate);
> +/* 0 for task context, |i>0| for kmsan_context_state[i]. */
> +DECLARE_PER_CPU(int, kmsan_context_level);
> +DECLARE_PER_CPU(int, kmsan_in_interrupt);
> +DECLARE_PER_CPU(bool, kmsan_in_softirq);
> +DECLARE_PER_CPU(bool, kmsan_in_nmi);
> +
> +extern spinlock_t report_lock;
> +
> +/* Stolen from kernel/printk/internal.h */
> +#define PRINTK_SAFE_CONTEXT_MASK 0x3fffffff
Is this used anywhere?
> +/* Called by kmsan_report.c under a lock. */
> +#define kmsan_pr_err(...) pr_err(__VA_ARGS__)
Why is this macro needed? It's never redefined, so in the places it is
used, you can just use pr_err. For readability I would avoid unnecessary
aliases, but if there is a genuine reason this may be needed in future,
I would just add a comment.
> +/* Used in other places - doesn't require a lock. */
> +#define kmsan_pr_locked(...) \
> + do { \
> + unsigned long flags; \
> + spin_lock_irqsave(&report_lock, flags); \
> + pr_err(__VA_ARGS__); \
> + spin_unlock_irqrestore(&report_lock, flags); \
> + } while (0)
Is this macro needed? The only reason it sort of makes sense is to
serialize a report with other printing, but otherwise pr_err already
makes sure things are serialized properly.
> +void kmsan_print_origin(depot_stack_handle_t origin);
> +void kmsan_report(depot_stack_handle_t origin,
> + void *address, int size, int off_first, int off_last,
> + const void *user_addr, int reason);
> +
> +
> +enum KMSAN_BUG_REASON {
> + REASON_ANY = 0,
> + REASON_COPY_TO_USER = 1,
> + REASON_USE_AFTER_FREE = 2,
> + REASON_SUBMIT_URB = 3,
> +};
Is it required to explicitly assign constants to these?
> +
> +/*
> + * When a compiler hook is invoked, it may make a call to instrumented code
> + * and eventually call itself recursively. To avoid that, we protect the
> + * runtime entry points with ENTER_RUNTIME()/LEAVE_RUNTIME() macros and exit
> + * the hook if IN_RUNTIME() is true. But when an interrupt occurs inside the
> + * runtime, the hooks won’t run either, which may lead to errors.
> + * Therefore we have to disable interrupts inside the runtime.
> + */
> +DECLARE_PER_CPU(int, kmsan_in_runtime);
> +#define IN_RUNTIME() (this_cpu_read(kmsan_in_runtime))
> +#define ENTER_RUNTIME(irq_flags) \
> + do { \
> + preempt_disable(); \
> + local_irq_save(irq_flags); \
> + stop_nmi(); \
> + this_cpu_inc(kmsan_in_runtime); \
> + BUG_ON(this_cpu_read(kmsan_in_runtime) > 1); \
> + } while (0)
> +#define LEAVE_RUNTIME(irq_flags) \
> + do { \
> + this_cpu_dec(kmsan_in_runtime); \
> + if (this_cpu_read(kmsan_in_runtime)) { \
> + kmsan_pr_err("kmsan_in_runtime: %d\n", \
> + this_cpu_read(kmsan_in_runtime)); \
> + BUG(); \
> + } \
> + restart_nmi(); \
> + local_irq_restore(irq_flags); \
> + preempt_enable(); } while (0)
Could these not be macros, and instead be static __always_inline
functions?
> +
> +void kmsan_memcpy_metadata(void *dst, void *src, size_t n);
> +void kmsan_memmove_metadata(void *dst, void *src, size_t n);
> +
> +depot_stack_handle_t kmsan_save_stack(void);
> +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags,
> + unsigned int extra_bits);
> +void kmsan_internal_poison_shadow(void *address, size_t size, gfp_t flags,
> + unsigned int poison_flags);
> +void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked);
> +void kmsan_internal_memset_shadow(void *address, int b, size_t size,
> + bool checked);
> +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id);
> +void kmsan_write_aligned_origin(void *var, size_t size, u32 origin);
> +
> +void kmsan_internal_task_create(struct task_struct *task);
> +void kmsan_internal_set_origin(void *addr, int size, u32 origin);
> +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked);
> +
> +struct kmsan_context_state *task_kmsan_context_state(void);
> +
> +bool metadata_is_contiguous(void *addr, size_t size, bool is_origin);
> +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr,
> + int reason);
> +
> +struct page *vmalloc_to_page_or_null(void *vaddr);
> +
> +/* Declared in mm/vmalloc.c */
> +void __vunmap_page_range(unsigned long addr, unsigned long end);
> +int __vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages);
> +
> +void *kmsan_internal_return_address(int arg);
> +bool kmsan_internal_is_module_addr(void *vaddr);
> +bool kmsan_internal_is_vmalloc_addr(void *addr);
> +
> +#endif /* __MM_KMSAN_KMSAN_H */
> diff --git a/mm/kmsan/kmsan_entry.c b/mm/kmsan/kmsan_entry.c
> new file mode 100644
> index 000000000000..47bc7736f1a9
> --- /dev/null
> +++ b/mm/kmsan/kmsan_entry.c
> @@ -0,0 +1,118 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN hooks for entry_64.S
> + *
> + * Copyright (C) 2018-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include "kmsan.h"
> +
> +static void kmsan_context_enter(void)
> +{
> + int level = this_cpu_read(kmsan_context_level) + 1;
> +
> + BUG_ON(level >= KMSAN_NESTED_CONTEXT_MAX);
> + this_cpu_write(kmsan_context_level, level);
> +}
> +
> +static void kmsan_context_exit(void)
> +{
> + int level = this_cpu_read(kmsan_context_level) - 1;
> +
> + BUG_ON(level < 0);
> + this_cpu_write(kmsan_context_level, level);
> +}
These are not preemption-safe. this_cpu_dec_return followed by the
BUG_ON should be sufficient. Similarly above and below (using
this_cpu_add_return)
> +void kmsan_interrupt_enter(void)
> +{
> + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> +
> + /* Turns out it's possible for in_interrupt to be >0 here. */
> + kmsan_context_enter();
> + BUG_ON(in_interrupt > 1);
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_interrupt, in_interrupt + 1);
> +}
> +EXPORT_SYMBOL(kmsan_interrupt_enter);
> +
> +void kmsan_interrupt_exit(void)
> +{
> + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> +
> + BUG_ON(!in_interrupt);
> + kmsan_context_exit();
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_interrupt, in_interrupt - 1);
> +}
> +EXPORT_SYMBOL(kmsan_interrupt_exit);
Why exactly does kmsan_in_interrupt need to be maintained here? I can't
see them being used anywhere else. Is it only for the BUG_ON?
> +void kmsan_softirq_enter(void)
> +{
> + bool in_softirq = this_cpu_read(kmsan_in_softirq);
> +
> + BUG_ON(in_softirq);
> + kmsan_context_enter();
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_softirq, true);
> +}
> +EXPORT_SYMBOL(kmsan_softirq_enter);
> +
> +void kmsan_softirq_exit(void)
> +{
> + bool in_softirq = this_cpu_read(kmsan_in_softirq);
> +
> + BUG_ON(!in_softirq);
> + kmsan_context_exit();
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_softirq, false);
> +}
> +EXPORT_SYMBOL(kmsan_softirq_exit);
Same question here for kmsan_in_softirq.
> +void kmsan_nmi_enter(void)
> +{
> + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> +
> + BUG_ON(in_nmi);
> + BUG_ON(preempt_count() & NMI_MASK);
> + kmsan_context_enter();
> + this_cpu_write(kmsan_in_nmi, true);
> +}
> +EXPORT_SYMBOL(kmsan_nmi_enter);
> +
> +void kmsan_nmi_exit(void)
> +{
> + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> +
> + BUG_ON(!in_nmi);
> + BUG_ON(preempt_count() & NMI_MASK);
> + kmsan_context_exit();
> + this_cpu_write(kmsan_in_nmi, false);
> +
> +}
> +EXPORT_SYMBOL(kmsan_nmi_exit);
And same question here for kmsan_in_nmi.
> +void kmsan_ist_enter(u64 shift_ist)
> +{
> + kmsan_context_enter();
> +}
> +EXPORT_SYMBOL(kmsan_ist_enter);
> +
> +void kmsan_ist_exit(u64 shift_ist)
> +{
> + kmsan_context_exit();
> +}
> +EXPORT_SYMBOL(kmsan_ist_exit);
> +
> +void kmsan_unpoison_pt_regs(struct pt_regs *regs)
> +{
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + kmsan_internal_unpoison_shadow(regs, sizeof(*regs), /*checked*/true);
> +}
> +EXPORT_SYMBOL(kmsan_unpoison_pt_regs);
> diff --git a/mm/kmsan/kmsan_hooks.c b/mm/kmsan/kmsan_hooks.c
> new file mode 100644
> index 000000000000..13a6ed809d81
> --- /dev/null
> +++ b/mm/kmsan/kmsan_hooks.c
> @@ -0,0 +1,422 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN hooks for kernel subsystems.
> + *
> + * These functions handle creation of KMSAN metadata for memory allocations.
> + *
> + * Copyright (C) 2018-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <asm/cacheflush.h>
> +#include <linux/dma-direction.h>
> +#include <linux/gfp.h>
> +#include <linux/mm.h>
> +#include <linux/mm_types.h>
> +#include <linux/skbuff.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include "../slab.h"
> +#include "kmsan.h"
> +
> +/*
> + * The functions may call back to instrumented code, which, in turn, may call
> + * these hooks again. To avoid re-entrancy, we use __GFP_NO_KMSAN_SHADOW.
> + * Instrumented functions shouldn't be called under
> + * ENTER_RUNTIME()/LEAVE_RUNTIME(), because this will lead to skipping
> + * effects of functions like memset() inside instrumented code.
> + */
> +/* Called from kernel/kthread.c, kernel/fork.c */
> +void kmsan_task_create(struct task_struct *task)
> +{
> + unsigned long irq_flags;
> +
> + if (!task)
> + return;
> + ENTER_RUNTIME(irq_flags);
> + kmsan_internal_task_create(task);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_task_create);
> +
> +
> +/* Called from kernel/exit.c */
> +void kmsan_task_exit(struct task_struct *task)
> +{
> + unsigned long irq_flags;
> + struct kmsan_task_state *state = &task->kmsan;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + state->allow_reporting = false;
> +
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_task_exit);
> +
> +/* Called from mm/slub.c */
> +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (unlikely(object == NULL))
> + return;
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * There's a ctor or this is an RCU cache - do nothing. The memory
> + * status hasn't changed since last use.
> + */
> + if (s->ctor || (s->flags & SLAB_TYPESAFE_BY_RCU))
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + if (flags & __GFP_ZERO) {
> + kmsan_internal_unpoison_shadow(object, s->object_size,
> + KMSAN_POISON_CHECK);
> + } else {
> + kmsan_internal_poison_shadow(object, s->object_size, flags,
> + KMSAN_POISON_CHECK);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_slab_alloc);
> +
> +/* Called from mm/slub.c */
> +void kmsan_slab_free(struct kmem_cache *s, void *object)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> +
> + /* RCU slabs could be legally used after free within the RCU period */
> + if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
> + goto leave;
> + if (s->ctor)
> + goto leave;
> + kmsan_internal_poison_shadow(object, s->object_size,
> + GFP_KERNEL,
> + KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
> +leave:
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_slab_free);
> +
> +/* Called from mm/slub.c */
> +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (unlikely(ptr == NULL))
> + return;
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + if (flags & __GFP_ZERO) {
> + kmsan_internal_unpoison_shadow((void *)ptr, size,
> + /*checked*/true);
> + } else {
> + kmsan_internal_poison_shadow((void *)ptr, size, flags,
> + KMSAN_POISON_CHECK);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_kmalloc_large);
> +
> +/* Called from mm/slub.c */
> +void kmsan_kfree_large(const void *ptr)
> +{
> + struct page *page;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + page = virt_to_head_page((void *)ptr);
> + BUG_ON(ptr != page_address(page));
> + kmsan_internal_poison_shadow(
> + (void *)ptr, PAGE_SIZE << compound_order(page), GFP_KERNEL,
> + KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_kfree_large);
> +
> +static unsigned long vmalloc_shadow(unsigned long addr)
> +{
> + return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_SHADOW);
> +}
> +
> +static unsigned long vmalloc_origin(unsigned long addr)
> +{
> + return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_ORIGIN);
> +}
> +
> +/* Called from mm/vmalloc.c */
> +void kmsan_vunmap_page_range(unsigned long start, unsigned long end)
> +{
> + __vunmap_page_range(vmalloc_shadow(start), vmalloc_shadow(end));
> + __vunmap_page_range(vmalloc_origin(start), vmalloc_origin(end));
> +}
> +EXPORT_SYMBOL(kmsan_vunmap_page_range);
> +
> +/* Called from lib/ioremap.c */
> +/*
> + * This function creates new shadow/origin pages for the physical pages mapped
> + * into the virtual memory. If those physical pages already had shadow/origin,
> + * those are ignored.
> + */
> +void kmsan_ioremap_page_range(unsigned long start, unsigned long end,
> + phys_addr_t phys_addr, pgprot_t prot)
> +{
> + unsigned long irq_flags;
> + struct page *shadow, *origin;
> + int i, nr;
> + unsigned long off = 0;
> + gfp_t gfp_mask = GFP_KERNEL | __GFP_ZERO | __GFP_NO_KMSAN_SHADOW;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + nr = (end - start) / PAGE_SIZE;
> + ENTER_RUNTIME(irq_flags);
> + for (i = 0; i < nr; i++, off += PAGE_SIZE) {
> + shadow = alloc_pages(gfp_mask, 1);
> + origin = alloc_pages(gfp_mask, 1);
> + __vmap_page_range_noflush(vmalloc_shadow(start + off),
> + vmalloc_shadow(start + off + PAGE_SIZE),
> + prot, &shadow);
> + __vmap_page_range_noflush(vmalloc_origin(start + off),
> + vmalloc_origin(start + off + PAGE_SIZE),
> + prot, &origin);
> + }
> + flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
> + flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_ioremap_page_range);
> +
> +void kmsan_iounmap_page_range(unsigned long start, unsigned long end)
> +{
> + int i, nr;
> + struct page *shadow, *origin;
> + unsigned long v_shadow, v_origin;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + nr = (end - start) / PAGE_SIZE;
> + ENTER_RUNTIME(irq_flags);
> + v_shadow = (unsigned long)vmalloc_shadow(start);
> + v_origin = (unsigned long)vmalloc_origin(start);
> + for (i = 0; i < nr; i++, v_shadow += PAGE_SIZE, v_origin += PAGE_SIZE) {
> + shadow = vmalloc_to_page_or_null((void *)v_shadow);
> + origin = vmalloc_to_page_or_null((void *)v_origin);
> + __vunmap_page_range(v_shadow, v_shadow + PAGE_SIZE);
> + __vunmap_page_range(v_origin, v_origin + PAGE_SIZE);
> + if (shadow)
> + __free_pages(shadow, 1);
> + if (origin)
> + __free_pages(origin, 1);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_iounmap_page_range);
> +
> +/* Called from include/linux/uaccess.h, include/linux/uaccess.h */
> +void kmsan_copy_to_user(const void *to, const void *from,
> + size_t to_copy, size_t left)
> +{
> + void *shadow;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * At this point we've copied the memory already. It's hard to check it
> + * before copying, as the size of actually copied buffer is unknown.
> + */
> +
> + /* copy_to_user() may copy zero bytes. No need to check. */
> + if (!to_copy)
> + return;
> + /* Or maybe copy_to_user() failed to copy anything. */
> + if (to_copy == left)
> + return;
> + if ((u64)to < TASK_SIZE) {
> + /* This is a user memory access, check it. */
> + kmsan_internal_check_memory((void *)from, to_copy - left, to,
> + REASON_COPY_TO_USER);
> + return;
> + }
> + /* Otherwise this is a kernel memory access. This happens when a compat
> + * syscall passes an argument allocated on the kernel stack to a real
> + * syscall.
> + * Don't check anything, just copy the shadow of the copied bytes.
> + */
> + shadow = kmsan_get_metadata((void *)to, to_copy - left, META_SHADOW);
> + if (shadow)
> + kmsan_memcpy_metadata((void *)to, (void *)from, to_copy - left);
> +}
> +EXPORT_SYMBOL(kmsan_copy_to_user);
> +
> +void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + /* The users may want to poison/unpoison random memory. */
> + kmsan_internal_poison_shadow((void *)address, size, flags,
> + KMSAN_POISON_NOCHECK);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_poison_shadow);
> +
> +void kmsan_unpoison_shadow(const void *address, size_t size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + /* The users may want to poison/unpoison random memory. */
> + kmsan_internal_unpoison_shadow((void *)address, size,
> + KMSAN_POISON_NOCHECK);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_unpoison_shadow);
> +
> +void kmsan_check_memory(const void *addr, size_t size)
> +{
> + return kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0,
> + REASON_ANY);
> +}
> +EXPORT_SYMBOL(kmsan_check_memory);
> +
> +void kmsan_gup_pgd_range(struct page **pages, int nr)
> +{
> + int i;
> + void *page_addr;
> +
> + /*
> + * gup_pgd_range() has just created a number of new pages that KMSAN
> + * treats as uninitialized. In the case they belong to the userspace
> + * memory, unpoison the corresponding kernel pages.
> + */
> + for (i = 0; i < nr; i++) {
> + page_addr = page_address(pages[i]);
> + if (((u64)page_addr < TASK_SIZE) &&
> + ((u64)page_addr + PAGE_SIZE < TASK_SIZE))
> + kmsan_unpoison_shadow(page_addr, PAGE_SIZE);
> + }
> +
> +}
> +EXPORT_SYMBOL(kmsan_gup_pgd_range);
> +
> +/* Helper function to check an SKB. */
> +void kmsan_check_skb(const struct sk_buff *skb)
> +{
> + int start = skb_headlen(skb);
> + struct sk_buff *frag_iter;
> + int i, copy = 0;
> + skb_frag_t *f;
> + u32 p_off, p_len, copied;
> + struct page *p;
> + u8 *vaddr;
> +
> + if (!skb || !skb->len)
> + return;
> +
> + kmsan_internal_check_memory(skb->data, skb_headlen(skb), 0, REASON_ANY);
> + if (skb_is_nonlinear(skb)) {
> + for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
> + f = &skb_shinfo(skb)->frags[i];
> +
> + skb_frag_foreach_page(f,
> + skb_frag_off(f) - start,
> + copy, p, p_off, p_len, copied) {
> +
> + vaddr = kmap_atomic(p);
> + kmsan_internal_check_memory(vaddr + p_off,
> + p_len, /*user_addr*/ 0,
> + REASON_ANY);
> + kunmap_atomic(vaddr);
> + }
> + }
> + }
> + skb_walk_frags(skb, frag_iter)
> + kmsan_check_skb(frag_iter);
> +}
> +EXPORT_SYMBOL(kmsan_check_skb);
> +
> +/* Helper function to check an URB. */
> +void kmsan_handle_urb(const struct urb *urb, bool is_out)
> +{
> + if (!urb)
> + return;
> + if (is_out)
> + kmsan_internal_check_memory(urb->transfer_buffer,
> + urb->transfer_buffer_length,
> + /*user_addr*/ 0, REASON_SUBMIT_URB);
> + else
> + kmsan_internal_unpoison_shadow(urb->transfer_buffer,
> + urb->transfer_buffer_length,
> + /*checked*/false);
> +}
> +EXPORT_SYMBOL(kmsan_handle_urb);
> +
> +static void kmsan_handle_dma_page(const void *addr, size_t size,
> + enum dma_data_direction dir)
> +{
> + switch (dir) {
> + case DMA_BIDIRECTIONAL:
> + kmsan_internal_check_memory((void *)addr, size, /*user_addr*/0,
> + REASON_ANY);
> + kmsan_internal_unpoison_shadow((void *)addr, size,
> + /*checked*/false);
> + break;
> + case DMA_TO_DEVICE:
> + kmsan_internal_check_memory((void *)addr, size, /*user_addr*/0,
> + REASON_ANY);
> + break;
> + case DMA_FROM_DEVICE:
> + kmsan_internal_unpoison_shadow((void *)addr, size,
> + /*checked*/false);
> + break;
> + case DMA_NONE:
> + break;
> + }
> +}
> +
> +/* Helper function to handle DMA data transfers. */
> +void kmsan_handle_dma(const void *addr, size_t size,
> + enum dma_data_direction dir)
> +{
> + u64 page_offset, to_go, uaddr = (u64)addr;
> +
> + /*
> + * The kernel may occasionally give us adjacent DMA pages not belonging
> + * to the same allocation. Process them separately to avoid triggering
> + * internal KMSAN checks.
> + */
> + while (size > 0) {
> + page_offset = uaddr % PAGE_SIZE;
> + to_go = min(PAGE_SIZE - page_offset, (u64)size);
> + kmsan_handle_dma_page((void *)uaddr, to_go, dir);
> + uaddr += to_go;
> + size -= to_go;
> + }
> +}
> +EXPORT_SYMBOL(kmsan_handle_dma);
> diff --git a/mm/kmsan/kmsan_init.c b/mm/kmsan/kmsan_init.c
> new file mode 100644
> index 000000000000..2816e7075a30
> --- /dev/null
> +++ b/mm/kmsan/kmsan_init.c
> @@ -0,0 +1,88 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN initialization routines.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include "kmsan.h"
> +
> +#include <asm/cpu_entry_area.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +
> +#define NUM_FUTURE_RANGES 128
> +struct start_end_pair {
> + void *start, *end;
> +};
> +
> +static struct start_end_pair start_end_pairs[NUM_FUTURE_RANGES] __initdata;
> +static int future_index __initdata;
> +
> +/*
> + * Record a range of memory for which the metadata pages will be created once
> + * the page allocator becomes available.
> + * TODO(glider): squash together ranges belonging to the same page.
> + */
What needs to be done to address the TODO? Just adding a comment is
fine (or if the TODO can be resolved that's also fine).
> +static void __init kmsan_record_future_shadow_range(void *start, void *end)
> +{
> + BUG_ON(future_index == NUM_FUTURE_RANGES);
> + BUG_ON((start >= end) || !start || !end);
> + start_end_pairs[future_index].start = start;
> + start_end_pairs[future_index].end = end;
> + future_index++;
> +}
> +
> +extern char _sdata[], _edata[];
> +
> +
> +
> +/*
> + * Initialize the shadow for existing mappings during kernel initialization.
> + * These include kernel text/data sections, NODE_DATA and future ranges
> + * registered while creating other data (e.g. percpu).
> + *
> + * Allocations via memblock can be only done before slab is initialized.
> + */
> +void __init kmsan_initialize_shadow(void)
> +{
> + int nid;
> + u64 i;
> + const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE);
> + phys_addr_t p_start, p_end;
> +
> + for_each_reserved_mem_region(i, &p_start, &p_end) {
> + kmsan_record_future_shadow_range(phys_to_virt(p_start),
> + phys_to_virt(p_end+1));
> + }
> + /* Allocate shadow for .data */
> + kmsan_record_future_shadow_range(_sdata, _edata);
> +
> + /*
> + * TODO(glider): alloc_node_data() in arch/x86/mm/numa.c uses
> + * sizeof(pg_data_t).
> + */
What needs to be done to address the TODO? Just adding a comment is
fine (or if the TODO can be resolved that's also fine).
> + for_each_online_node(nid)
> + kmsan_record_future_shadow_range(
> + NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size);
> +
> + for (i = 0; i < future_index; i++)
> + kmsan_init_alloc_meta_for_range(start_end_pairs[i].start,
> + start_end_pairs[i].end);
> +}
> +EXPORT_SYMBOL(kmsan_initialize_shadow);
> +
> +void __init kmsan_initialize(void)
> +{
> + /* Assuming current is init_task */
> + kmsan_internal_task_create(current);
> + kmsan_pr_locked("Starting KernelMemorySanitizer\n");
> + kmsan_ready = true;
> +}
> +EXPORT_SYMBOL(kmsan_initialize);
> diff --git a/mm/kmsan/kmsan_instr.c b/mm/kmsan/kmsan_instr.c
> new file mode 100644
> index 000000000000..7695daf2d88a
> --- /dev/null
> +++ b/mm/kmsan/kmsan_instr.c
> @@ -0,0 +1,259 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN compiler API.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include "kmsan.h"
> +#include <linux/gfp.h>
> +#include <linux/mm.h>
> +
> +static bool is_bad_asm_addr(void *addr, u64 size, bool is_store)
> +{
> + if ((u64)addr < TASK_SIZE)
> + return true;
> + if (!kmsan_get_metadata(addr, size, META_SHADOW))
> + return true;
> + return false;
> +}
> +
> +struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, u64 size)
> +{
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false);
> +}
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n);
> +
> +struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, u64 size)
> +{
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true);
> +}
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n);
> +
> +#define DECLARE_METADATA_PTR_GETTER(size) \
> +struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size(void *addr) \
> +{ \
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); \
> +} \
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \
> + \
> +struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size(void *addr) \
> +{ \
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); \
> +} \
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size)
> +
> +DECLARE_METADATA_PTR_GETTER(1);
> +DECLARE_METADATA_PTR_GETTER(2);
> +DECLARE_METADATA_PTR_GETTER(4);
> +DECLARE_METADATA_PTR_GETTER(8);
> +
> +void __msan_instrument_asm_store(void *addr, u64 size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * Most of the accesses are below 32 bytes. The two exceptions so far
> + * are clwb() (64 bytes) and FPU state (512 bytes).
> + * It's unlikely that the assembly will touch more than 512 bytes.
> + */
> + if (size > 512)
> + size = 8;
> + if (is_bad_asm_addr(addr, size, /*is_store*/true))
> + return;
> + ENTER_RUNTIME(irq_flags);
> + /* Unpoisoning the memory on best effort. */
> + kmsan_internal_unpoison_shadow(addr, size, /*checked*/false);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_instrument_asm_store);
> +
> +void *__msan_memmove(void *dst, void *src, u64 n)
> +{
> + void *result;
> + void *shadow_dst;
> +
> + result = __memmove(dst, src, n);
> + if (!n)
> + /* Some people call memmove() with zero length. */
> + return result;
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + /* Ok to skip address check here, we'll do it later. */
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
> +
> + if (!shadow_dst)
> + /* Can happen e.g. if the memory is untracked. */
> + return result;
> +
> + kmsan_memmove_metadata(dst, src, n);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memmove);
> +
> +void *__msan_memmove_nosanitize(void *dst, void *src, u64 n)
> +{
> + return __memmove(dst, src, n);
> +}
> +EXPORT_SYMBOL(__msan_memmove_nosanitize);
> +
> +void *__msan_memcpy(void *dst, const void *src, u64 n)
> +{
> + void *result;
> + void *shadow_dst;
> +
> + result = __memcpy(dst, src, n);
> + if (!n)
> + /* Some people call memcpy() with zero length. */
> + return result;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + /* Ok to skip address check here, we'll do it later. */
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
> + if (!shadow_dst)
> + /* Can happen e.g. if the memory is untracked. */
> + return result;
> +
> + kmsan_memcpy_metadata(dst, (void *)src, n);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memcpy);
> +
> +void *__msan_memcpy_nosanitize(void *dst, void *src, u64 n)
> +{
> + return __memcpy(dst, src, n);
> +}
> +EXPORT_SYMBOL(__msan_memcpy_nosanitize);
> +
> +void *__msan_memset(void *dst, int c, size_t n)
> +{
> + void *result;
> + unsigned long irq_flags;
> + depot_stack_handle_t new_origin;
> + unsigned int shadow;
> +
> + result = __memset(dst, c, n);
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + ENTER_RUNTIME(irq_flags);
> + shadow = 0;
> + kmsan_internal_memset_shadow(dst, shadow, n, /*checked*/false);
> + new_origin = 0;
> + kmsan_internal_set_origin(dst, n, new_origin);
> + LEAVE_RUNTIME(irq_flags);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memset);
> +
> +void *__msan_memset_nosanitize(void *dst, int c, size_t n)
> +{
> + return __memset(dst, c, n);
> +}
> +EXPORT_SYMBOL(__msan_memset_nosanitize);
> +
> +depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin)
> +{
> + depot_stack_handle_t ret = 0;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return ret;
> +
> + /* Creating new origins may allocate memory. */
> + ENTER_RUNTIME(irq_flags);
> + ret = kmsan_internal_chain_origin(origin);
> + LEAVE_RUNTIME(irq_flags);
> + return ret;
> +}
> +EXPORT_SYMBOL(__msan_chain_origin);
> +
> +void __msan_poison_alloca(void *address, u64 size, char *descr)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[4];
> + unsigned long irq_flags;
> + u64 size_copy = size, to_fill;
> + u64 addr_copy = (u64)address;
> + u64 page_offset;
> + void *shadow_start;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + while (size_copy) {
> + page_offset = addr_copy % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, size_copy);
> + shadow_start = kmsan_get_metadata((void *)addr_copy, to_fill,
> + META_SHADOW);
> + addr_copy += to_fill;
> + size_copy -= to_fill;
> + if (!shadow_start)
> + /* Can happen e.g. if the memory is untracked. */
> + continue;
> + __memset(shadow_start, -1, to_fill);
> + }
> +
> + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN;
> + entries[1] = (u64)descr;
> + entries[2] = (u64)__builtin_return_address(0);
> + entries[3] = (u64)kmsan_internal_return_address(1);
> +
> + /* stack_depot_save() may allocate memory. */
> + ENTER_RUNTIME(irq_flags);
> + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
> + LEAVE_RUNTIME(irq_flags);
> + kmsan_internal_set_origin(address, size, handle);
> +}
> +EXPORT_SYMBOL(__msan_poison_alloca);
> +
> +void __msan_unpoison_alloca(void *address, u64 size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + /* Assuming the shadow exists. */
> + kmsan_internal_unpoison_shadow(address, size, /*checked*/true);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_unpoison_alloca);
> +
> +void __msan_warning(u32 origin)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(origin, /*address*/0, /*size*/0,
> + /*off_first*/0, /*off_last*/0, /*user_addr*/0, REASON_ANY);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_warning);
> +
> +struct kmsan_context_state *__msan_get_context_state(void)
> +{
> + struct kmsan_context_state *ret;
> +
> + ret = task_kmsan_context_state();
> + BUG_ON(!ret);
> + return ret;
> +}
> +EXPORT_SYMBOL(__msan_get_context_state);
> diff --git a/mm/kmsan/kmsan_report.c b/mm/kmsan/kmsan_report.c
> new file mode 100644
> index 000000000000..443ab9c1e8bf
> --- /dev/null
> +++ b/mm/kmsan/kmsan_report.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN error reporting routines.
> + *
> + * Copyright (C) 2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <linux/console.h>
> +#include <linux/stackdepot.h>
> +#include <linux/stacktrace.h>
> +
> +#include "kmsan.h"
> +
> +DEFINE_SPINLOCK(report_lock);
> +
> +void kmsan_print_origin(depot_stack_handle_t origin)
> +{
> + unsigned long *entries = NULL, *chained_entries = NULL;
> + unsigned long nr_entries, chained_nr_entries, magic;
> + char *descr = NULL;
> + void *pc1 = NULL, *pc2 = NULL;
> + depot_stack_handle_t head;
> +
> + if (!origin) {
> + kmsan_pr_err("Origin not found, presumably a false report.\n");
> + return;
> + }
> +
> + while (true) {
> + nr_entries = stack_depot_fetch(origin, &entries);
> + magic = nr_entries ? (entries[0] & KMSAN_MAGIC_MASK) : 0;
> + if ((nr_entries == 4) && (magic == KMSAN_ALLOCA_MAGIC_ORIGIN)) {
> + descr = (char *)entries[1];
> + pc1 = (void *)entries[2];
> + pc2 = (void *)entries[3];
> + kmsan_pr_err("Local variable description: %s\n", descr);
> + kmsan_pr_err("Variable was created at:\n");
> + kmsan_pr_err(" %pS\n", pc1);
> + kmsan_pr_err(" %pS\n", pc2);
> + break;
> + }
> + if ((nr_entries == 3) &&
> + (magic == KMSAN_CHAIN_MAGIC_ORIGIN_FULL)) {
> + head = entries[1];
> + origin = entries[2];
> + kmsan_pr_err("Uninit was stored to memory at:\n");
> + chained_nr_entries =
> + stack_depot_fetch(head, &chained_entries);
> + stack_trace_print(chained_entries, chained_nr_entries,
> + 0);
> + kmsan_pr_err("\n");
> + continue;
> + }
> + kmsan_pr_err("Uninit was created at:\n");
> + if (entries)
> + stack_trace_print(entries, nr_entries, 0);
> + else
> + kmsan_pr_err("No stack\n");
> + break;
> + }
> +}
> +
> +void kmsan_report(depot_stack_handle_t origin,
> + void *address, int size, int off_first, int off_last,
> + const void *user_addr, int reason)
> +{
> + unsigned long flags;
> + unsigned long *entries;
> + unsigned int nr_entries;
> + bool is_uaf = false;
> + char *bug_type = NULL;
> +
> + if (!kmsan_ready)
> + return;
> + if (!current->kmsan.allow_reporting)
> + return;
> + if (!origin)
> + return;
> +
> + nr_entries = stack_depot_fetch(origin, &entries);
> +
> + current->kmsan.allow_reporting = false;
> + spin_lock_irqsave(&report_lock, flags);
> + kmsan_pr_err("=====================================================\n");
> + if (get_dsh_extra_bits(origin) & 1)
> + is_uaf = true;
> + switch (reason) {
> + case REASON_ANY:
> + bug_type = is_uaf ? "use-after-free" : "uninit-value";
> + break;
> + case REASON_COPY_TO_USER:
> + bug_type = is_uaf ? "kernel-infoleak-after-free" :
> + "kernel-infoleak";
> + break;
> + case REASON_SUBMIT_URB:
> + bug_type = is_uaf ? "kernel-usb-infoleak-after-free" :
> + "kernel-usb-infoleak";
> + break;
> + }
> + kmsan_pr_err("BUG: KMSAN: %s in %pS\n",
> + bug_type, kmsan_internal_return_address(2));
> + dump_stack();
> + kmsan_pr_err("\n");
> +
> + kmsan_print_origin(origin);
> +
> + if (size) {
> + kmsan_pr_err("\n");
> + if (off_first == off_last)
> + kmsan_pr_err("Byte %d of %d is uninitialized\n",
> + off_first, size);
> + else
> + kmsan_pr_err("Bytes %d-%d of %d are uninitialized\n",
> + off_first, off_last, size);
> + }
> + if (address)
> + kmsan_pr_err("Memory access of size %d starts at %px\n",
> + size, address);
> + if (user_addr && reason == REASON_COPY_TO_USER)
> + kmsan_pr_err("Data copied to user address %px\n", user_addr);
> + kmsan_pr_err("=====================================================\n");
> + add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
> + spin_unlock_irqrestore(&report_lock, flags);
> + if (panic_on_warn)
> + panic("panic_on_warn set ...\n");
> + current->kmsan.allow_reporting = true;
> +}
> diff --git a/mm/kmsan/kmsan_shadow.c b/mm/kmsan/kmsan_shadow.c
> new file mode 100644
> index 000000000000..06801d76e6b8
> --- /dev/null
> +++ b/mm/kmsan/kmsan_shadow.c
> @@ -0,0 +1,543 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN shadow implementation.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <asm/cpu_entry_area.h>
> +#include <asm/page.h>
> +#include <asm/pgtable_64_types.h>
> +#include <asm/tlbflush.h>
> +#include <linux/memblock.h>
> +#include <linux/mm_types.h>
> +#include <linux/percpu-defs.h>
> +#include <linux/slab.h>
> +#include <linux/smp.h>
> +#include <linux/stddef.h>
> +
> +#include "kmsan.h"
> +#include "kmsan_shadow.h"
> +
> +#define shadow_page_for(page) \
> + ((page)->shadow)
> +
> +#define origin_page_for(page) \
> + ((page)->origin)
> +
> +#define shadow_ptr_for(page) \
> + (page_address((page)->shadow))
> +
> +#define origin_ptr_for(page) \
> + (page_address((page)->origin))
> +
> +#define has_shadow_page(page) \
> + (!!((page)->shadow))
> +
> +#define has_origin_page(page) \
> + (!!((page)->origin))
> +
> +#define set_no_shadow_origin_page(page) \
> + do { \
> + (page)->shadow = NULL; \
> + (page)->origin = NULL; \
> + } while (0) /**/
> +
> +#define is_ignored_page(page) \
> + (!!(((u64)((page)->shadow)) % 2))
> +
> +#define ignore_page(pg) \
> + ((pg)->shadow = (struct page *)((u64)((pg)->shadow) | 1)) \
> +
> +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
> +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
> +
> +/*
> + * Dummy load and store pages to be used when the real metadata is unavailable.
> + * There are separate pages for loads and stores, so that every load returns a
> + * zero, and every store doesn't affect other stores.
> + */
> +char dummy_load_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +char dummy_store_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +
> +/*
> + * Taken from arch/x86/mm/physaddr.h to avoid using an instrumented version.
> + */
> +static int kmsan_phys_addr_valid(unsigned long addr)
> +{
> +#ifdef CONFIG_PHYS_ADDR_T_64BIT
> + return !(addr >> boot_cpu_data.x86_phys_bits);
> +#else
> + return 1;
> +#endif
> +}
> +
> +/*
> + * Taken from arch/x86/mm/physaddr.c to avoid using an instrumented version.
> + */
> +static bool kmsan_virt_addr_valid(void *addr)
> +{
> + unsigned long x = (unsigned long)addr;
> + unsigned long y = x - __START_KERNEL_map;
> +
> + /* use the carry flag to determine if x was < __START_KERNEL_map */
> + if (unlikely(x > y)) {
> + x = y + phys_base;
> +
> + if (y >= KERNEL_IMAGE_SIZE)
> + return false;
> + } else {
> + x = y + (__START_KERNEL_map - PAGE_OFFSET);
> +
> + /* carry flag will be set if starting x was >= PAGE_OFFSET */
> + if ((x > y) || !kmsan_phys_addr_valid(x))
> + return false;
> + }
> +
> + return pfn_valid(x >> PAGE_SHIFT);
> +}
> +
> +static unsigned long vmalloc_meta(void *addr, bool is_origin)
> +{
> + unsigned long addr64 = (unsigned long)addr, off;
> +
> + BUG_ON(is_origin && !IS_ALIGNED(addr64, ORIGIN_SIZE));
> + if (kmsan_internal_is_vmalloc_addr(addr)) {
> + return addr64 + (is_origin ? VMALLOC_ORIGIN_OFFSET
> + : VMALLOC_SHADOW_OFFSET);
> + }
> + if (kmsan_internal_is_module_addr(addr)) {
> + off = addr64 - MODULES_VADDR;
> + return off + (is_origin ? MODULES_ORIGIN_START
> + : MODULES_SHADOW_START);
> + }
> + return 0;
> +}
> +
> +static void *get_cea_meta_or_null(void *addr, bool is_origin)
> +{
> + int cpu = smp_processor_id();
> + int off;
> + char *metadata_array;
> +
> + if (((u64)addr < CPU_ENTRY_AREA_BASE) ||
> + ((u64)addr >= (CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE)))
> + return NULL;
> + off = (char *)addr - (char *)get_cpu_entry_area(cpu);
> + if ((off < 0) || (off >= CPU_ENTRY_AREA_SIZE))
> + return NULL;
> + metadata_array = is_origin ? cpu_entry_area_origin :
> + cpu_entry_area_shadow;
> + return &per_cpu(metadata_array[off], cpu);
> +}
> +
> +static struct page *virt_to_page_or_null(void *vaddr)
> +{
> + if (kmsan_virt_addr_valid(vaddr))
> + return virt_to_page(vaddr);
> + else
> + return NULL;
> +}
> +
> +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *address, u64 size,
> + bool store)
> +{
> + struct shadow_origin_ptr ret;
> + struct page *page;
> + u64 pad, offset, o_offset;
> + const u64 addr64 = (u64)address;
> + u64 o_addr64 = (u64)address;
> + void *shadow;
> +
> + if (size > PAGE_SIZE) {
> + WARN(1, "size too big in %s(%px, %d, %d)\n",
> + __func__, address, size, store);
> + BUG();
> + }
> + if (store) {
> + ret.s = dummy_store_page;
> + ret.o = dummy_store_page;
> + } else {
> + ret.s = dummy_load_page;
> + ret.o = dummy_load_page;
> + }
> + if (!kmsan_ready || IN_RUNTIME())
> + return ret;
> + BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW));
> +
> + if (!IS_ALIGNED(addr64, ORIGIN_SIZE)) {
> + pad = addr64 % ORIGIN_SIZE;
> + o_addr64 -= pad;
> + }
> +
> + if (kmsan_internal_is_vmalloc_addr(address) ||
> + kmsan_internal_is_module_addr(address)) {
> + ret.s = (void *)vmalloc_meta(address, META_SHADOW);
> + ret.o = (void *)vmalloc_meta((void *)o_addr64, META_ORIGIN);
> + return ret;
> + }
> +
> + if (!kmsan_virt_addr_valid(address)) {
> + page = vmalloc_to_page_or_null(address);
> + if (page)
> + goto next;
> + shadow = get_cea_meta_or_null(address, META_SHADOW);
> + if (shadow) {
> + ret.s = shadow;
> + ret.o = get_cea_meta_or_null((void *)o_addr64,
> + META_ORIGIN);
> + return ret;
> + }
> + }
> + page = virt_to_page_or_null(address);
> + if (!page)
> + return ret;
> +next:
> + if (is_ignored_page(page))
> + return ret;
> +
> + if (!has_shadow_page(page) || !has_origin_page(page))
> + return ret;
> + offset = addr64 % PAGE_SIZE;
> + o_offset = o_addr64 % PAGE_SIZE;
> +
> + if (offset + size - 1 > PAGE_SIZE) {
> + /*
> + * The access overflows the current page and touches the
> + * subsequent ones. Make sure the shadow/origin pages are also
> + * consequent.
> + */
> + BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW));
> + }
> +
> + ret.s = shadow_ptr_for(page) + offset;
> + ret.o = origin_ptr_for(page) + o_offset;
> + return ret;
> +}
> +
> +/*
> + * Obtain the shadow or origin pointer for the given address, or NULL if there's
> + * none. The caller must check the return value for being non-NULL if needed.
> + * The return value of this function should not depend on whether we're in the
> + * runtime or not.
> + */
> +void *kmsan_get_metadata(void *address, size_t size, bool is_origin)
> +{
> + struct page *page;
> + void *ret;
> + u64 addr = (u64)address, pad, off;
> +
> + if (is_origin && !IS_ALIGNED(addr, ORIGIN_SIZE)) {
> + pad = addr % ORIGIN_SIZE;
> + addr -= pad;
> + size += pad;
> + }
> + address = (void *)addr;
> + if (kmsan_internal_is_vmalloc_addr(address) ||
> + kmsan_internal_is_module_addr(address)) {
> + return (void *)vmalloc_meta(address, is_origin);
> + }
> +
> + if (!kmsan_virt_addr_valid(address)) {
> + page = vmalloc_to_page_or_null(address);
> + if (page)
> + goto next;
> + ret = get_cea_meta_or_null(address, is_origin);
> + if (ret)
> + return ret;
> + }
> + page = virt_to_page_or_null(address);
> + if (!page)
> + return NULL;
> +next:
> + if (is_ignored_page(page))
> + return NULL;
> + if (!has_shadow_page(page) || !has_origin_page(page))
> + return NULL;
> + off = addr % PAGE_SIZE;
> +
> + ret = (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off;
> + return ret;
> +}
> +
> +void __init kmsan_init_alloc_meta_for_range(void *start, void *end)
> +{
> + u64 addr, size;
> + struct page *page;
> + void *shadow, *origin;
> + struct page *shadow_p, *origin_p;
> +
> + start = (void *)ALIGN_DOWN((u64)start, PAGE_SIZE);
> + size = ALIGN((u64)end - (u64)start, PAGE_SIZE);
> + shadow = memblock_alloc(size, PAGE_SIZE);
> + origin = memblock_alloc(size, PAGE_SIZE);
> + for (addr = 0; addr < size; addr += PAGE_SIZE) {
> + page = virt_to_page_or_null((char *)start + addr);
> + shadow_p = virt_to_page_or_null((char *)shadow + addr);
> + set_no_shadow_origin_page(shadow_p);
> + shadow_page_for(page) = shadow_p;
> + origin_p = virt_to_page_or_null((char *)origin + addr);
> + set_no_shadow_origin_page(origin_p);
> + origin_page_for(page) = origin_p;
> + }
> +}
> +
> +/* Called from mm/memory.c */
> +void kmsan_copy_page_meta(struct page *dst, struct page *src)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + if (!has_shadow_page(src)) {
> + /* TODO(glider): are we leaking pages here? */
> + set_no_shadow_origin_page(dst);
> + return;
> + }
> + if (!has_shadow_page(dst))
> + return;
> + if (is_ignored_page(src)) {
> + ignore_page(dst);
> + return;
> + }
> +
> + ENTER_RUNTIME(irq_flags);
> + __memcpy(shadow_ptr_for(dst), shadow_ptr_for(src),
> + PAGE_SIZE);
> + BUG_ON(!has_origin_page(src) || !has_origin_page(dst));
> + __memcpy(origin_ptr_for(dst), origin_ptr_for(src),
> + PAGE_SIZE);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_copy_page_meta);
> +
> +/* Helper function to allocate page metadata. */
> +static int kmsan_internal_alloc_meta_for_pages(struct page *page,
> + unsigned int order,
> + gfp_t flags, int node)
> +{
> + struct page *shadow, *origin;
> + int pages = 1 << order;
> + int i;
> + bool initialized = (flags & __GFP_ZERO) || !kmsan_ready;
> + depot_stack_handle_t handle;
> +
> + if (flags & __GFP_NO_KMSAN_SHADOW) {
> + for (i = 0; i < pages; i++)
> + set_no_shadow_origin_page(&page[i]);
> + return 0;
> + }
> +
> + /* TODO(glider): must we override the flags? */
> + flags = GFP_ATOMIC;
> + if (initialized)
> + flags |= __GFP_ZERO;
> + shadow = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order);
> + if (!shadow) {
> + for (i = 0; i < pages; i++) {
> + set_no_shadow_origin_page(&page[i]);
> + set_no_shadow_origin_page(&page[i]);
> + }
> + return -ENOMEM;
> + }
> + if (!initialized)
> + __memset(page_address(shadow), -1, PAGE_SIZE * pages);
> +
> + origin = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order);
> + /* Assume we've allocated the origin. */
> + if (!origin) {
> + __free_pages(shadow, order);
> + for (i = 0; i < pages; i++)
> + set_no_shadow_origin_page(&page[i]);
> + return -ENOMEM;
> + }
> +
> + if (!initialized) {
> + handle = kmsan_save_stack_with_flags(flags, /*extra_bits*/0);
> + /*
> + * Addresses are page-aligned, pages are contiguous, so it's ok
> + * to just fill the origin pages with |handle|.
> + */
> + for (i = 0; i < PAGE_SIZE * pages / sizeof(handle); i++) {
> + ((depot_stack_handle_t *)page_address(origin))[i] =
> + handle;
> + }
> + }
> +
> + for (i = 0; i < pages; i++) {
> + shadow_page_for(&page[i]) = &shadow[i];
> + set_no_shadow_origin_page(shadow_page_for(&page[i]));
> + origin_page_for(&page[i]) = &origin[i];
> + set_no_shadow_origin_page(origin_page_for(&page[i]));
> + }
> + return 0;
> +}
> +
> +/* Called from mm/page_alloc.c */
> +int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags)
> +{
> + unsigned long irq_flags;
> + int ret;
> +
> + if (IN_RUNTIME())
> + return 0;
> + ENTER_RUNTIME(irq_flags);
> + ret = kmsan_internal_alloc_meta_for_pages(page, order, flags, -1);
> + LEAVE_RUNTIME(irq_flags);
> + return ret;
> +}
> +
> +/* Called from mm/page_alloc.c */
> +void kmsan_free_page(struct page *page, unsigned int order)
> +{
> + struct page *shadow, *origin, *cur_page;
> + int pages = 1 << order;
> + int i;
> + unsigned long irq_flags;
> +
> + if (!shadow_page_for(page)) {
> + for (i = 0; i < pages; i++) {
> + cur_page = &page[i];
> + BUG_ON(shadow_page_for(cur_page));
> + }
> + return;
> + }
> +
> + if (!kmsan_ready) {
> + for (i = 0; i < pages; i++) {
> + cur_page = &page[i];
> + set_no_shadow_origin_page(cur_page);
> + }
> + return;
> + }
> +
> + if (IN_RUNTIME()) {
> + /*
> + * TODO(glider): looks legit. depot_save_stack() may call
> + * free_pages().
> + */
What needs to be done to address the TODO? Just adding a comment is
fine (or if the TODO can be resolved that's also fine).
> + return;
> + }
> +
> + ENTER_RUNTIME(irq_flags);
> + shadow = shadow_page_for(&page[0]);
> + origin = origin_page_for(&page[0]);
> +
> + /* TODO(glider): this is racy. */
Can this be fixed or does the race not matter -- in the latter case,
just remove the TODO and turn it into a NOTE or similar.
> + for (i = 0; i < pages; i++) {
> + BUG_ON(has_shadow_page(shadow_page_for(&page[i])));
> + BUG_ON(has_shadow_page(origin_page_for(&page[i])));
> + set_no_shadow_origin_page(&page[i]);
> + }
> + BUG_ON(has_shadow_page(shadow));
> + __free_pages(shadow, order);
> +
> + BUG_ON(has_shadow_page(origin));
> + __free_pages(origin, order);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_free_page);
> +
> +/* Called from mm/page_alloc.c */
> +void kmsan_split_page(struct page *page, unsigned int order)
> +{
> + struct page *shadow, *origin;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + if (!has_shadow_page(&page[0])) {
> + BUG_ON(has_origin_page(&page[0]));
> + LEAVE_RUNTIME(irq_flags);
> + return;
> + }
> + shadow = shadow_page_for(&page[0]);
> + split_page(shadow, order);
> +
> + origin = origin_page_for(&page[0]);
> + split_page(origin, order);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_split_page);
> +
> +/* Called from include/linux/highmem.h */
> +void kmsan_clear_page(void *page_addr)
> +{
> + struct page *page;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + BUG_ON(!IS_ALIGNED((u64)page_addr, PAGE_SIZE));
> + page = vmalloc_to_page_or_null(page_addr);
> + if (!page)
> + page = virt_to_page_or_null(page_addr);
> + if (!page || !has_shadow_page(page))
> + return;
> + __memset(shadow_ptr_for(page), 0, PAGE_SIZE);
> + BUG_ON(!has_origin_page(page));
> + __memset(origin_ptr_for(page), 0, PAGE_SIZE);
> +}
> +EXPORT_SYMBOL(kmsan_clear_page);
> +
> +/* Called from mm/vmalloc.c */
> +void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages)
> +{
> + int nr, i, mapped;
> + struct page **s_pages, **o_pages;
> + unsigned long shadow_start, shadow_end, origin_start, origin_end;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + shadow_start = vmalloc_meta((void *)start, META_SHADOW);
> + if (!shadow_start)
> + return;
> +
> + BUG_ON(start >= end);
> + nr = (end - start) / PAGE_SIZE;
> + s_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL);
> + o_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL);
> + if (!s_pages || !o_pages)
> + goto ret;
> + for (i = 0; i < nr; i++) {
> + s_pages[i] = shadow_page_for(pages[i]);
> + o_pages[i] = origin_page_for(pages[i]);
> + }
> + prot = __pgprot(pgprot_val(prot) | _PAGE_NX);
> + prot = PAGE_KERNEL;
> +
> + shadow_end = vmalloc_meta((void *)end, META_SHADOW);
> + origin_start = vmalloc_meta((void *)start, META_ORIGIN);
> + origin_end = vmalloc_meta((void *)end, META_ORIGIN);
> + mapped = __vmap_page_range_noflush(shadow_start, shadow_end,
> + prot, s_pages);
> + BUG_ON(mapped != nr);
> + flush_tlb_kernel_range(shadow_start, shadow_end);
> + mapped = __vmap_page_range_noflush(origin_start, origin_end,
> + prot, o_pages);
> + BUG_ON(mapped != nr);
> + flush_tlb_kernel_range(origin_start, origin_end);
> +ret:
> + kfree(s_pages);
> + kfree(o_pages);
> +}
> +
> +void kmsan_ignore_page(struct page *page, int order)
> +{
> + int pages = 1 << order;
> + int i;
> + struct page *cp;
> +
> + for (i = 0; i < pages; i++) {
> + cp = &page[i];
> + ignore_page(cp);
> + }
> +}
> diff --git a/mm/kmsan/kmsan_shadow.h b/mm/kmsan/kmsan_shadow.h
> new file mode 100644
> index 000000000000..eaa7f771b6a5
> --- /dev/null
> +++ b/mm/kmsan/kmsan_shadow.h
> @@ -0,0 +1,30 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN shadow API.
> + *
> + * This should be agnostic to shadow implementation details.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#ifndef __MM_KMSAN_KMSAN_SHADOW_H
> +#define __MM_KMSAN_KMSAN_SHADOW_H
> +
> +#include <asm/cpu_entry_area.h> /* for CPU_ENTRY_AREA_MAP_SIZE */
> +
> +struct shadow_origin_ptr {
> + void *s, *o;
> +};
> +
> +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size,
> + bool store);
> +void *kmsan_get_metadata(void *addr, size_t size, bool is_origin);
> +void __init kmsan_init_alloc_meta_for_range(void *start, void *end);
> +
> +#endif /* __MM_KMSAN_KMSAN_SHADOW_H */
> diff --git a/scripts/Makefile.kmsan b/scripts/Makefile.kmsan
> new file mode 100644
> index 000000000000..8b3844b66b22
> --- /dev/null
> +++ b/scripts/Makefile.kmsan
> @@ -0,0 +1,12 @@
> +ifdef CONFIG_KMSAN
> +
> +CFLAGS_KMSAN := -fsanitize=kernel-memory
> +
> +ifeq ($(call cc-option, $(CFLAGS_KMSAN) -Werror),)
> + ifneq ($(CONFIG_COMPILE_TEST),y)
> + $(warning Cannot use CONFIG_KMSAN: \
> + -fsanitize=kernel-memory is not supported by compiler)
> + endif
> +endif
> +
> +endif
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-29 16:07 ` Marco Elver
@ 2019-12-19 14:16 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-19 14:16 UTC (permalink / raw)
To: Marco Elver
Cc: Wolfram Sang, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K . Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik
> Could all these just be using '.macro .endm'?
Done in v4
> > +__no_sanitize_memory
> > +static inline unsigned long KMSAN_INIT_8(unsigned long value)
> > +{
> > + return value;
> > +}
>
> Should the above be __always_inline?
No. __always_inline forces a non-instrumented function to be inlined
into its instrumented caller, which results in the former being
instrumented. I've updated the comment to reflect that.
> Does it make sense to use u8, u16, u32, u64 here -- just in case it's
> ported to other architectures in future?
Done in v4.
> > + default: \
> > + BUILD_BUG_ON(1); \
> > + } \
> > + __ret; \
> > + }) /**/
>
> Is the /**/ needed?
No, as long as we use .macro and .endm.
>
> It would be good to add doc comments to all API functions.
Done in v4
> > +extern bool kmsan_ready;
>
> What does this variable mean. Would 'kmsan_enabled' be more accurate?
I think kmsan_inited is a better name, if we want to change it at all.
kmsan_enabled somewhat implies KMSAN can be disabled.
> This is in include/linux -- do they need a KMSAN_ prefix to not clash
> with other definitions?
Done in v4.
> > +#define KMSAN_PARAM_SIZE 800
> > +
> > +#define PARAM_ARRAY_SIZE (KMSAN_PARAM_SIZE / sizeof(depot_stack_handle_t))
>
> Similar here -- does it need a KMSAN_ prefix?
Done in v4.
> > +void kmsan_clear_page(void *page_addr);
>
> It would be good to have doc comments for each of them.
Done in v4.
> > +
> > +KMSAN_SANITIZE := n
> > +KCOV_INSTRUMENT := n
>
> Does KMSAN work together with UBSAN? In that case may this needs a
> UBSAN_SANITIZE := n
Done
> > +#include <linux/stackdepot.h>
> > +#include <linux/stacktrace.h>
> > +#include <linux/types.h>
> > +#include <linux/vmalloc.h>
> > +
> > +#include <linux/mmzone.h>
>
> Why the space above the mmzone.h include?
Removed it, also fixed the include order for this file.
> > +/*
> > + * Some kernel asm() calls mention the non-existing |__force_order| variable
> > + * in the asm constraints to preserve the order of accesses to control
> > + * registers. KMSAN turns those mentions into actual memory accesses, therefore
> > + * the variable is now required to link the kernel.
> > + */
> > +unsigned long __force_order;
>
> Not sure if this is related, but when compiling with KMSAN I get
>
> ERROR: "__force_order" [drivers/misc/lkdtm/lkdtm.ko] undefined!
>
> with a default config with KMSAN selected.
Added an EXPORT_SYMBOL to fix this.
>
> > +bool kmsan_ready;
> > +#define KMSAN_STACK_DEPTH 64
> > +#define MAX_CHAIN_DEPTH 7
>
> Should these defines be above the variable definitions?
Done
>
> Why not just 'panic("%s: ...", __func__, ...)' ?
>
> If the BUG() should not be here, then maybe just WARN_ON?
Replaced with panic().
> > +
> > +/*
> > + * TODO(glider): writing an initialized byte shouldn't zero out the origin, if
> > + * the remaining three bytes are uninitialized.
> > + */
>
> What needs to be done to address the TODO? Just adding a comment is
> fine (or if the TODO can be resolved that's also fine).
Filed https://github.com/google/kmsan/issues/70 to track this. This
isn't a showstopper.
> > + if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) {
> > + kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr);
> > + BUG();
>
> Just panic?
Done.
> > +/*
> > + * TODO(glider): this check shouldn't be performed for origin pages, because
> > + * they're always accessed after the shadow pages.
> > + */
>
> What needs to be done to address the TODO? Just adding a comment is
> fine (or if the TODO can be resolved that's also fine).
Dropped the TODO. This is somewhat perfectionist.
> > + if (origin_p) {
> > + kmsan_pr_locked("Origin: %08x\n", *origin_p);
> > + kmsan_print_origin(*origin_p);
> > + } else {
> > + kmsan_pr_locked("Origin: unavailable\n");
> > + }
>
> These repeated calls to kmsan_pr_locked seem unnecessary. There is
> nothing ensuring atomicity of all these print calls w.r.t. reporting.
Replaced them with pr_err().
> > +/* Stolen from kernel/printk/internal.h */
> > +#define PRINTK_SAFE_CONTEXT_MASK 0x3fffffff
>
> Is this used anywhere?
No. Removed it.
> > +/* Called by kmsan_report.c under a lock. */
> > +#define kmsan_pr_err(...) pr_err(__VA_ARGS__)
>
> Why is this macro needed? It's never redefined, so in the places it is
> used, you can just use pr_err. For readability I would avoid unnecessary
> aliases, but if there is a genuine reason this may be needed in future,
> I would just add a comment.
I've removed the macro.
> > +/* Used in other places - doesn't require a lock. */
> > +#define kmsan_pr_locked(...) \
> > + do { \
> > + unsigned long flags; \
> > + spin_lock_irqsave(&report_lock, flags); \
> > + pr_err(__VA_ARGS__); \
> > + spin_unlock_irqrestore(&report_lock, flags); \
> > + } while (0)
>
> Is this macro needed? The only reason it sort of makes sense is to
> serialize a report with other printing, but otherwise pr_err already
> makes sure things are serialized properly.
Yes, this was the intention. On the other hand, this lock doesn't
prevent non-KMSAN code from messing up KMSAN reports, so it makes
little sense. Maybe we can just keep the spinlock to separate the
reports from each other.
> > +enum KMSAN_BUG_REASON {
> > + REASON_ANY = 0,
> > + REASON_COPY_TO_USER = 1,
> > + REASON_USE_AFTER_FREE = 2,
> > + REASON_SUBMIT_URB = 3,
> > +};
>
> Is it required to explicitly assign constants to these?
No. Removed the constants.
> > +#define LEAVE_RUNTIME(irq_flags) \
> > + do { \
> > + this_cpu_dec(kmsan_in_runtime); \
> > + if (this_cpu_read(kmsan_in_runtime)) { \
> > + kmsan_pr_err("kmsan_in_runtime: %d\n", \
> > + this_cpu_read(kmsan_in_runtime)); \
> > + BUG(); \
> > + } \
> > + restart_nmi(); \
> > + local_irq_restore(irq_flags); \
> > + preempt_enable(); } while (0)
>
> Could these not be macros, and instead be static __always_inline
> functions?
Done
> > +static void kmsan_context_exit(void)
> > +{
> > + int level = this_cpu_read(kmsan_context_level) - 1;
> > +
> > + BUG_ON(level < 0);
> > + this_cpu_write(kmsan_context_level, level);
> > +}
>
> These are not preemption-safe. this_cpu_dec_return followed by the
> BUG_ON should be sufficient. Similarly above and below (using
> this_cpu_add_return)
Good catch, thank you!
> > +void kmsan_interrupt_exit(void)
> > +{
> > + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> > +
> > + BUG_ON(!in_interrupt);
> > + kmsan_context_exit();
> > + /* Can't check preempt_count() here, it may be zero. */
> > + this_cpu_write(kmsan_in_interrupt, in_interrupt - 1);
> > +}
> > +EXPORT_SYMBOL(kmsan_interrupt_exit);
>
> Why exactly does kmsan_in_interrupt need to be maintained here? I can't
> see them being used anywhere else. Is it only for the BUG_ON?
Yes, initially some consistency checks made sense. I think it's safe
to delete them now.
> > +void kmsan_softirq_exit(void)
> > +{
> > + bool in_softirq = this_cpu_read(kmsan_in_softirq);
> > +
> > + BUG_ON(!in_softirq);
> > + kmsan_context_exit();
> > + /* Can't check preempt_count() here, it may be zero. */
> > + this_cpu_write(kmsan_in_softirq, false);
> > +}
> > +EXPORT_SYMBOL(kmsan_softirq_exit);
>
> Same question here for kmsan_in_softirq.
Ditto
> > +void kmsan_nmi_exit(void)
> > +{
> > + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> > +
> > + BUG_ON(!in_nmi);
> > + BUG_ON(preempt_count() & NMI_MASK);
> > + kmsan_context_exit();
> > + this_cpu_write(kmsan_in_nmi, false);
> > +
> > +}
> > +EXPORT_SYMBOL(kmsan_nmi_exit);
>
> And same question here for kmsan_in_nmi.
Ditto.
> > +
> > +/*
> > + * Record a range of memory for which the metadata pages will be created once
> > + * the page allocator becomes available.
> > + * TODO(glider): squash together ranges belonging to the same page.
> > + */
>
> What needs to be done to address the TODO? Just adding a comment is
> fine (or if the TODO can be resolved that's also fine).
Removed the TODO. There's a problem with non-contiguous pages which is
tracked at https://github.com/google/kmsan/issues/71
> > + /*
> > + * TODO(glider): alloc_node_data() in arch/x86/mm/numa.c uses
> > + * sizeof(pg_data_t).
> > + */
>
> What needs to be done to address the TODO? Just adding a comment is
> fine (or if the TODO can be resolved that's also fine).
Resolved this (the code is actually correct)
> > +
> > + if (IN_RUNTIME()) {
> > + /*
> > + * TODO(glider): looks legit. depot_save_stack() may call
> > + * free_pages().
> > + */
>
> What needs to be done to address the TODO? Just adding a comment is
> fine (or if the TODO can be resolved that's also fine).
I've just dropped this if-clause.
> > + return;
> > + }
> > +
> > + ENTER_RUNTIME(irq_flags);
> > + shadow = shadow_page_for(&page[0]);
> > + origin = origin_page_for(&page[0]);
> > +
> > + /* TODO(glider): this is racy. */
>
> Can this be fixed or does the race not matter -- in the latter case,
> just remove the TODO and turn it into a NOTE or similar.
It doesn't matter. Removed the comment.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-22 11:25 ` [PATCH RFC v3 10/36] kmsan: add KMSAN runtime glider
2019-11-24 19:44 ` Wolfram Sang
2019-11-29 16:07 ` Marco Elver
@ 2019-12-02 15:39 ` Andrey Konovalov
2019-12-20 18:58 ` Alexander Potapenko
2019-12-03 14:34 ` Andrey Konovalov
3 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-12-02 15:39 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Wolfram Sang, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel,
Arnd Bergmann, Christoph Hellwig, Christoph Hellwig,
darrick.wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, ericvh, Greg Kroah-Hartman, harry.wentland,
Herbert Xu, iii, mingo, Jason Wang, Jens Axboe, Marek Szyprowski,
Marco Elver, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S . Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
sergey.senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, gor
On Fri, Nov 22, 2019 at 12:26 PM <glider@google.com> wrote:
A few generic comments:
1. There's a lot of TODOs in the code. They either need to be resolved
or removed.
2. This patch is huge, would it be possible to split it? One way to do
this is to have two parts, one that adds the headers and empty hooks,
and the other one that adds hooks implementations. Or something like
that, if that's feasible at all.
[...]
> +/*
> + * Assembly bits to safely invoke KMSAN hooks from .S files.
> + *
> + * Adopted from KTSAN assembly hooks implementation by Dmitry Vyukov:
> + * https://github.com/google/ktsan/blob/ktsan/arch/x86/include/asm/ktsan.h
This link can get stale. Maybe just link the repo?
[...]
> +/*
> + * KMSAN checks.
This comment just repeats the file name. Maybe worth mentioning what
exactly we are checking here, and how this header is different from
kmsan.h. Perhaps some of the functions declared here should be moved
there as well.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
[...]
> +#ifndef LINUX_KMSAN_H
> +#define LINUX_KMSAN_H
> +
> +#include <linux/gfp.h>
> +#include <linux/stackdepot.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +struct page;
> +struct kmem_cache;
> +struct task_struct;
> +struct vm_struct;
> +
> +
Remove unneeded whitespace.
[...]
> +void kmsan_internal_memset_shadow(void *addr, int b, size_t size,
> + bool checked)
> +{
> + void *shadow_start;
> + u64 page_offset, address = (u64)addr;
> + size_t to_fill;
> +
> + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
> + while (size) {
> + page_offset = address % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, (u64)size);
> + shadow_start = kmsan_get_metadata((void *)address, to_fill,
> + META_SHADOW);
> + if (!shadow_start) {
> + if (checked) {
> + kmsan_pr_locked("WARNING: not memsetting %d bytes starting at %px, because the shadow is NULL\n", to_fill, address);
Why not use WARN()?
[...]
> +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[3];
> + u64 magic = KMSAN_CHAIN_MAGIC_ORIGIN_FULL;
> + int depth = 0;
> + static int skipped;
> + u32 extra_bits;
> +
> + if (!kmsan_ready)
> + return 0;
Do we need this check here?
> +
> + if (!id)
> + return id;
And this one?
> + /*
> + * Make sure we have enough spare bits in |id| to hold the UAF bit and
> + * the chain depth.
> + */
> + BUILD_BUG_ON((1 << STACK_DEPOT_EXTRA_BITS) <= (MAX_CHAIN_DEPTH << 1));
> +
> + extra_bits = get_dsh_extra_bits(id);
> +
> + depth = extra_bits >> 1;
> + if (depth >= MAX_CHAIN_DEPTH) {
> + skipped++;
> + if (skipped % 10000 == 0) {
> + kmsan_pr_locked("not chained %d origins\n", skipped);
> + dump_stack();
> + kmsan_print_origin(id);
> + }
> + return id;
> + }
> + depth++;
> + /* Lowest bit is the UAF flag, higher bits hold the depth. */
> + extra_bits = (depth << 1) | (extra_bits & 1);
Please add some helper functions/macros to work with extra_bits.
[...]
> +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked)
> +{
> + if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) {
> + kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr);
Why not use WARN()?
[...]
> +void kmsan_internal_task_create(struct task_struct *task);
> +void kmsan_internal_set_origin(void *addr, int size, u32 origin);
> +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked);
> +
> +struct kmsan_context_state *task_kmsan_context_state(void);
s/task_kmsan_context_state/kmsan_task_context_state/ or something like that.
[...]
> +void kmsan_interrupt_enter(void)
> +{
> + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> +
> + /* Turns out it's possible for in_interrupt to be >0 here. */
Why/how? Expand the comment.
[...]
> +void kmsan_nmi_enter(void)
> +{
> + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> +
> + BUG_ON(in_nmi);
> + BUG_ON(preempt_count() & NMI_MASK);
BUG_ON(in_nmi())?
> +/*
> + * The functions may call back to instrumented code, which, in turn, may call
> + * these hooks again. To avoid re-entrancy, we use __GFP_NO_KMSAN_SHADOW.
> + * Instrumented functions shouldn't be called under
> + * ENTER_RUNTIME()/LEAVE_RUNTIME(), because this will lead to skipping
> + * effects of functions like memset() inside instrumented code.
> + */
Add empty line.
> +/* Called from kernel/kthread.c, kernel/fork.c */
> +void kmsan_task_create(struct task_struct *task)
> +{
> + unsigned long irq_flags;
> +
> + if (!task)
> + return;
> + ENTER_RUNTIME(irq_flags);
> + kmsan_internal_task_create(task);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_task_create);
> +
> +
Remove empty line.
> +/* Called from kernel/exit.c */
> +void kmsan_task_exit(struct task_struct *task)
> +{
> + unsigned long irq_flags;
> + struct kmsan_task_state *state = &task->kmsan;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + state->allow_reporting = false;
> +
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_task_exit);
> +
> +/* Called from mm/slub.c */
> +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (unlikely(object == NULL))
> + return;
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * There's a ctor or this is an RCU cache - do nothing. The memory
> + * status hasn't changed since last use.
> + */
> + if (s->ctor || (s->flags & SLAB_TYPESAFE_BY_RCU))
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + if (flags & __GFP_ZERO) {
No {} needed here.
> + kmsan_internal_unpoison_shadow(object, s->object_size,
> + KMSAN_POISON_CHECK);
> + } else {
> + kmsan_internal_poison_shadow(object, s->object_size, flags,
> + KMSAN_POISON_CHECK);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_slab_alloc);
> +
> +/* Called from mm/slub.c */
> +void kmsan_slab_free(struct kmem_cache *s, void *object)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> +
> + /* RCU slabs could be legally used after free within the RCU period */
> + if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
> + goto leave;
> + if (s->ctor)
Why don't we poison if there's a ctor? Some comment is needed.
> + goto leave;
> + kmsan_internal_poison_shadow(object, s->object_size,
> + GFP_KERNEL,
> + KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
> +leave:
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_slab_free);
> +
> +/* Called from mm/slub.c */
> +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (unlikely(ptr == NULL))
> + return;
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + if (flags & __GFP_ZERO) {
No {} needed here.
[...]
> +void kmsan_check_skb(const struct sk_buff *skb)
> +{
> + int start = skb_headlen(skb);
> + struct sk_buff *frag_iter;
> + int i, copy = 0;
> + skb_frag_t *f;
> + u32 p_off, p_len, copied;
> + struct page *p;
> + u8 *vaddr;
> +
> + if (!skb || !skb->len)
We either need to check !skb before skb_headlen() or drop the check.
> + return;
> +
> + kmsan_internal_check_memory(skb->data, skb_headlen(skb), 0, REASON_ANY);
Use start instead of calling skb_headlen(skb) again. Or just remove
start and always call skb_headlen(skb).
> + if (skb_is_nonlinear(skb)) {
> + for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
> + f = &skb_shinfo(skb)->frags[i];
> +
> + skb_frag_foreach_page(f,
> + skb_frag_off(f) - start,
> + copy, p, p_off, p_len, copied) {
> +
> + vaddr = kmap_atomic(p);
> + kmsan_internal_check_memory(vaddr + p_off,
> + p_len, /*user_addr*/ 0,
> + REASON_ANY);
> + kunmap_atomic(vaddr);
> + }
> + }
> + }
> + skb_walk_frags(skb, frag_iter)
> + kmsan_check_skb(frag_iter);
Hm, won't this recursively walk the same list multiple times?
[...]
> +static void __init kmsan_record_future_shadow_range(void *start, void *end)
> +{
> + BUG_ON(future_index == NUM_FUTURE_RANGES);
> + BUG_ON((start >= end) || !start || !end);
> + start_end_pairs[future_index].start = start;
> + start_end_pairs[future_index].end = end;
> + future_index++;
> +}
> +
> +extern char _sdata[], _edata[];
#include <asm/sections.h>?
> +
> +
> +
Remove excessive whitespace.
> +/*
> + * Initialize the shadow for existing mappings during kernel initialization.
> + * These include kernel text/data sections, NODE_DATA and future ranges
> + * registered while creating other data (e.g. percpu).
> + *
> + * Allocations via memblock can be only done before slab is initialized.
> + */
> +void __init kmsan_initialize_shadow(void)
> +{
> + int nid;
> + u64 i;
> + const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE);
> + phys_addr_t p_start, p_end;
> +
> + for_each_reserved_mem_region(i, &p_start, &p_end) {
No need for {} here.
> + kmsan_record_future_shadow_range(phys_to_virt(p_start),
> + phys_to_virt(p_end+1));
> + }
> + /* Allocate shadow for .data */
> + kmsan_record_future_shadow_range(_sdata, _edata);
> +
> + /*
> + * TODO(glider): alloc_node_data() in arch/x86/mm/numa.c uses
> + * sizeof(pg_data_t).
> + */
> + for_each_online_node(nid)
> + kmsan_record_future_shadow_range(
> + NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size);
Remove tab before (char *)NODE_DATA(nid).
[...]
> +void __msan_instrument_asm_store(void *addr, u64 size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * Most of the accesses are below 32 bytes. The two exceptions so far
> + * are clwb() (64 bytes) and FPU state (512 bytes).
> + * It's unlikely that the assembly will touch more than 512 bytes.
> + */
> + if (size > 512)
Maybe do (WARN_ON(size > 512)) if this is something that we would want
to detect?
> + size = 8;
> + if (is_bad_asm_addr(addr, size, /*is_store*/true))
> + return;
> + ENTER_RUNTIME(irq_flags);
> + /* Unpoisoning the memory on best effort. */
> + kmsan_internal_unpoison_shadow(addr, size, /*checked*/false);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_instrument_asm_store);
> +
> +void *__msan_memmove(void *dst, void *src, u64 n)
> +{
> + void *result;
> + void *shadow_dst;
> +
> + result = __memmove(dst, src, n);
> + if (!n)
> + /* Some people call memmove() with zero length. */
> + return result;
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + /* Ok to skip address check here, we'll do it later. */
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
kmsan_memmove_metadata() performs this check, do we need it here? Same
goes for other callers of kmsan_memmove/memcpy_metadata().
> +
> + if (!shadow_dst)
> + /* Can happen e.g. if the memory is untracked. */
> + return result;
> +
> + kmsan_memmove_metadata(dst, src, n);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memmove);
[...]
> +void *__msan_memset(void *dst, int c, size_t n)
> +{
> + void *result;
> + unsigned long irq_flags;
> + depot_stack_handle_t new_origin;
> + unsigned int shadow;
> +
> + result = __memset(dst, c, n);
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + ENTER_RUNTIME(irq_flags);
> + shadow = 0;
> + kmsan_internal_memset_shadow(dst, shadow, n, /*checked*/false);
> + new_origin = 0;
> + kmsan_internal_set_origin(dst, n, new_origin);
Do we need variables for shadow and new_origin here?
> + LEAVE_RUNTIME(irq_flags);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memset);
[...]
> +void __msan_poison_alloca(void *address, u64 size, char *descr)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[4];
> + unsigned long irq_flags;
> + u64 size_copy = size, to_fill;
> + u64 addr_copy = (u64)address;
> + u64 page_offset;
> + void *shadow_start;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + while (size_copy) {
Why not call kmsan_internal_poison_shadow()/kmsan_internal_memset_shadow()
here instead of doing this manually?
> + page_offset = addr_copy % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, size_copy);
> + shadow_start = kmsan_get_metadata((void *)addr_copy, to_fill,
> + META_SHADOW);
> + addr_copy += to_fill;
> + size_copy -= to_fill;
> + if (!shadow_start)
> + /* Can happen e.g. if the memory is untracked. */
> + continue;
> + __memset(shadow_start, -1, to_fill);
> + }
> +
> + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN;
> + entries[1] = (u64)descr;
> + entries[2] = (u64)__builtin_return_address(0);
> + entries[3] = (u64)kmsan_internal_return_address(1);
> +
> + /* stack_depot_save() may allocate memory. */
> + ENTER_RUNTIME(irq_flags);
> + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
> + LEAVE_RUNTIME(irq_flags);
> + kmsan_internal_set_origin(address, size, handle);
> +}
> +EXPORT_SYMBOL(__msan_poison_alloca);
> +
> +void __msan_unpoison_alloca(void *address, u64 size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + /* Assuming the shadow exists. */
Why do we assume that shadow exists here, but not in
__msan_poison_alloca()? Please expand the comment.
> + kmsan_internal_unpoison_shadow(address, size, /*checked*/true);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_unpoison_alloca);
[...]
> +void kmsan_print_origin(depot_stack_handle_t origin)
> +{
> + unsigned long *entries = NULL, *chained_entries = NULL;
> + unsigned long nr_entries, chained_nr_entries, magic;
> + char *descr = NULL;
> + void *pc1 = NULL, *pc2 = NULL;
> + depot_stack_handle_t head;
> +
> + if (!origin) {
In some cases the caller of kmsan_print_origin() performs this check
and prints a differently formatted message (metadata_is_contiguous())
or no message at all (kmsan_report()). Some unification would be food.
> + kmsan_pr_err("Origin not found, presumably a false report.\n");
> + return;
> + }
> +
> + while (true) {
> + nr_entries = stack_depot_fetch(origin, &entries);
> + magic = nr_entries ? (entries[0] & KMSAN_MAGIC_MASK) : 0;
> + if ((nr_entries == 4) && (magic == KMSAN_ALLOCA_MAGIC_ORIGIN)) {
> + descr = (char *)entries[1];
> + pc1 = (void *)entries[2];
> + pc2 = (void *)entries[3];
> + kmsan_pr_err("Local variable description: %s\n", descr);
> + kmsan_pr_err("Variable was created at:\n");
A shorter way: "Local variable %s created at: ...".
> + kmsan_pr_err(" %pS\n", pc1);
> + kmsan_pr_err(" %pS\n", pc2);
> + break;
> + }
> + if ((nr_entries == 3) &&
> + (magic == KMSAN_CHAIN_MAGIC_ORIGIN_FULL)) {
> + head = entries[1];
> + origin = entries[2];
> + kmsan_pr_err("Uninit was stored to memory at:\n");
> + chained_nr_entries =
> + stack_depot_fetch(head, &chained_entries);
> + stack_trace_print(chained_entries, chained_nr_entries,
> + 0);
> + kmsan_pr_err("\n");
> + continue;
> + }
> + kmsan_pr_err("Uninit was created at:\n");
> + if (entries)
Should this rather check nr_entries?
> + stack_trace_print(entries, nr_entries, 0);
> + else
> + kmsan_pr_err("No stack\n");
KASAN says "(stack is not available)" here. Makes sense to unify with this.
> + break;
> + }
> +}
> +
> +void kmsan_report(depot_stack_handle_t origin,
> + void *address, int size, int off_first, int off_last,
> + const void *user_addr, int reason)
It's not really clear what off_first and off_last arguments are, and
how the range that they describe is different from [address, address +
size). Some comment would be good.
> +{
> + unsigned long flags;
> + unsigned long *entries;
> + unsigned int nr_entries;
> + bool is_uaf = false;
> + char *bug_type = NULL;
> +
> + if (!kmsan_ready)
> + return;
> + if (!current->kmsan.allow_reporting)
> + return;
> + if (!origin)
> + return;
> +
> + nr_entries = stack_depot_fetch(origin, &entries);
Do we need this here?
[...]
> +#define shadow_page_for(page) \
> + ((page)->shadow)
> +
> +#define origin_page_for(page) \
> + ((page)->origin)
> +
> +#define shadow_ptr_for(page) \
> + (page_address((page)->shadow))
> +
> +#define origin_ptr_for(page) \
> + (page_address((page)->origin))
> +
> +#define has_shadow_page(page) \
> + (!!((page)->shadow))
> +
> +#define has_origin_page(page) \
> + (!!((page)->origin))
Something like this would take less space:
#define shadow_page_for(page) ((page)->shadow)
#define origin_page_for(page) ((page)->origin)
...
> +
> +#define set_no_shadow_origin_page(page) \
> + do { \
> + (page)->shadow = NULL; \
> + (page)->origin = NULL; \
> + } while (0) /**/
> +
> +#define is_ignored_page(page) \
> + (!!(((u64)((page)->shadow)) % 2))
> +
> +#define ignore_page(pg) \
> + ((pg)->shadow = (struct page *)((u64)((pg)->shadow) | 1)) \
> +
> +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
> +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
> +
> +/*
> + * Dummy load and store pages to be used when the real metadata is unavailable.
> + * There are separate pages for loads and stores, so that every load returns a
> + * zero, and every store doesn't affect other stores.
every store doesn't affect other _reads_?
[...]
> +static unsigned long vmalloc_meta(void *addr, bool is_origin)
> +{
> + unsigned long addr64 = (unsigned long)addr, off;
> +
> + BUG_ON(is_origin && !IS_ALIGNED(addr64, ORIGIN_SIZE));
> + if (kmsan_internal_is_vmalloc_addr(addr)) {
No need for {} here.
> + return addr64 + (is_origin ? VMALLOC_ORIGIN_OFFSET
> + : VMALLOC_SHADOW_OFFSET);
> + }
> + if (kmsan_internal_is_module_addr(addr)) {
> + off = addr64 - MODULES_VADDR;
> + return off + (is_origin ? MODULES_ORIGIN_START
> + : MODULES_SHADOW_START);
> + }
> + return 0;
> +}
[...]
> +/*
> + * Obtain the shadow or origin pointer for the given address, or NULL if there's
> + * none. The caller must check the return value for being non-NULL if needed.
> + * The return value of this function should not depend on whether we're in the
> + * runtime or not.
> + */
> +void *kmsan_get_metadata(void *address, size_t size, bool is_origin)
This looks very similar to kmsan_get_shadow_origin_ptr(), would it be
possible to unify them somehow or to split out common parts into a
helper function?
> +{
> + struct page *page;
> + void *ret;
> + u64 addr = (u64)address, pad, off;
> +
> + if (is_origin && !IS_ALIGNED(addr, ORIGIN_SIZE)) {
> + pad = addr % ORIGIN_SIZE;
> + addr -= pad;
> + size += pad;
> + }
> + address = (void *)addr;
> + if (kmsan_internal_is_vmalloc_addr(address) ||
> + kmsan_internal_is_module_addr(address)) {
No need for {} here.
> + return (void *)vmalloc_meta(address, is_origin);
> + }
> +
> + if (!kmsan_virt_addr_valid(address)) {
> + page = vmalloc_to_page_or_null(address);
> + if (page)
> + goto next;
> + ret = get_cea_meta_or_null(address, is_origin);
> + if (ret)
> + return ret;
> + }
> + page = virt_to_page_or_null(address);
> + if (!page)
> + return NULL;
> +next:
> + if (is_ignored_page(page))
> + return NULL;
> + if (!has_shadow_page(page) || !has_origin_page(page))
> + return NULL;
> + off = addr % PAGE_SIZE;
> +
> + ret = (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off;
> + return ret;
> +}
[...]
> +void kmsan_ignore_page(struct page *page, int order)
> +{
> + int pages = 1 << order;
> + int i;
> + struct page *cp;
> +
> + for (i = 0; i < pages; i++) {
> + cp = &page[i];
> + ignore_page(cp);
ignore_page(&page[i])
> + }
> +}
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-12-02 15:39 ` Andrey Konovalov
@ 2019-12-20 18:58 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-20 18:58 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Wolfram Sang, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel,
Arnd Bergmann, Christoph Hellwig, Christoph Hellwig,
Darrick J. Wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik
> 1. There's a lot of TODOs in the code. They either need to be resolved
> or removed.
Done in v4
> 2. This patch is huge, would it be possible to split it? One way to do
> this is to have two parts, one that adds the headers and empty hooks,
> and the other one that adds hooks implementations. Or something like
> that, if that's feasible at all.
I've split away kmsan_hooks.c, kmsan_entry.c and kmsan_instr.c
Let's see if that helps.
> > + * Adopted from KTSAN assembly hooks implementation by Dmitry Vyukov:
> > + * https://github.com/google/ktsan/blob/ktsan/arch/x86/include/asm/ktsan.h
>
> This link can get stale. Maybe just link the repo?
Guess there's not much code left to credit, only a series of push
instructions. Not sure it's worth it. I've removed this comment.
> > + * KMSAN checks.
>
> This comment just repeats the file name. Maybe worth mentioning what
> exactly we are checking here, and how this header is different from
> kmsan.h. Perhaps some of the functions declared here should be moved
> there as well.
I've expanded the comment a bit, added doc comments and moved the
functions not supposed to be widely used to kmsan.h
> > +struct task_struct;
> > +struct vm_struct;
> > +
> > +
>
> Remove unneeded whitespace.
Done
> > + if (checked) {
> > + kmsan_pr_locked("WARNING: not memsetting %d bytes starting at %px, because the shadow is NULL\n", to_fill, address);
>
> Why not use WARN()?
changed this to panic()
> > +
> > + if (!kmsan_ready)
> > + return 0;
>
> Do we need this check here?
>
right, this is an internal function, callers must ensure we're not in
the runtime.
> > +
> > + if (!id)
> > + return id;
>
> And this one?
>
This one we do need, as there're cases in which the caller may pass a
zero origin to us.
> > + /* Lowest bit is the UAF flag, higher bits hold the depth. */
> > + extra_bits = (depth << 1) | (extra_bits & 1);
>
> Please add some helper functions/macros to work with extra_bits.
Done
> > + if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) {
> > + kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr);
>
> Why not use WARN()?
changed this to panic()
> > +
> > +struct kmsan_context_state *task_kmsan_context_state(void);
>
> s/task_kmsan_context_state/kmsan_task_context_state/ or something like that.
changed to kmsan_task_context_state
> > +{
> > + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> > +
> > + /* Turns out it's possible for in_interrupt to be >0 here. */
>
> Why/how? Expand the comment.
Dropped this function
>
> [...]
>
> > +void kmsan_nmi_enter(void)
> > +{
> > + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> > +
> > + BUG_ON(in_nmi);
> > + BUG_ON(preempt_count() & NMI_MASK);
>
> BUG_ON(in_nmi())?
I've actually dropped context-specific functions, leaving only
kmsan_context_{enter,exit}
> > +/*
> > + * The functions may call back to instrumented code, which, in turn, may call
> > + * these hooks again. To avoid re-entrancy, we use __GFP_NO_KMSAN_SHADOW.
> > + * Instrumented functions shouldn't be called under
> > + * ENTER_RUNTIME()/LEAVE_RUNTIME(), because this will lead to skipping
> > + * effects of functions like memset() inside instrumented code.
> > + */
>
> Add empty line.
Done
> > + LEAVE_RUNTIME(irq_flags);
> > +}
> > +EXPORT_SYMBOL(kmsan_task_create);
> > +
> > +
>
> Remove empty line.
Done
> > + return;
> > +
> > + ENTER_RUNTIME(irq_flags);
> > + if (flags & __GFP_ZERO) {
>
> No {} needed here.
Done
> > + if (s->ctor)
>
> Why don't we poison if there's a ctor? Some comment is needed.
Done
> > + if (!kmsan_ready || IN_RUNTIME())
> > + return;
> > + ENTER_RUNTIME(irq_flags);
> > + if (flags & __GFP_ZERO) {
>
> No {} needed here.
Done
> > + u8 *vaddr;
> > +
> > + if (!skb || !skb->len)
>
> We either need to check !skb before skb_headlen() or drop the check.
Done
> > + kmsan_internal_check_memory(skb->data, skb_headlen(skb), 0, REASON_ANY);
>
> Use start instead of calling skb_headlen(skb) again. Or just remove
> start and always call skb_headlen(skb).
Done
> > + skb_walk_frags(skb, frag_iter)
> > + kmsan_check_skb(frag_iter);
>
> Hm, won't this recursively walk the same list multiple times?
It should not. See the implementation of skb_dump()
> > +}
> > +
> > +extern char _sdata[], _edata[];
>
> #include <asm/sections.h>?
Didn't know that, thanks!
>
> > +
> > +
> > +
>
> Remove excessive whitespace.
Done
> > +
> > + for_each_reserved_mem_region(i, &p_start, &p_end) {
>
> No need for {} here.
Done
>
> > + for_each_online_node(nid)
> > + kmsan_record_future_shadow_range(
> > + NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size);
>
> Remove tab before (char *)NODE_DATA(nid).
Done
> > + * It's unlikely that the assembly will touch more than 512 bytes.
> > + */
> > + if (size > 512)
>
> Maybe do (WARN_ON(size > 512)) if this is something that we would want
> to detect?
added a WARN_ONCE to that branch
> > + /* Ok to skip address check here, we'll do it later. */
> > + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
>
> kmsan_memmove_metadata() performs this check, do we need it here? Same
> goes for other callers of kmsan_memmove/memcpy_metadata().
>
You're right. I've removed the extra checks.
> > + kmsan_internal_memset_shadow(dst, shadow, n, /*checked*/false);
> > + new_origin = 0;
> > + kmsan_internal_set_origin(dst, n, new_origin);
>
> Do we need variables for shadow and new_origin here?
No. They were here in the hope to make __msan_memset() use shadow of
|c| to initialize |dst|. See https://github.com/google/kmsan/issues/63
> > + if (!kmsan_ready || IN_RUNTIME())
> > + return;
> > +
> > + while (size_copy) {
>
> Why not call kmsan_internal_poison_shadow()/kmsan_internal_memset_shadow()
> here instead of doing this manually?
Done
> > + if (!kmsan_ready || IN_RUNTIME())
> > + return;
> > +
> > + ENTER_RUNTIME(irq_flags);
> > + /* Assuming the shadow exists. */
>
> Why do we assume that shadow exists here, but not in
> __msan_poison_alloca()? Please expand the comment.
We can safely assume that in both cases, and it's a bug if the shadow
doesn't exist. I've removed the misleading comment.
> In some cases the caller of kmsan_print_origin() performs this check
> and prints a differently formatted message (metadata_is_contiguous())
> or no message at all (kmsan_report()). Some unification would be food.
>
Let's just bail out from kmsan_print_origin if the origin is zero.
Only metadata_is_contiguous() may actually pass a zero origin, and the
message there is enough already.
> > + kmsan_pr_err("Local variable description: %s\n", descr);
> > + kmsan_pr_err("Variable was created at:\n");
>
> A shorter way: "Local variable %s created at: ...".
Done
> > + kmsan_pr_err("Uninit was created at:\n");
> > + if (entries)
>
> Should this rather check nr_entries?
SGTM
> > + stack_trace_print(entries, nr_entries, 0);
> > + else
> > + kmsan_pr_err("No stack\n");
>
> KASAN says "(stack is not available)" here. Makes sense to unify with this.
Done
>
> > + break;
> > + }
> > +}
> > +
> > +void kmsan_report(depot_stack_handle_t origin,
> > + void *address, int size, int off_first, int off_last,
> > + const void *user_addr, int reason)
>
> It's not really clear what off_first and off_last arguments are, and
> how the range that they describe is different from [address, address +
> size). Some comment would be good.
Added a doc comment to kmsan_report()
> > +
> > + nr_entries = stack_depot_fetch(origin, &entries);
>
> Do we need this here?
No, nor do we need nr_entries and entries
> > +#define has_origin_page(page) \
> > + (!!((page)->origin))
>
> Something like this would take less space:
>
> #define shadow_page_for(page) ((page)->shadow)
> #define origin_page_for(page) ((page)->origin)
> ...
Done
> > + * Dummy load and store pages to be used when the real metadata is unavailable.
> > + * There are separate pages for loads and stores, so that every load returns a
> > + * zero, and every store doesn't affect other stores.
>
> every store doesn't affect other _reads_?
Ack
> > + BUG_ON(is_origin && !IS_ALIGNED(addr64, ORIGIN_SIZE));
> > + if (kmsan_internal_is_vmalloc_addr(addr)) {
>
> No need for {} here.
Done
> > + * none. The caller must check the return value for being non-NULL if needed.
> > + * The return value of this function should not depend on whether we're in the
> > + * runtime or not.
> > + */
> > +void *kmsan_get_metadata(void *address, size_t size, bool is_origin)
>
> This looks very similar to kmsan_get_shadow_origin_ptr(), would it be
> possible to unify them somehow or to split out common parts into a
> helper function?
I've rewritten kmsan_get_shadow_origin_ptr() to use
kmsan_get_metadata(). It might have become a bit slower (still worth
looking into), but less spaghetti code now.
> > + if (kmsan_internal_is_vmalloc_addr(address) ||
> > + kmsan_internal_is_module_addr(address)) {
>
> No need for {} here.
Done
> > + for (i = 0; i < pages; i++) {
> > + cp = &page[i];
> > + ignore_page(cp);
>
> ignore_page(&page[i])
Done
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 10/36] kmsan: add KMSAN runtime
2019-11-22 11:25 ` [PATCH RFC v3 10/36] kmsan: add KMSAN runtime glider
` (2 preceding siblings ...)
2019-12-02 15:39 ` Andrey Konovalov
@ 2019-12-03 14:34 ` Andrey Konovalov
3 siblings, 0 replies; 120+ messages in thread
From: Andrey Konovalov @ 2019-12-03 14:34 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Wolfram Sang, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel,
Arnd Bergmann, Christoph Hellwig, Christoph Hellwig,
darrick.wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, ericvh, Greg Kroah-Hartman, harry.wentland,
Herbert Xu, iii, mingo, Jason Wang, Jens Axboe, Marek Szyprowski,
Marco Elver, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S . Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
sergey.senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, gor
On Fri, Nov 22, 2019 at 12:26 PM <glider@google.com> wrote:
>
> This patch adds the KernelMemorySanitizer runtime and associated files:
>
> - arch/x86/include/asm/kmsan.h: assembly definitions for hooking
> interrupt handlers;
> - include/linux/kmsan-checks.h: user API to enable/disable KMSAN,kmsan_handle_urb
> poison/unpoison memory etc.
> - include/linux/kmsan.h: declarations of KMSAN memory hooks to be
> referenced outside KMSAN runtime
> - lib/Kconfig.kmsan: declarations for CONFIG_KMSAN and
> CONFIG_TEST_KMSAN
> - mm/kmsan/Makefile: boilerplate Makefile
> - mm/kmsan/kmsan.h: internal KMSAN declarations
> - mm/kmsan/kmsan.c: core functions that operate with shadow and
> origin memory and perform checks, utility functions
> - mm/kmsan/kmsan_entry.c: KMSAN hooks for entry_64.S
> - mm/kmsan/kmsan_hooks.c: KMSAN hooks for kernel subsystems
> - mm/kmsan/kmsan_init.c: KMSAN initialization routines
> - mm/kmsan/kmsan_instr.c: functions called by KMSAN instrumentation
> - scripts/Makefile.kmsan: CFLAGS_KMSAN
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Wolfram Sang <wsa@the-dreams.de>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
> v2:
> - dropped kmsan_handle_vprintk()
> - use locking for single kmsan_pr_err() calls
> - don't try to understand we're inside printk()
> v3:
> - fix an endless loop in __msan_poison_alloca()
> - implement kmsan_handle_dma()
> - dropped kmsan_handle_i2c_transfer()
> - fixed compilation with UNWINDER_ORC
> - dropped assembly hooks for system calls
>
> Change-Id: I4b3a7aba6d5804afac4f5f7274cadf8675b6e119
> ---
> arch/x86/Kconfig | 1 +
> arch/x86/include/asm/kmsan.h | 117 ++++++++
> include/linux/kmsan-checks.h | 122 ++++++++
> include/linux/kmsan.h | 143 +++++++++
> lib/Kconfig.debug | 2 +
> lib/Kconfig.kmsan | 22 ++
> mm/kmsan/Makefile | 4 +
> mm/kmsan/kmsan.c | 563 +++++++++++++++++++++++++++++++++++
> mm/kmsan/kmsan.h | 146 +++++++++
> mm/kmsan/kmsan_entry.c | 118 ++++++++
> mm/kmsan/kmsan_hooks.c | 422 ++++++++++++++++++++++++++
> mm/kmsan/kmsan_init.c | 88 ++++++
> mm/kmsan/kmsan_instr.c | 259 ++++++++++++++++
> mm/kmsan/kmsan_report.c | 133 +++++++++
> mm/kmsan/kmsan_shadow.c | 543 +++++++++++++++++++++++++++++++++
> mm/kmsan/kmsan_shadow.h | 30 ++
> scripts/Makefile.kmsan | 12 +
> 17 files changed, 2725 insertions(+)
> create mode 100644 arch/x86/include/asm/kmsan.h
> create mode 100644 include/linux/kmsan-checks.h
> create mode 100644 include/linux/kmsan.h
> create mode 100644 lib/Kconfig.kmsan
> create mode 100644 mm/kmsan/Makefile
> create mode 100644 mm/kmsan/kmsan.c
> create mode 100644 mm/kmsan/kmsan.h
> create mode 100644 mm/kmsan/kmsan_entry.c
> create mode 100644 mm/kmsan/kmsan_hooks.c
> create mode 100644 mm/kmsan/kmsan_init.c
> create mode 100644 mm/kmsan/kmsan_instr.c
> create mode 100644 mm/kmsan/kmsan_report.c
> create mode 100644 mm/kmsan/kmsan_shadow.c
> create mode 100644 mm/kmsan/kmsan_shadow.h
> create mode 100644 scripts/Makefile.kmsan
>
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index d6e1faa28c58..3f83a5c53808 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -135,6 +135,7 @@ config X86
> select HAVE_ARCH_JUMP_LABEL
> select HAVE_ARCH_JUMP_LABEL_RELATIVE
> select HAVE_ARCH_KASAN if X86_64
> + select HAVE_ARCH_KMSAN if X86_64
> select HAVE_ARCH_KGDB
> select HAVE_ARCH_MMAP_RND_BITS if MMU
> select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT
> diff --git a/arch/x86/include/asm/kmsan.h b/arch/x86/include/asm/kmsan.h
> new file mode 100644
> index 000000000000..fc5f1224a059
> --- /dev/null
> +++ b/arch/x86/include/asm/kmsan.h
> @@ -0,0 +1,117 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Assembly bits to safely invoke KMSAN hooks from .S files.
> + *
> + * Adopted from KTSAN assembly hooks implementation by Dmitry Vyukov:
> + * https://github.com/google/ktsan/blob/ktsan/arch/x86/include/asm/ktsan.h
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +#ifndef _ASM_X86_KMSAN_H
> +#define _ASM_X86_KMSAN_H
> +
> +#ifdef CONFIG_KMSAN
> +
> +#define KMSAN_PUSH_REGS \
> + pushq %rax; \
> + pushq %rcx; \
> + pushq %rdx; \
> + pushq %rdi; \
> + pushq %rsi; \
> + pushq %r8; \
> + pushq %r9; \
> + pushq %r10; \
> + pushq %r11; \
> +/**/
> +
> +#define KMSAN_POP_REGS \
> + popq %r11; \
> + popq %r10; \
> + popq %r9; \
> + popq %r8; \
> + popq %rsi; \
> + popq %rdi; \
> + popq %rdx; \
> + popq %rcx; \
> + popq %rax; \
> +/**/
> +
> +#define KMSAN_INTERRUPT_ENTER \
> + KMSAN_PUSH_REGS \
> + call kmsan_interrupt_enter; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_INTERRUPT_EXIT \
> + KMSAN_PUSH_REGS \
> + call kmsan_interrupt_exit; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_SOFTIRQ_ENTER \
> + KMSAN_PUSH_REGS \
> + call kmsan_softirq_enter; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_SOFTIRQ_EXIT \
> + KMSAN_PUSH_REGS \
> + call kmsan_softirq_exit; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_NMI_ENTER \
> + KMSAN_PUSH_REGS \
> + call kmsan_nmi_enter; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_NMI_EXIT \
> + KMSAN_PUSH_REGS \
> + call kmsan_nmi_exit; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_IST_ENTER(shift_ist) \
> + KMSAN_PUSH_REGS \
> + movq $shift_ist, %rdi; \
> + call kmsan_ist_enter; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_IST_EXIT(shift_ist) \
> + KMSAN_PUSH_REGS \
> + movq $shift_ist, %rdi; \
> + call kmsan_ist_exit; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +#define KMSAN_UNPOISON_PT_REGS \
> + KMSAN_PUSH_REGS \
> + call kmsan_unpoison_pt_regs; \
> + KMSAN_POP_REGS \
> +/**/
> +
> +
> +#else /* ifdef CONFIG_KMSAN */
> +
> +#define KMSAN_INTERRUPT_ENTER
> +#define KMSAN_INTERRUPT_EXIT
> +#define KMSAN_SOFTIRQ_ENTER
> +#define KMSAN_SOFTIRQ_EXIT
> +#define KMSAN_NMI_ENTER
> +#define KMSAN_NMI_EXIT
> +#define KMSAN_SYSCALL_ENTER
> +#define KMSAN_SYSCALL_EXIT
> +#define KMSAN_IST_ENTER(shift_ist)
> +#define KMSAN_IST_EXIT(shift_ist)
> +#define KMSAN_UNPOISON_PT_REGS
> +
> +#endif /* ifdef CONFIG_KMSAN */
> +#endif /* ifndef _ASM_X86_KMSAN_H */
> diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h
> new file mode 100644
> index 000000000000..623854e88d4b
> --- /dev/null
> +++ b/include/linux/kmsan-checks.h
> @@ -0,0 +1,122 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN checks.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#ifndef _LINUX_KMSAN_CHECKS_H
> +#define _LINUX_KMSAN_CHECKS_H
> +
> +#include <linux/bug.h>
> +#include <linux/dma-direction.h>
> +#include <linux/types.h>
> +
> +struct page;
> +struct sk_buff;
> +struct urb;
> +
> +#ifdef CONFIG_KMSAN
> +
> +/*
> + * Helper functions that mark the return value initialized.
> + * Note that Clang ignores the inline attribute in the cases when a no_sanitize
> + * function is called from an instrumented one.
> + */
> +
> +__no_sanitize_memory
> +static inline unsigned char KMSAN_INIT_1(unsigned char value)
> +{
> + return value;
> +}
> +
> +__no_sanitize_memory
> +static inline unsigned short KMSAN_INIT_2(unsigned short value)
> +{
> + return value;
> +}
> +
> +__no_sanitize_memory
> +static inline unsigned int KMSAN_INIT_4(unsigned int value)
> +{
> + return value;
> +}
> +
> +__no_sanitize_memory
> +static inline unsigned long KMSAN_INIT_8(unsigned long value)
> +{
> + return value;
> +}
> +
> +#define KMSAN_INIT_VALUE(val) \
> + ({ \
> + typeof(val) __ret; \
> + switch (sizeof(val)) { \
> + case 1: \
> + *(unsigned char *)&__ret = KMSAN_INIT_1( \
> + (unsigned char)val); \
> + break; \
> + case 2: \
> + *(unsigned short *)&__ret = KMSAN_INIT_2( \
> + (unsigned short)val); \
> + break; \
> + case 4: \
> + *(unsigned int *)&__ret = KMSAN_INIT_4( \
> + (unsigned int)val); \
> + break; \
> + case 8: \
> + *(unsigned long *)&__ret = KMSAN_INIT_8( \
> + (unsigned long)val); \
> + break; \
> + default: \
> + BUILD_BUG_ON(1); \
> + } \
> + __ret; \
> + }) /**/
> +
> +void kmsan_ignore_page(struct page *page, int order);
> +void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags);
> +void kmsan_unpoison_shadow(const void *address, size_t size);
> +void kmsan_check_memory(const void *address, size_t size);
> +void kmsan_check_skb(const struct sk_buff *skb);
> +void kmsan_handle_dma(const void *address, size_t size,
> + enum dma_data_direction direction);
> +void kmsan_handle_urb(const struct urb *urb, bool is_out);
> +void kmsan_copy_to_user(const void *to, const void *from, size_t to_copy,
> + size_t left);
> +void *__msan_memcpy(void *dst, const void *src, u64 n);
> +void kmsan_enter_runtime(unsigned long *flags);
> +void kmsan_leave_runtime(unsigned long *flags);
> +
> +#else
> +
> +#define KMSAN_INIT_VALUE(value) (value)
> +
> +static inline void kmsan_ignore_page(struct page *page, int order) {}
> +static inline void kmsan_poison_shadow(const void *address, size_t size,
> + gfp_t flags) {}
> +static inline void kmsan_unpoison_shadow(const void *address, size_t size) {}
> +static inline void kmsan_check_memory(const void *address, size_t size) {}
> +static inline void kmsan_check_skb(const struct sk_buff *skb) {}
> +static inline void kmsan_handle_urb(const struct urb *urb, bool is_out) {}
> +static inline void kmsan_handle_dma(const void *address, size_t size,
> + enum dma_data_direction direction) {}
> +static inline void kmsan_copy_to_user(
> + const void *to, const void *from, size_t to_copy, size_t left) {}
> +static inline void *__msan_memcpy(void *dst, const void *src, size_t n)
> +{
> + return NULL;
> +}
> +
> +static inline void kmsan_enter_runtime(unsigned long *flags) {}
> +static inline void kmsan_leave_runtime(unsigned long *flags) {}
> +
> +#endif
> +
> +#endif /* _LINUX_KMSAN_CHECKS_H */
> diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h
> new file mode 100644
> index 000000000000..f5638bac368e
> --- /dev/null
> +++ b/include/linux/kmsan.h
> @@ -0,0 +1,143 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN API for subsystems.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +#ifndef LINUX_KMSAN_H
> +#define LINUX_KMSAN_H
> +
> +#include <linux/gfp.h>
> +#include <linux/stackdepot.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +struct page;
> +struct kmem_cache;
> +struct task_struct;
> +struct vm_struct;
> +
> +
> +extern bool kmsan_ready;
> +
> +#ifdef CONFIG_KMSAN
> +void __init kmsan_initialize_shadow(void);
> +void __init kmsan_initialize(void);
> +
> +/* These constants are defined in the MSan LLVM instrumentation pass. */
> +#define RETVAL_SIZE 800
> +#define KMSAN_PARAM_SIZE 800
> +
> +#define PARAM_ARRAY_SIZE (KMSAN_PARAM_SIZE / sizeof(depot_stack_handle_t))
> +
> +struct kmsan_context_state {
> + char param_tls[KMSAN_PARAM_SIZE];
> + char retval_tls[RETVAL_SIZE];
> + char va_arg_tls[KMSAN_PARAM_SIZE];
> + char va_arg_origin_tls[KMSAN_PARAM_SIZE];
> + u64 va_arg_overflow_size_tls;
> + depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE];
> + depot_stack_handle_t retval_origin_tls;
> + depot_stack_handle_t origin_tls;
> +};
> +
> +struct kmsan_task_state {
> + bool allow_reporting;
> + struct kmsan_context_state cstate;
> +};
> +
> +void kmsan_task_create(struct task_struct *task);
> +void kmsan_task_exit(struct task_struct *task);
> +void kmsan_alloc_shadow_for_region(void *start, size_t size);
> +int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags);
> +void kmsan_gup_pgd_range(struct page **pages, int nr);
> +void kmsan_free_page(struct page *page, unsigned int order);
> +void kmsan_split_page(struct page *page, unsigned int order);
> +void kmsan_copy_page_meta(struct page *dst, struct page *src);
> +
> +void kmsan_poison_slab(struct page *page, gfp_t flags);
> +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
> +void kmsan_kfree_large(const void *ptr);
> +void kmsan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
> + gfp_t flags);
> +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
> +void kmsan_slab_free(struct kmem_cache *s, void *object);
> +
> +void kmsan_slab_setup_object(struct kmem_cache *s, void *object);
> +void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
> + size_t size, void *object);
> +
> +/* vmap */
> +void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages);
> +void kmsan_vunmap_page_range(unsigned long addr, unsigned long end);
> +
> +/* ioremap */
> +void kmsan_ioremap_page_range(unsigned long addr, unsigned long end,
> + phys_addr_t phys_addr, pgprot_t prot);
> +void kmsan_iounmap_page_range(unsigned long start, unsigned long end);
> +
> +void kmsan_softirq_enter(void);
> +void kmsan_softirq_exit(void);
> +
> +void kmsan_clear_page(void *page_addr);
> +
> +#else
> +
> +static inline void __init kmsan_initialize_shadow(void) { }
> +static inline void __init kmsan_initialize(void) { }
> +
> +static inline void kmsan_task_create(struct task_struct *task) {}
> +static inline void kmsan_task_exit(struct task_struct *task) {}
> +static inline void kmsan_alloc_shadow_for_region(void *start, size_t size) {}
> +static inline int kmsan_alloc_page(struct page *page, unsigned int order,
> + gfp_t flags)
> +{
> + return 0;
> +}
> +static inline void kmsan_gup_pgd_range(struct page **pages, int nr) {}
> +static inline void kmsan_free_page(struct page *page, unsigned int order) {}
> +static inline void kmsan_split_page(struct page *page, unsigned int order) {}
> +static inline void kmsan_copy_page_meta(struct page *dst, struct page *src) {}
> +
> +static inline void kmsan_poison_slab(struct page *page, gfp_t flags) {}
> +static inline void kmsan_kmalloc_large(const void *ptr, size_t size,
> + gfp_t flags) {}
> +static inline void kmsan_kfree_large(const void *ptr) {}
> +static inline void kmsan_kmalloc(struct kmem_cache *s, const void *object,
> + size_t size, gfp_t flags) {}
> +static inline void kmsan_slab_alloc(struct kmem_cache *s, void *object,
> + gfp_t flags) {}
> +static inline void kmsan_slab_free(struct kmem_cache *s, void *object) {}
> +
> +static inline void kmsan_slab_setup_object(struct kmem_cache *s,
> + void *object) {}
> +static inline void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
> + size_t size, void *object) {}
> +
> +static inline void kmsan_vmap_page_range_noflush(unsigned long start,
> + unsigned long end,
> + pgprot_t prot,
> + struct page **pages) {}
> +static inline void kmsan_vunmap_page_range(unsigned long start,
> + unsigned long end) {}
> +
> +static inline void kmsan_ioremap_page_range(unsigned long start,
> + unsigned long end,
> + phys_addr_t phys_addr,
> + pgprot_t prot) {}
> +static inline void kmsan_iounmap_page_range(unsigned long start,
> + unsigned long end) {}
> +static inline void kmsan_softirq_enter(void) {}
> +static inline void kmsan_softirq_exit(void) {}
> +
> +static inline void kmsan_clear_page(void *page_addr) {}
> +#endif
> +
> +#endif /* LINUX_KMSAN_H */
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 93d97f9b0157..75c36318943d 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -756,6 +756,8 @@ config DEBUG_STACKOVERFLOW
>
> source "lib/Kconfig.kasan"
>
> +source "lib/Kconfig.kmsan"
> +
> endmenu # "Memory Debugging"
>
> config ARCH_HAS_KCOV
> diff --git a/lib/Kconfig.kmsan b/lib/Kconfig.kmsan
> new file mode 100644
> index 000000000000..187dddfcf220
> --- /dev/null
> +++ b/lib/Kconfig.kmsan
> @@ -0,0 +1,22 @@
> +config HAVE_ARCH_KMSAN
> + bool
> +
> +if HAVE_ARCH_KMSAN
> +
> +config KMSAN
> + bool "KMSAN: detector of uninitialized memory use"
> + depends on SLUB && !KASAN
> + select STACKDEPOT
> + help
> + KMSAN is a dynamic detector of uses of uninitialized memory in the
> + kernel. It is based on compiler instrumentation provided by Clang
> + and thus requires Clang 10.0.0+ to build.
> +
> +config TEST_KMSAN
> + tristate "Module for testing KMSAN for bug detection"
> + depends on m && KMSAN
> + help
> + Test module that can trigger various uses of uninitialized memory
> + detectable by KMSAN.
> +
> +endif
> diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile
> new file mode 100644
> index 000000000000..ccf6d2d00a7a
> --- /dev/null
> +++ b/mm/kmsan/Makefile
> @@ -0,0 +1,4 @@
> +obj-y := kmsan.o kmsan_instr.o kmsan_init.o kmsan_entry.o kmsan_hooks.o kmsan_report.o kmsan_shadow.o
> +
> +KMSAN_SANITIZE := n
> +KCOV_INSTRUMENT := n
> diff --git a/mm/kmsan/kmsan.c b/mm/kmsan/kmsan.c
> new file mode 100644
> index 000000000000..21e97d4b1a99
> --- /dev/null
> +++ b/mm/kmsan/kmsan.c
> @@ -0,0 +1,563 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN runtime library.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <asm/page.h>
> +#include <linux/compiler.h>
> +#include <linux/export.h>
> +#include <linux/highmem.h>
> +#include <linux/interrupt.h>
> +#include <linux/kernel.h>
> +#include <linux/kmsan.h>
> +#include <linux/memory.h>
> +#include <linux/mm.h>
> +#include <linux/preempt.h>
> +#include <linux/percpu-defs.h>
> +#include <linux/mm_types.h>
> +#include <linux/slab.h>
> +#include <linux/stackdepot.h>
> +#include <linux/stacktrace.h>
> +#include <linux/types.h>
> +#include <linux/vmalloc.h>
> +
> +#include <linux/mmzone.h>
> +
> +#include "../slab.h"
> +#include "kmsan.h"
> +
> +/*
> + * Some kernel asm() calls mention the non-existing |__force_order| variable
> + * in the asm constraints to preserve the order of accesses to control
> + * registers. KMSAN turns those mentions into actual memory accesses, therefore
> + * the variable is now required to link the kernel.
> + */
> +unsigned long __force_order;
> +
> +bool kmsan_ready;
> +#define KMSAN_STACK_DEPTH 64
> +#define MAX_CHAIN_DEPTH 7
> +
> +/*
> + * According to Documentation/x86/kernel-stacks, kernel code can run on the
> + * following stacks:
> + * - regular task stack - when executing the task code
> + * - interrupt stack - when handling external hardware interrupts and softirqs
> + * - NMI stack
> + * 0 is for regular interrupts, 1 for softirqs, 2 for NMI.
> + * Because interrupts may nest, trying to use a new context for every new
> + * interrupt.
> + */
> +/* [0] for dummy per-CPU context. */
> +DEFINE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
> + kmsan_percpu_cstate);
> +/* 0 for task context, |i>0| for kmsan_context_state[i]. */
> +DEFINE_PER_CPU(int, kmsan_context_level);
> +DEFINE_PER_CPU(int, kmsan_in_interrupt);
> +DEFINE_PER_CPU(bool, kmsan_in_softirq);
> +DEFINE_PER_CPU(bool, kmsan_in_nmi);
> +DEFINE_PER_CPU(int, kmsan_in_runtime);
> +
> +struct kmsan_context_state *task_kmsan_context_state(void)
> +{
> + int cpu = smp_processor_id();
> + int level = this_cpu_read(kmsan_context_level);
> + struct kmsan_context_state *ret;
> +
> + if (!kmsan_ready || IN_RUNTIME()) {
> + ret = &per_cpu(kmsan_percpu_cstate[0], cpu);
> + __memset(ret, 0, sizeof(struct kmsan_context_state));
> + return ret;
> + }
> +
> + if (!level)
> + ret = ¤t->kmsan.cstate;
> + else
> + ret = &per_cpu(kmsan_percpu_cstate[level], cpu);
> + return ret;
> +}
> +
> +void kmsan_internal_task_create(struct task_struct *task)
> +{
> + struct kmsan_task_state *state = &task->kmsan;
> +
> + __memset(state, 0, sizeof(struct kmsan_task_state));
> + state->allow_reporting = true;
> +}
> +
> +void kmsan_internal_memset_shadow(void *addr, int b, size_t size,
> + bool checked)
> +{
> + void *shadow_start;
> + u64 page_offset, address = (u64)addr;
> + size_t to_fill;
> +
> + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
> + while (size) {
> + page_offset = address % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, (u64)size);
> + shadow_start = kmsan_get_metadata((void *)address, to_fill,
> + META_SHADOW);
> + if (!shadow_start) {
> + if (checked) {
> + kmsan_pr_locked("WARNING: not memsetting %d bytes starting at %px, because the shadow is NULL\n", to_fill, address);
> + BUG();
> + }
> + /* Otherwise just move on. */
> + } else {
> + __memset(shadow_start, b, to_fill);
> + }
> + address += to_fill;
> + size -= to_fill;
> + }
> +}
> +
> +void kmsan_internal_poison_shadow(void *address, size_t size,
> + gfp_t flags, unsigned int poison_flags)
> +{
> + bool checked = poison_flags & KMSAN_POISON_CHECK;
> + depot_stack_handle_t handle;
> + u32 extra_bits = 0;
> +
> + if (poison_flags & KMSAN_POISON_FREE)
> + extra_bits = 1;
> + kmsan_internal_memset_shadow(address, -1, size, checked);
> + handle = kmsan_save_stack_with_flags(flags, extra_bits);
> + kmsan_set_origin_checked(address, size, handle, checked);
> +}
> +
> +void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked)
> +{
> + kmsan_internal_memset_shadow(address, 0, size, checked);
> + kmsan_set_origin_checked(address, size, 0, checked);
> +}
> +
> +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags,
> + unsigned int reserved)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[KMSAN_STACK_DEPTH];
> + unsigned int nr_entries;
> +
> + nr_entries = stack_trace_save(entries, KMSAN_STACK_DEPTH, 0);
> + filter_irq_stacks(entries, nr_entries);
> +
> + /* Don't sleep (see might_sleep_if() in __alloc_pages_nodemask()). */
> + flags &= ~__GFP_DIRECT_RECLAIM;
> +
> + handle = stack_depot_save(entries, nr_entries, flags);
> + return set_dsh_extra_bits(handle, reserved);
> +}
> +
> +/*
> + * Depending on the value of is_memmove, this serves as both a memcpy and a
> + * memmove implementation.
> + *
> + * As with the regular memmove, do the following:
> + * - if src and dst don't overlap, use memcpy();
> + * - if src and dst overlap:
> + * - if src > dst, use memcpy();
> + * - if src < dst, use reverse-memcpy.
> + * Why this is correct:
> + * - problems may arise if for some part of the overlapping region we
> + * overwrite its shadow with a new value before copying it somewhere.
> + * But there's a 1:1 mapping between the kernel memory and its shadow,
> + * therefore if this doesn't happen with the kernel memory it can't happen
> + * with the shadow.
> + */
> +void kmsan_memcpy_memmove_metadata(void *dst, void *src, size_t n,
> + bool is_memmove)
> +{
> + void *shadow_src, *shadow_dst;
> + depot_stack_handle_t *origin_src, *origin_dst;
> + int src_slots, dst_slots, i, iter, step, skip_bits;
> + depot_stack_handle_t old_origin = 0, chain_origin, new_origin = 0;
> + u32 *align_shadow_src, shadow;
> + bool backwards;
> +
> + BUG_ON(!metadata_is_contiguous(dst, n, META_SHADOW));
> + BUG_ON(!metadata_is_contiguous(src, n, META_SHADOW));
> +
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
> + if (!shadow_dst)
> + return;
> +
> + shadow_src = kmsan_get_metadata(src, n, META_SHADOW);
> + if (!shadow_src) {
> + /*
> + * |src| is untracked: zero out destination shadow, ignore the
> + * origins, we're done.
> + */
> + __memset(shadow_dst, 0, n);
> + return;
> + }
> + if (is_memmove)
> + __memmove(shadow_dst, shadow_src, n);
> + else
> + __memcpy(shadow_dst, shadow_src, n);
> +
> + origin_dst = kmsan_get_metadata(dst, n, META_ORIGIN);
> + origin_src = kmsan_get_metadata(src, n, META_ORIGIN);
> + BUG_ON(!origin_dst || !origin_src);
> + BUG_ON(!metadata_is_contiguous(dst, n, META_ORIGIN));
> + BUG_ON(!metadata_is_contiguous(src, n, META_ORIGIN));
> + src_slots = (ALIGN((u64)src + n, ORIGIN_SIZE) -
> + ALIGN_DOWN((u64)src, ORIGIN_SIZE)) / ORIGIN_SIZE;
> + dst_slots = (ALIGN((u64)dst + n, ORIGIN_SIZE) -
> + ALIGN_DOWN((u64)dst, ORIGIN_SIZE)) / ORIGIN_SIZE;
> + BUG_ON(!src_slots || !dst_slots);
> + BUG_ON((src_slots < 1) || (dst_slots < 1));
> + BUG_ON((src_slots - dst_slots > 1) || (dst_slots - src_slots < -1));
> +
> + backwards = is_memmove && (dst > src);
> + i = backwards ? min(src_slots, dst_slots) - 1 : 0;
> + iter = backwards ? -1 : 1;
> +
> + align_shadow_src = (u32 *)ALIGN_DOWN((u64)shadow_src, ORIGIN_SIZE);
> + for (step = 0; step < min(src_slots, dst_slots); step++, i += iter) {
> + BUG_ON(i < 0);
> + shadow = align_shadow_src[i];
> + if (i == 0) {
> + /*
> + * If |src| isn't aligned on ORIGIN_SIZE, don't
> + * look at the first |src % ORIGIN_SIZE| bytes
> + * of the first shadow slot.
> + */
> + skip_bits = ((u64)src % ORIGIN_SIZE) * 8;
> + shadow = (shadow << skip_bits) >> skip_bits;
> + }
> + if (i == src_slots - 1) {
> + /*
> + * If |src + n| isn't aligned on
> + * ORIGIN_SIZE, don't look at the last
> + * |(src + n) % ORIGIN_SIZE| bytes of the
> + * last shadow slot.
> + */
> + skip_bits = (((u64)src + n) % ORIGIN_SIZE) * 8;
> + shadow = (shadow >> skip_bits) << skip_bits;
> + }
> + /*
> + * Overwrite the origin only if the corresponding
> + * shadow is nonempty.
> + */
> + if (origin_src[i] && (origin_src[i] != old_origin) && shadow) {
> + old_origin = origin_src[i];
> + chain_origin = kmsan_internal_chain_origin(old_origin);
> + /*
> + * kmsan_internal_chain_origin() may return
> + * NULL, but we don't want to lose the previous
> + * origin value.
> + */
> + if (chain_origin)
> + new_origin = chain_origin;
> + else
> + new_origin = old_origin;
> + }
> + if (shadow)
> + origin_dst[i] = new_origin;
> + else
> + origin_dst[i] = 0;
> + }
> +}
> +
> +void kmsan_memcpy_metadata(void *dst, void *src, size_t n)
> +{
> + kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/false);
> +}
> +
> +void kmsan_memmove_metadata(void *dst, void *src, size_t n)
> +{
> + kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/true);
> +}
> +
> +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[3];
> + u64 magic = KMSAN_CHAIN_MAGIC_ORIGIN_FULL;
> + int depth = 0;
> + static int skipped;
> + u32 extra_bits;
> +
> + if (!kmsan_ready)
> + return 0;
> +
> + if (!id)
> + return id;
> + /*
> + * Make sure we have enough spare bits in |id| to hold the UAF bit and
> + * the chain depth.
> + */
> + BUILD_BUG_ON((1 << STACK_DEPOT_EXTRA_BITS) <= (MAX_CHAIN_DEPTH << 1));
> +
> + extra_bits = get_dsh_extra_bits(id);
> +
> + depth = extra_bits >> 1;
> + if (depth >= MAX_CHAIN_DEPTH) {
> + skipped++;
> + if (skipped % 10000 == 0) {
> + kmsan_pr_locked("not chained %d origins\n", skipped);
> + dump_stack();
> + kmsan_print_origin(id);
> + }
> + return id;
> + }
> + depth++;
> + /* Lowest bit is the UAF flag, higher bits hold the depth. */
> + extra_bits = (depth << 1) | (extra_bits & 1);
> + /* TODO(glider): how do we figure out we've dropped some frames? */
> + entries[0] = magic + depth;
> + entries[1] = kmsan_save_stack_with_flags(GFP_ATOMIC, extra_bits);
> + entries[2] = id;
> + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
> + return set_dsh_extra_bits(handle, extra_bits);
> +}
> +
> +void kmsan_write_aligned_origin(void *var, size_t size, u32 origin)
> +{
> + u32 *var_cast = (u32 *)var;
> + int i;
> +
> + BUG_ON((u64)var_cast % ORIGIN_SIZE);
> + BUG_ON(size % ORIGIN_SIZE);
> + for (i = 0; i < size / ORIGIN_SIZE; i++)
> + var_cast[i] = origin;
> +}
> +
> +/*
> + * TODO(glider): writing an initialized byte shouldn't zero out the origin, if
> + * the remaining three bytes are uninitialized.
> + */
> +void kmsan_internal_set_origin(void *addr, int size, u32 origin)
> +{
> + void *origin_start;
> + u64 address = (u64)addr, page_offset;
> + size_t to_fill, pad = 0;
> +
> + if (!IS_ALIGNED(address, ORIGIN_SIZE)) {
> + pad = address % ORIGIN_SIZE;
> + address -= pad;
> + size += pad;
> + }
> +
> + while (size > 0) {
> + page_offset = address % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, (u64)size);
> + /* write at least ORIGIN_SIZE bytes */
> + to_fill = ALIGN(to_fill, ORIGIN_SIZE);
> + BUG_ON(!to_fill);
> + origin_start = kmsan_get_metadata((void *)address, to_fill,
> + META_ORIGIN);
> + address += to_fill;
> + size -= to_fill;
> + if (!origin_start)
> + /* Can happen e.g. if the memory is untracked. */
> + continue;
> + kmsan_write_aligned_origin(origin_start, to_fill, origin);
> + }
> +}
> +
> +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked)
> +{
> + if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) {
> + kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr);
> + BUG();
> + }
> + kmsan_internal_set_origin(addr, size, origin);
> +}
> +
> +struct page *vmalloc_to_page_or_null(void *vaddr)
> +{
> + struct page *page;
> +
> + if (!kmsan_internal_is_vmalloc_addr(vaddr) &&
> + !kmsan_internal_is_module_addr(vaddr))
> + return NULL;
> + page = vmalloc_to_page(vaddr);
> + if (pfn_valid(page_to_pfn(page)))
> + return page;
> + else
> + return NULL;
> +}
> +
> +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr,
> + int reason)
> +{
> + unsigned long irq_flags;
> + unsigned long addr64 = (unsigned long)addr;
> + unsigned char *shadow = NULL;
> + depot_stack_handle_t *origin = NULL;
> + depot_stack_handle_t cur_origin = 0, new_origin = 0;
> + int cur_off_start = -1;
> + int i, chunk_size;
> + size_t pos = 0;
> +
> + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW));
> + if (size <= 0)
> + return;
> + while (pos < size) {
> + chunk_size = min(size - pos,
> + PAGE_SIZE - ((addr64 + pos) % PAGE_SIZE));
> + shadow = kmsan_get_metadata((void *)(addr64 + pos), chunk_size,
> + META_SHADOW);
> + if (!shadow) {
> + /*
> + * This page is untracked. If there were uninitialized
> + * bytes before, report them.
> + */
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size,
> + cur_off_start, pos - 1, user_addr,
> + reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> + cur_origin = 0;
> + cur_off_start = -1;
> + pos += chunk_size;
> + continue;
> + }
> + for (i = 0; i < chunk_size; i++) {
> + if (!shadow[i]) {
> + /*
> + * This byte is unpoisoned. If there were
> + * poisoned bytes before, report them.
> + */
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size,
> + cur_off_start, pos + i - 1,
> + user_addr, reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> + cur_origin = 0;
> + cur_off_start = -1;
> + continue;
> + }
> + origin = kmsan_get_metadata((void *)(addr64 + pos + i),
> + chunk_size - i, META_ORIGIN);
> + BUG_ON(!origin);
> + new_origin = *origin;
> + /*
> + * Encountered new origin - report the previous
> + * uninitialized range.
> + */
> + if (cur_origin != new_origin) {
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size,
> + cur_off_start, pos + i - 1,
> + user_addr, reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> + cur_origin = new_origin;
> + cur_off_start = pos + i;
> + }
> + }
> + pos += chunk_size;
> + }
> + BUG_ON(pos != size);
> + if (cur_origin) {
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(cur_origin, addr, size, cur_off_start, pos - 1,
> + user_addr, reason);
> + LEAVE_RUNTIME(irq_flags);
> + }
> +}
> +
> +/*
> + * TODO(glider): this check shouldn't be performed for origin pages, because
> + * they're always accessed after the shadow pages.
> + */
> +bool metadata_is_contiguous(void *addr, size_t size, bool is_origin)
> +{
> + u64 cur_addr = (u64)addr, next_addr;
> + char *cur_meta = NULL, *next_meta = NULL;
> + depot_stack_handle_t *origin_p;
> + bool all_untracked = false;
> + const char *fname = is_origin ? "origin" : "shadow";
> +
> + if (!size)
> + return true;
> +
> + /* The whole range belongs to the same page. */
> + if (ALIGN_DOWN(cur_addr + size - 1, PAGE_SIZE) ==
> + ALIGN_DOWN(cur_addr, PAGE_SIZE))
> + return true;
> + cur_meta = kmsan_get_metadata((void *)cur_addr, 1, is_origin);
> + if (!cur_meta)
> + all_untracked = true;
> + for (next_addr = cur_addr + PAGE_SIZE; next_addr < (u64)addr + size;
> + cur_addr = next_addr,
> + cur_meta = next_meta,
> + next_addr += PAGE_SIZE) {
> + next_meta = kmsan_get_metadata((void *)next_addr, 1, is_origin);
> + if (!next_meta) {
> + if (!all_untracked)
> + goto report;
> + continue;
> + }
> + if ((u64)cur_meta == ((u64)next_meta - PAGE_SIZE))
> + continue;
> + goto report;
> + }
> + return true;
> +
> +report:
> + kmsan_pr_locked("BUG: attempting to access two shadow page ranges.\n");
> + dump_stack();
> + kmsan_pr_locked("\n");
> + kmsan_pr_locked("Access of size %d at %px.\n", size, addr);
> + kmsan_pr_locked("Addresses belonging to different ranges: %px and %px\n",
> + cur_addr, next_addr);
> + kmsan_pr_locked("page[0].%s: %px, page[1].%s: %px\n",
> + fname, cur_meta, fname, next_meta);
> + origin_p = kmsan_get_metadata(addr, 1, META_ORIGIN);
> + if (origin_p) {
> + kmsan_pr_locked("Origin: %08x\n", *origin_p);
> + kmsan_print_origin(*origin_p);
> + } else {
> + kmsan_pr_locked("Origin: unavailable\n");
> + }
> + return false;
> +}
> +
> +/*
> + * Dummy replacement for __builtin_return_address() which may crash without
> + * frame pointers.
> + */
> +void *kmsan_internal_return_address(int arg)
> +{
> +#ifdef CONFIG_UNWINDER_FRAME_POINTER
> + switch (arg) {
> + case 1:
> + return __builtin_return_address(1);
> + case 2:
> + return __builtin_return_address(2);
> + default:
> + BUG();
> + }
> +#else
> + unsigned long entries[1];
> +
> + stack_trace_save(entries, 1, arg);
> + return (void *)entries[0];
> +#endif
> +}
> +
> +bool kmsan_internal_is_module_addr(void *vaddr)
> +{
> + return ((u64)vaddr >= MODULES_VADDR) && ((u64)vaddr < MODULES_END);
> +}
> +
> +bool kmsan_internal_is_vmalloc_addr(void *addr)
> +{
> + return ((u64)addr >= VMALLOC_START) && ((u64)addr < VMALLOC_END);
> +}
> diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h
> new file mode 100644
> index 000000000000..8760feef39bf
> --- /dev/null
> +++ b/mm/kmsan/kmsan.h
> @@ -0,0 +1,146 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN internal declarations.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#ifndef __MM_KMSAN_KMSAN_H
> +#define __MM_KMSAN_KMSAN_H
> +
> +#include <asm/pgtable_64_types.h>
> +#include <linux/irqflags.h>
> +#include <linux/sched.h>
> +#include <linux/stackdepot.h>
> +#include <linux/stacktrace.h>
> +#include <linux/nmi.h>
> +#include <linux/mm.h>
> +#include <linux/printk.h>
> +
> +#include "kmsan_shadow.h"
> +
> +#define KMSAN_MAGIC_MASK 0xffffffffff00
> +#define KMSAN_ALLOCA_MAGIC_ORIGIN 0x4110c4071900
> +#define KMSAN_CHAIN_MAGIC_ORIGIN_FULL 0xd419170cba00
> +
> +#define KMSAN_POISON_NOCHECK 0x0
> +#define KMSAN_POISON_CHECK 0x1
> +#define KMSAN_POISON_FREE 0x2
> +
> +#define ORIGIN_SIZE 4
> +
> +#define META_SHADOW (false)
> +#define META_ORIGIN (true)
> +
> +#define KMSAN_NESTED_CONTEXT_MAX (8)
> +/* [0] for dummy per-CPU context */
> +DECLARE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX],
> + kmsan_percpu_cstate);
> +/* 0 for task context, |i>0| for kmsan_context_state[i]. */
> +DECLARE_PER_CPU(int, kmsan_context_level);
> +DECLARE_PER_CPU(int, kmsan_in_interrupt);
> +DECLARE_PER_CPU(bool, kmsan_in_softirq);
> +DECLARE_PER_CPU(bool, kmsan_in_nmi);
> +
> +extern spinlock_t report_lock;
> +
> +/* Stolen from kernel/printk/internal.h */
> +#define PRINTK_SAFE_CONTEXT_MASK 0x3fffffff
> +
> +/* Called by kmsan_report.c under a lock. */
> +#define kmsan_pr_err(...) pr_err(__VA_ARGS__)
> +
> +/* Used in other places - doesn't require a lock. */
> +#define kmsan_pr_locked(...) \
> + do { \
> + unsigned long flags; \
> + spin_lock_irqsave(&report_lock, flags); \
> + pr_err(__VA_ARGS__); \
> + spin_unlock_irqrestore(&report_lock, flags); \
> + } while (0)
> +
> +void kmsan_print_origin(depot_stack_handle_t origin);
> +void kmsan_report(depot_stack_handle_t origin,
> + void *address, int size, int off_first, int off_last,
> + const void *user_addr, int reason);
> +
> +
> +enum KMSAN_BUG_REASON {
> + REASON_ANY = 0,
> + REASON_COPY_TO_USER = 1,
> + REASON_USE_AFTER_FREE = 2,
> + REASON_SUBMIT_URB = 3,
> +};
> +
> +/*
> + * When a compiler hook is invoked, it may make a call to instrumented code
> + * and eventually call itself recursively. To avoid that, we protect the
> + * runtime entry points with ENTER_RUNTIME()/LEAVE_RUNTIME() macros and exit
> + * the hook if IN_RUNTIME() is true. But when an interrupt occurs inside the
> + * runtime, the hooks won’t run either, which may lead to errors.
> + * Therefore we have to disable interrupts inside the runtime.
> + */
> +DECLARE_PER_CPU(int, kmsan_in_runtime);
> +#define IN_RUNTIME() (this_cpu_read(kmsan_in_runtime))
> +#define ENTER_RUNTIME(irq_flags) \
> + do { \
> + preempt_disable(); \
> + local_irq_save(irq_flags); \
> + stop_nmi(); \
> + this_cpu_inc(kmsan_in_runtime); \
> + BUG_ON(this_cpu_read(kmsan_in_runtime) > 1); \
> + } while (0)
> +#define LEAVE_RUNTIME(irq_flags) \
> + do { \
> + this_cpu_dec(kmsan_in_runtime); \
> + if (this_cpu_read(kmsan_in_runtime)) { \
> + kmsan_pr_err("kmsan_in_runtime: %d\n", \
> + this_cpu_read(kmsan_in_runtime)); \
> + BUG(); \
> + } \
> + restart_nmi(); \
> + local_irq_restore(irq_flags); \
> + preempt_enable(); } while (0)
> +
> +void kmsan_memcpy_metadata(void *dst, void *src, size_t n);
> +void kmsan_memmove_metadata(void *dst, void *src, size_t n);
> +
> +depot_stack_handle_t kmsan_save_stack(void);
> +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags,
> + unsigned int extra_bits);
> +void kmsan_internal_poison_shadow(void *address, size_t size, gfp_t flags,
> + unsigned int poison_flags);
> +void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked);
> +void kmsan_internal_memset_shadow(void *address, int b, size_t size,
> + bool checked);
> +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id);
> +void kmsan_write_aligned_origin(void *var, size_t size, u32 origin);
> +
> +void kmsan_internal_task_create(struct task_struct *task);
> +void kmsan_internal_set_origin(void *addr, int size, u32 origin);
> +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked);
> +
> +struct kmsan_context_state *task_kmsan_context_state(void);
> +
> +bool metadata_is_contiguous(void *addr, size_t size, bool is_origin);
> +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr,
> + int reason);
> +
> +struct page *vmalloc_to_page_or_null(void *vaddr);
> +
> +/* Declared in mm/vmalloc.c */
> +void __vunmap_page_range(unsigned long addr, unsigned long end);
> +int __vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages);
> +
> +void *kmsan_internal_return_address(int arg);
> +bool kmsan_internal_is_module_addr(void *vaddr);
> +bool kmsan_internal_is_vmalloc_addr(void *addr);
> +
> +#endif /* __MM_KMSAN_KMSAN_H */
> diff --git a/mm/kmsan/kmsan_entry.c b/mm/kmsan/kmsan_entry.c
> new file mode 100644
> index 000000000000..47bc7736f1a9
> --- /dev/null
> +++ b/mm/kmsan/kmsan_entry.c
> @@ -0,0 +1,118 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN hooks for entry_64.S
> + *
> + * Copyright (C) 2018-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include "kmsan.h"
> +
> +static void kmsan_context_enter(void)
> +{
> + int level = this_cpu_read(kmsan_context_level) + 1;
> +
> + BUG_ON(level >= KMSAN_NESTED_CONTEXT_MAX);
> + this_cpu_write(kmsan_context_level, level);
> +}
> +
> +static void kmsan_context_exit(void)
> +{
> + int level = this_cpu_read(kmsan_context_level) - 1;
> +
> + BUG_ON(level < 0);
> + this_cpu_write(kmsan_context_level, level);
> +}
> +
> +void kmsan_interrupt_enter(void)
> +{
> + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> +
> + /* Turns out it's possible for in_interrupt to be >0 here. */
> + kmsan_context_enter();
> + BUG_ON(in_interrupt > 1);
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_interrupt, in_interrupt + 1);
> +}
> +EXPORT_SYMBOL(kmsan_interrupt_enter);
> +
> +void kmsan_interrupt_exit(void)
> +{
> + int in_interrupt = this_cpu_read(kmsan_in_interrupt);
> +
> + BUG_ON(!in_interrupt);
> + kmsan_context_exit();
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_interrupt, in_interrupt - 1);
> +}
> +EXPORT_SYMBOL(kmsan_interrupt_exit);
> +
> +void kmsan_softirq_enter(void)
> +{
> + bool in_softirq = this_cpu_read(kmsan_in_softirq);
> +
> + BUG_ON(in_softirq);
> + kmsan_context_enter();
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_softirq, true);
> +}
> +EXPORT_SYMBOL(kmsan_softirq_enter);
> +
> +void kmsan_softirq_exit(void)
> +{
> + bool in_softirq = this_cpu_read(kmsan_in_softirq);
> +
> + BUG_ON(!in_softirq);
> + kmsan_context_exit();
> + /* Can't check preempt_count() here, it may be zero. */
> + this_cpu_write(kmsan_in_softirq, false);
> +}
> +EXPORT_SYMBOL(kmsan_softirq_exit);
> +
> +void kmsan_nmi_enter(void)
> +{
> + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> +
> + BUG_ON(in_nmi);
> + BUG_ON(preempt_count() & NMI_MASK);
> + kmsan_context_enter();
> + this_cpu_write(kmsan_in_nmi, true);
> +}
> +EXPORT_SYMBOL(kmsan_nmi_enter);
> +
> +void kmsan_nmi_exit(void)
> +{
> + bool in_nmi = this_cpu_read(kmsan_in_nmi);
> +
> + BUG_ON(!in_nmi);
> + BUG_ON(preempt_count() & NMI_MASK);
> + kmsan_context_exit();
> + this_cpu_write(kmsan_in_nmi, false);
> +
> +}
> +EXPORT_SYMBOL(kmsan_nmi_exit);
> +
> +void kmsan_ist_enter(u64 shift_ist)
> +{
> + kmsan_context_enter();
> +}
> +EXPORT_SYMBOL(kmsan_ist_enter);
> +
> +void kmsan_ist_exit(u64 shift_ist)
> +{
> + kmsan_context_exit();
> +}
> +EXPORT_SYMBOL(kmsan_ist_exit);
> +
> +void kmsan_unpoison_pt_regs(struct pt_regs *regs)
> +{
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + kmsan_internal_unpoison_shadow(regs, sizeof(*regs), /*checked*/true);
> +}
> +EXPORT_SYMBOL(kmsan_unpoison_pt_regs);
> diff --git a/mm/kmsan/kmsan_hooks.c b/mm/kmsan/kmsan_hooks.c
> new file mode 100644
> index 000000000000..13a6ed809d81
> --- /dev/null
> +++ b/mm/kmsan/kmsan_hooks.c
> @@ -0,0 +1,422 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN hooks for kernel subsystems.
> + *
> + * These functions handle creation of KMSAN metadata for memory allocations.
> + *
> + * Copyright (C) 2018-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <asm/cacheflush.h>
> +#include <linux/dma-direction.h>
> +#include <linux/gfp.h>
> +#include <linux/mm.h>
> +#include <linux/mm_types.h>
> +#include <linux/skbuff.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include "../slab.h"
> +#include "kmsan.h"
> +
> +/*
> + * The functions may call back to instrumented code, which, in turn, may call
> + * these hooks again. To avoid re-entrancy, we use __GFP_NO_KMSAN_SHADOW.
> + * Instrumented functions shouldn't be called under
> + * ENTER_RUNTIME()/LEAVE_RUNTIME(), because this will lead to skipping
> + * effects of functions like memset() inside instrumented code.
> + */
> +/* Called from kernel/kthread.c, kernel/fork.c */
> +void kmsan_task_create(struct task_struct *task)
> +{
> + unsigned long irq_flags;
> +
> + if (!task)
> + return;
> + ENTER_RUNTIME(irq_flags);
> + kmsan_internal_task_create(task);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_task_create);
> +
> +
> +/* Called from kernel/exit.c */
> +void kmsan_task_exit(struct task_struct *task)
> +{
> + unsigned long irq_flags;
> + struct kmsan_task_state *state = &task->kmsan;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + state->allow_reporting = false;
> +
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_task_exit);
> +
> +/* Called from mm/slub.c */
> +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (unlikely(object == NULL))
> + return;
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * There's a ctor or this is an RCU cache - do nothing. The memory
> + * status hasn't changed since last use.
> + */
> + if (s->ctor || (s->flags & SLAB_TYPESAFE_BY_RCU))
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + if (flags & __GFP_ZERO) {
> + kmsan_internal_unpoison_shadow(object, s->object_size,
> + KMSAN_POISON_CHECK);
> + } else {
> + kmsan_internal_poison_shadow(object, s->object_size, flags,
> + KMSAN_POISON_CHECK);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_slab_alloc);
> +
> +/* Called from mm/slub.c */
> +void kmsan_slab_free(struct kmem_cache *s, void *object)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> +
> + /* RCU slabs could be legally used after free within the RCU period */
> + if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
> + goto leave;
> + if (s->ctor)
> + goto leave;
> + kmsan_internal_poison_shadow(object, s->object_size,
> + GFP_KERNEL,
> + KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
> +leave:
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_slab_free);
> +
> +/* Called from mm/slub.c */
> +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (unlikely(ptr == NULL))
> + return;
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + if (flags & __GFP_ZERO) {
> + kmsan_internal_unpoison_shadow((void *)ptr, size,
> + /*checked*/true);
> + } else {
> + kmsan_internal_poison_shadow((void *)ptr, size, flags,
> + KMSAN_POISON_CHECK);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_kmalloc_large);
> +
> +/* Called from mm/slub.c */
> +void kmsan_kfree_large(const void *ptr)
> +{
> + struct page *page;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + page = virt_to_head_page((void *)ptr);
> + BUG_ON(ptr != page_address(page));
> + kmsan_internal_poison_shadow(
> + (void *)ptr, PAGE_SIZE << compound_order(page), GFP_KERNEL,
> + KMSAN_POISON_CHECK | KMSAN_POISON_FREE);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_kfree_large);
> +
> +static unsigned long vmalloc_shadow(unsigned long addr)
> +{
> + return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_SHADOW);
> +}
> +
> +static unsigned long vmalloc_origin(unsigned long addr)
> +{
> + return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_ORIGIN);
> +}
> +
> +/* Called from mm/vmalloc.c */
> +void kmsan_vunmap_page_range(unsigned long start, unsigned long end)
> +{
> + __vunmap_page_range(vmalloc_shadow(start), vmalloc_shadow(end));
> + __vunmap_page_range(vmalloc_origin(start), vmalloc_origin(end));
> +}
> +EXPORT_SYMBOL(kmsan_vunmap_page_range);
> +
> +/* Called from lib/ioremap.c */
> +/*
> + * This function creates new shadow/origin pages for the physical pages mapped
> + * into the virtual memory. If those physical pages already had shadow/origin,
> + * those are ignored.
> + */
> +void kmsan_ioremap_page_range(unsigned long start, unsigned long end,
> + phys_addr_t phys_addr, pgprot_t prot)
> +{
> + unsigned long irq_flags;
> + struct page *shadow, *origin;
> + int i, nr;
> + unsigned long off = 0;
> + gfp_t gfp_mask = GFP_KERNEL | __GFP_ZERO | __GFP_NO_KMSAN_SHADOW;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + nr = (end - start) / PAGE_SIZE;
> + ENTER_RUNTIME(irq_flags);
> + for (i = 0; i < nr; i++, off += PAGE_SIZE) {
> + shadow = alloc_pages(gfp_mask, 1);
> + origin = alloc_pages(gfp_mask, 1);
> + __vmap_page_range_noflush(vmalloc_shadow(start + off),
> + vmalloc_shadow(start + off + PAGE_SIZE),
> + prot, &shadow);
> + __vmap_page_range_noflush(vmalloc_origin(start + off),
> + vmalloc_origin(start + off + PAGE_SIZE),
> + prot, &origin);
> + }
> + flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end));
> + flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end));
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_ioremap_page_range);
> +
> +void kmsan_iounmap_page_range(unsigned long start, unsigned long end)
> +{
> + int i, nr;
> + struct page *shadow, *origin;
> + unsigned long v_shadow, v_origin;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + nr = (end - start) / PAGE_SIZE;
> + ENTER_RUNTIME(irq_flags);
> + v_shadow = (unsigned long)vmalloc_shadow(start);
> + v_origin = (unsigned long)vmalloc_origin(start);
> + for (i = 0; i < nr; i++, v_shadow += PAGE_SIZE, v_origin += PAGE_SIZE) {
> + shadow = vmalloc_to_page_or_null((void *)v_shadow);
> + origin = vmalloc_to_page_or_null((void *)v_origin);
> + __vunmap_page_range(v_shadow, v_shadow + PAGE_SIZE);
> + __vunmap_page_range(v_origin, v_origin + PAGE_SIZE);
> + if (shadow)
> + __free_pages(shadow, 1);
> + if (origin)
> + __free_pages(origin, 1);
> + }
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_iounmap_page_range);
> +
> +/* Called from include/linux/uaccess.h, include/linux/uaccess.h */
> +void kmsan_copy_to_user(const void *to, const void *from,
> + size_t to_copy, size_t left)
> +{
> + void *shadow;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * At this point we've copied the memory already. It's hard to check it
> + * before copying, as the size of actually copied buffer is unknown.
> + */
> +
> + /* copy_to_user() may copy zero bytes. No need to check. */
> + if (!to_copy)
> + return;
> + /* Or maybe copy_to_user() failed to copy anything. */
> + if (to_copy == left)
> + return;
> + if ((u64)to < TASK_SIZE) {
> + /* This is a user memory access, check it. */
> + kmsan_internal_check_memory((void *)from, to_copy - left, to,
> + REASON_COPY_TO_USER);
> + return;
> + }
> + /* Otherwise this is a kernel memory access. This happens when a compat
> + * syscall passes an argument allocated on the kernel stack to a real
> + * syscall.
> + * Don't check anything, just copy the shadow of the copied bytes.
> + */
> + shadow = kmsan_get_metadata((void *)to, to_copy - left, META_SHADOW);
> + if (shadow)
> + kmsan_memcpy_metadata((void *)to, (void *)from, to_copy - left);
> +}
> +EXPORT_SYMBOL(kmsan_copy_to_user);
> +
> +void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + /* The users may want to poison/unpoison random memory. */
> + kmsan_internal_poison_shadow((void *)address, size, flags,
> + KMSAN_POISON_NOCHECK);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_poison_shadow);
> +
> +void kmsan_unpoison_shadow(const void *address, size_t size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + /* The users may want to poison/unpoison random memory. */
> + kmsan_internal_unpoison_shadow((void *)address, size,
> + KMSAN_POISON_NOCHECK);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_unpoison_shadow);
> +
> +void kmsan_check_memory(const void *addr, size_t size)
> +{
> + return kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0,
> + REASON_ANY);
> +}
> +EXPORT_SYMBOL(kmsan_check_memory);
> +
> +void kmsan_gup_pgd_range(struct page **pages, int nr)
> +{
> + int i;
> + void *page_addr;
> +
> + /*
> + * gup_pgd_range() has just created a number of new pages that KMSAN
> + * treats as uninitialized. In the case they belong to the userspace
> + * memory, unpoison the corresponding kernel pages.
> + */
> + for (i = 0; i < nr; i++) {
> + page_addr = page_address(pages[i]);
> + if (((u64)page_addr < TASK_SIZE) &&
> + ((u64)page_addr + PAGE_SIZE < TASK_SIZE))
> + kmsan_unpoison_shadow(page_addr, PAGE_SIZE);
> + }
> +
> +}
> +EXPORT_SYMBOL(kmsan_gup_pgd_range);
> +
> +/* Helper function to check an SKB. */
> +void kmsan_check_skb(const struct sk_buff *skb)
> +{
> + int start = skb_headlen(skb);
> + struct sk_buff *frag_iter;
> + int i, copy = 0;
> + skb_frag_t *f;
> + u32 p_off, p_len, copied;
> + struct page *p;
> + u8 *vaddr;
> +
> + if (!skb || !skb->len)
> + return;
> +
> + kmsan_internal_check_memory(skb->data, skb_headlen(skb), 0, REASON_ANY);
> + if (skb_is_nonlinear(skb)) {
> + for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
> + f = &skb_shinfo(skb)->frags[i];
> +
> + skb_frag_foreach_page(f,
> + skb_frag_off(f) - start,
> + copy, p, p_off, p_len, copied) {
> +
> + vaddr = kmap_atomic(p);
> + kmsan_internal_check_memory(vaddr + p_off,
> + p_len, /*user_addr*/ 0,
> + REASON_ANY);
> + kunmap_atomic(vaddr);
> + }
> + }
> + }
> + skb_walk_frags(skb, frag_iter)
> + kmsan_check_skb(frag_iter);
> +}
> +EXPORT_SYMBOL(kmsan_check_skb);
> +
> +/* Helper function to check an URB. */
> +void kmsan_handle_urb(const struct urb *urb, bool is_out)
> +{
> + if (!urb)
> + return;
Here we also need to make sure that urb->setup_packet is fully
initialized for both is_out and !is_out requests.
> + if (is_out)
> + kmsan_internal_check_memory(urb->transfer_buffer,
> + urb->transfer_buffer_length,
> + /*user_addr*/ 0, REASON_SUBMIT_URB);
> + else
> + kmsan_internal_unpoison_shadow(urb->transfer_buffer,
> + urb->transfer_buffer_length,
> + /*checked*/false);
> +}
> +EXPORT_SYMBOL(kmsan_handle_urb);
> +
> +static void kmsan_handle_dma_page(const void *addr, size_t size,
> + enum dma_data_direction dir)
> +{
> + switch (dir) {
> + case DMA_BIDIRECTIONAL:
> + kmsan_internal_check_memory((void *)addr, size, /*user_addr*/0,
> + REASON_ANY);
> + kmsan_internal_unpoison_shadow((void *)addr, size,
> + /*checked*/false);
> + break;
> + case DMA_TO_DEVICE:
> + kmsan_internal_check_memory((void *)addr, size, /*user_addr*/0,
> + REASON_ANY);
> + break;
> + case DMA_FROM_DEVICE:
> + kmsan_internal_unpoison_shadow((void *)addr, size,
> + /*checked*/false);
> + break;
> + case DMA_NONE:
> + break;
> + }
> +}
> +
> +/* Helper function to handle DMA data transfers. */
> +void kmsan_handle_dma(const void *addr, size_t size,
> + enum dma_data_direction dir)
> +{
> + u64 page_offset, to_go, uaddr = (u64)addr;
> +
> + /*
> + * The kernel may occasionally give us adjacent DMA pages not belonging
> + * to the same allocation. Process them separately to avoid triggering
> + * internal KMSAN checks.
> + */
> + while (size > 0) {
> + page_offset = uaddr % PAGE_SIZE;
> + to_go = min(PAGE_SIZE - page_offset, (u64)size);
> + kmsan_handle_dma_page((void *)uaddr, to_go, dir);
> + uaddr += to_go;
> + size -= to_go;
> + }
> +}
> +EXPORT_SYMBOL(kmsan_handle_dma);
> diff --git a/mm/kmsan/kmsan_init.c b/mm/kmsan/kmsan_init.c
> new file mode 100644
> index 000000000000..2816e7075a30
> --- /dev/null
> +++ b/mm/kmsan/kmsan_init.c
> @@ -0,0 +1,88 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN initialization routines.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include "kmsan.h"
> +
> +#include <asm/cpu_entry_area.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +
> +#define NUM_FUTURE_RANGES 128
> +struct start_end_pair {
> + void *start, *end;
> +};
> +
> +static struct start_end_pair start_end_pairs[NUM_FUTURE_RANGES] __initdata;
> +static int future_index __initdata;
> +
> +/*
> + * Record a range of memory for which the metadata pages will be created once
> + * the page allocator becomes available.
> + * TODO(glider): squash together ranges belonging to the same page.
> + */
> +static void __init kmsan_record_future_shadow_range(void *start, void *end)
> +{
> + BUG_ON(future_index == NUM_FUTURE_RANGES);
> + BUG_ON((start >= end) || !start || !end);
> + start_end_pairs[future_index].start = start;
> + start_end_pairs[future_index].end = end;
> + future_index++;
> +}
> +
> +extern char _sdata[], _edata[];
> +
> +
> +
> +/*
> + * Initialize the shadow for existing mappings during kernel initialization.
> + * These include kernel text/data sections, NODE_DATA and future ranges
> + * registered while creating other data (e.g. percpu).
> + *
> + * Allocations via memblock can be only done before slab is initialized.
> + */
> +void __init kmsan_initialize_shadow(void)
> +{
> + int nid;
> + u64 i;
> + const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE);
> + phys_addr_t p_start, p_end;
> +
> + for_each_reserved_mem_region(i, &p_start, &p_end) {
> + kmsan_record_future_shadow_range(phys_to_virt(p_start),
> + phys_to_virt(p_end+1));
> + }
> + /* Allocate shadow for .data */
> + kmsan_record_future_shadow_range(_sdata, _edata);
> +
> + /*
> + * TODO(glider): alloc_node_data() in arch/x86/mm/numa.c uses
> + * sizeof(pg_data_t).
> + */
> + for_each_online_node(nid)
> + kmsan_record_future_shadow_range(
> + NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size);
> +
> + for (i = 0; i < future_index; i++)
> + kmsan_init_alloc_meta_for_range(start_end_pairs[i].start,
> + start_end_pairs[i].end);
> +}
> +EXPORT_SYMBOL(kmsan_initialize_shadow);
> +
> +void __init kmsan_initialize(void)
> +{
> + /* Assuming current is init_task */
> + kmsan_internal_task_create(current);
> + kmsan_pr_locked("Starting KernelMemorySanitizer\n");
> + kmsan_ready = true;
> +}
> +EXPORT_SYMBOL(kmsan_initialize);
> diff --git a/mm/kmsan/kmsan_instr.c b/mm/kmsan/kmsan_instr.c
> new file mode 100644
> index 000000000000..7695daf2d88a
> --- /dev/null
> +++ b/mm/kmsan/kmsan_instr.c
> @@ -0,0 +1,259 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN compiler API.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include "kmsan.h"
> +#include <linux/gfp.h>
> +#include <linux/mm.h>
> +
> +static bool is_bad_asm_addr(void *addr, u64 size, bool is_store)
> +{
> + if ((u64)addr < TASK_SIZE)
> + return true;
> + if (!kmsan_get_metadata(addr, size, META_SHADOW))
> + return true;
> + return false;
> +}
> +
> +struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, u64 size)
> +{
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false);
> +}
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n);
> +
> +struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, u64 size)
> +{
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true);
> +}
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n);
> +
> +#define DECLARE_METADATA_PTR_GETTER(size) \
> +struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size(void *addr) \
> +{ \
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); \
> +} \
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \
> + \
> +struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size(void *addr) \
> +{ \
> + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); \
> +} \
> +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size)
> +
> +DECLARE_METADATA_PTR_GETTER(1);
> +DECLARE_METADATA_PTR_GETTER(2);
> +DECLARE_METADATA_PTR_GETTER(4);
> +DECLARE_METADATA_PTR_GETTER(8);
> +
> +void __msan_instrument_asm_store(void *addr, u64 size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + /*
> + * Most of the accesses are below 32 bytes. The two exceptions so far
> + * are clwb() (64 bytes) and FPU state (512 bytes).
> + * It's unlikely that the assembly will touch more than 512 bytes.
> + */
> + if (size > 512)
> + size = 8;
> + if (is_bad_asm_addr(addr, size, /*is_store*/true))
> + return;
> + ENTER_RUNTIME(irq_flags);
> + /* Unpoisoning the memory on best effort. */
> + kmsan_internal_unpoison_shadow(addr, size, /*checked*/false);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_instrument_asm_store);
> +
> +void *__msan_memmove(void *dst, void *src, u64 n)
> +{
> + void *result;
> + void *shadow_dst;
> +
> + result = __memmove(dst, src, n);
> + if (!n)
> + /* Some people call memmove() with zero length. */
> + return result;
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + /* Ok to skip address check here, we'll do it later. */
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
> +
> + if (!shadow_dst)
> + /* Can happen e.g. if the memory is untracked. */
> + return result;
> +
> + kmsan_memmove_metadata(dst, src, n);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memmove);
> +
> +void *__msan_memmove_nosanitize(void *dst, void *src, u64 n)
> +{
> + return __memmove(dst, src, n);
> +}
> +EXPORT_SYMBOL(__msan_memmove_nosanitize);
> +
> +void *__msan_memcpy(void *dst, const void *src, u64 n)
> +{
> + void *result;
> + void *shadow_dst;
> +
> + result = __memcpy(dst, src, n);
> + if (!n)
> + /* Some people call memcpy() with zero length. */
> + return result;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + /* Ok to skip address check here, we'll do it later. */
> + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW);
> + if (!shadow_dst)
> + /* Can happen e.g. if the memory is untracked. */
> + return result;
> +
> + kmsan_memcpy_metadata(dst, (void *)src, n);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memcpy);
> +
> +void *__msan_memcpy_nosanitize(void *dst, void *src, u64 n)
> +{
> + return __memcpy(dst, src, n);
> +}
> +EXPORT_SYMBOL(__msan_memcpy_nosanitize);
> +
> +void *__msan_memset(void *dst, int c, size_t n)
> +{
> + void *result;
> + unsigned long irq_flags;
> + depot_stack_handle_t new_origin;
> + unsigned int shadow;
> +
> + result = __memset(dst, c, n);
> + if (!kmsan_ready || IN_RUNTIME())
> + return result;
> +
> + ENTER_RUNTIME(irq_flags);
> + shadow = 0;
> + kmsan_internal_memset_shadow(dst, shadow, n, /*checked*/false);
> + new_origin = 0;
> + kmsan_internal_set_origin(dst, n, new_origin);
> + LEAVE_RUNTIME(irq_flags);
> +
> + return result;
> +}
> +EXPORT_SYMBOL(__msan_memset);
> +
> +void *__msan_memset_nosanitize(void *dst, int c, size_t n)
> +{
> + return __memset(dst, c, n);
> +}
> +EXPORT_SYMBOL(__msan_memset_nosanitize);
> +
> +depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin)
> +{
> + depot_stack_handle_t ret = 0;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return ret;
> +
> + /* Creating new origins may allocate memory. */
> + ENTER_RUNTIME(irq_flags);
> + ret = kmsan_internal_chain_origin(origin);
> + LEAVE_RUNTIME(irq_flags);
> + return ret;
> +}
> +EXPORT_SYMBOL(__msan_chain_origin);
> +
> +void __msan_poison_alloca(void *address, u64 size, char *descr)
> +{
> + depot_stack_handle_t handle;
> + unsigned long entries[4];
> + unsigned long irq_flags;
> + u64 size_copy = size, to_fill;
> + u64 addr_copy = (u64)address;
> + u64 page_offset;
> + void *shadow_start;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + while (size_copy) {
> + page_offset = addr_copy % PAGE_SIZE;
> + to_fill = min(PAGE_SIZE - page_offset, size_copy);
> + shadow_start = kmsan_get_metadata((void *)addr_copy, to_fill,
> + META_SHADOW);
> + addr_copy += to_fill;
> + size_copy -= to_fill;
> + if (!shadow_start)
> + /* Can happen e.g. if the memory is untracked. */
> + continue;
> + __memset(shadow_start, -1, to_fill);
> + }
> +
> + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN;
> + entries[1] = (u64)descr;
> + entries[2] = (u64)__builtin_return_address(0);
> + entries[3] = (u64)kmsan_internal_return_address(1);
> +
> + /* stack_depot_save() may allocate memory. */
> + ENTER_RUNTIME(irq_flags);
> + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC);
> + LEAVE_RUNTIME(irq_flags);
> + kmsan_internal_set_origin(address, size, handle);
> +}
> +EXPORT_SYMBOL(__msan_poison_alloca);
> +
> +void __msan_unpoison_alloca(void *address, u64 size)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + /* Assuming the shadow exists. */
> + kmsan_internal_unpoison_shadow(address, size, /*checked*/true);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_unpoison_alloca);
> +
> +void __msan_warning(u32 origin)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + ENTER_RUNTIME(irq_flags);
> + kmsan_report(origin, /*address*/0, /*size*/0,
> + /*off_first*/0, /*off_last*/0, /*user_addr*/0, REASON_ANY);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(__msan_warning);
> +
> +struct kmsan_context_state *__msan_get_context_state(void)
> +{
> + struct kmsan_context_state *ret;
> +
> + ret = task_kmsan_context_state();
> + BUG_ON(!ret);
> + return ret;
> +}
> +EXPORT_SYMBOL(__msan_get_context_state);
> diff --git a/mm/kmsan/kmsan_report.c b/mm/kmsan/kmsan_report.c
> new file mode 100644
> index 000000000000..443ab9c1e8bf
> --- /dev/null
> +++ b/mm/kmsan/kmsan_report.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN error reporting routines.
> + *
> + * Copyright (C) 2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <linux/console.h>
> +#include <linux/stackdepot.h>
> +#include <linux/stacktrace.h>
> +
> +#include "kmsan.h"
> +
> +DEFINE_SPINLOCK(report_lock);
> +
> +void kmsan_print_origin(depot_stack_handle_t origin)
> +{
> + unsigned long *entries = NULL, *chained_entries = NULL;
> + unsigned long nr_entries, chained_nr_entries, magic;
> + char *descr = NULL;
> + void *pc1 = NULL, *pc2 = NULL;
> + depot_stack_handle_t head;
> +
> + if (!origin) {
> + kmsan_pr_err("Origin not found, presumably a false report.\n");
> + return;
> + }
> +
> + while (true) {
> + nr_entries = stack_depot_fetch(origin, &entries);
> + magic = nr_entries ? (entries[0] & KMSAN_MAGIC_MASK) : 0;
> + if ((nr_entries == 4) && (magic == KMSAN_ALLOCA_MAGIC_ORIGIN)) {
> + descr = (char *)entries[1];
> + pc1 = (void *)entries[2];
> + pc2 = (void *)entries[3];
> + kmsan_pr_err("Local variable description: %s\n", descr);
> + kmsan_pr_err("Variable was created at:\n");
> + kmsan_pr_err(" %pS\n", pc1);
> + kmsan_pr_err(" %pS\n", pc2);
> + break;
> + }
> + if ((nr_entries == 3) &&
> + (magic == KMSAN_CHAIN_MAGIC_ORIGIN_FULL)) {
> + head = entries[1];
> + origin = entries[2];
> + kmsan_pr_err("Uninit was stored to memory at:\n");
> + chained_nr_entries =
> + stack_depot_fetch(head, &chained_entries);
> + stack_trace_print(chained_entries, chained_nr_entries,
> + 0);
> + kmsan_pr_err("\n");
> + continue;
> + }
> + kmsan_pr_err("Uninit was created at:\n");
> + if (entries)
> + stack_trace_print(entries, nr_entries, 0);
> + else
> + kmsan_pr_err("No stack\n");
> + break;
> + }
> +}
> +
> +void kmsan_report(depot_stack_handle_t origin,
> + void *address, int size, int off_first, int off_last,
> + const void *user_addr, int reason)
> +{
> + unsigned long flags;
> + unsigned long *entries;
> + unsigned int nr_entries;
> + bool is_uaf = false;
> + char *bug_type = NULL;
> +
> + if (!kmsan_ready)
> + return;
> + if (!current->kmsan.allow_reporting)
> + return;
> + if (!origin)
> + return;
> +
> + nr_entries = stack_depot_fetch(origin, &entries);
> +
> + current->kmsan.allow_reporting = false;
> + spin_lock_irqsave(&report_lock, flags);
> + kmsan_pr_err("=====================================================\n");
> + if (get_dsh_extra_bits(origin) & 1)
> + is_uaf = true;
> + switch (reason) {
> + case REASON_ANY:
> + bug_type = is_uaf ? "use-after-free" : "uninit-value";
> + break;
> + case REASON_COPY_TO_USER:
> + bug_type = is_uaf ? "kernel-infoleak-after-free" :
> + "kernel-infoleak";
> + break;
> + case REASON_SUBMIT_URB:
> + bug_type = is_uaf ? "kernel-usb-infoleak-after-free" :
> + "kernel-usb-infoleak";
> + break;
> + }
> + kmsan_pr_err("BUG: KMSAN: %s in %pS\n",
> + bug_type, kmsan_internal_return_address(2));
> + dump_stack();
> + kmsan_pr_err("\n");
> +
> + kmsan_print_origin(origin);
> +
> + if (size) {
> + kmsan_pr_err("\n");
> + if (off_first == off_last)
> + kmsan_pr_err("Byte %d of %d is uninitialized\n",
> + off_first, size);
> + else
> + kmsan_pr_err("Bytes %d-%d of %d are uninitialized\n",
> + off_first, off_last, size);
> + }
> + if (address)
> + kmsan_pr_err("Memory access of size %d starts at %px\n",
> + size, address);
> + if (user_addr && reason == REASON_COPY_TO_USER)
> + kmsan_pr_err("Data copied to user address %px\n", user_addr);
> + kmsan_pr_err("=====================================================\n");
> + add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
> + spin_unlock_irqrestore(&report_lock, flags);
> + if (panic_on_warn)
> + panic("panic_on_warn set ...\n");
> + current->kmsan.allow_reporting = true;
> +}
> diff --git a/mm/kmsan/kmsan_shadow.c b/mm/kmsan/kmsan_shadow.c
> new file mode 100644
> index 000000000000..06801d76e6b8
> --- /dev/null
> +++ b/mm/kmsan/kmsan_shadow.c
> @@ -0,0 +1,543 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * KMSAN shadow implementation.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <asm/cpu_entry_area.h>
> +#include <asm/page.h>
> +#include <asm/pgtable_64_types.h>
> +#include <asm/tlbflush.h>
> +#include <linux/memblock.h>
> +#include <linux/mm_types.h>
> +#include <linux/percpu-defs.h>
> +#include <linux/slab.h>
> +#include <linux/smp.h>
> +#include <linux/stddef.h>
> +
> +#include "kmsan.h"
> +#include "kmsan_shadow.h"
> +
> +#define shadow_page_for(page) \
> + ((page)->shadow)
> +
> +#define origin_page_for(page) \
> + ((page)->origin)
> +
> +#define shadow_ptr_for(page) \
> + (page_address((page)->shadow))
> +
> +#define origin_ptr_for(page) \
> + (page_address((page)->origin))
> +
> +#define has_shadow_page(page) \
> + (!!((page)->shadow))
> +
> +#define has_origin_page(page) \
> + (!!((page)->origin))
> +
> +#define set_no_shadow_origin_page(page) \
> + do { \
> + (page)->shadow = NULL; \
> + (page)->origin = NULL; \
> + } while (0) /**/
> +
> +#define is_ignored_page(page) \
> + (!!(((u64)((page)->shadow)) % 2))
> +
> +#define ignore_page(pg) \
> + ((pg)->shadow = (struct page *)((u64)((pg)->shadow) | 1)) \
> +
> +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow);
> +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin);
> +
> +/*
> + * Dummy load and store pages to be used when the real metadata is unavailable.
> + * There are separate pages for loads and stores, so that every load returns a
> + * zero, and every store doesn't affect other stores.
> + */
> +char dummy_load_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +char dummy_store_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +
> +/*
> + * Taken from arch/x86/mm/physaddr.h to avoid using an instrumented version.
> + */
> +static int kmsan_phys_addr_valid(unsigned long addr)
> +{
> +#ifdef CONFIG_PHYS_ADDR_T_64BIT
> + return !(addr >> boot_cpu_data.x86_phys_bits);
> +#else
> + return 1;
> +#endif
> +}
> +
> +/*
> + * Taken from arch/x86/mm/physaddr.c to avoid using an instrumented version.
> + */
> +static bool kmsan_virt_addr_valid(void *addr)
> +{
> + unsigned long x = (unsigned long)addr;
> + unsigned long y = x - __START_KERNEL_map;
> +
> + /* use the carry flag to determine if x was < __START_KERNEL_map */
> + if (unlikely(x > y)) {
> + x = y + phys_base;
> +
> + if (y >= KERNEL_IMAGE_SIZE)
> + return false;
> + } else {
> + x = y + (__START_KERNEL_map - PAGE_OFFSET);
> +
> + /* carry flag will be set if starting x was >= PAGE_OFFSET */
> + if ((x > y) || !kmsan_phys_addr_valid(x))
> + return false;
> + }
> +
> + return pfn_valid(x >> PAGE_SHIFT);
> +}
> +
> +static unsigned long vmalloc_meta(void *addr, bool is_origin)
> +{
> + unsigned long addr64 = (unsigned long)addr, off;
> +
> + BUG_ON(is_origin && !IS_ALIGNED(addr64, ORIGIN_SIZE));
> + if (kmsan_internal_is_vmalloc_addr(addr)) {
> + return addr64 + (is_origin ? VMALLOC_ORIGIN_OFFSET
> + : VMALLOC_SHADOW_OFFSET);
> + }
> + if (kmsan_internal_is_module_addr(addr)) {
> + off = addr64 - MODULES_VADDR;
> + return off + (is_origin ? MODULES_ORIGIN_START
> + : MODULES_SHADOW_START);
> + }
> + return 0;
> +}
> +
> +static void *get_cea_meta_or_null(void *addr, bool is_origin)
> +{
> + int cpu = smp_processor_id();
> + int off;
> + char *metadata_array;
> +
> + if (((u64)addr < CPU_ENTRY_AREA_BASE) ||
> + ((u64)addr >= (CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE)))
> + return NULL;
> + off = (char *)addr - (char *)get_cpu_entry_area(cpu);
> + if ((off < 0) || (off >= CPU_ENTRY_AREA_SIZE))
> + return NULL;
> + metadata_array = is_origin ? cpu_entry_area_origin :
> + cpu_entry_area_shadow;
> + return &per_cpu(metadata_array[off], cpu);
> +}
> +
> +static struct page *virt_to_page_or_null(void *vaddr)
> +{
> + if (kmsan_virt_addr_valid(vaddr))
> + return virt_to_page(vaddr);
> + else
> + return NULL;
> +}
> +
> +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *address, u64 size,
> + bool store)
> +{
> + struct shadow_origin_ptr ret;
> + struct page *page;
> + u64 pad, offset, o_offset;
> + const u64 addr64 = (u64)address;
> + u64 o_addr64 = (u64)address;
> + void *shadow;
> +
> + if (size > PAGE_SIZE) {
> + WARN(1, "size too big in %s(%px, %d, %d)\n",
> + __func__, address, size, store);
> + BUG();
> + }
> + if (store) {
> + ret.s = dummy_store_page;
> + ret.o = dummy_store_page;
> + } else {
> + ret.s = dummy_load_page;
> + ret.o = dummy_load_page;
> + }
> + if (!kmsan_ready || IN_RUNTIME())
> + return ret;
> + BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW));
> +
> + if (!IS_ALIGNED(addr64, ORIGIN_SIZE)) {
> + pad = addr64 % ORIGIN_SIZE;
> + o_addr64 -= pad;
> + }
> +
> + if (kmsan_internal_is_vmalloc_addr(address) ||
> + kmsan_internal_is_module_addr(address)) {
> + ret.s = (void *)vmalloc_meta(address, META_SHADOW);
> + ret.o = (void *)vmalloc_meta((void *)o_addr64, META_ORIGIN);
> + return ret;
> + }
> +
> + if (!kmsan_virt_addr_valid(address)) {
> + page = vmalloc_to_page_or_null(address);
> + if (page)
> + goto next;
> + shadow = get_cea_meta_or_null(address, META_SHADOW);
> + if (shadow) {
> + ret.s = shadow;
> + ret.o = get_cea_meta_or_null((void *)o_addr64,
> + META_ORIGIN);
> + return ret;
> + }
> + }
> + page = virt_to_page_or_null(address);
> + if (!page)
> + return ret;
> +next:
> + if (is_ignored_page(page))
> + return ret;
> +
> + if (!has_shadow_page(page) || !has_origin_page(page))
> + return ret;
> + offset = addr64 % PAGE_SIZE;
> + o_offset = o_addr64 % PAGE_SIZE;
> +
> + if (offset + size - 1 > PAGE_SIZE) {
> + /*
> + * The access overflows the current page and touches the
> + * subsequent ones. Make sure the shadow/origin pages are also
> + * consequent.
> + */
> + BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW));
> + }
> +
> + ret.s = shadow_ptr_for(page) + offset;
> + ret.o = origin_ptr_for(page) + o_offset;
> + return ret;
> +}
> +
> +/*
> + * Obtain the shadow or origin pointer for the given address, or NULL if there's
> + * none. The caller must check the return value for being non-NULL if needed.
> + * The return value of this function should not depend on whether we're in the
> + * runtime or not.
> + */
> +void *kmsan_get_metadata(void *address, size_t size, bool is_origin)
> +{
> + struct page *page;
> + void *ret;
> + u64 addr = (u64)address, pad, off;
> +
> + if (is_origin && !IS_ALIGNED(addr, ORIGIN_SIZE)) {
> + pad = addr % ORIGIN_SIZE;
> + addr -= pad;
> + size += pad;
> + }
> + address = (void *)addr;
> + if (kmsan_internal_is_vmalloc_addr(address) ||
> + kmsan_internal_is_module_addr(address)) {
> + return (void *)vmalloc_meta(address, is_origin);
> + }
> +
> + if (!kmsan_virt_addr_valid(address)) {
> + page = vmalloc_to_page_or_null(address);
> + if (page)
> + goto next;
> + ret = get_cea_meta_or_null(address, is_origin);
> + if (ret)
> + return ret;
> + }
> + page = virt_to_page_or_null(address);
> + if (!page)
> + return NULL;
> +next:
> + if (is_ignored_page(page))
> + return NULL;
> + if (!has_shadow_page(page) || !has_origin_page(page))
> + return NULL;
> + off = addr % PAGE_SIZE;
> +
> + ret = (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off;
> + return ret;
> +}
> +
> +void __init kmsan_init_alloc_meta_for_range(void *start, void *end)
> +{
> + u64 addr, size;
> + struct page *page;
> + void *shadow, *origin;
> + struct page *shadow_p, *origin_p;
> +
> + start = (void *)ALIGN_DOWN((u64)start, PAGE_SIZE);
> + size = ALIGN((u64)end - (u64)start, PAGE_SIZE);
> + shadow = memblock_alloc(size, PAGE_SIZE);
> + origin = memblock_alloc(size, PAGE_SIZE);
> + for (addr = 0; addr < size; addr += PAGE_SIZE) {
> + page = virt_to_page_or_null((char *)start + addr);
> + shadow_p = virt_to_page_or_null((char *)shadow + addr);
> + set_no_shadow_origin_page(shadow_p);
> + shadow_page_for(page) = shadow_p;
> + origin_p = virt_to_page_or_null((char *)origin + addr);
> + set_no_shadow_origin_page(origin_p);
> + origin_page_for(page) = origin_p;
> + }
> +}
> +
> +/* Called from mm/memory.c */
> +void kmsan_copy_page_meta(struct page *dst, struct page *src)
> +{
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + if (!has_shadow_page(src)) {
> + /* TODO(glider): are we leaking pages here? */
> + set_no_shadow_origin_page(dst);
> + return;
> + }
> + if (!has_shadow_page(dst))
> + return;
> + if (is_ignored_page(src)) {
> + ignore_page(dst);
> + return;
> + }
> +
> + ENTER_RUNTIME(irq_flags);
> + __memcpy(shadow_ptr_for(dst), shadow_ptr_for(src),
> + PAGE_SIZE);
> + BUG_ON(!has_origin_page(src) || !has_origin_page(dst));
> + __memcpy(origin_ptr_for(dst), origin_ptr_for(src),
> + PAGE_SIZE);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_copy_page_meta);
> +
> +/* Helper function to allocate page metadata. */
> +static int kmsan_internal_alloc_meta_for_pages(struct page *page,
> + unsigned int order,
> + gfp_t flags, int node)
> +{
> + struct page *shadow, *origin;
> + int pages = 1 << order;
> + int i;
> + bool initialized = (flags & __GFP_ZERO) || !kmsan_ready;
> + depot_stack_handle_t handle;
> +
> + if (flags & __GFP_NO_KMSAN_SHADOW) {
> + for (i = 0; i < pages; i++)
> + set_no_shadow_origin_page(&page[i]);
> + return 0;
> + }
> +
> + /* TODO(glider): must we override the flags? */
> + flags = GFP_ATOMIC;
> + if (initialized)
> + flags |= __GFP_ZERO;
> + shadow = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order);
> + if (!shadow) {
> + for (i = 0; i < pages; i++) {
> + set_no_shadow_origin_page(&page[i]);
> + set_no_shadow_origin_page(&page[i]);
> + }
> + return -ENOMEM;
> + }
> + if (!initialized)
> + __memset(page_address(shadow), -1, PAGE_SIZE * pages);
> +
> + origin = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order);
> + /* Assume we've allocated the origin. */
> + if (!origin) {
> + __free_pages(shadow, order);
> + for (i = 0; i < pages; i++)
> + set_no_shadow_origin_page(&page[i]);
> + return -ENOMEM;
> + }
> +
> + if (!initialized) {
> + handle = kmsan_save_stack_with_flags(flags, /*extra_bits*/0);
> + /*
> + * Addresses are page-aligned, pages are contiguous, so it's ok
> + * to just fill the origin pages with |handle|.
> + */
> + for (i = 0; i < PAGE_SIZE * pages / sizeof(handle); i++) {
> + ((depot_stack_handle_t *)page_address(origin))[i] =
> + handle;
> + }
> + }
> +
> + for (i = 0; i < pages; i++) {
> + shadow_page_for(&page[i]) = &shadow[i];
> + set_no_shadow_origin_page(shadow_page_for(&page[i]));
> + origin_page_for(&page[i]) = &origin[i];
> + set_no_shadow_origin_page(origin_page_for(&page[i]));
> + }
> + return 0;
> +}
> +
> +/* Called from mm/page_alloc.c */
> +int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags)
> +{
> + unsigned long irq_flags;
> + int ret;
> +
> + if (IN_RUNTIME())
> + return 0;
> + ENTER_RUNTIME(irq_flags);
> + ret = kmsan_internal_alloc_meta_for_pages(page, order, flags, -1);
> + LEAVE_RUNTIME(irq_flags);
> + return ret;
> +}
> +
> +/* Called from mm/page_alloc.c */
> +void kmsan_free_page(struct page *page, unsigned int order)
> +{
> + struct page *shadow, *origin, *cur_page;
> + int pages = 1 << order;
> + int i;
> + unsigned long irq_flags;
> +
> + if (!shadow_page_for(page)) {
> + for (i = 0; i < pages; i++) {
> + cur_page = &page[i];
> + BUG_ON(shadow_page_for(cur_page));
> + }
> + return;
> + }
> +
> + if (!kmsan_ready) {
> + for (i = 0; i < pages; i++) {
> + cur_page = &page[i];
> + set_no_shadow_origin_page(cur_page);
> + }
> + return;
> + }
> +
> + if (IN_RUNTIME()) {
> + /*
> + * TODO(glider): looks legit. depot_save_stack() may call
> + * free_pages().
> + */
> + return;
> + }
> +
> + ENTER_RUNTIME(irq_flags);
> + shadow = shadow_page_for(&page[0]);
> + origin = origin_page_for(&page[0]);
> +
> + /* TODO(glider): this is racy. */
> + for (i = 0; i < pages; i++) {
> + BUG_ON(has_shadow_page(shadow_page_for(&page[i])));
> + BUG_ON(has_shadow_page(origin_page_for(&page[i])));
> + set_no_shadow_origin_page(&page[i]);
> + }
> + BUG_ON(has_shadow_page(shadow));
> + __free_pages(shadow, order);
> +
> + BUG_ON(has_shadow_page(origin));
> + __free_pages(origin, order);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_free_page);
> +
> +/* Called from mm/page_alloc.c */
> +void kmsan_split_page(struct page *page, unsigned int order)
> +{
> + struct page *shadow, *origin;
> + unsigned long irq_flags;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> +
> + ENTER_RUNTIME(irq_flags);
> + if (!has_shadow_page(&page[0])) {
> + BUG_ON(has_origin_page(&page[0]));
> + LEAVE_RUNTIME(irq_flags);
> + return;
> + }
> + shadow = shadow_page_for(&page[0]);
> + split_page(shadow, order);
> +
> + origin = origin_page_for(&page[0]);
> + split_page(origin, order);
> + LEAVE_RUNTIME(irq_flags);
> +}
> +EXPORT_SYMBOL(kmsan_split_page);
> +
> +/* Called from include/linux/highmem.h */
> +void kmsan_clear_page(void *page_addr)
> +{
> + struct page *page;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + BUG_ON(!IS_ALIGNED((u64)page_addr, PAGE_SIZE));
> + page = vmalloc_to_page_or_null(page_addr);
> + if (!page)
> + page = virt_to_page_or_null(page_addr);
> + if (!page || !has_shadow_page(page))
> + return;
> + __memset(shadow_ptr_for(page), 0, PAGE_SIZE);
> + BUG_ON(!has_origin_page(page));
> + __memset(origin_ptr_for(page), 0, PAGE_SIZE);
> +}
> +EXPORT_SYMBOL(kmsan_clear_page);
> +
> +/* Called from mm/vmalloc.c */
> +void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages)
> +{
> + int nr, i, mapped;
> + struct page **s_pages, **o_pages;
> + unsigned long shadow_start, shadow_end, origin_start, origin_end;
> +
> + if (!kmsan_ready || IN_RUNTIME())
> + return;
> + shadow_start = vmalloc_meta((void *)start, META_SHADOW);
> + if (!shadow_start)
> + return;
> +
> + BUG_ON(start >= end);
> + nr = (end - start) / PAGE_SIZE;
> + s_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL);
> + o_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL);
> + if (!s_pages || !o_pages)
> + goto ret;
> + for (i = 0; i < nr; i++) {
> + s_pages[i] = shadow_page_for(pages[i]);
> + o_pages[i] = origin_page_for(pages[i]);
> + }
> + prot = __pgprot(pgprot_val(prot) | _PAGE_NX);
> + prot = PAGE_KERNEL;
> +
> + shadow_end = vmalloc_meta((void *)end, META_SHADOW);
> + origin_start = vmalloc_meta((void *)start, META_ORIGIN);
> + origin_end = vmalloc_meta((void *)end, META_ORIGIN);
> + mapped = __vmap_page_range_noflush(shadow_start, shadow_end,
> + prot, s_pages);
> + BUG_ON(mapped != nr);
> + flush_tlb_kernel_range(shadow_start, shadow_end);
> + mapped = __vmap_page_range_noflush(origin_start, origin_end,
> + prot, o_pages);
> + BUG_ON(mapped != nr);
> + flush_tlb_kernel_range(origin_start, origin_end);
> +ret:
> + kfree(s_pages);
> + kfree(o_pages);
> +}
> +
> +void kmsan_ignore_page(struct page *page, int order)
> +{
> + int pages = 1 << order;
> + int i;
> + struct page *cp;
> +
> + for (i = 0; i < pages; i++) {
> + cp = &page[i];
> + ignore_page(cp);
> + }
> +}
> diff --git a/mm/kmsan/kmsan_shadow.h b/mm/kmsan/kmsan_shadow.h
> new file mode 100644
> index 000000000000..eaa7f771b6a5
> --- /dev/null
> +++ b/mm/kmsan/kmsan_shadow.h
> @@ -0,0 +1,30 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * KMSAN shadow API.
> + *
> + * This should be agnostic to shadow implementation details.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#ifndef __MM_KMSAN_KMSAN_SHADOW_H
> +#define __MM_KMSAN_KMSAN_SHADOW_H
> +
> +#include <asm/cpu_entry_area.h> /* for CPU_ENTRY_AREA_MAP_SIZE */
> +
> +struct shadow_origin_ptr {
> + void *s, *o;
> +};
> +
> +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size,
> + bool store);
> +void *kmsan_get_metadata(void *addr, size_t size, bool is_origin);
> +void __init kmsan_init_alloc_meta_for_range(void *start, void *end);
> +
> +#endif /* __MM_KMSAN_KMSAN_SHADOW_H */
> diff --git a/scripts/Makefile.kmsan b/scripts/Makefile.kmsan
> new file mode 100644
> index 000000000000..8b3844b66b22
> --- /dev/null
> +++ b/scripts/Makefile.kmsan
> @@ -0,0 +1,12 @@
> +ifdef CONFIG_KMSAN
> +
> +CFLAGS_KMSAN := -fsanitize=kernel-memory
> +
> +ifeq ($(call cc-option, $(CFLAGS_KMSAN) -Werror),)
> + ifneq ($(CONFIG_COMPILE_TEST),y)
> + $(warning Cannot use CONFIG_KMSAN: \
> + -fsanitize=kernel-memory is not supported by compiler)
> + endif
> +endif
> +
> +endif
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 11/36] kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (9 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 10/36] kmsan: add KMSAN runtime glider
@ 2019-11-22 11:25 ` glider
2019-11-29 14:52 ` Andrey Konovalov
2019-11-22 11:25 ` [PATCH RFC v3 12/36] kmsan: define READ_ONCE_NOCHECK() glider
` (24 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Andrey Ryabinin, Jens Axboe, Andy Lutomirski, Vegard Nossum,
Dmitry Vyukov, Christoph Hellwig, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, ard.biesheuvel,
arnd, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
edumazet, ericvh, gregkh, harry.wentland, herbert, iii, mingo,
jasowang, m.szyprowski, elver, mark.rutland, martin.petersen,
schwidefsky, willy, mst, monstr, pmladek, cai, rdunlap,
robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso, tglx,
gor, wsa
We assume nothing interesting could happen in stackdepot, and save some
memory by not tracking stackdepot allocations with KMSAN.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: linux-mm@kvack.org
---
Change-Id: Ic3ec9b3dff3fff2732d874508a3582fb26ff0b1f
---
lib/stackdepot.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index e2f000a9fad8..2b053c429454 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -294,7 +294,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
* contexts and I/O.
*/
alloc_flags &= ~GFP_ZONEMASK;
- alloc_flags &= (GFP_ATOMIC | GFP_KERNEL);
+ alloc_flags &= (GFP_ATOMIC | GFP_KERNEL |
+ __GFP_NO_KMSAN_SHADOW);
alloc_flags |= __GFP_NOWARN;
page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER);
if (page)
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 11/36] kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot
2019-11-22 11:25 ` [PATCH RFC v3 11/36] kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot glider
@ 2019-11-29 14:52 ` Andrey Konovalov
2019-12-03 14:27 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 14:52 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Andrey Ryabinin, Jens Axboe, Andy Lutomirski, Vegard Nossum,
Dmitry Vyukov, Christoph Hellwig, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Ard Biesheuvel,
Arnd Bergmann, Christoph Hellwig, darrick.wong, David S. Miller,
Dmitry Torokhov, ebiggers, Eric Dumazet, ericvh,
Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii, mingo,
Jason Wang, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, pmladek, Qian Cai,
Randy Dunlap, Robin Murphy, sergey.senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> We assume nothing interesting could happen in stackdepot, and save some
> memory by not tracking stackdepot allocations with KMSAN.
I think it makes sense to clarify here that "nothing interesting"
means no uninitialized memory can come from stackdepot or something
like that.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: Ic3ec9b3dff3fff2732d874508a3582fb26ff0b1f
> ---
> lib/stackdepot.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> index e2f000a9fad8..2b053c429454 100644
> --- a/lib/stackdepot.c
> +++ b/lib/stackdepot.c
> @@ -294,7 +294,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
> * contexts and I/O.
> */
> alloc_flags &= ~GFP_ZONEMASK;
> - alloc_flags &= (GFP_ATOMIC | GFP_KERNEL);
> + alloc_flags &= (GFP_ATOMIC | GFP_KERNEL |
> + __GFP_NO_KMSAN_SHADOW);
> alloc_flags |= __GFP_NOWARN;
> page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER);
> if (page)
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 11/36] kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot
2019-11-29 14:52 ` Andrey Konovalov
@ 2019-12-03 14:27 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 14:27 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Andrey Ryabinin, Jens Axboe, Andy Lutomirski, Vegard Nossum,
Dmitry Vyukov, Christoph Hellwig, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Ard Biesheuvel,
Arnd Bergmann, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Marek Szyprowski, Marco Elver, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S . Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, Nov 29, 2019 at 3:52 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > We assume nothing interesting could happen in stackdepot, and save some
> > memory by not tracking stackdepot allocations with KMSAN.
>
> I think it makes sense to clarify here that "nothing interesting"
> means no uninitialized memory can come from stackdepot or something
> like that.
Agreed. Will update the patch description.
I've also noticed this patch does nothing to ensure the allocations
are performed with __GFP_NO_KMSAN_SHADOW.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> > Cc: Jens Axboe <axboe@kernel.dk>
> > Cc: Andy Lutomirski <luto@kernel.org>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: Ic3ec9b3dff3fff2732d874508a3582fb26ff0b1f
> > ---
> > lib/stackdepot.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/stackdepot.c b/lib/stackdepot.c
> > index e2f000a9fad8..2b053c429454 100644
> > --- a/lib/stackdepot.c
> > +++ b/lib/stackdepot.c
> > @@ -294,7 +294,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries,
> > * contexts and I/O.
> > */
> > alloc_flags &= ~GFP_ZONEMASK;
> > - alloc_flags &= (GFP_ATOMIC | GFP_KERNEL);
> > + alloc_flags &= (GFP_ATOMIC | GFP_KERNEL |
> > + __GFP_NO_KMSAN_SHADOW);
> > alloc_flags |= __GFP_NOWARN;
Should have added __GFP_NO_KMSAN_SHADOW here.
> > page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER);
> > if (page)
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 12/36] kmsan: define READ_ONCE_NOCHECK()
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (10 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 11/36] kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot glider
@ 2019-11-22 11:25 ` glider
2019-12-02 10:03 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 13/36] kmsan: make READ_ONCE_TASK_STACK() return initialized values glider
` (23 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Mark Rutland, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, martin.petersen, schwidefsky, willy, mst,
monstr, pmladek, cai, rdunlap, robin.murphy, sergey.senozhatsky,
rostedt, tiwai, tytso, tglx, gor, wsa
READ_ONCE_NOCHECK() is already used by KASAN to ignore memory accesses
from e.g. stack unwinders.
Define READ_ONCE_NOCHECK() for KMSAN so that it returns initialized
values. This helps defeat false positives from leftover stack contents.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v3:
- removed unnecessary #ifdef as requested by Mark Rutland
Change-Id: Ib38369ba038ab3b581d8e45b81036c3304fb79cb
---
include/linux/compiler.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 5e88e7e33abe..99d40f31a2c3 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -270,9 +270,9 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s
/*
* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need
- * to hide memory access from KASAN.
+ * to hide memory access from KASAN or KMSAN.
*/
-#define READ_ONCE_NOCHECK(x) __READ_ONCE(x, 0)
+#define READ_ONCE_NOCHECK(x) KMSAN_INIT_VALUE(__READ_ONCE(x, 0))
static __no_kasan_or_inline
unsigned long read_word_at_a_time(const void *addr)
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 12/36] kmsan: define READ_ONCE_NOCHECK()
2019-11-22 11:25 ` [PATCH RFC v3 12/36] kmsan: define READ_ONCE_NOCHECK() glider
@ 2019-12-02 10:03 ` Marco Elver
2019-12-03 12:45 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-02 10:03 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Mark Rutland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, adilger.kernel,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann, hch,
Christoph Hellwig, darrick.wong, davem, dmitry.torokhov,
Eric Biggers, Eric Dumazet, ericvh, gregkh, harry.wentland,
herbert, iii, mingo, jasowang, Jens Axboe, m.szyprowski,
martin.petersen, schwidefsky, Matthew Wilcox, mst, Michal Simek,
pmladek, Qian Cai, Randy Dunlap, robin.murphy,
Sergey Senozhatsky, Steven Rostedt, tiwai, tytso,
Thomas Gleixner, gor, wsa
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> READ_ONCE_NOCHECK() is already used by KASAN to ignore memory accesses
> from e.g. stack unwinders.
> Define READ_ONCE_NOCHECK() for KMSAN so that it returns initialized
> values. This helps defeat false positives from leftover stack contents.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
> v3:
> - removed unnecessary #ifdef as requested by Mark Rutland
>
> Change-Id: Ib38369ba038ab3b581d8e45b81036c3304fb79cb
> ---
> include/linux/compiler.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index 5e88e7e33abe..99d40f31a2c3 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -270,9 +270,9 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s
>
> /*
> * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need
> - * to hide memory access from KASAN.
> + * to hide memory access from KASAN or KMSAN.
> */
> -#define READ_ONCE_NOCHECK(x) __READ_ONCE(x, 0)
> +#define READ_ONCE_NOCHECK(x) KMSAN_INIT_VALUE(__READ_ONCE(x, 0))
I think this needs:
#include <linux/kmsan-checks.h>
above.
> static __no_kasan_or_inline
> unsigned long read_word_at_a_time(const void *addr)
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 12/36] kmsan: define READ_ONCE_NOCHECK()
2019-12-02 10:03 ` Marco Elver
@ 2019-12-03 12:45 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 12:45 UTC (permalink / raw)
To: Marco Elver
Cc: Mark Rutland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Martin K . Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 11:03 AM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
> >
> > READ_ONCE_NOCHECK() is already used by KASAN to ignore memory accesses
> > from e.g. stack unwinders.
> > Define READ_ONCE_NOCHECK() for KMSAN so that it returns initialized
> > values. This helps defeat false positives from leftover stack contents.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Mark Rutland <mark.rutland@arm.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> > v3:
> > - removed unnecessary #ifdef as requested by Mark Rutland
> >
> > Change-Id: Ib38369ba038ab3b581d8e45b81036c3304fb79cb
> > ---
> > include/linux/compiler.h | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> > index 5e88e7e33abe..99d40f31a2c3 100644
> > --- a/include/linux/compiler.h
> > +++ b/include/linux/compiler.h
> > @@ -270,9 +270,9 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s
> >
> > /*
> > * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need
> > - * to hide memory access from KASAN.
> > + * to hide memory access from KASAN or KMSAN.
> > */
> > -#define READ_ONCE_NOCHECK(x) __READ_ONCE(x, 0)
> > +#define READ_ONCE_NOCHECK(x) KMSAN_INIT_VALUE(__READ_ONCE(x, 0))
>
> I think this needs:
> #include <linux/kmsan-checks.h>
> above.
Ack, will include in v4.
> > static __no_kasan_or_inline
> > unsigned long read_word_at_a_time(const void *addr)
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 13/36] kmsan: make READ_ONCE_TASK_STACK() return initialized values
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (11 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 12/36] kmsan: define READ_ONCE_NOCHECK() glider
@ 2019-11-22 11:25 ` glider
2019-12-02 10:07 ` Marco Elver
2019-11-22 11:25 ` [PATCH RFC v3 14/36] kmsan: x86: sync metadata pages on page fault glider
` (22 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
To avoid false positives, assume that reading from the task stack
always produces initialized values.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: Ie73e5a41fdc8195699928e65f5cbe0d3d3c9e2fa
---
arch/x86/include/asm/unwind.h | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/arch/x86/include/asm/unwind.h b/arch/x86/include/asm/unwind.h
index 499578f7e6d7..f60c2bd1ddf2 100644
--- a/arch/x86/include/asm/unwind.h
+++ b/arch/x86/include/asm/unwind.h
@@ -100,9 +100,10 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
#endif
/*
- * This disables KASAN checking when reading a value from another task's stack,
- * since the other task could be running on another CPU and could have poisoned
- * the stack in the meantime.
+ * This disables KASAN/KMSAN checking when reading a value from another task's
+ * stack, since the other task could be running on another CPU and could have
+ * poisoned the stack in the meantime. Frame pointers are uninitialized by
+ * default, so for KMSAN we mark the return value initialized unconditionally.
*/
#define READ_ONCE_TASK_STACK(task, x) \
({ \
@@ -111,7 +112,7 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
val = READ_ONCE(x); \
else \
val = READ_ONCE_NOCHECK(x); \
- val; \
+ KMSAN_INIT_VALUE(val); \
})
static inline bool task_on_another_cpu(struct task_struct *task)
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 13/36] kmsan: make READ_ONCE_TASK_STACK() return initialized values
2019-11-22 11:25 ` [PATCH RFC v3 13/36] kmsan: make READ_ONCE_TASK_STACK() return initialized values glider
@ 2019-12-02 10:07 ` Marco Elver
2019-12-05 15:52 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-02 10:07 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> To avoid false positives, assume that reading from the task stack
> always produces initialized values.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
Acked-by: Marco Elver <elver@google.com>
assuming previous patch's include for kmsan-checks.h is added.
> ---
>
> Change-Id: Ie73e5a41fdc8195699928e65f5cbe0d3d3c9e2fa
> ---
> arch/x86/include/asm/unwind.h | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/include/asm/unwind.h b/arch/x86/include/asm/unwind.h
> index 499578f7e6d7..f60c2bd1ddf2 100644
> --- a/arch/x86/include/asm/unwind.h
> +++ b/arch/x86/include/asm/unwind.h
> @@ -100,9 +100,10 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
> #endif
>
> /*
> - * This disables KASAN checking when reading a value from another task's stack,
> - * since the other task could be running on another CPU and could have poisoned
> - * the stack in the meantime.
> + * This disables KASAN/KMSAN checking when reading a value from another task's
> + * stack, since the other task could be running on another CPU and could have
> + * poisoned the stack in the meantime. Frame pointers are uninitialized by
> + * default, so for KMSAN we mark the return value initialized unconditionally.
> */
> #define READ_ONCE_TASK_STACK(task, x) \
> ({ \
> @@ -111,7 +112,7 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
> val = READ_ONCE(x); \
> else \
> val = READ_ONCE_NOCHECK(x); \
> - val; \
> + KMSAN_INIT_VALUE(val); \
> })
>
> static inline bool task_on_another_cpu(struct task_struct *task)
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 13/36] kmsan: make READ_ONCE_TASK_STACK() return initialized values
2019-12-02 10:07 ` Marco Elver
@ 2019-12-05 15:52 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 15:52 UTC (permalink / raw)
To: Marco Elver
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 11:08 AM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
> >
> > To avoid false positives, assume that reading from the task stack
> > always produces initialized values.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
>
> Acked-by: Marco Elver <elver@google.com>
>
> assuming previous patch's include for kmsan-checks.h is added.
Yes, I'll add that include here as well in v4.
> > ---
> >
> > Change-Id: Ie73e5a41fdc8195699928e65f5cbe0d3d3c9e2fa
> > ---
> > arch/x86/include/asm/unwind.h | 9 +++++----
> > 1 file changed, 5 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/x86/include/asm/unwind.h b/arch/x86/include/asm/unwind.h
> > index 499578f7e6d7..f60c2bd1ddf2 100644
> > --- a/arch/x86/include/asm/unwind.h
> > +++ b/arch/x86/include/asm/unwind.h
> > @@ -100,9 +100,10 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
> > #endif
> >
> > /*
> > - * This disables KASAN checking when reading a value from another task's stack,
> > - * since the other task could be running on another CPU and could have poisoned
> > - * the stack in the meantime.
> > + * This disables KASAN/KMSAN checking when reading a value from another task's
> > + * stack, since the other task could be running on another CPU and could have
> > + * poisoned the stack in the meantime. Frame pointers are uninitialized by
> > + * default, so for KMSAN we mark the return value initialized unconditionally.
> > */
> > #define READ_ONCE_TASK_STACK(task, x) \
> > ({ \
> > @@ -111,7 +112,7 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
> > val = READ_ONCE(x); \
> > else \
> > val = READ_ONCE_NOCHECK(x); \
> > - val; \
> > + KMSAN_INIT_VALUE(val); \
> > })
> >
> > static inline bool task_on_another_cpu(struct task_struct *task)
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 14/36] kmsan: x86: sync metadata pages on page fault
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (12 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 13/36] kmsan: make READ_ONCE_TASK_STACK() return initialized values glider
@ 2019-11-22 11:25 ` glider
2019-11-22 11:26 ` [PATCH RFC v3 15/36] kmsan: add tests for KMSAN glider
` (21 subsequent siblings)
35 siblings, 0 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:25 UTC (permalink / raw)
To: Ingo Molnar, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, jasowang, axboe, m.szyprowski,
elver, mark.rutland, martin.petersen, schwidefsky, willy, mst,
monstr, pmladek, cai, rdunlap, robin.murphy, sergey.senozhatsky,
rostedt, tiwai, tytso, tglx, gor, wsa
KMSAN assumes shadow and origin pages for every allocated page are
accessible. For pages in vmalloc region those metadata pages reside in
[VMALLOC_END, VMALLOC_META_END), therefore we must sync a bigger memory
region.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I0d54855489870ef1180b37fe2120b601da464bf7
---
arch/x86/mm/fault.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 9ceacd1156db..d582337ba45d 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -320,7 +320,17 @@ static void dump_pagetable(unsigned long address)
void vmalloc_sync_all(void)
{
+#ifdef CONFIG_KMSAN
+ /*
+ * For KMSAN, make sure metadata pages for vmalloc area and modules are
+ * also synced.
+ */
+ sync_global_pgds(VMALLOC_START & PGDIR_MASK, VMALLOC_META_END);
+ sync_global_pgds(MODULES_SHADOW_START & PGDIR_MASK,
+ MODULES_ORIGIN_END);
+#else
sync_global_pgds(VMALLOC_START & PGDIR_MASK, VMALLOC_END);
+#endif
}
/*
@@ -337,7 +347,17 @@ static noinline int vmalloc_fault(unsigned long address)
pte_t *pte;
/* Make sure we are in vmalloc area: */
+#ifdef CONFIG_KMSAN
+ /*
+ * For KMSAN, make sure metadata pages for vmalloc area and modules are
+ * also synced.
+ */
+ if (!(address >= VMALLOC_START && address < VMALLOC_META_END) &&
+ !(address >= MODULES_SHADOW_START &&
+ address < MODULES_ORIGIN_END))
+#else
if (!(address >= VMALLOC_START && address < VMALLOC_END))
+#endif
return -1;
/*
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH RFC v3 15/36] kmsan: add tests for KMSAN
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (13 preceding siblings ...)
2019-11-22 11:25 ` [PATCH RFC v3 14/36] kmsan: x86: sync metadata pages on page fault glider
@ 2019-11-22 11:26 ` glider
2019-11-29 14:14 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 16/36] crypto: kmsan: disable accelerated configs under KMSAN glider
` (20 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
The initial commit adds several tests that trigger KMSAN warnings in
simple cases.
To use, build the kernel with CONFIG_TEST_KMSAN and do
`insmod test_kmsan.ko`
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v2:
- added printk_test()
Change-Id: I287e86ae83a82b770f2baa46e5bbdce1dfa65195
---
lib/Makefile | 1 +
lib/test_kmsan.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 232 insertions(+)
create mode 100644 lib/test_kmsan.c
diff --git a/lib/Makefile b/lib/Makefile
index 58a3e1b1a868..08fcb37499a0 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -65,6 +65,7 @@ CFLAGS_test_kasan.o += $(call cc-disable-warning, vla)
obj-$(CONFIG_TEST_UBSAN) += test_ubsan.o
CFLAGS_test_ubsan.o += $(call cc-disable-warning, vla)
UBSAN_SANITIZE_test_ubsan.o := y
+obj-$(CONFIG_TEST_KMSAN) += test_kmsan.o
obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o
obj-$(CONFIG_TEST_LIST_SORT) += test_list_sort.o
obj-$(CONFIG_TEST_LKM) += test_module.o
diff --git a/lib/test_kmsan.c b/lib/test_kmsan.c
new file mode 100644
index 000000000000..dcbe02adbdb0
--- /dev/null
+++ b/lib/test_kmsan.c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Module for testing KMSAN.
+ *
+ * Copyright (C) 2017-2019 Google LLC
+ * Author: Alexander Potapenko <glider@google.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+/*
+ * Tests below use noinline and volatile to work around compiler optimizations
+ * that may mask KMSAN bugs.
+ */
+#define pr_fmt(fmt) "kmsan test: %s : " fmt, __func__
+
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/slab.h>
+#include <linux/kmsan-checks.h>
+
+#define CHECK(x) \
+ do { \
+ if (x) \
+ pr_info(#x " is true\n"); \
+ else \
+ pr_info(#x " is false\n"); \
+ } while (0)
+
+noinline void use_integer(int cond)
+{
+ CHECK(cond);
+}
+
+int signed_sum3(int a, int b, int c)
+{
+ return a + b + c;
+}
+
+noinline void uninit_kmalloc_test(void)
+{
+ int *ptr;
+
+ pr_info("-----------------------------\n");
+ pr_info("uninitialized kmalloc test (UMR report)\n");
+ ptr = kmalloc(sizeof(int), GFP_KERNEL);
+ pr_info("kmalloc returned %p\n", ptr);
+ CHECK(*ptr);
+}
+noinline void init_kmalloc_test(void)
+{
+ int *ptr;
+
+ pr_info("-----------------------------\n");
+ pr_info("initialized kmalloc test (no reports)\n");
+ ptr = kmalloc(sizeof(int), GFP_KERNEL);
+ memset(ptr, 0, sizeof(int));
+ pr_info("kmalloc returned %p\n", ptr);
+ CHECK(*ptr);
+}
+
+noinline void init_kzalloc_test(void)
+{
+ int *ptr;
+
+ pr_info("-----------------------------\n");
+ pr_info("initialized kzalloc test (no reports)\n");
+ ptr = kzalloc(sizeof(int), GFP_KERNEL);
+ pr_info("kzalloc returned %p\n", ptr);
+ CHECK(*ptr);
+}
+
+noinline void uninit_multiple_args_test(void)
+{
+ volatile int a;
+ volatile char b = 3, c;
+
+ pr_info("-----------------------------\n");
+ pr_info("uninitialized local passed to fn (UMR report)\n");
+ CHECK(signed_sum3(a, b, c));
+}
+
+noinline void uninit_stack_var_test(void)
+{
+ int cond;
+
+ pr_info("-----------------------------\n");
+ pr_info("uninitialized stack variable (UMR report)\n");
+ CHECK(cond);
+}
+
+noinline void init_stack_var_test(void)
+{
+ volatile int cond = 1;
+
+ pr_info("-----------------------------\n");
+ pr_info("initialized stack variable (no reports)\n");
+ CHECK(cond);
+}
+
+noinline void two_param_fn_2(int arg1, int arg2)
+{
+ CHECK(arg1);
+ CHECK(arg2);
+}
+
+noinline void one_param_fn(int arg)
+{
+ two_param_fn_2(arg, arg);
+ CHECK(arg);
+}
+
+noinline void two_param_fn(int arg1, int arg2)
+{
+ int init = 0;
+
+ one_param_fn(init);
+ CHECK(arg1);
+ CHECK(arg2);
+}
+
+void params_test(void)
+{
+ int uninit, init = 1;
+
+ two_param_fn(uninit, init);
+}
+
+noinline void do_uninit_local_array(char *array, int start, int stop)
+{
+ int i;
+ volatile char uninit;
+
+ for (i = start; i < stop; i++)
+ array[i] = uninit;
+}
+
+noinline void uninit_kmsan_check_memory_test(void)
+{
+ volatile char local_array[8];
+
+ pr_info("-----------------------------\n");
+ pr_info("kmsan_check_memory() called on uninit local (UMR report)\n");
+ do_uninit_local_array((char *)local_array, 5, 7);
+
+ kmsan_check_memory((char *)local_array, 8);
+}
+
+noinline void init_kmsan_vmap_vunmap_test(void)
+{
+ const int npages = 2;
+ struct page *pages[npages];
+ void *vbuf;
+ int i;
+
+ pr_info("-----------------------------\n");
+ pr_info("pages initialized via vmap (no reports)\n");
+
+ for (i = 0; i < npages; i++)
+ pages[i] = alloc_page(GFP_KERNEL);
+ vbuf = vmap(pages, npages, VM_MAP, PAGE_KERNEL);
+ memset(vbuf, 0xfe, npages * PAGE_SIZE);
+ for (i = 0; i < npages; i++)
+ kmsan_check_memory(page_address(pages[i]), PAGE_SIZE);
+
+ if (vbuf)
+ vunmap(vbuf);
+ for (i = 0; i < npages; i++)
+ if (pages[i])
+ __free_page(pages[i]);
+}
+
+noinline void init_vmalloc(void)
+{
+ char *buf;
+ int npages = 8, i;
+
+ pr_info("-----------------------------\n");
+ pr_info("pages initialized via vmap (no reports)\n");
+ buf = vmalloc(PAGE_SIZE * npages);
+ buf[0] = 1;
+ memset(buf, 0xfe, PAGE_SIZE * npages);
+ CHECK(buf[0]);
+ for (i = 0; i < npages; i++)
+ kmsan_check_memory(&buf[PAGE_SIZE * i], PAGE_SIZE);
+ vfree(buf);
+}
+
+noinline void uaf_test(void)
+{
+ volatile int *var;
+
+ pr_info("-----------------------------\n");
+ pr_info("use-after-free in kmalloc-ed buffer (UMR report)\n");
+ var = kmalloc(80, GFP_KERNEL);
+ var[3] = 0xfeedface;
+ kfree((int *)var);
+ CHECK(var[3]);
+}
+
+noinline void printk_test(void)
+{
+ volatile int uninit;
+
+ pr_info("-----------------------------\n");
+ pr_info("uninit local passed to pr_info() (UMR report)\n");
+ pr_info("%px contains %d\n", &uninit, uninit);
+}
+
+static noinline int __init kmsan_tests_init(void)
+{
+ uninit_kmalloc_test();
+ init_kmalloc_test();
+ init_kzalloc_test();
+ uninit_multiple_args_test();
+ uninit_stack_var_test();
+ init_stack_var_test();
+ uninit_kmsan_check_memory_test();
+ init_kmsan_vmap_vunmap_test();
+ init_vmalloc();
+ uaf_test();
+ printk_test();
+ return -EAGAIN;
+}
+
+module_init(kmsan_tests_init);
+MODULE_LICENSE("GPL");
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 15/36] kmsan: add tests for KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 15/36] kmsan: add tests for KMSAN glider
@ 2019-11-29 14:14 ` Andrey Konovalov
2019-12-05 14:30 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 14:14 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, ebiggers, Eric Dumazet, ericvh,
Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii, mingo,
Jason Wang, axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, pmladek, Qian Cai,
Randy Dunlap, Robin Murphy, sergey.senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> The initial commit adds several tests that trigger KMSAN warnings in
> simple cases.
> To use, build the kernel with CONFIG_TEST_KMSAN and do
> `insmod test_kmsan.ko`
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
> v2:
> - added printk_test()
>
> Change-Id: I287e86ae83a82b770f2baa46e5bbdce1dfa65195
> ---
> lib/Makefile | 1 +
> lib/test_kmsan.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 232 insertions(+)
> create mode 100644 lib/test_kmsan.c
>
> diff --git a/lib/Makefile b/lib/Makefile
> index 58a3e1b1a868..08fcb37499a0 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -65,6 +65,7 @@ CFLAGS_test_kasan.o += $(call cc-disable-warning, vla)
> obj-$(CONFIG_TEST_UBSAN) += test_ubsan.o
> CFLAGS_test_ubsan.o += $(call cc-disable-warning, vla)
> UBSAN_SANITIZE_test_ubsan.o := y
> +obj-$(CONFIG_TEST_KMSAN) += test_kmsan.o
> obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o
> obj-$(CONFIG_TEST_LIST_SORT) += test_list_sort.o
> obj-$(CONFIG_TEST_LKM) += test_module.o
> diff --git a/lib/test_kmsan.c b/lib/test_kmsan.c
> new file mode 100644
> index 000000000000..dcbe02adbdb0
> --- /dev/null
> +++ b/lib/test_kmsan.c
> @@ -0,0 +1,231 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Module for testing KMSAN.
> + *
> + * Copyright (C) 2017-2019 Google LLC
> + * Author: Alexander Potapenko <glider@google.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +/*
> + * Tests below use noinline and volatile to work around compiler optimizations
> + * that may mask KMSAN bugs.
> + */
> +#define pr_fmt(fmt) "kmsan test: %s : " fmt, __func__
> +
> +#include <linux/mm.h>
> +#include <linux/module.h>
> +#include <linux/printk.h>
> +#include <linux/slab.h>
> +#include <linux/kmsan-checks.h>
> +
> +#define CHECK(x) \
> + do { \
> + if (x) \
> + pr_info(#x " is true\n"); \
> + else \
> + pr_info(#x " is false\n"); \
> + } while (0)
> +
> +noinline void use_integer(int cond)
> +{
> + CHECK(cond);
> +}
This function seems to be unused.
> +
> +int signed_sum3(int a, int b, int c)
> +{
> + return a + b + c;
> +}
> +
> +noinline void uninit_kmalloc_test(void)
> +{
> + int *ptr;
> +
> + pr_info("-----------------------------\n");
> + pr_info("uninitialized kmalloc test (UMR report)\n");
> + ptr = kmalloc(sizeof(int), GFP_KERNEL);
> + pr_info("kmalloc returned %p\n", ptr);
> + CHECK(*ptr);
> +}
> +noinline void init_kmalloc_test(void)
> +{
> + int *ptr;
> +
> + pr_info("-----------------------------\n");
> + pr_info("initialized kmalloc test (no reports)\n");
> + ptr = kmalloc(sizeof(int), GFP_KERNEL);
> + memset(ptr, 0, sizeof(int));
> + pr_info("kmalloc returned %p\n", ptr);
> + CHECK(*ptr);
> +}
> +
> +noinline void init_kzalloc_test(void)
> +{
> + int *ptr;
> +
> + pr_info("-----------------------------\n");
> + pr_info("initialized kzalloc test (no reports)\n");
> + ptr = kzalloc(sizeof(int), GFP_KERNEL);
> + pr_info("kzalloc returned %p\n", ptr);
> + CHECK(*ptr);
> +}
> +
> +noinline void uninit_multiple_args_test(void)
> +{
> + volatile int a;
> + volatile char b = 3, c;
> +
> + pr_info("-----------------------------\n");
> + pr_info("uninitialized local passed to fn (UMR report)\n");
> + CHECK(signed_sum3(a, b, c));
> +}
> +
> +noinline void uninit_stack_var_test(void)
> +{
> + int cond;
> +
> + pr_info("-----------------------------\n");
> + pr_info("uninitialized stack variable (UMR report)\n");
> + CHECK(cond);
> +}
> +
> +noinline void init_stack_var_test(void)
> +{
> + volatile int cond = 1;
> +
> + pr_info("-----------------------------\n");
> + pr_info("initialized stack variable (no reports)\n");
> + CHECK(cond);
> +}
> +
> +noinline void two_param_fn_2(int arg1, int arg2)
> +{
> + CHECK(arg1);
> + CHECK(arg2);
> +}
> +
> +noinline void one_param_fn(int arg)
> +{
> + two_param_fn_2(arg, arg);
> + CHECK(arg);
> +}
> +
> +noinline void two_param_fn(int arg1, int arg2)
> +{
> + int init = 0;
> +
> + one_param_fn(init);
> + CHECK(arg1);
> + CHECK(arg2);
> +}
> +
> +void params_test(void)
> +{
> + int uninit, init = 1;
> +
> + two_param_fn(uninit, init);
> +}
This one as well.
> +
> +noinline void do_uninit_local_array(char *array, int start, int stop)
> +{
> + int i;
> + volatile char uninit;
> +
> + for (i = start; i < stop; i++)
> + array[i] = uninit;
> +}
> +
> +noinline void uninit_kmsan_check_memory_test(void)
> +{
> + volatile char local_array[8];
> +
> + pr_info("-----------------------------\n");
> + pr_info("kmsan_check_memory() called on uninit local (UMR report)\n");
> + do_uninit_local_array((char *)local_array, 5, 7);
> +
> + kmsan_check_memory((char *)local_array, 8);
> +}
> +
> +noinline void init_kmsan_vmap_vunmap_test(void)
> +{
> + const int npages = 2;
> + struct page *pages[npages];
> + void *vbuf;
> + int i;
> +
> + pr_info("-----------------------------\n");
> + pr_info("pages initialized via vmap (no reports)\n");
> +
> + for (i = 0; i < npages; i++)
> + pages[i] = alloc_page(GFP_KERNEL);
> + vbuf = vmap(pages, npages, VM_MAP, PAGE_KERNEL);
> + memset(vbuf, 0xfe, npages * PAGE_SIZE);
> + for (i = 0; i < npages; i++)
> + kmsan_check_memory(page_address(pages[i]), PAGE_SIZE);
> +
> + if (vbuf)
> + vunmap(vbuf);
> + for (i = 0; i < npages; i++)
> + if (pages[i])
> + __free_page(pages[i]);
> +}
> +
> +noinline void init_vmalloc(void)
> +{
> + char *buf;
> + int npages = 8, i;
> +
> + pr_info("-----------------------------\n");
> + pr_info("pages initialized via vmap (no reports)\n");
> + buf = vmalloc(PAGE_SIZE * npages);
> + buf[0] = 1;
> + memset(buf, 0xfe, PAGE_SIZE * npages);
> + CHECK(buf[0]);
> + for (i = 0; i < npages; i++)
> + kmsan_check_memory(&buf[PAGE_SIZE * i], PAGE_SIZE);
> + vfree(buf);
> +}
> +
> +noinline void uaf_test(void)
> +{
> + volatile int *var;
> +
> + pr_info("-----------------------------\n");
> + pr_info("use-after-free in kmalloc-ed buffer (UMR report)\n");
> + var = kmalloc(80, GFP_KERNEL);
> + var[3] = 0xfeedface;
> + kfree((int *)var);
> + CHECK(var[3]);
> +}
> +
> +noinline void printk_test(void)
> +{
> + volatile int uninit;
> +
> + pr_info("-----------------------------\n");
> + pr_info("uninit local passed to pr_info() (UMR report)\n");
> + pr_info("%px contains %d\n", &uninit, uninit);
> +}
> +
> +static noinline int __init kmsan_tests_init(void)
> +{
> + uninit_kmalloc_test();
> + init_kmalloc_test();
> + init_kzalloc_test();
> + uninit_multiple_args_test();
> + uninit_stack_var_test();
> + init_stack_var_test();
> + uninit_kmsan_check_memory_test();
> + init_kmsan_vmap_vunmap_test();
> + init_vmalloc();
Perhaps s/init_vmalloc/init_vmalloc_test?
> + uaf_test();
> + printk_test();
> + return -EAGAIN;
> +}
> +
> +module_init(kmsan_tests_init);
> +MODULE_LICENSE("GPL");
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 15/36] kmsan: add tests for KMSAN
2019-11-29 14:14 ` Andrey Konovalov
@ 2019-12-05 14:30 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 14:30 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Fri, Nov 29, 2019 at 3:15 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > The initial commit adds several tests that trigger KMSAN warnings in
> > simple cases.
> > To use, build the kernel with CONFIG_TEST_KMSAN and do
> > `insmod test_kmsan.ko`
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> > v2:
> > - added printk_test()
> >
> > Change-Id: I287e86ae83a82b770f2baa46e5bbdce1dfa65195
> > ---
> > lib/Makefile | 1 +
> > lib/test_kmsan.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 232 insertions(+)
> > create mode 100644 lib/test_kmsan.c
> >
> > diff --git a/lib/Makefile b/lib/Makefile
> > index 58a3e1b1a868..08fcb37499a0 100644
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -65,6 +65,7 @@ CFLAGS_test_kasan.o += $(call cc-disable-warning, vla)
> > obj-$(CONFIG_TEST_UBSAN) += test_ubsan.o
> > CFLAGS_test_ubsan.o += $(call cc-disable-warning, vla)
> > UBSAN_SANITIZE_test_ubsan.o := y
> > +obj-$(CONFIG_TEST_KMSAN) += test_kmsan.o
> > obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o
> > obj-$(CONFIG_TEST_LIST_SORT) += test_list_sort.o
> > obj-$(CONFIG_TEST_LKM) += test_module.o
> > diff --git a/lib/test_kmsan.c b/lib/test_kmsan.c
> > new file mode 100644
> > index 000000000000..dcbe02adbdb0
> > --- /dev/null
> > +++ b/lib/test_kmsan.c
> > @@ -0,0 +1,231 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Module for testing KMSAN.
> > + *
> > + * Copyright (C) 2017-2019 Google LLC
> > + * Author: Alexander Potapenko <glider@google.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + */
> > +
> > +/*
> > + * Tests below use noinline and volatile to work around compiler optimizations
> > + * that may mask KMSAN bugs.
> > + */
> > +#define pr_fmt(fmt) "kmsan test: %s : " fmt, __func__
> > +
> > +#include <linux/mm.h>
> > +#include <linux/module.h>
> > +#include <linux/printk.h>
> > +#include <linux/slab.h>
> > +#include <linux/kmsan-checks.h>
> > +
> > +#define CHECK(x) \
> > + do { \
> > + if (x) \
> > + pr_info(#x " is true\n"); \
> > + else \
> > + pr_info(#x " is false\n"); \
> > + } while (0)
> > +
> > +noinline void use_integer(int cond)
> > +{
> > + CHECK(cond);
> > +}
>
> This function seems to be unused.
Removed it, thanks!
> > +
> > +int signed_sum3(int a, int b, int c)
> > +{
> > + return a + b + c;
> > +}
> > +
> > +noinline void uninit_kmalloc_test(void)
> > +{
> > + int *ptr;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("uninitialized kmalloc test (UMR report)\n");
> > + ptr = kmalloc(sizeof(int), GFP_KERNEL);
> > + pr_info("kmalloc returned %p\n", ptr);
> > + CHECK(*ptr);
> > +}
> > +noinline void init_kmalloc_test(void)
> > +{
> > + int *ptr;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("initialized kmalloc test (no reports)\n");
> > + ptr = kmalloc(sizeof(int), GFP_KERNEL);
> > + memset(ptr, 0, sizeof(int));
> > + pr_info("kmalloc returned %p\n", ptr);
> > + CHECK(*ptr);
> > +}
> > +
> > +noinline void init_kzalloc_test(void)
> > +{
> > + int *ptr;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("initialized kzalloc test (no reports)\n");
> > + ptr = kzalloc(sizeof(int), GFP_KERNEL);
> > + pr_info("kzalloc returned %p\n", ptr);
> > + CHECK(*ptr);
> > +}
> > +
> > +noinline void uninit_multiple_args_test(void)
> > +{
> > + volatile int a;
> > + volatile char b = 3, c;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("uninitialized local passed to fn (UMR report)\n");
> > + CHECK(signed_sum3(a, b, c));
> > +}
> > +
> > +noinline void uninit_stack_var_test(void)
> > +{
> > + int cond;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("uninitialized stack variable (UMR report)\n");
> > + CHECK(cond);
> > +}
> > +
> > +noinline void init_stack_var_test(void)
> > +{
> > + volatile int cond = 1;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("initialized stack variable (no reports)\n");
> > + CHECK(cond);
> > +}
> > +
> > +noinline void two_param_fn_2(int arg1, int arg2)
> > +{
> > + CHECK(arg1);
> > + CHECK(arg2);
> > +}
> > +
> > +noinline void one_param_fn(int arg)
> > +{
> > + two_param_fn_2(arg, arg);
> > + CHECK(arg);
> > +}
> > +
> > +noinline void two_param_fn(int arg1, int arg2)
> > +{
> > + int init = 0;
> > +
> > + one_param_fn(init);
> > + CHECK(arg1);
> > + CHECK(arg2);
> > +}
> > +
> > +void params_test(void)
> > +{
> > + int uninit, init = 1;
> > +
> > + two_param_fn(uninit, init);
> > +}
>
> This one as well.
Added params_test to kmsan_tests_init() below.
> > +
> > +noinline void do_uninit_local_array(char *array, int start, int stop)
> > +{
> > + int i;
> > + volatile char uninit;
> > +
> > + for (i = start; i < stop; i++)
> > + array[i] = uninit;
> > +}
> > +
> > +noinline void uninit_kmsan_check_memory_test(void)
> > +{
> > + volatile char local_array[8];
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("kmsan_check_memory() called on uninit local (UMR report)\n");
> > + do_uninit_local_array((char *)local_array, 5, 7);
> > +
> > + kmsan_check_memory((char *)local_array, 8);
> > +}
> > +
> > +noinline void init_kmsan_vmap_vunmap_test(void)
> > +{
> > + const int npages = 2;
> > + struct page *pages[npages];
> > + void *vbuf;
> > + int i;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("pages initialized via vmap (no reports)\n");
> > +
> > + for (i = 0; i < npages; i++)
> > + pages[i] = alloc_page(GFP_KERNEL);
> > + vbuf = vmap(pages, npages, VM_MAP, PAGE_KERNEL);
> > + memset(vbuf, 0xfe, npages * PAGE_SIZE);
> > + for (i = 0; i < npages; i++)
> > + kmsan_check_memory(page_address(pages[i]), PAGE_SIZE);
> > +
> > + if (vbuf)
> > + vunmap(vbuf);
> > + for (i = 0; i < npages; i++)
> > + if (pages[i])
> > + __free_page(pages[i]);
> > +}
> > +
> > +noinline void init_vmalloc(void)
> > +{
> > + char *buf;
> > + int npages = 8, i;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("pages initialized via vmap (no reports)\n");
> > + buf = vmalloc(PAGE_SIZE * npages);
> > + buf[0] = 1;
> > + memset(buf, 0xfe, PAGE_SIZE * npages);
> > + CHECK(buf[0]);
> > + for (i = 0; i < npages; i++)
> > + kmsan_check_memory(&buf[PAGE_SIZE * i], PAGE_SIZE);
> > + vfree(buf);
> > +}
> > +
> > +noinline void uaf_test(void)
> > +{
> > + volatile int *var;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("use-after-free in kmalloc-ed buffer (UMR report)\n");
> > + var = kmalloc(80, GFP_KERNEL);
> > + var[3] = 0xfeedface;
> > + kfree((int *)var);
> > + CHECK(var[3]);
> > +}
> > +
> > +noinline void printk_test(void)
> > +{
> > + volatile int uninit;
> > +
> > + pr_info("-----------------------------\n");
> > + pr_info("uninit local passed to pr_info() (UMR report)\n");
> > + pr_info("%px contains %d\n", &uninit, uninit);
> > +}
> > +
> > +static noinline int __init kmsan_tests_init(void)
> > +{
> > + uninit_kmalloc_test();
> > + init_kmalloc_test();
> > + init_kzalloc_test();
> > + uninit_multiple_args_test();
> > + uninit_stack_var_test();
> > + init_stack_var_test();
> > + uninit_kmsan_check_memory_test();
> > + init_kmsan_vmap_vunmap_test();
> > + init_vmalloc();
>
> Perhaps s/init_vmalloc/init_vmalloc_test?
Done
>
> > + uaf_test();
> > + printk_test();
> > + return -EAGAIN;
> > +}
> > +
> > +module_init(kmsan_tests_init);
> > +MODULE_LICENSE("GPL");
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 16/36] crypto: kmsan: disable accelerated configs under KMSAN
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (14 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 15/36] kmsan: add tests for KMSAN glider
@ 2019-11-22 11:26 ` glider
2019-12-02 13:25 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 17/36] kmsan: x86: disable UNWINDER_ORC " glider
` (19 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Herbert Xu, David S. Miller, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, iii, mingo,
jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso,
tglx, gor, wsa
KMSAN is unable to understand when initialized values come from assembly.
Disable accelerated configs in KMSAN builds to prevent false positive
reports.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: Iddc71a2a27360e036d719c0940ebf15553cf8de8
---
crypto/Kconfig | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 52 insertions(+)
diff --git a/crypto/Kconfig b/crypto/Kconfig
index 9e524044d312..502a75f1b597 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -309,11 +309,15 @@ config CRYPTO_AEGIS128
config CRYPTO_AEGIS128_SIMD
bool "Support SIMD acceleration for AEGIS-128"
depends on CRYPTO_AEGIS128 && ((ARM || ARM64) && KERNEL_MODE_NEON)
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
default y
config CRYPTO_AEGIS128_AESNI_SSE2
tristate "AEGIS-128 AEAD algorithm (x86_64 AESNI+SSE2 implementation)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_AEAD
select CRYPTO_SIMD
help
@@ -571,6 +575,8 @@ config CRYPTO_CRC32C
config CRYPTO_CRC32C_INTEL
tristate "CRC32c INTEL hardware acceleration"
depends on X86
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_HASH
help
In Intel processor with SSE4.2 supported, the processor will
@@ -611,6 +617,8 @@ config CRYPTO_CRC32
config CRYPTO_CRC32_PCLMUL
tristate "CRC32 PCLMULQDQ hardware acceleration"
depends on X86
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_HASH
select CRC32
help
@@ -649,6 +657,8 @@ config CRYPTO_CRCT10DIF
config CRYPTO_CRCT10DIF_PCLMUL
tristate "CRCT10DIF PCLMULQDQ hardware acceleration"
depends on X86 && 64BIT && CRC_T10DIF
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_HASH
help
For x86_64 processors with SSE4.2 and PCLMULQDQ supported,
@@ -695,6 +705,8 @@ config CRYPTO_POLY1305
config CRYPTO_POLY1305_X86_64
tristate "Poly1305 authenticator algorithm (x86_64/SSE2/AVX2)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_POLY1305
help
Poly1305 authenticator algorithm, RFC7539.
@@ -814,6 +826,8 @@ config CRYPTO_SHA1
config CRYPTO_SHA1_SSSE3
tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_SHA1
select CRYPTO_HASH
help
@@ -825,6 +839,8 @@ config CRYPTO_SHA1_SSSE3
config CRYPTO_SHA256_SSSE3
tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_SHA256
select CRYPTO_HASH
help
@@ -837,6 +853,8 @@ config CRYPTO_SHA256_SSSE3
config CRYPTO_SHA512_SSSE3
tristate "SHA512 digest algorithm (SSSE3/AVX/AVX2)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_SHA512
select CRYPTO_HASH
help
@@ -1011,6 +1029,8 @@ config CRYPTO_WP512
config CRYPTO_GHASH_CLMUL_NI_INTEL
tristate "GHASH hash function (CLMUL-NI accelerated)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_CRYPTD
help
This is the x86_64 CLMUL-NI accelerated implementation of
@@ -1064,6 +1084,8 @@ config CRYPTO_AES_TI
config CRYPTO_AES_NI_INTEL
tristate "AES cipher algorithms (AES-NI)"
depends on X86
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_AEAD
select CRYPTO_LIB_AES
select CRYPTO_ALGAPI
@@ -1190,6 +1212,8 @@ config CRYPTO_BLOWFISH_COMMON
config CRYPTO_BLOWFISH_X86_64
tristate "Blowfish cipher algorithm (x86_64)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_BLOWFISH_COMMON
help
@@ -1221,6 +1245,8 @@ config CRYPTO_CAMELLIA_X86_64
tristate "Camellia cipher algorithm (x86_64)"
depends on X86 && 64BIT
depends on CRYPTO
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_GLUE_HELPER_X86
help
@@ -1238,6 +1264,8 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64
tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX)"
depends on X86 && 64BIT
depends on CRYPTO
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_CAMELLIA_X86_64
select CRYPTO_GLUE_HELPER_X86
@@ -1258,6 +1286,8 @@ config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64
tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)"
depends on X86 && 64BIT
depends on CRYPTO
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_CAMELLIA_AESNI_AVX_X86_64
help
Camellia cipher algorithm module (x86_64/AES-NI/AVX2).
@@ -1303,6 +1333,8 @@ config CRYPTO_CAST5
config CRYPTO_CAST5_AVX_X86_64
tristate "CAST5 (CAST-128) cipher algorithm (x86_64/AVX)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_CAST5
select CRYPTO_CAST_COMMON
@@ -1325,6 +1357,8 @@ config CRYPTO_CAST6
config CRYPTO_CAST6_AVX_X86_64
tristate "CAST6 (CAST-256) cipher algorithm (x86_64/AVX)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_CAST6
select CRYPTO_CAST_COMMON
@@ -1360,6 +1394,8 @@ config CRYPTO_DES_SPARC64
config CRYPTO_DES3_EDE_X86_64
tristate "Triple DES EDE cipher algorithm (x86-64)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_LIB_DES
help
@@ -1426,6 +1462,8 @@ config CRYPTO_CHACHA20
config CRYPTO_CHACHA20_X86_64
tristate "ChaCha stream cipher algorithms (x86_64/SSSE3/AVX2/AVX-512VL)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_CHACHA20
help
@@ -1462,6 +1500,8 @@ config CRYPTO_SERPENT
config CRYPTO_SERPENT_SSE2_X86_64
tristate "Serpent cipher algorithm (x86_64/SSE2)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_GLUE_HELPER_X86
select CRYPTO_SERPENT
@@ -1481,6 +1521,8 @@ config CRYPTO_SERPENT_SSE2_X86_64
config CRYPTO_SERPENT_SSE2_586
tristate "Serpent cipher algorithm (i586/SSE2)"
depends on X86 && !64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_GLUE_HELPER_X86
select CRYPTO_SERPENT
@@ -1500,6 +1542,8 @@ config CRYPTO_SERPENT_SSE2_586
config CRYPTO_SERPENT_AVX_X86_64
tristate "Serpent cipher algorithm (x86_64/AVX)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_GLUE_HELPER_X86
select CRYPTO_SERPENT
@@ -1520,6 +1564,8 @@ config CRYPTO_SERPENT_AVX_X86_64
config CRYPTO_SERPENT_AVX2_X86_64
tristate "Serpent cipher algorithm (x86_64/AVX2)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_SERPENT_AVX_X86_64
help
Serpent cipher algorithm, by Anderson, Biham & Knudsen.
@@ -1615,6 +1661,8 @@ config CRYPTO_TWOFISH_586
config CRYPTO_TWOFISH_X86_64
tristate "Twofish cipher algorithm (x86_64)"
depends on (X86 || UML_X86) && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_ALGAPI
select CRYPTO_TWOFISH_COMMON
help
@@ -1631,6 +1679,8 @@ config CRYPTO_TWOFISH_X86_64
config CRYPTO_TWOFISH_X86_64_3WAY
tristate "Twofish cipher algorithm (x86_64, 3-way parallel)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_TWOFISH_COMMON
select CRYPTO_TWOFISH_X86_64
@@ -1652,6 +1702,8 @@ config CRYPTO_TWOFISH_X86_64_3WAY
config CRYPTO_TWOFISH_AVX_X86_64
tristate "Twofish cipher algorithm (x86_64/AVX)"
depends on X86 && 64BIT
+ # Disable under KMSAN to prevent false positives from assembly.
+ depends on !KMSAN
select CRYPTO_BLKCIPHER
select CRYPTO_GLUE_HELPER_X86
select CRYPTO_SIMD
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 16/36] crypto: kmsan: disable accelerated configs under KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 16/36] crypto: kmsan: disable accelerated configs under KMSAN glider
@ 2019-12-02 13:25 ` Marco Elver
2019-12-05 14:51 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-02 13:25 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Herbert Xu, David S. Miller, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> KMSAN is unable to understand when initialized values come from assembly.
> Disable accelerated configs in KMSAN builds to prevent false positive
> reports.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: Iddc71a2a27360e036d719c0940ebf15553cf8de8
> ---
> crypto/Kconfig | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 52 insertions(+)
>
> diff --git a/crypto/Kconfig b/crypto/Kconfig
> index 9e524044d312..502a75f1b597 100644
> --- a/crypto/Kconfig
> +++ b/crypto/Kconfig
> @@ -309,11 +309,15 @@ config CRYPTO_AEGIS128
> config CRYPTO_AEGIS128_SIMD
> bool "Support SIMD acceleration for AEGIS-128"
> depends on CRYPTO_AEGIS128 && ((ARM || ARM64) && KERNEL_MODE_NEON)
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
Probably fine to put this on one line:
depends on !KMSAN # avoid false positives from assembly
likewise for all below.
> default y
>
> config CRYPTO_AEGIS128_AESNI_SSE2
> tristate "AEGIS-128 AEAD algorithm (x86_64 AESNI+SSE2 implementation)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_AEAD
> select CRYPTO_SIMD
> help
> @@ -571,6 +575,8 @@ config CRYPTO_CRC32C
> config CRYPTO_CRC32C_INTEL
> tristate "CRC32c INTEL hardware acceleration"
> depends on X86
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_HASH
> help
> In Intel processor with SSE4.2 supported, the processor will
> @@ -611,6 +617,8 @@ config CRYPTO_CRC32
> config CRYPTO_CRC32_PCLMUL
> tristate "CRC32 PCLMULQDQ hardware acceleration"
> depends on X86
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_HASH
> select CRC32
> help
> @@ -649,6 +657,8 @@ config CRYPTO_CRCT10DIF
> config CRYPTO_CRCT10DIF_PCLMUL
> tristate "CRCT10DIF PCLMULQDQ hardware acceleration"
> depends on X86 && 64BIT && CRC_T10DIF
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_HASH
> help
> For x86_64 processors with SSE4.2 and PCLMULQDQ supported,
> @@ -695,6 +705,8 @@ config CRYPTO_POLY1305
> config CRYPTO_POLY1305_X86_64
> tristate "Poly1305 authenticator algorithm (x86_64/SSE2/AVX2)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_POLY1305
> help
> Poly1305 authenticator algorithm, RFC7539.
> @@ -814,6 +826,8 @@ config CRYPTO_SHA1
> config CRYPTO_SHA1_SSSE3
> tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_SHA1
> select CRYPTO_HASH
> help
> @@ -825,6 +839,8 @@ config CRYPTO_SHA1_SSSE3
> config CRYPTO_SHA256_SSSE3
> tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_SHA256
> select CRYPTO_HASH
> help
> @@ -837,6 +853,8 @@ config CRYPTO_SHA256_SSSE3
> config CRYPTO_SHA512_SSSE3
> tristate "SHA512 digest algorithm (SSSE3/AVX/AVX2)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_SHA512
> select CRYPTO_HASH
> help
> @@ -1011,6 +1029,8 @@ config CRYPTO_WP512
> config CRYPTO_GHASH_CLMUL_NI_INTEL
> tristate "GHASH hash function (CLMUL-NI accelerated)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_CRYPTD
> help
> This is the x86_64 CLMUL-NI accelerated implementation of
> @@ -1064,6 +1084,8 @@ config CRYPTO_AES_TI
> config CRYPTO_AES_NI_INTEL
> tristate "AES cipher algorithms (AES-NI)"
> depends on X86
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_AEAD
> select CRYPTO_LIB_AES
> select CRYPTO_ALGAPI
> @@ -1190,6 +1212,8 @@ config CRYPTO_BLOWFISH_COMMON
> config CRYPTO_BLOWFISH_X86_64
> tristate "Blowfish cipher algorithm (x86_64)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_BLOWFISH_COMMON
> help
> @@ -1221,6 +1245,8 @@ config CRYPTO_CAMELLIA_X86_64
> tristate "Camellia cipher algorithm (x86_64)"
> depends on X86 && 64BIT
> depends on CRYPTO
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_GLUE_HELPER_X86
> help
> @@ -1238,6 +1264,8 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64
> tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX)"
> depends on X86 && 64BIT
> depends on CRYPTO
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_CAMELLIA_X86_64
> select CRYPTO_GLUE_HELPER_X86
> @@ -1258,6 +1286,8 @@ config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64
> tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)"
> depends on X86 && 64BIT
> depends on CRYPTO
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_CAMELLIA_AESNI_AVX_X86_64
> help
> Camellia cipher algorithm module (x86_64/AES-NI/AVX2).
> @@ -1303,6 +1333,8 @@ config CRYPTO_CAST5
> config CRYPTO_CAST5_AVX_X86_64
> tristate "CAST5 (CAST-128) cipher algorithm (x86_64/AVX)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_CAST5
> select CRYPTO_CAST_COMMON
> @@ -1325,6 +1357,8 @@ config CRYPTO_CAST6
> config CRYPTO_CAST6_AVX_X86_64
> tristate "CAST6 (CAST-256) cipher algorithm (x86_64/AVX)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_CAST6
> select CRYPTO_CAST_COMMON
> @@ -1360,6 +1394,8 @@ config CRYPTO_DES_SPARC64
> config CRYPTO_DES3_EDE_X86_64
> tristate "Triple DES EDE cipher algorithm (x86-64)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_LIB_DES
> help
> @@ -1426,6 +1462,8 @@ config CRYPTO_CHACHA20
> config CRYPTO_CHACHA20_X86_64
> tristate "ChaCha stream cipher algorithms (x86_64/SSSE3/AVX2/AVX-512VL)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_CHACHA20
> help
> @@ -1462,6 +1500,8 @@ config CRYPTO_SERPENT
> config CRYPTO_SERPENT_SSE2_X86_64
> tristate "Serpent cipher algorithm (x86_64/SSE2)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_GLUE_HELPER_X86
> select CRYPTO_SERPENT
> @@ -1481,6 +1521,8 @@ config CRYPTO_SERPENT_SSE2_X86_64
> config CRYPTO_SERPENT_SSE2_586
> tristate "Serpent cipher algorithm (i586/SSE2)"
> depends on X86 && !64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_GLUE_HELPER_X86
> select CRYPTO_SERPENT
> @@ -1500,6 +1542,8 @@ config CRYPTO_SERPENT_SSE2_586
> config CRYPTO_SERPENT_AVX_X86_64
> tristate "Serpent cipher algorithm (x86_64/AVX)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_GLUE_HELPER_X86
> select CRYPTO_SERPENT
> @@ -1520,6 +1564,8 @@ config CRYPTO_SERPENT_AVX_X86_64
> config CRYPTO_SERPENT_AVX2_X86_64
> tristate "Serpent cipher algorithm (x86_64/AVX2)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_SERPENT_AVX_X86_64
> help
> Serpent cipher algorithm, by Anderson, Biham & Knudsen.
> @@ -1615,6 +1661,8 @@ config CRYPTO_TWOFISH_586
> config CRYPTO_TWOFISH_X86_64
> tristate "Twofish cipher algorithm (x86_64)"
> depends on (X86 || UML_X86) && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_ALGAPI
> select CRYPTO_TWOFISH_COMMON
> help
> @@ -1631,6 +1679,8 @@ config CRYPTO_TWOFISH_X86_64
> config CRYPTO_TWOFISH_X86_64_3WAY
> tristate "Twofish cipher algorithm (x86_64, 3-way parallel)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_TWOFISH_COMMON
> select CRYPTO_TWOFISH_X86_64
> @@ -1652,6 +1702,8 @@ config CRYPTO_TWOFISH_X86_64_3WAY
> config CRYPTO_TWOFISH_AVX_X86_64
> tristate "Twofish cipher algorithm (x86_64/AVX)"
> depends on X86 && 64BIT
> + # Disable under KMSAN to prevent false positives from assembly.
> + depends on !KMSAN
> select CRYPTO_BLKCIPHER
> select CRYPTO_GLUE_HELPER_X86
> select CRYPTO_SIMD
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 16/36] crypto: kmsan: disable accelerated configs under KMSAN
2019-12-02 13:25 ` Marco Elver
@ 2019-12-05 14:51 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 14:51 UTC (permalink / raw)
To: Marco Elver
Cc: Herbert Xu, David S. Miller, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 2:25 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
> >
> > KMSAN is unable to understand when initialized values come from assembly.
> > Disable accelerated configs in KMSAN builds to prevent false positive
> > reports.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Herbert Xu <herbert@gondor.apana.org.au>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: Iddc71a2a27360e036d719c0940ebf15553cf8de8
> > ---
> > crypto/Kconfig | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 52 insertions(+)
> >
> > diff --git a/crypto/Kconfig b/crypto/Kconfig
> > index 9e524044d312..502a75f1b597 100644
> > --- a/crypto/Kconfig
> > +++ b/crypto/Kconfig
> > @@ -309,11 +309,15 @@ config CRYPTO_AEGIS128
> > config CRYPTO_AEGIS128_SIMD
> > bool "Support SIMD acceleration for AEGIS-128"
> > depends on CRYPTO_AEGIS128 && ((ARM || ARM64) && KERNEL_MODE_NEON)
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
>
> Probably fine to put this on one line:
>
> depends on !KMSAN # avoid false positives from assembly
Done in v4.
> likewise for all below.
>
> > default y
> >
> > config CRYPTO_AEGIS128_AESNI_SSE2
> > tristate "AEGIS-128 AEAD algorithm (x86_64 AESNI+SSE2 implementation)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_AEAD
> > select CRYPTO_SIMD
> > help
> > @@ -571,6 +575,8 @@ config CRYPTO_CRC32C
> > config CRYPTO_CRC32C_INTEL
> > tristate "CRC32c INTEL hardware acceleration"
> > depends on X86
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_HASH
> > help
> > In Intel processor with SSE4.2 supported, the processor will
> > @@ -611,6 +617,8 @@ config CRYPTO_CRC32
> > config CRYPTO_CRC32_PCLMUL
> > tristate "CRC32 PCLMULQDQ hardware acceleration"
> > depends on X86
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_HASH
> > select CRC32
> > help
> > @@ -649,6 +657,8 @@ config CRYPTO_CRCT10DIF
> > config CRYPTO_CRCT10DIF_PCLMUL
> > tristate "CRCT10DIF PCLMULQDQ hardware acceleration"
> > depends on X86 && 64BIT && CRC_T10DIF
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_HASH
> > help
> > For x86_64 processors with SSE4.2 and PCLMULQDQ supported,
> > @@ -695,6 +705,8 @@ config CRYPTO_POLY1305
> > config CRYPTO_POLY1305_X86_64
> > tristate "Poly1305 authenticator algorithm (x86_64/SSE2/AVX2)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_POLY1305
> > help
> > Poly1305 authenticator algorithm, RFC7539.
> > @@ -814,6 +826,8 @@ config CRYPTO_SHA1
> > config CRYPTO_SHA1_SSSE3
> > tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_SHA1
> > select CRYPTO_HASH
> > help
> > @@ -825,6 +839,8 @@ config CRYPTO_SHA1_SSSE3
> > config CRYPTO_SHA256_SSSE3
> > tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_SHA256
> > select CRYPTO_HASH
> > help
> > @@ -837,6 +853,8 @@ config CRYPTO_SHA256_SSSE3
> > config CRYPTO_SHA512_SSSE3
> > tristate "SHA512 digest algorithm (SSSE3/AVX/AVX2)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_SHA512
> > select CRYPTO_HASH
> > help
> > @@ -1011,6 +1029,8 @@ config CRYPTO_WP512
> > config CRYPTO_GHASH_CLMUL_NI_INTEL
> > tristate "GHASH hash function (CLMUL-NI accelerated)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_CRYPTD
> > help
> > This is the x86_64 CLMUL-NI accelerated implementation of
> > @@ -1064,6 +1084,8 @@ config CRYPTO_AES_TI
> > config CRYPTO_AES_NI_INTEL
> > tristate "AES cipher algorithms (AES-NI)"
> > depends on X86
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_AEAD
> > select CRYPTO_LIB_AES
> > select CRYPTO_ALGAPI
> > @@ -1190,6 +1212,8 @@ config CRYPTO_BLOWFISH_COMMON
> > config CRYPTO_BLOWFISH_X86_64
> > tristate "Blowfish cipher algorithm (x86_64)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_BLOWFISH_COMMON
> > help
> > @@ -1221,6 +1245,8 @@ config CRYPTO_CAMELLIA_X86_64
> > tristate "Camellia cipher algorithm (x86_64)"
> > depends on X86 && 64BIT
> > depends on CRYPTO
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_GLUE_HELPER_X86
> > help
> > @@ -1238,6 +1264,8 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64
> > tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX)"
> > depends on X86 && 64BIT
> > depends on CRYPTO
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_CAMELLIA_X86_64
> > select CRYPTO_GLUE_HELPER_X86
> > @@ -1258,6 +1286,8 @@ config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64
> > tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)"
> > depends on X86 && 64BIT
> > depends on CRYPTO
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_CAMELLIA_AESNI_AVX_X86_64
> > help
> > Camellia cipher algorithm module (x86_64/AES-NI/AVX2).
> > @@ -1303,6 +1333,8 @@ config CRYPTO_CAST5
> > config CRYPTO_CAST5_AVX_X86_64
> > tristate "CAST5 (CAST-128) cipher algorithm (x86_64/AVX)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_CAST5
> > select CRYPTO_CAST_COMMON
> > @@ -1325,6 +1357,8 @@ config CRYPTO_CAST6
> > config CRYPTO_CAST6_AVX_X86_64
> > tristate "CAST6 (CAST-256) cipher algorithm (x86_64/AVX)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_CAST6
> > select CRYPTO_CAST_COMMON
> > @@ -1360,6 +1394,8 @@ config CRYPTO_DES_SPARC64
> > config CRYPTO_DES3_EDE_X86_64
> > tristate "Triple DES EDE cipher algorithm (x86-64)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_LIB_DES
> > help
> > @@ -1426,6 +1462,8 @@ config CRYPTO_CHACHA20
> > config CRYPTO_CHACHA20_X86_64
> > tristate "ChaCha stream cipher algorithms (x86_64/SSSE3/AVX2/AVX-512VL)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_CHACHA20
> > help
> > @@ -1462,6 +1500,8 @@ config CRYPTO_SERPENT
> > config CRYPTO_SERPENT_SSE2_X86_64
> > tristate "Serpent cipher algorithm (x86_64/SSE2)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_GLUE_HELPER_X86
> > select CRYPTO_SERPENT
> > @@ -1481,6 +1521,8 @@ config CRYPTO_SERPENT_SSE2_X86_64
> > config CRYPTO_SERPENT_SSE2_586
> > tristate "Serpent cipher algorithm (i586/SSE2)"
> > depends on X86 && !64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_GLUE_HELPER_X86
> > select CRYPTO_SERPENT
> > @@ -1500,6 +1542,8 @@ config CRYPTO_SERPENT_SSE2_586
> > config CRYPTO_SERPENT_AVX_X86_64
> > tristate "Serpent cipher algorithm (x86_64/AVX)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_GLUE_HELPER_X86
> > select CRYPTO_SERPENT
> > @@ -1520,6 +1564,8 @@ config CRYPTO_SERPENT_AVX_X86_64
> > config CRYPTO_SERPENT_AVX2_X86_64
> > tristate "Serpent cipher algorithm (x86_64/AVX2)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_SERPENT_AVX_X86_64
> > help
> > Serpent cipher algorithm, by Anderson, Biham & Knudsen.
> > @@ -1615,6 +1661,8 @@ config CRYPTO_TWOFISH_586
> > config CRYPTO_TWOFISH_X86_64
> > tristate "Twofish cipher algorithm (x86_64)"
> > depends on (X86 || UML_X86) && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_ALGAPI
> > select CRYPTO_TWOFISH_COMMON
> > help
> > @@ -1631,6 +1679,8 @@ config CRYPTO_TWOFISH_X86_64
> > config CRYPTO_TWOFISH_X86_64_3WAY
> > tristate "Twofish cipher algorithm (x86_64, 3-way parallel)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_TWOFISH_COMMON
> > select CRYPTO_TWOFISH_X86_64
> > @@ -1652,6 +1702,8 @@ config CRYPTO_TWOFISH_X86_64_3WAY
> > config CRYPTO_TWOFISH_AVX_X86_64
> > tristate "Twofish cipher algorithm (x86_64/AVX)"
> > depends on X86 && 64BIT
> > + # Disable under KMSAN to prevent false positives from assembly.
> > + depends on !KMSAN
> > select CRYPTO_BLKCIPHER
> > select CRYPTO_GLUE_HELPER_X86
> > select CRYPTO_SIMD
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 17/36] kmsan: x86: disable UNWINDER_ORC under KMSAN
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (15 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 16/36] crypto: kmsan: disable accelerated configs under KMSAN glider
@ 2019-11-22 11:26 ` glider
2019-12-02 13:30 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT glider
` (18 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Qian Cai, Christoph Hellwig, Herbert Xu, Harry Wentland,
Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
KMSAN doesn't currently support UNWINDER_ORC, causing the kernel to
freeze at boot time.
See http://github.com/google/kmsan/issues/48.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Qian Cai <cai@lca.pw>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
This patch is part of "kmsan: Kconfig changes to disable options
incompatible with KMSAN", which was split into smaller pieces.
Change-Id: I9cb6ebbaeb9a38e9e1d015c68ab77d40420a7ad0
---
arch/x86/Kconfig.debug | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
index bf9cd83de777..db3cd6147829 100644
--- a/arch/x86/Kconfig.debug
+++ b/arch/x86/Kconfig.debug
@@ -292,6 +292,9 @@ choice
config UNWINDER_ORC
bool "ORC unwinder"
depends on X86_64
+ # KMSAN doesn't support UNWINDER_ORC yet,
+ # see https://github.com/google/kmsan/issues/48.
+ depends on !KMSAN
select STACK_VALIDATION
---help---
This option enables the ORC (Oops Rewind Capability) unwinder for
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 17/36] kmsan: x86: disable UNWINDER_ORC under KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 17/36] kmsan: x86: disable UNWINDER_ORC " glider
@ 2019-12-02 13:30 ` Marco Elver
0 siblings, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-12-02 13:30 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Qian Cai, Christoph Hellwig, Herbert Xu, Harry Wentland,
Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> KMSAN doesn't currently support UNWINDER_ORC, causing the kernel to
> freeze at boot time.
> See http://github.com/google/kmsan/issues/48.
The Github issues doesn't mention much else -- it might be useful for
future people to elaborate why it doesn't work? If it is unclear why
it doesn't work, then it's fine to mention this in commit message as
well I suppose.
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: Qian Cai <cai@lca.pw>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
> This patch is part of "kmsan: Kconfig changes to disable options
> incompatible with KMSAN", which was split into smaller pieces.
>
> Change-Id: I9cb6ebbaeb9a38e9e1d015c68ab77d40420a7ad0
> ---
> arch/x86/Kconfig.debug | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
> index bf9cd83de777..db3cd6147829 100644
> --- a/arch/x86/Kconfig.debug
> +++ b/arch/x86/Kconfig.debug
> @@ -292,6 +292,9 @@ choice
> config UNWINDER_ORC
> bool "ORC unwinder"
> depends on X86_64
> + # KMSAN doesn't support UNWINDER_ORC yet,
> + # see https://github.com/google/kmsan/issues/48.
> + depends on !KMSAN
> select STACK_VALIDATION
> ---help---
> This option enables the ORC (Oops Rewind Capability) unwinder for
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (16 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 17/36] kmsan: x86: disable UNWINDER_ORC " glider
@ 2019-11-22 11:26 ` glider
2019-12-02 13:33 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 20/36] kmsan: x86: increase stack sizes in KMSAN builds glider
` (17 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Eric Biggers, Qian Cai, Christoph Hellwig, Herbert Xu,
Harry Wentland, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, darrick.wong, davem, dmitry.torokhov,
edumazet, ericvh, gregkh, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
KMSAN doesn't currently support lock debugging, causing the kernel to
enter infinite recursion at boot time.
See https://github.com/google/kmsan/issues/57.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Eric Biggers <ebiggers@google.com>
Cc: Qian Cai <cai@lca.pw>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
This patch is part of "kmsan: Kconfig changes to disable options
incompatible with KMSAN", which was split into smaller pieces.
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 75c36318943d..a3f6f5d68593 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)"
config LOCK_DEBUGGING_SUPPORT
bool
depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
+ # KMSAN is incompatible with lockdep,
+ # see https://github.com/google/kmsan/issues/57.
+ depends on !KMSAN
default y
config PROVE_LOCKING
Change-Id: I4f97edc8a02d8ca208fc914e55e8f0c23d74eac8
---
lib/Kconfig.debug | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 75c36318943d..a3f6f5d68593 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)"
config LOCK_DEBUGGING_SUPPORT
bool
depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
+ # KMSAN is incompatible with lockdep,
+ # see https://github.com/google/kmsan/issues/57.
+ depends on !KMSAN
default y
config PROVE_LOCKING
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-11-22 11:26 ` [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT glider
@ 2019-12-02 13:33 ` Marco Elver
2019-12-03 14:34 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-02 13:33 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Eric Biggers, Qian Cai, Christoph Hellwig, Herbert Xu,
Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> KMSAN doesn't currently support lock debugging, causing the kernel to
> enter infinite recursion at boot time.
> See https://github.com/google/kmsan/issues/57.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: Eric Biggers <ebiggers@google.com>
> Cc: Qian Cai <cai@lca.pw>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
> This patch is part of "kmsan: Kconfig changes to disable options
> incompatible with KMSAN", which was split into smaller pieces.
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 75c36318943d..a3f6f5d68593 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)"
> config LOCK_DEBUGGING_SUPPORT
> bool
> depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
> + # KMSAN is incompatible with lockdep,
> + # see https://github.com/google/kmsan/issues/57.
> + depends on !KMSAN
> default y
>
> config PROVE_LOCKING
>
> Change-Id: I4f97edc8a02d8ca208fc914e55e8f0c23d74eac8
> ---
> lib/Kconfig.debug | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 75c36318943d..a3f6f5d68593 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)"
> config LOCK_DEBUGGING_SUPPORT
> bool
> depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
> + # KMSAN is incompatible with lockdep,
> + # see https://github.com/google/kmsan/issues/57.
Does it make sense to get this working? Or rather, is it feasible to
get working? If not, I would just remove the reference to the Github
issue, and declare that KMSAN is incompatible with lockdep.
> + depends on !KMSAN
> default y
>
> config PROVE_LOCKING
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-02 13:33 ` Marco Elver
@ 2019-12-03 14:34 ` Alexander Potapenko
2019-12-03 15:00 ` Qian Cai
0 siblings, 1 reply; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 14:34 UTC (permalink / raw)
To: Marco Elver
Cc: Eric Biggers, Qian Cai, Christoph Hellwig, Herbert Xu,
Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 2:33 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
> >
> > KMSAN doesn't currently support lock debugging, causing the kernel to
> > enter infinite recursion at boot time.
> > See https://github.com/google/kmsan/issues/57.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > Cc: Eric Biggers <ebiggers@google.com>
> > Cc: Qian Cai <cai@lca.pw>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: Herbert Xu <herbert@gondor.apana.org.au>
> > Cc: Harry Wentland <harry.wentland@amd.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> >
> > ---
> > This patch is part of "kmsan: Kconfig changes to disable options
> > incompatible with KMSAN", which was split into smaller pieces.
> >
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 75c36318943d..a3f6f5d68593 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)"
> > config LOCK_DEBUGGING_SUPPORT
> > bool
> > depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
> > + # KMSAN is incompatible with lockdep,
> > + # see https://github.com/google/kmsan/issues/57.
> > + depends on !KMSAN
> > default y
> >
> > config PROVE_LOCKING
> >
> > Change-Id: I4f97edc8a02d8ca208fc914e55e8f0c23d74eac8
> > ---
> > lib/Kconfig.debug | 3 +++
> > 1 file changed, 3 insertions(+)
> >
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 75c36318943d..a3f6f5d68593 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)"
> > config LOCK_DEBUGGING_SUPPORT
> > bool
> > depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
> > + # KMSAN is incompatible with lockdep,
> > + # see https://github.com/google/kmsan/issues/57.
>
> Does it make sense to get this working? Or rather, is it feasible to
> get working? If not, I would just remove the reference to the Github
> issue, and declare that KMSAN is incompatible with lockdep.
Agreed.
At this point I don't really know why KMSAN and lockdep don't play
well together, but I'm not expecting anyone to use them together
either.
> > + depends on !KMSAN
> > default y
> >
> > config PROVE_LOCKING
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-03 14:34 ` Alexander Potapenko
@ 2019-12-03 15:00 ` Qian Cai
2019-12-03 15:14 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Qian Cai @ 2019-12-03 15:00 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Marco Elver, Eric Biggers, Christoph Hellwig, Herbert Xu,
Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
> On Dec 3, 2019, at 9:35 AM, Alexander Potapenko <glider@google.com> wrote:
>
> At this point I don't really know why KMSAN and lockdep don't play
> well together, but I'm not expecting anyone to use them together
> either.
Of course people will use those together. For example, distro debug kernel variants.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-03 15:00 ` Qian Cai
@ 2019-12-03 15:14 ` Alexander Potapenko
2019-12-03 18:02 ` Qian Cai
2019-12-03 18:38 ` Steven Rostedt
0 siblings, 2 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 15:14 UTC (permalink / raw)
To: Qian Cai
Cc: Marco Elver, Eric Biggers, Christoph Hellwig, Herbert Xu,
Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, Dec 3, 2019 at 4:00 PM Qian Cai <cai@lca.pw> wrote:
>
>
>
> > On Dec 3, 2019, at 9:35 AM, Alexander Potapenko <glider@google.com> wrote:
> >
> > At this point I don't really know why KMSAN and lockdep don't play
> > well together, but I'm not expecting anyone to use them together
> > either.
>
> Of course people will use those together. For example, distro debug kernel variants.
Some tools are just not designed to work together.
For example, you won't be able to compile the kernel with both KASAN
and KMSAN enabled at the same time.
Lockdep doesn't require any instrumentation to work, so it _might_ be
possible to make it work with KMSAN, but it will probably still slow
down the things to an unacceptable level.
I'm inclining towards disabling the two together for now, unless
anyone is willing to address that issue.
Please let me know if you think I need to keep the link to
https://github.com/google/kmsan/issues/57 in the Kconfig comment,
right now it looks like:
# KMSAN is currently incompatible with lockdep.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-03 15:14 ` Alexander Potapenko
@ 2019-12-03 18:02 ` Qian Cai
2019-12-03 18:38 ` Steven Rostedt
1 sibling, 0 replies; 120+ messages in thread
From: Qian Cai @ 2019-12-03 18:02 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Marco Elver, Eric Biggers, Christoph Hellwig, Herbert Xu,
Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
> On Dec 3, 2019, at 10:14 AM, Alexander Potapenko <glider@google.com> wrote:
>
> Some tools are just not designed to work together.
> For example, you won't be able to compile the kernel with both KASAN
> and KMSAN enabled at the same time.
>
> Lockdep doesn't require any instrumentation to work, so it _might_ be
> possible to make it work with KMSAN, but it will probably still slow
> down the things to an unacceptable level.
> I'm inclining towards disabling the two together for now, unless
> anyone is willing to address that issue.
>
> Please let me know if you think I need to keep the link to
> https://github.com/google/kmsan/issues/57 in the Kconfig comment,
> right now it looks like:
>
> # KMSAN is currently incompatible with lockdep.
Then, I don’t see much value with KMSAN get merged. Although In theory, it does cover a bit more than the existing memory debugging infrastructure, but the question is does it worth it? Maybe it needs more data to show a list of bugs it found that existing debugging options could not find?
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-03 15:14 ` Alexander Potapenko
2019-12-03 18:02 ` Qian Cai
@ 2019-12-03 18:38 ` Steven Rostedt
2019-12-04 8:41 ` Alexander Potapenko
1 sibling, 1 reply; 120+ messages in thread
From: Steven Rostedt @ 2019-12-03 18:38 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Qian Cai, Marco Elver, Eric Biggers, Christoph Hellwig,
Herbert Xu, Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Takashi Iwai, Theodore Ts'o,
Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, 3 Dec 2019 16:14:17 +0100
Alexander Potapenko <glider@google.com> wrote:
> On Tue, Dec 3, 2019 at 4:00 PM Qian Cai <cai@lca.pw> wrote:
> >
> >
> >
> > > On Dec 3, 2019, at 9:35 AM, Alexander Potapenko <glider@google.com> wrote:
> > >
> > > At this point I don't really know why KMSAN and lockdep don't play
> > > well together, but I'm not expecting anyone to use them together
> > > either.
> >
> > Of course people will use those together. For example, distro debug kernel variants.
> Some tools are just not designed to work together.
> For example, you won't be able to compile the kernel with both KASAN
> and KMSAN enabled at the same time.
>
> Lockdep doesn't require any instrumentation to work, so it _might_ be
> possible to make it work with KMSAN, but it will probably still slow
> down the things to an unacceptable level.
> I'm inclining towards disabling the two together for now, unless
> anyone is willing to address that issue.
Note, I'm much more interested in someone running with lockdep than
with KMSAN. Thus, you may not get as much use if you do not work with
lockdep. I enable lockdep first when testing out my code. If KMSAN is
not compatible, it wont get enabled.
>
> Please let me know if you think I need to keep the link to
> https://github.com/google/kmsan/issues/57 in the Kconfig comment,
> right now it looks like:
>
> # KMSAN is currently incompatible with lockdep.
>
Function tracing had lots of issues with lockdep, but I worked hard to
make sure they could be compatible. This usually required having the
lockdep code not be traced. Is it possible to have the same with KMSAN.
That is, have KMSAN not look at anything that lockdep does?
-- Steve
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-03 18:38 ` Steven Rostedt
@ 2019-12-04 8:41 ` Alexander Potapenko
2019-12-04 12:22 ` Petr Mladek
0 siblings, 1 reply; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-04 8:41 UTC (permalink / raw)
To: Steven Rostedt
Cc: Qian Cai, Marco Elver, Eric Biggers, Christoph Hellwig,
Herbert Xu, Harry Wentland, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Takashi Iwai, Theodore Ts'o,
Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, Dec 3, 2019 at 7:38 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Tue, 3 Dec 2019 16:14:17 +0100
> Alexander Potapenko <glider@google.com> wrote:
>
> > On Tue, Dec 3, 2019 at 4:00 PM Qian Cai <cai@lca.pw> wrote:
> > >
> > >
> > >
> > > > On Dec 3, 2019, at 9:35 AM, Alexander Potapenko <glider@google.com> wrote:
> > > >
> > > > At this point I don't really know why KMSAN and lockdep don't play
> > > > well together, but I'm not expecting anyone to use them together
> > > > either.
> > >
> > > Of course people will use those together. For example, distro debug kernel variants.
> > Some tools are just not designed to work together.
> > For example, you won't be able to compile the kernel with both KASAN
> > and KMSAN enabled at the same time.
> >
> > Lockdep doesn't require any instrumentation to work, so it _might_ be
> > possible to make it work with KMSAN, but it will probably still slow
> > down the things to an unacceptable level.
> > I'm inclining towards disabling the two together for now, unless
> > anyone is willing to address that issue.
>
> Note, I'm much more interested in someone running with lockdep than
> with KMSAN. Thus, you may not get as much use if you do not work with
> lockdep. I enable lockdep first when testing out my code. If KMSAN is
> not compatible, it wont get enabled.
>
> >
> > Please let me know if you think I need to keep the link to
> > https://github.com/google/kmsan/issues/57 in the Kconfig comment,
> > right now it looks like:
> >
> > # KMSAN is currently incompatible with lockdep.
> >
>
> Function tracing had lots of issues with lockdep, but I worked hard to
> make sure they could be compatible. This usually required having the
> lockdep code not be traced. Is it possible to have the same with KMSAN.
> That is, have KMSAN not look at anything that lockdep does?
>
> -- Steve
Qian, Steve, thanks for the data points.
I thought it might be more valuable to cut some edges to make KMSAN
available to early users, but in the case most developers use a single
build for testing it indeed makes more sense to fix lockdep
interoperability.
I'll look into that.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-04 8:41 ` Alexander Potapenko
@ 2019-12-04 12:22 ` Petr Mladek
2019-12-04 13:12 ` Qian Cai
0 siblings, 1 reply; 120+ messages in thread
From: Petr Mladek @ 2019-12-04 12:22 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Steven Rostedt, Qian Cai, Marco Elver, Eric Biggers,
Christoph Hellwig, Herbert Xu, Harry Wentland, Vegard Nossum,
Dmitry Vyukov, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Wed 2019-12-04 09:41:15, Alexander Potapenko wrote:
> On Tue, Dec 3, 2019 at 7:38 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > On Tue, 3 Dec 2019 16:14:17 +0100
> > Alexander Potapenko <glider@google.com> wrote:
> >
> > > On Tue, Dec 3, 2019 at 4:00 PM Qian Cai <cai@lca.pw> wrote:
> > > >
> > > >
> > > >
> > > > > On Dec 3, 2019, at 9:35 AM, Alexander Potapenko <glider@google.com> wrote:
> > > > >
> > > > > At this point I don't really know why KMSAN and lockdep don't play
> > > > > well together, but I'm not expecting anyone to use them together
> > > > > either.
> > > >
> > > > Of course people will use those together. For example, distro debug kernel variants.
> > > Some tools are just not designed to work together.
> > > For example, you won't be able to compile the kernel with both KASAN
> > > and KMSAN enabled at the same time.
> > >
> > > Lockdep doesn't require any instrumentation to work, so it _might_ be
> > > possible to make it work with KMSAN, but it will probably still slow
> > > down the things to an unacceptable level.
> > > I'm inclining towards disabling the two together for now, unless
> > > anyone is willing to address that issue.
> >
> > Note, I'm much more interested in someone running with lockdep than
> > with KMSAN. Thus, you may not get as much use if you do not work with
> > lockdep. I enable lockdep first when testing out my code. If KMSAN is
> > not compatible, it wont get enabled.
> >
> > >
> > > Please let me know if you think I need to keep the link to
> > > https://github.com/google/kmsan/issues/57 in the Kconfig comment,
> > > right now it looks like:
> > >
> > > # KMSAN is currently incompatible with lockdep.
> > >
> >
> > Function tracing had lots of issues with lockdep, but I worked hard to
> > make sure they could be compatible. This usually required having the
> > lockdep code not be traced. Is it possible to have the same with KMSAN.
> > That is, have KMSAN not look at anything that lockdep does?
> >
> > -- Steve
> Qian, Steve, thanks for the data points.
> I thought it might be more valuable to cut some edges to make KMSAN
> available to early users,
Makes sense to me.
> but in the case most developers use a single
> build for testing it indeed makes more sense to fix lockdep
> interoperability.
> I'll look into that.
IMHO, a generic distro debug kernel could not enable debugging
options that would significantly slow down the kernel. Most users
would refuse using such kernels. Also they might slow down
the system to the point that many problems won't be visible.
For example, I do not see KASAN enabled in SUSE debug kernel.
I guess that it will be the same with KMSAN. These options
can be enabled in custom kernels when debugging particular
problems.
Best Regards,
Petr
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-04 12:22 ` Petr Mladek
@ 2019-12-04 13:12 ` Qian Cai
2019-12-04 16:24 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Qian Cai @ 2019-12-04 13:12 UTC (permalink / raw)
To: Petr Mladek
Cc: Alexander Potapenko, Steven Rostedt, Marco Elver, Eric Biggers,
Christoph Hellwig, Herbert Xu, Harry Wentland, Vegard Nossum,
Dmitry Vyukov, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
> On Dec 4, 2019, at 7:22 AM, Petr Mladek <pmladek@suse.com> wrote:
>
> IMHO, a generic distro debug kernel could not enable debugging
> options that would significantly slow down the kernel. Most users
> would refuse using such kernels. Also they might slow down
> the system to the point that many problems won't be visible.
>
> For example, I do not see KASAN enabled in SUSE debug kernel.
> I guess that it will be the same with KMSAN. These options
> can be enabled in custom kernels when debugging particular
> problems.
No, KASAN is one of most important debug options to find general issues. It was actually used in many debug kernels like those in Google. In contrast, it will find many issues compared to without even though the performance would suffer in some degrees.
I would even argue that since KASAN is so valuable that it is also desirable to get KMSAN to work together with KASAN by improving the compilers. Otherwise, it is a struggling to choose between KASAN and KMSAN for general debug kernels as KASAN could also cover a subset of KMSAN coverage.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-04 13:12 ` Qian Cai
@ 2019-12-04 16:24 ` Alexander Potapenko
2019-12-04 18:03 ` Qian Cai
0 siblings, 1 reply; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-04 16:24 UTC (permalink / raw)
To: Qian Cai
Cc: Petr Mladek, Steven Rostedt, Marco Elver, Eric Biggers,
Christoph Hellwig, Herbert Xu, Harry Wentland, Vegard Nossum,
Dmitry Vyukov, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Wed, Dec 4, 2019 at 2:12 PM Qian Cai <cai@lca.pw> wrote:
>
>
>
> > On Dec 4, 2019, at 7:22 AM, Petr Mladek <pmladek@suse.com> wrote:
> >
> > IMHO, a generic distro debug kernel could not enable debugging
> > options that would significantly slow down the kernel. Most users
> > would refuse using such kernels. Also they might slow down
> > the system to the point that many problems won't be visible.
> >
> > For example, I do not see KASAN enabled in SUSE debug kernel.
> > I guess that it will be the same with KMSAN. These options
> > can be enabled in custom kernels when debugging particular
> > problems.
>
> No, KASAN is one of most important debug options to find general issues. It was actually used in many debug kernels like those in Google. In contrast, it will find many issues compared to without even though the performance would suffer in some degrees.
>
> I would even argue that since KASAN is so valuable that it is also desirable to get KMSAN to work together with KASAN by improving the compilers. Otherwise, it is a struggling to choose between KASAN and KMSAN for general debug kernels as KASAN could also cover a subset of KMSAN coverage.
Turns out it was fairly easy to make KMSAN work with lockdep, sorry
for the noise.
I'll send the updated patch that disables instrumentation of
kernel/locking/lockdep.c in v4.
While on this, I want to make a point on KMSAN features that seem to
be misunderstood.
KMSAN uses precise bit-to-bit shadow mapping and compiler
instrumentation for shadow tracking to tell whether a value is used in
a comparison, pointer dereference or is copied to hardware or
userspace.
As long as the use of such value doesn't cause an immediate crash or a
memory corruption, KASAN is simply unable to detect such bugs.
Neither are any other existing tools, although they may also report
errors later, after the use of uninitialized values.
There's a list of bugs found by syzbot running KMSAN
(https://syzkaller.appspot.com/upstream/fixed?manager=ci-upstream-kmsan-gce),
and I would say more than a hundred of those is unique to KMSAN.
One can think of KMSAN as a _fast_ replacement of kmemcheck.
Unfortunately, the latter bit-rotted at some point and was deleted
from the kernel (I don't think distro debug kernels ever used it
though).
From our experience building userspace tools, making KMSAN and KASAN
work together isn't worth it.
The resulting monster will be slower than any of the two tools and
will be using more memory than both of them together.
This means that anyone who's using at least two machines for fuzzing
or continuous integration will prefer running two different tools on
them instead of running KMSAN+KASAN together on every machine.
That said, I'll try my best to keep existing low-overhead debug
configs working with KMSAN, but it still won't solve the need of
running orthogonal kernel configs for testing.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT
2019-12-04 16:24 ` Alexander Potapenko
@ 2019-12-04 18:03 ` Qian Cai
0 siblings, 0 replies; 120+ messages in thread
From: Qian Cai @ 2019-12-04 18:03 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Petr Mladek, Steven Rostedt, Marco Elver, Eric Biggers,
Christoph Hellwig, Herbert Xu, Harry Wentland, Vegard Nossum,
Dmitry Vyukov, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
> On Dec 4, 2019, at 11:24 AM, Alexander Potapenko <glider@google.com> wrote:
>
> One can think of KMSAN as a _fast_ replacement of kmemcheck.
> Unfortunately, the latter bit-rotted at some point and was deleted
> from the kernel (I don't think distro debug kernels ever used it
> though).
>
> From our experience building userspace tools, making KMSAN and KASAN
> work together isn't worth it.
> The resulting monster will be slower than any of the two tools and
> will be using more memory than both of them together.
> This means that anyone who's using at least two machines for fuzzing
> or continuous integration will prefer running two different tools on
> them instead of running KMSAN+KASAN together on every machine.
>
> That said, I'll try my best to keep existing low-overhead debug
> configs working with KMSAN, but it still won't solve the need of
> running orthogonal kernel configs for testing.
The problem is that we have too many memory-related debugging options in kernel. Those tend to break over time mainly because not many paid kernel developers would care about them, and most of Kernel developers nowadays are paid. Not many employers would like Google to spend money on long-term health of the kernel because those debugging options does not generate revenues. For example, kmemleak was broken on NUMA for many releases and lockdep PROVE_LOCKING still has too many unfixed splats right now which would render it almost useless for general debugging because it will disable itself after found a splat.
Every of those does somewhat different and yet have many overlapping that yet nobody put any effort to consolidate them. It is even more cumbersome for the options to break without notice because it does not play well with popular ones like KASAN given code for those features are spanning all over the kernel code base, so they are sensitive to changes from other subsystems. It put a burden for bug hunters to enable them because it is going to be expensive increasing test runs time exponentially and maintenance cost of different debug kernels.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 20/36] kmsan: x86: increase stack sizes in KMSAN builds
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (17 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 18/36] kmsan: disable LOCK_DEBUGGING_SUPPORT glider
@ 2019-11-22 11:26 ` glider
2019-12-02 14:31 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts glider
` (16 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin,
ard.biesheuvel, arnd, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso,
tglx, gor, wsa
KMSAN instruments the code heavily, increasing register pressure and
preventing functions from being inlined. As a result, the kernel
requires more stack space to run.
Rename KASAN_STACK_ORDER to EXTRA_STACK_ORDER and set EXTRA_STACK_ORDER
to 2 for KMSAN builds, effectively making the stacks 4 times larger.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: linux-mm@kvack.org
---
Change-Id: I1d9df161419a885bf654abff141e247366895b68
---
arch/x86/include/asm/page_64_types.h | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h
index 288b065955b7..ea9fbf09f43b 100644
--- a/arch/x86/include/asm/page_64_types.h
+++ b/arch/x86/include/asm/page_64_types.h
@@ -7,18 +7,20 @@
#endif
#ifdef CONFIG_KASAN
-#define KASAN_STACK_ORDER 1
+#define EXTRA_STACK_ORDER 1
+#elif defined(CONFIG_KMSAN)
+#define EXTRA_STACK_ORDER 2
#else
-#define KASAN_STACK_ORDER 0
+#define EXTRA_STACK_ORDER 0
#endif
-#define THREAD_SIZE_ORDER (2 + KASAN_STACK_ORDER)
+#define THREAD_SIZE_ORDER (2 + EXTRA_STACK_ORDER)
#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER)
-#define EXCEPTION_STACK_ORDER (0 + KASAN_STACK_ORDER)
+#define EXCEPTION_STACK_ORDER (0 + EXTRA_STACK_ORDER)
#define EXCEPTION_STKSZ (PAGE_SIZE << EXCEPTION_STACK_ORDER)
-#define IRQ_STACK_ORDER (2 + KASAN_STACK_ORDER)
+#define IRQ_STACK_ORDER (2 + EXTRA_STACK_ORDER)
#define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER)
/*
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 20/36] kmsan: x86: increase stack sizes in KMSAN builds
2019-11-22 11:26 ` [PATCH RFC v3 20/36] kmsan: x86: increase stack sizes in KMSAN builds glider
@ 2019-12-02 14:31 ` Marco Elver
0 siblings, 0 replies; 120+ messages in thread
From: Marco Elver @ 2019-12-02 14:31 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Darrick J. Wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> KMSAN instruments the code heavily, increasing register pressure and
> preventing functions from being inlined. As a result, the kernel
> requires more stack space to run.
>
> Rename KASAN_STACK_ORDER to EXTRA_STACK_ORDER and set EXTRA_STACK_ORDER
> to 2 for KMSAN builds, effectively making the stacks 4 times larger.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: linux-mm@kvack.org
Acked-by: Marco Elver <elver@google.com>
Thanks,
-- Marco
> ---
>
> Change-Id: I1d9df161419a885bf654abff141e247366895b68
> ---
> arch/x86/include/asm/page_64_types.h | 12 +++++++-----
> 1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h
> index 288b065955b7..ea9fbf09f43b 100644
> --- a/arch/x86/include/asm/page_64_types.h
> +++ b/arch/x86/include/asm/page_64_types.h
> @@ -7,18 +7,20 @@
> #endif
>
> #ifdef CONFIG_KASAN
> -#define KASAN_STACK_ORDER 1
> +#define EXTRA_STACK_ORDER 1
> +#elif defined(CONFIG_KMSAN)
> +#define EXTRA_STACK_ORDER 2
> #else
> -#define KASAN_STACK_ORDER 0
> +#define EXTRA_STACK_ORDER 0
> #endif
>
> -#define THREAD_SIZE_ORDER (2 + KASAN_STACK_ORDER)
> +#define THREAD_SIZE_ORDER (2 + EXTRA_STACK_ORDER)
> #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER)
>
> -#define EXCEPTION_STACK_ORDER (0 + KASAN_STACK_ORDER)
> +#define EXCEPTION_STACK_ORDER (0 + EXTRA_STACK_ORDER)
> #define EXCEPTION_STKSZ (PAGE_SIZE << EXCEPTION_STACK_ORDER)
>
> -#define IRQ_STACK_ORDER (2 + KASAN_STACK_ORDER)
> +#define IRQ_STACK_ORDER (2 + EXTRA_STACK_ORDER)
> #define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER)
>
> /*
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (18 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 20/36] kmsan: x86: increase stack sizes in KMSAN builds glider
@ 2019-11-22 11:26 ` glider
2019-11-29 15:07 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 22/36] kmsan: mm: call KMSAN hooks from SLUB code glider
` (15 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Ard Biesheuvel, Thomas Gleixner, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
arnd, hch, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
edumazet, ericvh, gregkh, harry.wentland, herbert, iii, mingo,
jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso,
gor, wsa
Instrumenting some files with KMSAN will result in kernel being unable
to link, boot or crashing at runtime for various reasons (e.g. infinite
recursion caused by instrumentation hooks calling instrumented code again).
Disable KMSAN in the following places:
- arch/x86/boot and arch/x86/realmode/rm, as KMSAN doesn't work for i386;
- arch/x86/entry/vdso, which isn't linked with KMSAN runtime;
- three files in arch/x86/kernel - boot problems;
- arch/x86/mm/cpu_entry_area.c - recursion;
- EFI stub - build failures;
- kcov, stackdepot - recursion.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I90961eabf2dcb9ae992aed259088953bad5e4d6d
---
arch/x86/boot/Makefile | 2 ++
arch/x86/boot/compressed/Makefile | 2 ++
arch/x86/entry/vdso/Makefile | 3 +++
arch/x86/kernel/Makefile | 4 ++++
arch/x86/kernel/cpu/Makefile | 1 +
arch/x86/mm/Makefile | 2 ++
arch/x86/realmode/rm/Makefile | 2 ++
drivers/firmware/efi/libstub/Makefile | 1 +
kernel/Makefile | 1 +
lib/Makefile | 1 +
10 files changed, 19 insertions(+)
diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
index e2839b5c246c..c039abd4c81f 100644
--- a/arch/x86/boot/Makefile
+++ b/arch/x86/boot/Makefile
@@ -10,6 +10,8 @@
#
KASAN_SANITIZE := n
+# KMSAN doesn't work for i386
+KMSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Kernel does not boot with kcov instrumentation here.
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 6b84afdd7538..9efe2d9fca4c 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -18,6 +18,8 @@
# compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
KASAN_SANITIZE := n
+# KMSAN doesn't work for i386
+KMSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 0f2154106d01..000467a1a4f2 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -11,6 +11,9 @@ include $(srctree)/lib/vdso/Makefile
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
+# Undefined references to KMSAN hooks.
+KMSAN_SANITIZE_vclock_gettime.o := n
+KMSAN_SANITIZE_vgetcpu.o := n
UBSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 3578ad248bc9..ce39972a7edf 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -28,6 +28,10 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n
KASAN_SANITIZE_stacktrace.o := n
KASAN_SANITIZE_paravirt.o := n
+# Work around reboot loop.
+KMSAN_SANITIZE_head$(BITS).o := n
+KMSAN_SANITIZE_nmi.o := n
+
OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o := y
OBJECT_FILES_NON_STANDARD_test_nx.o := y
OBJECT_FILES_NON_STANDARD_paravirt_patch.o := y
diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
index d7a1e5a9331c..41f4f8f2f2f0 100644
--- a/arch/x86/kernel/cpu/Makefile
+++ b/arch/x86/kernel/cpu/Makefile
@@ -12,6 +12,7 @@ endif
# If these files are instrumented, boot hangs during the first second.
KCOV_INSTRUMENT_common.o := n
KCOV_INSTRUMENT_perf_event.o := n
+KMSAN_SANITIZE_common.o := n
# Make sure load_percpu_segment has no stackprotector
nostackp := $(call cc-option, -fno-stack-protector)
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 84373dc9b341..42cb3a6409b0 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -7,6 +7,8 @@ KCOV_INSTRUMENT_mem_encrypt_identity.o := n
KASAN_SANITIZE_mem_encrypt.o := n
KASAN_SANITIZE_mem_encrypt_identity.o := n
+KMSAN_SANITIZE_cpu_entry_area.o := n
+
ifdef CONFIG_FUNCTION_TRACER
CFLAGS_REMOVE_mem_encrypt.o = -pg
CFLAGS_REMOVE_mem_encrypt_identity.o = -pg
diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile
index f60501a384f9..27e7bc0bbdde 100644
--- a/arch/x86/realmode/rm/Makefile
+++ b/arch/x86/realmode/rm/Makefile
@@ -7,6 +7,8 @@
#
#
KASAN_SANITIZE := n
+# KMSAN doesn't work for i386
+KMSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index 0460c7581220..11869c17a64c 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -32,6 +32,7 @@ KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \
GCOV_PROFILE := n
KASAN_SANITIZE := n
+KMSAN_SANITIZE := n
UBSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
diff --git a/kernel/Makefile b/kernel/Makefile
index daad787fb795..5fd6fbca2592 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -30,6 +30,7 @@ KCOV_INSTRUMENT_extable.o := n
# Don't self-instrument.
KCOV_INSTRUMENT_kcov.o := n
KASAN_SANITIZE_kcov.o := n
+KMSAN_SANITIZE_kcov.o := n
CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
# cond_syscall is currently not LTO compatible
diff --git a/lib/Makefile b/lib/Makefile
index 08fcb37499a0..ae6e57d857b0 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -222,6 +222,7 @@ obj-$(CONFIG_IRQ_POLL) += irq_poll.o
CFLAGS_stackdepot.o += -fno-builtin
obj-$(CONFIG_STACKDEPOT) += stackdepot.o
KASAN_SANITIZE_stackdepot.o := n
+KMSAN_SANITIZE_stackdepot.o := n
KCOV_INSTRUMENT_stackdepot.o := n
libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts
2019-11-22 11:26 ` [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts glider
@ 2019-11-29 15:07 ` Andrey Konovalov
2019-12-10 10:35 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 15:07 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Ard Biesheuvel, Thomas Gleixner, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Andy Lutomirski, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, pmladek,
Qian Cai, Randy Dunlap, Robin Murphy, sergey.senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> Instrumenting some files with KMSAN will result in kernel being unable
> to link, boot or crashing at runtime for various reasons (e.g. infinite
> recursion caused by instrumentation hooks calling instrumented code again).
>
> Disable KMSAN in the following places:
> - arch/x86/boot and arch/x86/realmode/rm, as KMSAN doesn't work for i386;
> - arch/x86/entry/vdso, which isn't linked with KMSAN runtime;
> - three files in arch/x86/kernel - boot problems;
> - arch/x86/mm/cpu_entry_area.c - recursion;
> - EFI stub - build failures;
> - kcov, stackdepot - recursion.
It makes sense to unify comments explaining the reasons for
KMSAN_SANITIZE := n with KCSAN patches:
https://patchwork.kernel.org/patch/11244145/
https://patchwork.kernel.org/patch/11244161/
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: I90961eabf2dcb9ae992aed259088953bad5e4d6d
> ---
> arch/x86/boot/Makefile | 2 ++
> arch/x86/boot/compressed/Makefile | 2 ++
> arch/x86/entry/vdso/Makefile | 3 +++
> arch/x86/kernel/Makefile | 4 ++++
> arch/x86/kernel/cpu/Makefile | 1 +
> arch/x86/mm/Makefile | 2 ++
> arch/x86/realmode/rm/Makefile | 2 ++
> drivers/firmware/efi/libstub/Makefile | 1 +
> kernel/Makefile | 1 +
> lib/Makefile | 1 +
> 10 files changed, 19 insertions(+)
>
> diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
> index e2839b5c246c..c039abd4c81f 100644
> --- a/arch/x86/boot/Makefile
> +++ b/arch/x86/boot/Makefile
> @@ -10,6 +10,8 @@
> #
>
> KASAN_SANITIZE := n
> +# KMSAN doesn't work for i386
> +KMSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
>
> # Kernel does not boot with kcov instrumentation here.
> diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> index 6b84afdd7538..9efe2d9fca4c 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -18,6 +18,8 @@
> # compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
>
> KASAN_SANITIZE := n
> +# KMSAN doesn't work for i386
> +KMSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
>
> # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
> diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
> index 0f2154106d01..000467a1a4f2 100644
> --- a/arch/x86/entry/vdso/Makefile
> +++ b/arch/x86/entry/vdso/Makefile
> @@ -11,6 +11,9 @@ include $(srctree)/lib/vdso/Makefile
>
> KBUILD_CFLAGS += $(DISABLE_LTO)
> KASAN_SANITIZE := n
> +# Undefined references to KMSAN hooks.
> +KMSAN_SANITIZE_vclock_gettime.o := n
> +KMSAN_SANITIZE_vgetcpu.o := n
> UBSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
>
> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> index 3578ad248bc9..ce39972a7edf 100644
> --- a/arch/x86/kernel/Makefile
> +++ b/arch/x86/kernel/Makefile
> @@ -28,6 +28,10 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n
> KASAN_SANITIZE_stacktrace.o := n
> KASAN_SANITIZE_paravirt.o := n
>
> +# Work around reboot loop.
> +KMSAN_SANITIZE_head$(BITS).o := n
> +KMSAN_SANITIZE_nmi.o := n
> +
> OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o := y
> OBJECT_FILES_NON_STANDARD_test_nx.o := y
> OBJECT_FILES_NON_STANDARD_paravirt_patch.o := y
> diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
> index d7a1e5a9331c..41f4f8f2f2f0 100644
> --- a/arch/x86/kernel/cpu/Makefile
> +++ b/arch/x86/kernel/cpu/Makefile
> @@ -12,6 +12,7 @@ endif
> # If these files are instrumented, boot hangs during the first second.
> KCOV_INSTRUMENT_common.o := n
> KCOV_INSTRUMENT_perf_event.o := n
> +KMSAN_SANITIZE_common.o := n
>
> # Make sure load_percpu_segment has no stackprotector
> nostackp := $(call cc-option, -fno-stack-protector)
> diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
> index 84373dc9b341..42cb3a6409b0 100644
> --- a/arch/x86/mm/Makefile
> +++ b/arch/x86/mm/Makefile
> @@ -7,6 +7,8 @@ KCOV_INSTRUMENT_mem_encrypt_identity.o := n
> KASAN_SANITIZE_mem_encrypt.o := n
> KASAN_SANITIZE_mem_encrypt_identity.o := n
>
> +KMSAN_SANITIZE_cpu_entry_area.o := n
> +
> ifdef CONFIG_FUNCTION_TRACER
> CFLAGS_REMOVE_mem_encrypt.o = -pg
> CFLAGS_REMOVE_mem_encrypt_identity.o = -pg
> diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile
> index f60501a384f9..27e7bc0bbdde 100644
> --- a/arch/x86/realmode/rm/Makefile
> +++ b/arch/x86/realmode/rm/Makefile
> @@ -7,6 +7,8 @@
> #
> #
> KASAN_SANITIZE := n
> +# KMSAN doesn't work for i386
> +KMSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
>
> # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
> diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> index 0460c7581220..11869c17a64c 100644
> --- a/drivers/firmware/efi/libstub/Makefile
> +++ b/drivers/firmware/efi/libstub/Makefile
> @@ -32,6 +32,7 @@ KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \
>
> GCOV_PROFILE := n
> KASAN_SANITIZE := n
> +KMSAN_SANITIZE := n
> UBSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
>
> diff --git a/kernel/Makefile b/kernel/Makefile
> index daad787fb795..5fd6fbca2592 100644
> --- a/kernel/Makefile
> +++ b/kernel/Makefile
> @@ -30,6 +30,7 @@ KCOV_INSTRUMENT_extable.o := n
> # Don't self-instrument.
> KCOV_INSTRUMENT_kcov.o := n
> KASAN_SANITIZE_kcov.o := n
> +KMSAN_SANITIZE_kcov.o := n
> CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
>
> # cond_syscall is currently not LTO compatible
> diff --git a/lib/Makefile b/lib/Makefile
> index 08fcb37499a0..ae6e57d857b0 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -222,6 +222,7 @@ obj-$(CONFIG_IRQ_POLL) += irq_poll.o
> CFLAGS_stackdepot.o += -fno-builtin
> obj-$(CONFIG_STACKDEPOT) += stackdepot.o
> KASAN_SANITIZE_stackdepot.o := n
> +KMSAN_SANITIZE_stackdepot.o := n
> KCOV_INSTRUMENT_stackdepot.o := n
>
> libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts
2019-11-29 15:07 ` Andrey Konovalov
@ 2019-12-10 10:35 ` Alexander Potapenko
2019-12-10 12:38 ` Alexander Potapenko
2019-12-10 12:43 ` Qian Cai
0 siblings, 2 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-10 10:35 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Ard Biesheuvel, Thomas Gleixner, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Andy Lutomirski, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Vasily Gorbik, Wolfram Sang
On Fri, Nov 29, 2019 at 4:07 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > Instrumenting some files with KMSAN will result in kernel being unable
> > to link, boot or crashing at runtime for various reasons (e.g. infinite
> > recursion caused by instrumentation hooks calling instrumented code again).
> >
> > Disable KMSAN in the following places:
> > - arch/x86/boot and arch/x86/realmode/rm, as KMSAN doesn't work for i386;
> > - arch/x86/entry/vdso, which isn't linked with KMSAN runtime;
> > - three files in arch/x86/kernel - boot problems;
> > - arch/x86/mm/cpu_entry_area.c - recursion;
> > - EFI stub - build failures;
> > - kcov, stackdepot - recursion.
>
> It makes sense to unify comments explaining the reasons for
> KMSAN_SANITIZE := n with KCSAN patches:
In principle, yes.
But KCSAN hasn't reached upstream yet, and rebasing on top of it will
increase the number of moving parts to take into account.
> https://patchwork.kernel.org/patch/11244145/
> https://patchwork.kernel.org/patch/11244161/
>
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: I90961eabf2dcb9ae992aed259088953bad5e4d6d
> > ---
> > arch/x86/boot/Makefile | 2 ++
> > arch/x86/boot/compressed/Makefile | 2 ++
> > arch/x86/entry/vdso/Makefile | 3 +++
> > arch/x86/kernel/Makefile | 4 ++++
> > arch/x86/kernel/cpu/Makefile | 1 +
> > arch/x86/mm/Makefile | 2 ++
> > arch/x86/realmode/rm/Makefile | 2 ++
> > drivers/firmware/efi/libstub/Makefile | 1 +
> > kernel/Makefile | 1 +
> > lib/Makefile | 1 +
> > 10 files changed, 19 insertions(+)
> >
> > diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
> > index e2839b5c246c..c039abd4c81f 100644
> > --- a/arch/x86/boot/Makefile
> > +++ b/arch/x86/boot/Makefile
> > @@ -10,6 +10,8 @@
> > #
> >
> > KASAN_SANITIZE := n
> > +# KMSAN doesn't work for i386
> > +KMSAN_SANITIZE := n
> > OBJECT_FILES_NON_STANDARD := y
> >
> > # Kernel does not boot with kcov instrumentation here.
> > diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> > index 6b84afdd7538..9efe2d9fca4c 100644
> > --- a/arch/x86/boot/compressed/Makefile
> > +++ b/arch/x86/boot/compressed/Makefile
> > @@ -18,6 +18,8 @@
> > # compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
> >
> > KASAN_SANITIZE := n
> > +# KMSAN doesn't work for i386
> > +KMSAN_SANITIZE := n
> > OBJECT_FILES_NON_STANDARD := y
> >
> > # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
> > diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
> > index 0f2154106d01..000467a1a4f2 100644
> > --- a/arch/x86/entry/vdso/Makefile
> > +++ b/arch/x86/entry/vdso/Makefile
> > @@ -11,6 +11,9 @@ include $(srctree)/lib/vdso/Makefile
> >
> > KBUILD_CFLAGS += $(DISABLE_LTO)
> > KASAN_SANITIZE := n
> > +# Undefined references to KMSAN hooks.
> > +KMSAN_SANITIZE_vclock_gettime.o := n
> > +KMSAN_SANITIZE_vgetcpu.o := n
> > UBSAN_SANITIZE := n
> > OBJECT_FILES_NON_STANDARD := y
> >
> > diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> > index 3578ad248bc9..ce39972a7edf 100644
> > --- a/arch/x86/kernel/Makefile
> > +++ b/arch/x86/kernel/Makefile
> > @@ -28,6 +28,10 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n
> > KASAN_SANITIZE_stacktrace.o := n
> > KASAN_SANITIZE_paravirt.o := n
> >
> > +# Work around reboot loop.
> > +KMSAN_SANITIZE_head$(BITS).o := n
> > +KMSAN_SANITIZE_nmi.o := n
> > +
> > OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o := y
> > OBJECT_FILES_NON_STANDARD_test_nx.o := y
> > OBJECT_FILES_NON_STANDARD_paravirt_patch.o := y
> > diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
> > index d7a1e5a9331c..41f4f8f2f2f0 100644
> > --- a/arch/x86/kernel/cpu/Makefile
> > +++ b/arch/x86/kernel/cpu/Makefile
> > @@ -12,6 +12,7 @@ endif
> > # If these files are instrumented, boot hangs during the first second.
> > KCOV_INSTRUMENT_common.o := n
> > KCOV_INSTRUMENT_perf_event.o := n
> > +KMSAN_SANITIZE_common.o := n
> >
> > # Make sure load_percpu_segment has no stackprotector
> > nostackp := $(call cc-option, -fno-stack-protector)
> > diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
> > index 84373dc9b341..42cb3a6409b0 100644
> > --- a/arch/x86/mm/Makefile
> > +++ b/arch/x86/mm/Makefile
> > @@ -7,6 +7,8 @@ KCOV_INSTRUMENT_mem_encrypt_identity.o := n
> > KASAN_SANITIZE_mem_encrypt.o := n
> > KASAN_SANITIZE_mem_encrypt_identity.o := n
> >
> > +KMSAN_SANITIZE_cpu_entry_area.o := n
> > +
> > ifdef CONFIG_FUNCTION_TRACER
> > CFLAGS_REMOVE_mem_encrypt.o = -pg
> > CFLAGS_REMOVE_mem_encrypt_identity.o = -pg
> > diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile
> > index f60501a384f9..27e7bc0bbdde 100644
> > --- a/arch/x86/realmode/rm/Makefile
> > +++ b/arch/x86/realmode/rm/Makefile
> > @@ -7,6 +7,8 @@
> > #
> > #
> > KASAN_SANITIZE := n
> > +# KMSAN doesn't work for i386
> > +KMSAN_SANITIZE := n
> > OBJECT_FILES_NON_STANDARD := y
> >
> > # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
> > diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> > index 0460c7581220..11869c17a64c 100644
> > --- a/drivers/firmware/efi/libstub/Makefile
> > +++ b/drivers/firmware/efi/libstub/Makefile
> > @@ -32,6 +32,7 @@ KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \
> >
> > GCOV_PROFILE := n
> > KASAN_SANITIZE := n
> > +KMSAN_SANITIZE := n
> > UBSAN_SANITIZE := n
> > OBJECT_FILES_NON_STANDARD := y
> >
> > diff --git a/kernel/Makefile b/kernel/Makefile
> > index daad787fb795..5fd6fbca2592 100644
> > --- a/kernel/Makefile
> > +++ b/kernel/Makefile
> > @@ -30,6 +30,7 @@ KCOV_INSTRUMENT_extable.o := n
> > # Don't self-instrument.
> > KCOV_INSTRUMENT_kcov.o := n
> > KASAN_SANITIZE_kcov.o := n
> > +KMSAN_SANITIZE_kcov.o := n
> > CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
> >
> > # cond_syscall is currently not LTO compatible
> > diff --git a/lib/Makefile b/lib/Makefile
> > index 08fcb37499a0..ae6e57d857b0 100644
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -222,6 +222,7 @@ obj-$(CONFIG_IRQ_POLL) += irq_poll.o
> > CFLAGS_stackdepot.o += -fno-builtin
> > obj-$(CONFIG_STACKDEPOT) += stackdepot.o
> > KASAN_SANITIZE_stackdepot.o := n
> > +KMSAN_SANITIZE_stackdepot.o := n
> > KCOV_INSTRUMENT_stackdepot.o := n
> >
> > libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts
2019-12-10 10:35 ` Alexander Potapenko
@ 2019-12-10 12:38 ` Alexander Potapenko
2019-12-10 12:43 ` Qian Cai
1 sibling, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-10 12:38 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Ard Biesheuvel, Thomas Gleixner, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Andy Lutomirski, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Vasily Gorbik, Wolfram Sang
On Tue, Dec 10, 2019 at 11:35 AM Alexander Potapenko <glider@google.com> wrote:
>
> On Fri, Nov 29, 2019 at 4:07 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> > >
> > > Instrumenting some files with KMSAN will result in kernel being unable
> > > to link, boot or crashing at runtime for various reasons (e.g. infinite
> > > recursion caused by instrumentation hooks calling instrumented code again).
> > >
> > > Disable KMSAN in the following places:
> > > - arch/x86/boot and arch/x86/realmode/rm, as KMSAN doesn't work for i386;
> > > - arch/x86/entry/vdso, which isn't linked with KMSAN runtime;
> > > - three files in arch/x86/kernel - boot problems;
> > > - arch/x86/mm/cpu_entry_area.c - recursion;
> > > - EFI stub - build failures;
> > > - kcov, stackdepot - recursion.
> >
> > It makes sense to unify comments explaining the reasons for
> > KMSAN_SANITIZE := n with KCSAN patches:
> In principle, yes.
> But KCSAN hasn't reached upstream yet, and rebasing on top of it will
> increase the number of moving parts to take into account.
Ah, I see. Yes, I'll add meaningful comments in v4.
>
> > https://patchwork.kernel.org/patch/11244145/
> > https://patchwork.kernel.org/patch/11244161/
> >
> > >
> > > Signed-off-by: Alexander Potapenko <glider@google.com>
> > > To: Alexander Potapenko <glider@google.com>
> > > Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Cc: Thomas Gleixner <tglx@linutronix.de>
> > > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > > Cc: Dmitry Vyukov <dvyukov@google.com>
> > > Cc: linux-mm@kvack.org
> > > ---
> > >
> > > Change-Id: I90961eabf2dcb9ae992aed259088953bad5e4d6d
> > > ---
> > > arch/x86/boot/Makefile | 2 ++
> > > arch/x86/boot/compressed/Makefile | 2 ++
> > > arch/x86/entry/vdso/Makefile | 3 +++
> > > arch/x86/kernel/Makefile | 4 ++++
> > > arch/x86/kernel/cpu/Makefile | 1 +
> > > arch/x86/mm/Makefile | 2 ++
> > > arch/x86/realmode/rm/Makefile | 2 ++
> > > drivers/firmware/efi/libstub/Makefile | 1 +
> > > kernel/Makefile | 1 +
> > > lib/Makefile | 1 +
> > > 10 files changed, 19 insertions(+)
> > >
> > > diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
> > > index e2839b5c246c..c039abd4c81f 100644
> > > --- a/arch/x86/boot/Makefile
> > > +++ b/arch/x86/boot/Makefile
> > > @@ -10,6 +10,8 @@
> > > #
> > >
> > > KASAN_SANITIZE := n
> > > +# KMSAN doesn't work for i386
> > > +KMSAN_SANITIZE := n
> > > OBJECT_FILES_NON_STANDARD := y
> > >
> > > # Kernel does not boot with kcov instrumentation here.
> > > diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> > > index 6b84afdd7538..9efe2d9fca4c 100644
> > > --- a/arch/x86/boot/compressed/Makefile
> > > +++ b/arch/x86/boot/compressed/Makefile
> > > @@ -18,6 +18,8 @@
> > > # compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
> > >
> > > KASAN_SANITIZE := n
> > > +# KMSAN doesn't work for i386
> > > +KMSAN_SANITIZE := n
> > > OBJECT_FILES_NON_STANDARD := y
> > >
> > > # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
> > > diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
> > > index 0f2154106d01..000467a1a4f2 100644
> > > --- a/arch/x86/entry/vdso/Makefile
> > > +++ b/arch/x86/entry/vdso/Makefile
> > > @@ -11,6 +11,9 @@ include $(srctree)/lib/vdso/Makefile
> > >
> > > KBUILD_CFLAGS += $(DISABLE_LTO)
> > > KASAN_SANITIZE := n
> > > +# Undefined references to KMSAN hooks.
> > > +KMSAN_SANITIZE_vclock_gettime.o := n
> > > +KMSAN_SANITIZE_vgetcpu.o := n
> > > UBSAN_SANITIZE := n
> > > OBJECT_FILES_NON_STANDARD := y
> > >
> > > diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
> > > index 3578ad248bc9..ce39972a7edf 100644
> > > --- a/arch/x86/kernel/Makefile
> > > +++ b/arch/x86/kernel/Makefile
> > > @@ -28,6 +28,10 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n
> > > KASAN_SANITIZE_stacktrace.o := n
> > > KASAN_SANITIZE_paravirt.o := n
> > >
> > > +# Work around reboot loop.
> > > +KMSAN_SANITIZE_head$(BITS).o := n
> > > +KMSAN_SANITIZE_nmi.o := n
> > > +
> > > OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o := y
> > > OBJECT_FILES_NON_STANDARD_test_nx.o := y
> > > OBJECT_FILES_NON_STANDARD_paravirt_patch.o := y
> > > diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
> > > index d7a1e5a9331c..41f4f8f2f2f0 100644
> > > --- a/arch/x86/kernel/cpu/Makefile
> > > +++ b/arch/x86/kernel/cpu/Makefile
> > > @@ -12,6 +12,7 @@ endif
> > > # If these files are instrumented, boot hangs during the first second.
> > > KCOV_INSTRUMENT_common.o := n
> > > KCOV_INSTRUMENT_perf_event.o := n
> > > +KMSAN_SANITIZE_common.o := n
> > >
> > > # Make sure load_percpu_segment has no stackprotector
> > > nostackp := $(call cc-option, -fno-stack-protector)
> > > diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
> > > index 84373dc9b341..42cb3a6409b0 100644
> > > --- a/arch/x86/mm/Makefile
> > > +++ b/arch/x86/mm/Makefile
> > > @@ -7,6 +7,8 @@ KCOV_INSTRUMENT_mem_encrypt_identity.o := n
> > > KASAN_SANITIZE_mem_encrypt.o := n
> > > KASAN_SANITIZE_mem_encrypt_identity.o := n
> > >
> > > +KMSAN_SANITIZE_cpu_entry_area.o := n
> > > +
> > > ifdef CONFIG_FUNCTION_TRACER
> > > CFLAGS_REMOVE_mem_encrypt.o = -pg
> > > CFLAGS_REMOVE_mem_encrypt_identity.o = -pg
> > > diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile
> > > index f60501a384f9..27e7bc0bbdde 100644
> > > --- a/arch/x86/realmode/rm/Makefile
> > > +++ b/arch/x86/realmode/rm/Makefile
> > > @@ -7,6 +7,8 @@
> > > #
> > > #
> > > KASAN_SANITIZE := n
> > > +# KMSAN doesn't work for i386
> > > +KMSAN_SANITIZE := n
> > > OBJECT_FILES_NON_STANDARD := y
> > >
> > > # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
> > > diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> > > index 0460c7581220..11869c17a64c 100644
> > > --- a/drivers/firmware/efi/libstub/Makefile
> > > +++ b/drivers/firmware/efi/libstub/Makefile
> > > @@ -32,6 +32,7 @@ KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \
> > >
> > > GCOV_PROFILE := n
> > > KASAN_SANITIZE := n
> > > +KMSAN_SANITIZE := n
> > > UBSAN_SANITIZE := n
> > > OBJECT_FILES_NON_STANDARD := y
> > >
> > > diff --git a/kernel/Makefile b/kernel/Makefile
> > > index daad787fb795..5fd6fbca2592 100644
> > > --- a/kernel/Makefile
> > > +++ b/kernel/Makefile
> > > @@ -30,6 +30,7 @@ KCOV_INSTRUMENT_extable.o := n
> > > # Don't self-instrument.
> > > KCOV_INSTRUMENT_kcov.o := n
> > > KASAN_SANITIZE_kcov.o := n
> > > +KMSAN_SANITIZE_kcov.o := n
> > > CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
> > >
> > > # cond_syscall is currently not LTO compatible
> > > diff --git a/lib/Makefile b/lib/Makefile
> > > index 08fcb37499a0..ae6e57d857b0 100644
> > > --- a/lib/Makefile
> > > +++ b/lib/Makefile
> > > @@ -222,6 +222,7 @@ obj-$(CONFIG_IRQ_POLL) += irq_poll.o
> > > CFLAGS_stackdepot.o += -fno-builtin
> > > obj-$(CONFIG_STACKDEPOT) += stackdepot.o
> > > KASAN_SANITIZE_stackdepot.o := n
> > > +KMSAN_SANITIZE_stackdepot.o := n
> > > KCOV_INSTRUMENT_stackdepot.o := n
> > >
> > > libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \
> > > --
> > > 2.24.0.432.g9d3f5f5b63-goog
> > >
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts
2019-12-10 10:35 ` Alexander Potapenko
2019-12-10 12:38 ` Alexander Potapenko
@ 2019-12-10 12:43 ` Qian Cai
1 sibling, 0 replies; 120+ messages in thread
From: Qian Cai @ 2019-12-10 12:43 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Andrey Konovalov, Ard Biesheuvel, Thomas Gleixner, Vegard Nossum,
Dmitry Vyukov, Linux Memory Management List, Alexander Viro,
Andreas Dilger, Andrew Morton, Andrey Ryabinin, Andy Lutomirski,
Arnd Bergmann, Christoph Hellwig, Christoph Hellwig,
Darrick J. Wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, Petr Mladek,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Vasily Gorbik, Wolfram Sang
> On Dec 10, 2019, at 5:35 AM, Alexander Potapenko <glider@google.com> wrote:
>
> In principle, yes.
> But KCSAN hasn't reached upstream yet, and rebasing on top of it will
> increase the number of moving parts to take into account.
KCSAN is now in linux-next, so it makes sense to sync with it.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 22/36] kmsan: mm: call KMSAN hooks from SLUB code
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (19 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 21/36] kmsan: disable KMSAN instrumentation for certain kernel parts glider
@ 2019-11-22 11:26 ` glider
2019-12-02 15:36 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed glider
` (14 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Andrew Morton, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
In order to report uninitialized memory coming from heap allocations
KMSAN has to poison them unless they're created with __GFP_ZERO.
It's handy that we need KMSAN hooks in the places where
init_on_alloc/init_on_free initialization is performed.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v3:
- reverted unrelated whitespace changes
Change-Id: I51103b7981d3aabed747d0c85cbdc85568665871
---
mm/slub.c | 34 +++++++++++++++++++++++++++++-----
1 file changed, 29 insertions(+), 5 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index b25c807a111f..b5d2be1ac755 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -21,6 +21,8 @@
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/kasan.h>
+#include <linux/kmsan.h>
+#include <linux/kmsan-checks.h> /* KMSAN_INIT_VALUE */
#include <linux/cpu.h>
#include <linux/cpuset.h>
#include <linux/mempolicy.h>
@@ -285,17 +287,27 @@ static void prefetch_freepointer(const struct kmem_cache *s, void *object)
prefetch(object + s->offset);
}
+/*
+ * When running under KMSAN, get_freepointer_safe() may return an uninitialized
+ * pointer value in the case the current thread loses the race for the next
+ * memory chunk in the freelist. In that case this_cpu_cmpxchg_double() in
+ * slab_alloc_node() will fail, so the uninitialized value won't be used, but
+ * KMSAN will still check all arguments of cmpxchg because of imperfect
+ * handling of inline assembly.
+ * To work around this problem, use KMSAN_INIT_VALUE() to force initialize the
+ * return value of get_freepointer_safe().
+ */
static inline void *get_freepointer_safe(struct kmem_cache *s, void *object)
{
unsigned long freepointer_addr;
void *p;
if (!debug_pagealloc_enabled())
- return get_freepointer(s, object);
+ return KMSAN_INIT_VALUE(get_freepointer(s, object));
freepointer_addr = (unsigned long)object + s->offset;
probe_kernel_read(&p, (void **)freepointer_addr, sizeof(p));
- return freelist_ptr(s, p, freepointer_addr);
+ return KMSAN_INIT_VALUE(freelist_ptr(s, p, freepointer_addr));
}
static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp)
@@ -1390,6 +1402,7 @@ static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags)
ptr = kasan_kmalloc_large(ptr, size, flags);
/* As ptr might get tagged, call kmemleak hook after KASAN. */
kmemleak_alloc(ptr, size, 1, flags);
+ kmsan_kmalloc_large(ptr, size, flags);
return ptr;
}
@@ -1397,6 +1410,7 @@ static __always_inline void kfree_hook(void *x)
{
kmemleak_free(x);
kasan_kfree_large(x, _RET_IP_);
+ kmsan_kfree_large(x);
}
static __always_inline bool slab_free_hook(struct kmem_cache *s, void *x)
@@ -1453,6 +1467,12 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s,
} while (object != old_tail);
}
+ do {
+ object = next;
+ next = get_freepointer(s, object);
+ kmsan_slab_free(s, object);
+ } while (object != old_tail);
+
/*
* Compiler cannot detect this function can be removed if slab_free_hook()
* evaluates to nothing. Thus, catch all relevant config debug options here.
@@ -2776,6 +2796,7 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s,
if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object)
memset(object, 0, s->object_size);
+ kmsan_slab_alloc(s, object, gfpflags);
slab_post_alloc_hook(s, gfpflags, 1, &object);
return object;
@@ -3157,7 +3178,7 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
void **p)
{
struct kmem_cache_cpu *c;
- int i;
+ int i, j;
/* memcg and kmem_cache debug support */
s = slab_pre_alloc_hook(s, flags);
@@ -3198,11 +3219,11 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
/* Clear memory outside IRQ disabled fastpath loop */
if (unlikely(slab_want_init_on_alloc(flags, s))) {
- int j;
-
for (j = 0; j < i; j++)
memset(p[j], 0, s->object_size);
}
+ for (j = 0; j < i; j++)
+ kmsan_slab_alloc(s, p[j], flags);
/* memcg and kmem_cache debug support */
slab_post_alloc_hook(s, flags, size, p);
@@ -3803,6 +3824,7 @@ static int __init setup_slub_min_objects(char *str)
__setup("slub_min_objects=", setup_slub_min_objects);
+__no_sanitize_memory
void *__kmalloc(size_t size, gfp_t flags)
{
struct kmem_cache *s;
@@ -5717,6 +5739,7 @@ static char *create_unique_id(struct kmem_cache *s)
p += sprintf(p, "%07u", s->size);
BUG_ON(p > name + ID_STR_LENGTH - 1);
+ kmsan_unpoison_shadow(name, p - name);
return name;
}
@@ -5866,6 +5889,7 @@ static int sysfs_slab_alias(struct kmem_cache *s, const char *name)
al->name = name;
al->next = alias_list;
alias_list = al;
+ kmsan_unpoison_shadow(al, sizeof(struct saved_alias));
return 0;
}
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 22/36] kmsan: mm: call KMSAN hooks from SLUB code
2019-11-22 11:26 ` [PATCH RFC v3 22/36] kmsan: mm: call KMSAN hooks from SLUB code glider
@ 2019-12-02 15:36 ` Marco Elver
2019-12-10 12:07 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-02 15:36 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Andrew Morton, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
>
> In order to report uninitialized memory coming from heap allocations
> KMSAN has to poison them unless they're created with __GFP_ZERO.
>
> It's handy that we need KMSAN hooks in the places where
> init_on_alloc/init_on_free initialization is performed.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
> v3:
> - reverted unrelated whitespace changes
>
> Change-Id: I51103b7981d3aabed747d0c85cbdc85568665871
> ---
> mm/slub.c | 34 +++++++++++++++++++++++++++++-----
> 1 file changed, 29 insertions(+), 5 deletions(-)
>
> diff --git a/mm/slub.c b/mm/slub.c
> index b25c807a111f..b5d2be1ac755 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -21,6 +21,8 @@
> #include <linux/proc_fs.h>
> #include <linux/seq_file.h>
> #include <linux/kasan.h>
> +#include <linux/kmsan.h>
> +#include <linux/kmsan-checks.h> /* KMSAN_INIT_VALUE */
> #include <linux/cpu.h>
> #include <linux/cpuset.h>
> #include <linux/mempolicy.h>
> @@ -285,17 +287,27 @@ static void prefetch_freepointer(const struct kmem_cache *s, void *object)
> prefetch(object + s->offset);
> }
>
> +/*
> + * When running under KMSAN, get_freepointer_safe() may return an uninitialized
> + * pointer value in the case the current thread loses the race for the next
> + * memory chunk in the freelist. In that case this_cpu_cmpxchg_double() in
> + * slab_alloc_node() will fail, so the uninitialized value won't be used, but
> + * KMSAN will still check all arguments of cmpxchg because of imperfect
> + * handling of inline assembly.
> + * To work around this problem, use KMSAN_INIT_VALUE() to force initialize the
> + * return value of get_freepointer_safe().
> + */
Isn't this a general problem with cmpxchg? I.e. does other code using
it have the same problem?
Would it be better to just use KMSAN_INIT_VALUE in cmpxchg, rather
than having the one-off workaround here?
Thanks,
-- Marco
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 22/36] kmsan: mm: call KMSAN hooks from SLUB code
2019-12-02 15:36 ` Marco Elver
@ 2019-12-10 12:07 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-10 12:07 UTC (permalink / raw)
To: Marco Elver
Cc: Andrew Morton, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 4:37 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:27, <glider@google.com> wrote:
> >
> > In order to report uninitialized memory coming from heap allocations
> > KMSAN has to poison them unless they're created with __GFP_ZERO.
> >
> > It's handy that we need KMSAN hooks in the places where
> > init_on_alloc/init_on_free initialization is performed.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> > v3:
> > - reverted unrelated whitespace changes
> >
> > Change-Id: I51103b7981d3aabed747d0c85cbdc85568665871
> > ---
> > mm/slub.c | 34 +++++++++++++++++++++++++++++-----
> > 1 file changed, 29 insertions(+), 5 deletions(-)
> >
> > diff --git a/mm/slub.c b/mm/slub.c
> > index b25c807a111f..b5d2be1ac755 100644
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -21,6 +21,8 @@
> > #include <linux/proc_fs.h>
> > #include <linux/seq_file.h>
> > #include <linux/kasan.h>
> > +#include <linux/kmsan.h>
> > +#include <linux/kmsan-checks.h> /* KMSAN_INIT_VALUE */
> > #include <linux/cpu.h>
> > #include <linux/cpuset.h>
> > #include <linux/mempolicy.h>
> > @@ -285,17 +287,27 @@ static void prefetch_freepointer(const struct kmem_cache *s, void *object)
> > prefetch(object + s->offset);
> > }
> >
> > +/*
> > + * When running under KMSAN, get_freepointer_safe() may return an uninitialized
> > + * pointer value in the case the current thread loses the race for the next
> > + * memory chunk in the freelist. In that case this_cpu_cmpxchg_double() in
> > + * slab_alloc_node() will fail, so the uninitialized value won't be used, but
> > + * KMSAN will still check all arguments of cmpxchg because of imperfect
> > + * handling of inline assembly.
> > + * To work around this problem, use KMSAN_INIT_VALUE() to force initialize the
> > + * return value of get_freepointer_safe().
> > + */
>
> Isn't this a general problem with cmpxchg? I.e. does other code using
> it have the same problem?
>
> Would it be better to just use KMSAN_INIT_VALUE in cmpxchg, rather
> than having the one-off workaround here?
I don't think so. It's normally still an error to pass an
uninitialized arg to cmpxchg, as in other cases it may be copied to
the result.
> Thanks,
> -- Marco
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (20 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 22/36] kmsan: mm: call KMSAN hooks from SLUB code glider
@ 2019-11-22 11:26 ` glider
2019-11-26 10:17 ` Petr Mladek
2019-11-29 16:21 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 24/36] kmsan: disable instrumentation of certain functions glider
` (13 subsequent siblings)
35 siblings, 2 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Andrew Morton, Greg Kroah-Hartman, Eric Dumazet, Wolfram Sang,
Petr Mladek, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, ericvh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, cai, rdunlap,
robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso, tglx,
gor
Insert KMSAN hooks that check for potential memory errors and/or make
necessary bookkeeping changes:
- allocate/split/deallocate metadata pages in
alloc_pages()/split_page()/free_page();
- clear page shadow and origins in clear_page(), copy_user_highpage();
- copy page metadata in copy_highpage(), wp_page_copy();
- handle vmap()/vunmap()/iounmap();
- handle task creation and deletion;
- initialize result of vscnprintf() in vprintk_store();
- call softirq entry/exit hooks in kernel/softirq.c;
- check/initialize memory sent to/read from USB, I2C, and network
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Wolfram Sang <wsa@the-dreams.de>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v2:
- dropped call to kmsan_handle_vprintk, updated comment in printk.c
v3:
- put KMSAN_INIT_VALUE on a separate line in vprintk_store()
- dropped call to kmsan_handle_i2c_transfer()
Change-Id: I1250a928d9263bf71fdaa067a070bdee686ef47b
---
arch/x86/include/asm/page_64.h | 13 +++++++++++++
arch/x86/mm/ioremap.c | 3 +++
drivers/usb/core/urb.c | 2 ++
include/linux/highmem.h | 4 ++++
kernel/exit.c | 2 ++
kernel/fork.c | 2 ++
kernel/kthread.c | 2 ++
kernel/printk/printk.c | 6 ++++++
kernel/softirq.c | 5 +++++
lib/ioremap.c | 5 +++++
mm/compaction.c | 9 +++++++++
mm/gup.c | 3 +++
mm/memory.c | 2 ++
mm/page_alloc.c | 16 ++++++++++++++++
mm/vmalloc.c | 23 +++++++++++++++++++++--
net/sched/sch_generic.c | 2 ++
16 files changed, 97 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h
index 939b1cff4a7b..0ba43d93414f 100644
--- a/arch/x86/include/asm/page_64.h
+++ b/arch/x86/include/asm/page_64.h
@@ -44,14 +44,27 @@ void clear_page_orig(void *page);
void clear_page_rep(void *page);
void clear_page_erms(void *page);
+/* This is an assembly header, avoid including too much of kmsan.h */
+#ifdef CONFIG_KMSAN
+void kmsan_clear_page(void *page_addr);
+#endif
+__no_sanitize_memory
static inline void clear_page(void *page)
{
+#ifdef CONFIG_KMSAN
+ /* alternative_call_2() changes |page|. */
+ void *page_copy = page;
+#endif
alternative_call_2(clear_page_orig,
clear_page_rep, X86_FEATURE_REP_GOOD,
clear_page_erms, X86_FEATURE_ERMS,
"=D" (page),
"0" (page)
: "cc", "memory", "rax", "rcx");
+#ifdef CONFIG_KMSAN
+ /* Clear KMSAN shadow for the pages that have it. */
+ kmsan_clear_page(page_copy);
+#endif
}
void copy_page(void *to, void *from);
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index a39dcdb5ae34..fdb2abc11a82 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -7,6 +7,7 @@
* (C) Copyright 1995 1996 Linus Torvalds
*/
+#include <linux/kmsan.h>
#include <linux/memblock.h>
#include <linux/init.h>
#include <linux/io.h>
@@ -451,6 +452,8 @@ void iounmap(volatile void __iomem *addr)
return;
}
+ kmsan_iounmap_page_range((unsigned long)addr,
+ (unsigned long)addr + get_vm_area_size(p));
free_memtype(p->phys_addr, p->phys_addr + get_vm_area_size(p));
/* Finally remove it */
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index 0eab79f82ce4..5bdb54d71c2e 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -8,6 +8,7 @@
#include <linux/bitops.h>
#include <linux/slab.h>
#include <linux/log2.h>
+#include <linux/kmsan-checks.h>
#include <linux/usb.h>
#include <linux/wait.h>
#include <linux/usb/hcd.h>
@@ -401,6 +402,7 @@ int usb_submit_urb(struct urb *urb, gfp_t mem_flags)
URB_SETUP_MAP_SINGLE | URB_SETUP_MAP_LOCAL |
URB_DMA_SG_COMBINED);
urb->transfer_flags |= (is_out ? URB_DIR_OUT : URB_DIR_IN);
+ kmsan_handle_urb(urb, is_out);
if (xfertype != USB_ENDPOINT_XFER_CONTROL &&
dev->state < USB_STATE_CONFIGURED)
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index ea5cdbd8c2c3..623b56f48685 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -5,6 +5,7 @@
#include <linux/fs.h>
#include <linux/kernel.h>
#include <linux/bug.h>
+#include <linux/kmsan.h>
#include <linux/mm.h>
#include <linux/uaccess.h>
#include <linux/hardirq.h>
@@ -255,6 +256,8 @@ static inline void copy_user_highpage(struct page *to, struct page *from,
vfrom = kmap_atomic(from);
vto = kmap_atomic(to);
copy_user_page(vto, vfrom, vaddr, to);
+ /* User pages don't have shadow, just clear the destination. */
+ kmsan_clear_page(page_address(to));
kunmap_atomic(vto);
kunmap_atomic(vfrom);
}
@@ -270,6 +273,7 @@ static inline void copy_highpage(struct page *to, struct page *from)
vfrom = kmap_atomic(from);
vto = kmap_atomic(to);
copy_page(vto, vfrom);
+ kmsan_copy_page_meta(to, from);
kunmap_atomic(vto);
kunmap_atomic(vfrom);
}
diff --git a/kernel/exit.c b/kernel/exit.c
index a46a50d67002..9e3ce929110b 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -60,6 +60,7 @@
#include <linux/writeback.h>
#include <linux/shm.h>
#include <linux/kcov.h>
+#include <linux/kmsan.h>
#include <linux/random.h>
#include <linux/rcuwait.h>
#include <linux/compat.h>
@@ -719,6 +720,7 @@ void __noreturn do_exit(long code)
profile_task_exit(tsk);
kcov_task_exit(tsk);
+ kmsan_task_exit(tsk);
WARN_ON(blk_needs_flush_plug(tsk));
diff --git a/kernel/fork.c b/kernel/fork.c
index bcdf53125210..0f08952a42dc 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -37,6 +37,7 @@
#include <linux/fdtable.h>
#include <linux/iocontext.h>
#include <linux/key.h>
+#include <linux/kmsan.h>
#include <linux/binfmts.h>
#include <linux/mman.h>
#include <linux/mmu_notifier.h>
@@ -931,6 +932,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
account_kernel_stack(tsk, 1);
kcov_task_init(tsk);
+ kmsan_task_create(tsk);
#ifdef CONFIG_FAULT_INJECTION
tsk->fail_nth = 0;
diff --git a/kernel/kthread.c b/kernel/kthread.c
index b262f47046ca..33ca743ca8b5 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -17,6 +17,7 @@
#include <linux/unistd.h>
#include <linux/file.h>
#include <linux/export.h>
+#include <linux/kmsan.h>
#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/freezer.h>
@@ -350,6 +351,7 @@ struct task_struct *__kthread_create_on_node(int (*threadfn)(void *data),
set_cpus_allowed_ptr(task, cpu_all_mask);
}
kfree(create);
+ kmsan_task_create(task);
return task;
}
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index ca65327a6de8..c9ef7fb0906f 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1915,6 +1915,12 @@ int vprintk_store(int facility, int level,
* prefix which might be passed-in as a parameter.
*/
text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
+ /*
+ * If any of vscnprintf() arguments is uninitialized, KMSAN will report
+ * one or more errors and also probably mark text_len as uninitialized.
+ * Initialize |text_len| to prevent the errors from spreading further.
+ */
+ text_len = KMSAN_INIT_VALUE(text_len);
/* mark and strip a trailing newline */
if (text_len && text[text_len-1] == '\n') {
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 0427a86743a4..6d566dd68b35 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -11,6 +11,7 @@
#include <linux/export.h>
#include <linux/kernel_stat.h>
+#include <linux/kmsan.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/mm.h>
@@ -370,7 +371,9 @@ static inline void invoke_softirq(void)
* it is the irq stack, because it should be near empty
* at this stage.
*/
+ kmsan_softirq_enter();
__do_softirq();
+ kmsan_softirq_exit();
#else
/*
* Otherwise, irq_exit() is called on the task stack that can
@@ -600,7 +603,9 @@ static void run_ksoftirqd(unsigned int cpu)
* We can safely run softirq on inline stack, as we are not deep
* in the task stack here.
*/
+ kmsan_softirq_enter();
__do_softirq();
+ kmsan_softirq_exit();
local_irq_enable();
cond_resched();
return;
diff --git a/lib/ioremap.c b/lib/ioremap.c
index 0a2ffadc6d71..5f830cee5bfc 100644
--- a/lib/ioremap.c
+++ b/lib/ioremap.c
@@ -6,6 +6,7 @@
*
* (C) Copyright 1995 1996 Linus Torvalds
*/
+#include <linux/kmsan.h>
#include <linux/vmalloc.h>
#include <linux/mm.h>
#include <linux/sched.h>
@@ -214,6 +215,8 @@ int ioremap_page_range(unsigned long addr,
unsigned long start;
unsigned long next;
int err;
+ unsigned long old_addr = addr;
+ phys_addr_t old_phys_addr = phys_addr;
might_sleep();
BUG_ON(addr >= end);
@@ -228,6 +231,8 @@ int ioremap_page_range(unsigned long addr,
} while (pgd++, phys_addr += (next - addr), addr = next, addr != end);
flush_cache_vmap(start, end);
+ if (!err)
+ kmsan_ioremap_page_range(old_addr, end, old_phys_addr, prot);
return err;
}
diff --git a/mm/compaction.c b/mm/compaction.c
index 672d3c78c6ab..720a8a4dafec 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -84,6 +84,15 @@ static void split_map_pages(struct list_head *list)
for (i = 0; i < nr_pages; i++) {
list_add(&page->lru, &tmp_list);
+#ifdef CONFIG_KMSAN
+ /*
+ * TODO(glider): we may lose the metadata when copying
+ * something to these pages. Need to allocate shadow
+ * and origin pages here instead.
+ */
+ page->shadow = NULL;
+ page->origin = NULL;
+#endif
page++;
}
}
diff --git a/mm/gup.c b/mm/gup.c
index 8f236a335ae9..8f5f99772278 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -4,6 +4,7 @@
#include <linux/err.h>
#include <linux/spinlock.h>
+#include <linux/kmsan.h>
#include <linux/mm.h>
#include <linux/memremap.h>
#include <linux/pagemap.h>
@@ -2349,6 +2350,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
gup_fast_permitted(start, end)) {
local_irq_save(flags);
gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
+ kmsan_gup_pgd_range(pages, nr);
local_irq_restore(flags);
}
@@ -2418,6 +2420,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages,
gup_fast_permitted(start, end)) {
local_irq_disable();
gup_pgd_range(addr, end, gup_flags, pages, &nr);
+ kmsan_gup_pgd_range(pages, nr);
local_irq_enable();
ret = nr;
}
diff --git a/mm/memory.c b/mm/memory.c
index b1ca51a079f2..48ceacc06e2d 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -51,6 +51,7 @@
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/memremap.h>
+#include <linux/kmsan.h>
#include <linux/ksm.h>
#include <linux/rmap.h>
#include <linux/export.h>
@@ -2328,6 +2329,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
if (!new_page)
goto oom;
cow_user_page(new_page, old_page, vmf->address, vma);
+ kmsan_copy_page_meta(new_page, old_page);
}
if (mem_cgroup_try_charge_delay(new_page, mm, GFP_KERNEL, &memcg, false))
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index ecc3dbad606b..c98e4441c7c0 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -26,6 +26,8 @@
#include <linux/compiler.h>
#include <linux/kernel.h>
#include <linux/kasan.h>
+#include <linux/kmsan.h>
+#include <linux/kmsan-checks.h>
#include <linux/module.h>
#include <linux/suspend.h>
#include <linux/pagevec.h>
@@ -1133,6 +1135,7 @@ static __always_inline bool free_pages_prepare(struct page *page,
VM_BUG_ON_PAGE(PageTail(page), page);
trace_mm_page_free(page, order);
+ kmsan_free_page(page, order);
/*
* Check tail pages before head page information is cleared to
@@ -3121,6 +3124,7 @@ void split_page(struct page *page, unsigned int order)
VM_BUG_ON_PAGE(PageCompound(page), page);
VM_BUG_ON_PAGE(!page_count(page), page);
+ kmsan_split_page(page, order);
for (i = 1; i < (1 << order); i++)
set_page_refcounted(page + i);
split_page_owner(page, order);
@@ -3253,6 +3257,13 @@ static struct page *rmqueue_pcplist(struct zone *preferred_zone,
/*
* Allocate a page from the given zone. Use pcplists for order-0 allocations.
*/
+/*
+ * TODO(glider): rmqueue() may call __msan_poison_alloca() through a call to
+ * set_pfnblock_flags_mask(). If __msan_poison_alloca() attempts to allocate
+ * pages for the stack depot, it may call rmqueue() again, which will result
+ * in a deadlock.
+ */
+__no_sanitize_memory
static inline
struct page *rmqueue(struct zone *preferred_zone,
struct zone *zone, unsigned int order,
@@ -4781,6 +4792,11 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
trace_mm_page_alloc(page, order, alloc_mask, ac.migratetype);
+ if (page)
+ if (kmsan_alloc_page(page, order, gfp_mask)) {
+ __free_pages(page, order);
+ page = NULL;
+ }
return page;
}
EXPORT_SYMBOL(__alloc_pages_nodemask);
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index a3c70e275f4e..bdf66ffcf02c 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -29,6 +29,7 @@
#include <linux/rcupdate.h>
#include <linux/pfn.h>
#include <linux/kmemleak.h>
+#include <linux/kmsan.h>
#include <linux/atomic.h>
#include <linux/compiler.h>
#include <linux/llist.h>
@@ -119,7 +120,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end)
} while (p4d++, addr = next, addr != end);
}
-static void vunmap_page_range(unsigned long addr, unsigned long end)
+/* Exported for KMSAN, visible in mm/kmsan/kmsan.h only. */
+void __vunmap_page_range(unsigned long addr, unsigned long end)
{
pgd_t *pgd;
unsigned long next;
@@ -133,6 +135,12 @@ static void vunmap_page_range(unsigned long addr, unsigned long end)
vunmap_p4d_range(pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
+EXPORT_SYMBOL(__vunmap_page_range);
+static void vunmap_page_range(unsigned long addr, unsigned long end)
+{
+ kmsan_vunmap_page_range(addr, end);
+ __vunmap_page_range(addr, end);
+}
static int vmap_pte_range(pmd_t *pmd, unsigned long addr,
unsigned long end, pgprot_t prot, struct page **pages, int *nr)
@@ -216,8 +224,11 @@ static int vmap_p4d_range(pgd_t *pgd, unsigned long addr,
* will have pfns corresponding to the "pages" array.
*
* Ie. pte at addr+N*PAGE_SIZE shall point to pfn corresponding to pages[N]
+ *
+ * This function is exported for use in KMSAN, but is only declared in KMSAN
+ * headers.
*/
-static int vmap_page_range_noflush(unsigned long start, unsigned long end,
+int __vmap_page_range_noflush(unsigned long start, unsigned long end,
pgprot_t prot, struct page **pages)
{
pgd_t *pgd;
@@ -237,6 +248,14 @@ static int vmap_page_range_noflush(unsigned long start, unsigned long end,
return nr;
}
+EXPORT_SYMBOL(__vmap_page_range_noflush);
+
+static int vmap_page_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages)
+{
+ kmsan_vmap_page_range_noflush(start, end, prot, pages);
+ return __vmap_page_range_noflush(start, end, prot, pages);
+}
static int vmap_page_range(unsigned long start, unsigned long end,
pgprot_t prot, struct page **pages)
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 17bd8f539bc7..fd22c4a4ba42 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -11,6 +11,7 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/kmsan-checks.h>
#include <linux/sched.h>
#include <linux/string.h>
#include <linux/errno.h>
@@ -659,6 +660,7 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc)
} else {
qdisc->empty = true;
}
+ kmsan_check_skb(skb);
return skb;
}
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed
2019-11-22 11:26 ` [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed glider
@ 2019-11-26 10:17 ` Petr Mladek
2019-11-26 10:52 ` Alexander Potapenko
2019-11-29 16:21 ` Andrey Konovalov
1 sibling, 1 reply; 120+ messages in thread
From: Petr Mladek @ 2019-11-26 10:17 UTC (permalink / raw)
To: glider
Cc: Andrew Morton, Greg Kroah-Hartman, Eric Dumazet, Wolfram Sang,
Vegard Nossum, Dmitry Vyukov, linux-mm, viro, adilger.kernel,
andreyknvl, aryabinin, luto, ard.biesheuvel, arnd, hch, hch,
darrick.wong, davem, dmitry.torokhov, ebiggers, ericvh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor
On Fri 2019-11-22 12:26:08, glider@google.com wrote:
> Insert KMSAN hooks that check for potential memory errors and/or make
> necessary bookkeeping changes:
> - allocate/split/deallocate metadata pages in
> alloc_pages()/split_page()/free_page();
> - clear page shadow and origins in clear_page(), copy_user_highpage();
> - copy page metadata in copy_highpage(), wp_page_copy();
> - handle vmap()/vunmap()/iounmap();
> - handle task creation and deletion;
> - initialize result of vscnprintf() in vprintk_store();
> - call softirq entry/exit hooks in kernel/softirq.c;
> - check/initialize memory sent to/read from USB, I2C, and network
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> index ca65327a6de8..c9ef7fb0906f 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -1915,6 +1915,12 @@ int vprintk_store(int facility, int level,
> * prefix which might be passed-in as a parameter.
> */
> text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
> + /*
> + * If any of vscnprintf() arguments is uninitialized, KMSAN will report
> + * one or more errors and also probably mark text_len as uninitialized.
> + * Initialize |text_len| to prevent the errors from spreading further.
> + */
> + text_len = KMSAN_INIT_VALUE(text_len);
>
> /* mark and strip a trailing newline */
> if (text_len && text[text_len-1] == '\n') {
Acked-by: Petr Mladek <pmladek@suse.com> # for printk.c
Best Regards,
Petr
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed
2019-11-26 10:17 ` Petr Mladek
@ 2019-11-26 10:52 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-11-26 10:52 UTC (permalink / raw)
To: Petr Mladek
Cc: Andrew Morton, Greg Kroah-Hartman, Eric Dumazet, Wolfram Sang,
Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann, hch,
Christoph Hellwig, darrick.wong, David Miller, Dmitry Torokhov,
Eric Biggers, Eric Van Hensbergen, Harry Wentland, Herbert Xu,
Ilya Leoshkevich, Ingo Molnar, jasowang, Jens Axboe,
Marek Szyprowski, Marco Elver, Mark Rutland, Martin K . Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik
On Tue, Nov 26, 2019 at 11:17 AM Petr Mladek <pmladek@suse.com> wrote:
>
> On Fri 2019-11-22 12:26:08, glider@google.com wrote:
> > Insert KMSAN hooks that check for potential memory errors and/or make
> > necessary bookkeeping changes:
> > - allocate/split/deallocate metadata pages in
> > alloc_pages()/split_page()/free_page();
> > - clear page shadow and origins in clear_page(), copy_user_highpage();
> > - copy page metadata in copy_highpage(), wp_page_copy();
> > - handle vmap()/vunmap()/iounmap();
> > - handle task creation and deletion;
> > - initialize result of vscnprintf() in vprintk_store();
> > - call softirq entry/exit hooks in kernel/softirq.c;
> > - check/initialize memory sent to/read from USB, I2C, and network
I forgot to remove I2C from patch description, will do in the next patch series.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
>
> > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> > index ca65327a6de8..c9ef7fb0906f 100644
> > --- a/kernel/printk/printk.c
> > +++ b/kernel/printk/printk.c
> > @@ -1915,6 +1915,12 @@ int vprintk_store(int facility, int level,
> > * prefix which might be passed-in as a parameter.
> > */
> > text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
> > + /*
> > + * If any of vscnprintf() arguments is uninitialized, KMSAN will report
> > + * one or more errors and also probably mark text_len as uninitialized.
> > + * Initialize |text_len| to prevent the errors from spreading further.
> > + */
> > + text_len = KMSAN_INIT_VALUE(text_len);
> >
> > /* mark and strip a trailing newline */
> > if (text_len && text[text_len-1] == '\n') {
>
> Acked-by: Petr Mladek <pmladek@suse.com> # for printk.c
>
> Best Regards,
> Petr
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed
2019-11-22 11:26 ` [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed glider
2019-11-26 10:17 ` Petr Mladek
@ 2019-11-29 16:21 ` Andrey Konovalov
2019-12-16 11:30 ` Alexander Potapenko
1 sibling, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 16:21 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Andrew Morton, Greg Kroah-Hartman, Eric Dumazet, Wolfram Sang,
Petr Mladek, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, ericvh,
harry.wentland, Herbert Xu, iii, mingo, Jason Wang, Jens Axboe,
Marek Szyprowski, Marco Elver, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S . Tsirkin,
Michal Simek, Qian Cai, Randy Dunlap, Robin Murphy,
sergey.senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, gor
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> Insert KMSAN hooks that check for potential memory errors and/or make
> necessary bookkeeping changes:
I think it makes sense to split this patch into 2+ parts, the first
one adds hooks for internal KMSAN bookkeeping:
> - allocate/split/deallocate metadata pages in
> alloc_pages()/split_page()/free_page();
> - clear page shadow and origins in clear_page(), copy_user_highpage();
> - copy page metadata in copy_highpage(), wp_page_copy();
> - handle vmap()/vunmap()/iounmap();
> - handle task creation and deletion;
> - call softirq entry/exit hooks in kernel/softirq.c;
And the other ones do other things:
> - initialize result of vscnprintf() in vprintk_store();
> - check/initialize memory sent to/read from USB, I2C, and network
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Eric Dumazet <edumazet@google.com>
> Cc: Wolfram Sang <wsa@the-dreams.de>
> Cc: Petr Mladek <pmladek@suse.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
>
> v2:
> - dropped call to kmsan_handle_vprintk, updated comment in printk.c
>
> v3:
> - put KMSAN_INIT_VALUE on a separate line in vprintk_store()
> - dropped call to kmsan_handle_i2c_transfer()
>
> Change-Id: I1250a928d9263bf71fdaa067a070bdee686ef47b
> ---
> arch/x86/include/asm/page_64.h | 13 +++++++++++++
> arch/x86/mm/ioremap.c | 3 +++
> drivers/usb/core/urb.c | 2 ++
> include/linux/highmem.h | 4 ++++
> kernel/exit.c | 2 ++
> kernel/fork.c | 2 ++
> kernel/kthread.c | 2 ++
> kernel/printk/printk.c | 6 ++++++
> kernel/softirq.c | 5 +++++
> lib/ioremap.c | 5 +++++
> mm/compaction.c | 9 +++++++++
> mm/gup.c | 3 +++
> mm/memory.c | 2 ++
> mm/page_alloc.c | 16 ++++++++++++++++
> mm/vmalloc.c | 23 +++++++++++++++++++++--
> net/sched/sch_generic.c | 2 ++
> 16 files changed, 97 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h
> index 939b1cff4a7b..0ba43d93414f 100644
> --- a/arch/x86/include/asm/page_64.h
> +++ b/arch/x86/include/asm/page_64.h
> @@ -44,14 +44,27 @@ void clear_page_orig(void *page);
> void clear_page_rep(void *page);
> void clear_page_erms(void *page);
>
> +/* This is an assembly header, avoid including too much of kmsan.h */
> +#ifdef CONFIG_KMSAN
> +void kmsan_clear_page(void *page_addr);
> +#endif
> +__no_sanitize_memory
> static inline void clear_page(void *page)
> {
> +#ifdef CONFIG_KMSAN
> + /* alternative_call_2() changes |page|. */
> + void *page_copy = page;
> +#endif
> alternative_call_2(clear_page_orig,
> clear_page_rep, X86_FEATURE_REP_GOOD,
> clear_page_erms, X86_FEATURE_ERMS,
> "=D" (page),
> "0" (page)
> : "cc", "memory", "rax", "rcx");
> +#ifdef CONFIG_KMSAN
> + /* Clear KMSAN shadow for the pages that have it. */
> + kmsan_clear_page(page_copy);
> +#endif
> }
>
> void copy_page(void *to, void *from);
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index a39dcdb5ae34..fdb2abc11a82 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -7,6 +7,7 @@
> * (C) Copyright 1995 1996 Linus Torvalds
> */
>
> +#include <linux/kmsan.h>
> #include <linux/memblock.h>
> #include <linux/init.h>
> #include <linux/io.h>
> @@ -451,6 +452,8 @@ void iounmap(volatile void __iomem *addr)
> return;
> }
>
> + kmsan_iounmap_page_range((unsigned long)addr,
> + (unsigned long)addr + get_vm_area_size(p));
> free_memtype(p->phys_addr, p->phys_addr + get_vm_area_size(p));
>
> /* Finally remove it */
> diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
> index 0eab79f82ce4..5bdb54d71c2e 100644
> --- a/drivers/usb/core/urb.c
> +++ b/drivers/usb/core/urb.c
> @@ -8,6 +8,7 @@
> #include <linux/bitops.h>
> #include <linux/slab.h>
> #include <linux/log2.h>
> +#include <linux/kmsan-checks.h>
> #include <linux/usb.h>
> #include <linux/wait.h>
> #include <linux/usb/hcd.h>
> @@ -401,6 +402,7 @@ int usb_submit_urb(struct urb *urb, gfp_t mem_flags)
> URB_SETUP_MAP_SINGLE | URB_SETUP_MAP_LOCAL |
> URB_DMA_SG_COMBINED);
> urb->transfer_flags |= (is_out ? URB_DIR_OUT : URB_DIR_IN);
> + kmsan_handle_urb(urb, is_out);
>
> if (xfertype != USB_ENDPOINT_XFER_CONTROL &&
> dev->state < USB_STATE_CONFIGURED)
> diff --git a/include/linux/highmem.h b/include/linux/highmem.h
> index ea5cdbd8c2c3..623b56f48685 100644
> --- a/include/linux/highmem.h
> +++ b/include/linux/highmem.h
> @@ -5,6 +5,7 @@
> #include <linux/fs.h>
> #include <linux/kernel.h>
> #include <linux/bug.h>
> +#include <linux/kmsan.h>
> #include <linux/mm.h>
> #include <linux/uaccess.h>
> #include <linux/hardirq.h>
> @@ -255,6 +256,8 @@ static inline void copy_user_highpage(struct page *to, struct page *from,
> vfrom = kmap_atomic(from);
> vto = kmap_atomic(to);
> copy_user_page(vto, vfrom, vaddr, to);
> + /* User pages don't have shadow, just clear the destination. */
> + kmsan_clear_page(page_address(to));
> kunmap_atomic(vto);
> kunmap_atomic(vfrom);
> }
> @@ -270,6 +273,7 @@ static inline void copy_highpage(struct page *to, struct page *from)
> vfrom = kmap_atomic(from);
> vto = kmap_atomic(to);
> copy_page(vto, vfrom);
> + kmsan_copy_page_meta(to, from);
> kunmap_atomic(vto);
> kunmap_atomic(vfrom);
> }
> diff --git a/kernel/exit.c b/kernel/exit.c
> index a46a50d67002..9e3ce929110b 100644
> --- a/kernel/exit.c
> +++ b/kernel/exit.c
> @@ -60,6 +60,7 @@
> #include <linux/writeback.h>
> #include <linux/shm.h>
> #include <linux/kcov.h>
> +#include <linux/kmsan.h>
> #include <linux/random.h>
> #include <linux/rcuwait.h>
> #include <linux/compat.h>
> @@ -719,6 +720,7 @@ void __noreturn do_exit(long code)
>
> profile_task_exit(tsk);
> kcov_task_exit(tsk);
> + kmsan_task_exit(tsk);
>
> WARN_ON(blk_needs_flush_plug(tsk));
>
> diff --git a/kernel/fork.c b/kernel/fork.c
> index bcdf53125210..0f08952a42dc 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -37,6 +37,7 @@
> #include <linux/fdtable.h>
> #include <linux/iocontext.h>
> #include <linux/key.h>
> +#include <linux/kmsan.h>
> #include <linux/binfmts.h>
> #include <linux/mman.h>
> #include <linux/mmu_notifier.h>
> @@ -931,6 +932,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
> account_kernel_stack(tsk, 1);
>
> kcov_task_init(tsk);
> + kmsan_task_create(tsk);
>
> #ifdef CONFIG_FAULT_INJECTION
> tsk->fail_nth = 0;
> diff --git a/kernel/kthread.c b/kernel/kthread.c
> index b262f47046ca..33ca743ca8b5 100644
> --- a/kernel/kthread.c
> +++ b/kernel/kthread.c
> @@ -17,6 +17,7 @@
> #include <linux/unistd.h>
> #include <linux/file.h>
> #include <linux/export.h>
> +#include <linux/kmsan.h>
> #include <linux/mutex.h>
> #include <linux/slab.h>
> #include <linux/freezer.h>
> @@ -350,6 +351,7 @@ struct task_struct *__kthread_create_on_node(int (*threadfn)(void *data),
> set_cpus_allowed_ptr(task, cpu_all_mask);
> }
> kfree(create);
> + kmsan_task_create(task);
> return task;
> }
>
> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> index ca65327a6de8..c9ef7fb0906f 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -1915,6 +1915,12 @@ int vprintk_store(int facility, int level,
> * prefix which might be passed-in as a parameter.
> */
> text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
> + /*
> + * If any of vscnprintf() arguments is uninitialized, KMSAN will report
> + * one or more errors and also probably mark text_len as uninitialized.
> + * Initialize |text_len| to prevent the errors from spreading further.
> + */
> + text_len = KMSAN_INIT_VALUE(text_len);
>
> /* mark and strip a trailing newline */
> if (text_len && text[text_len-1] == '\n') {
> diff --git a/kernel/softirq.c b/kernel/softirq.c
> index 0427a86743a4..6d566dd68b35 100644
> --- a/kernel/softirq.c
> +++ b/kernel/softirq.c
> @@ -11,6 +11,7 @@
>
> #include <linux/export.h>
> #include <linux/kernel_stat.h>
> +#include <linux/kmsan.h>
> #include <linux/interrupt.h>
> #include <linux/init.h>
> #include <linux/mm.h>
> @@ -370,7 +371,9 @@ static inline void invoke_softirq(void)
> * it is the irq stack, because it should be near empty
> * at this stage.
> */
> + kmsan_softirq_enter();
> __do_softirq();
> + kmsan_softirq_exit();
> #else
> /*
> * Otherwise, irq_exit() is called on the task stack that can
> @@ -600,7 +603,9 @@ static void run_ksoftirqd(unsigned int cpu)
> * We can safely run softirq on inline stack, as we are not deep
> * in the task stack here.
> */
> + kmsan_softirq_enter();
> __do_softirq();
> + kmsan_softirq_exit();
> local_irq_enable();
> cond_resched();
> return;
> diff --git a/lib/ioremap.c b/lib/ioremap.c
> index 0a2ffadc6d71..5f830cee5bfc 100644
> --- a/lib/ioremap.c
> +++ b/lib/ioremap.c
> @@ -6,6 +6,7 @@
> *
> * (C) Copyright 1995 1996 Linus Torvalds
> */
> +#include <linux/kmsan.h>
> #include <linux/vmalloc.h>
> #include <linux/mm.h>
> #include <linux/sched.h>
> @@ -214,6 +215,8 @@ int ioremap_page_range(unsigned long addr,
> unsigned long start;
> unsigned long next;
> int err;
> + unsigned long old_addr = addr;
> + phys_addr_t old_phys_addr = phys_addr;
>
> might_sleep();
> BUG_ON(addr >= end);
> @@ -228,6 +231,8 @@ int ioremap_page_range(unsigned long addr,
> } while (pgd++, phys_addr += (next - addr), addr = next, addr != end);
>
> flush_cache_vmap(start, end);
> + if (!err)
> + kmsan_ioremap_page_range(old_addr, end, old_phys_addr, prot);
>
> return err;
> }
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 672d3c78c6ab..720a8a4dafec 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -84,6 +84,15 @@ static void split_map_pages(struct list_head *list)
>
> for (i = 0; i < nr_pages; i++) {
> list_add(&page->lru, &tmp_list);
> +#ifdef CONFIG_KMSAN
> + /*
> + * TODO(glider): we may lose the metadata when copying
> + * something to these pages. Need to allocate shadow
> + * and origin pages here instead.
> + */
> + page->shadow = NULL;
> + page->origin = NULL;
> +#endif
> page++;
> }
> }
> diff --git a/mm/gup.c b/mm/gup.c
> index 8f236a335ae9..8f5f99772278 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -4,6 +4,7 @@
> #include <linux/err.h>
> #include <linux/spinlock.h>
>
> +#include <linux/kmsan.h>
> #include <linux/mm.h>
> #include <linux/memremap.h>
> #include <linux/pagemap.h>
> @@ -2349,6 +2350,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
> gup_fast_permitted(start, end)) {
> local_irq_save(flags);
> gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
> + kmsan_gup_pgd_range(pages, nr);
> local_irq_restore(flags);
> }
>
> @@ -2418,6 +2420,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages,
> gup_fast_permitted(start, end)) {
> local_irq_disable();
> gup_pgd_range(addr, end, gup_flags, pages, &nr);
> + kmsan_gup_pgd_range(pages, nr);
> local_irq_enable();
> ret = nr;
> }
> diff --git a/mm/memory.c b/mm/memory.c
> index b1ca51a079f2..48ceacc06e2d 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -51,6 +51,7 @@
> #include <linux/highmem.h>
> #include <linux/pagemap.h>
> #include <linux/memremap.h>
> +#include <linux/kmsan.h>
> #include <linux/ksm.h>
> #include <linux/rmap.h>
> #include <linux/export.h>
> @@ -2328,6 +2329,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
> if (!new_page)
> goto oom;
> cow_user_page(new_page, old_page, vmf->address, vma);
> + kmsan_copy_page_meta(new_page, old_page);
> }
>
> if (mem_cgroup_try_charge_delay(new_page, mm, GFP_KERNEL, &memcg, false))
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index ecc3dbad606b..c98e4441c7c0 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -26,6 +26,8 @@
> #include <linux/compiler.h>
> #include <linux/kernel.h>
> #include <linux/kasan.h>
> +#include <linux/kmsan.h>
> +#include <linux/kmsan-checks.h>
> #include <linux/module.h>
> #include <linux/suspend.h>
> #include <linux/pagevec.h>
> @@ -1133,6 +1135,7 @@ static __always_inline bool free_pages_prepare(struct page *page,
> VM_BUG_ON_PAGE(PageTail(page), page);
>
> trace_mm_page_free(page, order);
> + kmsan_free_page(page, order);
>
> /*
> * Check tail pages before head page information is cleared to
> @@ -3121,6 +3124,7 @@ void split_page(struct page *page, unsigned int order)
> VM_BUG_ON_PAGE(PageCompound(page), page);
> VM_BUG_ON_PAGE(!page_count(page), page);
>
> + kmsan_split_page(page, order);
> for (i = 1; i < (1 << order); i++)
> set_page_refcounted(page + i);
> split_page_owner(page, order);
> @@ -3253,6 +3257,13 @@ static struct page *rmqueue_pcplist(struct zone *preferred_zone,
> /*
> * Allocate a page from the given zone. Use pcplists for order-0 allocations.
> */
> +/*
> + * TODO(glider): rmqueue() may call __msan_poison_alloca() through a call to
> + * set_pfnblock_flags_mask(). If __msan_poison_alloca() attempts to allocate
> + * pages for the stack depot, it may call rmqueue() again, which will result
> + * in a deadlock.
> + */
> +__no_sanitize_memory
> static inline
> struct page *rmqueue(struct zone *preferred_zone,
> struct zone *zone, unsigned int order,
> @@ -4781,6 +4792,11 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
>
> trace_mm_page_alloc(page, order, alloc_mask, ac.migratetype);
>
> + if (page)
> + if (kmsan_alloc_page(page, order, gfp_mask)) {
> + __free_pages(page, order);
> + page = NULL;
> + }
> return page;
> }
> EXPORT_SYMBOL(__alloc_pages_nodemask);
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index a3c70e275f4e..bdf66ffcf02c 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -29,6 +29,7 @@
> #include <linux/rcupdate.h>
> #include <linux/pfn.h>
> #include <linux/kmemleak.h>
> +#include <linux/kmsan.h>
> #include <linux/atomic.h>
> #include <linux/compiler.h>
> #include <linux/llist.h>
> @@ -119,7 +120,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end)
> } while (p4d++, addr = next, addr != end);
> }
>
> -static void vunmap_page_range(unsigned long addr, unsigned long end)
> +/* Exported for KMSAN, visible in mm/kmsan/kmsan.h only. */
> +void __vunmap_page_range(unsigned long addr, unsigned long end)
> {
> pgd_t *pgd;
> unsigned long next;
> @@ -133,6 +135,12 @@ static void vunmap_page_range(unsigned long addr, unsigned long end)
> vunmap_p4d_range(pgd, addr, next);
> } while (pgd++, addr = next, addr != end);
> }
> +EXPORT_SYMBOL(__vunmap_page_range);
> +static void vunmap_page_range(unsigned long addr, unsigned long end)
> +{
> + kmsan_vunmap_page_range(addr, end);
> + __vunmap_page_range(addr, end);
> +}
>
> static int vmap_pte_range(pmd_t *pmd, unsigned long addr,
> unsigned long end, pgprot_t prot, struct page **pages, int *nr)
> @@ -216,8 +224,11 @@ static int vmap_p4d_range(pgd_t *pgd, unsigned long addr,
> * will have pfns corresponding to the "pages" array.
> *
> * Ie. pte at addr+N*PAGE_SIZE shall point to pfn corresponding to pages[N]
> + *
> + * This function is exported for use in KMSAN, but is only declared in KMSAN
> + * headers.
> */
> -static int vmap_page_range_noflush(unsigned long start, unsigned long end,
> +int __vmap_page_range_noflush(unsigned long start, unsigned long end,
> pgprot_t prot, struct page **pages)
> {
> pgd_t *pgd;
> @@ -237,6 +248,14 @@ static int vmap_page_range_noflush(unsigned long start, unsigned long end,
>
> return nr;
> }
> +EXPORT_SYMBOL(__vmap_page_range_noflush);
> +
> +static int vmap_page_range_noflush(unsigned long start, unsigned long end,
> + pgprot_t prot, struct page **pages)
> +{
> + kmsan_vmap_page_range_noflush(start, end, prot, pages);
> + return __vmap_page_range_noflush(start, end, prot, pages);
> +}
>
> static int vmap_page_range(unsigned long start, unsigned long end,
> pgprot_t prot, struct page **pages)
> diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
> index 17bd8f539bc7..fd22c4a4ba42 100644
> --- a/net/sched/sch_generic.c
> +++ b/net/sched/sch_generic.c
> @@ -11,6 +11,7 @@
> #include <linux/module.h>
> #include <linux/types.h>
> #include <linux/kernel.h>
> +#include <linux/kmsan-checks.h>
> #include <linux/sched.h>
> #include <linux/string.h>
> #include <linux/errno.h>
> @@ -659,6 +660,7 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc)
> } else {
> qdisc->empty = true;
> }
> + kmsan_check_skb(skb);
>
> return skb;
> }
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed
2019-11-29 16:21 ` Andrey Konovalov
@ 2019-12-16 11:30 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-16 11:30 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Andrew Morton, Greg Kroah-Hartman, Eric Dumazet, Wolfram Sang,
Petr Mladek, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Van Hensbergen, Harry Wentland, Herbert Xu,
Ilya Leoshkevich, Ingo Molnar, Jason Wang, Jens Axboe,
Marek Szyprowski, Marco Elver, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S . Tsirkin,
Michal Simek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik
On Fri, Nov 29, 2019 at 5:22 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > Insert KMSAN hooks that check for potential memory errors and/or make
> > necessary bookkeeping changes:
>
> I think it makes sense to split this patch into 2+ parts, the first
> one adds hooks for internal KMSAN bookkeeping:
>
Done in v4.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 24/36] kmsan: disable instrumentation of certain functions
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (21 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 23/36] kmsan: call KMSAN hooks where needed glider
@ 2019-11-22 11:26 ` glider
2019-11-29 14:59 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 25/36] kmsan: unpoison |tlb| in arch_tlb_gather_mmu() glider
` (12 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Thomas Gleixner, Andrew Morton, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, gor, wsa
Some functions are called from handwritten assembly, and therefore don't
have their arguments' metadata fully set up by the instrumentation code.
Mark them with __no_sanitize_memory to avoid false positives from
spreading further.
Certain functions perform task switching, so that the value of |current|
is different as they proceed. Because KMSAN state pointer is only read
once at the beginning of the function, touching it after |current| has
changed may be dangerous.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v3:
- removed TODOs from comments
Change-Id: I684d23dac5a22eb0a4cea71993cb934302b17cea
---
arch/x86/entry/common.c | 1 +
arch/x86/include/asm/irq_regs.h | 1 +
arch/x86/include/asm/syscall_wrapper.h | 1 +
arch/x86/kernel/apic/apic.c | 1 +
arch/x86/kernel/dumpstack_64.c | 1 +
arch/x86/kernel/process_64.c | 5 +++++
arch/x86/kernel/traps.c | 12 ++++++++++--
arch/x86/kernel/uprobes.c | 7 ++++++-
kernel/profile.c | 1 +
kernel/sched/core.c | 6 ++++++
10 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 3f8e22615812..0dd5b2acb355 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -275,6 +275,7 @@ __visible inline void syscall_return_slowpath(struct pt_regs *regs)
}
#ifdef CONFIG_X86_64
+__no_sanitize_memory
__visible void do_syscall_64(unsigned long nr, struct pt_regs *regs)
{
struct thread_info *ti;
diff --git a/arch/x86/include/asm/irq_regs.h b/arch/x86/include/asm/irq_regs.h
index 187ce59aea28..d65a00bd6f02 100644
--- a/arch/x86/include/asm/irq_regs.h
+++ b/arch/x86/include/asm/irq_regs.h
@@ -14,6 +14,7 @@
DECLARE_PER_CPU(struct pt_regs *, irq_regs);
+__no_sanitize_memory
static inline struct pt_regs *get_irq_regs(void)
{
return __this_cpu_read(irq_regs);
diff --git a/arch/x86/include/asm/syscall_wrapper.h b/arch/x86/include/asm/syscall_wrapper.h
index e046a405743d..43910ce1b53b 100644
--- a/arch/x86/include/asm/syscall_wrapper.h
+++ b/arch/x86/include/asm/syscall_wrapper.h
@@ -159,6 +159,7 @@
ALLOW_ERROR_INJECTION(__x64_sys##name, ERRNO); \
static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)); \
static inline long __do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__));\
+ __no_sanitize_memory \
asmlinkage long __x64_sys##name(const struct pt_regs *regs) \
{ \
return __se_sys##name(SC_X86_64_REGS_TO_ARGS(x,__VA_ARGS__));\
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 9e2dd2b296cd..7b24bda22c38 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1118,6 +1118,7 @@ static void local_apic_timer_interrupt(void)
* [ if a single-CPU system runs an SMP kernel then we call the local
* interrupt as well. Thus we cannot inline the local irq ... ]
*/
+__no_sanitize_memory /* |regs| may be uninitialized */
__visible void __irq_entry smp_apic_timer_interrupt(struct pt_regs *regs)
{
struct pt_regs *old_regs = set_irq_regs(regs);
diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c
index 753b8cfe8b8a..ba883d282a43 100644
--- a/arch/x86/kernel/dumpstack_64.c
+++ b/arch/x86/kernel/dumpstack_64.c
@@ -143,6 +143,7 @@ static bool in_irq_stack(unsigned long *stack, struct stack_info *info)
return true;
}
+__no_sanitize_memory
int get_stack_info(unsigned long *stack, struct task_struct *task,
struct stack_info *info, unsigned long *visit_mask)
{
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index af64519b2695..70e33150a83a 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -500,6 +500,11 @@ void compat_start_thread(struct pt_regs *regs, u32 new_ip, u32 new_sp)
* Kprobes not supported here. Set the probe on schedule instead.
* Function graph tracer not supported too.
*/
+/*
+ * Avoid touching KMSAN state or reporting anything here, as __switch_to() does
+ * weird things with tasks.
+ */
+__no_sanitize_memory
__visible __notrace_funcgraph struct task_struct *
__switch_to(struct task_struct *prev_p, struct task_struct *next_p)
{
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 4bb0f8447112..a94282d1f60b 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -618,7 +618,10 @@ NOKPROBE_SYMBOL(do_int3);
* Help handler running on a per-cpu (IST or entry trampoline) stack
* to switch to the normal thread stack if the interrupted code was in
* user mode. The actual stack switch is done in entry_64.S
+ *
+ * This function switches the registers - don't instrument it with KMSAN!
*/
+__no_sanitize_memory
asmlinkage __visible notrace struct pt_regs *sync_regs(struct pt_regs *eregs)
{
struct pt_regs *regs = (struct pt_regs *)this_cpu_read(cpu_current_top_of_stack) - 1;
@@ -634,6 +637,11 @@ struct bad_iret_stack {
};
asmlinkage __visible notrace
+/*
+ * Dark magic happening here, let's not instrument this function.
+ * Also avoid copying any metadata by using raw __memmove().
+ */
+__no_sanitize_memory
struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s)
{
/*
@@ -648,10 +656,10 @@ struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s)
(struct bad_iret_stack *)this_cpu_read(cpu_tss_rw.x86_tss.sp0) - 1;
/* Copy the IRET target to the new stack. */
- memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8);
+ __memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8);
/* Copy the remainder of the stack from the current stack. */
- memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
+ __memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
BUG_ON(!user_mode(&new_stack->regs));
return new_stack;
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 8cd745ef8c7b..bcd4bf5a909f 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -8,6 +8,7 @@
* Jim Keniston
*/
#include <linux/kernel.h>
+#include <linux/kmsan-checks.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
#include <linux/uprobes.h>
@@ -997,9 +998,13 @@ int arch_uprobe_post_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)
int arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data)
{
struct die_args *args = data;
- struct pt_regs *regs = args->regs;
+ struct pt_regs *regs;
int ret = NOTIFY_DONE;
+ kmsan_unpoison_shadow(args, sizeof(*args));
+ regs = args->regs;
+ if (regs)
+ kmsan_unpoison_shadow(regs, sizeof(*regs));
/* We are only interested in userspace traps */
if (regs && !user_mode(regs))
return NOTIFY_DONE;
diff --git a/kernel/profile.c b/kernel/profile.c
index af7c94bf5fa1..835a5b66d1a4 100644
--- a/kernel/profile.c
+++ b/kernel/profile.c
@@ -399,6 +399,7 @@ void profile_hits(int type, void *__pc, unsigned int nr_hits)
}
EXPORT_SYMBOL_GPL(profile_hits);
+__no_sanitize_memory
void profile_tick(int type)
{
struct pt_regs *regs = get_irq_regs();
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index dd05a378631a..674d36fe9d44 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -475,6 +475,7 @@ void wake_q_add_safe(struct wake_q_head *head, struct task_struct *task)
put_task_struct(task);
}
+__no_sanitize_memory /* context switching here */
void wake_up_q(struct wake_q_head *head)
{
struct wake_q_node *node = head->first;
@@ -3180,6 +3181,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,
* past. prev == current is still correct but we need to recalculate this_rq
* because prev may have moved to another CPU.
*/
+__no_sanitize_memory /* |current| changes here */
static struct rq *finish_task_switch(struct task_struct *prev)
__releases(rq->lock)
{
@@ -3986,6 +3988,7 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
*
* WARNING: must be called with preemption disabled!
*/
+__no_sanitize_memory /* |current| changes here */
static void __sched notrace __schedule(bool preempt)
{
struct task_struct *prev, *next;
@@ -4605,6 +4608,7 @@ int task_prio(const struct task_struct *p)
*
* Return: 1 if the CPU is currently idle. 0 otherwise.
*/
+__no_sanitize_memory /* nothing to report here */
int idle_cpu(int cpu)
{
struct rq *rq = cpu_rq(cpu);
@@ -6544,6 +6548,7 @@ static struct kmem_cache *task_group_cache __read_mostly;
DECLARE_PER_CPU(cpumask_var_t, load_balance_mask);
DECLARE_PER_CPU(cpumask_var_t, select_idle_mask);
+__no_sanitize_memory
void __init sched_init(void)
{
unsigned long ptr = 0;
@@ -6716,6 +6721,7 @@ static inline int preempt_count_equals(int preempt_offset)
return (nested == preempt_offset);
}
+__no_sanitize_memory /* expect the arguments to be initialized */
void __might_sleep(const char *file, int line, int preempt_offset)
{
/*
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 24/36] kmsan: disable instrumentation of certain functions
2019-11-22 11:26 ` [PATCH RFC v3 24/36] kmsan: disable instrumentation of certain functions glider
@ 2019-11-29 14:59 ` Andrey Konovalov
2019-12-18 10:02 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 14:59 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Thomas Gleixner, Andrew Morton, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, pmladek,
Qian Cai, Randy Dunlap, Robin Murphy, sergey.senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> Some functions are called from handwritten assembly, and therefore don't
> have their arguments' metadata fully set up by the instrumentation code.
> Mark them with __no_sanitize_memory to avoid false positives from
> spreading further.
> Certain functions perform task switching, so that the value of |current|
> is different as they proceed. Because KMSAN state pointer is only read
> once at the beginning of the function, touching it after |current| has
> changed may be dangerous.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
> v3:
> - removed TODOs from comments
>
> Change-Id: I684d23dac5a22eb0a4cea71993cb934302b17cea
> ---
> arch/x86/entry/common.c | 1 +
> arch/x86/include/asm/irq_regs.h | 1 +
> arch/x86/include/asm/syscall_wrapper.h | 1 +
> arch/x86/kernel/apic/apic.c | 1 +
> arch/x86/kernel/dumpstack_64.c | 1 +
> arch/x86/kernel/process_64.c | 5 +++++
> arch/x86/kernel/traps.c | 12 ++++++++++--
> arch/x86/kernel/uprobes.c | 7 ++++++-
> kernel/profile.c | 1 +
> kernel/sched/core.c | 6 ++++++
> 10 files changed, 33 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
> index 3f8e22615812..0dd5b2acb355 100644
> --- a/arch/x86/entry/common.c
> +++ b/arch/x86/entry/common.c
> @@ -275,6 +275,7 @@ __visible inline void syscall_return_slowpath(struct pt_regs *regs)
> }
>
> #ifdef CONFIG_X86_64
> +__no_sanitize_memory
> __visible void do_syscall_64(unsigned long nr, struct pt_regs *regs)
> {
> struct thread_info *ti;
> diff --git a/arch/x86/include/asm/irq_regs.h b/arch/x86/include/asm/irq_regs.h
> index 187ce59aea28..d65a00bd6f02 100644
> --- a/arch/x86/include/asm/irq_regs.h
> +++ b/arch/x86/include/asm/irq_regs.h
> @@ -14,6 +14,7 @@
>
> DECLARE_PER_CPU(struct pt_regs *, irq_regs);
>
> +__no_sanitize_memory
> static inline struct pt_regs *get_irq_regs(void)
> {
> return __this_cpu_read(irq_regs);
> diff --git a/arch/x86/include/asm/syscall_wrapper.h b/arch/x86/include/asm/syscall_wrapper.h
> index e046a405743d..43910ce1b53b 100644
> --- a/arch/x86/include/asm/syscall_wrapper.h
> +++ b/arch/x86/include/asm/syscall_wrapper.h
> @@ -159,6 +159,7 @@
> ALLOW_ERROR_INJECTION(__x64_sys##name, ERRNO); \
> static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)); \
> static inline long __do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__));\
> + __no_sanitize_memory \
> asmlinkage long __x64_sys##name(const struct pt_regs *regs) \
> { \
> return __se_sys##name(SC_X86_64_REGS_TO_ARGS(x,__VA_ARGS__));\
> diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
> index 9e2dd2b296cd..7b24bda22c38 100644
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -1118,6 +1118,7 @@ static void local_apic_timer_interrupt(void)
> * [ if a single-CPU system runs an SMP kernel then we call the local
> * interrupt as well. Thus we cannot inline the local irq ... ]
> */
> +__no_sanitize_memory /* |regs| may be uninitialized */
The comment style around __no_sanitize_memory looks very different for
different call sites, perhaps it makes sense to unify it somehow.
> __visible void __irq_entry smp_apic_timer_interrupt(struct pt_regs *regs)
> {
> struct pt_regs *old_regs = set_irq_regs(regs);
> diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c
> index 753b8cfe8b8a..ba883d282a43 100644
> --- a/arch/x86/kernel/dumpstack_64.c
> +++ b/arch/x86/kernel/dumpstack_64.c
> @@ -143,6 +143,7 @@ static bool in_irq_stack(unsigned long *stack, struct stack_info *info)
> return true;
> }
>
> +__no_sanitize_memory
> int get_stack_info(unsigned long *stack, struct task_struct *task,
> struct stack_info *info, unsigned long *visit_mask)
> {
> diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
> index af64519b2695..70e33150a83a 100644
> --- a/arch/x86/kernel/process_64.c
> +++ b/arch/x86/kernel/process_64.c
> @@ -500,6 +500,11 @@ void compat_start_thread(struct pt_regs *regs, u32 new_ip, u32 new_sp)
> * Kprobes not supported here. Set the probe on schedule instead.
> * Function graph tracer not supported too.
> */
> +/*
> + * Avoid touching KMSAN state or reporting anything here, as __switch_to() does
> + * weird things with tasks.
> + */
> +__no_sanitize_memory
> __visible __notrace_funcgraph struct task_struct *
> __switch_to(struct task_struct *prev_p, struct task_struct *next_p)
> {
> diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
> index 4bb0f8447112..a94282d1f60b 100644
> --- a/arch/x86/kernel/traps.c
> +++ b/arch/x86/kernel/traps.c
> @@ -618,7 +618,10 @@ NOKPROBE_SYMBOL(do_int3);
> * Help handler running on a per-cpu (IST or entry trampoline) stack
> * to switch to the normal thread stack if the interrupted code was in
> * user mode. The actual stack switch is done in entry_64.S
> + *
> + * This function switches the registers - don't instrument it with KMSAN!
> */
> +__no_sanitize_memory
> asmlinkage __visible notrace struct pt_regs *sync_regs(struct pt_regs *eregs)
> {
> struct pt_regs *regs = (struct pt_regs *)this_cpu_read(cpu_current_top_of_stack) - 1;
> @@ -634,6 +637,11 @@ struct bad_iret_stack {
> };
>
> asmlinkage __visible notrace
> +/*
> + * Dark magic happening here, let's not instrument this function.
> + * Also avoid copying any metadata by using raw __memmove().
> + */
> +__no_sanitize_memory
> struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s)
> {
> /*
> @@ -648,10 +656,10 @@ struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s)
> (struct bad_iret_stack *)this_cpu_read(cpu_tss_rw.x86_tss.sp0) - 1;
>
> /* Copy the IRET target to the new stack. */
> - memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8);
> + __memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8);
>
> /* Copy the remainder of the stack from the current stack. */
> - memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
> + __memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
Looks like this change should go into a separate patch.
>
> BUG_ON(!user_mode(&new_stack->regs));
> return new_stack;
> diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
> index 8cd745ef8c7b..bcd4bf5a909f 100644
> --- a/arch/x86/kernel/uprobes.c
> +++ b/arch/x86/kernel/uprobes.c
> @@ -8,6 +8,7 @@
> * Jim Keniston
> */
> #include <linux/kernel.h>
> +#include <linux/kmsan-checks.h>
> #include <linux/sched.h>
> #include <linux/ptrace.h>
> #include <linux/uprobes.h>
> @@ -997,9 +998,13 @@ int arch_uprobe_post_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)
> int arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data)
> {
> struct die_args *args = data;
> - struct pt_regs *regs = args->regs;
> + struct pt_regs *regs;
> int ret = NOTIFY_DONE;
>
> + kmsan_unpoison_shadow(args, sizeof(*args));
> + regs = args->regs;
> + if (regs)
> + kmsan_unpoison_shadow(regs, sizeof(*regs));
This one as well.
> /* We are only interested in userspace traps */
> if (regs && !user_mode(regs))
> return NOTIFY_DONE;
> diff --git a/kernel/profile.c b/kernel/profile.c
> index af7c94bf5fa1..835a5b66d1a4 100644
> --- a/kernel/profile.c
> +++ b/kernel/profile.c
> @@ -399,6 +399,7 @@ void profile_hits(int type, void *__pc, unsigned int nr_hits)
> }
> EXPORT_SYMBOL_GPL(profile_hits);
>
> +__no_sanitize_memory
> void profile_tick(int type)
> {
> struct pt_regs *regs = get_irq_regs();
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index dd05a378631a..674d36fe9d44 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -475,6 +475,7 @@ void wake_q_add_safe(struct wake_q_head *head, struct task_struct *task)
> put_task_struct(task);
> }
>
> +__no_sanitize_memory /* context switching here */
> void wake_up_q(struct wake_q_head *head)
> {
> struct wake_q_node *node = head->first;
> @@ -3180,6 +3181,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,
> * past. prev == current is still correct but we need to recalculate this_rq
> * because prev may have moved to another CPU.
> */
> +__no_sanitize_memory /* |current| changes here */
> static struct rq *finish_task_switch(struct task_struct *prev)
> __releases(rq->lock)
> {
> @@ -3986,6 +3988,7 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
> *
> * WARNING: must be called with preemption disabled!
> */
> +__no_sanitize_memory /* |current| changes here */
> static void __sched notrace __schedule(bool preempt)
> {
> struct task_struct *prev, *next;
> @@ -4605,6 +4608,7 @@ int task_prio(const struct task_struct *p)
> *
> * Return: 1 if the CPU is currently idle. 0 otherwise.
> */
> +__no_sanitize_memory /* nothing to report here */
> int idle_cpu(int cpu)
> {
> struct rq *rq = cpu_rq(cpu);
> @@ -6544,6 +6548,7 @@ static struct kmem_cache *task_group_cache __read_mostly;
> DECLARE_PER_CPU(cpumask_var_t, load_balance_mask);
> DECLARE_PER_CPU(cpumask_var_t, select_idle_mask);
>
> +__no_sanitize_memory
> void __init sched_init(void)
> {
> unsigned long ptr = 0;
> @@ -6716,6 +6721,7 @@ static inline int preempt_count_equals(int preempt_offset)
> return (nested == preempt_offset);
> }
>
> +__no_sanitize_memory /* expect the arguments to be initialized */
> void __might_sleep(const char *file, int line, int preempt_offset)
> {
> /*
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 24/36] kmsan: disable instrumentation of certain functions
2019-11-29 14:59 ` Andrey Konovalov
@ 2019-12-18 10:02 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-18 10:02 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Thomas Gleixner, Andrew Morton, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Vasily Gorbik, Wolfram Sang
> The comment style around __no_sanitize_memory looks very different for
> different call sites, perhaps it makes sense to unify it somehow.
Done in v4.
> > +/*
> > + * Dark magic happening here, let's not instrument this function.
> > + * Also avoid copying any metadata by using raw __memmove().
> > + */
> > +__no_sanitize_memory
> > struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s)
> > {
> > /*
> > @@ -648,10 +656,10 @@ struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s)
> > (struct bad_iret_stack *)this_cpu_read(cpu_tss_rw.x86_tss.sp0) - 1;
> >
> > /* Copy the IRET target to the new stack. */
> > - memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8);
> > + __memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8);
> >
> > /* Copy the remainder of the stack from the current stack. */
> > - memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
> > + __memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
>
> Looks like this change should go into a separate patch.
I disagree. Both the __no_sanitize_memory annotation and __memmove()
calls serve the same purpose: avoid calling KMSAN code from
fixup_bad_iret().
I don't think it makes sense to separate them.
> > + kmsan_unpoison_shadow(args, sizeof(*args));
> > + regs = args->regs;
> > + if (regs)
> > + kmsan_unpoison_shadow(regs, sizeof(*regs));
>
> This one as well.
Done in v4.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 25/36] kmsan: unpoison |tlb| in arch_tlb_gather_mmu()
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (22 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 24/36] kmsan: disable instrumentation of certain functions glider
@ 2019-11-22 11:26 ` glider
2019-11-29 15:08 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 26/36] kmsan: use __msan_memcpy() where possible glider
` (11 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
This is a hack to reduce stackdepot pressure.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I22a201e7e4f67ed74f8129072f12e5351b26103a
---
mm/mmu_gather.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
index 7d70e5c78f97..8c5ea2d2e7d5 100644
--- a/mm/mmu_gather.c
+++ b/mm/mmu_gather.c
@@ -1,6 +1,7 @@
#include <linux/gfp.h>
#include <linux/highmem.h>
#include <linux/kernel.h>
+#include <linux/kmsan-checks.h>
#include <linux/mmdebug.h>
#include <linux/mm_types.h>
#include <linux/pagemap.h>
@@ -206,6 +207,15 @@ void tlb_flush_mmu(struct mmu_gather *tlb)
void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
unsigned long start, unsigned long end)
{
+ /*
+ * TODO(glider): struct mmu_gather contains 7 1-bit fields packed into a
+ * 32-bit unsigned int value. The remaining 25 bits remain uninitialized
+ * and are never used, but KMSAN updates the origin for them in
+ * zap_pXX_range() in mm/memory.c, thus creating very long origin
+ * chains. This is technically correct, but consumes too much memory.
+ * Unpoisoning the whole structure will prevent creating such chains.
+ */
+ kmsan_unpoison_shadow(tlb, sizeof(*tlb));
tlb->mm = mm;
/* Is it from 0 to ~0? */
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 25/36] kmsan: unpoison |tlb| in arch_tlb_gather_mmu()
2019-11-22 11:26 ` [PATCH RFC v3 25/36] kmsan: unpoison |tlb| in arch_tlb_gather_mmu() glider
@ 2019-11-29 15:08 ` Andrey Konovalov
2019-12-03 14:19 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 15:08 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, pmladek,
Qian Cai, Randy Dunlap, Robin Murphy, sergey.senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> This is a hack to reduce stackdepot pressure.
I think this patch needs a better description :) Like in the comment below.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
>
> Change-Id: I22a201e7e4f67ed74f8129072f12e5351b26103a
> ---
> mm/mmu_gather.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
> index 7d70e5c78f97..8c5ea2d2e7d5 100644
> --- a/mm/mmu_gather.c
> +++ b/mm/mmu_gather.c
> @@ -1,6 +1,7 @@
> #include <linux/gfp.h>
> #include <linux/highmem.h>
> #include <linux/kernel.h>
> +#include <linux/kmsan-checks.h>
> #include <linux/mmdebug.h>
> #include <linux/mm_types.h>
> #include <linux/pagemap.h>
> @@ -206,6 +207,15 @@ void tlb_flush_mmu(struct mmu_gather *tlb)
> void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
> unsigned long start, unsigned long end)
> {
> + /*
> + * TODO(glider): struct mmu_gather contains 7 1-bit fields packed into a
Remove TODO?
> + * 32-bit unsigned int value. The remaining 25 bits remain uninitialized
> + * and are never used, but KMSAN updates the origin for them in
> + * zap_pXX_range() in mm/memory.c, thus creating very long origin
> + * chains. This is technically correct, but consumes too much memory.
> + * Unpoisoning the whole structure will prevent creating such chains.
> + */
> + kmsan_unpoison_shadow(tlb, sizeof(*tlb));
> tlb->mm = mm;
>
> /* Is it from 0 to ~0? */
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 25/36] kmsan: unpoison |tlb| in arch_tlb_gather_mmu()
2019-11-29 15:08 ` Andrey Konovalov
@ 2019-12-03 14:19 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 14:19 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Fri, Nov 29, 2019 at 4:09 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > This is a hack to reduce stackdepot pressure.
>
> I think this patch needs a better description :) Like in the comment below.
>
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> >
> > ---
> >
> > Change-Id: I22a201e7e4f67ed74f8129072f12e5351b26103a
> > ---
> > mm/mmu_gather.c | 10 ++++++++++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
> > index 7d70e5c78f97..8c5ea2d2e7d5 100644
> > --- a/mm/mmu_gather.c
> > +++ b/mm/mmu_gather.c
> > @@ -1,6 +1,7 @@
> > #include <linux/gfp.h>
> > #include <linux/highmem.h>
> > #include <linux/kernel.h>
> > +#include <linux/kmsan-checks.h>
> > #include <linux/mmdebug.h>
> > #include <linux/mm_types.h>
> > #include <linux/pagemap.h>
> > @@ -206,6 +207,15 @@ void tlb_flush_mmu(struct mmu_gather *tlb)
> > void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
> > unsigned long start, unsigned long end)
> > {
> > + /*
> > + * TODO(glider): struct mmu_gather contains 7 1-bit fields packed into a
>
> Remove TODO?
Will do, thanks!
> > + * 32-bit unsigned int value. The remaining 25 bits remain uninitialized
> > + * and are never used, but KMSAN updates the origin for them in
> > + * zap_pXX_range() in mm/memory.c, thus creating very long origin
> > + * chains. This is technically correct, but consumes too much memory.
> > + * Unpoisoning the whole structure will prevent creating such chains.
> > + */
> > + kmsan_unpoison_shadow(tlb, sizeof(*tlb));
> > tlb->mm = mm;
> >
> > /* Is it from 0 to ~0? */
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 26/36] kmsan: use __msan_memcpy() where possible.
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (23 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 25/36] kmsan: unpoison |tlb| in arch_tlb_gather_mmu() glider
@ 2019-11-22 11:26 ` glider
2019-11-29 15:13 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends glider
` (10 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
Unless stated otherwise (by explicitly calling __memcpy()) we want all
memcpy() calls to call __msan_memcpy() so that shadow and origin values
are updated accordingly.
Bootloader must still the default string functions to avoid crashes.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v3:
- use default string functions in the bootloader
Change-Id: Ib2512ce5aa8d457453dd38caa12f58f002166813
---
arch/x86/boot/compressed/misc.h | 1 +
arch/x86/include/asm/string_64.h | 9 ++++++++-
include/linux/compiler.h | 9 ++++++++-
include/linux/string.h | 2 ++
4 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
index c8181392f70d..dd4bd8c5d97a 100644
--- a/arch/x86/boot/compressed/misc.h
+++ b/arch/x86/boot/compressed/misc.h
@@ -12,6 +12,7 @@
#undef CONFIG_PARAVIRT_XXL
#undef CONFIG_PARAVIRT_SPINLOCKS
#undef CONFIG_KASAN
+#undef CONFIG_KMSAN
/* cpu_feature_enabled() cannot be used this early */
#define USE_EARLY_PGTABLE_L5
diff --git a/arch/x86/include/asm/string_64.h b/arch/x86/include/asm/string_64.h
index 75314c3dbe47..d3c76d910c23 100644
--- a/arch/x86/include/asm/string_64.h
+++ b/arch/x86/include/asm/string_64.h
@@ -11,7 +11,13 @@
function. */
#define __HAVE_ARCH_MEMCPY 1
+#if defined(CONFIG_KMSAN)
+#undef memcpy
+/* __msan_memcpy() is defined in compiler.h */
+#define memcpy(dst, src, len) __msan_memcpy(dst, src, len)
+#else
extern void *memcpy(void *to, const void *from, size_t len);
+#endif
extern void *__memcpy(void *to, const void *from, size_t len);
#define __HAVE_ARCH_MEMSET
@@ -64,7 +70,8 @@ char *strcpy(char *dest, const char *src);
char *strcat(char *dest, const char *src);
int strcmp(const char *cs, const char *ct);
-#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
+#if (defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)) || \
+ (defined(CONFIG_KMSAN) && !defined(__SANITIZE_MEMORY__))
/*
* For files that not instrumented (e.g. mm/slub.c) we
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 99d40f31a2c3..9ce11f4f4cb2 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -179,6 +179,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
#include <uapi/linux/types.h>
+#ifdef CONFIG_KMSAN
+void *__msan_memcpy(void *dst, const void *src, u64 size);
+#define __DO_MEMCPY(res, p, size) __msan_memcpy(res, p, size)
+#else
+#define __DO_MEMCPY(res, p, size) __builtin_memcpy(res, p, size)
+#endif
+
#define __READ_ONCE_SIZE \
({ \
switch (size) { \
@@ -188,7 +195,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
case 8: *(__u64 *)res = *(volatile __u64 *)p; break; \
default: \
barrier(); \
- __builtin_memcpy((void *)res, (const void *)p, size); \
+ __DO_MEMCPY((void *)res, (const void *)p, size); \
barrier(); \
} \
})
diff --git a/include/linux/string.h b/include/linux/string.h
index b6ccdc2c7f02..5d8ce09cba2e 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -363,6 +363,7 @@ __FORTIFY_INLINE void *memset(void *p, int c, __kernel_size_t size)
return __builtin_memset(p, c, size);
}
+#ifndef CONFIG_KMSAN
__FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
{
size_t p_size = __builtin_object_size(p, 0);
@@ -377,6 +378,7 @@ __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
fortify_panic(__func__);
return __builtin_memcpy(p, q, size);
}
+#endif
__FORTIFY_INLINE void *memmove(void *p, const void *q, __kernel_size_t size)
{
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 26/36] kmsan: use __msan_memcpy() where possible.
2019-11-22 11:26 ` [PATCH RFC v3 26/36] kmsan: use __msan_memcpy() where possible glider
@ 2019-11-29 15:13 ` Andrey Konovalov
2019-12-05 15:46 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 15:13 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, pmladek,
Qian Cai, Randy Dunlap, Robin Murphy, sergey.senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> Unless stated otherwise (by explicitly calling __memcpy()) we want all
> memcpy() calls to call __msan_memcpy() so that shadow and origin values
> are updated accordingly.
Why do we only do this for memcpy() but not for memove() and others?
>
> Bootloader must still the default string functions to avoid crashes.
must still use
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
> v3:
> - use default string functions in the bootloader
>
> Change-Id: Ib2512ce5aa8d457453dd38caa12f58f002166813
> ---
> arch/x86/boot/compressed/misc.h | 1 +
> arch/x86/include/asm/string_64.h | 9 ++++++++-
> include/linux/compiler.h | 9 ++++++++-
> include/linux/string.h | 2 ++
> 4 files changed, 19 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
> index c8181392f70d..dd4bd8c5d97a 100644
> --- a/arch/x86/boot/compressed/misc.h
> +++ b/arch/x86/boot/compressed/misc.h
> @@ -12,6 +12,7 @@
> #undef CONFIG_PARAVIRT_XXL
> #undef CONFIG_PARAVIRT_SPINLOCKS
> #undef CONFIG_KASAN
> +#undef CONFIG_KMSAN
>
> /* cpu_feature_enabled() cannot be used this early */
> #define USE_EARLY_PGTABLE_L5
> diff --git a/arch/x86/include/asm/string_64.h b/arch/x86/include/asm/string_64.h
> index 75314c3dbe47..d3c76d910c23 100644
> --- a/arch/x86/include/asm/string_64.h
> +++ b/arch/x86/include/asm/string_64.h
> @@ -11,7 +11,13 @@
> function. */
>
> #define __HAVE_ARCH_MEMCPY 1
> +#if defined(CONFIG_KMSAN)
> +#undef memcpy
> +/* __msan_memcpy() is defined in compiler.h */
> +#define memcpy(dst, src, len) __msan_memcpy(dst, src, len)
> +#else
> extern void *memcpy(void *to, const void *from, size_t len);
> +#endif
> extern void *__memcpy(void *to, const void *from, size_t len);
>
> #define __HAVE_ARCH_MEMSET
> @@ -64,7 +70,8 @@ char *strcpy(char *dest, const char *src);
> char *strcat(char *dest, const char *src);
> int strcmp(const char *cs, const char *ct);
>
> -#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
> +#if (defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)) || \
> + (defined(CONFIG_KMSAN) && !defined(__SANITIZE_MEMORY__))
>
> /*
> * For files that not instrumented (e.g. mm/slub.c) we
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index 99d40f31a2c3..9ce11f4f4cb2 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -179,6 +179,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
>
> #include <uapi/linux/types.h>
>
> +#ifdef CONFIG_KMSAN
> +void *__msan_memcpy(void *dst, const void *src, u64 size);
> +#define __DO_MEMCPY(res, p, size) __msan_memcpy(res, p, size)
> +#else
> +#define __DO_MEMCPY(res, p, size) __builtin_memcpy(res, p, size)
> +#endif
> +
> #define __READ_ONCE_SIZE \
> ({ \
> switch (size) { \
> @@ -188,7 +195,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
> case 8: *(__u64 *)res = *(volatile __u64 *)p; break; \
> default: \
> barrier(); \
> - __builtin_memcpy((void *)res, (const void *)p, size); \
> + __DO_MEMCPY((void *)res, (const void *)p, size); \
> barrier(); \
> } \
> })
> diff --git a/include/linux/string.h b/include/linux/string.h
> index b6ccdc2c7f02..5d8ce09cba2e 100644
> --- a/include/linux/string.h
> +++ b/include/linux/string.h
> @@ -363,6 +363,7 @@ __FORTIFY_INLINE void *memset(void *p, int c, __kernel_size_t size)
> return __builtin_memset(p, c, size);
> }
>
> +#ifndef CONFIG_KMSAN
> __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
> {
> size_t p_size = __builtin_object_size(p, 0);
> @@ -377,6 +378,7 @@ __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
> fortify_panic(__func__);
> return __builtin_memcpy(p, q, size);
> }
> +#endif
>
> __FORTIFY_INLINE void *memmove(void *p, const void *q, __kernel_size_t size)
> {
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 26/36] kmsan: use __msan_memcpy() where possible.
2019-11-29 15:13 ` Andrey Konovalov
@ 2019-12-05 15:46 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 15:46 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Fri, Nov 29, 2019 at 4:13 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > Unless stated otherwise (by explicitly calling __memcpy()) we want all
> > memcpy() calls to call __msan_memcpy() so that shadow and origin values
> > are updated accordingly.
>
> Why do we only do this for memcpy() but not for memove() and others?
Hm, interesting.
Looks like I simply forgot to add memset() and memmove(). Could have
costed us some false negatives.
> >
> > Bootloader must still the default string functions to avoid crashes.
>
> must still use
Ack
>
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> > v3:
> > - use default string functions in the bootloader
> >
> > Change-Id: Ib2512ce5aa8d457453dd38caa12f58f002166813
> > ---
> > arch/x86/boot/compressed/misc.h | 1 +
> > arch/x86/include/asm/string_64.h | 9 ++++++++-
> > include/linux/compiler.h | 9 ++++++++-
> > include/linux/string.h | 2 ++
> > 4 files changed, 19 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
> > index c8181392f70d..dd4bd8c5d97a 100644
> > --- a/arch/x86/boot/compressed/misc.h
> > +++ b/arch/x86/boot/compressed/misc.h
> > @@ -12,6 +12,7 @@
> > #undef CONFIG_PARAVIRT_XXL
> > #undef CONFIG_PARAVIRT_SPINLOCKS
> > #undef CONFIG_KASAN
> > +#undef CONFIG_KMSAN
> >
> > /* cpu_feature_enabled() cannot be used this early */
> > #define USE_EARLY_PGTABLE_L5
> > diff --git a/arch/x86/include/asm/string_64.h b/arch/x86/include/asm/string_64.h
> > index 75314c3dbe47..d3c76d910c23 100644
> > --- a/arch/x86/include/asm/string_64.h
> > +++ b/arch/x86/include/asm/string_64.h
> > @@ -11,7 +11,13 @@
> > function. */
> >
> > #define __HAVE_ARCH_MEMCPY 1
> > +#if defined(CONFIG_KMSAN)
> > +#undef memcpy
> > +/* __msan_memcpy() is defined in compiler.h */
> > +#define memcpy(dst, src, len) __msan_memcpy(dst, src, len)
> > +#else
> > extern void *memcpy(void *to, const void *from, size_t len);
> > +#endif
> > extern void *__memcpy(void *to, const void *from, size_t len);
> >
> > #define __HAVE_ARCH_MEMSET
> > @@ -64,7 +70,8 @@ char *strcpy(char *dest, const char *src);
> > char *strcat(char *dest, const char *src);
> > int strcmp(const char *cs, const char *ct);
> >
> > -#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
> > +#if (defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)) || \
> > + (defined(CONFIG_KMSAN) && !defined(__SANITIZE_MEMORY__))
> >
> > /*
> > * For files that not instrumented (e.g. mm/slub.c) we
> > diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> > index 99d40f31a2c3..9ce11f4f4cb2 100644
> > --- a/include/linux/compiler.h
> > +++ b/include/linux/compiler.h
> > @@ -179,6 +179,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
> >
> > #include <uapi/linux/types.h>
> >
> > +#ifdef CONFIG_KMSAN
> > +void *__msan_memcpy(void *dst, const void *src, u64 size);
> > +#define __DO_MEMCPY(res, p, size) __msan_memcpy(res, p, size)
> > +#else
> > +#define __DO_MEMCPY(res, p, size) __builtin_memcpy(res, p, size)
> > +#endif
> > +
> > #define __READ_ONCE_SIZE \
> > ({ \
> > switch (size) { \
> > @@ -188,7 +195,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
> > case 8: *(__u64 *)res = *(volatile __u64 *)p; break; \
> > default: \
> > barrier(); \
> > - __builtin_memcpy((void *)res, (const void *)p, size); \
> > + __DO_MEMCPY((void *)res, (const void *)p, size); \
> > barrier(); \
> > } \
> > })
> > diff --git a/include/linux/string.h b/include/linux/string.h
> > index b6ccdc2c7f02..5d8ce09cba2e 100644
> > --- a/include/linux/string.h
> > +++ b/include/linux/string.h
> > @@ -363,6 +363,7 @@ __FORTIFY_INLINE void *memset(void *p, int c, __kernel_size_t size)
> > return __builtin_memset(p, c, size);
> > }
> >
> > +#ifndef CONFIG_KMSAN
> > __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
> > {
> > size_t p_size = __builtin_object_size(p, 0);
> > @@ -377,6 +378,7 @@ __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
> > fortify_panic(__func__);
> > return __builtin_memcpy(p, q, size);
> > }
> > +#endif
> >
> > __FORTIFY_INLINE void *memmove(void *p, const void *q, __kernel_size_t size)
> > {
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (24 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 26/36] kmsan: use __msan_memcpy() where possible glider
@ 2019-11-22 11:26 ` glider
2019-11-29 15:34 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 28/36] kmsan: enable KMSAN builds glider
` (9 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Alexander Viro, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
Memory that is copied from userspace must be unpoisoned.
Before copying memory to userspace, check it and report an error if it
contains uninitialized bits.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
v3:
- fixed compilation errors reported by kbuild test bot
Change-Id: I38428b9c7d1909b8441dcec1749b080494a7af99
---
arch/x86/include/asm/uaccess.h | 12 ++++++++++++
include/asm-generic/cacheflush.h | 7 ++++++-
include/asm-generic/uaccess.h | 12 ++++++++++--
include/linux/uaccess.h | 32 +++++++++++++++++++++++++++-----
lib/iov_iter.c | 6 ++++++
lib/usercopy.c | 6 +++++-
6 files changed, 66 insertions(+), 9 deletions(-)
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 61d93f062a36..ac4b26583f7c 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -6,6 +6,7 @@
*/
#include <linux/compiler.h>
#include <linux/kasan-checks.h>
+#include <linux/kmsan-checks.h>
#include <linux/string.h>
#include <asm/asm.h>
#include <asm/page.h>
@@ -174,6 +175,7 @@ __typeof__(__builtin_choose_expr(sizeof(x) > sizeof(0UL), 0ULL, 0UL))
ASM_CALL_CONSTRAINT \
: "0" (ptr), "i" (sizeof(*(ptr)))); \
(x) = (__force __typeof__(*(ptr))) __val_gu; \
+ kmsan_unpoison_shadow(&(x), sizeof(*(ptr))); \
__builtin_expect(__ret_gu, 0); \
})
@@ -248,6 +250,7 @@ extern void __put_user_8(void);
__chk_user_ptr(ptr); \
might_fault(); \
__pu_val = x; \
+ kmsan_check_memory(&(__pu_val), sizeof(*(ptr))); \
switch (sizeof(*(ptr))) { \
case 1: \
__put_user_x(1, __pu_val, ptr, __ret_pu); \
@@ -270,7 +273,9 @@ extern void __put_user_8(void);
#define __put_user_size(x, ptr, size, label) \
do { \
+ __typeof__(*(ptr)) __pus_val = x; \
__chk_user_ptr(ptr); \
+ kmsan_check_memory(&(__pus_val), size); \
switch (size) { \
case 1: \
__put_user_goto(x, ptr, "b", "b", "iq", label); \
@@ -295,7 +300,10 @@ do { \
*/
#define __put_user_size_ex(x, ptr, size) \
do { \
+ __typeof__(*(ptr)) __puse_val; \
__chk_user_ptr(ptr); \
+ __puse_val = x; \
+ kmsan_check_memory(&(__puse_val), size); \
switch (size) { \
case 1: \
__put_user_asm_ex(x, ptr, "b", "b", "iq"); \
@@ -363,6 +371,7 @@ do { \
default: \
(x) = __get_user_bad(); \
} \
+ kmsan_unpoison_shadow(&(x), size); \
} while (0)
#define __get_user_asm(x, addr, err, itype, rtype, ltype, errret) \
@@ -413,6 +422,7 @@ do { \
default: \
(x) = __get_user_bad(); \
} \
+ kmsan_unpoison_shadow(&(x), size); \
} while (0)
#define __get_user_asm_ex(x, addr, itype, rtype, ltype) \
@@ -428,11 +438,13 @@ do { \
#define __put_user_nocheck(x, ptr, size) \
({ \
__label__ __pu_label; \
+ __typeof__(*(ptr)) __pun_val = x; \
int __pu_err = -EFAULT; \
__typeof__(*(ptr)) __pu_val = (x); \
__typeof__(ptr) __pu_ptr = (ptr); \
__typeof__(size) __pu_size = (size); \
__uaccess_begin(); \
+ kmsan_check_memory(&(__pun_val), size); \
__put_user_size(__pu_val, __pu_ptr, __pu_size, __pu_label); \
__pu_err = 0; \
__pu_label: \
diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h
index a950a22c4890..707531dccf5e 100644
--- a/include/asm-generic/cacheflush.h
+++ b/include/asm-generic/cacheflush.h
@@ -4,6 +4,7 @@
/* Keep includes the same across arches. */
#include <linux/mm.h>
+#include <linux/kmsan-checks.h>
#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
@@ -72,10 +73,14 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
#define copy_to_user_page(vma, page, vaddr, dst, src, len) \
do { \
+ kmsan_check_memory(src, len); \
memcpy(dst, src, len); \
flush_icache_user_range(vma, page, vaddr, len); \
} while (0)
#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
- memcpy(dst, src, len)
+ do { \
+ memcpy(dst, src, len); \
+ kmsan_unpoison_shadow(dst, len); \
+ } while (0)
#endif /* __ASM_CACHEFLUSH_H */
diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h
index e935318804f8..508ee649aeef 100644
--- a/include/asm-generic/uaccess.h
+++ b/include/asm-generic/uaccess.h
@@ -142,7 +142,11 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
static inline int __put_user_fn(size_t size, void __user *ptr, void *x)
{
- return unlikely(raw_copy_to_user(ptr, x, size)) ? -EFAULT : 0;
+ int n;
+
+ n = raw_copy_to_user(ptr, x, size);
+ kmsan_copy_to_user(ptr, x, size, n);
+ return unlikely(n) ? -EFAULT : 0;
}
#define __put_user_fn(sz, u, k) __put_user_fn(sz, u, k)
@@ -203,7 +207,11 @@ extern int __put_user_bad(void) __attribute__((noreturn));
#ifndef __get_user_fn
static inline int __get_user_fn(size_t size, const void __user *ptr, void *x)
{
- return unlikely(raw_copy_from_user(x, ptr, size)) ? -EFAULT : 0;
+ int copied, to_copy = size;
+
+ copied = raw_copy_from_user(x, ptr, size);
+ kmsan_unpoison_shadow(x, to_copy - copied);
+ return unlikely(copied) ? -EFAULT : 0;
}
#define __get_user_fn(sz, u, k) __get_user_fn(sz, u, k)
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index d4ee6e942562..7550d11a8077 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -5,6 +5,7 @@
#include <linux/sched.h>
#include <linux/thread_info.h>
#include <linux/kasan-checks.h>
+#include <linux/kmsan-checks.h>
#define uaccess_kernel() segment_eq(get_fs(), KERNEL_DS)
@@ -58,18 +59,26 @@
static __always_inline __must_check unsigned long
__copy_from_user_inatomic(void *to, const void __user *from, unsigned long n)
{
+ unsigned long to_copy = n;
+
kasan_check_write(to, n);
check_object_size(to, n, false);
- return raw_copy_from_user(to, from, n);
+ n = raw_copy_from_user(to, from, n);
+ kmsan_unpoison_shadow(to, to_copy - n);
+ return n;
}
static __always_inline __must_check unsigned long
__copy_from_user(void *to, const void __user *from, unsigned long n)
{
+ unsigned long to_copy = n;
+
might_fault();
kasan_check_write(to, n);
check_object_size(to, n, false);
- return raw_copy_from_user(to, from, n);
+ n = raw_copy_from_user(to, from, n);
+ kmsan_unpoison_shadow(to, to_copy - n);
+ return n;
}
/**
@@ -88,29 +97,39 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
static __always_inline __must_check unsigned long
__copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
{
+ unsigned long to_copy = n;
+
kasan_check_read(from, n);
check_object_size(from, n, true);
- return raw_copy_to_user(to, from, n);
+ n = raw_copy_to_user(to, from, n);
+ kmsan_copy_to_user((const void *)to, from, to_copy, n);
+ return n;
}
static __always_inline __must_check unsigned long
__copy_to_user(void __user *to, const void *from, unsigned long n)
{
+ unsigned long to_copy = n;
+
might_fault();
kasan_check_read(from, n);
check_object_size(from, n, true);
- return raw_copy_to_user(to, from, n);
+ n = raw_copy_to_user(to, from, n);
+ kmsan_copy_to_user((const void *)to, from, to_copy, n);
+ return n;
}
#ifdef INLINE_COPY_FROM_USER
static inline __must_check unsigned long
_copy_from_user(void *to, const void __user *from, unsigned long n)
{
- unsigned long res = n;
+ unsigned long res = n, to_copy = n;
+
might_fault();
if (likely(access_ok(from, n))) {
kasan_check_write(to, n);
res = raw_copy_from_user(to, from, n);
+ kmsan_unpoison_shadow(to, to_copy - res);
}
if (unlikely(res))
memset(to + (n - res), 0, res);
@@ -125,10 +144,13 @@ _copy_from_user(void *, const void __user *, unsigned long);
static inline __must_check unsigned long
_copy_to_user(void __user *to, const void *from, unsigned long n)
{
+ unsigned long to_copy = n;
+
might_fault();
if (access_ok(to, n)) {
kasan_check_read(from, n);
n = raw_copy_to_user(to, from, n);
+ kmsan_copy_to_user(to, from, to_copy, n);
}
return n;
}
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 639d5e7014c1..f038676068b2 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -137,18 +137,24 @@
static int copyout(void __user *to, const void *from, size_t n)
{
+ size_t to_copy = n;
+
if (access_ok(to, n)) {
kasan_check_read(from, n);
n = raw_copy_to_user(to, from, n);
+ kmsan_copy_to_user(to, from, to_copy, n);
}
return n;
}
static int copyin(void *to, const void __user *from, size_t n)
{
+ size_t to_copy = n;
+
if (access_ok(from, n)) {
kasan_check_write(to, n);
n = raw_copy_from_user(to, from, n);
+ kmsan_unpoison_shadow(to, to_copy - n);
}
return n;
}
diff --git a/lib/usercopy.c b/lib/usercopy.c
index cbb4d9ec00f2..abfd93edecba 100644
--- a/lib/usercopy.c
+++ b/lib/usercopy.c
@@ -1,4 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
+#include <linux/kmsan-checks.h>
#include <linux/uaccess.h>
#include <linux/bitops.h>
@@ -7,11 +8,12 @@
#ifndef INLINE_COPY_FROM_USER
unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
{
- unsigned long res = n;
+ unsigned long res = n, to_copy = n;
might_fault();
if (likely(access_ok(from, n))) {
kasan_check_write(to, n);
res = raw_copy_from_user(to, from, n);
+ kmsan_unpoison_shadow(to, to_copy - res);
}
if (unlikely(res))
memset(to + (n - res), 0, res);
@@ -23,10 +25,12 @@ EXPORT_SYMBOL(_copy_from_user);
#ifndef INLINE_COPY_TO_USER
unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
{
+ unsigned long to_copy = n;
might_fault();
if (likely(access_ok(to, n))) {
kasan_check_read(from, n);
n = raw_copy_to_user(to, from, n);
+ kmsan_copy_to_user(to, from, to_copy, n);
}
return n;
}
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends
2019-11-22 11:26 ` [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends glider
@ 2019-11-29 15:34 ` Andrey Konovalov
2019-12-05 16:00 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 15:34 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Alexander Viro, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Andreas Dilger, Andrew Morton,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S . Tsirkin, Michal Simek, pmladek,
Qian Cai, Randy Dunlap, Robin Murphy, sergey.senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> Memory that is copied from userspace must be unpoisoned.
> Before copying memory to userspace, check it and report an error if it
> contains uninitialized bits.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
> v3:
> - fixed compilation errors reported by kbuild test bot
>
> Change-Id: I38428b9c7d1909b8441dcec1749b080494a7af99
> ---
> arch/x86/include/asm/uaccess.h | 12 ++++++++++++
> include/asm-generic/cacheflush.h | 7 ++++++-
> include/asm-generic/uaccess.h | 12 ++++++++++--
> include/linux/uaccess.h | 32 +++++++++++++++++++++++++++-----
> lib/iov_iter.c | 6 ++++++
> lib/usercopy.c | 6 +++++-
> 6 files changed, 66 insertions(+), 9 deletions(-)
>
> diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
> index 61d93f062a36..ac4b26583f7c 100644
> --- a/arch/x86/include/asm/uaccess.h
> +++ b/arch/x86/include/asm/uaccess.h
> @@ -6,6 +6,7 @@
> */
> #include <linux/compiler.h>
> #include <linux/kasan-checks.h>
> +#include <linux/kmsan-checks.h>
> #include <linux/string.h>
> #include <asm/asm.h>
> #include <asm/page.h>
> @@ -174,6 +175,7 @@ __typeof__(__builtin_choose_expr(sizeof(x) > sizeof(0UL), 0ULL, 0UL))
> ASM_CALL_CONSTRAINT \
> : "0" (ptr), "i" (sizeof(*(ptr)))); \
> (x) = (__force __typeof__(*(ptr))) __val_gu; \
> + kmsan_unpoison_shadow(&(x), sizeof(*(ptr))); \
> __builtin_expect(__ret_gu, 0); \
> })
>
> @@ -248,6 +250,7 @@ extern void __put_user_8(void);
> __chk_user_ptr(ptr); \
> might_fault(); \
> __pu_val = x; \
> + kmsan_check_memory(&(__pu_val), sizeof(*(ptr))); \
> switch (sizeof(*(ptr))) { \
> case 1: \
> __put_user_x(1, __pu_val, ptr, __ret_pu); \
> @@ -270,7 +273,9 @@ extern void __put_user_8(void);
>
> #define __put_user_size(x, ptr, size, label) \
> do { \
> + __typeof__(*(ptr)) __pus_val = x; \
> __chk_user_ptr(ptr); \
> + kmsan_check_memory(&(__pus_val), size); \
> switch (size) { \
> case 1: \
> __put_user_goto(x, ptr, "b", "b", "iq", label); \
> @@ -295,7 +300,10 @@ do { \
> */
> #define __put_user_size_ex(x, ptr, size) \
> do { \
> + __typeof__(*(ptr)) __puse_val; \
Can we do = x here?
> __chk_user_ptr(ptr); \
> + __puse_val = x; \
> + kmsan_check_memory(&(__puse_val), size); \
> switch (size) { \
> case 1: \
> __put_user_asm_ex(x, ptr, "b", "b", "iq"); \
> @@ -363,6 +371,7 @@ do { \
> default: \
> (x) = __get_user_bad(); \
> } \
> + kmsan_unpoison_shadow(&(x), size); \
> } while (0)
>
> #define __get_user_asm(x, addr, err, itype, rtype, ltype, errret) \
> @@ -413,6 +422,7 @@ do { \
> default: \
> (x) = __get_user_bad(); \
> } \
> + kmsan_unpoison_shadow(&(x), size); \
> } while (0)
>
> #define __get_user_asm_ex(x, addr, itype, rtype, ltype) \
> @@ -428,11 +438,13 @@ do { \
> #define __put_user_nocheck(x, ptr, size) \
> ({ \
> __label__ __pu_label; \
> + __typeof__(*(ptr)) __pun_val = x; \
Not sure if this matters, but two lines below do (x).
Also, why can't we use __pu_val instead of defining __pun_val?
> int __pu_err = -EFAULT; \
> __typeof__(*(ptr)) __pu_val = (x); \
> __typeof__(ptr) __pu_ptr = (ptr); \
> __typeof__(size) __pu_size = (size); \
> __uaccess_begin(); \
> + kmsan_check_memory(&(__pun_val), size); \
> __put_user_size(__pu_val, __pu_ptr, __pu_size, __pu_label); \
> __pu_err = 0; \
> __pu_label: \
> diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h
> index a950a22c4890..707531dccf5e 100644
> --- a/include/asm-generic/cacheflush.h
> +++ b/include/asm-generic/cacheflush.h
> @@ -4,6 +4,7 @@
>
> /* Keep includes the same across arches. */
> #include <linux/mm.h>
> +#include <linux/kmsan-checks.h>
>
> #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
>
> @@ -72,10 +73,14 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
>
> #define copy_to_user_page(vma, page, vaddr, dst, src, len) \
> do { \
> + kmsan_check_memory(src, len); \
> memcpy(dst, src, len); \
> flush_icache_user_range(vma, page, vaddr, len); \
> } while (0)
> #define copy_from_user_page(vma, page, vaddr, dst, src, len) \
> - memcpy(dst, src, len)
> + do { \
> + memcpy(dst, src, len); \
> + kmsan_unpoison_shadow(dst, len); \
> + } while (0)
>
> #endif /* __ASM_CACHEFLUSH_H */
> diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h
> index e935318804f8..508ee649aeef 100644
> --- a/include/asm-generic/uaccess.h
> +++ b/include/asm-generic/uaccess.h
> @@ -142,7 +142,11 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
>
> static inline int __put_user_fn(size_t size, void __user *ptr, void *x)
> {
> - return unlikely(raw_copy_to_user(ptr, x, size)) ? -EFAULT : 0;
> + int n;
> +
> + n = raw_copy_to_user(ptr, x, size);
> + kmsan_copy_to_user(ptr, x, size, n);
> + return unlikely(n) ? -EFAULT : 0;
> }
>
> #define __put_user_fn(sz, u, k) __put_user_fn(sz, u, k)
> @@ -203,7 +207,11 @@ extern int __put_user_bad(void) __attribute__((noreturn));
> #ifndef __get_user_fn
> static inline int __get_user_fn(size_t size, const void __user *ptr, void *x)
> {
> - return unlikely(raw_copy_from_user(x, ptr, size)) ? -EFAULT : 0;
> + int copied, to_copy = size;
> +
> + copied = raw_copy_from_user(x, ptr, size);
> + kmsan_unpoison_shadow(x, to_copy - copied);
> + return unlikely(copied) ? -EFAULT : 0;
> }
>
> #define __get_user_fn(sz, u, k) __get_user_fn(sz, u, k)
> diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
> index d4ee6e942562..7550d11a8077 100644
> --- a/include/linux/uaccess.h
> +++ b/include/linux/uaccess.h
> @@ -5,6 +5,7 @@
> #include <linux/sched.h>
> #include <linux/thread_info.h>
> #include <linux/kasan-checks.h>
> +#include <linux/kmsan-checks.h>
>
> #define uaccess_kernel() segment_eq(get_fs(), KERNEL_DS)
>
> @@ -58,18 +59,26 @@
> static __always_inline __must_check unsigned long
> __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n)
> {
> + unsigned long to_copy = n;
> +
> kasan_check_write(to, n);
> check_object_size(to, n, false);
> - return raw_copy_from_user(to, from, n);
> + n = raw_copy_from_user(to, from, n);
> + kmsan_unpoison_shadow(to, to_copy - n);
> + return n;
> }
>
> static __always_inline __must_check unsigned long
> __copy_from_user(void *to, const void __user *from, unsigned long n)
> {
> + unsigned long to_copy = n;
This is confusing. I think we need a var for raw_copy_from_user()
return value instead. Same in functions above and below.
> +
> might_fault();
> kasan_check_write(to, n);
> check_object_size(to, n, false);
> - return raw_copy_from_user(to, from, n);
> + n = raw_copy_from_user(to, from, n);
> + kmsan_unpoison_shadow(to, to_copy - n);
> + return n;
> }
>
> /**
> @@ -88,29 +97,39 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
> static __always_inline __must_check unsigned long
> __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
> {
> + unsigned long to_copy = n;
> +
> kasan_check_read(from, n);
> check_object_size(from, n, true);
> - return raw_copy_to_user(to, from, n);
> + n = raw_copy_to_user(to, from, n);
> + kmsan_copy_to_user((const void *)to, from, to_copy, n);
> + return n;
> }
>
> static __always_inline __must_check unsigned long
> __copy_to_user(void __user *to, const void *from, unsigned long n)
> {
> + unsigned long to_copy = n;
> +
> might_fault();
> kasan_check_read(from, n);
> check_object_size(from, n, true);
> - return raw_copy_to_user(to, from, n);
> + n = raw_copy_to_user(to, from, n);
> + kmsan_copy_to_user((const void *)to, from, to_copy, n);
> + return n;
> }
>
> #ifdef INLINE_COPY_FROM_USER
> static inline __must_check unsigned long
> _copy_from_user(void *to, const void __user *from, unsigned long n)
> {
> - unsigned long res = n;
> + unsigned long res = n, to_copy = n;
> +
> might_fault();
> if (likely(access_ok(from, n))) {
> kasan_check_write(to, n);
> res = raw_copy_from_user(to, from, n);
> + kmsan_unpoison_shadow(to, to_copy - res);
> }
> if (unlikely(res))
> memset(to + (n - res), 0, res);
> @@ -125,10 +144,13 @@ _copy_from_user(void *, const void __user *, unsigned long);
> static inline __must_check unsigned long
> _copy_to_user(void __user *to, const void *from, unsigned long n)
> {
> + unsigned long to_copy = n;
> +
> might_fault();
> if (access_ok(to, n)) {
> kasan_check_read(from, n);
> n = raw_copy_to_user(to, from, n);
> + kmsan_copy_to_user(to, from, to_copy, n);
> }
> return n;
> }
> diff --git a/lib/iov_iter.c b/lib/iov_iter.c
> index 639d5e7014c1..f038676068b2 100644
> --- a/lib/iov_iter.c
> +++ b/lib/iov_iter.c
> @@ -137,18 +137,24 @@
>
> static int copyout(void __user *to, const void *from, size_t n)
> {
> + size_t to_copy = n;
> +
> if (access_ok(to, n)) {
> kasan_check_read(from, n);
> n = raw_copy_to_user(to, from, n);
> + kmsan_copy_to_user(to, from, to_copy, n);
> }
> return n;
> }
>
> static int copyin(void *to, const void __user *from, size_t n)
> {
> + size_t to_copy = n;
> +
> if (access_ok(from, n)) {
> kasan_check_write(to, n);
> n = raw_copy_from_user(to, from, n);
> + kmsan_unpoison_shadow(to, to_copy - n);
> }
> return n;
> }
> diff --git a/lib/usercopy.c b/lib/usercopy.c
> index cbb4d9ec00f2..abfd93edecba 100644
> --- a/lib/usercopy.c
> +++ b/lib/usercopy.c
> @@ -1,4 +1,5 @@
> // SPDX-License-Identifier: GPL-2.0
> +#include <linux/kmsan-checks.h>
> #include <linux/uaccess.h>
> #include <linux/bitops.h>
>
> @@ -7,11 +8,12 @@
> #ifndef INLINE_COPY_FROM_USER
> unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
> {
> - unsigned long res = n;
> + unsigned long res = n, to_copy = n;
> might_fault();
> if (likely(access_ok(from, n))) {
> kasan_check_write(to, n);
> res = raw_copy_from_user(to, from, n);
> + kmsan_unpoison_shadow(to, to_copy - res);
> }
> if (unlikely(res))
> memset(to + (n - res), 0, res);
> @@ -23,10 +25,12 @@ EXPORT_SYMBOL(_copy_from_user);
> #ifndef INLINE_COPY_TO_USER
> unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
> {
> + unsigned long to_copy = n;
> might_fault();
> if (likely(access_ok(to, n))) {
> kasan_check_read(from, n);
> n = raw_copy_to_user(to, from, n);
> + kmsan_copy_to_user(to, from, to_copy, n);
> }
> return n;
> }
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends
2019-11-29 15:34 ` Andrey Konovalov
@ 2019-12-05 16:00 ` Alexander Potapenko
2019-12-05 16:44 ` Andrey Konovalov
0 siblings, 1 reply; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 16:00 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Alexander Viro, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Andreas Dilger, Andrew Morton,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Fri, Nov 29, 2019 at 4:34 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > Memory that is copied from userspace must be unpoisoned.
> > Before copying memory to userspace, check it and report an error if it
> > contains uninitialized bits.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> > v3:
> > - fixed compilation errors reported by kbuild test bot
> >
> > Change-Id: I38428b9c7d1909b8441dcec1749b080494a7af99
> > ---
> > arch/x86/include/asm/uaccess.h | 12 ++++++++++++
> > include/asm-generic/cacheflush.h | 7 ++++++-
> > include/asm-generic/uaccess.h | 12 ++++++++++--
> > include/linux/uaccess.h | 32 +++++++++++++++++++++++++++-----
> > lib/iov_iter.c | 6 ++++++
> > lib/usercopy.c | 6 +++++-
> > 6 files changed, 66 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
> > index 61d93f062a36..ac4b26583f7c 100644
> > --- a/arch/x86/include/asm/uaccess.h
> > +++ b/arch/x86/include/asm/uaccess.h
> > @@ -6,6 +6,7 @@
> > */
> > #include <linux/compiler.h>
> > #include <linux/kasan-checks.h>
> > +#include <linux/kmsan-checks.h>
> > #include <linux/string.h>
> > #include <asm/asm.h>
> > #include <asm/page.h>
> > @@ -174,6 +175,7 @@ __typeof__(__builtin_choose_expr(sizeof(x) > sizeof(0UL), 0ULL, 0UL))
> > ASM_CALL_CONSTRAINT \
> > : "0" (ptr), "i" (sizeof(*(ptr)))); \
> > (x) = (__force __typeof__(*(ptr))) __val_gu; \
> > + kmsan_unpoison_shadow(&(x), sizeof(*(ptr))); \
> > __builtin_expect(__ret_gu, 0); \
> > })
> >
> > @@ -248,6 +250,7 @@ extern void __put_user_8(void);
> > __chk_user_ptr(ptr); \
> > might_fault(); \
> > __pu_val = x; \
> > + kmsan_check_memory(&(__pu_val), sizeof(*(ptr))); \
> > switch (sizeof(*(ptr))) { \
> > case 1: \
> > __put_user_x(1, __pu_val, ptr, __ret_pu); \
> > @@ -270,7 +273,9 @@ extern void __put_user_8(void);
> >
> > #define __put_user_size(x, ptr, size, label) \
> > do { \
> > + __typeof__(*(ptr)) __pus_val = x; \
> > __chk_user_ptr(ptr); \
> > + kmsan_check_memory(&(__pus_val), size); \
> > switch (size) { \
> > case 1: \
> > __put_user_goto(x, ptr, "b", "b", "iq", label); \
> > @@ -295,7 +300,10 @@ do { \
> > */
> > #define __put_user_size_ex(x, ptr, size) \
> > do { \
> > + __typeof__(*(ptr)) __puse_val; \
>
> Can we do = x here?
Yes. Fixed, thanks!
> > __chk_user_ptr(ptr); \
> > + __puse_val = x; \
> > + kmsan_check_memory(&(__puse_val), size); \
> > switch (size) { \
> > case 1: \
> > __put_user_asm_ex(x, ptr, "b", "b", "iq"); \
> > @@ -363,6 +371,7 @@ do { \
> > default: \
> > (x) = __get_user_bad(); \
> > } \
> > + kmsan_unpoison_shadow(&(x), size); \
> > } while (0)
> >
> > #define __get_user_asm(x, addr, err, itype, rtype, ltype, errret) \
> > @@ -413,6 +422,7 @@ do { \
> > default: \
> > (x) = __get_user_bad(); \
> > } \
> > + kmsan_unpoison_shadow(&(x), size); \
> > } while (0)
> >
> > #define __get_user_asm_ex(x, addr, itype, rtype, ltype) \
> > @@ -428,11 +438,13 @@ do { \
> > #define __put_user_nocheck(x, ptr, size) \
> > ({ \
> > __label__ __pu_label; \
> > + __typeof__(*(ptr)) __pun_val = x; \
>
> Not sure if this matters, but two lines below do (x).
Right.
> Also, why can't we use __pu_val instead of defining __pun_val?
Will do.
>
> > int __pu_err = -EFAULT; \
> > __typeof__(*(ptr)) __pu_val = (x); \
> > __typeof__(ptr) __pu_ptr = (ptr); \
> > __typeof__(size) __pu_size = (size); \
> > __uaccess_begin(); \
> > + kmsan_check_memory(&(__pun_val), size); \
> > __put_user_size(__pu_val, __pu_ptr, __pu_size, __pu_label); \
> > __pu_err = 0; \
> > __pu_label: \
> > diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h
> > index a950a22c4890..707531dccf5e 100644
> > --- a/include/asm-generic/cacheflush.h
> > +++ b/include/asm-generic/cacheflush.h
> > @@ -4,6 +4,7 @@
> >
> > /* Keep includes the same across arches. */
> > #include <linux/mm.h>
> > +#include <linux/kmsan-checks.h>
> >
> > #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
> >
> > @@ -72,10 +73,14 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
> >
> > #define copy_to_user_page(vma, page, vaddr, dst, src, len) \
> > do { \
> > + kmsan_check_memory(src, len); \
> > memcpy(dst, src, len); \
> > flush_icache_user_range(vma, page, vaddr, len); \
> > } while (0)
> > #define copy_from_user_page(vma, page, vaddr, dst, src, len) \
> > - memcpy(dst, src, len)
> > + do { \
> > + memcpy(dst, src, len); \
> > + kmsan_unpoison_shadow(dst, len); \
> > + } while (0)
> >
> > #endif /* __ASM_CACHEFLUSH_H */
> > diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h
> > index e935318804f8..508ee649aeef 100644
> > --- a/include/asm-generic/uaccess.h
> > +++ b/include/asm-generic/uaccess.h
> > @@ -142,7 +142,11 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
> >
> > static inline int __put_user_fn(size_t size, void __user *ptr, void *x)
> > {
> > - return unlikely(raw_copy_to_user(ptr, x, size)) ? -EFAULT : 0;
> > + int n;
> > +
> > + n = raw_copy_to_user(ptr, x, size);
> > + kmsan_copy_to_user(ptr, x, size, n);
> > + return unlikely(n) ? -EFAULT : 0;
> > }
> >
> > #define __put_user_fn(sz, u, k) __put_user_fn(sz, u, k)
> > @@ -203,7 +207,11 @@ extern int __put_user_bad(void) __attribute__((noreturn));
> > #ifndef __get_user_fn
> > static inline int __get_user_fn(size_t size, const void __user *ptr, void *x)
> > {
> > - return unlikely(raw_copy_from_user(x, ptr, size)) ? -EFAULT : 0;
> > + int copied, to_copy = size;
> > +
> > + copied = raw_copy_from_user(x, ptr, size);
> > + kmsan_unpoison_shadow(x, to_copy - copied);
> > + return unlikely(copied) ? -EFAULT : 0;
> > }
> >
> > #define __get_user_fn(sz, u, k) __get_user_fn(sz, u, k)
> > diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
> > index d4ee6e942562..7550d11a8077 100644
> > --- a/include/linux/uaccess.h
> > +++ b/include/linux/uaccess.h
> > @@ -5,6 +5,7 @@
> > #include <linux/sched.h>
> > #include <linux/thread_info.h>
> > #include <linux/kasan-checks.h>
> > +#include <linux/kmsan-checks.h>
> >
> > #define uaccess_kernel() segment_eq(get_fs(), KERNEL_DS)
> >
> > @@ -58,18 +59,26 @@
> > static __always_inline __must_check unsigned long
> > __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n)
> > {
> > + unsigned long to_copy = n;
> > +
> > kasan_check_write(to, n);
> > check_object_size(to, n, false);
> > - return raw_copy_from_user(to, from, n);
> > + n = raw_copy_from_user(to, from, n);
> > + kmsan_unpoison_shadow(to, to_copy - n);
> > + return n;
> > }
> >
> > static __always_inline __must_check unsigned long
> > __copy_from_user(void *to, const void __user *from, unsigned long n)
> > {
> > + unsigned long to_copy = n;
>
> This is confusing. I think we need a var for raw_copy_from_user()
> return value instead. Same in functions above and below.
raw_copy_from_user() returns the number of bytes _not_ copied from
userspace. So in the case it returns 0 we need to unpoison to_copy
bytes.
> > +
> > might_fault();
> > kasan_check_write(to, n);
> > check_object_size(to, n, false);
> > - return raw_copy_from_user(to, from, n);
> > + n = raw_copy_from_user(to, from, n);
> > + kmsan_unpoison_shadow(to, to_copy - n);
> > + return n;
> > }
> >
> > /**
> > @@ -88,29 +97,39 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
> > static __always_inline __must_check unsigned long
> > __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
> > {
> > + unsigned long to_copy = n;
> > +
> > kasan_check_read(from, n);
> > check_object_size(from, n, true);
> > - return raw_copy_to_user(to, from, n);
> > + n = raw_copy_to_user(to, from, n);
> > + kmsan_copy_to_user((const void *)to, from, to_copy, n);
> > + return n;
> > }
> >
> > static __always_inline __must_check unsigned long
> > __copy_to_user(void __user *to, const void *from, unsigned long n)
> > {
> > + unsigned long to_copy = n;
> > +
> > might_fault();
> > kasan_check_read(from, n);
> > check_object_size(from, n, true);
> > - return raw_copy_to_user(to, from, n);
> > + n = raw_copy_to_user(to, from, n);
> > + kmsan_copy_to_user((const void *)to, from, to_copy, n);
> > + return n;
> > }
> >
> > #ifdef INLINE_COPY_FROM_USER
> > static inline __must_check unsigned long
> > _copy_from_user(void *to, const void __user *from, unsigned long n)
> > {
> > - unsigned long res = n;
> > + unsigned long res = n, to_copy = n;
> > +
> > might_fault();
> > if (likely(access_ok(from, n))) {
> > kasan_check_write(to, n);
> > res = raw_copy_from_user(to, from, n);
> > + kmsan_unpoison_shadow(to, to_copy - res);
> > }
> > if (unlikely(res))
> > memset(to + (n - res), 0, res);
> > @@ -125,10 +144,13 @@ _copy_from_user(void *, const void __user *, unsigned long);
> > static inline __must_check unsigned long
> > _copy_to_user(void __user *to, const void *from, unsigned long n)
> > {
> > + unsigned long to_copy = n;
> > +
> > might_fault();
> > if (access_ok(to, n)) {
> > kasan_check_read(from, n);
> > n = raw_copy_to_user(to, from, n);
> > + kmsan_copy_to_user(to, from, to_copy, n);
> > }
> > return n;
> > }
> > diff --git a/lib/iov_iter.c b/lib/iov_iter.c
> > index 639d5e7014c1..f038676068b2 100644
> > --- a/lib/iov_iter.c
> > +++ b/lib/iov_iter.c
> > @@ -137,18 +137,24 @@
> >
> > static int copyout(void __user *to, const void *from, size_t n)
> > {
> > + size_t to_copy = n;
> > +
> > if (access_ok(to, n)) {
> > kasan_check_read(from, n);
> > n = raw_copy_to_user(to, from, n);
> > + kmsan_copy_to_user(to, from, to_copy, n);
> > }
> > return n;
> > }
> >
> > static int copyin(void *to, const void __user *from, size_t n)
> > {
> > + size_t to_copy = n;
> > +
> > if (access_ok(from, n)) {
> > kasan_check_write(to, n);
> > n = raw_copy_from_user(to, from, n);
> > + kmsan_unpoison_shadow(to, to_copy - n);
> > }
> > return n;
> > }
> > diff --git a/lib/usercopy.c b/lib/usercopy.c
> > index cbb4d9ec00f2..abfd93edecba 100644
> > --- a/lib/usercopy.c
> > +++ b/lib/usercopy.c
> > @@ -1,4 +1,5 @@
> > // SPDX-License-Identifier: GPL-2.0
> > +#include <linux/kmsan-checks.h>
> > #include <linux/uaccess.h>
> > #include <linux/bitops.h>
> >
> > @@ -7,11 +8,12 @@
> > #ifndef INLINE_COPY_FROM_USER
> > unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
> > {
> > - unsigned long res = n;
> > + unsigned long res = n, to_copy = n;
> > might_fault();
> > if (likely(access_ok(from, n))) {
> > kasan_check_write(to, n);
> > res = raw_copy_from_user(to, from, n);
> > + kmsan_unpoison_shadow(to, to_copy - res);
> > }
> > if (unlikely(res))
> > memset(to + (n - res), 0, res);
> > @@ -23,10 +25,12 @@ EXPORT_SYMBOL(_copy_from_user);
> > #ifndef INLINE_COPY_TO_USER
> > unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
> > {
> > + unsigned long to_copy = n;
> > might_fault();
> > if (likely(access_ok(to, n))) {
> > kasan_check_read(from, n);
> > n = raw_copy_to_user(to, from, n);
> > + kmsan_copy_to_user(to, from, to_copy, n);
> > }
> > return n;
> > }
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends
2019-12-05 16:00 ` Alexander Potapenko
@ 2019-12-05 16:44 ` Andrey Konovalov
2019-12-11 14:22 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-12-05 16:44 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Alexander Viro, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Andreas Dilger, Andrew Morton,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Thu, Dec 5, 2019 at 5:00 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Fri, Nov 29, 2019 at 4:34 PM Andrey Konovalov <andreyknvl@google.com> wrote:
> >
> > On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> > >
> > > Memory that is copied from userspace must be unpoisoned.
> > > Before copying memory to userspace, check it and report an error if it
> > > contains uninitialized bits.
> > >
> > > Signed-off-by: Alexander Potapenko <glider@google.com>
> > > To: Alexander Potapenko <glider@google.com>
> > > Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> > > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > > Cc: Dmitry Vyukov <dvyukov@google.com>
> > > Cc: linux-mm@kvack.org
> > > ---
> > > v3:
> > > - fixed compilation errors reported by kbuild test bot
> > >
> > > Change-Id: I38428b9c7d1909b8441dcec1749b080494a7af99
> > > ---
> > > arch/x86/include/asm/uaccess.h | 12 ++++++++++++
> > > include/asm-generic/cacheflush.h | 7 ++++++-
> > > include/asm-generic/uaccess.h | 12 ++++++++++--
> > > include/linux/uaccess.h | 32 +++++++++++++++++++++++++++-----
> > > lib/iov_iter.c | 6 ++++++
> > > lib/usercopy.c | 6 +++++-
> > > 6 files changed, 66 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
> > > index 61d93f062a36..ac4b26583f7c 100644
> > > --- a/arch/x86/include/asm/uaccess.h
> > > +++ b/arch/x86/include/asm/uaccess.h
> > > @@ -6,6 +6,7 @@
> > > */
> > > #include <linux/compiler.h>
> > > #include <linux/kasan-checks.h>
> > > +#include <linux/kmsan-checks.h>
> > > #include <linux/string.h>
> > > #include <asm/asm.h>
> > > #include <asm/page.h>
> > > @@ -174,6 +175,7 @@ __typeof__(__builtin_choose_expr(sizeof(x) > sizeof(0UL), 0ULL, 0UL))
> > > ASM_CALL_CONSTRAINT \
> > > : "0" (ptr), "i" (sizeof(*(ptr)))); \
> > > (x) = (__force __typeof__(*(ptr))) __val_gu; \
> > > + kmsan_unpoison_shadow(&(x), sizeof(*(ptr))); \
> > > __builtin_expect(__ret_gu, 0); \
> > > })
> > >
> > > @@ -248,6 +250,7 @@ extern void __put_user_8(void);
> > > __chk_user_ptr(ptr); \
> > > might_fault(); \
> > > __pu_val = x; \
> > > + kmsan_check_memory(&(__pu_val), sizeof(*(ptr))); \
> > > switch (sizeof(*(ptr))) { \
> > > case 1: \
> > > __put_user_x(1, __pu_val, ptr, __ret_pu); \
> > > @@ -270,7 +273,9 @@ extern void __put_user_8(void);
> > >
> > > #define __put_user_size(x, ptr, size, label) \
> > > do { \
> > > + __typeof__(*(ptr)) __pus_val = x; \
> > > __chk_user_ptr(ptr); \
> > > + kmsan_check_memory(&(__pus_val), size); \
> > > switch (size) { \
> > > case 1: \
> > > __put_user_goto(x, ptr, "b", "b", "iq", label); \
> > > @@ -295,7 +300,10 @@ do { \
> > > */
> > > #define __put_user_size_ex(x, ptr, size) \
> > > do { \
> > > + __typeof__(*(ptr)) __puse_val; \
> >
> > Can we do = x here?
> Yes. Fixed, thanks!
> > > __chk_user_ptr(ptr); \
> > > + __puse_val = x; \
> > > + kmsan_check_memory(&(__puse_val), size); \
> > > switch (size) { \
> > > case 1: \
> > > __put_user_asm_ex(x, ptr, "b", "b", "iq"); \
> > > @@ -363,6 +371,7 @@ do { \
> > > default: \
> > > (x) = __get_user_bad(); \
> > > } \
> > > + kmsan_unpoison_shadow(&(x), size); \
> > > } while (0)
> > >
> > > #define __get_user_asm(x, addr, err, itype, rtype, ltype, errret) \
> > > @@ -413,6 +422,7 @@ do { \
> > > default: \
> > > (x) = __get_user_bad(); \
> > > } \
> > > + kmsan_unpoison_shadow(&(x), size); \
> > > } while (0)
> > >
> > > #define __get_user_asm_ex(x, addr, itype, rtype, ltype) \
> > > @@ -428,11 +438,13 @@ do { \
> > > #define __put_user_nocheck(x, ptr, size) \
> > > ({ \
> > > __label__ __pu_label; \
> > > + __typeof__(*(ptr)) __pun_val = x; \
> >
> > Not sure if this matters, but two lines below do (x).
> Right.
> > Also, why can't we use __pu_val instead of defining __pun_val?
> Will do.
> >
> > > int __pu_err = -EFAULT; \
> > > __typeof__(*(ptr)) __pu_val = (x); \
> > > __typeof__(ptr) __pu_ptr = (ptr); \
> > > __typeof__(size) __pu_size = (size); \
> > > __uaccess_begin(); \
> > > + kmsan_check_memory(&(__pun_val), size); \
> > > __put_user_size(__pu_val, __pu_ptr, __pu_size, __pu_label); \
> > > __pu_err = 0; \
> > > __pu_label: \
> > > diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h
> > > index a950a22c4890..707531dccf5e 100644
> > > --- a/include/asm-generic/cacheflush.h
> > > +++ b/include/asm-generic/cacheflush.h
> > > @@ -4,6 +4,7 @@
> > >
> > > /* Keep includes the same across arches. */
> > > #include <linux/mm.h>
> > > +#include <linux/kmsan-checks.h>
> > >
> > > #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
> > >
> > > @@ -72,10 +73,14 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
> > >
> > > #define copy_to_user_page(vma, page, vaddr, dst, src, len) \
> > > do { \
> > > + kmsan_check_memory(src, len); \
> > > memcpy(dst, src, len); \
> > > flush_icache_user_range(vma, page, vaddr, len); \
> > > } while (0)
> > > #define copy_from_user_page(vma, page, vaddr, dst, src, len) \
> > > - memcpy(dst, src, len)
> > > + do { \
> > > + memcpy(dst, src, len); \
> > > + kmsan_unpoison_shadow(dst, len); \
> > > + } while (0)
> > >
> > > #endif /* __ASM_CACHEFLUSH_H */
> > > diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h
> > > index e935318804f8..508ee649aeef 100644
> > > --- a/include/asm-generic/uaccess.h
> > > +++ b/include/asm-generic/uaccess.h
> > > @@ -142,7 +142,11 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
> > >
> > > static inline int __put_user_fn(size_t size, void __user *ptr, void *x)
> > > {
> > > - return unlikely(raw_copy_to_user(ptr, x, size)) ? -EFAULT : 0;
> > > + int n;
> > > +
> > > + n = raw_copy_to_user(ptr, x, size);
> > > + kmsan_copy_to_user(ptr, x, size, n);
> > > + return unlikely(n) ? -EFAULT : 0;
> > > }
> > >
> > > #define __put_user_fn(sz, u, k) __put_user_fn(sz, u, k)
> > > @@ -203,7 +207,11 @@ extern int __put_user_bad(void) __attribute__((noreturn));
> > > #ifndef __get_user_fn
> > > static inline int __get_user_fn(size_t size, const void __user *ptr, void *x)
> > > {
> > > - return unlikely(raw_copy_from_user(x, ptr, size)) ? -EFAULT : 0;
> > > + int copied, to_copy = size;
> > > +
> > > + copied = raw_copy_from_user(x, ptr, size);
> > > + kmsan_unpoison_shadow(x, to_copy - copied);
> > > + return unlikely(copied) ? -EFAULT : 0;
> > > }
> > >
> > > #define __get_user_fn(sz, u, k) __get_user_fn(sz, u, k)
> > > diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
> > > index d4ee6e942562..7550d11a8077 100644
> > > --- a/include/linux/uaccess.h
> > > +++ b/include/linux/uaccess.h
> > > @@ -5,6 +5,7 @@
> > > #include <linux/sched.h>
> > > #include <linux/thread_info.h>
> > > #include <linux/kasan-checks.h>
> > > +#include <linux/kmsan-checks.h>
> > >
> > > #define uaccess_kernel() segment_eq(get_fs(), KERNEL_DS)
> > >
> > > @@ -58,18 +59,26 @@
> > > static __always_inline __must_check unsigned long
> > > __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n)
> > > {
> > > + unsigned long to_copy = n;
> > > +
> > > kasan_check_write(to, n);
> > > check_object_size(to, n, false);
> > > - return raw_copy_from_user(to, from, n);
> > > + n = raw_copy_from_user(to, from, n);
> > > + kmsan_unpoison_shadow(to, to_copy - n);
> > > + return n;
> > > }
> > >
> > > static __always_inline __must_check unsigned long
> > > __copy_from_user(void *to, const void __user *from, unsigned long n)
> > > {
> > > + unsigned long to_copy = n;
> >
> > This is confusing. I think we need a var for raw_copy_from_user()
> > return value instead. Same in functions above and below.
> raw_copy_from_user() returns the number of bytes _not_ copied from
> userspace. So in the case it returns 0 we need to unpoison to_copy
> bytes.
My point was that reasiggning the value of n to the return value of
raw_copy_from_user(), but saving the original n into to_copy is
confusing. It would be easier to read this code, is n was left as is,
and there was a new var for raw_copy_from_user() return value.
> > > +
> > > might_fault();
> > > kasan_check_write(to, n);
> > > check_object_size(to, n, false);
> > > - return raw_copy_from_user(to, from, n);
> > > + n = raw_copy_from_user(to, from, n);
> > > + kmsan_unpoison_shadow(to, to_copy - n);
> > > + return n;
> > > }
> > >
> > > /**
> > > @@ -88,29 +97,39 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
> > > static __always_inline __must_check unsigned long
> > > __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
> > > {
> > > + unsigned long to_copy = n;
> > > +
> > > kasan_check_read(from, n);
> > > check_object_size(from, n, true);
> > > - return raw_copy_to_user(to, from, n);
> > > + n = raw_copy_to_user(to, from, n);
> > > + kmsan_copy_to_user((const void *)to, from, to_copy, n);
> > > + return n;
> > > }
> > >
> > > static __always_inline __must_check unsigned long
> > > __copy_to_user(void __user *to, const void *from, unsigned long n)
> > > {
> > > + unsigned long to_copy = n;
> > > +
> > > might_fault();
> > > kasan_check_read(from, n);
> > > check_object_size(from, n, true);
> > > - return raw_copy_to_user(to, from, n);
> > > + n = raw_copy_to_user(to, from, n);
> > > + kmsan_copy_to_user((const void *)to, from, to_copy, n);
> > > + return n;
> > > }
> > >
> > > #ifdef INLINE_COPY_FROM_USER
> > > static inline __must_check unsigned long
> > > _copy_from_user(void *to, const void __user *from, unsigned long n)
> > > {
> > > - unsigned long res = n;
> > > + unsigned long res = n, to_copy = n;
> > > +
> > > might_fault();
> > > if (likely(access_ok(from, n))) {
> > > kasan_check_write(to, n);
> > > res = raw_copy_from_user(to, from, n);
> > > + kmsan_unpoison_shadow(to, to_copy - res);
> > > }
> > > if (unlikely(res))
> > > memset(to + (n - res), 0, res);
> > > @@ -125,10 +144,13 @@ _copy_from_user(void *, const void __user *, unsigned long);
> > > static inline __must_check unsigned long
> > > _copy_to_user(void __user *to, const void *from, unsigned long n)
> > > {
> > > + unsigned long to_copy = n;
> > > +
> > > might_fault();
> > > if (access_ok(to, n)) {
> > > kasan_check_read(from, n);
> > > n = raw_copy_to_user(to, from, n);
> > > + kmsan_copy_to_user(to, from, to_copy, n);
> > > }
> > > return n;
> > > }
> > > diff --git a/lib/iov_iter.c b/lib/iov_iter.c
> > > index 639d5e7014c1..f038676068b2 100644
> > > --- a/lib/iov_iter.c
> > > +++ b/lib/iov_iter.c
> > > @@ -137,18 +137,24 @@
> > >
> > > static int copyout(void __user *to, const void *from, size_t n)
> > > {
> > > + size_t to_copy = n;
> > > +
> > > if (access_ok(to, n)) {
> > > kasan_check_read(from, n);
> > > n = raw_copy_to_user(to, from, n);
> > > + kmsan_copy_to_user(to, from, to_copy, n);
> > > }
> > > return n;
> > > }
> > >
> > > static int copyin(void *to, const void __user *from, size_t n)
> > > {
> > > + size_t to_copy = n;
> > > +
> > > if (access_ok(from, n)) {
> > > kasan_check_write(to, n);
> > > n = raw_copy_from_user(to, from, n);
> > > + kmsan_unpoison_shadow(to, to_copy - n);
> > > }
> > > return n;
> > > }
> > > diff --git a/lib/usercopy.c b/lib/usercopy.c
> > > index cbb4d9ec00f2..abfd93edecba 100644
> > > --- a/lib/usercopy.c
> > > +++ b/lib/usercopy.c
> > > @@ -1,4 +1,5 @@
> > > // SPDX-License-Identifier: GPL-2.0
> > > +#include <linux/kmsan-checks.h>
> > > #include <linux/uaccess.h>
> > > #include <linux/bitops.h>
> > >
> > > @@ -7,11 +8,12 @@
> > > #ifndef INLINE_COPY_FROM_USER
> > > unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
> > > {
> > > - unsigned long res = n;
> > > + unsigned long res = n, to_copy = n;
> > > might_fault();
> > > if (likely(access_ok(from, n))) {
> > > kasan_check_write(to, n);
> > > res = raw_copy_from_user(to, from, n);
> > > + kmsan_unpoison_shadow(to, to_copy - res);
> > > }
> > > if (unlikely(res))
> > > memset(to + (n - res), 0, res);
> > > @@ -23,10 +25,12 @@ EXPORT_SYMBOL(_copy_from_user);
> > > #ifndef INLINE_COPY_TO_USER
> > > unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
> > > {
> > > + unsigned long to_copy = n;
> > > might_fault();
> > > if (likely(access_ok(to, n))) {
> > > kasan_check_read(from, n);
> > > n = raw_copy_to_user(to, from, n);
> > > + kmsan_copy_to_user(to, from, to_copy, n);
> > > }
> > > return n;
> > > }
> > > --
> > > 2.24.0.432.g9d3f5f5b63-goog
> > >
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends
2019-12-05 16:44 ` Andrey Konovalov
@ 2019-12-11 14:22 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-11 14:22 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Alexander Viro, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Andreas Dilger, Andrew Morton,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
> My point was that reasiggning the value of n to the return value of
> raw_copy_from_user(), but saving the original n into to_copy is
> confusing. It would be easier to read this code, is n was left as is,
> and there was a new var for raw_copy_from_user() return value.
>
Ok, will do in v4.
Thanks!
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 28/36] kmsan: enable KMSAN builds
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (25 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 27/36] kmsan: hooks for copy_to_user() and friends glider
@ 2019-11-22 11:26 ` glider
2019-11-29 15:55 ` Andrey Konovalov
2019-11-22 11:26 ` [PATCH RFC v3 29/36] kmsan: handle /dev/[u]random glider
` (8 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, m.szyprowski,
elver, mark.rutland, martin.petersen, schwidefsky, willy, mst,
monstr, pmladek, cai, rdunlap, robin.murphy, sergey.senozhatsky,
rostedt, tiwai, tytso, tglx, gor, wsa
Make KMSAN usable by adding the necessary Makefile bits and calling
KMSAN initialization routines (kmsan_initialize_shadow() and
kmsan_initialize()) from init/main.c
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
This patch was previously called "kmsan: Changing existing files to
enable KMSAN builds". Logically unrelated parts of it were split away.
Change-Id: I37e0b7f2d2f2b0aeac5753ff9d6b411485fc374e
---
Makefile | 3 ++-
init/main.c | 3 +++
mm/Makefile | 1 +
scripts/Makefile.lib | 6 ++++++
4 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 79be70bf2899..181ae2dac415 100644
--- a/Makefile
+++ b/Makefile
@@ -478,7 +478,7 @@ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS
export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
-export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN
+export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KMSAN
export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE
export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
@@ -898,6 +898,7 @@ KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none)
endif
include scripts/Makefile.kasan
+include scripts/Makefile.kmsan
include scripts/Makefile.extrawarn
include scripts/Makefile.ubsan
diff --git a/init/main.c b/init/main.c
index 91f6ebb30ef0..afcca2a38348 100644
--- a/init/main.c
+++ b/init/main.c
@@ -32,6 +32,7 @@
#include <linux/nmi.h>
#include <linux/percpu.h>
#include <linux/kmod.h>
+#include <linux/kmsan.h>
#include <linux/vmalloc.h>
#include <linux/kernel_stat.h>
#include <linux/start_kernel.h>
@@ -554,6 +555,7 @@ static void __init mm_init(void)
*/
page_ext_init_flatmem();
report_meminit();
+ kmsan_initialize_shadow();
mem_init();
kmem_cache_init();
kmemleak_init();
@@ -625,6 +627,7 @@ asmlinkage __visible void __init start_kernel(void)
sort_main_extable();
trap_init();
mm_init();
+ kmsan_initialize();
ftrace_init();
diff --git a/mm/Makefile b/mm/Makefile
index d996846697ef..419e6e02dfaf 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -71,6 +71,7 @@ obj-$(CONFIG_PAGE_POISONING) += page_poison.o
obj-$(CONFIG_SLAB) += slab.o
obj-$(CONFIG_SLUB) += slub.o
obj-$(CONFIG_KASAN) += kasan/
+obj-$(CONFIG_KMSAN) += kmsan/
obj-$(CONFIG_FAILSLAB) += failslab.o
obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
obj-$(CONFIG_MEMTEST) += memtest.o
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 179d55af5852..f9f38f7c1cd4 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -140,6 +140,12 @@ _c_flags += $(if $(patsubst n%,, \
$(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE))
endif
+ifeq ($(CONFIG_KMSAN),y)
+_c_flags += $(if $(patsubst n%,, \
+ $(KMSAN_SANITIZE_$(basetarget).o)$(KMSAN_SANITIZE)y), \
+ $(CFLAGS_KMSAN))
+endif
+
ifeq ($(CONFIG_UBSAN),y)
_c_flags += $(if $(patsubst n%,, \
$(UBSAN_SANITIZE_$(basetarget).o)$(UBSAN_SANITIZE)$(CONFIG_UBSAN_SANITIZE_ALL)), \
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 28/36] kmsan: enable KMSAN builds
2019-11-22 11:26 ` [PATCH RFC v3 28/36] kmsan: enable KMSAN builds glider
@ 2019-11-29 15:55 ` Andrey Konovalov
2019-12-11 12:51 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Andrey Konovalov @ 2019-11-29 15:55 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, darrick.wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
ericvh, Greg Kroah-Hartman, harry.wentland, Herbert Xu, iii,
mingo, Jason Wang, Marek Szyprowski, Marco Elver, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S . Tsirkin, Michal Simek, pmladek, Qian Cai,
Randy Dunlap, Robin Murphy, sergey.senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, gor, wsa
On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
>
> Make KMSAN usable by adding the necessary Makefile bits and calling
> KMSAN initialization routines (kmsan_initialize_shadow() and
> kmsan_initialize()) from init/main.c
I think it makes sense to split these two parts into separate patches as well.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
> This patch was previously called "kmsan: Changing existing files to
> enable KMSAN builds". Logically unrelated parts of it were split away.
>
> Change-Id: I37e0b7f2d2f2b0aeac5753ff9d6b411485fc374e
> ---
> Makefile | 3 ++-
> init/main.c | 3 +++
> mm/Makefile | 1 +
> scripts/Makefile.lib | 6 ++++++
> 4 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 79be70bf2899..181ae2dac415 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -478,7 +478,7 @@ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
>
> export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS
> export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
> -export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN
> +export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KMSAN
> export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE
> export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
> export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
> @@ -898,6 +898,7 @@ KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none)
> endif
>
> include scripts/Makefile.kasan
> +include scripts/Makefile.kmsan
> include scripts/Makefile.extrawarn
> include scripts/Makefile.ubsan
>
> diff --git a/init/main.c b/init/main.c
> index 91f6ebb30ef0..afcca2a38348 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -32,6 +32,7 @@
> #include <linux/nmi.h>
> #include <linux/percpu.h>
> #include <linux/kmod.h>
> +#include <linux/kmsan.h>
> #include <linux/vmalloc.h>
> #include <linux/kernel_stat.h>
> #include <linux/start_kernel.h>
> @@ -554,6 +555,7 @@ static void __init mm_init(void)
> */
> page_ext_init_flatmem();
> report_meminit();
> + kmsan_initialize_shadow();
> mem_init();
> kmem_cache_init();
> kmemleak_init();
> @@ -625,6 +627,7 @@ asmlinkage __visible void __init start_kernel(void)
> sort_main_extable();
> trap_init();
> mm_init();
> + kmsan_initialize();
>
> ftrace_init();
>
> diff --git a/mm/Makefile b/mm/Makefile
> index d996846697ef..419e6e02dfaf 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -71,6 +71,7 @@ obj-$(CONFIG_PAGE_POISONING) += page_poison.o
> obj-$(CONFIG_SLAB) += slab.o
> obj-$(CONFIG_SLUB) += slub.o
> obj-$(CONFIG_KASAN) += kasan/
> +obj-$(CONFIG_KMSAN) += kmsan/
> obj-$(CONFIG_FAILSLAB) += failslab.o
> obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
> obj-$(CONFIG_MEMTEST) += memtest.o
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 179d55af5852..f9f38f7c1cd4 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -140,6 +140,12 @@ _c_flags += $(if $(patsubst n%,, \
> $(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE))
> endif
>
> +ifeq ($(CONFIG_KMSAN),y)
> +_c_flags += $(if $(patsubst n%,, \
> + $(KMSAN_SANITIZE_$(basetarget).o)$(KMSAN_SANITIZE)y), \
> + $(CFLAGS_KMSAN))
> +endif
> +
> ifeq ($(CONFIG_UBSAN),y)
> _c_flags += $(if $(patsubst n%,, \
> $(UBSAN_SANITIZE_$(basetarget).o)$(UBSAN_SANITIZE)$(CONFIG_UBSAN_SANITIZE_ALL)), \
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 28/36] kmsan: enable KMSAN builds
2019-11-29 15:55 ` Andrey Konovalov
@ 2019-12-11 12:51 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-11 12:51 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Alexander Viro, Andreas Dilger,
Andrew Morton, Andrey Ryabinin, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Marek Szyprowski, Marco Elver, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S . Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, Nov 29, 2019 at 4:55 PM Andrey Konovalov <andreyknvl@google.com> wrote:
>
> On Fri, Nov 22, 2019 at 12:27 PM <glider@google.com> wrote:
> >
> > Make KMSAN usable by adding the necessary Makefile bits and calling
> > KMSAN initialization routines (kmsan_initialize_shadow() and
> > kmsan_initialize()) from init/main.c
>
> I think it makes sense to split these two parts into separate patches as well.
>
Done in v4.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 29/36] kmsan: handle /dev/[u]random
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (26 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 28/36] kmsan: enable KMSAN builds glider
@ 2019-11-22 11:26 ` glider
2019-11-22 11:26 ` [PATCH RFC v3 30/36] kmsan: virtio: check/unpoison scatterlist in vring_map_one_sg() glider
` (7 subsequent siblings)
35 siblings, 0 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Andrew Morton, Jens Axboe, Theodore Ts'o, Dmitry Torokhov,
Martin K. Petersen, Michael S. Tsirkin, Christoph Hellwig,
Eric Dumazet, Eric Van Hensbergen, Takashi Iwai, Vegard Nossum,
Dmitry Vyukov, Matthew Wilcox, linux-mm
Cc: glider, viro, adilger.kernel, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, darrick.wong, davem, ebiggers, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, m.szyprowski,
elver, mark.rutland, schwidefsky, monstr, pmladek, cai, rdunlap,
robin.murphy, sergey.senozhatsky, rostedt, tglx, gor, wsa
The random number generator may use uninitialized memory, but it may not
return uninitialized values. Unpoison the output buffer in
_extract_crng() to prevent false reports.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: "Theodore Ts'o" <tytso@mit.edu>
Cc: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Eric Van Hensbergen <ericvh@gmail.com>
Cc: Takashi Iwai <tiwai@suse.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: linux-mm@kvack.org
---
This patch was previously known as "kmsan: unpoisoning buffers from
devices etc.", but it turned out to be possible to drop most of the
annotations from that patch, so it only relates to /dev/random now.
Change-Id: Id460e7a86ce564f1357469f53d0c7410ca08f0e9
---
drivers/char/random.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/char/random.c b/drivers/char/random.c
index de434feb873a..9b1639c73b65 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -320,6 +320,7 @@
#include <linux/fs.h>
#include <linux/genhd.h>
#include <linux/interrupt.h>
+#include <linux/kmsan-checks.h>
#include <linux/mm.h>
#include <linux/nodemask.h>
#include <linux/spinlock.h>
@@ -1061,6 +1062,11 @@ static void _extract_crng(struct crng_state *crng,
spin_lock_irqsave(&crng->lock, flags);
if (arch_get_random_long(&v))
crng->state[14] ^= v;
+ /*
+ * Regardless of where the random data comes from, KMSAN should treat
+ * it as initialized.
+ */
+ kmsan_unpoison_shadow(crng->state, sizeof(crng->state));
chacha20_block(&crng->state[0], out);
if (crng->state[12] == 0)
crng->state[13]++;
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH RFC v3 30/36] kmsan: virtio: check/unpoison scatterlist in vring_map_one_sg()
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (27 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 29/36] kmsan: handle /dev/[u]random glider
@ 2019-11-22 11:26 ` glider
2019-11-22 11:26 ` [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN glider
` (6 subsequent siblings)
35 siblings, 0 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, Michael S. Tsirkin, Jason Wang, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, axboe, m.szyprowski, elver,
mark.rutland, martin.petersen, schwidefsky, willy, monstr,
pmladek, cai, rdunlap, robin.murphy, sergey.senozhatsky, rostedt,
tiwai, tytso, tglx, gor, wsa
If vring doesn't use the DMA API, KMSAN is unable to tell whether the
memory is initialized by hardware. Explicitly call kmsan_handle_dma()
from vring_map_one_sg() in this case to prevent false positives.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: linux-mm@kvack.org
---
Change-Id: Icc8678289b7084139320fc503898a67aa9803458
---
drivers/virtio/virtio_ring.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index a8041e451e9e..2f8da5eeb3a0 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -11,6 +11,7 @@
#include <linux/module.h>
#include <linux/hrtimer.h>
#include <linux/dma-mapping.h>
+#include <linux/kmsan-checks.h>
#include <xen/xen.h>
#ifdef DEBUG
@@ -326,8 +327,15 @@ static dma_addr_t vring_map_one_sg(const struct vring_virtqueue *vq,
struct scatterlist *sg,
enum dma_data_direction direction)
{
- if (!vq->use_dma_api)
+ if (!vq->use_dma_api) {
+ /*
+ * If DMA is not used, KMSAN doesn't know that the scatterlist
+ * is initialized by the hardware. Explicitly check/unpoison it
+ * depending on the direction.
+ */
+ kmsan_handle_dma(sg_virt(sg), sg->length, direction);
return (dma_addr_t)sg_phys(sg);
+ }
/*
* We can't use dma_map_sg, because we don't use scatterlists in
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (28 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 30/36] kmsan: virtio: check/unpoison scatterlist in vring_map_one_sg() glider
@ 2019-11-22 11:26 ` glider
2019-12-02 15:51 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 32/36] kmsan: add iomap support glider
` (5 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Vegard Nossum, Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, hch, darrick.wong, davem,
dmitry.torokhov, ebiggers, edumazet, ericvh, gregkh,
harry.wentland, herbert, iii, mingo, jasowang, axboe,
m.szyprowski, elver, mark.rutland, martin.petersen, schwidefsky,
willy, mst, monstr, pmladek, cai, rdunlap, robin.murphy,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
Disable the efficient 8-byte reading under KMSAN to avoid false positives.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c
---
lib/string.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/string.c b/lib/string.c
index 08ec58cc673b..15efdc51bda6 100644
--- a/lib/string.c
+++ b/lib/string.c
@@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count)
if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
return -E2BIG;
-#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+/**
+ * Disable the efficient 8-byte reading under KMSAN to avoid false positives.
+ */
+#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN)
/*
* If src is unaligned, don't cross a page boundary,
* since we don't know if the next page is mapped.
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN glider
@ 2019-12-02 15:51 ` Marco Elver
2019-12-02 16:23 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-02 15:51 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
>
> Disable the efficient 8-byte reading under KMSAN to avoid false positives.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
>
> ---
>
> Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c
> ---
> lib/string.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/lib/string.c b/lib/string.c
> index 08ec58cc673b..15efdc51bda6 100644
> --- a/lib/string.c
> +++ b/lib/string.c
> @@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count)
> if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
> return -E2BIG;
>
> -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> +/**
Why a doc comment?
> + * Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> + */
AFAIK the CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS case is about
unaligned accesses crossing page boundaries. In the #else case it's
still going to do word-at-a-time if both src and dest are aligned, so
the comment above is somewhat inaccurate.
> +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN)
> /*
> * If src is unaligned, don't cross a page boundary,
> * since we don't know if the next page is mapped.
> --
> 2.24.0.432.g9d3f5f5b63-goog
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN
2019-12-02 15:51 ` Marco Elver
@ 2019-12-02 16:23 ` Alexander Potapenko
2019-12-03 11:19 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-02 16:23 UTC (permalink / raw)
To: Marco Elver
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 4:51 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> >
> > Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> >
> > ---
> >
> > Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c
> > ---
> > lib/string.c | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/string.c b/lib/string.c
> > index 08ec58cc673b..15efdc51bda6 100644
> > --- a/lib/string.c
> > +++ b/lib/string.c
> > @@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count)
> > if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
> > return -E2BIG;
> >
> > -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> > +/**
>
> Why a doc comment?
Will fix, thanks!
> > + * Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > + */
>
> AFAIK the CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS case is about
> unaligned accesses crossing page boundaries. In the #else case it's
> still going to do word-at-a-time if both src and dest are aligned, so
> the comment above is somewhat inaccurate.
Yes, this makes little sense.
Reading word-at-a-time shouldn't induce any errors, although it may
generate redundant stack IDs for values that will never be used.
I'll try to drop this patch.
> > +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN)
> > /*
> > * If src is unaligned, don't cross a page boundary,
> > * since we don't know if the next page is mapped.
> > --
> > 2.24.0.432.g9d3f5f5b63-goog
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN
2019-12-02 16:23 ` Alexander Potapenko
@ 2019-12-03 11:19 ` Alexander Potapenko
2019-12-03 11:24 ` Marco Elver
0 siblings, 1 reply; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 11:19 UTC (permalink / raw)
To: Marco Elver
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Mon, Dec 2, 2019 at 5:23 PM Alexander Potapenko <glider@google.com> wrote:
>
> On Mon, Dec 2, 2019 at 4:51 PM Marco Elver <elver@google.com> wrote:
> >
> > On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> > >
> > > Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > >
> > > Signed-off-by: Alexander Potapenko <glider@google.com>
> > > To: Alexander Potapenko <glider@google.com>
> > > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > > Cc: Dmitry Vyukov <dvyukov@google.com>
> > > Cc: linux-mm@kvack.org
> > >
> > > ---
> > >
> > > Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c
> > > ---
> > > lib/string.c | 5 ++++-
> > > 1 file changed, 4 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/lib/string.c b/lib/string.c
> > > index 08ec58cc673b..15efdc51bda6 100644
> > > --- a/lib/string.c
> > > +++ b/lib/string.c
> > > @@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count)
> > > if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
> > > return -E2BIG;
> > >
> > > -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> > > +/**
> >
> > Why a doc comment?
> Will fix, thanks!
> > > + * Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > > + */
> >
> > AFAIK the CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS case is about
> > unaligned accesses crossing page boundaries. In the #else case it's
> > still going to do word-at-a-time if both src and dest are aligned, so
> > the comment above is somewhat inaccurate.
> Yes, this makes little sense.
> Reading word-at-a-time shouldn't induce any errors, although it may
> generate redundant stack IDs for values that will never be used.
> I'll try to drop this patch.
Turns out the patch is still needed, as read_word_at_a_time may read
uninitialized bytes which are then used in comparisons.
I've changed the patch to always set max=0 under KMSAN:
https://github.com/google/kmsan/commit/3ff43863bf53dd871a3d4dc4fbb2a76d79b4db4f
Will include this version in v4 series.
>
> > > +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN)
> > > /*
> > > * If src is unaligned, don't cross a page boundary,
> > > * since we don't know if the next page is mapped.
> > > --
> > > 2.24.0.432.g9d3f5f5b63-goog
> > >
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN
2019-12-03 11:19 ` Alexander Potapenko
@ 2019-12-03 11:24 ` Marco Elver
2019-12-03 11:27 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-03 11:24 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, 3 Dec 2019 at 12:19, Alexander Potapenko <glider@google.com> wrote:
>
> On Mon, Dec 2, 2019 at 5:23 PM Alexander Potapenko <glider@google.com> wrote:
> >
> > On Mon, Dec 2, 2019 at 4:51 PM Marco Elver <elver@google.com> wrote:
> > >
> > > On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> > > >
> > > > Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > > >
> > > > Signed-off-by: Alexander Potapenko <glider@google.com>
> > > > To: Alexander Potapenko <glider@google.com>
> > > > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > > > Cc: Dmitry Vyukov <dvyukov@google.com>
> > > > Cc: linux-mm@kvack.org
> > > >
> > > > ---
> > > >
> > > > Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c
> > > > ---
> > > > lib/string.c | 5 ++++-
> > > > 1 file changed, 4 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/lib/string.c b/lib/string.c
> > > > index 08ec58cc673b..15efdc51bda6 100644
> > > > --- a/lib/string.c
> > > > +++ b/lib/string.c
> > > > @@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count)
> > > > if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
> > > > return -E2BIG;
> > > >
> > > > -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> > > > +/**
> > >
> > > Why a doc comment?
> > Will fix, thanks!
> > > > + * Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > > > + */
> > >
> > > AFAIK the CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS case is about
> > > unaligned accesses crossing page boundaries. In the #else case it's
> > > still going to do word-at-a-time if both src and dest are aligned, so
> > > the comment above is somewhat inaccurate.
> > Yes, this makes little sense.
> > Reading word-at-a-time shouldn't induce any errors, although it may
> > generate redundant stack IDs for values that will never be used.
> > I'll try to drop this patch.
> Turns out the patch is still needed, as read_word_at_a_time may read
> uninitialized bytes which are then used in comparisons.
> I've changed the patch to always set max=0 under KMSAN:
> https://github.com/google/kmsan/commit/3ff43863bf53dd871a3d4dc4fbb2a76d79b4db4f
> Will include this version in v4 series.
So was the previous version working because all strscpy where this was
relevant were unaligned?
Re new patch: I think here it should be trivial to use 'if
(IS_ENABLED(CONFIG_KMSAN))' instead of #ifdef.
Thanks,
-- Marco
> > > > +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN)
> > > > /*
> > > > * If src is unaligned, don't cross a page boundary,
> > > > * since we don't know if the next page is mapped.
> > > > --
> > > > 2.24.0.432.g9d3f5f5b63-goog
> > > >
> >
> >
> >
> > --
> > Alexander Potapenko
> > Software Engineer
> >
> > Google Germany GmbH
> > Erika-Mann-Straße, 33
> > 80636 München
> >
> > Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> > Registergericht und -nummer: Hamburg, HRB 86891
> > Sitz der Gesellschaft: Hamburg
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN
2019-12-03 11:24 ` Marco Elver
@ 2019-12-03 11:27 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 11:27 UTC (permalink / raw)
To: Marco Elver
Cc: Vegard Nossum, Dmitry Vyukov, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Eric Biggers, Eric Dumazet,
Eric Van Hensbergen, Greg Kroah-Hartman, Harry Wentland,
Herbert Xu, Ilya Leoshkevich, Ingo Molnar, Jason Wang,
Jens Axboe, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, Dec 3, 2019 at 12:25 PM Marco Elver <elver@google.com> wrote:
>
> On Tue, 3 Dec 2019 at 12:19, Alexander Potapenko <glider@google.com> wrote:
> >
> > On Mon, Dec 2, 2019 at 5:23 PM Alexander Potapenko <glider@google.com> wrote:
> > >
> > > On Mon, Dec 2, 2019 at 4:51 PM Marco Elver <elver@google.com> wrote:
> > > >
> > > > On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> > > > >
> > > > > Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > > > >
> > > > > Signed-off-by: Alexander Potapenko <glider@google.com>
> > > > > To: Alexander Potapenko <glider@google.com>
> > > > > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > > > > Cc: Dmitry Vyukov <dvyukov@google.com>
> > > > > Cc: linux-mm@kvack.org
> > > > >
> > > > > ---
> > > > >
> > > > > Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c
> > > > > ---
> > > > > lib/string.c | 5 ++++-
> > > > > 1 file changed, 4 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/lib/string.c b/lib/string.c
> > > > > index 08ec58cc673b..15efdc51bda6 100644
> > > > > --- a/lib/string.c
> > > > > +++ b/lib/string.c
> > > > > @@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count)
> > > > > if (count == 0 || WARN_ON_ONCE(count > INT_MAX))
> > > > > return -E2BIG;
> > > > >
> > > > > -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> > > > > +/**
> > > >
> > > > Why a doc comment?
> > > Will fix, thanks!
> > > > > + * Disable the efficient 8-byte reading under KMSAN to avoid false positives.
> > > > > + */
> > > >
> > > > AFAIK the CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS case is about
> > > > unaligned accesses crossing page boundaries. In the #else case it's
> > > > still going to do word-at-a-time if both src and dest are aligned, so
> > > > the comment above is somewhat inaccurate.
> > > Yes, this makes little sense.
> > > Reading word-at-a-time shouldn't induce any errors, although it may
> > > generate redundant stack IDs for values that will never be used.
> > > I'll try to drop this patch.
> > Turns out the patch is still needed, as read_word_at_a_time may read
> > uninitialized bytes which are then used in comparisons.
> > I've changed the patch to always set max=0 under KMSAN:
> > https://github.com/google/kmsan/commit/3ff43863bf53dd871a3d4dc4fbb2a76d79b4db4f
> > Will include this version in v4 series.
>
> So was the previous version working because all strscpy where this was
> relevant were unaligned?
Right.
> Re new patch: I think here it should be trivial to use 'if
> (IS_ENABLED(CONFIG_KMSAN))' instead of #ifdef.
Yes, sorry, I always forget about IS_ENABLED.
> Thanks,
> -- Marco
>
> > > > > +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN)
> > > > > /*
> > > > > * If src is unaligned, don't cross a page boundary,
> > > > > * since we don't know if the next page is mapped.
> > > > > --
> > > > > 2.24.0.432.g9d3f5f5b63-goog
> > > > >
> > >
> > >
> > >
> > > --
> > > Alexander Potapenko
> > > Software Engineer
> > >
> > > Google Germany GmbH
> > > Erika-Mann-Straße, 33
> > > 80636 München
> > >
> > > Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> > > Registergericht und -nummer: Hamburg, HRB 86891
> > > Sitz der Gesellschaft: Hamburg
> >
> >
> >
> > --
> > Alexander Potapenko
> > Software Engineer
> >
> > Google Germany GmbH
> > Erika-Mann-Straße, 33
> > 80636 München
> >
> > Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> > Registergericht und -nummer: Hamburg, HRB 86891
> > Sitz der Gesellschaft: Hamburg
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 32/36] kmsan: add iomap support
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (29 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 31/36] kmsan: disable strscpy() optimization under KMSAN glider
@ 2019-11-22 11:26 ` glider
2019-12-03 12:50 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 33/36] kmsan: dma: unpoison memory mapped by dma_direct_map_page() glider
` (4 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Christoph Hellwig, Darrick J. Wong, Vegard Nossum, Dmitry Vyukov,
linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, davem, dmitry.torokhov, ebiggers,
edumazet, ericvh, gregkh, harry.wentland, herbert, iii, mingo,
jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso,
tglx, gor, wsa
Functions from lib/iomap.c interact with hardware, so KMSAN must ensure
that:
- every read function returns an initialized value
- every write function checks values before sending them to hardware.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: Iacd96265e56398d8c111637ddad3cad727e48c8d
---
lib/iomap.c | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/lib/iomap.c b/lib/iomap.c
index e909ab71e995..3ef20b8138e2 100644
--- a/lib/iomap.c
+++ b/lib/iomap.c
@@ -6,6 +6,7 @@
*/
#include <linux/pci.h>
#include <linux/io.h>
+#include <linux/kmsan-checks.h>
#include <linux/export.h>
@@ -70,26 +71,31 @@ static void bad_io_access(unsigned long port, const char *access)
#define mmio_read64be(addr) swab64(readq(addr))
#endif
+__no_sanitize_memory
unsigned int ioread8(void __iomem *addr)
{
IO_COND(addr, return inb(port), return readb(addr));
return 0xff;
}
+__no_sanitize_memory
unsigned int ioread16(void __iomem *addr)
{
IO_COND(addr, return inw(port), return readw(addr));
return 0xffff;
}
+__no_sanitize_memory
unsigned int ioread16be(void __iomem *addr)
{
IO_COND(addr, return pio_read16be(port), return mmio_read16be(addr));
return 0xffff;
}
+__no_sanitize_memory
unsigned int ioread32(void __iomem *addr)
{
IO_COND(addr, return inl(port), return readl(addr));
return 0xffffffff;
}
+__no_sanitize_memory
unsigned int ioread32be(void __iomem *addr)
{
IO_COND(addr, return pio_read32be(port), return mmio_read32be(addr));
@@ -142,18 +148,21 @@ static u64 pio_read64be_hi_lo(unsigned long port)
return lo | (hi << 32);
}
+__no_sanitize_memory
u64 ioread64_lo_hi(void __iomem *addr)
{
IO_COND(addr, return pio_read64_lo_hi(port), return readq(addr));
return 0xffffffffffffffffULL;
}
+__no_sanitize_memory
u64 ioread64_hi_lo(void __iomem *addr)
{
IO_COND(addr, return pio_read64_hi_lo(port), return readq(addr));
return 0xffffffffffffffffULL;
}
+__no_sanitize_memory
u64 ioread64be_lo_hi(void __iomem *addr)
{
IO_COND(addr, return pio_read64be_lo_hi(port),
@@ -161,6 +170,7 @@ u64 ioread64be_lo_hi(void __iomem *addr)
return 0xffffffffffffffffULL;
}
+__no_sanitize_memory
u64 ioread64be_hi_lo(void __iomem *addr)
{
IO_COND(addr, return pio_read64be_hi_lo(port),
@@ -188,22 +198,32 @@ EXPORT_SYMBOL(ioread64be_hi_lo);
void iowrite8(u8 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, outb(val,port), writeb(val, addr));
}
void iowrite16(u16 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, outw(val,port), writew(val, addr));
}
void iowrite16be(u16 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, pio_write16be(val,port), mmio_write16be(val, addr));
}
void iowrite32(u32 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, outl(val,port), writel(val, addr));
}
void iowrite32be(u32 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, pio_write32be(val,port), mmio_write32be(val, addr));
}
EXPORT_SYMBOL(iowrite8);
@@ -239,24 +259,32 @@ static void pio_write64be_hi_lo(u64 val, unsigned long port)
void iowrite64_lo_hi(u64 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, pio_write64_lo_hi(val, port),
writeq(val, addr));
}
void iowrite64_hi_lo(u64 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, pio_write64_hi_lo(val, port),
writeq(val, addr));
}
void iowrite64be_lo_hi(u64 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, pio_write64be_lo_hi(val, port),
mmio_write64be(val, addr));
}
void iowrite64be_hi_lo(u64 val, void __iomem *addr)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(&val, sizeof(val));
IO_COND(addr, pio_write64be_hi_lo(val, port),
mmio_write64be(val, addr));
}
@@ -328,14 +356,20 @@ static inline void mmio_outsl(void __iomem *addr, const u32 *src, int count)
void ioread8_rep(void __iomem *addr, void *dst, unsigned long count)
{
IO_COND(addr, insb(port,dst,count), mmio_insb(addr, dst, count));
+ /* KMSAN must treat values read from devices as initialized. */
+ kmsan_unpoison_shadow(dst, count);
}
void ioread16_rep(void __iomem *addr, void *dst, unsigned long count)
{
IO_COND(addr, insw(port,dst,count), mmio_insw(addr, dst, count));
+ /* KMSAN must treat values read from devices as initialized. */
+ kmsan_unpoison_shadow(dst, count * 2);
}
void ioread32_rep(void __iomem *addr, void *dst, unsigned long count)
{
IO_COND(addr, insl(port,dst,count), mmio_insl(addr, dst, count));
+ /* KMSAN must treat values read from devices as initialized. */
+ kmsan_unpoison_shadow(dst, count * 4);
}
EXPORT_SYMBOL(ioread8_rep);
EXPORT_SYMBOL(ioread16_rep);
@@ -343,14 +377,20 @@ EXPORT_SYMBOL(ioread32_rep);
void iowrite8_rep(void __iomem *addr, const void *src, unsigned long count)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(src, count);
IO_COND(addr, outsb(port, src, count), mmio_outsb(addr, src, count));
}
void iowrite16_rep(void __iomem *addr, const void *src, unsigned long count)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(src, count);
IO_COND(addr, outsw(port, src, count), mmio_outsw(addr, src, count));
}
void iowrite32_rep(void __iomem *addr, const void *src, unsigned long count)
{
+ /* Make sure uninitialized memory isn't copied to devices. */
+ kmsan_check_memory(src, count);
IO_COND(addr, outsl(port, src,count), mmio_outsl(addr, src, count));
}
EXPORT_SYMBOL(iowrite8_rep);
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 32/36] kmsan: add iomap support
2019-11-22 11:26 ` [PATCH RFC v3 32/36] kmsan: add iomap support glider
@ 2019-12-03 12:50 ` Marco Elver
2019-12-03 14:07 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-03 12:50 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Christoph Hellwig, Darrick J. Wong, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, David S. Miller, Dmitry Torokhov,
Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
>
> Functions from lib/iomap.c interact with hardware, so KMSAN must ensure
> that:
> - every read function returns an initialized value
> - every write function checks values before sending them to hardware.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: Christoph Hellwig <hch@infradead.org>
> Cc: Darrick J. Wong <darrick.wong@oracle.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: Iacd96265e56398d8c111637ddad3cad727e48c8d
> ---
> lib/iomap.c | 40 ++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 40 insertions(+)
>
> diff --git a/lib/iomap.c b/lib/iomap.c
> index e909ab71e995..3ef20b8138e2 100644
> --- a/lib/iomap.c
> +++ b/lib/iomap.c
> @@ -6,6 +6,7 @@
[...]
> void iowrite8_rep(void __iomem *addr, const void *src, unsigned long count)
> {
> + /* Make sure uninitialized memory isn't copied to devices. */
> + kmsan_check_memory(src, count);
> IO_COND(addr, outsb(port, src, count), mmio_outsb(addr, src, count));
> }
> void iowrite16_rep(void __iomem *addr, const void *src, unsigned long count)
> {
> + /* Make sure uninitialized memory isn't copied to devices. */
> + kmsan_check_memory(src, count);
Should this be 'count * 2' ?
> IO_COND(addr, outsw(port, src, count), mmio_outsw(addr, src, count));
> }
> void iowrite32_rep(void __iomem *addr, const void *src, unsigned long count)
> {
> + /* Make sure uninitialized memory isn't copied to devices. */
> + kmsan_check_memory(src, count);
Should this be 'count * 4' ?
Thanks,
-- Marco
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 32/36] kmsan: add iomap support
2019-12-03 12:50 ` Marco Elver
@ 2019-12-03 14:07 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 14:07 UTC (permalink / raw)
To: Marco Elver
Cc: Christoph Hellwig, Darrick J. Wong, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andreas Dilger,
Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, David S. Miller, Dmitry Torokhov,
Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Theodore Ts'o, Thomas Gleixner,
Vasily Gorbik, Wolfram Sang
On Tue, Dec 3, 2019 at 1:51 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> >
> > Functions from lib/iomap.c interact with hardware, so KMSAN must ensure
> > that:
> > - every read function returns an initialized value
> > - every write function checks values before sending them to hardware.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > Cc: Christoph Hellwig <hch@infradead.org>
> > Cc: Darrick J. Wong <darrick.wong@oracle.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: Iacd96265e56398d8c111637ddad3cad727e48c8d
> > ---
> > lib/iomap.c | 40 ++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 40 insertions(+)
> >
> > diff --git a/lib/iomap.c b/lib/iomap.c
> > index e909ab71e995..3ef20b8138e2 100644
> > --- a/lib/iomap.c
> > +++ b/lib/iomap.c
> > @@ -6,6 +6,7 @@
> [...]
> > void iowrite8_rep(void __iomem *addr, const void *src, unsigned long count)
> > {
> > + /* Make sure uninitialized memory isn't copied to devices. */
> > + kmsan_check_memory(src, count);
> > IO_COND(addr, outsb(port, src, count), mmio_outsb(addr, src, count));
> > }
> > void iowrite16_rep(void __iomem *addr, const void *src, unsigned long count)
> > {
> > + /* Make sure uninitialized memory isn't copied to devices. */
> > + kmsan_check_memory(src, count);
>
> Should this be 'count * 2' ?
Good catch, thanks!
Will fix in v4.
>
> > IO_COND(addr, outsw(port, src, count), mmio_outsw(addr, src, count));
> > }
> > void iowrite32_rep(void __iomem *addr, const void *src, unsigned long count)
> > {
> > + /* Make sure uninitialized memory isn't copied to devices. */
> > + kmsan_check_memory(src, count);
>
> Should this be 'count * 4' ?
Ditto
> Thanks,
> -- Marco
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 33/36] kmsan: dma: unpoison memory mapped by dma_direct_map_page()
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (30 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 32/36] kmsan: add iomap support glider
@ 2019-11-22 11:26 ` glider
2019-11-22 11:26 ` [PATCH RFC v3 34/36] kmsan: disable physical page merging in biovec glider
` (3 subsequent siblings)
35 siblings, 0 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Christoph Hellwig, Marek Szyprowski, Robin Murphy, Vegard Nossum,
Dmitry Vyukov, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin, luto,
ard.biesheuvel, arnd, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, elver, mark.rutland, martin.petersen,
schwidefsky, willy, mst, monstr, pmladek, cai, rdunlap,
sergey.senozhatsky, rostedt, tiwai, tytso, tglx, gor, wsa
KMSAN doesn't know about DMA memory writes performed by devices.
We unpoison such memory when it's mapped to avoid false positive
reports.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: Ib1019ed531fea69f88b5cdec3d1e27403f2f3d64
---
kernel/dma/direct.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 8402b29c280f..3539337f76d9 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -341,6 +341,7 @@ dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
report_addr(dev, dma_addr, size);
return DMA_MAPPING_ERROR;
}
+ kmsan_handle_dma(page_address(page) + offset, size, dir);
if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
arch_sync_dma_for_device(dev, phys, size, dir);
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH RFC v3 34/36] kmsan: disable physical page merging in biovec
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (31 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 33/36] kmsan: dma: unpoison memory mapped by dma_direct_map_page() glider
@ 2019-11-22 11:26 ` glider
2019-12-03 12:54 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN glider
` (2 subsequent siblings)
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, aryabinin,
ard.biesheuvel, arnd, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso,
tglx, gor, wsa
KMSAN metadata for consequent physical pages may be inconsequent,
therefore accessing such pages together may lead to metadata
corruption.
We disable merging pages in biovec to prevent such corruptions.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: linux-mm@kvack.org
---
Change-Id: Id2f2babaf662ac44675c4f2790f4a80ddc328fa7
---
block/blk.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/block/blk.h b/block/blk.h
index 47fba9362e60..9ee271a22423 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -78,6 +78,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q,
phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset;
phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset;
+#ifdef CONFIG_KMSAN
+ /*
+ * Merging consequent physical pages may not work correctly under KMSAN
+ * if their metadata pages aren't consequent. Just disable merging.
+ */
+ return false;
+#endif
if (addr1 + vec1->bv_len != addr2)
return false;
if (xen_domain() && !xen_biovec_phys_mergeable(vec1, vec2->bv_page))
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 34/36] kmsan: disable physical page merging in biovec
2019-11-22 11:26 ` [PATCH RFC v3 34/36] kmsan: disable physical page merging in biovec glider
@ 2019-12-03 12:54 ` Marco Elver
2019-12-03 13:38 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-03 12:54 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Darrick J. Wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
>
> KMSAN metadata for consequent physical pages may be inconsequent,
> therefore accessing such pages together may lead to metadata
> corruption.
> We disable merging pages in biovec to prevent such corruptions.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: Id2f2babaf662ac44675c4f2790f4a80ddc328fa7
> ---
> block/blk.h | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/block/blk.h b/block/blk.h
> index 47fba9362e60..9ee271a22423 100644
> --- a/block/blk.h
> +++ b/block/blk.h
> @@ -78,6 +78,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q,
> phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset;
> phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset;
>
> +#ifdef CONFIG_KMSAN
Prefer 'if (IS_ENABLED(CONFIG_KMSAN))'.
Thanks,
-- Marco
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 34/36] kmsan: disable physical page merging in biovec
2019-12-03 12:54 ` Marco Elver
@ 2019-12-03 13:38 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-03 13:38 UTC (permalink / raw)
To: Marco Elver
Cc: Jens Axboe, Andy Lutomirski, Vegard Nossum, Dmitry Vyukov,
Christoph Hellwig, Linux Memory Management List, Al Viro,
Andreas Dilger, Andrew Morton, Andrey Konovalov, Andrey Ryabinin,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Darrick J. Wong, David S. Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Marek Szyprowski, Mark Rutland, Martin K. Petersen,
Martin Schwidefsky, Matthew Wilcox, Michael S. Tsirkin,
Michal Simek, Petr Mladek, Qian Cai, Randy Dunlap, Robin Murphy,
Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, Dec 3, 2019 at 1:54 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> >
> > KMSAN metadata for consequent physical pages may be inconsequent,
> > therefore accessing such pages together may lead to metadata
> > corruption.
> > We disable merging pages in biovec to prevent such corruptions.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Jens Axboe <axboe@kernel.dk>
> > Cc: Andy Lutomirski <luto@kernel.org>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: Id2f2babaf662ac44675c4f2790f4a80ddc328fa7
> > ---
> > block/blk.h | 7 +++++++
> > 1 file changed, 7 insertions(+)
> >
> > diff --git a/block/blk.h b/block/blk.h
> > index 47fba9362e60..9ee271a22423 100644
> > --- a/block/blk.h
> > +++ b/block/blk.h
> > @@ -78,6 +78,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q,
> > phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset;
> > phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset;
> >
> > +#ifdef CONFIG_KMSAN
>
> Prefer 'if (IS_ENABLED(CONFIG_KMSAN))'.
WBD in v4.
> Thanks,
> -- Marco
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (32 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 34/36] kmsan: disable physical page merging in biovec glider
@ 2019-11-22 11:26 ` glider
2019-11-25 16:05 ` Robin Murphy
2019-12-03 14:22 ` Marco Elver
2019-11-22 11:26 ` [PATCH RFC v3 36/36] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN glider
2019-11-29 14:39 ` [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure Marco Elver
35 siblings, 2 replies; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Theodore Ts'o, Andreas Dilger, Vegard Nossum, Dmitry Vyukov,
linux-mm
Cc: glider, viro, akpm, andreyknvl, aryabinin, luto, ard.biesheuvel,
arnd, hch, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
edumazet, ericvh, gregkh, harry.wentland, herbert, iii, mingo,
jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, robin.murphy, sergey.senozhatsky, rostedt, tiwai, tglx,
gor, wsa
KMSAN doesn't allow treating adjacent memory pages as such, if they were
allocated by different alloc_pages() calls.
ext4_mpage_readpages() however does so: adjacent pages end up being passed
together to dma_direct_map_sg().
To prevent this, jump directly to the buffer_head-based read function in
KMSAN builds.
Signed-off-by: Alexander Potapenko <glider@google.com>
Cc: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andreas Dilger <adilger.kernel@dilger.ca>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: linux-mm@kvack.org
---
Change-Id: I54ae8af536626a988e6398ff18a06c179b0ce034
---
fs/ext4/readpage.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
index a30b203fa461..a3bb9e3ce5de 100644
--- a/fs/ext4/readpage.c
+++ b/fs/ext4/readpage.c
@@ -252,6 +252,17 @@ int ext4_mpage_readpages(struct address_space *mapping,
if (page_has_buffers(page))
goto confused;
+#if defined(CONFIG_KMSAN)
+ /*
+ * The following code may treat adjacent pages allocated
+ * separately as a bigger contiguous allocation.
+ * KMSAN doesn't allow this, as the corresponding metadata
+ * pages may be separated.
+ * Skip this complex logic for KMSAN builds.
+ */
+ goto confused;
+#endif
+
block_in_file = (sector_t)page->index << (PAGE_SHIFT - blkbits);
last_block = block_in_file + nr_pages * blocks_per_page;
last_block_in_file = (ext4_readpage_limit(inode) +
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN glider
@ 2019-11-25 16:05 ` Robin Murphy
2019-11-25 17:03 ` Alexander Potapenko
2019-12-03 14:22 ` Marco Elver
1 sibling, 1 reply; 120+ messages in thread
From: Robin Murphy @ 2019-11-25 16:05 UTC (permalink / raw)
To: glider, Theodore Ts'o, Andreas Dilger, Vegard Nossum,
Dmitry Vyukov, linux-mm
Cc: viro, akpm, andreyknvl, aryabinin, luto, ard.biesheuvel, arnd,
hch, hch, darrick.wong, davem, dmitry.torokhov, ebiggers,
edumazet, ericvh, gregkh, harry.wentland, herbert, iii, mingo,
jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, monstr, pmladek, cai,
rdunlap, sergey.senozhatsky, rostedt, tiwai, tglx, gor, wsa
On 22/11/2019 11:26 am, glider@google.com wrote:
> KMSAN doesn't allow treating adjacent memory pages as such, if they were
> allocated by different alloc_pages() calls.
> ext4_mpage_readpages() however does so: adjacent pages end up being passed
> together to dma_direct_map_sg().
Urgh, there are definitely more places where physically-contiguous pages
are coalesced into single scatterlist entries - see
sg_alloc_table_from_pages() for instance. I wouldn't be surprised if
there are further open-coded versions hiding out in various other
drivers/subsystems too. Unless I've misunderstood, this seems like quite
an invasive limitation :(
Robin.
> To prevent this, jump directly to the buffer_head-based read function in
> KMSAN builds.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: "Theodore Ts'o" <tytso@mit.edu>
> Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: I54ae8af536626a988e6398ff18a06c179b0ce034
> ---
> fs/ext4/readpage.c | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
> diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
> index a30b203fa461..a3bb9e3ce5de 100644
> --- a/fs/ext4/readpage.c
> +++ b/fs/ext4/readpage.c
> @@ -252,6 +252,17 @@ int ext4_mpage_readpages(struct address_space *mapping,
> if (page_has_buffers(page))
> goto confused;
>
> +#if defined(CONFIG_KMSAN)
> + /*
> + * The following code may treat adjacent pages allocated
> + * separately as a bigger contiguous allocation.
> + * KMSAN doesn't allow this, as the corresponding metadata
> + * pages may be separated.
> + * Skip this complex logic for KMSAN builds.
> + */
> + goto confused;
> +#endif
> +
> block_in_file = (sector_t)page->index << (PAGE_SHIFT - blkbits);
> last_block = block_in_file + nr_pages * blocks_per_page;
> last_block_in_file = (ext4_readpage_limit(inode) +
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN
2019-11-25 16:05 ` Robin Murphy
@ 2019-11-25 17:03 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-11-25 17:03 UTC (permalink / raw)
To: Robin Murphy
Cc: Theodore Ts'o, Andreas Dilger, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andrew Morton,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, hch, Christoph Hellwig,
darrick.wong, David Miller, Dmitry Torokhov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
jasowang, Jens Axboe, Marek Szyprowski, Marco Elver,
Mark Rutland, Martin K . Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Mon, Nov 25, 2019 at 5:05 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 22/11/2019 11:26 am, glider@google.com wrote:
> > KMSAN doesn't allow treating adjacent memory pages as such, if they were
> > allocated by different alloc_pages() calls.
> > ext4_mpage_readpages() however does so: adjacent pages end up being passed
> > together to dma_direct_map_sg().
>
> Urgh, there are definitely more places where physically-contiguous pages
> are coalesced into single scatterlist entries - see
> sg_alloc_table_from_pages() for instance. I wouldn't be surprised if
> there are further open-coded versions hiding out in various other
> drivers/subsystems too. Unless I've misunderstood, this seems like quite
> an invasive limitation :(
You're right.
Other places haven't fired off so far, so I was just unaware of them,
but I'm anticipating more.
There are two possible solutions to that problem:
1. Allocate shadow and origin pages at fixed offset from the kernel page.
This is what we already do for vmalloc, but not for page_alloc(), as
it turned out to be quite hard.
Ideas on how to implement this approach are still welcome, because
it'll simplify the rest of the KMSAN runtime a lot.
2. Make all accesses touching non-contiguous pages access dummy shadow
pages instead, so that such accesses don't produce any uninitialized
values.
This is quite controversial, as it may prevent true positives from
being reported.
> Robin.
>
> > To prevent this, jump directly to the buffer_head-based read function in
> > KMSAN builds.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > Cc: "Theodore Ts'o" <tytso@mit.edu>
> > Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: I54ae8af536626a988e6398ff18a06c179b0ce034
> > ---
> > fs/ext4/readpage.c | 11 +++++++++++
> > 1 file changed, 11 insertions(+)
> >
> > diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
> > index a30b203fa461..a3bb9e3ce5de 100644
> > --- a/fs/ext4/readpage.c
> > +++ b/fs/ext4/readpage.c
> > @@ -252,6 +252,17 @@ int ext4_mpage_readpages(struct address_space *mapping,
> > if (page_has_buffers(page))
> > goto confused;
> >
> > +#if defined(CONFIG_KMSAN)
> > + /*
> > + * The following code may treat adjacent pages allocated
> > + * separately as a bigger contiguous allocation.
> > + * KMSAN doesn't allow this, as the corresponding metadata
> > + * pages may be separated.
> > + * Skip this complex logic for KMSAN builds.
> > + */
> > + goto confused;
> > +#endif
> > +
> > block_in_file = (sector_t)page->index << (PAGE_SHIFT - blkbits);
> > last_block = block_in_file + nr_pages * blocks_per_page;
> > last_block_in_file = (ext4_readpage_limit(inode) +
> >
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN glider
2019-11-25 16:05 ` Robin Murphy
@ 2019-12-03 14:22 ` Marco Elver
2019-12-05 14:31 ` Alexander Potapenko
1 sibling, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-03 14:22 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Theodore Ts'o, Andreas Dilger, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andrew Morton,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
>
> KMSAN doesn't allow treating adjacent memory pages as such, if they were
> allocated by different alloc_pages() calls.
> ext4_mpage_readpages() however does so: adjacent pages end up being passed
> together to dma_direct_map_sg().
> To prevent this, jump directly to the buffer_head-based read function in
> KMSAN builds.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> Cc: "Theodore Ts'o" <tytso@mit.edu>
> Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: linux-mm@kvack.org
> ---
>
> Change-Id: I54ae8af536626a988e6398ff18a06c179b0ce034
> ---
> fs/ext4/readpage.c | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
> diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
> index a30b203fa461..a3bb9e3ce5de 100644
> --- a/fs/ext4/readpage.c
> +++ b/fs/ext4/readpage.c
> @@ -252,6 +252,17 @@ int ext4_mpage_readpages(struct address_space *mapping,
> if (page_has_buffers(page))
> goto confused;
>
> +#if defined(CONFIG_KMSAN)
Prefer 'if (IS_ENABLED(CONFIG_KMSAN))'.
Thanks,
-- Marco
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN
2019-12-03 14:22 ` Marco Elver
@ 2019-12-05 14:31 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 14:31 UTC (permalink / raw)
To: Marco Elver
Cc: Theodore Ts'o, Andreas Dilger, Vegard Nossum, Dmitry Vyukov,
Linux Memory Management List, Al Viro, Andrew Morton,
Andrey Konovalov, Andrey Ryabinin, Andy Lutomirski,
Ard Biesheuvel, Arnd Bergmann, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Michal Simek, Petr Mladek,
Qian Cai, Randy Dunlap, Robin Murphy, Sergey Senozhatsky,
Steven Rostedt, Takashi Iwai, Thomas Gleixner, Vasily Gorbik,
Wolfram Sang
On Tue, Dec 3, 2019 at 3:22 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> >
> > KMSAN doesn't allow treating adjacent memory pages as such, if they were
> > allocated by different alloc_pages() calls.
> > ext4_mpage_readpages() however does so: adjacent pages end up being passed
> > together to dma_direct_map_sg().
> > To prevent this, jump directly to the buffer_head-based read function in
> > KMSAN builds.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > Cc: "Theodore Ts'o" <tytso@mit.edu>
> > Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > Change-Id: I54ae8af536626a988e6398ff18a06c179b0ce034
> > ---
> > fs/ext4/readpage.c | 11 +++++++++++
> > 1 file changed, 11 insertions(+)
> >
> > diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c
> > index a30b203fa461..a3bb9e3ce5de 100644
> > --- a/fs/ext4/readpage.c
> > +++ b/fs/ext4/readpage.c
> > @@ -252,6 +252,17 @@ int ext4_mpage_readpages(struct address_space *mapping,
> > if (page_has_buffers(page))
> > goto confused;
> >
> > +#if defined(CONFIG_KMSAN)
>
> Prefer 'if (IS_ENABLED(CONFIG_KMSAN))'.
Done in v4.
> Thanks,
> -- Marco
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH RFC v3 36/36] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (33 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 35/36] kmsan: ext4: skip block merging logic in ext4_mpage_readpages for KMSAN glider
@ 2019-11-22 11:26 ` glider
2019-12-03 14:17 ` Marco Elver
2019-11-29 14:39 ` [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure Marco Elver
35 siblings, 1 reply; 120+ messages in thread
From: glider @ 2019-11-22 11:26 UTC (permalink / raw)
To: Arnd Bergmann, Michal Simek, Andrey Ryabinin, Vegard Nossum,
Dmitry Vyukov, Randy Dunlap, linux-mm
Cc: glider, viro, adilger.kernel, akpm, andreyknvl, luto,
ard.biesheuvel, hch, hch, darrick.wong, davem, dmitry.torokhov,
ebiggers, edumazet, ericvh, gregkh, harry.wentland, herbert, iii,
mingo, jasowang, axboe, m.szyprowski, elver, mark.rutland,
martin.petersen, schwidefsky, willy, mst, pmladek, cai,
robin.murphy, sergey.senozhatsky, rostedt, tiwai, tytso, tglx,
gor, wsa
This is needed to allow memory tools like KASAN and KMSAN see the
memory accesses from the checksum code. Without CONFIG_GENERIC_CSUM the
tools can't see memory accesses originating from handwritten assembly
code.
For KASAN it's a question of detecting more bugs, for KMSAN using the C
implementation also helps avoid false positives originating from
seemingly uninitialized checksum values.
Signed-off-by: Alexander Potapenko <glider@google.com>
To: Alexander Potapenko <glider@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: linux-mm@kvack.org
---
v2:
- dropped the "default n" (as requested by Randy Dunlap)
Change-Id: I645e2c097253a8d5717ad87e2e2df6f6f67251f3
---
arch/x86/Kconfig | 4 ++++
arch/x86/include/asm/checksum.h | 10 +++++++---
arch/x86/lib/Makefile | 2 ++
3 files changed, 13 insertions(+), 3 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 3f83a5c53808..f497aae3dbf4 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,6 +272,10 @@ config GENERIC_ISA_DMA
def_bool y
depends on ISA_DMA_API
+config GENERIC_CSUM
+ bool
+ default y if KMSAN || KASAN
+
config GENERIC_BUG
def_bool y
depends on BUG
diff --git a/arch/x86/include/asm/checksum.h b/arch/x86/include/asm/checksum.h
index d79d1e622dcf..ab3464cbce26 100644
--- a/arch/x86/include/asm/checksum.h
+++ b/arch/x86/include/asm/checksum.h
@@ -1,6 +1,10 @@
/* SPDX-License-Identifier: GPL-2.0 */
-#ifdef CONFIG_X86_32
-# include <asm/checksum_32.h>
+#ifdef CONFIG_GENERIC_CSUM
+# include <asm-generic/checksum.h>
#else
-# include <asm/checksum_64.h>
+# ifdef CONFIG_X86_32
+# include <asm/checksum_32.h>
+# else
+# include <asm/checksum_64.h>
+# endif
#endif
diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
index 5246db42de45..bca9031de9ff 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
@@ -55,7 +55,9 @@ endif
lib-$(CONFIG_X86_USE_3DNOW) += mmx_32.o
else
obj-y += iomap_copy_64.o
+ifneq ($(CONFIG_GENERIC_CSUM),y)
lib-y += csum-partial_64.o csum-copy_64.o csum-wrappers_64.o
+endif
lib-y += clear_page_64.o copy_page_64.o
lib-y += memmove_64.o memset_64.o
lib-y += copy_user_64.o
--
2.24.0.432.g9d3f5f5b63-goog
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 36/36] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN
2019-11-22 11:26 ` [PATCH RFC v3 36/36] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN glider
@ 2019-12-03 14:17 ` Marco Elver
2019-12-05 14:37 ` Alexander Potapenko
0 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-12-03 14:17 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Arnd Bergmann, Michal Simek, Andrey Ryabinin, Vegard Nossum,
Dmitry Vyukov, Randy Dunlap, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andy Lutomirski, Ard Biesheuvel, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Petr Mladek, Qian Cai,
Robin Murphy, Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
>
> This is needed to allow memory tools like KASAN and KMSAN see the
> memory accesses from the checksum code. Without CONFIG_GENERIC_CSUM the
> tools can't see memory accesses originating from handwritten assembly
> code.
> For KASAN it's a question of detecting more bugs, for KMSAN using the C
> implementation also helps avoid false positives originating from
> seemingly uninitialized checksum values.
The files touched are only in x86. The title mentions 'net' -- is this
still accurate?
> Signed-off-by: Alexander Potapenko <glider@google.com>
> To: Alexander Potapenko <glider@google.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Michal Simek <monstr@monstr.eu>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: linux-mm@kvack.org
> ---
>
> v2:
> - dropped the "default n" (as requested by Randy Dunlap)
>
> Change-Id: I645e2c097253a8d5717ad87e2e2df6f6f67251f3
> ---
> arch/x86/Kconfig | 4 ++++
> arch/x86/include/asm/checksum.h | 10 +++++++---
> arch/x86/lib/Makefile | 2 ++
> 3 files changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 3f83a5c53808..f497aae3dbf4 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -272,6 +272,10 @@ config GENERIC_ISA_DMA
> def_bool y
> depends on ISA_DMA_API
>
> +config GENERIC_CSUM
> + bool
> + default y if KMSAN || KASAN
When would it be desirable to use GENERIC_CSUM without KMSAN or KASAN?
If it is generally a bad idea to disable GENERIC_CSUM with KMSAN and
KASAN, does it make more sense to just remove this config option and
unconditionally (if CONFIG_K{A,M}SAN) use the asm-generic version in
checksum.h?
This would simplify this patch and avoid introducing a config option
that will likely never be set explicitly.
> config GENERIC_BUG
> def_bool y
> depends on BUG
> diff --git a/arch/x86/include/asm/checksum.h b/arch/x86/include/asm/checksum.h
> index d79d1e622dcf..ab3464cbce26 100644
> --- a/arch/x86/include/asm/checksum.h
> +++ b/arch/x86/include/asm/checksum.h
> @@ -1,6 +1,10 @@
> /* SPDX-License-Identifier: GPL-2.0 */
> -#ifdef CONFIG_X86_32
> -# include <asm/checksum_32.h>
> +#ifdef CONFIG_GENERIC_CSUM
> +# include <asm-generic/checksum.h>
> #else
> -# include <asm/checksum_64.h>
> +# ifdef CONFIG_X86_32
> +# include <asm/checksum_32.h>
> +# else
> +# include <asm/checksum_64.h>
> +# endif
> #endif
> diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
> index 5246db42de45..bca9031de9ff 100644
> --- a/arch/x86/lib/Makefile
> +++ b/arch/x86/lib/Makefile
> @@ -55,7 +55,9 @@ endif
> lib-$(CONFIG_X86_USE_3DNOW) += mmx_32.o
> else
> obj-y += iomap_copy_64.o
> +ifneq ($(CONFIG_GENERIC_CSUM),y)
> lib-y += csum-partial_64.o csum-copy_64.o csum-wrappers_64.o
> +endif
> lib-y += clear_page_64.o copy_page_64.o
> lib-y += memmove_64.o memset_64.o
> lib-y += copy_user_64.o
Thanks,
-- Marco
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 36/36] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN
2019-12-03 14:17 ` Marco Elver
@ 2019-12-05 14:37 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-05 14:37 UTC (permalink / raw)
To: Marco Elver
Cc: Arnd Bergmann, Michal Simek, Andrey Ryabinin, Vegard Nossum,
Dmitry Vyukov, Randy Dunlap, Linux Memory Management List,
Al Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andy Lutomirski, Ard Biesheuvel, Christoph Hellwig,
Christoph Hellwig, Darrick J. Wong, David S. Miller,
Dmitry Torokhov, Eric Biggers, Eric Dumazet, Eric Van Hensbergen,
Greg Kroah-Hartman, Harry Wentland, Herbert Xu, Ilya Leoshkevich,
Ingo Molnar, Jason Wang, Jens Axboe, Marek Szyprowski,
Mark Rutland, Martin K. Petersen, Martin Schwidefsky,
Matthew Wilcox, Michael S. Tsirkin, Petr Mladek, Qian Cai,
Robin Murphy, Sergey Senozhatsky, Steven Rostedt, Takashi Iwai,
Theodore Ts'o, Thomas Gleixner, Vasily Gorbik, Wolfram Sang
On Tue, Dec 3, 2019 at 3:17 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:28, <glider@google.com> wrote:
> >
> > This is needed to allow memory tools like KASAN and KMSAN see the
> > memory accesses from the checksum code. Without CONFIG_GENERIC_CSUM the
> > tools can't see memory accesses originating from handwritten assembly
> > code.
> > For KASAN it's a question of detecting more bugs, for KMSAN using the C
> > implementation also helps avoid false positives originating from
> > seemingly uninitialized checksum values.
>
> The files touched are only in x86. The title mentions 'net' -- is this
> still accurate?
Not sure :/
I'll probably replace it with "x86:", as other patches to this file
don't mention "net".
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Arnd Bergmann <arnd@arndb.de>
> > Cc: Michal Simek <monstr@monstr.eu>
> > Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: Randy Dunlap <rdunlap@infradead.org>
> > Cc: linux-mm@kvack.org
> > ---
> >
> > v2:
> > - dropped the "default n" (as requested by Randy Dunlap)
> >
> > Change-Id: I645e2c097253a8d5717ad87e2e2df6f6f67251f3
> > ---
> > arch/x86/Kconfig | 4 ++++
> > arch/x86/include/asm/checksum.h | 10 +++++++---
> > arch/x86/lib/Makefile | 2 ++
> > 3 files changed, 13 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > index 3f83a5c53808..f497aae3dbf4 100644
> > --- a/arch/x86/Kconfig
> > +++ b/arch/x86/Kconfig
> > @@ -272,6 +272,10 @@ config GENERIC_ISA_DMA
> > def_bool y
> > depends on ISA_DMA_API
> >
> > +config GENERIC_CSUM
> > + bool
> > + default y if KMSAN || KASAN
>
> When would it be desirable to use GENERIC_CSUM without KMSAN or KASAN?
>
> If it is generally a bad idea to disable GENERIC_CSUM with KMSAN and
> KASAN, does it make more sense to just remove this config option and
> unconditionally (if CONFIG_K{A,M}SAN) use the asm-generic version in
> checksum.h?
>
> This would simplify this patch and avoid introducing a config option
> that will likely never be set explicitly.
There is already a similar option for MIPS and m68k, so I think it's
better to be consistent with them.
> > config GENERIC_BUG
> > def_bool y
> > depends on BUG
> > diff --git a/arch/x86/include/asm/checksum.h b/arch/x86/include/asm/checksum.h
> > index d79d1e622dcf..ab3464cbce26 100644
> > --- a/arch/x86/include/asm/checksum.h
> > +++ b/arch/x86/include/asm/checksum.h
> > @@ -1,6 +1,10 @@
> > /* SPDX-License-Identifier: GPL-2.0 */
> > -#ifdef CONFIG_X86_32
> > -# include <asm/checksum_32.h>
> > +#ifdef CONFIG_GENERIC_CSUM
> > +# include <asm-generic/checksum.h>
> > #else
> > -# include <asm/checksum_64.h>
> > +# ifdef CONFIG_X86_32
> > +# include <asm/checksum_32.h>
> > +# else
> > +# include <asm/checksum_64.h>
> > +# endif
> > #endif
> > diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
> > index 5246db42de45..bca9031de9ff 100644
> > --- a/arch/x86/lib/Makefile
> > +++ b/arch/x86/lib/Makefile
> > @@ -55,7 +55,9 @@ endif
> > lib-$(CONFIG_X86_USE_3DNOW) += mmx_32.o
> > else
> > obj-y += iomap_copy_64.o
> > +ifneq ($(CONFIG_GENERIC_CSUM),y)
> > lib-y += csum-partial_64.o csum-copy_64.o csum-wrappers_64.o
> > +endif
> > lib-y += clear_page_64.o copy_page_64.o
> > lib-y += memmove_64.o memset_64.o
> > lib-y += copy_user_64.o
>
> Thanks,
> -- Marco
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure
2019-11-22 11:25 [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure glider
` (34 preceding siblings ...)
2019-11-22 11:26 ` [PATCH RFC v3 36/36] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN glider
@ 2019-11-29 14:39 ` Marco Elver
2019-12-02 16:02 ` Alexander Potapenko
35 siblings, 1 reply; 120+ messages in thread
From: Marco Elver @ 2019-11-29 14:39 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Dmitry Vyukov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Jens Axboe, Marek Szyprowski, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S. Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Vegard Nossum, Wolfram Sang, Linux Memory Management List
On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
>
> KernelMemorySanitizer (KMSAN) is a detector of errors related to uses of
> uninitialized memory. It relies on compile-time Clang instrumentation
> (similar to MSan in the userspace:
> https://clang.llvm.org/docs/MemorySanitizer.html)
> and tracks the state of every bit of kernel memory, being able to report
> an error if uninitialized value is used in a condition, dereferenced or
> copied to userspace, USB or network.
>
> KMSAN has reported more than 200 bugs in the past two years, most of
> them with the help of syzkaller (http://syzkaller.appspot.com).
>
> The proposed patchset contains KMSAN runtime implementation together
> with small changes to other subsystems needed to make KMSAN work.
> The latter changes fall into several categories:
> - nice-to-have features that are independent from KMSAN but simplify
> its implementation (stackdepot changes, CONFIG_GENERIC_CSUM etc.);
> - Kconfig changes that prohibit options incompatible with KMSAN;
> - calls to KMSAN runtime functions that help KMSAN do the bookkeeping
> (e.g. tell it to allocate, copy or delete the metadata);
> - calls to KMSAN runtime functions that tell KMSAN to check memory
> escaping the kernel for uninitialized values. These are required to
> increase the number of true positive error reports;
> - calls to runtime functions that tell KMSAN to ignore certain memory
> ranges to avoid false negative reports. Most certainly there can be
> better ways to deal with every such report.
>
> This patchset allows one to boot and run a defconfig+KMSAN kernel on a QEMU
> without known major false positives. It however doesn't guarantee there
> are no false positives in drivers of certain devices or less tested
> subsystems, although KMSAN is actively tested on syzbot with quite a
> rich config.
>
> One may find it handy to review these patches in Gerrit:
> https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/1081
> I've ensured the Change-Id: tags stay away from commit descriptions.
>
> The patchset was generated relative to Linux v5.4-rc5.
>
> I also apologize for not sending every patch in the previous series
> to all recipients of patches from that series.
>
> Note: checkpatch.pl complains a lot about the use of BUG_ON in KMSAN
> source. I don't have a strong opinion on this, but KMSAN is a debugging
> tool, so any runtime invariant violation in it renders the tool useless.
> Therefore it doesn't make much sense to not terminate after a bug in
> KMSAN.
>
> Alexander Potapenko (36):
> stackdepot: check depot_index before accessing the stack slab
> stackdepot: build with -fno-builtin
> kasan: stackdepot: move filter_irq_stacks() to stackdepot.c
> stackdepot: reserve 5 extra bits in depot_stack_handle_t
> kmsan: add ReST documentation
> kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW
> kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__
> kmsan: reduce vmalloc space
> kmsan: add KMSAN bits to struct page and struct task_struct
> kmsan: add KMSAN runtime
> kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot
> kmsan: define READ_ONCE_NOCHECK()
> kmsan: make READ_ONCE_TASK_STACK() return initialized values
> kmsan: x86: sync metadata pages on page fault
> kmsan: add tests for KMSAN
> crypto: kmsan: disable accelerated configs under KMSAN
> kmsan: x86: disable UNWINDER_ORC under KMSAN
> kmsan: disable LOCK_DEBUGGING_SUPPORT
> kmsan: x86/asm: add KMSAN hooks to entry_64.S
> kmsan: x86: increase stack sizes in KMSAN builds
> kmsan: disable KMSAN instrumentation for certain kernel parts
> kmsan: mm: call KMSAN hooks from SLUB code
> kmsan: call KMSAN hooks where needed
> kmsan: disable instrumentation of certain functions
> kmsan: unpoison |tlb| in arch_tlb_gather_mmu()
> kmsan: use __msan_memcpy() where possible.
> kmsan: hooks for copy_to_user() and friends
> kmsan: enable KMSAN builds
> kmsan: handle /dev/[u]random
> kmsan: virtio: check/unpoison scatterlist in vring_map_one_sg()
> kmsan: disable strscpy() optimization under KMSAN
> kmsan: add iomap support
> kmsan: dma: unpoison memory mapped by dma_direct_map_page()
> kmsan: disable physical page merging in biovec
> kmsan: ext4: skip block merging logic in ext4_mpage_readpages for
> KMSAN
> net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for
> KASAN/KMSAN
>
> To: Alexander Potapenko <glider@google.com>
> Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Andrey Konovalov <andreyknvl@google.com>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Christoph Hellwig <hch@infradead.org>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Darrick J. Wong <darrick.wong@oracle.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Dmitry Torokhov <dmitry.torokhov@gmail.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Eric Biggers <ebiggers@google.com>
> Cc: Eric Dumazet <edumazet@google.com>
> Cc: Eric Van Hensbergen <ericvh@gmail.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
> Cc: Ilya Leoshkevich <iii@linux.ibm.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Jason Wang <jasowang@redhat.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Marco Elver <elver@google.com>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Martin K. Petersen <martin.petersen@oracle.com>
> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Cc: Matthew Wilcox <willy@infradead.org>
> Cc: "Michael S. Tsirkin" <mst@redhat.com>
> Cc: Michal Simek <monstr@monstr.eu>
> Cc: Petr Mladek <pmladek@suse.com>
> Cc: Qian Cai <cai@lca.pw>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Takashi Iwai <tiwai@suse.com>
> Cc: "Theodore Ts'o" <tytso@mit.edu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Vasily Gorbik <gor@linux.ibm.com>
> Cc: Vegard Nossum <vegard.nossum@oracle.com>
> Cc: Wolfram Sang <wsa@the-dreams.de>
> Cc: linux-mm@kvack.org
>
> Documentation/dev-tools/index.rst | 1 +
> Documentation/dev-tools/kmsan.rst | 418 ++++++++++++++++++
> Makefile | 3 +-
> arch/x86/Kconfig | 5 +
> arch/x86/Kconfig.debug | 3 +
> arch/x86/boot/Makefile | 2 +
> arch/x86/boot/compressed/Makefile | 2 +
> arch/x86/boot/compressed/misc.h | 1 +
> arch/x86/entry/common.c | 1 +
> arch/x86/entry/entry_64.S | 16 +
> arch/x86/entry/vdso/Makefile | 3 +
> arch/x86/include/asm/checksum.h | 10 +-
> arch/x86/include/asm/irq_regs.h | 1 +
> arch/x86/include/asm/kmsan.h | 117 +++++
> arch/x86/include/asm/page_64.h | 13 +
> arch/x86/include/asm/page_64_types.h | 12 +-
> arch/x86/include/asm/pgtable_64_types.h | 15 +
> arch/x86/include/asm/string_64.h | 9 +-
> arch/x86/include/asm/syscall_wrapper.h | 1 +
> arch/x86/include/asm/uaccess.h | 12 +
> arch/x86/include/asm/unwind.h | 9 +-
> arch/x86/kernel/Makefile | 4 +
> arch/x86/kernel/apic/apic.c | 1 +
> arch/x86/kernel/cpu/Makefile | 1 +
> arch/x86/kernel/dumpstack_64.c | 1 +
> arch/x86/kernel/process_64.c | 5 +
> arch/x86/kernel/traps.c | 12 +-
> arch/x86/kernel/uprobes.c | 7 +-
> arch/x86/lib/Makefile | 2 +
> arch/x86/mm/Makefile | 2 +
> arch/x86/mm/fault.c | 20 +
> arch/x86/mm/ioremap.c | 3 +
> arch/x86/realmode/rm/Makefile | 2 +
> block/blk.h | 7 +
> crypto/Kconfig | 52 +++
> drivers/char/random.c | 6 +
> drivers/firmware/efi/libstub/Makefile | 1 +
> drivers/usb/core/urb.c | 2 +
> drivers/virtio/virtio_ring.c | 10 +-
> fs/ext4/readpage.c | 11 +
> include/asm-generic/cacheflush.h | 7 +-
> include/asm-generic/uaccess.h | 12 +-
> include/linux/compiler-clang.h | 8 +
> include/linux/compiler-gcc.h | 5 +
> include/linux/compiler.h | 13 +-
> include/linux/gfp.h | 4 +-
> include/linux/highmem.h | 4 +
> include/linux/kmsan-checks.h | 122 +++++
> include/linux/kmsan.h | 143 ++++++
> include/linux/mm_types.h | 9 +
> include/linux/sched.h | 5 +
> include/linux/stackdepot.h | 10 +
> include/linux/string.h | 2 +
> include/linux/uaccess.h | 32 +-
> init/main.c | 3 +
> kernel/Makefile | 1 +
> kernel/dma/direct.c | 1 +
> kernel/exit.c | 2 +
> kernel/fork.c | 2 +
> kernel/kthread.c | 2 +
> kernel/printk/printk.c | 6 +
> kernel/profile.c | 1 +
> kernel/sched/core.c | 6 +
> kernel/softirq.c | 5 +
> lib/Kconfig.debug | 5 +
> lib/Kconfig.kmsan | 22 +
> lib/Makefile | 6 +
> lib/iomap.c | 40 ++
> lib/ioremap.c | 5 +
> lib/iov_iter.c | 6 +
> lib/stackdepot.c | 69 ++-
> lib/string.c | 5 +-
> lib/test_kmsan.c | 231 ++++++++++
> lib/usercopy.c | 6 +-
> mm/Makefile | 1 +
> mm/compaction.c | 9 +
> mm/gup.c | 3 +
> mm/kasan/common.c | 23 -
> mm/kmsan/Makefile | 4 +
> mm/kmsan/kmsan.c | 563 ++++++++++++++++++++++++
> mm/kmsan/kmsan.h | 146 ++++++
> mm/kmsan/kmsan_entry.c | 118 +++++
> mm/kmsan/kmsan_hooks.c | 422 ++++++++++++++++++
> mm/kmsan/kmsan_init.c | 88 ++++
> mm/kmsan/kmsan_instr.c | 259 +++++++++++
> mm/kmsan/kmsan_report.c | 133 ++++++
> mm/kmsan/kmsan_shadow.c | 543 +++++++++++++++++++++++
> mm/kmsan/kmsan_shadow.h | 30 ++
> mm/memory.c | 2 +
> mm/mmu_gather.c | 10 +
> mm/page_alloc.c | 16 +
> mm/slub.c | 34 +-
> mm/vmalloc.c | 23 +-
> net/sched/sch_generic.c | 2 +
> scripts/Makefile.kmsan | 12 +
> scripts/Makefile.lib | 6 +
> 96 files changed, 3983 insertions(+), 67 deletions(-)
> create mode 100644 Documentation/dev-tools/kmsan.rst
> create mode 100644 arch/x86/include/asm/kmsan.h
> create mode 100644 include/linux/kmsan-checks.h
> create mode 100644 include/linux/kmsan.h
> create mode 100644 lib/Kconfig.kmsan
> create mode 100644 lib/test_kmsan.c
> create mode 100644 mm/kmsan/Makefile
> create mode 100644 mm/kmsan/kmsan.c
> create mode 100644 mm/kmsan/kmsan.h
> create mode 100644 mm/kmsan/kmsan_entry.c
> create mode 100644 mm/kmsan/kmsan_hooks.c
> create mode 100644 mm/kmsan/kmsan_init.c
> create mode 100644 mm/kmsan/kmsan_instr.c
> create mode 100644 mm/kmsan/kmsan_report.c
> create mode 100644 mm/kmsan/kmsan_shadow.c
> create mode 100644 mm/kmsan/kmsan_shadow.h
> create mode 100644 scripts/Makefile.kmsan
There are currently lots of UACCESS warnings -- should the core
runtime functions be added to the whitelist in tools/objtool/check.c?
Thanks,
-- Marco
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure
2019-11-29 14:39 ` [PATCH RFC v3 00/36] Add KernelMemorySanitizer infrastructure Marco Elver
@ 2019-12-02 16:02 ` Alexander Potapenko
0 siblings, 0 replies; 120+ messages in thread
From: Alexander Potapenko @ 2019-12-02 16:02 UTC (permalink / raw)
To: Marco Elver
Cc: Alexander Viro, Andreas Dilger, Andrew Morton, Andrey Konovalov,
Andrey Ryabinin, Andy Lutomirski, Ard Biesheuvel, Arnd Bergmann,
Christoph Hellwig, Christoph Hellwig, Darrick J. Wong,
David S. Miller, Dmitry Torokhov, Dmitry Vyukov, Eric Biggers,
Eric Dumazet, Eric Van Hensbergen, Greg Kroah-Hartman,
Harry Wentland, Herbert Xu, Ilya Leoshkevich, Ingo Molnar,
Jason Wang, Jens Axboe, Marek Szyprowski, Mark Rutland,
Martin K. Petersen, Martin Schwidefsky, Matthew Wilcox,
Michael S. Tsirkin, Michal Simek, Petr Mladek, Qian Cai,
Randy Dunlap, Robin Murphy, Sergey Senozhatsky, Steven Rostedt,
Takashi Iwai, Theodore Ts'o, Thomas Gleixner, Vasily Gorbik,
Vegard Nossum, Wolfram Sang, Linux Memory Management List
On Fri, Nov 29, 2019 at 3:39 PM Marco Elver <elver@google.com> wrote:
>
> On Fri, 22 Nov 2019 at 12:26, <glider@google.com> wrote:
> >
> > KernelMemorySanitizer (KMSAN) is a detector of errors related to uses of
> > uninitialized memory. It relies on compile-time Clang instrumentation
> > (similar to MSan in the userspace:
> > https://clang.llvm.org/docs/MemorySanitizer.html)
> > and tracks the state of every bit of kernel memory, being able to report
> > an error if uninitialized value is used in a condition, dereferenced or
> > copied to userspace, USB or network.
> >
> > KMSAN has reported more than 200 bugs in the past two years, most of
> > them with the help of syzkaller (http://syzkaller.appspot.com).
> >
> > The proposed patchset contains KMSAN runtime implementation together
> > with small changes to other subsystems needed to make KMSAN work.
> > The latter changes fall into several categories:
> > - nice-to-have features that are independent from KMSAN but simplify
> > its implementation (stackdepot changes, CONFIG_GENERIC_CSUM etc.);
> > - Kconfig changes that prohibit options incompatible with KMSAN;
> > - calls to KMSAN runtime functions that help KMSAN do the bookkeeping
> > (e.g. tell it to allocate, copy or delete the metadata);
> > - calls to KMSAN runtime functions that tell KMSAN to check memory
> > escaping the kernel for uninitialized values. These are required to
> > increase the number of true positive error reports;
> > - calls to runtime functions that tell KMSAN to ignore certain memory
> > ranges to avoid false negative reports. Most certainly there can be
> > better ways to deal with every such report.
> >
> > This patchset allows one to boot and run a defconfig+KMSAN kernel on a QEMU
> > without known major false positives. It however doesn't guarantee there
> > are no false positives in drivers of certain devices or less tested
> > subsystems, although KMSAN is actively tested on syzbot with quite a
> > rich config.
> >
> > One may find it handy to review these patches in Gerrit:
> > https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/1081
> > I've ensured the Change-Id: tags stay away from commit descriptions.
> >
> > The patchset was generated relative to Linux v5.4-rc5.
> >
> > I also apologize for not sending every patch in the previous series
> > to all recipients of patches from that series.
> >
> > Note: checkpatch.pl complains a lot about the use of BUG_ON in KMSAN
> > source. I don't have a strong opinion on this, but KMSAN is a debugging
> > tool, so any runtime invariant violation in it renders the tool useless.
> > Therefore it doesn't make much sense to not terminate after a bug in
> > KMSAN.
> >
> > Alexander Potapenko (36):
> > stackdepot: check depot_index before accessing the stack slab
> > stackdepot: build with -fno-builtin
> > kasan: stackdepot: move filter_irq_stacks() to stackdepot.c
> > stackdepot: reserve 5 extra bits in depot_stack_handle_t
> > kmsan: add ReST documentation
> > kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW
> > kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__
> > kmsan: reduce vmalloc space
> > kmsan: add KMSAN bits to struct page and struct task_struct
> > kmsan: add KMSAN runtime
> > kmsan: stackdepot: don't allocate KMSAN metadata for stackdepot
> > kmsan: define READ_ONCE_NOCHECK()
> > kmsan: make READ_ONCE_TASK_STACK() return initialized values
> > kmsan: x86: sync metadata pages on page fault
> > kmsan: add tests for KMSAN
> > crypto: kmsan: disable accelerated configs under KMSAN
> > kmsan: x86: disable UNWINDER_ORC under KMSAN
> > kmsan: disable LOCK_DEBUGGING_SUPPORT
> > kmsan: x86/asm: add KMSAN hooks to entry_64.S
> > kmsan: x86: increase stack sizes in KMSAN builds
> > kmsan: disable KMSAN instrumentation for certain kernel parts
> > kmsan: mm: call KMSAN hooks from SLUB code
> > kmsan: call KMSAN hooks where needed
> > kmsan: disable instrumentation of certain functions
> > kmsan: unpoison |tlb| in arch_tlb_gather_mmu()
> > kmsan: use __msan_memcpy() where possible.
> > kmsan: hooks for copy_to_user() and friends
> > kmsan: enable KMSAN builds
> > kmsan: handle /dev/[u]random
> > kmsan: virtio: check/unpoison scatterlist in vring_map_one_sg()
> > kmsan: disable strscpy() optimization under KMSAN
> > kmsan: add iomap support
> > kmsan: dma: unpoison memory mapped by dma_direct_map_page()
> > kmsan: disable physical page merging in biovec
> > kmsan: ext4: skip block merging logic in ext4_mpage_readpages for
> > KMSAN
> > net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for
> > KASAN/KMSAN
> >
> > To: Alexander Potapenko <glider@google.com>
> > Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> > Cc: Andreas Dilger <adilger.kernel@dilger.ca>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Andrey Konovalov <andreyknvl@google.com>
> > Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> > Cc: Andy Lutomirski <luto@kernel.org>
> > Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Cc: Arnd Bergmann <arnd@arndb.de>
> > Cc: Christoph Hellwig <hch@infradead.org>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: Darrick J. Wong <darrick.wong@oracle.com>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: Dmitry Torokhov <dmitry.torokhov@gmail.com>
> > Cc: Dmitry Vyukov <dvyukov@google.com>
> > Cc: Eric Biggers <ebiggers@google.com>
> > Cc: Eric Dumazet <edumazet@google.com>
> > Cc: Eric Van Hensbergen <ericvh@gmail.com>
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Harry Wentland <harry.wentland@amd.com>
> > Cc: Herbert Xu <herbert@gondor.apana.org.au>
> > Cc: Ilya Leoshkevich <iii@linux.ibm.com>
> > Cc: Ingo Molnar <mingo@elte.hu>
> > Cc: Jason Wang <jasowang@redhat.com>
> > Cc: Jens Axboe <axboe@kernel.dk>
> > Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> > Cc: Marco Elver <elver@google.com>
> > Cc: Mark Rutland <mark.rutland@arm.com>
> > Cc: Martin K. Petersen <martin.petersen@oracle.com>
> > Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> > Cc: Matthew Wilcox <willy@infradead.org>
> > Cc: "Michael S. Tsirkin" <mst@redhat.com>
> > Cc: Michal Simek <monstr@monstr.eu>
> > Cc: Petr Mladek <pmladek@suse.com>
> > Cc: Qian Cai <cai@lca.pw>
> > Cc: Randy Dunlap <rdunlap@infradead.org>
> > Cc: Robin Murphy <robin.murphy@arm.com>
> > Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> > Cc: Takashi Iwai <tiwai@suse.com>
> > Cc: "Theodore Ts'o" <tytso@mit.edu>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Vasily Gorbik <gor@linux.ibm.com>
> > Cc: Vegard Nossum <vegard.nossum@oracle.com>
> > Cc: Wolfram Sang <wsa@the-dreams.de>
> > Cc: linux-mm@kvack.org
> >
> > Documentation/dev-tools/index.rst | 1 +
> > Documentation/dev-tools/kmsan.rst | 418 ++++++++++++++++++
> > Makefile | 3 +-
> > arch/x86/Kconfig | 5 +
> > arch/x86/Kconfig.debug | 3 +
> > arch/x86/boot/Makefile | 2 +
> > arch/x86/boot/compressed/Makefile | 2 +
> > arch/x86/boot/compressed/misc.h | 1 +
> > arch/x86/entry/common.c | 1 +
> > arch/x86/entry/entry_64.S | 16 +
> > arch/x86/entry/vdso/Makefile | 3 +
> > arch/x86/include/asm/checksum.h | 10 +-
> > arch/x86/include/asm/irq_regs.h | 1 +
> > arch/x86/include/asm/kmsan.h | 117 +++++
> > arch/x86/include/asm/page_64.h | 13 +
> > arch/x86/include/asm/page_64_types.h | 12 +-
> > arch/x86/include/asm/pgtable_64_types.h | 15 +
> > arch/x86/include/asm/string_64.h | 9 +-
> > arch/x86/include/asm/syscall_wrapper.h | 1 +
> > arch/x86/include/asm/uaccess.h | 12 +
> > arch/x86/include/asm/unwind.h | 9 +-
> > arch/x86/kernel/Makefile | 4 +
> > arch/x86/kernel/apic/apic.c | 1 +
> > arch/x86/kernel/cpu/Makefile | 1 +
> > arch/x86/kernel/dumpstack_64.c | 1 +
> > arch/x86/kernel/process_64.c | 5 +
> > arch/x86/kernel/traps.c | 12 +-
> > arch/x86/kernel/uprobes.c | 7 +-
> > arch/x86/lib/Makefile | 2 +
> > arch/x86/mm/Makefile | 2 +
> > arch/x86/mm/fault.c | 20 +
> > arch/x86/mm/ioremap.c | 3 +
> > arch/x86/realmode/rm/Makefile | 2 +
> > block/blk.h | 7 +
> > crypto/Kconfig | 52 +++
> > drivers/char/random.c | 6 +
> > drivers/firmware/efi/libstub/Makefile | 1 +
> > drivers/usb/core/urb.c | 2 +
> > drivers/virtio/virtio_ring.c | 10 +-
> > fs/ext4/readpage.c | 11 +
> > include/asm-generic/cacheflush.h | 7 +-
> > include/asm-generic/uaccess.h | 12 +-
> > include/linux/compiler-clang.h | 8 +
> > include/linux/compiler-gcc.h | 5 +
> > include/linux/compiler.h | 13 +-
> > include/linux/gfp.h | 4 +-
> > include/linux/highmem.h | 4 +
> > include/linux/kmsan-checks.h | 122 +++++
> > include/linux/kmsan.h | 143 ++++++
> > include/linux/mm_types.h | 9 +
> > include/linux/sched.h | 5 +
> > include/linux/stackdepot.h | 10 +
> > include/linux/string.h | 2 +
> > include/linux/uaccess.h | 32 +-
> > init/main.c | 3 +
> > kernel/Makefile | 1 +
> > kernel/dma/direct.c | 1 +
> > kernel/exit.c | 2 +
> > kernel/fork.c | 2 +
> > kernel/kthread.c | 2 +
> > kernel/printk/printk.c | 6 +
> > kernel/profile.c | 1 +
> > kernel/sched/core.c | 6 +
> > kernel/softirq.c | 5 +
> > lib/Kconfig.debug | 5 +
> > lib/Kconfig.kmsan | 22 +
> > lib/Makefile | 6 +
> > lib/iomap.c | 40 ++
> > lib/ioremap.c | 5 +
> > lib/iov_iter.c | 6 +
> > lib/stackdepot.c | 69 ++-
> > lib/string.c | 5 +-
> > lib/test_kmsan.c | 231 ++++++++++
> > lib/usercopy.c | 6 +-
> > mm/Makefile | 1 +
> > mm/compaction.c | 9 +
> > mm/gup.c | 3 +
> > mm/kasan/common.c | 23 -
> > mm/kmsan/Makefile | 4 +
> > mm/kmsan/kmsan.c | 563 ++++++++++++++++++++++++
> > mm/kmsan/kmsan.h | 146 ++++++
> > mm/kmsan/kmsan_entry.c | 118 +++++
> > mm/kmsan/kmsan_hooks.c | 422 ++++++++++++++++++
> > mm/kmsan/kmsan_init.c | 88 ++++
> > mm/kmsan/kmsan_instr.c | 259 +++++++++++
> > mm/kmsan/kmsan_report.c | 133 ++++++
> > mm/kmsan/kmsan_shadow.c | 543 +++++++++++++++++++++++
> > mm/kmsan/kmsan_shadow.h | 30 ++
> > mm/memory.c | 2 +
> > mm/mmu_gather.c | 10 +
> > mm/page_alloc.c | 16 +
> > mm/slub.c | 34 +-
> > mm/vmalloc.c | 23 +-
> > net/sched/sch_generic.c | 2 +
> > scripts/Makefile.kmsan | 12 +
> > scripts/Makefile.lib | 6 +
> > 96 files changed, 3983 insertions(+), 67 deletions(-)
> > create mode 100644 Documentation/dev-tools/kmsan.rst
> > create mode 100644 arch/x86/include/asm/kmsan.h
> > create mode 100644 include/linux/kmsan-checks.h
> > create mode 100644 include/linux/kmsan.h
> > create mode 100644 lib/Kconfig.kmsan
> > create mode 100644 lib/test_kmsan.c
> > create mode 100644 mm/kmsan/Makefile
> > create mode 100644 mm/kmsan/kmsan.c
> > create mode 100644 mm/kmsan/kmsan.h
> > create mode 100644 mm/kmsan/kmsan_entry.c
> > create mode 100644 mm/kmsan/kmsan_hooks.c
> > create mode 100644 mm/kmsan/kmsan_init.c
> > create mode 100644 mm/kmsan/kmsan_instr.c
> > create mode 100644 mm/kmsan/kmsan_report.c
> > create mode 100644 mm/kmsan/kmsan_shadow.c
> > create mode 100644 mm/kmsan/kmsan_shadow.h
> > create mode 100644 scripts/Makefile.kmsan
>
> There are currently lots of UACCESS warnings -- should the core
> runtime functions be added to the whitelist in tools/objtool/check.c?
I've tried to do this properly, but failed.
What I consider to be an intuitive solution is to add KMSAN interface
functions (e.g. __msan_metadata_ptr_for_load_X) to the whitelist in
tools/objtool/check.c.
None of these functions is expected to touch userspace memory, but
they can be called in the uaccess context, as the compiler adds them
to every memory access.
Turns out however it's not enough to whitelist those functions, as
they are viral: after whitelisting them I get warnings about their
callees.
The list of functions called by e.g. kmsan_get_shadow_origin_ptr()
include __warn_printk() and other non-KMSAN stuff, so I have to call
user_access_save()/user_access_restore() to keep those warnings from
spreading.
But this slows the whole runtime down by ~50% for no reason.
> Thanks,
> -- Marco
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
^ permalink raw reply [flat|nested] 120+ messages in thread