* [PATCH v1 0/8] SLAB support for KASAN
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
This patch set implements SLAB support for KASAN
Unlike SLUB, SLAB doesn't store allocation/deallocation stacks for heap
objects, therefore we reimplement this feature in mm/kasan/stackdepot.c.
The intention is to ultimately switch SLUB to use this implementation as
well, which will remove the dependency on SLUB_DEBUG.
Also neither SLUB nor SLAB delay the reuse of freed memory chunks, which
is necessary for better detection of use-after-free errors. We introduce
memory quarantine (mm/kasan/quarantine.c), which allows delayed reuse of
deallocated memory.
Alexander Potapenko (8):
kasan: Change the behavior of kmalloc_large_oob_right test
mm, kasan: SLAB support
mm, kasan: Added GFP flags to KASAN API
arch, ftrace: For KASAN put hard/soft IRQ entries into separate
sections
mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
kasan: Test fix: Warn if the UAF could not be detected in kmalloc_uaf2
kasan: Changed kmalloc_large_oob_right, added
kmalloc_pagealloc_oob_right
mm: kasan: Initial memory quarantine implementation
Documentation/kasan.txt | 5 +-
arch/arm/kernel/vmlinux.lds.S | 1 +
arch/arm64/kernel/vmlinux.lds.S | 1 +
arch/blackfin/kernel/vmlinux.lds.S | 1 +
arch/c6x/kernel/vmlinux.lds.S | 1 +
arch/metag/kernel/vmlinux.lds.S | 1 +
arch/microblaze/kernel/vmlinux.lds.S | 1 +
arch/mips/kernel/vmlinux.lds.S | 1 +
arch/nios2/kernel/vmlinux.lds.S | 1 +
arch/openrisc/kernel/vmlinux.lds.S | 1 +
arch/parisc/kernel/vmlinux.lds.S | 1 +
arch/powerpc/kernel/vmlinux.lds.S | 1 +
arch/s390/kernel/vmlinux.lds.S | 1 +
arch/sh/kernel/vmlinux.lds.S | 1 +
arch/sparc/kernel/vmlinux.lds.S | 1 +
arch/tile/kernel/vmlinux.lds.S | 1 +
arch/x86/kernel/Makefile | 1 +
arch/x86/kernel/vmlinux.lds.S | 1 +
include/asm-generic/vmlinux.lds.h | 12 +-
include/linux/ftrace.h | 31 ++--
include/linux/kasan.h | 63 +++++---
include/linux/slab.h | 6 +
include/linux/slab_def.h | 14 ++
include/linux/slub_def.h | 11 ++
kernel/softirq.c | 3 +-
lib/Kconfig.kasan | 4 +-
lib/test_kasan.c | 66 +++++++-
mm/Makefile | 1 +
mm/kasan/Makefile | 3 +
mm/kasan/kasan.c | 221 +++++++++++++++++++++++++--
mm/kasan/kasan.h | 52 +++++++
mm/kasan/quarantine.c | 284 +++++++++++++++++++++++++++++++++++
mm/kasan/report.c | 68 +++++++--
mm/kasan/stackdepot.c | 236 +++++++++++++++++++++++++++++
mm/mempool.c | 23 +--
mm/page_alloc.c | 2 +-
mm/slab.c | 56 ++++++-
mm/slab.h | 4 +
mm/slab_common.c | 8 +-
mm/slub.c | 21 +--
40 files changed, 1122 insertions(+), 89 deletions(-)
create mode 100644 mm/kasan/quarantine.c
create mode 100644 mm/kasan/stackdepot.c
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply [flat|nested] 78+ messages in thread
* [PATCH v1 0/8] SLAB support for KASAN
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
This patch set implements SLAB support for KASAN
Unlike SLUB, SLAB doesn't store allocation/deallocation stacks for heap
objects, therefore we reimplement this feature in mm/kasan/stackdepot.c.
The intention is to ultimately switch SLUB to use this implementation as
well, which will remove the dependency on SLUB_DEBUG.
Also neither SLUB nor SLAB delay the reuse of freed memory chunks, which
is necessary for better detection of use-after-free errors. We introduce
memory quarantine (mm/kasan/quarantine.c), which allows delayed reuse of
deallocated memory.
Alexander Potapenko (8):
kasan: Change the behavior of kmalloc_large_oob_right test
mm, kasan: SLAB support
mm, kasan: Added GFP flags to KASAN API
arch, ftrace: For KASAN put hard/soft IRQ entries into separate
sections
mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
kasan: Test fix: Warn if the UAF could not be detected in kmalloc_uaf2
kasan: Changed kmalloc_large_oob_right, added
kmalloc_pagealloc_oob_right
mm: kasan: Initial memory quarantine implementation
Documentation/kasan.txt | 5 +-
arch/arm/kernel/vmlinux.lds.S | 1 +
arch/arm64/kernel/vmlinux.lds.S | 1 +
arch/blackfin/kernel/vmlinux.lds.S | 1 +
arch/c6x/kernel/vmlinux.lds.S | 1 +
arch/metag/kernel/vmlinux.lds.S | 1 +
arch/microblaze/kernel/vmlinux.lds.S | 1 +
arch/mips/kernel/vmlinux.lds.S | 1 +
arch/nios2/kernel/vmlinux.lds.S | 1 +
arch/openrisc/kernel/vmlinux.lds.S | 1 +
arch/parisc/kernel/vmlinux.lds.S | 1 +
arch/powerpc/kernel/vmlinux.lds.S | 1 +
arch/s390/kernel/vmlinux.lds.S | 1 +
arch/sh/kernel/vmlinux.lds.S | 1 +
arch/sparc/kernel/vmlinux.lds.S | 1 +
arch/tile/kernel/vmlinux.lds.S | 1 +
arch/x86/kernel/Makefile | 1 +
arch/x86/kernel/vmlinux.lds.S | 1 +
include/asm-generic/vmlinux.lds.h | 12 +-
include/linux/ftrace.h | 31 ++--
include/linux/kasan.h | 63 +++++---
include/linux/slab.h | 6 +
include/linux/slab_def.h | 14 ++
include/linux/slub_def.h | 11 ++
kernel/softirq.c | 3 +-
lib/Kconfig.kasan | 4 +-
lib/test_kasan.c | 66 +++++++-
mm/Makefile | 1 +
mm/kasan/Makefile | 3 +
mm/kasan/kasan.c | 221 +++++++++++++++++++++++++--
mm/kasan/kasan.h | 52 +++++++
mm/kasan/quarantine.c | 284 +++++++++++++++++++++++++++++++++++
mm/kasan/report.c | 68 +++++++--
mm/kasan/stackdepot.c | 236 +++++++++++++++++++++++++++++
mm/mempool.c | 23 +--
mm/page_alloc.c | 2 +-
mm/slab.c | 56 ++++++-
mm/slab.h | 4 +
mm/slab_common.c | 8 +-
mm/slub.c | 21 +--
40 files changed, 1122 insertions(+), 89 deletions(-)
create mode 100644 mm/kasan/quarantine.c
create mode 100644 mm/kasan/stackdepot.c
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
depending on which allocator (SLAB or SLUB) is being used
Signed-off-by: Alexander Potapenko <glider@google.com>
---
lib/test_kasan.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index c32f3b0..66dd92f 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
static noinline void __init kmalloc_large_oob_right(void)
{
char *ptr;
- size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
+ size_t size;
+
+ if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
+ /*
+ * We're using the SLAB allocator. Allocate a chunk that fits
+ * into a slab.
+ */
+ size = KMALLOC_MAX_CACHE_SIZE - 256;
+ } else {
+ /*
+ * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
+ * We're using the SLUB allocator. Allocate a chunk that does
+ * not fit into a slab to trigger the page allocator.
+ */
+ size = KMALLOC_MAX_CACHE_SIZE + 10;
+ }
pr_info("kmalloc large allocation: out-of-bounds to right\n");
ptr = kmalloc(size, GFP_KERNEL);
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
depending on which allocator (SLAB or SLUB) is being used
Signed-off-by: Alexander Potapenko <glider@google.com>
---
lib/test_kasan.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index c32f3b0..66dd92f 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
static noinline void __init kmalloc_large_oob_right(void)
{
char *ptr;
- size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
+ size_t size;
+
+ if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
+ /*
+ * We're using the SLAB allocator. Allocate a chunk that fits
+ * into a slab.
+ */
+ size = KMALLOC_MAX_CACHE_SIZE - 256;
+ } else {
+ /*
+ * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
+ * We're using the SLUB allocator. Allocate a chunk that does
+ * not fit into a slab to trigger the page allocator.
+ */
+ size = KMALLOC_MAX_CACHE_SIZE + 10;
+ }
pr_info("kmalloc large allocation: out-of-bounds to right\n");
ptr = kmalloc(size, GFP_KERNEL);
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 2/8] mm, kasan: SLAB support
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
This patch adds KASAN hooks to SLAB allocator.
This patch is based on the "mm: kasan: unified support for SLUB and
SLAB allocators" patch originally prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
Documentation/kasan.txt | 5 ++-
include/linux/kasan.h | 12 +++++++
include/linux/slab.h | 6 ++++
include/linux/slab_def.h | 14 ++++++++
include/linux/slub_def.h | 11 ++++++
lib/Kconfig.kasan | 4 ++-
mm/Makefile | 1 +
mm/kasan/kasan.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++
mm/kasan/kasan.h | 34 ++++++++++++++++++
mm/kasan/report.c | 59 +++++++++++++++++++++++++------
mm/slab.c | 46 +++++++++++++++++++++---
mm/slab_common.c | 2 +-
12 files changed, 264 insertions(+), 21 deletions(-)
diff --git a/Documentation/kasan.txt b/Documentation/kasan.txt
index aa1e0c9..7dd95b3 100644
--- a/Documentation/kasan.txt
+++ b/Documentation/kasan.txt
@@ -12,8 +12,7 @@ KASAN uses compile-time instrumentation for checking every memory access,
therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is
required for detection of out-of-bounds accesses to stack or global variables.
-Currently KASAN is supported only for x86_64 architecture and requires the
-kernel to be built with the SLUB allocator.
+Currently KASAN is supported only for x86_64 architecture.
1. Usage
========
@@ -27,7 +26,7 @@ inline are compiler instrumentation types. The former produces smaller binary
the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC
version 5.0 or later.
-Currently KASAN works only with the SLUB memory allocator.
+KASAN works with both SLUB and SLAB memory allocators.
For better bug detection and nicer reporting, enable CONFIG_STACKTRACE.
To disable instrumentation for specific files or directories, add a line
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 4b9f85c..4405a35 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -46,6 +46,9 @@ void kasan_unpoison_shadow(const void *address, size_t size);
void kasan_alloc_pages(struct page *page, unsigned int order);
void kasan_free_pages(struct page *page, unsigned int order);
+void kasan_cache_create(struct kmem_cache *cache, size_t *size,
+ unsigned long *flags);
+
void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
void kasan_poison_object_data(struct kmem_cache *cache, void *object);
@@ -59,6 +62,11 @@ void kasan_krealloc(const void *object, size_t new_size);
void kasan_slab_alloc(struct kmem_cache *s, void *object);
void kasan_slab_free(struct kmem_cache *s, void *object);
+struct kasan_cache {
+ int alloc_meta_offset;
+ int free_meta_offset;
+};
+
int kasan_module_alloc(void *addr, size_t size);
void kasan_free_shadow(const struct vm_struct *vm);
@@ -72,6 +80,10 @@ static inline void kasan_disable_current(void) {}
static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
static inline void kasan_free_pages(struct page *page, unsigned int order) {}
+static inline void kasan_cache_create(struct kmem_cache *cache,
+ size_t *size,
+ unsigned long *flags) {}
+
static inline void kasan_poison_slab(struct page *page) {}
static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
void *object) {}
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 3ffee74..92f8558 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -92,6 +92,12 @@
# define SLAB_ACCOUNT 0x00000000UL
#endif
+#ifdef CONFIG_KASAN
+#define SLAB_KASAN 0x08000000UL
+#else
+#define SLAB_KASAN 0x00000000UL
+#endif
+
/* The following flags affect the page allocator grouping pages by mobility */
#define SLAB_RECLAIM_ACCOUNT 0x00020000UL /* Objects are reclaimable */
#define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */
diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
index 33d0490..a25804d 100644
--- a/include/linux/slab_def.h
+++ b/include/linux/slab_def.h
@@ -72,8 +72,22 @@ struct kmem_cache {
#ifdef CONFIG_MEMCG_KMEM
struct memcg_cache_params memcg_params;
#endif
+#ifdef CONFIG_KASAN
+ struct kasan_cache kasan_info;
+#endif
struct kmem_cache_node *node[MAX_NUMNODES];
};
+static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
+ void *x) {
+ void *object = x - (x - page->s_mem) % cache->size;
+ void *last_object = page->s_mem + (cache->num - 1) * cache->size;
+
+ if (unlikely(object > last_object))
+ return last_object;
+ else
+ return object;
+}
+
#endif /* _LINUX_SLAB_DEF_H */
diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
index 3388511..c553dad 100644
--- a/include/linux/slub_def.h
+++ b/include/linux/slub_def.h
@@ -129,4 +129,15 @@ static inline void *virt_to_obj(struct kmem_cache *s,
void object_err(struct kmem_cache *s, struct page *page,
u8 *object, char *reason);
+static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
+ void *x) {
+ void *object = x - (x - page_address(page)) % cache->size;
+ void *last_object = page_address(page) +
+ (page->objects - 1) * cache->size;
+ if (unlikely(object > last_object))
+ return last_object;
+ else
+ return object;
+}
+
#endif /* _LINUX_SLUB_DEF_H */
diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
index 0fee5ac..0e4d2b3 100644
--- a/lib/Kconfig.kasan
+++ b/lib/Kconfig.kasan
@@ -5,7 +5,7 @@ if HAVE_ARCH_KASAN
config KASAN
bool "KASan: runtime memory debugger"
- depends on SLUB_DEBUG
+ depends on SLUB_DEBUG || (SLAB && !DEBUG_SLAB)
select CONSTRUCTORS
help
Enables kernel address sanitizer - runtime memory debugger,
@@ -16,6 +16,8 @@ config KASAN
This feature consumes about 1/8 of available memory and brings about
~x3 performance slowdown.
For better error detection enable CONFIG_STACKTRACE.
+ Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB
+ (the resulting kernel does not boot).
choice
prompt "Instrumentation type"
diff --git a/mm/Makefile b/mm/Makefile
index 2ed4319..d675b37 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -3,6 +3,7 @@
#
KASAN_SANITIZE_slab_common.o := n
+KASAN_SANITIZE_slab.o := n
KASAN_SANITIZE_slub.o := n
mmu-y := nommu.o
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index bc0a8d8..84305c2 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -314,6 +314,59 @@ void kasan_free_pages(struct page *page, unsigned int order)
KASAN_FREE_PAGE);
}
+#ifdef CONFIG_SLAB
+/*
+ * Adaptive redzone policy taken from the userspace AddressSanitizer runtime.
+ * For larger allocations larger redzones are used.
+ */
+static size_t optimal_redzone(size_t object_size)
+{
+ int rz =
+ object_size <= 64 - 16 ? 16 :
+ object_size <= 128 - 32 ? 32 :
+ object_size <= 512 - 64 ? 64 :
+ object_size <= 4096 - 128 ? 128 :
+ object_size <= (1 << 14) - 256 ? 256 :
+ object_size <= (1 << 15) - 512 ? 512 :
+ object_size <= (1 << 16) - 1024 ? 1024 : 2048;
+ return rz;
+}
+
+void kasan_cache_create(struct kmem_cache *cache, size_t *size,
+ unsigned long *flags)
+{
+ int redzone_adjust;
+ /* Make sure the adjusted size is still less than
+ * KMALLOC_MAX_CACHE_SIZE.
+ * TODO: this check is only useful for SLAB, but not SLUB. We'll need
+ * to skip it for SLUB when it starts using kasan_cache_create().
+ */
+ if (*size > KMALLOC_MAX_CACHE_SIZE -
+ sizeof(struct kasan_alloc_meta) -
+ sizeof(struct kasan_free_meta))
+ return;
+ *flags |= SLAB_KASAN;
+ /* Add alloc meta. */
+ cache->kasan_info.alloc_meta_offset = *size;
+ *size += sizeof(struct kasan_alloc_meta);
+
+ /* Add free meta. */
+ if (cache->flags & SLAB_DESTROY_BY_RCU || cache->ctor ||
+ cache->object_size < sizeof(struct kasan_free_meta)) {
+ cache->kasan_info.free_meta_offset = *size;
+ *size += sizeof(struct kasan_free_meta);
+ }
+ redzone_adjust = optimal_redzone(cache->object_size) -
+ (*size - cache->object_size);
+ if (redzone_adjust > 0)
+ *size += redzone_adjust;
+ *size = min(KMALLOC_MAX_CACHE_SIZE,
+ max(*size,
+ cache->object_size +
+ optimal_redzone(cache->object_size)));
+}
+#endif
+
void kasan_poison_slab(struct page *page)
{
kasan_poison_shadow(page_address(page),
@@ -331,8 +384,36 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object)
kasan_poison_shadow(object,
round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
KASAN_KMALLOC_REDZONE);
+#ifdef CONFIG_SLAB
+ if (cache->flags & SLAB_KASAN) {
+ struct kasan_alloc_meta *alloc_info =
+ get_alloc_info(cache, object);
+ alloc_info->state = KASAN_STATE_INIT;
+ }
+#endif
}
+static inline void set_track(struct kasan_track *track)
+{
+ track->cpu = raw_smp_processor_id();
+ track->pid = current->pid;
+ track->when = jiffies;
+}
+
+#ifdef CONFIG_SLAB
+struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
+ const void *object)
+{
+ return (void *)object + cache->kasan_info.alloc_meta_offset;
+}
+
+struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
+ const void *object)
+{
+ return (void *)object + cache->kasan_info.free_meta_offset;
+}
+#endif
+
void kasan_slab_alloc(struct kmem_cache *cache, void *object)
{
kasan_kmalloc(cache, object, cache->object_size);
@@ -366,6 +447,16 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size)
kasan_unpoison_shadow(object, size);
kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
KASAN_KMALLOC_REDZONE);
+#ifdef CONFIG_SLAB
+ if (cache->flags & SLAB_KASAN) {
+ struct kasan_alloc_meta *alloc_info =
+ get_alloc_info(cache, object);
+
+ alloc_info->state = KASAN_STATE_ALLOC;
+ alloc_info->alloc_size = size;
+ set_track(&alloc_info->track);
+ }
+#endif
}
EXPORT_SYMBOL(kasan_kmalloc);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 4f6c62e..7b9e4ab9 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -54,6 +54,40 @@ struct kasan_global {
#endif
};
+/**
+ * Structures to keep alloc and free tracks *
+ */
+
+enum kasan_state {
+ KASAN_STATE_INIT,
+ KASAN_STATE_ALLOC,
+ KASAN_STATE_FREE
+};
+
+struct kasan_track {
+ u64 cpu : 6; /* for NR_CPUS = 64 */
+ u64 pid : 16; /* 65536 processes */
+ u64 when : 42; /* ~140 years */
+};
+
+struct kasan_alloc_meta {
+ u32 state : 2; /* enum kasan_state */
+ u32 alloc_size : 30;
+ struct kasan_track track;
+};
+
+struct kasan_free_meta {
+ /* Allocator freelist pointer, unused by KASAN. */
+ void **freelist;
+ struct kasan_track track;
+};
+
+struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
+ const void *object);
+struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
+ const void *object);
+
+
static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
{
return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET)
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 12f222d..2bf7218 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -115,6 +115,42 @@ static inline bool init_task_stack_addr(const void *addr)
sizeof(init_thread_union.stack));
}
+static void print_track(struct kasan_track *track)
+{
+ pr_err("PID = %lu, CPU = %lu, timestamp = %lu\n", track->pid,
+ track->cpu, track->when);
+}
+
+static void print_object(struct kmem_cache *cache, void *object)
+{
+ struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object);
+ struct kasan_free_meta *free_info;
+
+ pr_err("Object at %p, in cache %s\n", object, cache->name);
+ if (!(cache->flags & SLAB_KASAN))
+ return;
+ switch (alloc_info->state) {
+ case KASAN_STATE_INIT:
+ pr_err("Object not allocated yet\n");
+ break;
+ case KASAN_STATE_ALLOC:
+ pr_err("Object allocated with size %u bytes.\n",
+ alloc_info->alloc_size);
+ pr_err("Allocation:\n");
+ print_track(&alloc_info->track);
+ break;
+ case KASAN_STATE_FREE:
+ pr_err("Object freed, allocated with size %u bytes\n",
+ alloc_info->alloc_size);
+ free_info = get_free_info(cache, object);
+ pr_err("Allocation:\n");
+ print_track(&alloc_info->track);
+ pr_err("Deallocation:\n");
+ print_track(&free_info->track);
+ break;
+ }
+}
+
static void print_address_description(struct kasan_access_info *info)
{
const void *addr = info->access_addr;
@@ -126,17 +162,14 @@ static void print_address_description(struct kasan_access_info *info)
if (PageSlab(page)) {
void *object;
struct kmem_cache *cache = page->slab_cache;
- void *last_object;
-
- object = virt_to_obj(cache, page_address(page), addr);
- last_object = page_address(page) +
- page->objects * cache->size;
-
- if (unlikely(object > last_object))
- object = last_object; /* we hit into padding */
-
+ object = nearest_obj(cache, page,
+ (void *)info->access_addr);
+#ifdef CONFIG_SLAB
+ print_object(cache, object);
+#else
object_err(cache, page, object,
- "kasan: bad access detected");
+ "kasan: bad access detected");
+#endif
return;
}
dump_page(page, "kasan: bad access detected");
@@ -146,8 +179,9 @@ static void print_address_description(struct kasan_access_info *info)
if (!init_task_stack_addr(addr))
pr_err("Address belongs to variable %pS\n", addr);
}
-
+#ifdef CONFIG_SLUB
dump_stack();
+#endif
}
static bool row_is_guilty(const void *row, const void *guilty)
@@ -233,6 +267,9 @@ static void kasan_report_error(struct kasan_access_info *info)
dump_stack();
} else {
print_error_description(info);
+#ifdef CONFIG_SLAB
+ dump_stack();
+#endif
print_address_description(info);
print_shadow_for_address(info->first_bad_addr);
}
diff --git a/mm/slab.c b/mm/slab.c
index 6ecc697..739b89d 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -2196,6 +2196,7 @@ __kmem_cache_create (struct kmem_cache *cachep, unsigned long flags)
#endif
#endif
+ kasan_cache_create(cachep, &size, &flags);
/*
* Determine if the slab management is 'on' or 'off' slab.
* (bootstrapping cannot cope with offslab caches so don't do
@@ -2503,8 +2504,13 @@ static void cache_init_objs(struct kmem_cache *cachep,
* cache which they are a constructor for. Otherwise, deadlock.
* They must also be threaded.
*/
- if (cachep->ctor && !(cachep->flags & SLAB_POISON))
+ if (cachep->ctor && !(cachep->flags & SLAB_POISON)) {
+ kasan_unpoison_object_data(cachep,
+ objp + obj_offset(cachep));
cachep->ctor(objp + obj_offset(cachep));
+ kasan_poison_object_data(
+ cachep, objp + obj_offset(cachep));
+ }
if (cachep->flags & SLAB_RED_ZONE) {
if (*dbg_redzone2(cachep, objp) != RED_INACTIVE)
@@ -2519,8 +2525,11 @@ static void cache_init_objs(struct kmem_cache *cachep,
kernel_map_pages(virt_to_page(objp),
cachep->size / PAGE_SIZE, 0);
#else
- if (cachep->ctor)
+ if (cachep->ctor) {
+ kasan_unpoison_object_data(cachep, objp);
cachep->ctor(objp);
+ kasan_poison_object_data(cachep, objp);
+ }
#endif
set_obj_status(page, i, OBJECT_FREE);
set_free_obj(page, i, i);
@@ -2650,6 +2659,7 @@ static int cache_grow(struct kmem_cache *cachep,
slab_map_pages(cachep, page, freelist);
+ kasan_poison_slab(page);
cache_init_objs(cachep, page);
if (gfpflags_allow_blocking(local_flags))
@@ -3364,7 +3374,10 @@ free_done:
static inline void __cache_free(struct kmem_cache *cachep, void *objp,
unsigned long caller)
{
- struct array_cache *ac = cpu_cache_get(cachep);
+ struct array_cache *ac;
+
+ kasan_slab_free(cachep, objp);
+ ac = cpu_cache_get(cachep);
check_irq_off();
kmemleak_free_recursive(objp, cachep->flags);
@@ -3403,6 +3416,8 @@ static inline void __cache_free(struct kmem_cache *cachep, void *objp,
void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags)
{
void *ret = slab_alloc(cachep, flags, _RET_IP_);
+ if (ret)
+ kasan_slab_alloc(cachep, ret);
trace_kmem_cache_alloc(_RET_IP_, ret,
cachep->object_size, cachep->size, flags);
@@ -3432,6 +3447,8 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size)
ret = slab_alloc(cachep, flags, _RET_IP_);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
trace_kmalloc(_RET_IP_, ret,
size, cachep->size, flags);
return ret;
@@ -3455,6 +3472,8 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid)
{
void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
+ if (ret)
+ kasan_slab_alloc(cachep, ret);
trace_kmem_cache_alloc_node(_RET_IP_, ret,
cachep->object_size, cachep->size,
flags, nodeid);
@@ -3473,6 +3492,8 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep,
ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
trace_kmalloc_node(_RET_IP_, ret,
size, cachep->size,
flags, nodeid);
@@ -3485,11 +3506,16 @@ static __always_inline void *
__do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller)
{
struct kmem_cache *cachep;
+ void *ret;
cachep = kmalloc_slab(size, flags);
if (unlikely(ZERO_OR_NULL_PTR(cachep)))
return cachep;
- return kmem_cache_alloc_node_trace(cachep, flags, node, size);
+ ret = kmem_cache_alloc_node_trace(cachep, flags, node, size);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
+
+ return ret;
}
void *__kmalloc_node(size_t size, gfp_t flags, int node)
@@ -3523,6 +3549,8 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags,
return cachep;
ret = slab_alloc(cachep, flags, caller);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
trace_kmalloc(caller, ret,
size, cachep->size, flags);
@@ -4240,10 +4268,18 @@ module_init(slab_proc_init);
*/
size_t ksize(const void *objp)
{
+ size_t size;
+
BUG_ON(!objp);
if (unlikely(objp == ZERO_SIZE_PTR))
return 0;
- return virt_to_cache(objp)->object_size;
+ size = virt_to_cache(objp)->object_size;
+ /* We assume that ksize callers could use whole allocated area,
+ * so we need to unpoison this area.
+ */
+ kasan_krealloc(objp, size);
+
+ return size;
}
EXPORT_SYMBOL(ksize);
diff --git a/mm/slab_common.c b/mm/slab_common.c
index e016178..8d2531d 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -35,7 +35,7 @@ struct kmem_cache *kmem_cache;
*/
#define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
SLAB_TRACE | SLAB_DESTROY_BY_RCU | SLAB_NOLEAKTRACE | \
- SLAB_FAILSLAB)
+ SLAB_FAILSLAB | SLAB_KASAN)
#define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
SLAB_NOTRACK | SLAB_ACCOUNT)
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
This patch adds KASAN hooks to SLAB allocator.
This patch is based on the "mm: kasan: unified support for SLUB and
SLAB allocators" patch originally prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
Documentation/kasan.txt | 5 ++-
include/linux/kasan.h | 12 +++++++
include/linux/slab.h | 6 ++++
include/linux/slab_def.h | 14 ++++++++
include/linux/slub_def.h | 11 ++++++
lib/Kconfig.kasan | 4 ++-
mm/Makefile | 1 +
mm/kasan/kasan.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++
mm/kasan/kasan.h | 34 ++++++++++++++++++
mm/kasan/report.c | 59 +++++++++++++++++++++++++------
mm/slab.c | 46 +++++++++++++++++++++---
mm/slab_common.c | 2 +-
12 files changed, 264 insertions(+), 21 deletions(-)
diff --git a/Documentation/kasan.txt b/Documentation/kasan.txt
index aa1e0c9..7dd95b3 100644
--- a/Documentation/kasan.txt
+++ b/Documentation/kasan.txt
@@ -12,8 +12,7 @@ KASAN uses compile-time instrumentation for checking every memory access,
therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is
required for detection of out-of-bounds accesses to stack or global variables.
-Currently KASAN is supported only for x86_64 architecture and requires the
-kernel to be built with the SLUB allocator.
+Currently KASAN is supported only for x86_64 architecture.
1. Usage
========
@@ -27,7 +26,7 @@ inline are compiler instrumentation types. The former produces smaller binary
the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC
version 5.0 or later.
-Currently KASAN works only with the SLUB memory allocator.
+KASAN works with both SLUB and SLAB memory allocators.
For better bug detection and nicer reporting, enable CONFIG_STACKTRACE.
To disable instrumentation for specific files or directories, add a line
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 4b9f85c..4405a35 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -46,6 +46,9 @@ void kasan_unpoison_shadow(const void *address, size_t size);
void kasan_alloc_pages(struct page *page, unsigned int order);
void kasan_free_pages(struct page *page, unsigned int order);
+void kasan_cache_create(struct kmem_cache *cache, size_t *size,
+ unsigned long *flags);
+
void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
void kasan_poison_object_data(struct kmem_cache *cache, void *object);
@@ -59,6 +62,11 @@ void kasan_krealloc(const void *object, size_t new_size);
void kasan_slab_alloc(struct kmem_cache *s, void *object);
void kasan_slab_free(struct kmem_cache *s, void *object);
+struct kasan_cache {
+ int alloc_meta_offset;
+ int free_meta_offset;
+};
+
int kasan_module_alloc(void *addr, size_t size);
void kasan_free_shadow(const struct vm_struct *vm);
@@ -72,6 +80,10 @@ static inline void kasan_disable_current(void) {}
static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
static inline void kasan_free_pages(struct page *page, unsigned int order) {}
+static inline void kasan_cache_create(struct kmem_cache *cache,
+ size_t *size,
+ unsigned long *flags) {}
+
static inline void kasan_poison_slab(struct page *page) {}
static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
void *object) {}
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 3ffee74..92f8558 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -92,6 +92,12 @@
# define SLAB_ACCOUNT 0x00000000UL
#endif
+#ifdef CONFIG_KASAN
+#define SLAB_KASAN 0x08000000UL
+#else
+#define SLAB_KASAN 0x00000000UL
+#endif
+
/* The following flags affect the page allocator grouping pages by mobility */
#define SLAB_RECLAIM_ACCOUNT 0x00020000UL /* Objects are reclaimable */
#define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */
diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
index 33d0490..a25804d 100644
--- a/include/linux/slab_def.h
+++ b/include/linux/slab_def.h
@@ -72,8 +72,22 @@ struct kmem_cache {
#ifdef CONFIG_MEMCG_KMEM
struct memcg_cache_params memcg_params;
#endif
+#ifdef CONFIG_KASAN
+ struct kasan_cache kasan_info;
+#endif
struct kmem_cache_node *node[MAX_NUMNODES];
};
+static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
+ void *x) {
+ void *object = x - (x - page->s_mem) % cache->size;
+ void *last_object = page->s_mem + (cache->num - 1) * cache->size;
+
+ if (unlikely(object > last_object))
+ return last_object;
+ else
+ return object;
+}
+
#endif /* _LINUX_SLAB_DEF_H */
diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
index 3388511..c553dad 100644
--- a/include/linux/slub_def.h
+++ b/include/linux/slub_def.h
@@ -129,4 +129,15 @@ static inline void *virt_to_obj(struct kmem_cache *s,
void object_err(struct kmem_cache *s, struct page *page,
u8 *object, char *reason);
+static inline void *nearest_obj(struct kmem_cache *cache, struct page *page,
+ void *x) {
+ void *object = x - (x - page_address(page)) % cache->size;
+ void *last_object = page_address(page) +
+ (page->objects - 1) * cache->size;
+ if (unlikely(object > last_object))
+ return last_object;
+ else
+ return object;
+}
+
#endif /* _LINUX_SLUB_DEF_H */
diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan
index 0fee5ac..0e4d2b3 100644
--- a/lib/Kconfig.kasan
+++ b/lib/Kconfig.kasan
@@ -5,7 +5,7 @@ if HAVE_ARCH_KASAN
config KASAN
bool "KASan: runtime memory debugger"
- depends on SLUB_DEBUG
+ depends on SLUB_DEBUG || (SLAB && !DEBUG_SLAB)
select CONSTRUCTORS
help
Enables kernel address sanitizer - runtime memory debugger,
@@ -16,6 +16,8 @@ config KASAN
This feature consumes about 1/8 of available memory and brings about
~x3 performance slowdown.
For better error detection enable CONFIG_STACKTRACE.
+ Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB
+ (the resulting kernel does not boot).
choice
prompt "Instrumentation type"
diff --git a/mm/Makefile b/mm/Makefile
index 2ed4319..d675b37 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -3,6 +3,7 @@
#
KASAN_SANITIZE_slab_common.o := n
+KASAN_SANITIZE_slab.o := n
KASAN_SANITIZE_slub.o := n
mmu-y := nommu.o
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index bc0a8d8..84305c2 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -314,6 +314,59 @@ void kasan_free_pages(struct page *page, unsigned int order)
KASAN_FREE_PAGE);
}
+#ifdef CONFIG_SLAB
+/*
+ * Adaptive redzone policy taken from the userspace AddressSanitizer runtime.
+ * For larger allocations larger redzones are used.
+ */
+static size_t optimal_redzone(size_t object_size)
+{
+ int rz =
+ object_size <= 64 - 16 ? 16 :
+ object_size <= 128 - 32 ? 32 :
+ object_size <= 512 - 64 ? 64 :
+ object_size <= 4096 - 128 ? 128 :
+ object_size <= (1 << 14) - 256 ? 256 :
+ object_size <= (1 << 15) - 512 ? 512 :
+ object_size <= (1 << 16) - 1024 ? 1024 : 2048;
+ return rz;
+}
+
+void kasan_cache_create(struct kmem_cache *cache, size_t *size,
+ unsigned long *flags)
+{
+ int redzone_adjust;
+ /* Make sure the adjusted size is still less than
+ * KMALLOC_MAX_CACHE_SIZE.
+ * TODO: this check is only useful for SLAB, but not SLUB. We'll need
+ * to skip it for SLUB when it starts using kasan_cache_create().
+ */
+ if (*size > KMALLOC_MAX_CACHE_SIZE -
+ sizeof(struct kasan_alloc_meta) -
+ sizeof(struct kasan_free_meta))
+ return;
+ *flags |= SLAB_KASAN;
+ /* Add alloc meta. */
+ cache->kasan_info.alloc_meta_offset = *size;
+ *size += sizeof(struct kasan_alloc_meta);
+
+ /* Add free meta. */
+ if (cache->flags & SLAB_DESTROY_BY_RCU || cache->ctor ||
+ cache->object_size < sizeof(struct kasan_free_meta)) {
+ cache->kasan_info.free_meta_offset = *size;
+ *size += sizeof(struct kasan_free_meta);
+ }
+ redzone_adjust = optimal_redzone(cache->object_size) -
+ (*size - cache->object_size);
+ if (redzone_adjust > 0)
+ *size += redzone_adjust;
+ *size = min(KMALLOC_MAX_CACHE_SIZE,
+ max(*size,
+ cache->object_size +
+ optimal_redzone(cache->object_size)));
+}
+#endif
+
void kasan_poison_slab(struct page *page)
{
kasan_poison_shadow(page_address(page),
@@ -331,8 +384,36 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object)
kasan_poison_shadow(object,
round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE),
KASAN_KMALLOC_REDZONE);
+#ifdef CONFIG_SLAB
+ if (cache->flags & SLAB_KASAN) {
+ struct kasan_alloc_meta *alloc_info =
+ get_alloc_info(cache, object);
+ alloc_info->state = KASAN_STATE_INIT;
+ }
+#endif
}
+static inline void set_track(struct kasan_track *track)
+{
+ track->cpu = raw_smp_processor_id();
+ track->pid = current->pid;
+ track->when = jiffies;
+}
+
+#ifdef CONFIG_SLAB
+struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
+ const void *object)
+{
+ return (void *)object + cache->kasan_info.alloc_meta_offset;
+}
+
+struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
+ const void *object)
+{
+ return (void *)object + cache->kasan_info.free_meta_offset;
+}
+#endif
+
void kasan_slab_alloc(struct kmem_cache *cache, void *object)
{
kasan_kmalloc(cache, object, cache->object_size);
@@ -366,6 +447,16 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size)
kasan_unpoison_shadow(object, size);
kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
KASAN_KMALLOC_REDZONE);
+#ifdef CONFIG_SLAB
+ if (cache->flags & SLAB_KASAN) {
+ struct kasan_alloc_meta *alloc_info =
+ get_alloc_info(cache, object);
+
+ alloc_info->state = KASAN_STATE_ALLOC;
+ alloc_info->alloc_size = size;
+ set_track(&alloc_info->track);
+ }
+#endif
}
EXPORT_SYMBOL(kasan_kmalloc);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 4f6c62e..7b9e4ab9 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -54,6 +54,40 @@ struct kasan_global {
#endif
};
+/**
+ * Structures to keep alloc and free tracks *
+ */
+
+enum kasan_state {
+ KASAN_STATE_INIT,
+ KASAN_STATE_ALLOC,
+ KASAN_STATE_FREE
+};
+
+struct kasan_track {
+ u64 cpu : 6; /* for NR_CPUS = 64 */
+ u64 pid : 16; /* 65536 processes */
+ u64 when : 42; /* ~140 years */
+};
+
+struct kasan_alloc_meta {
+ u32 state : 2; /* enum kasan_state */
+ u32 alloc_size : 30;
+ struct kasan_track track;
+};
+
+struct kasan_free_meta {
+ /* Allocator freelist pointer, unused by KASAN. */
+ void **freelist;
+ struct kasan_track track;
+};
+
+struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
+ const void *object);
+struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
+ const void *object);
+
+
static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
{
return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET)
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 12f222d..2bf7218 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -115,6 +115,42 @@ static inline bool init_task_stack_addr(const void *addr)
sizeof(init_thread_union.stack));
}
+static void print_track(struct kasan_track *track)
+{
+ pr_err("PID = %lu, CPU = %lu, timestamp = %lu\n", track->pid,
+ track->cpu, track->when);
+}
+
+static void print_object(struct kmem_cache *cache, void *object)
+{
+ struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object);
+ struct kasan_free_meta *free_info;
+
+ pr_err("Object at %p, in cache %s\n", object, cache->name);
+ if (!(cache->flags & SLAB_KASAN))
+ return;
+ switch (alloc_info->state) {
+ case KASAN_STATE_INIT:
+ pr_err("Object not allocated yet\n");
+ break;
+ case KASAN_STATE_ALLOC:
+ pr_err("Object allocated with size %u bytes.\n",
+ alloc_info->alloc_size);
+ pr_err("Allocation:\n");
+ print_track(&alloc_info->track);
+ break;
+ case KASAN_STATE_FREE:
+ pr_err("Object freed, allocated with size %u bytes\n",
+ alloc_info->alloc_size);
+ free_info = get_free_info(cache, object);
+ pr_err("Allocation:\n");
+ print_track(&alloc_info->track);
+ pr_err("Deallocation:\n");
+ print_track(&free_info->track);
+ break;
+ }
+}
+
static void print_address_description(struct kasan_access_info *info)
{
const void *addr = info->access_addr;
@@ -126,17 +162,14 @@ static void print_address_description(struct kasan_access_info *info)
if (PageSlab(page)) {
void *object;
struct kmem_cache *cache = page->slab_cache;
- void *last_object;
-
- object = virt_to_obj(cache, page_address(page), addr);
- last_object = page_address(page) +
- page->objects * cache->size;
-
- if (unlikely(object > last_object))
- object = last_object; /* we hit into padding */
-
+ object = nearest_obj(cache, page,
+ (void *)info->access_addr);
+#ifdef CONFIG_SLAB
+ print_object(cache, object);
+#else
object_err(cache, page, object,
- "kasan: bad access detected");
+ "kasan: bad access detected");
+#endif
return;
}
dump_page(page, "kasan: bad access detected");
@@ -146,8 +179,9 @@ static void print_address_description(struct kasan_access_info *info)
if (!init_task_stack_addr(addr))
pr_err("Address belongs to variable %pS\n", addr);
}
-
+#ifdef CONFIG_SLUB
dump_stack();
+#endif
}
static bool row_is_guilty(const void *row, const void *guilty)
@@ -233,6 +267,9 @@ static void kasan_report_error(struct kasan_access_info *info)
dump_stack();
} else {
print_error_description(info);
+#ifdef CONFIG_SLAB
+ dump_stack();
+#endif
print_address_description(info);
print_shadow_for_address(info->first_bad_addr);
}
diff --git a/mm/slab.c b/mm/slab.c
index 6ecc697..739b89d 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -2196,6 +2196,7 @@ __kmem_cache_create (struct kmem_cache *cachep, unsigned long flags)
#endif
#endif
+ kasan_cache_create(cachep, &size, &flags);
/*
* Determine if the slab management is 'on' or 'off' slab.
* (bootstrapping cannot cope with offslab caches so don't do
@@ -2503,8 +2504,13 @@ static void cache_init_objs(struct kmem_cache *cachep,
* cache which they are a constructor for. Otherwise, deadlock.
* They must also be threaded.
*/
- if (cachep->ctor && !(cachep->flags & SLAB_POISON))
+ if (cachep->ctor && !(cachep->flags & SLAB_POISON)) {
+ kasan_unpoison_object_data(cachep,
+ objp + obj_offset(cachep));
cachep->ctor(objp + obj_offset(cachep));
+ kasan_poison_object_data(
+ cachep, objp + obj_offset(cachep));
+ }
if (cachep->flags & SLAB_RED_ZONE) {
if (*dbg_redzone2(cachep, objp) != RED_INACTIVE)
@@ -2519,8 +2525,11 @@ static void cache_init_objs(struct kmem_cache *cachep,
kernel_map_pages(virt_to_page(objp),
cachep->size / PAGE_SIZE, 0);
#else
- if (cachep->ctor)
+ if (cachep->ctor) {
+ kasan_unpoison_object_data(cachep, objp);
cachep->ctor(objp);
+ kasan_poison_object_data(cachep, objp);
+ }
#endif
set_obj_status(page, i, OBJECT_FREE);
set_free_obj(page, i, i);
@@ -2650,6 +2659,7 @@ static int cache_grow(struct kmem_cache *cachep,
slab_map_pages(cachep, page, freelist);
+ kasan_poison_slab(page);
cache_init_objs(cachep, page);
if (gfpflags_allow_blocking(local_flags))
@@ -3364,7 +3374,10 @@ free_done:
static inline void __cache_free(struct kmem_cache *cachep, void *objp,
unsigned long caller)
{
- struct array_cache *ac = cpu_cache_get(cachep);
+ struct array_cache *ac;
+
+ kasan_slab_free(cachep, objp);
+ ac = cpu_cache_get(cachep);
check_irq_off();
kmemleak_free_recursive(objp, cachep->flags);
@@ -3403,6 +3416,8 @@ static inline void __cache_free(struct kmem_cache *cachep, void *objp,
void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags)
{
void *ret = slab_alloc(cachep, flags, _RET_IP_);
+ if (ret)
+ kasan_slab_alloc(cachep, ret);
trace_kmem_cache_alloc(_RET_IP_, ret,
cachep->object_size, cachep->size, flags);
@@ -3432,6 +3447,8 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size)
ret = slab_alloc(cachep, flags, _RET_IP_);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
trace_kmalloc(_RET_IP_, ret,
size, cachep->size, flags);
return ret;
@@ -3455,6 +3472,8 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid)
{
void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
+ if (ret)
+ kasan_slab_alloc(cachep, ret);
trace_kmem_cache_alloc_node(_RET_IP_, ret,
cachep->object_size, cachep->size,
flags, nodeid);
@@ -3473,6 +3492,8 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep,
ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
trace_kmalloc_node(_RET_IP_, ret,
size, cachep->size,
flags, nodeid);
@@ -3485,11 +3506,16 @@ static __always_inline void *
__do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller)
{
struct kmem_cache *cachep;
+ void *ret;
cachep = kmalloc_slab(size, flags);
if (unlikely(ZERO_OR_NULL_PTR(cachep)))
return cachep;
- return kmem_cache_alloc_node_trace(cachep, flags, node, size);
+ ret = kmem_cache_alloc_node_trace(cachep, flags, node, size);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
+
+ return ret;
}
void *__kmalloc_node(size_t size, gfp_t flags, int node)
@@ -3523,6 +3549,8 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags,
return cachep;
ret = slab_alloc(cachep, flags, caller);
+ if (ret)
+ kasan_kmalloc(cachep, ret, size);
trace_kmalloc(caller, ret,
size, cachep->size, flags);
@@ -4240,10 +4268,18 @@ module_init(slab_proc_init);
*/
size_t ksize(const void *objp)
{
+ size_t size;
+
BUG_ON(!objp);
if (unlikely(objp == ZERO_SIZE_PTR))
return 0;
- return virt_to_cache(objp)->object_size;
+ size = virt_to_cache(objp)->object_size;
+ /* We assume that ksize callers could use whole allocated area,
+ * so we need to unpoison this area.
+ */
+ kasan_krealloc(objp, size);
+
+ return size;
}
EXPORT_SYMBOL(ksize);
diff --git a/mm/slab_common.c b/mm/slab_common.c
index e016178..8d2531d 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -35,7 +35,7 @@ struct kmem_cache *kmem_cache;
*/
#define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
SLAB_TRACE | SLAB_DESTROY_BY_RCU | SLAB_NOLEAKTRACE | \
- SLAB_FAILSLAB)
+ SLAB_FAILSLAB | SLAB_KASAN)
#define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
SLAB_NOTRACK | SLAB_ACCOUNT)
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 3/8] mm, kasan: Added GFP flags to KASAN API
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Add GFP flags to KASAN hooks for future patches to use.
This patch is based on the "mm: kasan: unified support for SLUB and
SLAB allocators" patch originally prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
include/linux/kasan.h | 19 +++++++++++--------
mm/kasan/kasan.c | 15 ++++++++-------
mm/mempool.c | 16 ++++++++--------
mm/slab.c | 14 +++++++-------
mm/slab_common.c | 4 ++--
mm/slub.c | 17 +++++++++--------
6 files changed, 45 insertions(+), 40 deletions(-)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 4405a35..bf71ab0 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -53,13 +53,14 @@ void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
void kasan_poison_object_data(struct kmem_cache *cache, void *object);
-void kasan_kmalloc_large(const void *ptr, size_t size);
+void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
void kasan_kfree_large(const void *ptr);
void kasan_kfree(void *ptr);
-void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size);
-void kasan_krealloc(const void *object, size_t new_size);
+void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
+ gfp_t flags);
+void kasan_krealloc(const void *object, size_t new_size, gfp_t flags);
-void kasan_slab_alloc(struct kmem_cache *s, void *object);
+void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
void kasan_slab_free(struct kmem_cache *s, void *object);
struct kasan_cache {
@@ -90,14 +91,16 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
static inline void kasan_poison_object_data(struct kmem_cache *cache,
void *object) {}
-static inline void kasan_kmalloc_large(void *ptr, size_t size) {}
+static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {}
static inline void kasan_kfree_large(const void *ptr) {}
static inline void kasan_kfree(void *ptr) {}
static inline void kasan_kmalloc(struct kmem_cache *s, const void *object,
- size_t size) {}
-static inline void kasan_krealloc(const void *object, size_t new_size) {}
+ size_t size, gfp_t flags) {}
+static inline void kasan_krealloc(const void *object, size_t new_size,
+ gfp_t flags) {}
-static inline void kasan_slab_alloc(struct kmem_cache *s, void *object) {}
+static inline void kasan_slab_alloc(struct kmem_cache *s, void *object,
+ gfp_t flags) {}
static inline void kasan_slab_free(struct kmem_cache *s, void *object) {}
static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index 84305c2..787224a 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -414,9 +414,9 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
}
#endif
-void kasan_slab_alloc(struct kmem_cache *cache, void *object)
+void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)
{
- kasan_kmalloc(cache, object, cache->object_size);
+ kasan_kmalloc(cache, object, cache->object_size, flags);
}
void kasan_slab_free(struct kmem_cache *cache, void *object)
@@ -431,7 +431,8 @@ void kasan_slab_free(struct kmem_cache *cache, void *object)
kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
}
-void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size)
+void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
+ gfp_t flags)
{
unsigned long redzone_start;
unsigned long redzone_end;
@@ -460,7 +461,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size)
}
EXPORT_SYMBOL(kasan_kmalloc);
-void kasan_kmalloc_large(const void *ptr, size_t size)
+void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
{
struct page *page;
unsigned long redzone_start;
@@ -479,7 +480,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size)
KASAN_PAGE_REDZONE);
}
-void kasan_krealloc(const void *object, size_t size)
+void kasan_krealloc(const void *object, size_t size, gfp_t flags)
{
struct page *page;
@@ -489,9 +490,9 @@ void kasan_krealloc(const void *object, size_t size)
page = virt_to_head_page(object);
if (unlikely(!PageSlab(page)))
- kasan_kmalloc_large(object, size);
+ kasan_kmalloc_large(object, size, flags);
else
- kasan_kmalloc(page->slab_cache, object, size);
+ kasan_kmalloc(page->slab_cache, object, size, flags);
}
void kasan_kfree(void *ptr)
diff --git a/mm/mempool.c b/mm/mempool.c
index 004d42b..b47c8a7 100644
--- a/mm/mempool.c
+++ b/mm/mempool.c
@@ -112,12 +112,12 @@ static void kasan_poison_element(mempool_t *pool, void *element)
kasan_free_pages(element, (unsigned long)pool->pool_data);
}
-static void kasan_unpoison_element(mempool_t *pool, void *element)
+static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
{
if (pool->alloc == mempool_alloc_slab)
- kasan_slab_alloc(pool->pool_data, element);
+ kasan_slab_alloc(pool->pool_data, element, flags);
if (pool->alloc == mempool_kmalloc)
- kasan_krealloc(element, (size_t)pool->pool_data);
+ kasan_krealloc(element, (size_t)pool->pool_data, flags);
if (pool->alloc == mempool_alloc_pages)
kasan_alloc_pages(element, (unsigned long)pool->pool_data);
}
@@ -130,13 +130,13 @@ static void add_element(mempool_t *pool, void *element)
pool->elements[pool->curr_nr++] = element;
}
-static void *remove_element(mempool_t *pool)
+static void *remove_element(mempool_t *pool, gfp_t flags)
{
void *element = pool->elements[--pool->curr_nr];
BUG_ON(pool->curr_nr < 0);
check_element(pool, element);
- kasan_unpoison_element(pool, element);
+ kasan_unpoison_element(pool, element, flags);
return element;
}
@@ -154,7 +154,7 @@ void mempool_destroy(mempool_t *pool)
return;
while (pool->curr_nr) {
- void *element = remove_element(pool);
+ void *element = remove_element(pool, GFP_KERNEL);
pool->free(element, pool->pool_data);
}
kfree(pool->elements);
@@ -250,7 +250,7 @@ int mempool_resize(mempool_t *pool, int new_min_nr)
spin_lock_irqsave(&pool->lock, flags);
if (new_min_nr <= pool->min_nr) {
while (new_min_nr < pool->curr_nr) {
- element = remove_element(pool);
+ element = remove_element(pool, GFP_KERNEL);
spin_unlock_irqrestore(&pool->lock, flags);
pool->free(element, pool->pool_data);
spin_lock_irqsave(&pool->lock, flags);
@@ -336,7 +336,7 @@ repeat_alloc:
spin_lock_irqsave(&pool->lock, flags);
if (likely(pool->curr_nr)) {
- element = remove_element(pool);
+ element = remove_element(pool, gfp_temp);
spin_unlock_irqrestore(&pool->lock, flags);
/* paired with rmb in mempool_free(), read comment there */
smp_wmb();
diff --git a/mm/slab.c b/mm/slab.c
index 739b89d..0ec7aa3 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -3417,7 +3417,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags)
{
void *ret = slab_alloc(cachep, flags, _RET_IP_);
if (ret)
- kasan_slab_alloc(cachep, ret);
+ kasan_slab_alloc(cachep, ret, flags);
trace_kmem_cache_alloc(_RET_IP_, ret,
cachep->object_size, cachep->size, flags);
@@ -3448,7 +3448,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size)
ret = slab_alloc(cachep, flags, _RET_IP_);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
trace_kmalloc(_RET_IP_, ret,
size, cachep->size, flags);
return ret;
@@ -3473,7 +3473,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid)
void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
if (ret)
- kasan_slab_alloc(cachep, ret);
+ kasan_slab_alloc(cachep, ret, flags);
trace_kmem_cache_alloc_node(_RET_IP_, ret,
cachep->object_size, cachep->size,
flags, nodeid);
@@ -3493,7 +3493,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep,
ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
trace_kmalloc_node(_RET_IP_, ret,
size, cachep->size,
flags, nodeid);
@@ -3513,7 +3513,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller)
return cachep;
ret = kmem_cache_alloc_node_trace(cachep, flags, node, size);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
return ret;
}
@@ -3550,7 +3550,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags,
ret = slab_alloc(cachep, flags, caller);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
trace_kmalloc(caller, ret,
size, cachep->size, flags);
@@ -4278,7 +4278,7 @@ size_t ksize(const void *objp)
/* We assume that ksize callers could use whole allocated area,
* so we need to unpoison this area.
*/
- kasan_krealloc(objp, size);
+ kasan_krealloc(objp, size, GFP_NOWAIT);
return size;
}
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 8d2531d..8478631 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1008,7 +1008,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order)
page = alloc_kmem_pages(flags, order);
ret = page ? page_address(page) : NULL;
kmemleak_alloc(ret, size, 1, flags);
- kasan_kmalloc_large(ret, size);
+ kasan_kmalloc_large(ret, size, flags);
return ret;
}
EXPORT_SYMBOL(kmalloc_order);
@@ -1189,7 +1189,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size,
ks = ksize(p);
if (ks >= new_size) {
- kasan_krealloc((void *)p, new_size);
+ kasan_krealloc((void *)p, new_size, flags);
return (void *)p;
}
diff --git a/mm/slub.c b/mm/slub.c
index b21fd24..945bbee 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1272,7 +1272,7 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node,
static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags)
{
kmemleak_alloc(ptr, size, 1, flags);
- kasan_kmalloc_large(ptr, size);
+ kasan_kmalloc_large(ptr, size, flags);
}
static inline void kfree_hook(const void *x)
@@ -1306,7 +1306,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
kmemcheck_slab_alloc(s, flags, object, slab_ksize(s));
kmemleak_alloc_recursive(object, s->object_size, 1,
s->flags, flags);
- kasan_slab_alloc(s, object);
+ kasan_slab_alloc(s, object, flags);
}
memcg_kmem_put_cache(s);
}
@@ -2590,7 +2590,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size)
{
void *ret = slab_alloc(s, gfpflags, _RET_IP_);
trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, gfpflags);
return ret;
}
EXPORT_SYMBOL(kmem_cache_alloc_trace);
@@ -2618,7 +2618,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s,
trace_kmalloc_node(_RET_IP_, ret,
size, s->size, gfpflags, node);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, gfpflags);
return ret;
}
EXPORT_SYMBOL(kmem_cache_alloc_node_trace);
@@ -3162,7 +3162,8 @@ static void early_kmem_cache_node_alloc(int node)
init_object(kmem_cache_node, n, SLUB_RED_ACTIVE);
init_tracking(kmem_cache_node, n);
#endif
- kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node));
+ kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node),
+ GFP_KERNEL);
init_kmem_cache_node(n);
inc_slabs_node(kmem_cache_node, node, page->objects);
@@ -3535,7 +3536,7 @@ void *__kmalloc(size_t size, gfp_t flags)
trace_kmalloc(_RET_IP_, ret, size, s->size, flags);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, flags);
return ret;
}
@@ -3580,7 +3581,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node)
trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, flags);
return ret;
}
@@ -3609,7 +3610,7 @@ size_t ksize(const void *object)
size_t size = __ksize(object);
/* We assume that ksize callers could use whole allocated area,
so we need unpoison this area. */
- kasan_krealloc(object, size);
+ kasan_krealloc(object, size, GFP_NOWAIT);
return size;
}
EXPORT_SYMBOL(ksize);
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 3/8] mm, kasan: Added GFP flags to KASAN API
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Add GFP flags to KASAN hooks for future patches to use.
This patch is based on the "mm: kasan: unified support for SLUB and
SLAB allocators" patch originally prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
include/linux/kasan.h | 19 +++++++++++--------
mm/kasan/kasan.c | 15 ++++++++-------
mm/mempool.c | 16 ++++++++--------
mm/slab.c | 14 +++++++-------
mm/slab_common.c | 4 ++--
mm/slub.c | 17 +++++++++--------
6 files changed, 45 insertions(+), 40 deletions(-)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 4405a35..bf71ab0 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -53,13 +53,14 @@ void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
void kasan_poison_object_data(struct kmem_cache *cache, void *object);
-void kasan_kmalloc_large(const void *ptr, size_t size);
+void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
void kasan_kfree_large(const void *ptr);
void kasan_kfree(void *ptr);
-void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size);
-void kasan_krealloc(const void *object, size_t new_size);
+void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
+ gfp_t flags);
+void kasan_krealloc(const void *object, size_t new_size, gfp_t flags);
-void kasan_slab_alloc(struct kmem_cache *s, void *object);
+void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
void kasan_slab_free(struct kmem_cache *s, void *object);
struct kasan_cache {
@@ -90,14 +91,16 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
static inline void kasan_poison_object_data(struct kmem_cache *cache,
void *object) {}
-static inline void kasan_kmalloc_large(void *ptr, size_t size) {}
+static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {}
static inline void kasan_kfree_large(const void *ptr) {}
static inline void kasan_kfree(void *ptr) {}
static inline void kasan_kmalloc(struct kmem_cache *s, const void *object,
- size_t size) {}
-static inline void kasan_krealloc(const void *object, size_t new_size) {}
+ size_t size, gfp_t flags) {}
+static inline void kasan_krealloc(const void *object, size_t new_size,
+ gfp_t flags) {}
-static inline void kasan_slab_alloc(struct kmem_cache *s, void *object) {}
+static inline void kasan_slab_alloc(struct kmem_cache *s, void *object,
+ gfp_t flags) {}
static inline void kasan_slab_free(struct kmem_cache *s, void *object) {}
static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index 84305c2..787224a 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -414,9 +414,9 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
}
#endif
-void kasan_slab_alloc(struct kmem_cache *cache, void *object)
+void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)
{
- kasan_kmalloc(cache, object, cache->object_size);
+ kasan_kmalloc(cache, object, cache->object_size, flags);
}
void kasan_slab_free(struct kmem_cache *cache, void *object)
@@ -431,7 +431,8 @@ void kasan_slab_free(struct kmem_cache *cache, void *object)
kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
}
-void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size)
+void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
+ gfp_t flags)
{
unsigned long redzone_start;
unsigned long redzone_end;
@@ -460,7 +461,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size)
}
EXPORT_SYMBOL(kasan_kmalloc);
-void kasan_kmalloc_large(const void *ptr, size_t size)
+void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
{
struct page *page;
unsigned long redzone_start;
@@ -479,7 +480,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size)
KASAN_PAGE_REDZONE);
}
-void kasan_krealloc(const void *object, size_t size)
+void kasan_krealloc(const void *object, size_t size, gfp_t flags)
{
struct page *page;
@@ -489,9 +490,9 @@ void kasan_krealloc(const void *object, size_t size)
page = virt_to_head_page(object);
if (unlikely(!PageSlab(page)))
- kasan_kmalloc_large(object, size);
+ kasan_kmalloc_large(object, size, flags);
else
- kasan_kmalloc(page->slab_cache, object, size);
+ kasan_kmalloc(page->slab_cache, object, size, flags);
}
void kasan_kfree(void *ptr)
diff --git a/mm/mempool.c b/mm/mempool.c
index 004d42b..b47c8a7 100644
--- a/mm/mempool.c
+++ b/mm/mempool.c
@@ -112,12 +112,12 @@ static void kasan_poison_element(mempool_t *pool, void *element)
kasan_free_pages(element, (unsigned long)pool->pool_data);
}
-static void kasan_unpoison_element(mempool_t *pool, void *element)
+static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
{
if (pool->alloc == mempool_alloc_slab)
- kasan_slab_alloc(pool->pool_data, element);
+ kasan_slab_alloc(pool->pool_data, element, flags);
if (pool->alloc == mempool_kmalloc)
- kasan_krealloc(element, (size_t)pool->pool_data);
+ kasan_krealloc(element, (size_t)pool->pool_data, flags);
if (pool->alloc == mempool_alloc_pages)
kasan_alloc_pages(element, (unsigned long)pool->pool_data);
}
@@ -130,13 +130,13 @@ static void add_element(mempool_t *pool, void *element)
pool->elements[pool->curr_nr++] = element;
}
-static void *remove_element(mempool_t *pool)
+static void *remove_element(mempool_t *pool, gfp_t flags)
{
void *element = pool->elements[--pool->curr_nr];
BUG_ON(pool->curr_nr < 0);
check_element(pool, element);
- kasan_unpoison_element(pool, element);
+ kasan_unpoison_element(pool, element, flags);
return element;
}
@@ -154,7 +154,7 @@ void mempool_destroy(mempool_t *pool)
return;
while (pool->curr_nr) {
- void *element = remove_element(pool);
+ void *element = remove_element(pool, GFP_KERNEL);
pool->free(element, pool->pool_data);
}
kfree(pool->elements);
@@ -250,7 +250,7 @@ int mempool_resize(mempool_t *pool, int new_min_nr)
spin_lock_irqsave(&pool->lock, flags);
if (new_min_nr <= pool->min_nr) {
while (new_min_nr < pool->curr_nr) {
- element = remove_element(pool);
+ element = remove_element(pool, GFP_KERNEL);
spin_unlock_irqrestore(&pool->lock, flags);
pool->free(element, pool->pool_data);
spin_lock_irqsave(&pool->lock, flags);
@@ -336,7 +336,7 @@ repeat_alloc:
spin_lock_irqsave(&pool->lock, flags);
if (likely(pool->curr_nr)) {
- element = remove_element(pool);
+ element = remove_element(pool, gfp_temp);
spin_unlock_irqrestore(&pool->lock, flags);
/* paired with rmb in mempool_free(), read comment there */
smp_wmb();
diff --git a/mm/slab.c b/mm/slab.c
index 739b89d..0ec7aa3 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -3417,7 +3417,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags)
{
void *ret = slab_alloc(cachep, flags, _RET_IP_);
if (ret)
- kasan_slab_alloc(cachep, ret);
+ kasan_slab_alloc(cachep, ret, flags);
trace_kmem_cache_alloc(_RET_IP_, ret,
cachep->object_size, cachep->size, flags);
@@ -3448,7 +3448,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size)
ret = slab_alloc(cachep, flags, _RET_IP_);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
trace_kmalloc(_RET_IP_, ret,
size, cachep->size, flags);
return ret;
@@ -3473,7 +3473,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid)
void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
if (ret)
- kasan_slab_alloc(cachep, ret);
+ kasan_slab_alloc(cachep, ret, flags);
trace_kmem_cache_alloc_node(_RET_IP_, ret,
cachep->object_size, cachep->size,
flags, nodeid);
@@ -3493,7 +3493,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep,
ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
trace_kmalloc_node(_RET_IP_, ret,
size, cachep->size,
flags, nodeid);
@@ -3513,7 +3513,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller)
return cachep;
ret = kmem_cache_alloc_node_trace(cachep, flags, node, size);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
return ret;
}
@@ -3550,7 +3550,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags,
ret = slab_alloc(cachep, flags, caller);
if (ret)
- kasan_kmalloc(cachep, ret, size);
+ kasan_kmalloc(cachep, ret, size, flags);
trace_kmalloc(caller, ret,
size, cachep->size, flags);
@@ -4278,7 +4278,7 @@ size_t ksize(const void *objp)
/* We assume that ksize callers could use whole allocated area,
* so we need to unpoison this area.
*/
- kasan_krealloc(objp, size);
+ kasan_krealloc(objp, size, GFP_NOWAIT);
return size;
}
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 8d2531d..8478631 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1008,7 +1008,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order)
page = alloc_kmem_pages(flags, order);
ret = page ? page_address(page) : NULL;
kmemleak_alloc(ret, size, 1, flags);
- kasan_kmalloc_large(ret, size);
+ kasan_kmalloc_large(ret, size, flags);
return ret;
}
EXPORT_SYMBOL(kmalloc_order);
@@ -1189,7 +1189,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size,
ks = ksize(p);
if (ks >= new_size) {
- kasan_krealloc((void *)p, new_size);
+ kasan_krealloc((void *)p, new_size, flags);
return (void *)p;
}
diff --git a/mm/slub.c b/mm/slub.c
index b21fd24..945bbee 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1272,7 +1272,7 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node,
static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags)
{
kmemleak_alloc(ptr, size, 1, flags);
- kasan_kmalloc_large(ptr, size);
+ kasan_kmalloc_large(ptr, size, flags);
}
static inline void kfree_hook(const void *x)
@@ -1306,7 +1306,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
kmemcheck_slab_alloc(s, flags, object, slab_ksize(s));
kmemleak_alloc_recursive(object, s->object_size, 1,
s->flags, flags);
- kasan_slab_alloc(s, object);
+ kasan_slab_alloc(s, object, flags);
}
memcg_kmem_put_cache(s);
}
@@ -2590,7 +2590,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size)
{
void *ret = slab_alloc(s, gfpflags, _RET_IP_);
trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, gfpflags);
return ret;
}
EXPORT_SYMBOL(kmem_cache_alloc_trace);
@@ -2618,7 +2618,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s,
trace_kmalloc_node(_RET_IP_, ret,
size, s->size, gfpflags, node);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, gfpflags);
return ret;
}
EXPORT_SYMBOL(kmem_cache_alloc_node_trace);
@@ -3162,7 +3162,8 @@ static void early_kmem_cache_node_alloc(int node)
init_object(kmem_cache_node, n, SLUB_RED_ACTIVE);
init_tracking(kmem_cache_node, n);
#endif
- kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node));
+ kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node),
+ GFP_KERNEL);
init_kmem_cache_node(n);
inc_slabs_node(kmem_cache_node, node, page->objects);
@@ -3535,7 +3536,7 @@ void *__kmalloc(size_t size, gfp_t flags)
trace_kmalloc(_RET_IP_, ret, size, s->size, flags);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, flags);
return ret;
}
@@ -3580,7 +3581,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node)
trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node);
- kasan_kmalloc(s, ret, size);
+ kasan_kmalloc(s, ret, size, flags);
return ret;
}
@@ -3609,7 +3610,7 @@ size_t ksize(const void *object)
size_t size = __ksize(object);
/* We assume that ksize callers could use whole allocated area,
so we need unpoison this area. */
- kasan_krealloc(object, size);
+ kasan_krealloc(object, size, GFP_NOWAIT);
return size;
}
EXPORT_SYMBOL(ksize);
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
KASAN needs to know whether the allocation happens in an IRQ handler.
This lets us strip everything below the IRQ entry point to reduce the number
of unique stack traces needed to be stored.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
arch/arm/kernel/vmlinux.lds.S | 1 +
arch/arm64/kernel/vmlinux.lds.S | 1 +
arch/blackfin/kernel/vmlinux.lds.S | 1 +
arch/c6x/kernel/vmlinux.lds.S | 1 +
arch/metag/kernel/vmlinux.lds.S | 1 +
arch/microblaze/kernel/vmlinux.lds.S | 1 +
arch/mips/kernel/vmlinux.lds.S | 1 +
arch/nios2/kernel/vmlinux.lds.S | 1 +
arch/openrisc/kernel/vmlinux.lds.S | 1 +
arch/parisc/kernel/vmlinux.lds.S | 1 +
arch/powerpc/kernel/vmlinux.lds.S | 1 +
arch/s390/kernel/vmlinux.lds.S | 1 +
arch/sh/kernel/vmlinux.lds.S | 1 +
arch/sparc/kernel/vmlinux.lds.S | 1 +
arch/tile/kernel/vmlinux.lds.S | 1 +
arch/x86/kernel/vmlinux.lds.S | 1 +
include/asm-generic/vmlinux.lds.h | 12 +++++++++++-
include/linux/ftrace.h | 31 ++++++++++++++++++++-----------
kernel/softirq.c | 3 ++-
19 files changed, 49 insertions(+), 13 deletions(-)
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index 8b60fde..28b690fc 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -105,6 +105,7 @@ SECTIONS
*(.exception.text)
__exception_text_end = .;
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
LOCK_TEXT
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index e3928f5..b9242b7 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -102,6 +102,7 @@ SECTIONS
*(.exception.text)
__exception_text_end = .;
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
LOCK_TEXT
diff --git a/arch/blackfin/kernel/vmlinux.lds.S b/arch/blackfin/kernel/vmlinux.lds.S
index c9eec84..d920b95 100644
--- a/arch/blackfin/kernel/vmlinux.lds.S
+++ b/arch/blackfin/kernel/vmlinux.lds.S
@@ -35,6 +35,7 @@ SECTIONS
#endif
LOCK_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
KPROBES_TEXT
#ifdef CONFIG_ROMKERNEL
__sinittext = .;
diff --git a/arch/c6x/kernel/vmlinux.lds.S b/arch/c6x/kernel/vmlinux.lds.S
index 5a6e141..50bc10f 100644
--- a/arch/c6x/kernel/vmlinux.lds.S
+++ b/arch/c6x/kernel/vmlinux.lds.S
@@ -72,6 +72,7 @@ SECTIONS
SCHED_TEXT
LOCK_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
KPROBES_TEXT
*(.fixup)
*(.gnu.warning)
diff --git a/arch/metag/kernel/vmlinux.lds.S b/arch/metag/kernel/vmlinux.lds.S
index e12055e..150ace9 100644
--- a/arch/metag/kernel/vmlinux.lds.S
+++ b/arch/metag/kernel/vmlinux.lds.S
@@ -24,6 +24,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.text.*)
*(.gnu.warning)
}
diff --git a/arch/microblaze/kernel/vmlinux.lds.S b/arch/microblaze/kernel/vmlinux.lds.S
index be9488d..0a47f04 100644
--- a/arch/microblaze/kernel/vmlinux.lds.S
+++ b/arch/microblaze/kernel/vmlinux.lds.S
@@ -36,6 +36,7 @@ SECTIONS {
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
. = ALIGN (4) ;
_etext = . ;
}
diff --git a/arch/mips/kernel/vmlinux.lds.S b/arch/mips/kernel/vmlinux.lds.S
index 0a93e83..54d653e 100644
--- a/arch/mips/kernel/vmlinux.lds.S
+++ b/arch/mips/kernel/vmlinux.lds.S
@@ -58,6 +58,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.text.*)
*(.fixup)
*(.gnu.warning)
diff --git a/arch/nios2/kernel/vmlinux.lds.S b/arch/nios2/kernel/vmlinux.lds.S
index 326fab4..e23e895 100644
--- a/arch/nios2/kernel/vmlinux.lds.S
+++ b/arch/nios2/kernel/vmlinux.lds.S
@@ -39,6 +39,7 @@ SECTIONS
SCHED_TEXT
LOCK_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
KPROBES_TEXT
} =0
_etext = .;
diff --git a/arch/openrisc/kernel/vmlinux.lds.S b/arch/openrisc/kernel/vmlinux.lds.S
index 2d69a85..d936de4 100644
--- a/arch/openrisc/kernel/vmlinux.lds.S
+++ b/arch/openrisc/kernel/vmlinux.lds.S
@@ -50,6 +50,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.text.__*)
_etext = .;
diff --git a/arch/parisc/kernel/vmlinux.lds.S b/arch/parisc/kernel/vmlinux.lds.S
index 308f290..f3ead0b 100644
--- a/arch/parisc/kernel/vmlinux.lds.S
+++ b/arch/parisc/kernel/vmlinux.lds.S
@@ -72,6 +72,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.text.do_softirq)
*(.text.sys_exit)
*(.text.do_sigaltstack)
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index d41fd0a..2dd91f7 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -55,6 +55,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
#ifdef CONFIG_PPC32
*(.got1)
diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index 445657f..0f41a82 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -28,6 +28,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.gnu.warning)
} :text = 0x0700
diff --git a/arch/sh/kernel/vmlinux.lds.S b/arch/sh/kernel/vmlinux.lds.S
index db88cbf..235a410 100644
--- a/arch/sh/kernel/vmlinux.lds.S
+++ b/arch/sh/kernel/vmlinux.lds.S
@@ -39,6 +39,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.gnu.warning)
_etext = .; /* End of text section */
diff --git a/arch/sparc/kernel/vmlinux.lds.S b/arch/sparc/kernel/vmlinux.lds.S
index f1a2f68..aadd321 100644
--- a/arch/sparc/kernel/vmlinux.lds.S
+++ b/arch/sparc/kernel/vmlinux.lds.S
@@ -48,6 +48,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.gnu.warning)
} = 0
_etext = .;
diff --git a/arch/tile/kernel/vmlinux.lds.S b/arch/tile/kernel/vmlinux.lds.S
index 0e059a0..378f5d8 100644
--- a/arch/tile/kernel/vmlinux.lds.S
+++ b/arch/tile/kernel/vmlinux.lds.S
@@ -45,6 +45,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
__fix_text_end = .; /* tile-cpack won't rearrange before this */
ALIGN_FUNCTION();
*(.hottext*)
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 74e4bf1..056a97a 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -102,6 +102,7 @@ SECTIONS
KPROBES_TEXT
ENTRY_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.gnu.warning)
/* End of text section */
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index c4bd0e2..b470421 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -456,7 +456,7 @@
*(.entry.text) \
VMLINUX_SYMBOL(__entry_text_end) = .;
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
#define IRQENTRY_TEXT \
ALIGN_FUNCTION(); \
VMLINUX_SYMBOL(__irqentry_text_start) = .; \
@@ -466,6 +466,16 @@
#define IRQENTRY_TEXT
#endif
+#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
+#define SOFTIRQENTRY_TEXT \
+ ALIGN_FUNCTION(); \
+ VMLINUX_SYMBOL(__softirqentry_text_start) = .; \
+ *(.softirqentry.text) \
+ VMLINUX_SYMBOL(__softirqentry_text_end) = .;
+#else
+#define SOFTIRQENTRY_TEXT
+#endif
+
/* Section used for early init (in .S files) */
#define HEAD_TEXT *(.head.text)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 0639dcc..2a143fa 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -762,6 +762,26 @@ struct ftrace_graph_ret {
typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
+#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
+/*
+ * We want to know which function is an entrypoint of a hardirq.
+ */
+#define __irq_entry __attribute__((__section__(".irqentry.text")))
+#define __softirq_entry \
+ __attribute__((__section__(".softirqentry.text")))
+
+/* Limits of hardirq entrypoints */
+extern char __irqentry_text_start[];
+extern char __irqentry_text_end[];
+/* Limits of softirq entrypoints */
+extern char __softirqentry_text_start[];
+extern char __softirqentry_text_end[];
+
+#else
+#define __irq_entry
+#define __softirq_entry
+#endif
+
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
/* for init task */
@@ -798,16 +818,6 @@ ftrace_push_return_trace(unsigned long ret, unsigned long func, int *depth,
*/
#define __notrace_funcgraph notrace
-/*
- * We want to which function is an entrypoint of a hardirq.
- * That will help us to put a signal on output.
- */
-#define __irq_entry __attribute__((__section__(".irqentry.text")))
-
-/* Limits of hardirq entrypoints */
-extern char __irqentry_text_start[];
-extern char __irqentry_text_end[];
-
#define FTRACE_NOTRACE_DEPTH 65536
#define FTRACE_RETFUNC_DEPTH 50
#define FTRACE_RETSTACK_ALLOC_SIZE 32
@@ -844,7 +854,6 @@ static inline void unpause_graph_tracing(void)
#else /* !CONFIG_FUNCTION_GRAPH_TRACER */
#define __notrace_funcgraph
-#define __irq_entry
#define INIT_FTRACE_GRAPH
static inline void ftrace_graph_init_task(struct task_struct *t) { }
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 479e443..2b31bb0 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -11,6 +11,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/export.h>
+#include <linux/ftrace.h>
#include <linux/kernel_stat.h>
#include <linux/interrupt.h>
#include <linux/init.h>
@@ -227,7 +228,7 @@ static inline bool lockdep_softirq_start(void) { return false; }
static inline void lockdep_softirq_end(bool in_hardirq) { }
#endif
-asmlinkage __visible void __do_softirq(void)
+asmlinkage __visible void __softirq_entry __do_softirq(void)
{
unsigned long end = jiffies + MAX_SOFTIRQ_TIME;
unsigned long old_flags = current->flags;
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
KASAN needs to know whether the allocation happens in an IRQ handler.
This lets us strip everything below the IRQ entry point to reduce the number
of unique stack traces needed to be stored.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
arch/arm/kernel/vmlinux.lds.S | 1 +
arch/arm64/kernel/vmlinux.lds.S | 1 +
arch/blackfin/kernel/vmlinux.lds.S | 1 +
arch/c6x/kernel/vmlinux.lds.S | 1 +
arch/metag/kernel/vmlinux.lds.S | 1 +
arch/microblaze/kernel/vmlinux.lds.S | 1 +
arch/mips/kernel/vmlinux.lds.S | 1 +
arch/nios2/kernel/vmlinux.lds.S | 1 +
arch/openrisc/kernel/vmlinux.lds.S | 1 +
arch/parisc/kernel/vmlinux.lds.S | 1 +
arch/powerpc/kernel/vmlinux.lds.S | 1 +
arch/s390/kernel/vmlinux.lds.S | 1 +
arch/sh/kernel/vmlinux.lds.S | 1 +
arch/sparc/kernel/vmlinux.lds.S | 1 +
arch/tile/kernel/vmlinux.lds.S | 1 +
arch/x86/kernel/vmlinux.lds.S | 1 +
include/asm-generic/vmlinux.lds.h | 12 +++++++++++-
include/linux/ftrace.h | 31 ++++++++++++++++++++-----------
kernel/softirq.c | 3 ++-
19 files changed, 49 insertions(+), 13 deletions(-)
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index 8b60fde..28b690fc 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -105,6 +105,7 @@ SECTIONS
*(.exception.text)
__exception_text_end = .;
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
LOCK_TEXT
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index e3928f5..b9242b7 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -102,6 +102,7 @@ SECTIONS
*(.exception.text)
__exception_text_end = .;
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
LOCK_TEXT
diff --git a/arch/blackfin/kernel/vmlinux.lds.S b/arch/blackfin/kernel/vmlinux.lds.S
index c9eec84..d920b95 100644
--- a/arch/blackfin/kernel/vmlinux.lds.S
+++ b/arch/blackfin/kernel/vmlinux.lds.S
@@ -35,6 +35,7 @@ SECTIONS
#endif
LOCK_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
KPROBES_TEXT
#ifdef CONFIG_ROMKERNEL
__sinittext = .;
diff --git a/arch/c6x/kernel/vmlinux.lds.S b/arch/c6x/kernel/vmlinux.lds.S
index 5a6e141..50bc10f 100644
--- a/arch/c6x/kernel/vmlinux.lds.S
+++ b/arch/c6x/kernel/vmlinux.lds.S
@@ -72,6 +72,7 @@ SECTIONS
SCHED_TEXT
LOCK_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
KPROBES_TEXT
*(.fixup)
*(.gnu.warning)
diff --git a/arch/metag/kernel/vmlinux.lds.S b/arch/metag/kernel/vmlinux.lds.S
index e12055e..150ace9 100644
--- a/arch/metag/kernel/vmlinux.lds.S
+++ b/arch/metag/kernel/vmlinux.lds.S
@@ -24,6 +24,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.text.*)
*(.gnu.warning)
}
diff --git a/arch/microblaze/kernel/vmlinux.lds.S b/arch/microblaze/kernel/vmlinux.lds.S
index be9488d..0a47f04 100644
--- a/arch/microblaze/kernel/vmlinux.lds.S
+++ b/arch/microblaze/kernel/vmlinux.lds.S
@@ -36,6 +36,7 @@ SECTIONS {
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
. = ALIGN (4) ;
_etext = . ;
}
diff --git a/arch/mips/kernel/vmlinux.lds.S b/arch/mips/kernel/vmlinux.lds.S
index 0a93e83..54d653e 100644
--- a/arch/mips/kernel/vmlinux.lds.S
+++ b/arch/mips/kernel/vmlinux.lds.S
@@ -58,6 +58,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.text.*)
*(.fixup)
*(.gnu.warning)
diff --git a/arch/nios2/kernel/vmlinux.lds.S b/arch/nios2/kernel/vmlinux.lds.S
index 326fab4..e23e895 100644
--- a/arch/nios2/kernel/vmlinux.lds.S
+++ b/arch/nios2/kernel/vmlinux.lds.S
@@ -39,6 +39,7 @@ SECTIONS
SCHED_TEXT
LOCK_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
KPROBES_TEXT
} =0
_etext = .;
diff --git a/arch/openrisc/kernel/vmlinux.lds.S b/arch/openrisc/kernel/vmlinux.lds.S
index 2d69a85..d936de4 100644
--- a/arch/openrisc/kernel/vmlinux.lds.S
+++ b/arch/openrisc/kernel/vmlinux.lds.S
@@ -50,6 +50,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.text.__*)
_etext = .;
diff --git a/arch/parisc/kernel/vmlinux.lds.S b/arch/parisc/kernel/vmlinux.lds.S
index 308f290..f3ead0b 100644
--- a/arch/parisc/kernel/vmlinux.lds.S
+++ b/arch/parisc/kernel/vmlinux.lds.S
@@ -72,6 +72,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.text.do_softirq)
*(.text.sys_exit)
*(.text.do_sigaltstack)
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index d41fd0a..2dd91f7 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -55,6 +55,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
#ifdef CONFIG_PPC32
*(.got1)
diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index 445657f..0f41a82 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -28,6 +28,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.gnu.warning)
} :text = 0x0700
diff --git a/arch/sh/kernel/vmlinux.lds.S b/arch/sh/kernel/vmlinux.lds.S
index db88cbf..235a410 100644
--- a/arch/sh/kernel/vmlinux.lds.S
+++ b/arch/sh/kernel/vmlinux.lds.S
@@ -39,6 +39,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.gnu.warning)
_etext = .; /* End of text section */
diff --git a/arch/sparc/kernel/vmlinux.lds.S b/arch/sparc/kernel/vmlinux.lds.S
index f1a2f68..aadd321 100644
--- a/arch/sparc/kernel/vmlinux.lds.S
+++ b/arch/sparc/kernel/vmlinux.lds.S
@@ -48,6 +48,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.gnu.warning)
} = 0
_etext = .;
diff --git a/arch/tile/kernel/vmlinux.lds.S b/arch/tile/kernel/vmlinux.lds.S
index 0e059a0..378f5d8 100644
--- a/arch/tile/kernel/vmlinux.lds.S
+++ b/arch/tile/kernel/vmlinux.lds.S
@@ -45,6 +45,7 @@ SECTIONS
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
__fix_text_end = .; /* tile-cpack won't rearrange before this */
ALIGN_FUNCTION();
*(.hottext*)
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 74e4bf1..056a97a 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -102,6 +102,7 @@ SECTIONS
KPROBES_TEXT
ENTRY_TEXT
IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
*(.fixup)
*(.gnu.warning)
/* End of text section */
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index c4bd0e2..b470421 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -456,7 +456,7 @@
*(.entry.text) \
VMLINUX_SYMBOL(__entry_text_end) = .;
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
#define IRQENTRY_TEXT \
ALIGN_FUNCTION(); \
VMLINUX_SYMBOL(__irqentry_text_start) = .; \
@@ -466,6 +466,16 @@
#define IRQENTRY_TEXT
#endif
+#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
+#define SOFTIRQENTRY_TEXT \
+ ALIGN_FUNCTION(); \
+ VMLINUX_SYMBOL(__softirqentry_text_start) = .; \
+ *(.softirqentry.text) \
+ VMLINUX_SYMBOL(__softirqentry_text_end) = .;
+#else
+#define SOFTIRQENTRY_TEXT
+#endif
+
/* Section used for early init (in .S files) */
#define HEAD_TEXT *(.head.text)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 0639dcc..2a143fa 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -762,6 +762,26 @@ struct ftrace_graph_ret {
typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
+#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
+/*
+ * We want to know which function is an entrypoint of a hardirq.
+ */
+#define __irq_entry __attribute__((__section__(".irqentry.text")))
+#define __softirq_entry \
+ __attribute__((__section__(".softirqentry.text")))
+
+/* Limits of hardirq entrypoints */
+extern char __irqentry_text_start[];
+extern char __irqentry_text_end[];
+/* Limits of softirq entrypoints */
+extern char __softirqentry_text_start[];
+extern char __softirqentry_text_end[];
+
+#else
+#define __irq_entry
+#define __softirq_entry
+#endif
+
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
/* for init task */
@@ -798,16 +818,6 @@ ftrace_push_return_trace(unsigned long ret, unsigned long func, int *depth,
*/
#define __notrace_funcgraph notrace
-/*
- * We want to which function is an entrypoint of a hardirq.
- * That will help us to put a signal on output.
- */
-#define __irq_entry __attribute__((__section__(".irqentry.text")))
-
-/* Limits of hardirq entrypoints */
-extern char __irqentry_text_start[];
-extern char __irqentry_text_end[];
-
#define FTRACE_NOTRACE_DEPTH 65536
#define FTRACE_RETFUNC_DEPTH 50
#define FTRACE_RETSTACK_ALLOC_SIZE 32
@@ -844,7 +854,6 @@ static inline void unpause_graph_tracing(void)
#else /* !CONFIG_FUNCTION_GRAPH_TRACER */
#define __notrace_funcgraph
-#define __irq_entry
#define INIT_FTRACE_GRAPH
static inline void ftrace_graph_init_task(struct task_struct *t) { }
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 479e443..2b31bb0 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -11,6 +11,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/export.h>
+#include <linux/ftrace.h>
#include <linux/kernel_stat.h>
#include <linux/interrupt.h>
#include <linux/init.h>
@@ -227,7 +228,7 @@ static inline bool lockdep_softirq_start(void) { return false; }
static inline void lockdep_softirq_end(bool in_hardirq) { }
#endif
-asmlinkage __visible void __do_softirq(void)
+asmlinkage __visible void __softirq_entry __do_softirq(void)
{
unsigned long end = jiffies + MAX_SOFTIRQ_TIME;
unsigned long old_flags = current->flags;
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Stack depot will allow KASAN store allocation/deallocation stack traces
for memory chunks. The stack traces are stored in a hash table and
referenced by handles which reside in the kasan_alloc_meta and
kasan_free_meta structures in the allocated memory chunks.
IRQ stack traces are cut below the IRQ entry point to avoid unnecessary
duplication.
Right now stackdepot support is only enabled in SLAB allocator.
Once KASAN features in SLAB are on par with those in SLUB we can switch
SLUB to stackdepot as well, thus removing the dependency on SLUB_DEBUG.
This patch is based on the "mm: kasan: stack depots" patch originally
prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
arch/x86/kernel/Makefile | 1 +
mm/kasan/Makefile | 3 +
mm/kasan/kasan.c | 51 +++++++++-
mm/kasan/kasan.h | 11 +++
mm/kasan/report.c | 8 ++
mm/kasan/stackdepot.c | 236 +++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 307 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index b1b78ff..500584d 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -19,6 +19,7 @@ endif
KASAN_SANITIZE_head$(BITS).o := n
KASAN_SANITIZE_dumpstack.o := n
KASAN_SANITIZE_dumpstack_$(BITS).o := n
+KASAN_SANITIZE_stacktrace.o := n
CFLAGS_irq.o := -I$(src)/../include/asm/trace
diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index 6471014..f952515 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -6,3 +6,6 @@ CFLAGS_REMOVE_kasan.o = -pg
CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
obj-y := kasan.o report.o kasan_init.o
+ifdef CONFIG_SLAB
+ obj-y += stackdepot.o
+endif
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index 787224a..b5d04ec 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -17,7 +17,9 @@
#define DISABLE_BRANCH_PROFILING
#include <linux/export.h>
+#include <linux/ftrace.h>
#include <linux/init.h>
+#include <linux/kasan.h>
#include <linux/kernel.h>
#include <linux/kmemleak.h>
#include <linux/memblock.h>
@@ -31,7 +33,6 @@
#include <linux/string.h>
#include <linux/types.h>
#include <linux/vmalloc.h>
-#include <linux/kasan.h>
#include "kasan.h"
#include "../slab.h"
@@ -393,23 +394,67 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object)
#endif
}
-static inline void set_track(struct kasan_track *track)
+static inline int in_irqentry_text(unsigned long ptr)
+{
+ return (ptr >= (unsigned long)&__irqentry_text_start &&
+ ptr < (unsigned long)&__irqentry_text_end) ||
+ (ptr >= (unsigned long)&__softirqentry_text_start &&
+ ptr < (unsigned long)&__softirqentry_text_end);
+}
+
+static inline void filter_irq_stacks(struct stack_trace *trace)
+{
+ int i;
+
+ if (!trace->nr_entries)
+ return;
+ for (i = 0; i < trace->nr_entries; i++)
+ if (in_irqentry_text(trace->entries[i])) {
+ /* Include the irqentry function into the stack. */
+ trace->nr_entries = i + 1;
+ break;
+ }
+}
+
+static inline kasan_stack_handle save_stack(gfp_t flags)
+{
+ unsigned long entries[KASAN_STACK_DEPTH];
+ struct stack_trace trace = {
+ .nr_entries = 0,
+ .entries = entries,
+ .max_entries = KASAN_STACK_DEPTH,
+ .skip = 0
+ };
+
+ save_stack_trace(&trace);
+ filter_irq_stacks(&trace);
+ if (trace.nr_entries != 0 &&
+ trace.entries[trace.nr_entries-1] == ULONG_MAX)
+ trace.nr_entries--;
+
+ return kasan_save_stack(&trace, flags);
+}
+
+static inline void set_track(struct kasan_track *track, gfp_t flags)
{
track->cpu = raw_smp_processor_id();
track->pid = current->pid;
track->when = jiffies;
+ track->stack = save_stack(flags);
}
#ifdef CONFIG_SLAB
struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
const void *object)
{
+ BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32);
return (void *)object + cache->kasan_info.alloc_meta_offset;
}
struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
const void *object)
{
+ BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
return (void *)object + cache->kasan_info.free_meta_offset;
}
#endif
@@ -455,7 +500,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
alloc_info->state = KASAN_STATE_ALLOC;
alloc_info->alloc_size = size;
- set_track(&alloc_info->track);
+ set_track(&alloc_info->track, flags);
}
#endif
}
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 7b9e4ab9..eb9de369 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -64,10 +64,15 @@ enum kasan_state {
KASAN_STATE_FREE
};
+#define KASAN_STACK_DEPTH 64
+#define KASAN_STACK_BITS (32) /* up to 16GB of stack storage */
+typedef u32 kasan_stack_handle;
+
struct kasan_track {
u64 cpu : 6; /* for NR_CPUS = 64 */
u64 pid : 16; /* 65536 processes */
u64 when : 42; /* ~140 years */
+ kasan_stack_handle stack : KASAN_STACK_BITS;
};
struct kasan_alloc_meta {
@@ -102,4 +107,10 @@ static inline bool kasan_report_enabled(void)
void kasan_report(unsigned long addr, size_t size,
bool is_write, unsigned long ip);
+struct stack_trace;
+
+kasan_stack_handle kasan_save_stack(struct stack_trace *trace, gfp_t flags);
+
+void kasan_fetch_stack(kasan_stack_handle handle, struct stack_trace *trace);
+
#endif
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 2bf7218..6c4afcd 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -119,6 +119,14 @@ static void print_track(struct kasan_track *track)
{
pr_err("PID = %lu, CPU = %lu, timestamp = %lu\n", track->pid,
track->cpu, track->when);
+ if (track->stack) {
+ struct stack_trace trace;
+
+ kasan_fetch_stack(track->stack, &trace);
+ print_stack_trace(&trace, 0);
+ } else {
+ pr_err("(stack is not available)\n");
+ }
}
static void print_object(struct kmem_cache *cache, void *object)
diff --git a/mm/kasan/stackdepot.c b/mm/kasan/stackdepot.c
new file mode 100644
index 0000000..e3026a5
--- /dev/null
+++ b/mm/kasan/stackdepot.c
@@ -0,0 +1,236 @@
+/*
+ * Stack depot
+ * KASAN needs to safe alloc and free stacks per object, but storing 2 stack
+ * traces per object is too much overhead (e.g. SLUB_DEBUG needs 256 bytes per
+ * object).
+ *
+ * Instead, stack depot maintains a hashtable of unique stacktraces. Since alloc
+ * and free stacks repeat a lot, we save about 100x space.
+ * Stacks are never removed from depot, so we store them contiguously one after
+ * another in a contiguos memory allocation.
+ */
+
+
+#include <linux/gfp.h>
+#include <linux/jhash.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/percpu.h>
+#include <linux/printk.h>
+#include <linux/stacktrace.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "kasan.h"
+
+#define STACK_ALLOC_ORDER 4 /* 'Slab' size order for stack depot, 16 pages */
+#define STACK_ALLOC_SIZE (1L << (PAGE_SHIFT + STACK_ALLOC_ORDER))
+#define STACK_ALLOC_ALIGN 4
+#define STACK_ALLOC_OFFSET_BITS (STACK_ALLOC_ORDER + PAGE_SHIFT - \
+ STACK_ALLOC_ALIGN)
+#define STACK_ALLOC_INDEX_BITS (KASAN_STACK_BITS - STACK_ALLOC_OFFSET_BITS)
+#define STACK_ALLOC_SLABS_CAP 1024
+#define STACK_ALLOC_MAX_SLABS \
+ (((1L << (STACK_ALLOC_INDEX_BITS)) < STACK_ALLOC_SLABS_CAP) ? \
+ (1L << (STACK_ALLOC_INDEX_BITS)) : STACK_ALLOC_SLABS_CAP)
+
+/* The compact structure to store the reference to stacks. */
+union handle_parts {
+ kasan_stack_handle handle;
+ struct {
+ u32 slabindex : STACK_ALLOC_INDEX_BITS;
+ u32 offset : STACK_ALLOC_OFFSET_BITS;
+ };
+};
+
+struct kasan_stack {
+ struct kasan_stack *next; /* Link in the hashtable */
+ u32 hash; /* Hash in the hastable */
+ u32 size; /* Number of frames in the stack */
+ union handle_parts handle;
+ unsigned long entries[1]; /* Variable-sized array of entries. */
+};
+
+static void *stack_slabs[STACK_ALLOC_MAX_SLABS];
+
+static int depot_index;
+static int next_slab_inited;
+static size_t depot_offset;
+static DEFINE_SPINLOCK(depot_lock);
+
+static bool init_stack_slab(void **prealloc)
+{
+ if (!*prealloc)
+ return false;
+ if (smp_load_acquire(&next_slab_inited))
+ return true;
+ if (stack_slabs[depot_index] == NULL) {
+ stack_slabs[depot_index] = *prealloc;
+ } else {
+ stack_slabs[depot_index + 1] = *prealloc;
+ smp_store_release(&next_slab_inited, 1);
+ }
+ *prealloc = NULL;
+ return true;
+}
+
+/* Allocation of a new stack in raw storage */
+static struct kasan_stack *kasan_alloc_stack(unsigned long *entries, int size,
+ u32 hash, void **prealloc, gfp_t alloc_flags)
+{
+ int required_size = offsetof(struct kasan_stack, entries) +
+ sizeof(unsigned long) * size;
+ struct kasan_stack *stack;
+
+ required_size = ALIGN(required_size, 1 << STACK_ALLOC_ALIGN);
+
+ if (unlikely(depot_offset + required_size > STACK_ALLOC_SIZE)) {
+ if (unlikely(depot_index + 1 >= STACK_ALLOC_MAX_SLABS)) {
+ WARN_ONCE(1, "Stack depot reached limit capacity");
+ return NULL;
+ }
+ depot_index++;
+ depot_offset = 0;
+ if (depot_index + 1 < STACK_ALLOC_MAX_SLABS)
+ smp_store_release(&next_slab_inited, 0);
+ }
+ init_stack_slab(prealloc);
+ if (stack_slabs[depot_index] == NULL)
+ return NULL;
+
+ stack = stack_slabs[depot_index] + depot_offset;
+
+ stack->hash = hash;
+ stack->size = size;
+ stack->handle.slabindex = depot_index;
+ stack->handle.offset = depot_offset >> STACK_ALLOC_ALIGN;
+ __memcpy(stack->entries, entries, size * sizeof(unsigned long));
+ depot_offset += required_size;
+
+ return stack;
+}
+
+#define STACK_HASH_ORDER 20
+#define STACK_HASH_SIZE (1L << STACK_HASH_ORDER)
+#define STACK_HASH_MASK (STACK_HASH_SIZE - 1)
+#define STACK_HASH_SEED 0x9747b28c
+
+static struct kasan_stack *stack_table[STACK_HASH_SIZE] = {
+ [0 ... STACK_HASH_SIZE - 1] = NULL
+};
+
+/* Calculate hash for a stack */
+static inline u32 hash_stack(unsigned long *entries, unsigned int size)
+{
+ return jhash2((u32 *)entries,
+ size * sizeof(unsigned long) / sizeof(u32),
+ STACK_HASH_SEED);
+}
+
+/* Find a stack that is equal to the one stored in entries in the hash */
+static inline struct kasan_stack *find_stack(struct kasan_stack *bucket,
+ unsigned long *entries, int size,
+ u32 hash)
+{
+ struct kasan_stack *found;
+
+ for (found = bucket; found; found = found->next) {
+ if (found->hash == hash &&
+ found->size == size &&
+ !memcmp(entries, found->entries,
+ size * sizeof(unsigned long))) {
+ return found;
+ }
+ }
+ return NULL;
+}
+
+void kasan_fetch_stack(kasan_stack_handle handle, struct stack_trace *trace)
+{
+ union handle_parts parts = { .handle = handle };
+ void *slab = stack_slabs[parts.slabindex];
+ size_t offset = parts.offset << STACK_ALLOC_ALIGN;
+ struct kasan_stack *stack = slab + offset;
+
+ trace->nr_entries = trace->max_entries = stack->size;
+ trace->entries = stack->entries;
+ trace->skip = 0;
+}
+
+/*
+ * kasan_save_stack - save stack in a stack depot.
+ * @trace - the stacktrace to save.
+ * @alloc_flags - flags for allocating additional memory if required.
+ *
+ * Returns the handle of the stack struct stored in depot.
+ */
+kasan_stack_handle kasan_save_stack(struct stack_trace *trace,
+ gfp_t alloc_flags)
+{
+ u32 hash;
+ kasan_stack_handle retval = 0;
+ struct kasan_stack *found = NULL, **bucket;
+ unsigned long flags;
+ struct page *page = NULL;
+ void *prealloc = NULL;
+
+ if (unlikely(trace->nr_entries == 0))
+ goto exit;
+
+ hash = hash_stack(trace->entries, trace->nr_entries);
+ /* Bad luck, we won't store this stack. */
+ if (hash == 0)
+ goto exit;
+
+ bucket = &stack_table[hash & STACK_HASH_MASK];
+
+ /* Fast path: look the stack trace up without locking. */
+ found = find_stack(smp_load_acquire(bucket), trace->entries,
+ trace->nr_entries, hash);
+ if (found)
+ goto exit;
+
+ /* Check if the current or the next stack slab need to be initialized.
+ * If so, allocate the memory - we won't be able to do that under the
+ * lock.
+ */
+ if (unlikely(!smp_load_acquire(&next_slab_inited))) {
+ if (!preempt_count() && !in_irq()) {
+ alloc_flags &= (__GFP_RECLAIM | __GFP_IO | __GFP_FS |
+ __GFP_NOWARN | __GFP_NORETRY |
+ __GFP_NOMEMALLOC | __GFP_DIRECT_RECLAIM);
+ page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER);
+ if (page)
+ prealloc = page_address(page);
+ }
+ }
+
+ spin_lock_irqsave(&depot_lock, flags);
+
+ found = find_stack(*bucket, trace->entries, trace->nr_entries, hash);
+ if (!found) {
+ struct kasan_stack *new =
+ kasan_alloc_stack(trace->entries, trace->nr_entries,
+ hash, &prealloc, alloc_flags);
+ if (new) {
+ new->next = *bucket;
+ smp_store_release(bucket, new);
+ found = new;
+ }
+ } else if (prealloc) {
+ /*
+ * We didn't need to store this stack trace, but let's keep
+ * the preallocated memory for the future.
+ */
+ WARN_ON(!init_stack_slab(&prealloc));
+ }
+
+ spin_unlock_irqrestore(&depot_lock, flags);
+exit:
+ if (prealloc)
+ /* Nobody used this memory, ok to free it. */
+ free_pages((unsigned long)prealloc, STACK_ALLOC_ORDER);
+ if (found)
+ retval = found->handle.handle;
+ return retval;
+}
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Stack depot will allow KASAN store allocation/deallocation stack traces
for memory chunks. The stack traces are stored in a hash table and
referenced by handles which reside in the kasan_alloc_meta and
kasan_free_meta structures in the allocated memory chunks.
IRQ stack traces are cut below the IRQ entry point to avoid unnecessary
duplication.
Right now stackdepot support is only enabled in SLAB allocator.
Once KASAN features in SLAB are on par with those in SLUB we can switch
SLUB to stackdepot as well, thus removing the dependency on SLUB_DEBUG.
This patch is based on the "mm: kasan: stack depots" patch originally
prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
arch/x86/kernel/Makefile | 1 +
mm/kasan/Makefile | 3 +
mm/kasan/kasan.c | 51 +++++++++-
mm/kasan/kasan.h | 11 +++
mm/kasan/report.c | 8 ++
mm/kasan/stackdepot.c | 236 +++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 307 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index b1b78ff..500584d 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -19,6 +19,7 @@ endif
KASAN_SANITIZE_head$(BITS).o := n
KASAN_SANITIZE_dumpstack.o := n
KASAN_SANITIZE_dumpstack_$(BITS).o := n
+KASAN_SANITIZE_stacktrace.o := n
CFLAGS_irq.o := -I$(src)/../include/asm/trace
diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index 6471014..f952515 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -6,3 +6,6 @@ CFLAGS_REMOVE_kasan.o = -pg
CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
obj-y := kasan.o report.o kasan_init.o
+ifdef CONFIG_SLAB
+ obj-y += stackdepot.o
+endif
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index 787224a..b5d04ec 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -17,7 +17,9 @@
#define DISABLE_BRANCH_PROFILING
#include <linux/export.h>
+#include <linux/ftrace.h>
#include <linux/init.h>
+#include <linux/kasan.h>
#include <linux/kernel.h>
#include <linux/kmemleak.h>
#include <linux/memblock.h>
@@ -31,7 +33,6 @@
#include <linux/string.h>
#include <linux/types.h>
#include <linux/vmalloc.h>
-#include <linux/kasan.h>
#include "kasan.h"
#include "../slab.h"
@@ -393,23 +394,67 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object)
#endif
}
-static inline void set_track(struct kasan_track *track)
+static inline int in_irqentry_text(unsigned long ptr)
+{
+ return (ptr >= (unsigned long)&__irqentry_text_start &&
+ ptr < (unsigned long)&__irqentry_text_end) ||
+ (ptr >= (unsigned long)&__softirqentry_text_start &&
+ ptr < (unsigned long)&__softirqentry_text_end);
+}
+
+static inline void filter_irq_stacks(struct stack_trace *trace)
+{
+ int i;
+
+ if (!trace->nr_entries)
+ return;
+ for (i = 0; i < trace->nr_entries; i++)
+ if (in_irqentry_text(trace->entries[i])) {
+ /* Include the irqentry function into the stack. */
+ trace->nr_entries = i + 1;
+ break;
+ }
+}
+
+static inline kasan_stack_handle save_stack(gfp_t flags)
+{
+ unsigned long entries[KASAN_STACK_DEPTH];
+ struct stack_trace trace = {
+ .nr_entries = 0,
+ .entries = entries,
+ .max_entries = KASAN_STACK_DEPTH,
+ .skip = 0
+ };
+
+ save_stack_trace(&trace);
+ filter_irq_stacks(&trace);
+ if (trace.nr_entries != 0 &&
+ trace.entries[trace.nr_entries-1] == ULONG_MAX)
+ trace.nr_entries--;
+
+ return kasan_save_stack(&trace, flags);
+}
+
+static inline void set_track(struct kasan_track *track, gfp_t flags)
{
track->cpu = raw_smp_processor_id();
track->pid = current->pid;
track->when = jiffies;
+ track->stack = save_stack(flags);
}
#ifdef CONFIG_SLAB
struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
const void *object)
{
+ BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32);
return (void *)object + cache->kasan_info.alloc_meta_offset;
}
struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
const void *object)
{
+ BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32);
return (void *)object + cache->kasan_info.free_meta_offset;
}
#endif
@@ -455,7 +500,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
alloc_info->state = KASAN_STATE_ALLOC;
alloc_info->alloc_size = size;
- set_track(&alloc_info->track);
+ set_track(&alloc_info->track, flags);
}
#endif
}
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 7b9e4ab9..eb9de369 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -64,10 +64,15 @@ enum kasan_state {
KASAN_STATE_FREE
};
+#define KASAN_STACK_DEPTH 64
+#define KASAN_STACK_BITS (32) /* up to 16GB of stack storage */
+typedef u32 kasan_stack_handle;
+
struct kasan_track {
u64 cpu : 6; /* for NR_CPUS = 64 */
u64 pid : 16; /* 65536 processes */
u64 when : 42; /* ~140 years */
+ kasan_stack_handle stack : KASAN_STACK_BITS;
};
struct kasan_alloc_meta {
@@ -102,4 +107,10 @@ static inline bool kasan_report_enabled(void)
void kasan_report(unsigned long addr, size_t size,
bool is_write, unsigned long ip);
+struct stack_trace;
+
+kasan_stack_handle kasan_save_stack(struct stack_trace *trace, gfp_t flags);
+
+void kasan_fetch_stack(kasan_stack_handle handle, struct stack_trace *trace);
+
#endif
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 2bf7218..6c4afcd 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -119,6 +119,14 @@ static void print_track(struct kasan_track *track)
{
pr_err("PID = %lu, CPU = %lu, timestamp = %lu\n", track->pid,
track->cpu, track->when);
+ if (track->stack) {
+ struct stack_trace trace;
+
+ kasan_fetch_stack(track->stack, &trace);
+ print_stack_trace(&trace, 0);
+ } else {
+ pr_err("(stack is not available)\n");
+ }
}
static void print_object(struct kmem_cache *cache, void *object)
diff --git a/mm/kasan/stackdepot.c b/mm/kasan/stackdepot.c
new file mode 100644
index 0000000..e3026a5
--- /dev/null
+++ b/mm/kasan/stackdepot.c
@@ -0,0 +1,236 @@
+/*
+ * Stack depot
+ * KASAN needs to safe alloc and free stacks per object, but storing 2 stack
+ * traces per object is too much overhead (e.g. SLUB_DEBUG needs 256 bytes per
+ * object).
+ *
+ * Instead, stack depot maintains a hashtable of unique stacktraces. Since alloc
+ * and free stacks repeat a lot, we save about 100x space.
+ * Stacks are never removed from depot, so we store them contiguously one after
+ * another in a contiguos memory allocation.
+ */
+
+
+#include <linux/gfp.h>
+#include <linux/jhash.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/percpu.h>
+#include <linux/printk.h>
+#include <linux/stacktrace.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "kasan.h"
+
+#define STACK_ALLOC_ORDER 4 /* 'Slab' size order for stack depot, 16 pages */
+#define STACK_ALLOC_SIZE (1L << (PAGE_SHIFT + STACK_ALLOC_ORDER))
+#define STACK_ALLOC_ALIGN 4
+#define STACK_ALLOC_OFFSET_BITS (STACK_ALLOC_ORDER + PAGE_SHIFT - \
+ STACK_ALLOC_ALIGN)
+#define STACK_ALLOC_INDEX_BITS (KASAN_STACK_BITS - STACK_ALLOC_OFFSET_BITS)
+#define STACK_ALLOC_SLABS_CAP 1024
+#define STACK_ALLOC_MAX_SLABS \
+ (((1L << (STACK_ALLOC_INDEX_BITS)) < STACK_ALLOC_SLABS_CAP) ? \
+ (1L << (STACK_ALLOC_INDEX_BITS)) : STACK_ALLOC_SLABS_CAP)
+
+/* The compact structure to store the reference to stacks. */
+union handle_parts {
+ kasan_stack_handle handle;
+ struct {
+ u32 slabindex : STACK_ALLOC_INDEX_BITS;
+ u32 offset : STACK_ALLOC_OFFSET_BITS;
+ };
+};
+
+struct kasan_stack {
+ struct kasan_stack *next; /* Link in the hashtable */
+ u32 hash; /* Hash in the hastable */
+ u32 size; /* Number of frames in the stack */
+ union handle_parts handle;
+ unsigned long entries[1]; /* Variable-sized array of entries. */
+};
+
+static void *stack_slabs[STACK_ALLOC_MAX_SLABS];
+
+static int depot_index;
+static int next_slab_inited;
+static size_t depot_offset;
+static DEFINE_SPINLOCK(depot_lock);
+
+static bool init_stack_slab(void **prealloc)
+{
+ if (!*prealloc)
+ return false;
+ if (smp_load_acquire(&next_slab_inited))
+ return true;
+ if (stack_slabs[depot_index] == NULL) {
+ stack_slabs[depot_index] = *prealloc;
+ } else {
+ stack_slabs[depot_index + 1] = *prealloc;
+ smp_store_release(&next_slab_inited, 1);
+ }
+ *prealloc = NULL;
+ return true;
+}
+
+/* Allocation of a new stack in raw storage */
+static struct kasan_stack *kasan_alloc_stack(unsigned long *entries, int size,
+ u32 hash, void **prealloc, gfp_t alloc_flags)
+{
+ int required_size = offsetof(struct kasan_stack, entries) +
+ sizeof(unsigned long) * size;
+ struct kasan_stack *stack;
+
+ required_size = ALIGN(required_size, 1 << STACK_ALLOC_ALIGN);
+
+ if (unlikely(depot_offset + required_size > STACK_ALLOC_SIZE)) {
+ if (unlikely(depot_index + 1 >= STACK_ALLOC_MAX_SLABS)) {
+ WARN_ONCE(1, "Stack depot reached limit capacity");
+ return NULL;
+ }
+ depot_index++;
+ depot_offset = 0;
+ if (depot_index + 1 < STACK_ALLOC_MAX_SLABS)
+ smp_store_release(&next_slab_inited, 0);
+ }
+ init_stack_slab(prealloc);
+ if (stack_slabs[depot_index] == NULL)
+ return NULL;
+
+ stack = stack_slabs[depot_index] + depot_offset;
+
+ stack->hash = hash;
+ stack->size = size;
+ stack->handle.slabindex = depot_index;
+ stack->handle.offset = depot_offset >> STACK_ALLOC_ALIGN;
+ __memcpy(stack->entries, entries, size * sizeof(unsigned long));
+ depot_offset += required_size;
+
+ return stack;
+}
+
+#define STACK_HASH_ORDER 20
+#define STACK_HASH_SIZE (1L << STACK_HASH_ORDER)
+#define STACK_HASH_MASK (STACK_HASH_SIZE - 1)
+#define STACK_HASH_SEED 0x9747b28c
+
+static struct kasan_stack *stack_table[STACK_HASH_SIZE] = {
+ [0 ... STACK_HASH_SIZE - 1] = NULL
+};
+
+/* Calculate hash for a stack */
+static inline u32 hash_stack(unsigned long *entries, unsigned int size)
+{
+ return jhash2((u32 *)entries,
+ size * sizeof(unsigned long) / sizeof(u32),
+ STACK_HASH_SEED);
+}
+
+/* Find a stack that is equal to the one stored in entries in the hash */
+static inline struct kasan_stack *find_stack(struct kasan_stack *bucket,
+ unsigned long *entries, int size,
+ u32 hash)
+{
+ struct kasan_stack *found;
+
+ for (found = bucket; found; found = found->next) {
+ if (found->hash == hash &&
+ found->size == size &&
+ !memcmp(entries, found->entries,
+ size * sizeof(unsigned long))) {
+ return found;
+ }
+ }
+ return NULL;
+}
+
+void kasan_fetch_stack(kasan_stack_handle handle, struct stack_trace *trace)
+{
+ union handle_parts parts = { .handle = handle };
+ void *slab = stack_slabs[parts.slabindex];
+ size_t offset = parts.offset << STACK_ALLOC_ALIGN;
+ struct kasan_stack *stack = slab + offset;
+
+ trace->nr_entries = trace->max_entries = stack->size;
+ trace->entries = stack->entries;
+ trace->skip = 0;
+}
+
+/*
+ * kasan_save_stack - save stack in a stack depot.
+ * @trace - the stacktrace to save.
+ * @alloc_flags - flags for allocating additional memory if required.
+ *
+ * Returns the handle of the stack struct stored in depot.
+ */
+kasan_stack_handle kasan_save_stack(struct stack_trace *trace,
+ gfp_t alloc_flags)
+{
+ u32 hash;
+ kasan_stack_handle retval = 0;
+ struct kasan_stack *found = NULL, **bucket;
+ unsigned long flags;
+ struct page *page = NULL;
+ void *prealloc = NULL;
+
+ if (unlikely(trace->nr_entries == 0))
+ goto exit;
+
+ hash = hash_stack(trace->entries, trace->nr_entries);
+ /* Bad luck, we won't store this stack. */
+ if (hash == 0)
+ goto exit;
+
+ bucket = &stack_table[hash & STACK_HASH_MASK];
+
+ /* Fast path: look the stack trace up without locking. */
+ found = find_stack(smp_load_acquire(bucket), trace->entries,
+ trace->nr_entries, hash);
+ if (found)
+ goto exit;
+
+ /* Check if the current or the next stack slab need to be initialized.
+ * If so, allocate the memory - we won't be able to do that under the
+ * lock.
+ */
+ if (unlikely(!smp_load_acquire(&next_slab_inited))) {
+ if (!preempt_count() && !in_irq()) {
+ alloc_flags &= (__GFP_RECLAIM | __GFP_IO | __GFP_FS |
+ __GFP_NOWARN | __GFP_NORETRY |
+ __GFP_NOMEMALLOC | __GFP_DIRECT_RECLAIM);
+ page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER);
+ if (page)
+ prealloc = page_address(page);
+ }
+ }
+
+ spin_lock_irqsave(&depot_lock, flags);
+
+ found = find_stack(*bucket, trace->entries, trace->nr_entries, hash);
+ if (!found) {
+ struct kasan_stack *new =
+ kasan_alloc_stack(trace->entries, trace->nr_entries,
+ hash, &prealloc, alloc_flags);
+ if (new) {
+ new->next = *bucket;
+ smp_store_release(bucket, new);
+ found = new;
+ }
+ } else if (prealloc) {
+ /*
+ * We didn't need to store this stack trace, but let's keep
+ * the preallocated memory for the future.
+ */
+ WARN_ON(!init_stack_slab(&prealloc));
+ }
+
+ spin_unlock_irqrestore(&depot_lock, flags);
+exit:
+ if (prealloc)
+ /* Nobody used this memory, ok to free it. */
+ free_pages((unsigned long)prealloc, STACK_ALLOC_ORDER);
+ if (found)
+ retval = found->handle.handle;
+ return retval;
+}
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 6/8] kasan: Test fix: Warn if the UAF could not be detected in kmalloc_uaf2
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Signed-off-by: Alexander Potapenko <glider@google.com>
---
lib/test_kasan.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 66dd92f..5498a78 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -286,6 +286,8 @@ static noinline void __init kmalloc_uaf2(void)
}
ptr1[40] = 'x';
+ if (ptr1 == ptr2)
+ pr_err("Could not detect use-after-free: ptr1 == ptr2\n");
kfree(ptr2);
}
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 6/8] kasan: Test fix: Warn if the UAF could not be detected in kmalloc_uaf2
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Signed-off-by: Alexander Potapenko <glider@google.com>
---
lib/test_kasan.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 66dd92f..5498a78 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -286,6 +286,8 @@ static noinline void __init kmalloc_uaf2(void)
}
ptr1[40] = 'x';
+ if (ptr1 == ptr2)
+ pr_err("Could not detect use-after-free: ptr1 == ptr2\n");
kfree(ptr2);
}
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 7/8] kasan: Changed kmalloc_large_oob_right, added kmalloc_pagealloc_oob_right
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Signed-off-by: Alexander Potapenko <glider@google.com>
---
lib/test_kasan.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 5498a78..822c804 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -65,7 +65,8 @@ static noinline void __init kmalloc_node_oob_right(void)
kfree(ptr);
}
-static noinline void __init kmalloc_large_oob_right(void)
+#ifdef CONFIG_SLUB
+static noinline void __init kmalloc_pagealloc_oob_right(void)
{
char *ptr;
size_t size;
@@ -85,6 +86,18 @@ static noinline void __init kmalloc_large_oob_right(void)
size = KMALLOC_MAX_CACHE_SIZE + 10;
}
+ ptr[size] = 0;
+ kfree(ptr);
+}
+#endif
+
+static noinline void __init kmalloc_large_oob_right(void)
+{
+ char *ptr;
+ size_t size = KMALLOC_MAX_CACHE_SIZE - 256;
+ /* Allocate a chunk that is large enough, but still fits into a slab
+ * and does not trigger the page allocator fallback in SLUB.
+ */
pr_info("kmalloc large allocation: out-of-bounds to right\n");
ptr = kmalloc(size, GFP_KERNEL);
if (!ptr) {
@@ -341,6 +354,9 @@ static int __init kmalloc_tests_init(void)
kmalloc_oob_right();
kmalloc_oob_left();
kmalloc_node_oob_right();
+#ifdef CONFIG_SLUB
+ kmalloc_pagealloc_oob_right();
+#endif
kmalloc_large_oob_right();
kmalloc_oob_krealloc_more();
kmalloc_oob_krealloc_less();
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 7/8] kasan: Changed kmalloc_large_oob_right, added kmalloc_pagealloc_oob_right
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Signed-off-by: Alexander Potapenko <glider@google.com>
---
lib/test_kasan.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 5498a78..822c804 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -65,7 +65,8 @@ static noinline void __init kmalloc_node_oob_right(void)
kfree(ptr);
}
-static noinline void __init kmalloc_large_oob_right(void)
+#ifdef CONFIG_SLUB
+static noinline void __init kmalloc_pagealloc_oob_right(void)
{
char *ptr;
size_t size;
@@ -85,6 +86,18 @@ static noinline void __init kmalloc_large_oob_right(void)
size = KMALLOC_MAX_CACHE_SIZE + 10;
}
+ ptr[size] = 0;
+ kfree(ptr);
+}
+#endif
+
+static noinline void __init kmalloc_large_oob_right(void)
+{
+ char *ptr;
+ size_t size = KMALLOC_MAX_CACHE_SIZE - 256;
+ /* Allocate a chunk that is large enough, but still fits into a slab
+ * and does not trigger the page allocator fallback in SLUB.
+ */
pr_info("kmalloc large allocation: out-of-bounds to right\n");
ptr = kmalloc(size, GFP_KERNEL);
if (!ptr) {
@@ -341,6 +354,9 @@ static int __init kmalloc_tests_init(void)
kmalloc_oob_right();
kmalloc_oob_left();
kmalloc_node_oob_right();
+#ifdef CONFIG_SLUB
+ kmalloc_pagealloc_oob_right();
+#endif
kmalloc_large_oob_right();
kmalloc_oob_krealloc_more();
kmalloc_oob_krealloc_less();
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-27 18:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Quarantine isolates freed objects in a separate queue. The objects are
returned to the allocator later, which helps to detect use-after-free
errors.
Freed objects are first added to per-cpu quarantine queues.
When a cache is destroyed or memory shrinking is requested, the objects
are moved into the global quarantine queue. Whenever a kmalloc call
allows memory reclaiming, the oldest objects are popped out of the
global queue until the total size of objects in quarantine is less than
3/4 of the maximum quarantine size (which is a fraction of installed
physical memory).
Right now quarantine support is only enabled in SLAB allocator.
Unification of KASAN features in SLAB and SLUB will be done later.
This patch is based on the "mm: kasan: quarantine" patch originally
prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
include/linux/kasan.h | 30 ++++--
lib/test_kasan.c | 29 ++++++
mm/kasan/Makefile | 2 +-
mm/kasan/kasan.c | 68 +++++++++++-
mm/kasan/kasan.h | 11 +-
mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
mm/kasan/report.c | 3 +-
mm/mempool.c | 7 +-
mm/page_alloc.c | 2 +-
mm/slab.c | 12 ++-
mm/slab.h | 4 +
mm/slab_common.c | 2 +
mm/slub.c | 4 +-
13 files changed, 435 insertions(+), 23 deletions(-)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index bf71ab0..355e722 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -44,24 +44,29 @@ static inline void kasan_disable_current(void)
void kasan_unpoison_shadow(const void *address, size_t size);
void kasan_alloc_pages(struct page *page, unsigned int order);
-void kasan_free_pages(struct page *page, unsigned int order);
+void kasan_poison_free_pages(struct page *page, unsigned int order);
void kasan_cache_create(struct kmem_cache *cache, size_t *size,
unsigned long *flags);
+void kasan_cache_shrink(struct kmem_cache *cache);
+void kasan_cache_destroy(struct kmem_cache *cache);
void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
void kasan_poison_object_data(struct kmem_cache *cache, void *object);
void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
-void kasan_kfree_large(const void *ptr);
-void kasan_kfree(void *ptr);
+void kasan_poison_kfree_large(const void *ptr);
+void kasan_poison_kfree(void *ptr);
void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
gfp_t flags);
void kasan_krealloc(const void *object, size_t new_size, gfp_t flags);
void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
-void kasan_slab_free(struct kmem_cache *s, void *object);
+/* kasan_slab_free() returns true if the object has been put into quarantine.
+ */
+bool kasan_slab_free(struct kmem_cache *s, void *object);
+void kasan_poison_slab_free(struct kmem_cache *s, void *object);
struct kasan_cache {
int alloc_meta_offset;
@@ -79,11 +84,14 @@ static inline void kasan_enable_current(void) {}
static inline void kasan_disable_current(void) {}
static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
-static inline void kasan_free_pages(struct page *page, unsigned int order) {}
+static inline void kasan_poison_free_pages(struct page *page,
+ unsigned int order) {}
static inline void kasan_cache_create(struct kmem_cache *cache,
size_t *size,
unsigned long *flags) {}
+static inline void kasan_cache_shrink(struct kmem_cache *cache) {}
+static inline void kasan_cache_destroy(struct kmem_cache *cache) {}
static inline void kasan_poison_slab(struct page *page) {}
static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
@@ -92,8 +100,8 @@ static inline void kasan_poison_object_data(struct kmem_cache *cache,
void *object) {}
static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {}
-static inline void kasan_kfree_large(const void *ptr) {}
-static inline void kasan_kfree(void *ptr) {}
+static inline void kasan_poison_kfree_large(const void *ptr) {}
+static inline void kasan_poison_kfree(void *ptr) {}
static inline void kasan_kmalloc(struct kmem_cache *s, const void *object,
size_t size, gfp_t flags) {}
static inline void kasan_krealloc(const void *object, size_t new_size,
@@ -101,7 +109,13 @@ static inline void kasan_krealloc(const void *object, size_t new_size,
static inline void kasan_slab_alloc(struct kmem_cache *s, void *object,
gfp_t flags) {}
-static inline void kasan_slab_free(struct kmem_cache *s, void *object) {}
+/* kasan_slab_free() returns true if the object has been put into quarantine.
+ */
+static inline bool kasan_slab_free(struct kmem_cache *s, void *object)
+{
+ return false;
+}
+static inline void kasan_poison_slab_free(struct kmem_cache *s, void *object) {}
static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
static inline void kasan_free_shadow(const struct vm_struct *vm) {}
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 822c804..6eb6d42 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -349,6 +349,32 @@ static noinline void __init kasan_stack_oob(void)
*(volatile char *)p;
}
+#ifdef CONFIG_SLAB
+static noinline void __init kasan_quarantine_cache(void)
+{
+ struct kmem_cache *cache = kmem_cache_create(
+ "test", 137, 8, GFP_KERNEL, NULL);
+ int i;
+
+ for (i = 0; i < 100; i++) {
+ void *p = kmem_cache_alloc(cache, GFP_KERNEL);
+
+ kmem_cache_free(cache, p);
+ p = kmalloc(sizeof(u64), GFP_KERNEL);
+ kfree(p);
+ }
+ kmem_cache_shrink(cache);
+ for (i = 0; i < 100; i++) {
+ u64 *p = kmem_cache_alloc(cache, GFP_KERNEL);
+
+ kmem_cache_free(cache, p);
+ p = kmalloc(sizeof(u64), GFP_KERNEL);
+ kfree(p);
+ }
+ kmem_cache_destroy(cache);
+}
+#endif
+
static int __init kmalloc_tests_init(void)
{
kmalloc_oob_right();
@@ -372,6 +398,9 @@ static int __init kmalloc_tests_init(void)
kmem_cache_oob();
kasan_stack_oob();
kasan_global_oob();
+#ifdef CONFIG_SLAB
+ kasan_quarantine_cache();
+#endif
return -EAGAIN;
}
diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index f952515..8e59350 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -7,5 +7,5 @@ CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
obj-y := kasan.o report.o kasan_init.o
ifdef CONFIG_SLAB
- obj-y += stackdepot.o
+ obj-y += stackdepot.o quarantine.o
endif
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index b5d04ec..43c079a 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -307,7 +307,7 @@ void kasan_alloc_pages(struct page *page, unsigned int order)
kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
}
-void kasan_free_pages(struct page *page, unsigned int order)
+void kasan_poison_free_pages(struct page *page, unsigned int order)
{
if (likely(!PageHighMem(page)))
kasan_poison_shadow(page_address(page),
@@ -368,6 +368,20 @@ void kasan_cache_create(struct kmem_cache *cache, size_t *size,
}
#endif
+void kasan_cache_shrink(struct kmem_cache *cache)
+{
+#ifdef CONFIG_SLAB
+ quarantine_remove_cache(cache);
+#endif
+}
+
+void kasan_cache_destroy(struct kmem_cache *cache)
+{
+#ifdef CONFIG_SLAB
+ quarantine_remove_cache(cache);
+#endif
+}
+
void kasan_poison_slab(struct page *page)
{
kasan_poison_shadow(page_address(page),
@@ -464,7 +478,7 @@ void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)
kasan_kmalloc(cache, object, cache->object_size, flags);
}
-void kasan_slab_free(struct kmem_cache *cache, void *object)
+void kasan_poison_slab_free(struct kmem_cache *cache, void *object)
{
unsigned long size = cache->object_size;
unsigned long rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
@@ -476,12 +490,53 @@ void kasan_slab_free(struct kmem_cache *cache, void *object)
kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
}
+bool kasan_slab_free(struct kmem_cache *cache, void *object)
+{
+#ifdef CONFIG_SLAB
+ /* RCU slabs could be legally used after free within the RCU period */
+ if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
+ return false;
+
+ if (likely(cache->flags & SLAB_KASAN)) {
+ struct kasan_alloc_meta *alloc_info =
+ get_alloc_info(cache, object);
+ struct kasan_free_meta *free_info =
+ get_free_info(cache, object);
+
+ switch (alloc_info->state) {
+ case KASAN_STATE_ALLOC:
+ alloc_info->state = KASAN_STATE_QUARANTINE;
+ quarantine_put(free_info, cache);
+ set_track(&free_info->track, GFP_NOWAIT);
+ kasan_poison_slab_free(cache, object);
+ return true;
+ case KASAN_STATE_QUARANTINE:
+ case KASAN_STATE_FREE:
+ pr_err("Double free");
+ dump_stack();
+ break;
+ default:
+ break;
+ }
+ }
+ return false;
+#else
+ kasan_poison_slab_free(cache, object);
+ return false;
+#endif
+}
+
void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
gfp_t flags)
{
unsigned long redzone_start;
unsigned long redzone_end;
+#ifdef CONFIG_SLAB
+ if (flags & __GFP_RECLAIM)
+ quarantine_reduce();
+#endif
+
if (unlikely(object == NULL))
return;
@@ -512,6 +567,11 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
unsigned long redzone_start;
unsigned long redzone_end;
+#ifdef CONFIG_SLAB
+ if (flags & __GFP_RECLAIM)
+ quarantine_reduce();
+#endif
+
if (unlikely(ptr == NULL))
return;
@@ -540,7 +600,7 @@ void kasan_krealloc(const void *object, size_t size, gfp_t flags)
kasan_kmalloc(page->slab_cache, object, size, flags);
}
-void kasan_kfree(void *ptr)
+void kasan_poison_kfree(void *ptr)
{
struct page *page;
@@ -553,7 +613,7 @@ void kasan_kfree(void *ptr)
kasan_slab_free(page->slab_cache, ptr);
}
-void kasan_kfree_large(const void *ptr)
+void kasan_poison_kfree_large(const void *ptr)
{
struct page *page = virt_to_page(ptr);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index eb9de369..0fe58d9 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -61,6 +61,7 @@ struct kasan_global {
enum kasan_state {
KASAN_STATE_INIT,
KASAN_STATE_ALLOC,
+ KASAN_STATE_QUARANTINE,
KASAN_STATE_FREE
};
@@ -82,8 +83,10 @@ struct kasan_alloc_meta {
};
struct kasan_free_meta {
- /* Allocator freelist pointer, unused by KASAN. */
- void **freelist;
+ /* This field is used while the object is in the quarantine.
+ * Otherwise it might be used for the allocator freelist.
+ */
+ void **quarantine_link;
struct kasan_track track;
};
@@ -113,4 +116,8 @@ kasan_stack_handle kasan_save_stack(struct stack_trace *trace, gfp_t flags);
void kasan_fetch_stack(kasan_stack_handle handle, struct stack_trace *trace);
+void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache);
+void quarantine_reduce(void);
+void quarantine_remove_cache(struct kmem_cache *cache);
+
#endif
diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c
new file mode 100644
index 0000000..10c0ad6
--- /dev/null
+++ b/mm/kasan/quarantine.c
@@ -0,0 +1,284 @@
+/* Kasan quarantine */
+
+#include <linux/gfp.h>
+#include <linux/hash.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/percpu.h>
+#include <linux/printk.h>
+#include <linux/shrinker.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "../slab.h"
+#include "kasan.h"
+
+/* Data structure and operations for quarantine queues. */
+
+/* Each queue is a signled-linked list, which also stores the total size of
+ * objects inside of it.
+ */
+struct qlist {
+ void **head;
+ void **tail;
+ size_t bytes;
+};
+
+#define QLIST_INIT { NULL, NULL, 0 }
+
+static inline bool empty_qlist(struct qlist *q)
+{
+ return !q->head;
+}
+
+static inline void init_qlist(struct qlist *q)
+{
+ q->head = q->tail = NULL;
+ q->bytes = 0;
+}
+
+static inline void qlist_put(struct qlist *q, void **qlink, size_t size)
+{
+ if (unlikely(empty_qlist(q)))
+ q->head = qlink;
+ else
+ *q->tail = qlink;
+ q->tail = qlink;
+ *qlink = NULL;
+ q->bytes += size;
+}
+
+static inline void **qlist_remove(struct qlist *q, void ***prev,
+ size_t size)
+{
+ void **qlink = *prev;
+
+ *prev = *qlink;
+ if (q->tail == qlink) {
+ if (q->head == qlink)
+ q->tail = NULL;
+ else
+ q->tail = (void **)prev;
+ }
+ q->bytes -= size;
+
+ return qlink;
+}
+
+static inline void qlist_move_all(struct qlist *from, struct qlist *to)
+{
+ if (unlikely(empty_qlist(from)))
+ return;
+
+ if (empty_qlist(to)) {
+ *to = *from;
+ init_qlist(from);
+ return;
+ }
+
+ *to->tail = from->head;
+ to->tail = from->tail;
+ to->bytes += from->bytes;
+
+ init_qlist(from);
+}
+
+static inline void qlist_move(struct qlist *from, void **last, struct qlist *to,
+ size_t size)
+{
+ if (unlikely(last == from->tail)) {
+ qlist_move_all(from, to);
+ return;
+ }
+ if (empty_qlist(to))
+ to->head = from->head;
+ else
+ *to->tail = from->head;
+ to->tail = last;
+ from->head = *last;
+ *last = NULL;
+ from->bytes -= size;
+ to->bytes += size;
+}
+
+
+/* The object quarantine consists of per-cpu queues and a global queue,
+ * guarded by quarantine_lock.
+ */
+static DEFINE_PER_CPU(struct qlist, cpu_quarantine);
+
+static struct qlist global_quarantine;
+static DEFINE_SPINLOCK(quarantine_lock);
+
+/* Maximum size of the global queue. */
+static unsigned long quarantine_size;
+
+/* The fraction of physical memory the quarantine is allowed to occupy.
+ * Quarantine doesn't support memory shrinker with SLAB allocator, so we keep
+ * the ratio low to avoid OOM.
+ */
+#define QUARANTINE_FRACTION 32
+
+#define QUARANTINE_LOW_SIZE (smp_load_acquire(&quarantine_size) * 3 / 4)
+#define QUARANTINE_PERCPU_SIZE (1 << 20)
+
+static inline struct kmem_cache *qlink_to_cache(void **qlink)
+{
+ return virt_to_head_page(qlink)->slab_cache;
+}
+
+static inline void *qlink_to_object(void **qlink, struct kmem_cache *cache)
+{
+ struct kasan_free_meta *free_info =
+ container_of((void ***)qlink, struct kasan_free_meta,
+ quarantine_link);
+
+ return ((void *)free_info) - cache->kasan_info.free_meta_offset;
+}
+
+static inline void qlink_free(void **qlink, struct kmem_cache *cache)
+{
+ void *object = qlink_to_object(qlink, cache);
+ struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object);
+ unsigned long flags;
+
+ local_irq_save(flags);
+ alloc_info->state = KASAN_STATE_FREE;
+ nokasan_free(cache, object, _THIS_IP_);
+ local_irq_restore(flags);
+}
+
+static inline void qlist_free_all(struct qlist *q, struct kmem_cache *cache)
+{
+ void **qlink;
+
+ if (unlikely(empty_qlist(q)))
+ return;
+
+ qlink = q->head;
+ while (qlink) {
+ struct kmem_cache *obj_cache =
+ cache ? cache : qlink_to_cache(qlink);
+ void **next = *qlink;
+
+ qlink_free(qlink, obj_cache);
+ qlink = next;
+ }
+ init_qlist(q);
+}
+
+void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache)
+{
+ unsigned long flags;
+ struct qlist *q;
+ struct qlist temp = QLIST_INIT;
+
+ local_irq_save(flags);
+
+ q = this_cpu_ptr(&cpu_quarantine);
+ qlist_put(q, (void **) &info->quarantine_link, cache->size);
+ if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE))
+ qlist_move_all(q, &temp);
+
+ local_irq_restore(flags);
+
+ if (unlikely(!empty_qlist(&temp))) {
+ spin_lock_irqsave(&quarantine_lock, flags);
+ qlist_move_all(&temp, &global_quarantine);
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+ }
+}
+
+void quarantine_reduce(void)
+{
+ size_t new_quarantine_size;
+ unsigned long flags;
+ struct qlist to_free = QLIST_INIT;
+ size_t size_to_free = 0;
+ void **last;
+
+ if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
+ smp_load_acquire(&quarantine_size)))
+ return;
+
+ spin_lock_irqsave(&quarantine_lock, flags);
+
+ /* Update quarantine size in case of hotplug. Allocate a fraction of
+ * the installed memory to quarantine minus per-cpu queue limits.
+ */
+ new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
+ QUARANTINE_FRACTION;
+ new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
+ smp_store_release(&quarantine_size, new_quarantine_size);
+
+ last = global_quarantine.head;
+ while (last) {
+ struct kmem_cache *cache = qlink_to_cache(last);
+
+ size_to_free += cache->size;
+ if (!*last || size_to_free >
+ global_quarantine.bytes - QUARANTINE_LOW_SIZE)
+ break;
+ last = (void **) *last;
+ }
+ qlist_move(&global_quarantine, last, &to_free, size_to_free);
+
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+
+ qlist_free_all(&to_free, NULL);
+}
+
+static inline void qlist_move_cache(struct qlist *from,
+ struct qlist *to,
+ struct kmem_cache *cache)
+{
+ void ***prev;
+
+ if (unlikely(empty_qlist(from)))
+ return;
+
+ prev = &from->head;
+ while (*prev) {
+ void **qlink = *prev;
+ struct kmem_cache *obj_cache = qlink_to_cache(qlink);
+
+ if (obj_cache == cache) {
+ if (unlikely(from->tail == qlink))
+ from->tail = (void **) prev;
+ *prev = (void **) *qlink;
+ from->bytes -= cache->size;
+ qlist_put(to, qlink, cache->size);
+ } else
+ prev = (void ***) *prev;
+ }
+}
+
+static void per_cpu_remove_cache(void *arg)
+{
+ struct kmem_cache *cache = arg;
+ struct qlist to_free = QLIST_INIT;
+ struct qlist *q;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ q = this_cpu_ptr(&cpu_quarantine);
+ qlist_move_cache(q, &to_free, cache);
+ local_irq_restore(flags);
+
+ qlist_free_all(&to_free, cache);
+}
+
+void quarantine_remove_cache(struct kmem_cache *cache)
+{
+ unsigned long flags;
+ struct qlist to_free = QLIST_INIT;
+
+ on_each_cpu(per_cpu_remove_cache, cache, 0);
+
+ spin_lock_irqsave(&quarantine_lock, flags);
+ qlist_move_cache(&global_quarantine, &to_free, cache);
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+
+ qlist_free_all(&to_free, cache);
+}
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 6c4afcd..a4dca25 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -148,7 +148,8 @@ static void print_object(struct kmem_cache *cache, void *object)
print_track(&alloc_info->track);
break;
case KASAN_STATE_FREE:
- pr_err("Object freed, allocated with size %u bytes\n",
+ case KASAN_STATE_QUARANTINE:
+ pr_err("Object freed, allocated with size %lu bytes\n",
alloc_info->alloc_size);
free_info = get_free_info(cache, object);
pr_err("Allocation:\n");
diff --git a/mm/mempool.c b/mm/mempool.c
index b47c8a7..4beeeef 100644
--- a/mm/mempool.c
+++ b/mm/mempool.c
@@ -105,11 +105,12 @@ static inline void poison_element(mempool_t *pool, void *element)
static void kasan_poison_element(mempool_t *pool, void *element)
{
if (pool->alloc == mempool_alloc_slab)
- kasan_slab_free(pool->pool_data, element);
+ kasan_poison_slab_free(pool->pool_data, element);
if (pool->alloc == mempool_kmalloc)
- kasan_kfree(element);
+ kasan_poison_kfree(element);
if (pool->alloc == mempool_alloc_pages)
- kasan_free_pages(element, (unsigned long)pool->pool_data);
+ kasan_poison_free_pages(element,
+ (unsigned long)pool->pool_data);
}
static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 63358d9..4f65587 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -980,7 +980,7 @@ static bool free_pages_prepare(struct page *page, unsigned int order)
trace_mm_page_free(page, order);
kmemcheck_free_shadow(page, order);
- kasan_free_pages(page, order);
+ kasan_poison_free_pages(page, order);
if (PageAnon(page))
page->mapping = NULL;
diff --git a/mm/slab.c b/mm/slab.c
index 0ec7aa3..e2fac67 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -3374,9 +3374,19 @@ free_done:
static inline void __cache_free(struct kmem_cache *cachep, void *objp,
unsigned long caller)
{
+#ifdef CONFIG_KASAN
+ if (!kasan_slab_free(cachep, objp))
+ /* The object has been put into the quarantine, don't touch it
+ * for now.
+ */
+ nokasan_free(cachep, objp, caller);
+}
+
+void nokasan_free(struct kmem_cache *cachep, void *objp, unsigned long caller)
+{
+#endif
struct array_cache *ac;
- kasan_slab_free(cachep, objp);
ac = cpu_cache_get(cachep);
check_irq_off();
diff --git a/mm/slab.h b/mm/slab.h
index c63b869..3f19e3f 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -372,4 +372,8 @@ void *slab_next(struct seq_file *m, void *p, loff_t *pos);
void slab_stop(struct seq_file *m, void *p);
int memcg_slab_show(struct seq_file *m, void *p);
+#ifdef CONFIG_KASAN
+void nokasan_free(struct kmem_cache *cache, void *x, unsigned long addr);
+#endif
+
#endif /* MM_SLAB_H */
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 8478631..8f2edde 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -710,6 +710,7 @@ void kmem_cache_destroy(struct kmem_cache *s)
get_online_cpus();
get_online_mems();
+ kasan_cache_destroy(s);
mutex_lock(&slab_mutex);
s->refcount--;
@@ -748,6 +749,7 @@ int kmem_cache_shrink(struct kmem_cache *cachep)
get_online_cpus();
get_online_mems();
+ kasan_cache_shrink(cachep);
ret = __kmem_cache_shrink(cachep, false);
put_online_mems();
put_online_cpus();
diff --git a/mm/slub.c b/mm/slub.c
index 945bbee..6fe45de 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1278,7 +1278,7 @@ static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags)
static inline void kfree_hook(const void *x)
{
kmemleak_free(x);
- kasan_kfree_large(x);
+ kasan_poison_kfree_large(x);
}
static inline struct kmem_cache *slab_pre_alloc_hook(struct kmem_cache *s,
@@ -1333,7 +1333,7 @@ static inline void slab_free_hook(struct kmem_cache *s, void *x)
if (!(s->flags & SLAB_DEBUG_OBJECTS))
debug_check_no_obj_freed(x, s->object_size);
- kasan_slab_free(s, x);
+ kasan_poison_slab_free(s, x);
}
static inline void slab_free_freelist_hook(struct kmem_cache *s,
--
2.7.0.rc3.207.g0ac5344
^ permalink raw reply related [flat|nested] 78+ messages in thread
* [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-01-27 18:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-27 18:25 UTC (permalink / raw)
To: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt
Cc: kasan-dev, linux-kernel, linux-mm
Quarantine isolates freed objects in a separate queue. The objects are
returned to the allocator later, which helps to detect use-after-free
errors.
Freed objects are first added to per-cpu quarantine queues.
When a cache is destroyed or memory shrinking is requested, the objects
are moved into the global quarantine queue. Whenever a kmalloc call
allows memory reclaiming, the oldest objects are popped out of the
global queue until the total size of objects in quarantine is less than
3/4 of the maximum quarantine size (which is a fraction of installed
physical memory).
Right now quarantine support is only enabled in SLAB allocator.
Unification of KASAN features in SLAB and SLUB will be done later.
This patch is based on the "mm: kasan: quarantine" patch originally
prepared by Dmitry Chernenkov.
Signed-off-by: Alexander Potapenko <glider@google.com>
---
include/linux/kasan.h | 30 ++++--
lib/test_kasan.c | 29 ++++++
mm/kasan/Makefile | 2 +-
mm/kasan/kasan.c | 68 +++++++++++-
mm/kasan/kasan.h | 11 +-
mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
mm/kasan/report.c | 3 +-
mm/mempool.c | 7 +-
mm/page_alloc.c | 2 +-
mm/slab.c | 12 ++-
mm/slab.h | 4 +
mm/slab_common.c | 2 +
mm/slub.c | 4 +-
13 files changed, 435 insertions(+), 23 deletions(-)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index bf71ab0..355e722 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -44,24 +44,29 @@ static inline void kasan_disable_current(void)
void kasan_unpoison_shadow(const void *address, size_t size);
void kasan_alloc_pages(struct page *page, unsigned int order);
-void kasan_free_pages(struct page *page, unsigned int order);
+void kasan_poison_free_pages(struct page *page, unsigned int order);
void kasan_cache_create(struct kmem_cache *cache, size_t *size,
unsigned long *flags);
+void kasan_cache_shrink(struct kmem_cache *cache);
+void kasan_cache_destroy(struct kmem_cache *cache);
void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
void kasan_poison_object_data(struct kmem_cache *cache, void *object);
void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags);
-void kasan_kfree_large(const void *ptr);
-void kasan_kfree(void *ptr);
+void kasan_poison_kfree_large(const void *ptr);
+void kasan_poison_kfree(void *ptr);
void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size,
gfp_t flags);
void kasan_krealloc(const void *object, size_t new_size, gfp_t flags);
void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
-void kasan_slab_free(struct kmem_cache *s, void *object);
+/* kasan_slab_free() returns true if the object has been put into quarantine.
+ */
+bool kasan_slab_free(struct kmem_cache *s, void *object);
+void kasan_poison_slab_free(struct kmem_cache *s, void *object);
struct kasan_cache {
int alloc_meta_offset;
@@ -79,11 +84,14 @@ static inline void kasan_enable_current(void) {}
static inline void kasan_disable_current(void) {}
static inline void kasan_alloc_pages(struct page *page, unsigned int order) {}
-static inline void kasan_free_pages(struct page *page, unsigned int order) {}
+static inline void kasan_poison_free_pages(struct page *page,
+ unsigned int order) {}
static inline void kasan_cache_create(struct kmem_cache *cache,
size_t *size,
unsigned long *flags) {}
+static inline void kasan_cache_shrink(struct kmem_cache *cache) {}
+static inline void kasan_cache_destroy(struct kmem_cache *cache) {}
static inline void kasan_poison_slab(struct page *page) {}
static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
@@ -92,8 +100,8 @@ static inline void kasan_poison_object_data(struct kmem_cache *cache,
void *object) {}
static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {}
-static inline void kasan_kfree_large(const void *ptr) {}
-static inline void kasan_kfree(void *ptr) {}
+static inline void kasan_poison_kfree_large(const void *ptr) {}
+static inline void kasan_poison_kfree(void *ptr) {}
static inline void kasan_kmalloc(struct kmem_cache *s, const void *object,
size_t size, gfp_t flags) {}
static inline void kasan_krealloc(const void *object, size_t new_size,
@@ -101,7 +109,13 @@ static inline void kasan_krealloc(const void *object, size_t new_size,
static inline void kasan_slab_alloc(struct kmem_cache *s, void *object,
gfp_t flags) {}
-static inline void kasan_slab_free(struct kmem_cache *s, void *object) {}
+/* kasan_slab_free() returns true if the object has been put into quarantine.
+ */
+static inline bool kasan_slab_free(struct kmem_cache *s, void *object)
+{
+ return false;
+}
+static inline void kasan_poison_slab_free(struct kmem_cache *s, void *object) {}
static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
static inline void kasan_free_shadow(const struct vm_struct *vm) {}
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 822c804..6eb6d42 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -349,6 +349,32 @@ static noinline void __init kasan_stack_oob(void)
*(volatile char *)p;
}
+#ifdef CONFIG_SLAB
+static noinline void __init kasan_quarantine_cache(void)
+{
+ struct kmem_cache *cache = kmem_cache_create(
+ "test", 137, 8, GFP_KERNEL, NULL);
+ int i;
+
+ for (i = 0; i < 100; i++) {
+ void *p = kmem_cache_alloc(cache, GFP_KERNEL);
+
+ kmem_cache_free(cache, p);
+ p = kmalloc(sizeof(u64), GFP_KERNEL);
+ kfree(p);
+ }
+ kmem_cache_shrink(cache);
+ for (i = 0; i < 100; i++) {
+ u64 *p = kmem_cache_alloc(cache, GFP_KERNEL);
+
+ kmem_cache_free(cache, p);
+ p = kmalloc(sizeof(u64), GFP_KERNEL);
+ kfree(p);
+ }
+ kmem_cache_destroy(cache);
+}
+#endif
+
static int __init kmalloc_tests_init(void)
{
kmalloc_oob_right();
@@ -372,6 +398,9 @@ static int __init kmalloc_tests_init(void)
kmem_cache_oob();
kasan_stack_oob();
kasan_global_oob();
+#ifdef CONFIG_SLAB
+ kasan_quarantine_cache();
+#endif
return -EAGAIN;
}
diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index f952515..8e59350 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -7,5 +7,5 @@ CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
obj-y := kasan.o report.o kasan_init.o
ifdef CONFIG_SLAB
- obj-y += stackdepot.o
+ obj-y += stackdepot.o quarantine.o
endif
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index b5d04ec..43c079a 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -307,7 +307,7 @@ void kasan_alloc_pages(struct page *page, unsigned int order)
kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
}
-void kasan_free_pages(struct page *page, unsigned int order)
+void kasan_poison_free_pages(struct page *page, unsigned int order)
{
if (likely(!PageHighMem(page)))
kasan_poison_shadow(page_address(page),
@@ -368,6 +368,20 @@ void kasan_cache_create(struct kmem_cache *cache, size_t *size,
}
#endif
+void kasan_cache_shrink(struct kmem_cache *cache)
+{
+#ifdef CONFIG_SLAB
+ quarantine_remove_cache(cache);
+#endif
+}
+
+void kasan_cache_destroy(struct kmem_cache *cache)
+{
+#ifdef CONFIG_SLAB
+ quarantine_remove_cache(cache);
+#endif
+}
+
void kasan_poison_slab(struct page *page)
{
kasan_poison_shadow(page_address(page),
@@ -464,7 +478,7 @@ void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)
kasan_kmalloc(cache, object, cache->object_size, flags);
}
-void kasan_slab_free(struct kmem_cache *cache, void *object)
+void kasan_poison_slab_free(struct kmem_cache *cache, void *object)
{
unsigned long size = cache->object_size;
unsigned long rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
@@ -476,12 +490,53 @@ void kasan_slab_free(struct kmem_cache *cache, void *object)
kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
}
+bool kasan_slab_free(struct kmem_cache *cache, void *object)
+{
+#ifdef CONFIG_SLAB
+ /* RCU slabs could be legally used after free within the RCU period */
+ if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
+ return false;
+
+ if (likely(cache->flags & SLAB_KASAN)) {
+ struct kasan_alloc_meta *alloc_info =
+ get_alloc_info(cache, object);
+ struct kasan_free_meta *free_info =
+ get_free_info(cache, object);
+
+ switch (alloc_info->state) {
+ case KASAN_STATE_ALLOC:
+ alloc_info->state = KASAN_STATE_QUARANTINE;
+ quarantine_put(free_info, cache);
+ set_track(&free_info->track, GFP_NOWAIT);
+ kasan_poison_slab_free(cache, object);
+ return true;
+ case KASAN_STATE_QUARANTINE:
+ case KASAN_STATE_FREE:
+ pr_err("Double free");
+ dump_stack();
+ break;
+ default:
+ break;
+ }
+ }
+ return false;
+#else
+ kasan_poison_slab_free(cache, object);
+ return false;
+#endif
+}
+
void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
gfp_t flags)
{
unsigned long redzone_start;
unsigned long redzone_end;
+#ifdef CONFIG_SLAB
+ if (flags & __GFP_RECLAIM)
+ quarantine_reduce();
+#endif
+
if (unlikely(object == NULL))
return;
@@ -512,6 +567,11 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)
unsigned long redzone_start;
unsigned long redzone_end;
+#ifdef CONFIG_SLAB
+ if (flags & __GFP_RECLAIM)
+ quarantine_reduce();
+#endif
+
if (unlikely(ptr == NULL))
return;
@@ -540,7 +600,7 @@ void kasan_krealloc(const void *object, size_t size, gfp_t flags)
kasan_kmalloc(page->slab_cache, object, size, flags);
}
-void kasan_kfree(void *ptr)
+void kasan_poison_kfree(void *ptr)
{
struct page *page;
@@ -553,7 +613,7 @@ void kasan_kfree(void *ptr)
kasan_slab_free(page->slab_cache, ptr);
}
-void kasan_kfree_large(const void *ptr)
+void kasan_poison_kfree_large(const void *ptr)
{
struct page *page = virt_to_page(ptr);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index eb9de369..0fe58d9 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -61,6 +61,7 @@ struct kasan_global {
enum kasan_state {
KASAN_STATE_INIT,
KASAN_STATE_ALLOC,
+ KASAN_STATE_QUARANTINE,
KASAN_STATE_FREE
};
@@ -82,8 +83,10 @@ struct kasan_alloc_meta {
};
struct kasan_free_meta {
- /* Allocator freelist pointer, unused by KASAN. */
- void **freelist;
+ /* This field is used while the object is in the quarantine.
+ * Otherwise it might be used for the allocator freelist.
+ */
+ void **quarantine_link;
struct kasan_track track;
};
@@ -113,4 +116,8 @@ kasan_stack_handle kasan_save_stack(struct stack_trace *trace, gfp_t flags);
void kasan_fetch_stack(kasan_stack_handle handle, struct stack_trace *trace);
+void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache);
+void quarantine_reduce(void);
+void quarantine_remove_cache(struct kmem_cache *cache);
+
#endif
diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c
new file mode 100644
index 0000000..10c0ad6
--- /dev/null
+++ b/mm/kasan/quarantine.c
@@ -0,0 +1,284 @@
+/* Kasan quarantine */
+
+#include <linux/gfp.h>
+#include <linux/hash.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/percpu.h>
+#include <linux/printk.h>
+#include <linux/shrinker.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "../slab.h"
+#include "kasan.h"
+
+/* Data structure and operations for quarantine queues. */
+
+/* Each queue is a signled-linked list, which also stores the total size of
+ * objects inside of it.
+ */
+struct qlist {
+ void **head;
+ void **tail;
+ size_t bytes;
+};
+
+#define QLIST_INIT { NULL, NULL, 0 }
+
+static inline bool empty_qlist(struct qlist *q)
+{
+ return !q->head;
+}
+
+static inline void init_qlist(struct qlist *q)
+{
+ q->head = q->tail = NULL;
+ q->bytes = 0;
+}
+
+static inline void qlist_put(struct qlist *q, void **qlink, size_t size)
+{
+ if (unlikely(empty_qlist(q)))
+ q->head = qlink;
+ else
+ *q->tail = qlink;
+ q->tail = qlink;
+ *qlink = NULL;
+ q->bytes += size;
+}
+
+static inline void **qlist_remove(struct qlist *q, void ***prev,
+ size_t size)
+{
+ void **qlink = *prev;
+
+ *prev = *qlink;
+ if (q->tail == qlink) {
+ if (q->head == qlink)
+ q->tail = NULL;
+ else
+ q->tail = (void **)prev;
+ }
+ q->bytes -= size;
+
+ return qlink;
+}
+
+static inline void qlist_move_all(struct qlist *from, struct qlist *to)
+{
+ if (unlikely(empty_qlist(from)))
+ return;
+
+ if (empty_qlist(to)) {
+ *to = *from;
+ init_qlist(from);
+ return;
+ }
+
+ *to->tail = from->head;
+ to->tail = from->tail;
+ to->bytes += from->bytes;
+
+ init_qlist(from);
+}
+
+static inline void qlist_move(struct qlist *from, void **last, struct qlist *to,
+ size_t size)
+{
+ if (unlikely(last == from->tail)) {
+ qlist_move_all(from, to);
+ return;
+ }
+ if (empty_qlist(to))
+ to->head = from->head;
+ else
+ *to->tail = from->head;
+ to->tail = last;
+ from->head = *last;
+ *last = NULL;
+ from->bytes -= size;
+ to->bytes += size;
+}
+
+
+/* The object quarantine consists of per-cpu queues and a global queue,
+ * guarded by quarantine_lock.
+ */
+static DEFINE_PER_CPU(struct qlist, cpu_quarantine);
+
+static struct qlist global_quarantine;
+static DEFINE_SPINLOCK(quarantine_lock);
+
+/* Maximum size of the global queue. */
+static unsigned long quarantine_size;
+
+/* The fraction of physical memory the quarantine is allowed to occupy.
+ * Quarantine doesn't support memory shrinker with SLAB allocator, so we keep
+ * the ratio low to avoid OOM.
+ */
+#define QUARANTINE_FRACTION 32
+
+#define QUARANTINE_LOW_SIZE (smp_load_acquire(&quarantine_size) * 3 / 4)
+#define QUARANTINE_PERCPU_SIZE (1 << 20)
+
+static inline struct kmem_cache *qlink_to_cache(void **qlink)
+{
+ return virt_to_head_page(qlink)->slab_cache;
+}
+
+static inline void *qlink_to_object(void **qlink, struct kmem_cache *cache)
+{
+ struct kasan_free_meta *free_info =
+ container_of((void ***)qlink, struct kasan_free_meta,
+ quarantine_link);
+
+ return ((void *)free_info) - cache->kasan_info.free_meta_offset;
+}
+
+static inline void qlink_free(void **qlink, struct kmem_cache *cache)
+{
+ void *object = qlink_to_object(qlink, cache);
+ struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object);
+ unsigned long flags;
+
+ local_irq_save(flags);
+ alloc_info->state = KASAN_STATE_FREE;
+ nokasan_free(cache, object, _THIS_IP_);
+ local_irq_restore(flags);
+}
+
+static inline void qlist_free_all(struct qlist *q, struct kmem_cache *cache)
+{
+ void **qlink;
+
+ if (unlikely(empty_qlist(q)))
+ return;
+
+ qlink = q->head;
+ while (qlink) {
+ struct kmem_cache *obj_cache =
+ cache ? cache : qlink_to_cache(qlink);
+ void **next = *qlink;
+
+ qlink_free(qlink, obj_cache);
+ qlink = next;
+ }
+ init_qlist(q);
+}
+
+void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache)
+{
+ unsigned long flags;
+ struct qlist *q;
+ struct qlist temp = QLIST_INIT;
+
+ local_irq_save(flags);
+
+ q = this_cpu_ptr(&cpu_quarantine);
+ qlist_put(q, (void **) &info->quarantine_link, cache->size);
+ if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE))
+ qlist_move_all(q, &temp);
+
+ local_irq_restore(flags);
+
+ if (unlikely(!empty_qlist(&temp))) {
+ spin_lock_irqsave(&quarantine_lock, flags);
+ qlist_move_all(&temp, &global_quarantine);
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+ }
+}
+
+void quarantine_reduce(void)
+{
+ size_t new_quarantine_size;
+ unsigned long flags;
+ struct qlist to_free = QLIST_INIT;
+ size_t size_to_free = 0;
+ void **last;
+
+ if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
+ smp_load_acquire(&quarantine_size)))
+ return;
+
+ spin_lock_irqsave(&quarantine_lock, flags);
+
+ /* Update quarantine size in case of hotplug. Allocate a fraction of
+ * the installed memory to quarantine minus per-cpu queue limits.
+ */
+ new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
+ QUARANTINE_FRACTION;
+ new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
+ smp_store_release(&quarantine_size, new_quarantine_size);
+
+ last = global_quarantine.head;
+ while (last) {
+ struct kmem_cache *cache = qlink_to_cache(last);
+
+ size_to_free += cache->size;
+ if (!*last || size_to_free >
+ global_quarantine.bytes - QUARANTINE_LOW_SIZE)
+ break;
+ last = (void **) *last;
+ }
+ qlist_move(&global_quarantine, last, &to_free, size_to_free);
+
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+
+ qlist_free_all(&to_free, NULL);
+}
+
+static inline void qlist_move_cache(struct qlist *from,
+ struct qlist *to,
+ struct kmem_cache *cache)
+{
+ void ***prev;
+
+ if (unlikely(empty_qlist(from)))
+ return;
+
+ prev = &from->head;
+ while (*prev) {
+ void **qlink = *prev;
+ struct kmem_cache *obj_cache = qlink_to_cache(qlink);
+
+ if (obj_cache == cache) {
+ if (unlikely(from->tail == qlink))
+ from->tail = (void **) prev;
+ *prev = (void **) *qlink;
+ from->bytes -= cache->size;
+ qlist_put(to, qlink, cache->size);
+ } else
+ prev = (void ***) *prev;
+ }
+}
+
+static void per_cpu_remove_cache(void *arg)
+{
+ struct kmem_cache *cache = arg;
+ struct qlist to_free = QLIST_INIT;
+ struct qlist *q;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ q = this_cpu_ptr(&cpu_quarantine);
+ qlist_move_cache(q, &to_free, cache);
+ local_irq_restore(flags);
+
+ qlist_free_all(&to_free, cache);
+}
+
+void quarantine_remove_cache(struct kmem_cache *cache)
+{
+ unsigned long flags;
+ struct qlist to_free = QLIST_INIT;
+
+ on_each_cpu(per_cpu_remove_cache, cache, 0);
+
+ spin_lock_irqsave(&quarantine_lock, flags);
+ qlist_move_cache(&global_quarantine, &to_free, cache);
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+
+ qlist_free_all(&to_free, cache);
+}
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 6c4afcd..a4dca25 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -148,7 +148,8 @@ static void print_object(struct kmem_cache *cache, void *object)
print_track(&alloc_info->track);
break;
case KASAN_STATE_FREE:
- pr_err("Object freed, allocated with size %u bytes\n",
+ case KASAN_STATE_QUARANTINE:
+ pr_err("Object freed, allocated with size %lu bytes\n",
alloc_info->alloc_size);
free_info = get_free_info(cache, object);
pr_err("Allocation:\n");
diff --git a/mm/mempool.c b/mm/mempool.c
index b47c8a7..4beeeef 100644
--- a/mm/mempool.c
+++ b/mm/mempool.c
@@ -105,11 +105,12 @@ static inline void poison_element(mempool_t *pool, void *element)
static void kasan_poison_element(mempool_t *pool, void *element)
{
if (pool->alloc == mempool_alloc_slab)
- kasan_slab_free(pool->pool_data, element);
+ kasan_poison_slab_free(pool->pool_data, element);
if (pool->alloc == mempool_kmalloc)
- kasan_kfree(element);
+ kasan_poison_kfree(element);
if (pool->alloc == mempool_alloc_pages)
- kasan_free_pages(element, (unsigned long)pool->pool_data);
+ kasan_poison_free_pages(element,
+ (unsigned long)pool->pool_data);
}
static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 63358d9..4f65587 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -980,7 +980,7 @@ static bool free_pages_prepare(struct page *page, unsigned int order)
trace_mm_page_free(page, order);
kmemcheck_free_shadow(page, order);
- kasan_free_pages(page, order);
+ kasan_poison_free_pages(page, order);
if (PageAnon(page))
page->mapping = NULL;
diff --git a/mm/slab.c b/mm/slab.c
index 0ec7aa3..e2fac67 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -3374,9 +3374,19 @@ free_done:
static inline void __cache_free(struct kmem_cache *cachep, void *objp,
unsigned long caller)
{
+#ifdef CONFIG_KASAN
+ if (!kasan_slab_free(cachep, objp))
+ /* The object has been put into the quarantine, don't touch it
+ * for now.
+ */
+ nokasan_free(cachep, objp, caller);
+}
+
+void nokasan_free(struct kmem_cache *cachep, void *objp, unsigned long caller)
+{
+#endif
struct array_cache *ac;
- kasan_slab_free(cachep, objp);
ac = cpu_cache_get(cachep);
check_irq_off();
diff --git a/mm/slab.h b/mm/slab.h
index c63b869..3f19e3f 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -372,4 +372,8 @@ void *slab_next(struct seq_file *m, void *p, loff_t *pos);
void slab_stop(struct seq_file *m, void *p);
int memcg_slab_show(struct seq_file *m, void *p);
+#ifdef CONFIG_KASAN
+void nokasan_free(struct kmem_cache *cache, void *x, unsigned long addr);
+#endif
+
#endif /* MM_SLAB_H */
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 8478631..8f2edde 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -710,6 +710,7 @@ void kmem_cache_destroy(struct kmem_cache *s)
get_online_cpus();
get_online_mems();
+ kasan_cache_destroy(s);
mutex_lock(&slab_mutex);
s->refcount--;
@@ -748,6 +749,7 @@ int kmem_cache_shrink(struct kmem_cache *cachep)
get_online_cpus();
get_online_mems();
+ kasan_cache_shrink(cachep);
ret = __kmem_cache_shrink(cachep, false);
put_online_mems();
put_online_cpus();
diff --git a/mm/slub.c b/mm/slub.c
index 945bbee..6fe45de 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1278,7 +1278,7 @@ static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags)
static inline void kfree_hook(const void *x)
{
kmemleak_free(x);
- kasan_kfree_large(x);
+ kasan_poison_kfree_large(x);
}
static inline struct kmem_cache *slab_pre_alloc_hook(struct kmem_cache *s,
@@ -1333,7 +1333,7 @@ static inline void slab_free_hook(struct kmem_cache *s, void *x)
if (!(s->flags & SLAB_DEBUG_OBJECTS))
debug_check_no_obj_freed(x, s->object_size);
- kasan_slab_free(s, x);
+ kasan_poison_slab_free(s, x);
}
static inline void slab_free_freelist_hook(struct kmem_cache *s,
--
2.7.0.rc3.207.g0ac5344
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-28 7:40 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-01-28 7:40 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
Hello,
On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
> Stack depot will allow KASAN store allocation/deallocation stack traces
> for memory chunks. The stack traces are stored in a hash table and
> referenced by handles which reside in the kasan_alloc_meta and
> kasan_free_meta structures in the allocated memory chunks.
Looks really nice!
Could it be more generalized to be used by other feature that need to
store stack trace such as tracepoint or page owner?
If it could be, there is one more requirement.
I understand the fact that entry is never removed from depot makes things
very simpler, but, for general usecases, it's better to use reference count
and allow to remove. Is it possible?
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-01-28 7:40 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-01-28 7:40 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
Hello,
On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
> Stack depot will allow KASAN store allocation/deallocation stack traces
> for memory chunks. The stack traces are stored in a hash table and
> referenced by handles which reside in the kasan_alloc_meta and
> kasan_free_meta structures in the allocated memory chunks.
Looks really nice!
Could it be more generalized to be used by other feature that need to
store stack trace such as tracepoint or page owner?
If it could be, there is one more requirement.
I understand the fact that entry is never removed from depot makes things
very simpler, but, for general usecases, it's better to use reference count
and allow to remove. Is it possible?
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-28 7:44 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-01-28 7:44 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
> This patch adds KASAN hooks to SLAB allocator.
>
> This patch is based on the "mm: kasan: unified support for SLUB and
> SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> ---
> Documentation/kasan.txt | 5 ++-
...
> +#ifdef CONFIG_SLAB
> +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
> + const void *object)
> +{
> + return (void *)object + cache->kasan_info.alloc_meta_offset;
> +}
> +
> +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
> + const void *object)
> +{
> + return (void *)object + cache->kasan_info.free_meta_offset;
> +}
> +#endif
I cannot find the place to store stack info for free. get_free_info()
isn't used except print_object(). Plese let me know where.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-01-28 7:44 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-01-28 7:44 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
> This patch adds KASAN hooks to SLAB allocator.
>
> This patch is based on the "mm: kasan: unified support for SLUB and
> SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> ---
> Documentation/kasan.txt | 5 ++-
...
> +#ifdef CONFIG_SLAB
> +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
> + const void *object)
> +{
> + return (void *)object + cache->kasan_info.alloc_meta_offset;
> +}
> +
> +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
> + const void *object)
> +{
> + return (void *)object + cache->kasan_info.free_meta_offset;
> +}
> +#endif
I cannot find the place to store stack info for free. get_free_info()
isn't used except print_object(). Plese let me know where.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-01-28 7:44 ` Joonsoo Kim
(?)
@ 2016-01-28 12:37 ` Alexander Potapenko
2016-01-28 13:29 ` Alexander Potapenko
-1 siblings, 1 reply; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-28 12:37 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, dvyukov,
ryabinin.a.a, linux-mm, adech.fo, akpm, rostedt
[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]
On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>
> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
> > This patch adds KASAN hooks to SLAB allocator.
> >
> > This patch is based on the "mm: kasan: unified support for SLUB and
> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
> >
> > Signed-off-by: Alexander Potapenko <glider@google.com>
> > ---
> > Documentation/kasan.txt | 5 ++-
>
> ...
>
> > +#ifdef CONFIG_SLAB
> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
> > + const void *object)
> > +{
> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
> > +}
> > +
> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
> > + const void *object)
> > +{
> > + return (void *)object + cache->kasan_info.free_meta_offset;
> > +}
> > +#endif
>
> I cannot find the place to store stack info for free. get_free_info()
> isn't used except print_object(). Plese let me know where.
This is covered by other patches in this patchset.
> Thanks.
[-- Attachment #2: Type: text/html, Size: 1664 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-01-28 7:40 ` Joonsoo Kim
(?)
@ 2016-01-28 12:51 ` Alexander Potapenko
2016-01-28 13:27 ` Alexander Potapenko
-1 siblings, 1 reply; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-28 12:51 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, dvyukov,
ryabinin.a.a, linux-mm, adech.fo, akpm, rostedt
[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]
On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>
> Hello,
>
> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
> > Stack depot will allow KASAN store allocation/deallocation stack traces
> > for memory chunks. The stack traces are stored in a hash table and
> > referenced by handles which reside in the kasan_alloc_meta and
> > kasan_free_meta structures in the allocated memory chunks.
>
> Looks really nice!
>
> Could it be more generalized to be used by other feature that need to
> store stack trace such as tracepoint or page owner?
Certainly yes, but see below.
> If it could be, there is one more requirement.
> I understand the fact that entry is never removed from depot makes things
> very simpler, but, for general usecases, it's better to use reference
count
> and allow to remove. Is it possible?
For our use case reference counting is not really necessary, and it would
introduce unwanted contention.
There are two possible options, each having its advantages and drawbacks:
we can let the clients store the refcounters directly in their stacks (more
universal, but harder to use for the clients), or keep the counters in the
depot but add an API that does not change them (easier for the clients, but
potentially error-prone).
I'd say it's better to actually find at least one more user for the stack
depot in order to understand the requirements, and refactor the code after
that.
> Thanks.
>
[-- Attachment #2: Type: text/html, Size: 1766 bytes --]
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-01-28 12:51 ` Alexander Potapenko
@ 2016-01-28 13:27 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-28 13:27 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>
> On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>
>> Hello,
>>
>> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>> > Stack depot will allow KASAN store allocation/deallocation stack traces
>> > for memory chunks. The stack traces are stored in a hash table and
>> > referenced by handles which reside in the kasan_alloc_meta and
>> > kasan_free_meta structures in the allocated memory chunks.
>>
>> Looks really nice!
>>
>> Could it be more generalized to be used by other feature that need to
>> store stack trace such as tracepoint or page owner?
> Certainly yes, but see below.
>
>> If it could be, there is one more requirement.
>> I understand the fact that entry is never removed from depot makes things
>> very simpler, but, for general usecases, it's better to use reference
>> count
>> and allow to remove. Is it possible?
> For our use case reference counting is not really necessary, and it would
> introduce unwanted contention.
> There are two possible options, each having its advantages and drawbacks: we
> can let the clients store the refcounters directly in their stacks (more
> universal, but harder to use for the clients), or keep the counters in the
> depot but add an API that does not change them (easier for the clients, but
> potentially error-prone).
>
> I'd say it's better to actually find at least one more user for the stack
> depot in order to understand the requirements, and refactor the code after
> that.
>> Thanks.
>>
(resending to linux-kernel@ because the previous mail bounced)
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-01-28 13:27 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-28 13:27 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>
> On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>
>> Hello,
>>
>> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>> > Stack depot will allow KASAN store allocation/deallocation stack traces
>> > for memory chunks. The stack traces are stored in a hash table and
>> > referenced by handles which reside in the kasan_alloc_meta and
>> > kasan_free_meta structures in the allocated memory chunks.
>>
>> Looks really nice!
>>
>> Could it be more generalized to be used by other feature that need to
>> store stack trace such as tracepoint or page owner?
> Certainly yes, but see below.
>
>> If it could be, there is one more requirement.
>> I understand the fact that entry is never removed from depot makes things
>> very simpler, but, for general usecases, it's better to use reference
>> count
>> and allow to remove. Is it possible?
> For our use case reference counting is not really necessary, and it would
> introduce unwanted contention.
> There are two possible options, each having its advantages and drawbacks: we
> can let the clients store the refcounters directly in their stacks (more
> universal, but harder to use for the clients), or keep the counters in the
> depot but add an API that does not change them (easier for the clients, but
> potentially error-prone).
>
> I'd say it's better to actually find at least one more user for the stack
> depot in order to understand the requirements, and refactor the code after
> that.
>> Thanks.
>>
(resending to linux-kernel@ because the previous mail bounced)
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-01-28 12:37 ` Alexander Potapenko
@ 2016-01-28 13:29 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-28 13:29 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>
> On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>
>> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>> > This patch adds KASAN hooks to SLAB allocator.
>> >
>> > This patch is based on the "mm: kasan: unified support for SLUB and
>> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>> >
>> > Signed-off-by: Alexander Potapenko <glider@google.com>
>> > ---
>> > Documentation/kasan.txt | 5 ++-
>>
>> ...
>>
>> > +#ifdef CONFIG_SLAB
>> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>> > + const void *object)
>> > +{
>> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>> > +}
>> > +
>> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>> > + const void *object)
>> > +{
>> > + return (void *)object + cache->kasan_info.free_meta_offset;
>> > +}
>> > +#endif
>>
>> I cannot find the place to store stack info for free. get_free_info()
>> isn't used except print_object(). Plese let me know where.
>
> This is covered by other patches in this patchset.
>
>> Thanks.
(resending to linux-kernel@ because the previous mail bounced)
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-01-28 13:29 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-28 13:29 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>
> On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>
>> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>> > This patch adds KASAN hooks to SLAB allocator.
>> >
>> > This patch is based on the "mm: kasan: unified support for SLUB and
>> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>> >
>> > Signed-off-by: Alexander Potapenko <glider@google.com>
>> > ---
>> > Documentation/kasan.txt | 5 ++-
>>
>> ...
>>
>> > +#ifdef CONFIG_SLAB
>> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>> > + const void *object)
>> > +{
>> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>> > +}
>> > +
>> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>> > + const void *object)
>> > +{
>> > + return (void *)object + cache->kasan_info.free_meta_offset;
>> > +}
>> > +#endif
>>
>> I cannot find the place to store stack info for free. get_free_info()
>> isn't used except print_object(). Plese let me know where.
>
> This is covered by other patches in this patchset.
>
>> Thanks.
(resending to linux-kernel@ because the previous mail bounced)
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-01-28 14:53 ` Steven Rostedt
-1 siblings, 0 replies; 78+ messages in thread
From: Steven Rostedt @ 2016-01-28 14:53 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, kasan-dev,
linux-kernel, linux-mm
On Wed, 27 Jan 2016 19:25:09 +0100
Alexander Potapenko <glider@google.com> wrote:
> --- a/include/linux/ftrace.h
> +++ b/include/linux/ftrace.h
> @@ -762,6 +762,26 @@ struct ftrace_graph_ret {
> typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
> typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
>
> +#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
> +/*
> + * We want to know which function is an entrypoint of a hardirq.
> + */
> +#define __irq_entry __attribute__((__section__(".irqentry.text")))
> +#define __softirq_entry \
> + __attribute__((__section__(".softirqentry.text")))
> +
> +/* Limits of hardirq entrypoints */
> +extern char __irqentry_text_start[];
> +extern char __irqentry_text_end[];
> +/* Limits of softirq entrypoints */
> +extern char __softirqentry_text_start[];
> +extern char __softirqentry_text_end[];
> +
> +#else
> +#define __irq_entry
> +#define __softirq_entry
> +#endif
> +
> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>
> /* for init task */
Since this is no longer just used for function tracing, perhaps the
code should be moved to include/linux/irq.h or something.
-- Steve
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
@ 2016-01-28 14:53 ` Steven Rostedt
0 siblings, 0 replies; 78+ messages in thread
From: Steven Rostedt @ 2016-01-28 14:53 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, kasan-dev,
linux-kernel, linux-mm
On Wed, 27 Jan 2016 19:25:09 +0100
Alexander Potapenko <glider@google.com> wrote:
> --- a/include/linux/ftrace.h
> +++ b/include/linux/ftrace.h
> @@ -762,6 +762,26 @@ struct ftrace_graph_ret {
> typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
> typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
>
> +#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
> +/*
> + * We want to know which function is an entrypoint of a hardirq.
> + */
> +#define __irq_entry __attribute__((__section__(".irqentry.text")))
> +#define __softirq_entry \
> + __attribute__((__section__(".softirqentry.text")))
> +
> +/* Limits of hardirq entrypoints */
> +extern char __irqentry_text_start[];
> +extern char __irqentry_text_end[];
> +/* Limits of softirq entrypoints */
> +extern char __softirqentry_text_start[];
> +extern char __softirqentry_text_end[];
> +
> +#else
> +#define __irq_entry
> +#define __softirq_entry
> +#endif
> +
> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>
> /* for init task */
Since this is no longer just used for function tracing, perhaps the
code should be moved to include/linux/irq.h or something.
-- Steve
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
2016-01-28 14:53 ` Steven Rostedt
@ 2016-01-29 11:33 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-29 11:33 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
Agreed. Once I receive more comments I will make a new patch set and
include this change as well.
On Thu, Jan 28, 2016 at 3:53 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Wed, 27 Jan 2016 19:25:09 +0100
> Alexander Potapenko <glider@google.com> wrote:
>
>> --- a/include/linux/ftrace.h
>> +++ b/include/linux/ftrace.h
>> @@ -762,6 +762,26 @@ struct ftrace_graph_ret {
>> typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
>> typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
>>
>> +#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
>> +/*
>> + * We want to know which function is an entrypoint of a hardirq.
>> + */
>> +#define __irq_entry __attribute__((__section__(".irqentry.text")))
>> +#define __softirq_entry \
>> + __attribute__((__section__(".softirqentry.text")))
>> +
>> +/* Limits of hardirq entrypoints */
>> +extern char __irqentry_text_start[];
>> +extern char __irqentry_text_end[];
>> +/* Limits of softirq entrypoints */
>> +extern char __softirqentry_text_start[];
>> +extern char __softirqentry_text_end[];
>> +
>> +#else
>> +#define __irq_entry
>> +#define __softirq_entry
>> +#endif
>> +
>> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>
>> /* for init task */
>
> Since this is no longer just used for function tracing, perhaps the
> code should be moved to include/linux/irq.h or something.
>
> -- Steve
>
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
@ 2016-01-29 11:33 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-29 11:33 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
Agreed. Once I receive more comments I will make a new patch set and
include this change as well.
On Thu, Jan 28, 2016 at 3:53 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Wed, 27 Jan 2016 19:25:09 +0100
> Alexander Potapenko <glider@google.com> wrote:
>
>> --- a/include/linux/ftrace.h
>> +++ b/include/linux/ftrace.h
>> @@ -762,6 +762,26 @@ struct ftrace_graph_ret {
>> typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
>> typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
>>
>> +#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
>> +/*
>> + * We want to know which function is an entrypoint of a hardirq.
>> + */
>> +#define __irq_entry __attribute__((__section__(".irqentry.text")))
>> +#define __softirq_entry \
>> + __attribute__((__section__(".softirqentry.text")))
>> +
>> +/* Limits of hardirq entrypoints */
>> +extern char __irqentry_text_start[];
>> +extern char __irqentry_text_end[];
>> +/* Limits of softirq entrypoints */
>> +extern char __softirqentry_text_start[];
>> +extern char __softirqentry_text_end[];
>> +
>> +#else
>> +#define __irq_entry
>> +#define __softirq_entry
>> +#endif
>> +
>> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>
>> /* for init task */
>
> Since this is no longer just used for function tracing, perhaps the
> code should be moved to include/linux/irq.h or something.
>
> -- Steve
>
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
2016-01-29 11:33 ` Alexander Potapenko
@ 2016-01-29 11:59 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-29 11:59 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
On the other hand, this will require including <linux/irq.h> into
various files that currently use __irq_section.
But that header has a comment saying:
/*
* Please do not include this file in generic code. There is currently
* no requirement for any architecture to implement anything held
* within this file.
*
* Thanks. --rmk
*/
Do we really want to put anything into that header?
On Fri, Jan 29, 2016 at 12:33 PM, Alexander Potapenko <glider@google.com> wrote:
> Agreed. Once I receive more comments I will make a new patch set and
> include this change as well.
>
> On Thu, Jan 28, 2016 at 3:53 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>> On Wed, 27 Jan 2016 19:25:09 +0100
>> Alexander Potapenko <glider@google.com> wrote:
>>
>>> --- a/include/linux/ftrace.h
>>> +++ b/include/linux/ftrace.h
>>> @@ -762,6 +762,26 @@ struct ftrace_graph_ret {
>>> typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
>>> typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
>>>
>>> +#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
>>> +/*
>>> + * We want to know which function is an entrypoint of a hardirq.
>>> + */
>>> +#define __irq_entry __attribute__((__section__(".irqentry.text")))
>>> +#define __softirq_entry \
>>> + __attribute__((__section__(".softirqentry.text")))
>>> +
>>> +/* Limits of hardirq entrypoints */
>>> +extern char __irqentry_text_start[];
>>> +extern char __irqentry_text_end[];
>>> +/* Limits of softirq entrypoints */
>>> +extern char __softirqentry_text_start[];
>>> +extern char __softirqentry_text_end[];
>>> +
>>> +#else
>>> +#define __irq_entry
>>> +#define __softirq_entry
>>> +#endif
>>> +
>>> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>>
>>> /* for init task */
>>
>> Since this is no longer just used for function tracing, perhaps the
>> code should be moved to include/linux/irq.h or something.
>>
>> -- Steve
>>
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
> Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
> leiten Sie diese bitte nicht weiter, informieren Sie den
> Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
> This e-mail is confidential. If you are not the right addressee please
> do not forward it, please inform the sender, and please erase this
> e-mail including any attachments. Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
@ 2016-01-29 11:59 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-01-29 11:59 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
On the other hand, this will require including <linux/irq.h> into
various files that currently use __irq_section.
But that header has a comment saying:
/*
* Please do not include this file in generic code. There is currently
* no requirement for any architecture to implement anything held
* within this file.
*
* Thanks. --rmk
*/
Do we really want to put anything into that header?
On Fri, Jan 29, 2016 at 12:33 PM, Alexander Potapenko <glider@google.com> wrote:
> Agreed. Once I receive more comments I will make a new patch set and
> include this change as well.
>
> On Thu, Jan 28, 2016 at 3:53 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>> On Wed, 27 Jan 2016 19:25:09 +0100
>> Alexander Potapenko <glider@google.com> wrote:
>>
>>> --- a/include/linux/ftrace.h
>>> +++ b/include/linux/ftrace.h
>>> @@ -762,6 +762,26 @@ struct ftrace_graph_ret {
>>> typedef void (*trace_func_graph_ret_t)(struct ftrace_graph_ret *); /* return */
>>> typedef int (*trace_func_graph_ent_t)(struct ftrace_graph_ent *); /* entry */
>>>
>>> +#if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
>>> +/*
>>> + * We want to know which function is an entrypoint of a hardirq.
>>> + */
>>> +#define __irq_entry __attribute__((__section__(".irqentry.text")))
>>> +#define __softirq_entry \
>>> + __attribute__((__section__(".softirqentry.text")))
>>> +
>>> +/* Limits of hardirq entrypoints */
>>> +extern char __irqentry_text_start[];
>>> +extern char __irqentry_text_end[];
>>> +/* Limits of softirq entrypoints */
>>> +extern char __softirqentry_text_start[];
>>> +extern char __softirqentry_text_end[];
>>> +
>>> +#else
>>> +#define __irq_entry
>>> +#define __softirq_entry
>>> +#endif
>>> +
>>> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>>
>>> /* for init task */
>>
>> Since this is no longer just used for function tracing, perhaps the
>> code should be moved to include/linux/irq.h or something.
>>
>> -- Steve
>>
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
> Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
> leiten Sie diese bitte nicht weiter, informieren Sie den
> Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
> This e-mail is confidential. If you are not the right addressee please
> do not forward it, please inform the sender, and please erase this
> e-mail including any attachments. Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
2016-01-29 11:59 ` Alexander Potapenko
@ 2016-01-29 14:45 ` Steven Rostedt
-1 siblings, 0 replies; 78+ messages in thread
From: Steven Rostedt @ 2016-01-29 14:45 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
On Fri, 29 Jan 2016 12:59:13 +0100
Alexander Potapenko <glider@google.com> wrote:
> On the other hand, this will require including <linux/irq.h> into
> various files that currently use __irq_section.
> But that header has a comment saying:
>
> /*
> * Please do not include this file in generic code. There is currently
> * no requirement for any architecture to implement anything held
> * within this file.
> *
> * Thanks. --rmk
> */
>
> Do we really want to put anything into that header?
>
What about interrupt.h?
It's just weird to have KSAN needing to pull in ftrace.h for irq work.
-- Steve
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
@ 2016-01-29 14:45 ` Steven Rostedt
0 siblings, 0 replies; 78+ messages in thread
From: Steven Rostedt @ 2016-01-29 14:45 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
On Fri, 29 Jan 2016 12:59:13 +0100
Alexander Potapenko <glider@google.com> wrote:
> On the other hand, this will require including <linux/irq.h> into
> various files that currently use __irq_section.
> But that header has a comment saying:
>
> /*
> * Please do not include this file in generic code. There is currently
> * no requirement for any architecture to implement anything held
> * within this file.
> *
> * Thanks. --rmk
> */
>
> Do we really want to put anything into that header?
>
What about interrupt.h?
It's just weird to have KSAN needing to pull in ftrace.h for irq work.
-- Steve
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-01-28 13:29 ` Alexander Potapenko
@ 2016-02-01 2:15 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-01 2:15 UTC (permalink / raw)
To: Alexander Potapenko
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
> >
> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
> >>
> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
> >> > This patch adds KASAN hooks to SLAB allocator.
> >> >
> >> > This patch is based on the "mm: kasan: unified support for SLUB and
> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
> >> >
> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
> >> > ---
> >> > Documentation/kasan.txt | 5 ++-
> >>
> >> ...
> >>
> >> > +#ifdef CONFIG_SLAB
> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
> >> > + const void *object)
> >> > +{
> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
> >> > +}
> >> > +
> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
> >> > + const void *object)
> >> > +{
> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
> >> > +}
> >> > +#endif
> >>
> >> I cannot find the place to store stack info for free. get_free_info()
> >> isn't used except print_object(). Plese let me know where.
> >
> > This is covered by other patches in this patchset.
This should be covered by this patch. Stroing and printing free_info
is already done on SLUB and it is meaningful without quarantain.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-02-01 2:15 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-01 2:15 UTC (permalink / raw)
To: Alexander Potapenko
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
> >
> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
> >>
> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
> >> > This patch adds KASAN hooks to SLAB allocator.
> >> >
> >> > This patch is based on the "mm: kasan: unified support for SLUB and
> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
> >> >
> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
> >> > ---
> >> > Documentation/kasan.txt | 5 ++-
> >>
> >> ...
> >>
> >> > +#ifdef CONFIG_SLAB
> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
> >> > + const void *object)
> >> > +{
> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
> >> > +}
> >> > +
> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
> >> > + const void *object)
> >> > +{
> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
> >> > +}
> >> > +#endif
> >>
> >> I cannot find the place to store stack info for free. get_free_info()
> >> isn't used except print_object(). Plese let me know where.
> >
> > This is covered by other patches in this patchset.
This should be covered by this patch. Stroing and printing free_info
is already done on SLUB and it is meaningful without quarantain.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-02-01 2:47 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-01 2:47 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
> Quarantine isolates freed objects in a separate queue. The objects are
> returned to the allocator later, which helps to detect use-after-free
> errors.
>
> Freed objects are first added to per-cpu quarantine queues.
> When a cache is destroyed or memory shrinking is requested, the objects
> are moved into the global quarantine queue. Whenever a kmalloc call
> allows memory reclaiming, the oldest objects are popped out of the
> global queue until the total size of objects in quarantine is less than
> 3/4 of the maximum quarantine size (which is a fraction of installed
> physical memory).
Just wondering why not using time based approach rather than size
based one. In heavy load condition, how much time do the object stay in
quarantine?
>
> Right now quarantine support is only enabled in SLAB allocator.
> Unification of KASAN features in SLAB and SLUB will be done later.
>
> This patch is based on the "mm: kasan: quarantine" patch originally
> prepared by Dmitry Chernenkov.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> ---
> include/linux/kasan.h | 30 ++++--
> lib/test_kasan.c | 29 ++++++
> mm/kasan/Makefile | 2 +-
> mm/kasan/kasan.c | 68 +++++++++++-
> mm/kasan/kasan.h | 11 +-
> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
> mm/kasan/report.c | 3 +-
> mm/mempool.c | 7 +-
> mm/page_alloc.c | 2 +-
> mm/slab.c | 12 ++-
> mm/slab.h | 4 +
> mm/slab_common.c | 2 +
> mm/slub.c | 4 +-
> 13 files changed, 435 insertions(+), 23 deletions(-)
>
...
> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
> +{
> +#ifdef CONFIG_SLAB
> + /* RCU slabs could be legally used after free within the RCU period */
> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
> + return false;
> +
> + if (likely(cache->flags & SLAB_KASAN)) {
> + struct kasan_alloc_meta *alloc_info =
> + get_alloc_info(cache, object);
> + struct kasan_free_meta *free_info =
> + get_free_info(cache, object);
> +
> + switch (alloc_info->state) {
> + case KASAN_STATE_ALLOC:
> + alloc_info->state = KASAN_STATE_QUARANTINE;
> + quarantine_put(free_info, cache);
quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
although it's not much meaningful without poisoning. But, I have an
idea to poison object on SLAB_DESTROY_BY_RCU cache.
quarantine_put() moves per cpu list to global queue when
list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
at that time, after then, we can poison objects. With appropriate size
setup, it would not be intrusive.
> + set_track(&free_info->track, GFP_NOWAIT);
set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
> + kasan_poison_slab_free(cache, object);
> + return true;
> + case KASAN_STATE_QUARANTINE:
> + case KASAN_STATE_FREE:
> + pr_err("Double free");
> + dump_stack();
> + break;
> + default:
> + break;
> + }
> + }
> + return false;
> +#else
> + kasan_poison_slab_free(cache, object);
> + return false;
> +#endif
> +}
> +
...
> +void quarantine_reduce(void)
> +{
> + size_t new_quarantine_size;
> + unsigned long flags;
> + struct qlist to_free = QLIST_INIT;
> + size_t size_to_free = 0;
> + void **last;
> +
> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
> + smp_load_acquire(&quarantine_size)))
> + return;
> +
> + spin_lock_irqsave(&quarantine_lock, flags);
> +
> + /* Update quarantine size in case of hotplug. Allocate a fraction of
> + * the installed memory to quarantine minus per-cpu queue limits.
> + */
> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
> + QUARANTINE_FRACTION;
> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
> + smp_store_release(&quarantine_size, new_quarantine_size);
> +
> + last = global_quarantine.head;
> + while (last) {
> + struct kmem_cache *cache = qlink_to_cache(last);
> +
> + size_to_free += cache->size;
> + if (!*last || size_to_free >
> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
> + break;
> + last = (void **) *last;
> + }
> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
> +
> + spin_unlock_irqrestore(&quarantine_lock, flags);
> +
> + qlist_free_all(&to_free, NULL);
> +}
Isn't it better to call quarantine_reduce() in shrink_slab()?
It will help to maximize quarantine time.
> +
> +static inline void qlist_move_cache(struct qlist *from,
> + struct qlist *to,
> + struct kmem_cache *cache)
> +{
> + void ***prev;
> +
> + if (unlikely(empty_qlist(from)))
> + return;
> +
> + prev = &from->head;
> + while (*prev) {
> + void **qlink = *prev;
> + struct kmem_cache *obj_cache = qlink_to_cache(qlink);
> +
> + if (obj_cache == cache) {
> + if (unlikely(from->tail == qlink))
> + from->tail = (void **) prev;
> + *prev = (void **) *qlink;
> + from->bytes -= cache->size;
> + qlist_put(to, qlink, cache->size);
> + } else
> + prev = (void ***) *prev;
> + }
> +}
> +
> +static void per_cpu_remove_cache(void *arg)
> +{
> + struct kmem_cache *cache = arg;
> + struct qlist to_free = QLIST_INIT;
> + struct qlist *q;
> + unsigned long flags;
> +
> + local_irq_save(flags);
> + q = this_cpu_ptr(&cpu_quarantine);
> + qlist_move_cache(q, &to_free, cache);
> + local_irq_restore(flags);
> +
> + qlist_free_all(&to_free, cache);
> +}
> +
> +void quarantine_remove_cache(struct kmem_cache *cache)
> +{
> + unsigned long flags;
> + struct qlist to_free = QLIST_INIT;
> +
> + on_each_cpu(per_cpu_remove_cache, cache, 0);
Should be called with wait = 1.
> +
> + spin_lock_irqsave(&quarantine_lock, flags);
> + qlist_move_cache(&global_quarantine, &to_free, cache);
> + spin_unlock_irqrestore(&quarantine_lock, flags);
> +
> + qlist_free_all(&to_free, cache);
> +}
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 6c4afcd..a4dca25 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -148,7 +148,8 @@ static void print_object(struct kmem_cache *cache, void *object)
> print_track(&alloc_info->track);
> break;
> case KASAN_STATE_FREE:
> - pr_err("Object freed, allocated with size %u bytes\n",
> + case KASAN_STATE_QUARANTINE:
> + pr_err("Object freed, allocated with size %lu bytes\n",
> alloc_info->alloc_size);
> free_info = get_free_info(cache, object);
> pr_err("Allocation:\n");
> diff --git a/mm/mempool.c b/mm/mempool.c
> index b47c8a7..4beeeef 100644
> --- a/mm/mempool.c
> +++ b/mm/mempool.c
> @@ -105,11 +105,12 @@ static inline void poison_element(mempool_t *pool, void *element)
> static void kasan_poison_element(mempool_t *pool, void *element)
> {
> if (pool->alloc == mempool_alloc_slab)
> - kasan_slab_free(pool->pool_data, element);
> + kasan_poison_slab_free(pool->pool_data, element);
> if (pool->alloc == mempool_kmalloc)
> - kasan_kfree(element);
> + kasan_poison_kfree(element);
> if (pool->alloc == mempool_alloc_pages)
> - kasan_free_pages(element, (unsigned long)pool->pool_data);
> + kasan_poison_free_pages(element,
> + (unsigned long)pool->pool_data);
> }
>
> static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 63358d9..4f65587 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -980,7 +980,7 @@ static bool free_pages_prepare(struct page *page, unsigned int order)
>
> trace_mm_page_free(page, order);
> kmemcheck_free_shadow(page, order);
> - kasan_free_pages(page, order);
> + kasan_poison_free_pages(page, order);
>
> if (PageAnon(page))
> page->mapping = NULL;
> diff --git a/mm/slab.c b/mm/slab.c
> index 0ec7aa3..e2fac67 100644
> --- a/mm/slab.c
> +++ b/mm/slab.c
> @@ -3374,9 +3374,19 @@ free_done:
> static inline void __cache_free(struct kmem_cache *cachep, void *objp,
> unsigned long caller)
> {
> +#ifdef CONFIG_KASAN
> + if (!kasan_slab_free(cachep, objp))
> + /* The object has been put into the quarantine, don't touch it
> + * for now.
> + */
> + nokasan_free(cachep, objp, caller);
> +}
> +
> +void nokasan_free(struct kmem_cache *cachep, void *objp, unsigned long caller)
> +{
> +#endif
It looks not good to me.
Converting __cache_free() to ____cache_free() and making
__cache_free() call ____cache_free() if (!kasan_slab_free()) looks
better to me and less error-prone.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-02-01 2:47 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-01 2:47 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, akpm, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
> Quarantine isolates freed objects in a separate queue. The objects are
> returned to the allocator later, which helps to detect use-after-free
> errors.
>
> Freed objects are first added to per-cpu quarantine queues.
> When a cache is destroyed or memory shrinking is requested, the objects
> are moved into the global quarantine queue. Whenever a kmalloc call
> allows memory reclaiming, the oldest objects are popped out of the
> global queue until the total size of objects in quarantine is less than
> 3/4 of the maximum quarantine size (which is a fraction of installed
> physical memory).
Just wondering why not using time based approach rather than size
based one. In heavy load condition, how much time do the object stay in
quarantine?
>
> Right now quarantine support is only enabled in SLAB allocator.
> Unification of KASAN features in SLAB and SLUB will be done later.
>
> This patch is based on the "mm: kasan: quarantine" patch originally
> prepared by Dmitry Chernenkov.
>
> Signed-off-by: Alexander Potapenko <glider@google.com>
> ---
> include/linux/kasan.h | 30 ++++--
> lib/test_kasan.c | 29 ++++++
> mm/kasan/Makefile | 2 +-
> mm/kasan/kasan.c | 68 +++++++++++-
> mm/kasan/kasan.h | 11 +-
> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
> mm/kasan/report.c | 3 +-
> mm/mempool.c | 7 +-
> mm/page_alloc.c | 2 +-
> mm/slab.c | 12 ++-
> mm/slab.h | 4 +
> mm/slab_common.c | 2 +
> mm/slub.c | 4 +-
> 13 files changed, 435 insertions(+), 23 deletions(-)
>
...
> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
> +{
> +#ifdef CONFIG_SLAB
> + /* RCU slabs could be legally used after free within the RCU period */
> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
> + return false;
> +
> + if (likely(cache->flags & SLAB_KASAN)) {
> + struct kasan_alloc_meta *alloc_info =
> + get_alloc_info(cache, object);
> + struct kasan_free_meta *free_info =
> + get_free_info(cache, object);
> +
> + switch (alloc_info->state) {
> + case KASAN_STATE_ALLOC:
> + alloc_info->state = KASAN_STATE_QUARANTINE;
> + quarantine_put(free_info, cache);
quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
although it's not much meaningful without poisoning. But, I have an
idea to poison object on SLAB_DESTROY_BY_RCU cache.
quarantine_put() moves per cpu list to global queue when
list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
at that time, after then, we can poison objects. With appropriate size
setup, it would not be intrusive.
> + set_track(&free_info->track, GFP_NOWAIT);
set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
> + kasan_poison_slab_free(cache, object);
> + return true;
> + case KASAN_STATE_QUARANTINE:
> + case KASAN_STATE_FREE:
> + pr_err("Double free");
> + dump_stack();
> + break;
> + default:
> + break;
> + }
> + }
> + return false;
> +#else
> + kasan_poison_slab_free(cache, object);
> + return false;
> +#endif
> +}
> +
...
> +void quarantine_reduce(void)
> +{
> + size_t new_quarantine_size;
> + unsigned long flags;
> + struct qlist to_free = QLIST_INIT;
> + size_t size_to_free = 0;
> + void **last;
> +
> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
> + smp_load_acquire(&quarantine_size)))
> + return;
> +
> + spin_lock_irqsave(&quarantine_lock, flags);
> +
> + /* Update quarantine size in case of hotplug. Allocate a fraction of
> + * the installed memory to quarantine minus per-cpu queue limits.
> + */
> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
> + QUARANTINE_FRACTION;
> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
> + smp_store_release(&quarantine_size, new_quarantine_size);
> +
> + last = global_quarantine.head;
> + while (last) {
> + struct kmem_cache *cache = qlink_to_cache(last);
> +
> + size_to_free += cache->size;
> + if (!*last || size_to_free >
> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
> + break;
> + last = (void **) *last;
> + }
> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
> +
> + spin_unlock_irqrestore(&quarantine_lock, flags);
> +
> + qlist_free_all(&to_free, NULL);
> +}
Isn't it better to call quarantine_reduce() in shrink_slab()?
It will help to maximize quarantine time.
> +
> +static inline void qlist_move_cache(struct qlist *from,
> + struct qlist *to,
> + struct kmem_cache *cache)
> +{
> + void ***prev;
> +
> + if (unlikely(empty_qlist(from)))
> + return;
> +
> + prev = &from->head;
> + while (*prev) {
> + void **qlink = *prev;
> + struct kmem_cache *obj_cache = qlink_to_cache(qlink);
> +
> + if (obj_cache == cache) {
> + if (unlikely(from->tail == qlink))
> + from->tail = (void **) prev;
> + *prev = (void **) *qlink;
> + from->bytes -= cache->size;
> + qlist_put(to, qlink, cache->size);
> + } else
> + prev = (void ***) *prev;
> + }
> +}
> +
> +static void per_cpu_remove_cache(void *arg)
> +{
> + struct kmem_cache *cache = arg;
> + struct qlist to_free = QLIST_INIT;
> + struct qlist *q;
> + unsigned long flags;
> +
> + local_irq_save(flags);
> + q = this_cpu_ptr(&cpu_quarantine);
> + qlist_move_cache(q, &to_free, cache);
> + local_irq_restore(flags);
> +
> + qlist_free_all(&to_free, cache);
> +}
> +
> +void quarantine_remove_cache(struct kmem_cache *cache)
> +{
> + unsigned long flags;
> + struct qlist to_free = QLIST_INIT;
> +
> + on_each_cpu(per_cpu_remove_cache, cache, 0);
Should be called with wait = 1.
> +
> + spin_lock_irqsave(&quarantine_lock, flags);
> + qlist_move_cache(&global_quarantine, &to_free, cache);
> + spin_unlock_irqrestore(&quarantine_lock, flags);
> +
> + qlist_free_all(&to_free, cache);
> +}
> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
> index 6c4afcd..a4dca25 100644
> --- a/mm/kasan/report.c
> +++ b/mm/kasan/report.c
> @@ -148,7 +148,8 @@ static void print_object(struct kmem_cache *cache, void *object)
> print_track(&alloc_info->track);
> break;
> case KASAN_STATE_FREE:
> - pr_err("Object freed, allocated with size %u bytes\n",
> + case KASAN_STATE_QUARANTINE:
> + pr_err("Object freed, allocated with size %lu bytes\n",
> alloc_info->alloc_size);
> free_info = get_free_info(cache, object);
> pr_err("Allocation:\n");
> diff --git a/mm/mempool.c b/mm/mempool.c
> index b47c8a7..4beeeef 100644
> --- a/mm/mempool.c
> +++ b/mm/mempool.c
> @@ -105,11 +105,12 @@ static inline void poison_element(mempool_t *pool, void *element)
> static void kasan_poison_element(mempool_t *pool, void *element)
> {
> if (pool->alloc == mempool_alloc_slab)
> - kasan_slab_free(pool->pool_data, element);
> + kasan_poison_slab_free(pool->pool_data, element);
> if (pool->alloc == mempool_kmalloc)
> - kasan_kfree(element);
> + kasan_poison_kfree(element);
> if (pool->alloc == mempool_alloc_pages)
> - kasan_free_pages(element, (unsigned long)pool->pool_data);
> + kasan_poison_free_pages(element,
> + (unsigned long)pool->pool_data);
> }
>
> static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 63358d9..4f65587 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -980,7 +980,7 @@ static bool free_pages_prepare(struct page *page, unsigned int order)
>
> trace_mm_page_free(page, order);
> kmemcheck_free_shadow(page, order);
> - kasan_free_pages(page, order);
> + kasan_poison_free_pages(page, order);
>
> if (PageAnon(page))
> page->mapping = NULL;
> diff --git a/mm/slab.c b/mm/slab.c
> index 0ec7aa3..e2fac67 100644
> --- a/mm/slab.c
> +++ b/mm/slab.c
> @@ -3374,9 +3374,19 @@ free_done:
> static inline void __cache_free(struct kmem_cache *cachep, void *objp,
> unsigned long caller)
> {
> +#ifdef CONFIG_KASAN
> + if (!kasan_slab_free(cachep, objp))
> + /* The object has been put into the quarantine, don't touch it
> + * for now.
> + */
> + nokasan_free(cachep, objp, caller);
> +}
> +
> +void nokasan_free(struct kmem_cache *cachep, void *objp, unsigned long caller)
> +{
> +#endif
It looks not good to me.
Converting __cache_free() to ____cache_free() and making
__cache_free() call ____cache_free() if (!kasan_slab_free()) looks
better to me and less error-prone.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-01-28 13:27 ` Alexander Potapenko
@ 2016-02-01 2:55 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-01 2:55 UTC (permalink / raw)
To: Alexander Potapenko
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
> >
> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
> >>
> >> Hello,
> >>
> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
> >> > for memory chunks. The stack traces are stored in a hash table and
> >> > referenced by handles which reside in the kasan_alloc_meta and
> >> > kasan_free_meta structures in the allocated memory chunks.
> >>
> >> Looks really nice!
> >>
> >> Could it be more generalized to be used by other feature that need to
> >> store stack trace such as tracepoint or page owner?
> > Certainly yes, but see below.
> >
> >> If it could be, there is one more requirement.
> >> I understand the fact that entry is never removed from depot makes things
> >> very simpler, but, for general usecases, it's better to use reference
> >> count
> >> and allow to remove. Is it possible?
> > For our use case reference counting is not really necessary, and it would
> > introduce unwanted contention.
Okay.
> > There are two possible options, each having its advantages and drawbacks: we
> > can let the clients store the refcounters directly in their stacks (more
> > universal, but harder to use for the clients), or keep the counters in the
> > depot but add an API that does not change them (easier for the clients, but
> > potentially error-prone).
> > I'd say it's better to actually find at least one more user for the stack
> > depot in order to understand the requirements, and refactor the code after
> > that.
I re-think the page owner case and it also may not need refcount.
For now, just moving this stuff to /lib would be helpful for other future user.
BTW, is there any performance number? I guess that it could affect
the performance.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-02-01 2:55 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-01 2:55 UTC (permalink / raw)
To: Alexander Potapenko
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
rostedt
On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
> >
> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
> >>
> >> Hello,
> >>
> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
> >> > for memory chunks. The stack traces are stored in a hash table and
> >> > referenced by handles which reside in the kasan_alloc_meta and
> >> > kasan_free_meta structures in the allocated memory chunks.
> >>
> >> Looks really nice!
> >>
> >> Could it be more generalized to be used by other feature that need to
> >> store stack trace such as tracepoint or page owner?
> > Certainly yes, but see below.
> >
> >> If it could be, there is one more requirement.
> >> I understand the fact that entry is never removed from depot makes things
> >> very simpler, but, for general usecases, it's better to use reference
> >> count
> >> and allow to remove. Is it possible?
> > For our use case reference counting is not really necessary, and it would
> > introduce unwanted contention.
Okay.
> > There are two possible options, each having its advantages and drawbacks: we
> > can let the clients store the refcounters directly in their stacks (more
> > universal, but harder to use for the clients), or keep the counters in the
> > depot but add an API that does not change them (easier for the clients, but
> > potentially error-prone).
> > I'd say it's better to actually find at least one more user for the stack
> > depot in order to understand the requirements, and refactor the code after
> > that.
I re-think the page owner case and it also may not need refcount.
For now, just moving this stuff to /lib would be helpful for other future user.
BTW, is there any performance number? I guess that it could affect
the performance.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
2016-01-27 18:25 ` Alexander Potapenko
@ 2016-02-02 5:34 ` Andrew Morton
-1 siblings, 0 replies; 78+ messages in thread
From: Andrew Morton @ 2016-02-02 5:34 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
> depending on which allocator (SLAB or SLUB) is being used
>
> ...
>
> --- a/lib/test_kasan.c
> +++ b/lib/test_kasan.c
> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
> static noinline void __init kmalloc_large_oob_right(void)
> {
> char *ptr;
> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
> + size_t size;
> +
> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
> + /*
> + * We're using the SLAB allocator. Allocate a chunk that fits
> + * into a slab.
> + */
> + size = KMALLOC_MAX_CACHE_SIZE - 256;
> + } else {
> + /*
> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
> + * We're using the SLUB allocator. Allocate a chunk that does
> + * not fit into a slab to trigger the page allocator.
> + */
> + size = KMALLOC_MAX_CACHE_SIZE + 10;
> + }
This seems a weird way of working out whether we're using SLAB or SLUB.
Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
something standardized rather than a weird just-happens-to-work like
this.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
@ 2016-02-02 5:34 ` Andrew Morton
0 siblings, 0 replies; 78+ messages in thread
From: Andrew Morton @ 2016-02-02 5:34 UTC (permalink / raw)
To: Alexander Potapenko
Cc: adech.fo, cl, dvyukov, ryabinin.a.a, rostedt, kasan-dev,
linux-kernel, linux-mm
On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
> depending on which allocator (SLAB or SLUB) is being used
>
> ...
>
> --- a/lib/test_kasan.c
> +++ b/lib/test_kasan.c
> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
> static noinline void __init kmalloc_large_oob_right(void)
> {
> char *ptr;
> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
> + size_t size;
> +
> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
> + /*
> + * We're using the SLAB allocator. Allocate a chunk that fits
> + * into a slab.
> + */
> + size = KMALLOC_MAX_CACHE_SIZE - 256;
> + } else {
> + /*
> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
> + * We're using the SLUB allocator. Allocate a chunk that does
> + * not fit into a slab to trigger the page allocator.
> + */
> + size = KMALLOC_MAX_CACHE_SIZE + 10;
> + }
This seems a weird way of working out whether we're using SLAB or SLUB.
Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
something standardized rather than a weird just-happens-to-work like
this.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
2016-02-02 5:34 ` Andrew Morton
@ 2016-02-02 15:29 ` Andrey Ryabinin
-1 siblings, 0 replies; 78+ messages in thread
From: Andrey Ryabinin @ 2016-02-02 15:29 UTC (permalink / raw)
To: Andrew Morton, Alexander Potapenko
Cc: adech.fo, cl, dvyukov, rostedt, kasan-dev, linux-kernel, linux-mm
On 02/02/2016 08:34 AM, Andrew Morton wrote:
> On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
>
>> depending on which allocator (SLAB or SLUB) is being used
>>
>> ...
>>
>> --- a/lib/test_kasan.c
>> +++ b/lib/test_kasan.c
>> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
>> static noinline void __init kmalloc_large_oob_right(void)
>> {
>> char *ptr;
>> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
>> + size_t size;
>> +
>> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
>> + /*
>> + * We're using the SLAB allocator. Allocate a chunk that fits
>> + * into a slab.
>> + */
>> + size = KMALLOC_MAX_CACHE_SIZE - 256;
>> + } else {
>> + /*
>> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
>> + * We're using the SLUB allocator. Allocate a chunk that does
>> + * not fit into a slab to trigger the page allocator.
>> + */
>> + size = KMALLOC_MAX_CACHE_SIZE + 10;
>> + }
>
> This seems a weird way of working out whether we're using SLAB or SLUB.
>
> Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
> something standardized rather than a weird just-happens-to-work like
> this.
>
Actually it would be simpler to not use KMALLOC_MAX_CACHE_SIZE at all.
Simply replace it with 2 or 3 PAGE_SIZEs.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
@ 2016-02-02 15:29 ` Andrey Ryabinin
0 siblings, 0 replies; 78+ messages in thread
From: Andrey Ryabinin @ 2016-02-02 15:29 UTC (permalink / raw)
To: Andrew Morton, Alexander Potapenko
Cc: adech.fo, cl, dvyukov, rostedt, kasan-dev, linux-kernel, linux-mm
On 02/02/2016 08:34 AM, Andrew Morton wrote:
> On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
>
>> depending on which allocator (SLAB or SLUB) is being used
>>
>> ...
>>
>> --- a/lib/test_kasan.c
>> +++ b/lib/test_kasan.c
>> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
>> static noinline void __init kmalloc_large_oob_right(void)
>> {
>> char *ptr;
>> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
>> + size_t size;
>> +
>> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
>> + /*
>> + * We're using the SLAB allocator. Allocate a chunk that fits
>> + * into a slab.
>> + */
>> + size = KMALLOC_MAX_CACHE_SIZE - 256;
>> + } else {
>> + /*
>> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
>> + * We're using the SLUB allocator. Allocate a chunk that does
>> + * not fit into a slab to trigger the page allocator.
>> + */
>> + size = KMALLOC_MAX_CACHE_SIZE + 10;
>> + }
>
> This seems a weird way of working out whether we're using SLAB or SLUB.
>
> Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
> something standardized rather than a weird just-happens-to-work like
> this.
>
Actually it would be simpler to not use KMALLOC_MAX_CACHE_SIZE at all.
Simply replace it with 2 or 3 PAGE_SIZEs.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
2016-02-02 15:29 ` Andrey Ryabinin
@ 2016-02-02 16:25 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-02 16:25 UTC (permalink / raw)
To: Andrey Ryabinin
Cc: Andrew Morton, Andrey Konovalov, Christoph Lameter,
Dmitriy Vyukov, Steven Rostedt, kasan-dev, linux-kernel,
linux-mm
The intention was to detect the situation in which a new allocator
appears for which we don't know how it behaves if we allocate more
than KMALLOC_MAX_CACHE_SIZE.
I agree this makes little sense and we can just stick to
CONFIG_SLAB/CONFIG_SLUB cases.
However I think it's better to keep 'size = KMALLOC_MAX_CACHE_SIZE +
something' to keep this code working in the case the value of
KMALLOC_MAX_CACHE_SIZE changes.
On Tue, Feb 2, 2016 at 4:29 PM, Andrey Ryabinin <ryabinin.a.a@gmail.com> wrote:
>
>
> On 02/02/2016 08:34 AM, Andrew Morton wrote:
>> On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
>>
>>> depending on which allocator (SLAB or SLUB) is being used
>>>
>>> ...
>>>
>>> --- a/lib/test_kasan.c
>>> +++ b/lib/test_kasan.c
>>> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
>>> static noinline void __init kmalloc_large_oob_right(void)
>>> {
>>> char *ptr;
>>> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
>>> + size_t size;
>>> +
>>> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
>>> + /*
>>> + * We're using the SLAB allocator. Allocate a chunk that fits
>>> + * into a slab.
>>> + */
>>> + size = KMALLOC_MAX_CACHE_SIZE - 256;
>>> + } else {
>>> + /*
>>> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
>>> + * We're using the SLUB allocator. Allocate a chunk that does
>>> + * not fit into a slab to trigger the page allocator.
>>> + */
>>> + size = KMALLOC_MAX_CACHE_SIZE + 10;
>>> + }
>>
>> This seems a weird way of working out whether we're using SLAB or SLUB.
>>
>> Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
>> something standardized rather than a weird just-happens-to-work like
>> this.
>>
>
> Actually it would be simpler to not use KMALLOC_MAX_CACHE_SIZE at all.
> Simply replace it with 2 or 3 PAGE_SIZEs.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
@ 2016-02-02 16:25 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-02 16:25 UTC (permalink / raw)
To: Andrey Ryabinin
Cc: Andrew Morton, Andrey Konovalov, Christoph Lameter,
Dmitriy Vyukov, Steven Rostedt, kasan-dev, linux-kernel,
linux-mm
The intention was to detect the situation in which a new allocator
appears for which we don't know how it behaves if we allocate more
than KMALLOC_MAX_CACHE_SIZE.
I agree this makes little sense and we can just stick to
CONFIG_SLAB/CONFIG_SLUB cases.
However I think it's better to keep 'size = KMALLOC_MAX_CACHE_SIZE +
something' to keep this code working in the case the value of
KMALLOC_MAX_CACHE_SIZE changes.
On Tue, Feb 2, 2016 at 4:29 PM, Andrey Ryabinin <ryabinin.a.a@gmail.com> wrote:
>
>
> On 02/02/2016 08:34 AM, Andrew Morton wrote:
>> On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
>>
>>> depending on which allocator (SLAB or SLUB) is being used
>>>
>>> ...
>>>
>>> --- a/lib/test_kasan.c
>>> +++ b/lib/test_kasan.c
>>> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
>>> static noinline void __init kmalloc_large_oob_right(void)
>>> {
>>> char *ptr;
>>> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
>>> + size_t size;
>>> +
>>> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
>>> + /*
>>> + * We're using the SLAB allocator. Allocate a chunk that fits
>>> + * into a slab.
>>> + */
>>> + size = KMALLOC_MAX_CACHE_SIZE - 256;
>>> + } else {
>>> + /*
>>> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
>>> + * We're using the SLUB allocator. Allocate a chunk that does
>>> + * not fit into a slab to trigger the page allocator.
>>> + */
>>> + size = KMALLOC_MAX_CACHE_SIZE + 10;
>>> + }
>>
>> This seems a weird way of working out whether we're using SLAB or SLUB.
>>
>> Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
>> something standardized rather than a weird just-happens-to-work like
>> this.
>>
>
> Actually it would be simpler to not use KMALLOC_MAX_CACHE_SIZE at all.
> Simply replace it with 2 or 3 PAGE_SIZEs.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
2016-02-02 16:25 ` Alexander Potapenko
@ 2016-02-15 14:05 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-15 14:05 UTC (permalink / raw)
To: Andrey Ryabinin
Cc: Andrew Morton, Andrey Konovalov, Christoph Lameter,
Dmitriy Vyukov, Steven Rostedt, kasan-dev, linux-kernel,
linux-mm
Turns out I've actually overintellectualized this test.
I've reworked this patch so that kmalloc_pagealloc_oob_right()
allocates a big chunk of memory from the page allocator (and therefore
is enabled only under CONFIG_SLUB).
kmalloc_large_oob_right() now allocates KMALLOC_MAX_CACHE_SIZE - 256
in both SLAB and SLUB modes.
I'll send the updated patch set later today.
On Tue, Feb 2, 2016 at 5:25 PM, Alexander Potapenko <glider@google.com> wrote:
> The intention was to detect the situation in which a new allocator
> appears for which we don't know how it behaves if we allocate more
> than KMALLOC_MAX_CACHE_SIZE.
> I agree this makes little sense and we can just stick to
> CONFIG_SLAB/CONFIG_SLUB cases.
>
> However I think it's better to keep 'size = KMALLOC_MAX_CACHE_SIZE +
> something' to keep this code working in the case the value of
> KMALLOC_MAX_CACHE_SIZE changes.
>
> On Tue, Feb 2, 2016 at 4:29 PM, Andrey Ryabinin <ryabinin.a.a@gmail.com> wrote:
>>
>>
>> On 02/02/2016 08:34 AM, Andrew Morton wrote:
>>> On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
>>>
>>>> depending on which allocator (SLAB or SLUB) is being used
>>>>
>>>> ...
>>>>
>>>> --- a/lib/test_kasan.c
>>>> +++ b/lib/test_kasan.c
>>>> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
>>>> static noinline void __init kmalloc_large_oob_right(void)
>>>> {
>>>> char *ptr;
>>>> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
>>>> + size_t size;
>>>> +
>>>> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
>>>> + /*
>>>> + * We're using the SLAB allocator. Allocate a chunk that fits
>>>> + * into a slab.
>>>> + */
>>>> + size = KMALLOC_MAX_CACHE_SIZE - 256;
>>>> + } else {
>>>> + /*
>>>> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
>>>> + * We're using the SLUB allocator. Allocate a chunk that does
>>>> + * not fit into a slab to trigger the page allocator.
>>>> + */
>>>> + size = KMALLOC_MAX_CACHE_SIZE + 10;
>>>> + }
>>>
>>> This seems a weird way of working out whether we're using SLAB or SLUB.
>>>
>>> Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
>>> something standardized rather than a weird just-happens-to-work like
>>> this.
>>>
>>
>> Actually it would be simpler to not use KMALLOC_MAX_CACHE_SIZE at all.
>> Simply replace it with 2 or 3 PAGE_SIZEs.
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
> Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
> leiten Sie diese bitte nicht weiter, informieren Sie den
> Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
> This e-mail is confidential. If you are not the right addressee please
> do not forward it, please inform the sender, and please erase this
> e-mail including any attachments. Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test
@ 2016-02-15 14:05 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-15 14:05 UTC (permalink / raw)
To: Andrey Ryabinin
Cc: Andrew Morton, Andrey Konovalov, Christoph Lameter,
Dmitriy Vyukov, Steven Rostedt, kasan-dev, linux-kernel,
linux-mm
Turns out I've actually overintellectualized this test.
I've reworked this patch so that kmalloc_pagealloc_oob_right()
allocates a big chunk of memory from the page allocator (and therefore
is enabled only under CONFIG_SLUB).
kmalloc_large_oob_right() now allocates KMALLOC_MAX_CACHE_SIZE - 256
in both SLAB and SLUB modes.
I'll send the updated patch set later today.
On Tue, Feb 2, 2016 at 5:25 PM, Alexander Potapenko <glider@google.com> wrote:
> The intention was to detect the situation in which a new allocator
> appears for which we don't know how it behaves if we allocate more
> than KMALLOC_MAX_CACHE_SIZE.
> I agree this makes little sense and we can just stick to
> CONFIG_SLAB/CONFIG_SLUB cases.
>
> However I think it's better to keep 'size = KMALLOC_MAX_CACHE_SIZE +
> something' to keep this code working in the case the value of
> KMALLOC_MAX_CACHE_SIZE changes.
>
> On Tue, Feb 2, 2016 at 4:29 PM, Andrey Ryabinin <ryabinin.a.a@gmail.com> wrote:
>>
>>
>> On 02/02/2016 08:34 AM, Andrew Morton wrote:
>>> On Wed, 27 Jan 2016 19:25:06 +0100 Alexander Potapenko <glider@google.com> wrote:
>>>
>>>> depending on which allocator (SLAB or SLUB) is being used
>>>>
>>>> ...
>>>>
>>>> --- a/lib/test_kasan.c
>>>> +++ b/lib/test_kasan.c
>>>> @@ -68,7 +68,22 @@ static noinline void __init kmalloc_node_oob_right(void)
>>>> static noinline void __init kmalloc_large_oob_right(void)
>>>> {
>>>> char *ptr;
>>>> - size_t size = KMALLOC_MAX_CACHE_SIZE + 10;
>>>> + size_t size;
>>>> +
>>>> + if (KMALLOC_MAX_CACHE_SIZE == KMALLOC_MAX_SIZE) {
>>>> + /*
>>>> + * We're using the SLAB allocator. Allocate a chunk that fits
>>>> + * into a slab.
>>>> + */
>>>> + size = KMALLOC_MAX_CACHE_SIZE - 256;
>>>> + } else {
>>>> + /*
>>>> + * KMALLOC_MAX_SIZE > KMALLOC_MAX_CACHE_SIZE.
>>>> + * We're using the SLUB allocator. Allocate a chunk that does
>>>> + * not fit into a slab to trigger the page allocator.
>>>> + */
>>>> + size = KMALLOC_MAX_CACHE_SIZE + 10;
>>>> + }
>>>
>>> This seems a weird way of working out whether we're using SLAB or SLUB.
>>>
>>> Can't we use, umm, #ifdef CONFIG_SLAB? If not that then let's cook up
>>> something standardized rather than a weird just-happens-to-work like
>>> this.
>>>
>>
>> Actually it would be simpler to not use KMALLOC_MAX_CACHE_SIZE at all.
>> Simply replace it with 2 or 3 PAGE_SIZEs.
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
> Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
> leiten Sie diese bitte nicht weiter, informieren Sie den
> Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
> This e-mail is confidential. If you are not the right addressee please
> do not forward it, please inform the sender, and please erase this
> e-mail including any attachments. Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
2016-01-29 14:45 ` Steven Rostedt
@ 2016-02-16 15:32 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-16 15:32 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
Ok, interrupt.h sounds good.
On Fri, Jan 29, 2016 at 3:45 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Fri, 29 Jan 2016 12:59:13 +0100
> Alexander Potapenko <glider@google.com> wrote:
>
>> On the other hand, this will require including <linux/irq.h> into
>> various files that currently use __irq_section.
>> But that header has a comment saying:
>>
>> /*
>> * Please do not include this file in generic code. There is currently
>> * no requirement for any architecture to implement anything held
>> * within this file.
>> *
>> * Thanks. --rmk
>> */
>>
>> Do we really want to put anything into that header?
>>
>
> What about interrupt.h?
>
> It's just weird to have KSAN needing to pull in ftrace.h for irq work.
>
> -- Steve
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections
@ 2016-02-16 15:32 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-16 15:32 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, kasan-dev, linux-kernel,
linux-mm
Ok, interrupt.h sounds good.
On Fri, Jan 29, 2016 at 3:45 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Fri, 29 Jan 2016 12:59:13 +0100
> Alexander Potapenko <glider@google.com> wrote:
>
>> On the other hand, this will require including <linux/irq.h> into
>> various files that currently use __irq_section.
>> But that header has a comment saying:
>>
>> /*
>> * Please do not include this file in generic code. There is currently
>> * no requirement for any architecture to implement anything held
>> * within this file.
>> *
>> * Thanks. --rmk
>> */
>>
>> Do we really want to put anything into that header?
>>
>
> What about interrupt.h?
>
> It's just weird to have KSAN needing to pull in ftrace.h for irq work.
>
> -- Steve
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-02-01 2:55 ` Joonsoo Kim
@ 2016-02-16 18:37 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-16 18:37 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
Steven Rostedt
On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>> >
>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>> >>
>> >> Hello,
>> >>
>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>> >> > for memory chunks. The stack traces are stored in a hash table and
>> >> > referenced by handles which reside in the kasan_alloc_meta and
>> >> > kasan_free_meta structures in the allocated memory chunks.
>> >>
>> >> Looks really nice!
>> >>
>> >> Could it be more generalized to be used by other feature that need to
>> >> store stack trace such as tracepoint or page owner?
>> > Certainly yes, but see below.
>> >
>> >> If it could be, there is one more requirement.
>> >> I understand the fact that entry is never removed from depot makes things
>> >> very simpler, but, for general usecases, it's better to use reference
>> >> count
>> >> and allow to remove. Is it possible?
>> > For our use case reference counting is not really necessary, and it would
>> > introduce unwanted contention.
>
> Okay.
>
>> > There are two possible options, each having its advantages and drawbacks: we
>> > can let the clients store the refcounters directly in their stacks (more
>> > universal, but harder to use for the clients), or keep the counters in the
>> > depot but add an API that does not change them (easier for the clients, but
>> > potentially error-prone).
>> > I'd say it's better to actually find at least one more user for the stack
>> > depot in order to understand the requirements, and refactor the code after
>> > that.
>
> I re-think the page owner case and it also may not need refcount.
> For now, just moving this stuff to /lib would be helpful for other future user.
I agree this code may need to be moved to /lib someday, but I wouldn't
hurry with that.
Right now it is quite KASAN-specific, and it's unclear yet whether
anyone else is going to use it.
I suggest we keep it in mm/kasan for now, and factor the common parts
into /lib when the need arises.
> BTW, is there any performance number? I guess that it could affect
> the performance.
I've compared the performance of KASAN with SLAB allocator on a small
synthetic benchmark in two modes: with stack depot enabled and with
kasan_save_stack() unconditionally returning 0.
In the former case 8% more time was spent in the kernel than in the latter case.
If I am not mistaking, for SLUB allocator the bookkeeping (enabled
with the slub_debug=UZ boot options) take only 1.5 time, so the
difference is worth looking into (at least before we switch SLUB to
stack depot).
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-02-16 18:37 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-16 18:37 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
Steven Rostedt
On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>> >
>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>> >>
>> >> Hello,
>> >>
>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>> >> > for memory chunks. The stack traces are stored in a hash table and
>> >> > referenced by handles which reside in the kasan_alloc_meta and
>> >> > kasan_free_meta structures in the allocated memory chunks.
>> >>
>> >> Looks really nice!
>> >>
>> >> Could it be more generalized to be used by other feature that need to
>> >> store stack trace such as tracepoint or page owner?
>> > Certainly yes, but see below.
>> >
>> >> If it could be, there is one more requirement.
>> >> I understand the fact that entry is never removed from depot makes things
>> >> very simpler, but, for general usecases, it's better to use reference
>> >> count
>> >> and allow to remove. Is it possible?
>> > For our use case reference counting is not really necessary, and it would
>> > introduce unwanted contention.
>
> Okay.
>
>> > There are two possible options, each having its advantages and drawbacks: we
>> > can let the clients store the refcounters directly in their stacks (more
>> > universal, but harder to use for the clients), or keep the counters in the
>> > depot but add an API that does not change them (easier for the clients, but
>> > potentially error-prone).
>> > I'd say it's better to actually find at least one more user for the stack
>> > depot in order to understand the requirements, and refactor the code after
>> > that.
>
> I re-think the page owner case and it also may not need refcount.
> For now, just moving this stuff to /lib would be helpful for other future user.
I agree this code may need to be moved to /lib someday, but I wouldn't
hurry with that.
Right now it is quite KASAN-specific, and it's unclear yet whether
anyone else is going to use it.
I suggest we keep it in mm/kasan for now, and factor the common parts
into /lib when the need arises.
> BTW, is there any performance number? I guess that it could affect
> the performance.
I've compared the performance of KASAN with SLAB allocator on a small
synthetic benchmark in two modes: with stack depot enabled and with
kasan_save_stack() unconditionally returning 0.
In the former case 8% more time was spent in the kernel than in the latter case.
If I am not mistaking, for SLUB allocator the bookkeeping (enabled
with the slub_debug=UZ boot options) take only 1.5 time, so the
difference is worth looking into (at least before we switch SLUB to
stack depot).
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-02-16 18:37 ` Alexander Potapenko
@ 2016-02-17 18:29 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-17 18:29 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
Steven Rostedt
On Tue, Feb 16, 2016 at 7:37 PM, Alexander Potapenko <glider@google.com> wrote:
> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>> >
>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>> >>
>>> >> Looks really nice!
>>> >>
>>> >> Could it be more generalized to be used by other feature that need to
>>> >> store stack trace such as tracepoint or page owner?
>>> > Certainly yes, but see below.
>>> >
>>> >> If it could be, there is one more requirement.
>>> >> I understand the fact that entry is never removed from depot makes things
>>> >> very simpler, but, for general usecases, it's better to use reference
>>> >> count
>>> >> and allow to remove. Is it possible?
>>> > For our use case reference counting is not really necessary, and it would
>>> > introduce unwanted contention.
>>
>> Okay.
>>
>>> > There are two possible options, each having its advantages and drawbacks: we
>>> > can let the clients store the refcounters directly in their stacks (more
>>> > universal, but harder to use for the clients), or keep the counters in the
>>> > depot but add an API that does not change them (easier for the clients, but
>>> > potentially error-prone).
>>> > I'd say it's better to actually find at least one more user for the stack
>>> > depot in order to understand the requirements, and refactor the code after
>>> > that.
>>
>> I re-think the page owner case and it also may not need refcount.
>> For now, just moving this stuff to /lib would be helpful for other future user.
> I agree this code may need to be moved to /lib someday, but I wouldn't
> hurry with that.
> Right now it is quite KASAN-specific, and it's unclear yet whether
> anyone else is going to use it.
> I suggest we keep it in mm/kasan for now, and factor the common parts
> into /lib when the need arises.
>
>> BTW, is there any performance number? I guess that it could affect
>> the performance.
> I've compared the performance of KASAN with SLAB allocator on a small
> synthetic benchmark in two modes: with stack depot enabled and with
> kasan_save_stack() unconditionally returning 0.
> In the former case 8% more time was spent in the kernel than in the latter case.
>
> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
> with the slub_debug=UZ boot options) take only 1.5 time, so the
> difference is worth looking into (at least before we switch SLUB to
> stack depot).
I've made additional measurements.
Previously I had been using a userspace benchmark that created and
destroyed pipes in a loop
(https://github.com/google/sanitizers/blob/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c).
Now I've made a kernel module that allocated and deallocated memory
chunks of different sizes in a loop.
There were two modes of operation:
1) all the allocations were made from the same function, therefore all
allocation/deallocation stacks were similar and there always was a hit
in the stackdepot hashtable
2) The allocations were made from 2^16 different stacks.
In the first case SLAB+stackdepot turned out to be 13% faster than
SLUB+slub_debug, in the second SLAB was 11% faster.
Note that in both cases and for both allocators most of the time (more
than 90%) was spent in the x86 stack unwinder, which is common for
both approaches.
Yet another observation regarding stackdepot: under a heavy load
(running Trinity for a hour, 101M allocations) the depot saturates at
around 20K records with the hashtable miss rate of 0.02%.
That said, I still cannot justify the results of the userspace
benchmark, but the slowdown of the stackdepot approach for SLAB sounds
acceptable, especially given the memory gain compared to SLUB
bookkeeping (which requires 128 bytes per memory allocation) and the
fact we'll be dealing with the fast path most of the time.
It will certainly be nice to compare SLUB+slub_debug to
SLUB+stackdepot once we start switching SLUB to stackdepot.
>
>> Thanks.
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
> Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
> leiten Sie diese bitte nicht weiter, informieren Sie den
> Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
> This e-mail is confidential. If you are not the right addressee please
> do not forward it, please inform the sender, and please erase this
> e-mail including any attachments. Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-02-17 18:29 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-17 18:29 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
Steven Rostedt
On Tue, Feb 16, 2016 at 7:37 PM, Alexander Potapenko <glider@google.com> wrote:
> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>> >
>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>> >>
>>> >> Looks really nice!
>>> >>
>>> >> Could it be more generalized to be used by other feature that need to
>>> >> store stack trace such as tracepoint or page owner?
>>> > Certainly yes, but see below.
>>> >
>>> >> If it could be, there is one more requirement.
>>> >> I understand the fact that entry is never removed from depot makes things
>>> >> very simpler, but, for general usecases, it's better to use reference
>>> >> count
>>> >> and allow to remove. Is it possible?
>>> > For our use case reference counting is not really necessary, and it would
>>> > introduce unwanted contention.
>>
>> Okay.
>>
>>> > There are two possible options, each having its advantages and drawbacks: we
>>> > can let the clients store the refcounters directly in their stacks (more
>>> > universal, but harder to use for the clients), or keep the counters in the
>>> > depot but add an API that does not change them (easier for the clients, but
>>> > potentially error-prone).
>>> > I'd say it's better to actually find at least one more user for the stack
>>> > depot in order to understand the requirements, and refactor the code after
>>> > that.
>>
>> I re-think the page owner case and it also may not need refcount.
>> For now, just moving this stuff to /lib would be helpful for other future user.
> I agree this code may need to be moved to /lib someday, but I wouldn't
> hurry with that.
> Right now it is quite KASAN-specific, and it's unclear yet whether
> anyone else is going to use it.
> I suggest we keep it in mm/kasan for now, and factor the common parts
> into /lib when the need arises.
>
>> BTW, is there any performance number? I guess that it could affect
>> the performance.
> I've compared the performance of KASAN with SLAB allocator on a small
> synthetic benchmark in two modes: with stack depot enabled and with
> kasan_save_stack() unconditionally returning 0.
> In the former case 8% more time was spent in the kernel than in the latter case.
>
> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
> with the slub_debug=UZ boot options) take only 1.5 time, so the
> difference is worth looking into (at least before we switch SLUB to
> stack depot).
I've made additional measurements.
Previously I had been using a userspace benchmark that created and
destroyed pipes in a loop
(https://github.com/google/sanitizers/blob/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c).
Now I've made a kernel module that allocated and deallocated memory
chunks of different sizes in a loop.
There were two modes of operation:
1) all the allocations were made from the same function, therefore all
allocation/deallocation stacks were similar and there always was a hit
in the stackdepot hashtable
2) The allocations were made from 2^16 different stacks.
In the first case SLAB+stackdepot turned out to be 13% faster than
SLUB+slub_debug, in the second SLAB was 11% faster.
Note that in both cases and for both allocators most of the time (more
than 90%) was spent in the x86 stack unwinder, which is common for
both approaches.
Yet another observation regarding stackdepot: under a heavy load
(running Trinity for a hour, 101M allocations) the depot saturates at
around 20K records with the hashtable miss rate of 0.02%.
That said, I still cannot justify the results of the userspace
benchmark, but the slowdown of the stackdepot approach for SLAB sounds
acceptable, especially given the memory gain compared to SLUB
bookkeeping (which requires 128 bytes per memory allocation) and the
fact we'll be dealing with the fast path most of the time.
It will certainly be nice to compare SLUB+slub_debug to
SLUB+stackdepot once we start switching SLUB to stackdepot.
>
>> Thanks.
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Straße, 33
> 80636 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
> Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
> leiten Sie diese bitte nicht weiter, informieren Sie den
> Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
> This e-mail is confidential. If you are not the right addressee please
> do not forward it, please inform the sender, and please erase this
> e-mail including any attachments. Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-02-16 18:37 ` Alexander Potapenko
@ 2016-02-18 7:58 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-18 7:58 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
2016-02-17 3:37 GMT+09:00 Alexander Potapenko <glider@google.com>:
> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>> >
>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>> >>
>>> >> Looks really nice!
>>> >>
>>> >> Could it be more generalized to be used by other feature that need to
>>> >> store stack trace such as tracepoint or page owner?
>>> > Certainly yes, but see below.
>>> >
>>> >> If it could be, there is one more requirement.
>>> >> I understand the fact that entry is never removed from depot makes things
>>> >> very simpler, but, for general usecases, it's better to use reference
>>> >> count
>>> >> and allow to remove. Is it possible?
>>> > For our use case reference counting is not really necessary, and it would
>>> > introduce unwanted contention.
>>
>> Okay.
>>
>>> > There are two possible options, each having its advantages and drawbacks: we
>>> > can let the clients store the refcounters directly in their stacks (more
>>> > universal, but harder to use for the clients), or keep the counters in the
>>> > depot but add an API that does not change them (easier for the clients, but
>>> > potentially error-prone).
>>> > I'd say it's better to actually find at least one more user for the stack
>>> > depot in order to understand the requirements, and refactor the code after
>>> > that.
>>
>> I re-think the page owner case and it also may not need refcount.
>> For now, just moving this stuff to /lib would be helpful for other future user.
> I agree this code may need to be moved to /lib someday, but I wouldn't
> hurry with that.
> Right now it is quite KASAN-specific, and it's unclear yet whether
> anyone else is going to use it.
> I suggest we keep it in mm/kasan for now, and factor the common parts
> into /lib when the need arises.
Please consider it one more time. I really have a plan to use it on page owner,
because using page owner requires too many memory for stack trace and
it changes system behaviour a lot.
Page owner uses following structure to store stack trace.
struct page_ext {
unsigned long flags;
#ifdef CONFIG_PAGE_OWNER
unsigned int order;
gfp_t gfp_mask;
unsigned int nr_entries;
int last_migrate_reason;
unsigned long trace_entries[8];
#endif
};
Using stack depot in page owner would be straight forward if stack depot
is in /lib. It is possible to move it when needed but it requires moving
a file and it would not be desirable.
>> BTW, is there any performance number? I guess that it could affect
>> the performance.
> I've compared the performance of KASAN with SLAB allocator on a small
> synthetic benchmark in two modes: with stack depot enabled and with
> kasan_save_stack() unconditionally returning 0.
> In the former case 8% more time was spent in the kernel than in the latter case.
>
> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
> with the slub_debug=UZ boot options) take only 1.5 time, so the
> difference is worth looking into (at least before we switch SLUB to
> stack depot).
Okay.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-02-18 7:58 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-18 7:58 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
2016-02-17 3:37 GMT+09:00 Alexander Potapenko <glider@google.com>:
> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>> >
>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>> >>
>>> >> Looks really nice!
>>> >>
>>> >> Could it be more generalized to be used by other feature that need to
>>> >> store stack trace such as tracepoint or page owner?
>>> > Certainly yes, but see below.
>>> >
>>> >> If it could be, there is one more requirement.
>>> >> I understand the fact that entry is never removed from depot makes things
>>> >> very simpler, but, for general usecases, it's better to use reference
>>> >> count
>>> >> and allow to remove. Is it possible?
>>> > For our use case reference counting is not really necessary, and it would
>>> > introduce unwanted contention.
>>
>> Okay.
>>
>>> > There are two possible options, each having its advantages and drawbacks: we
>>> > can let the clients store the refcounters directly in their stacks (more
>>> > universal, but harder to use for the clients), or keep the counters in the
>>> > depot but add an API that does not change them (easier for the clients, but
>>> > potentially error-prone).
>>> > I'd say it's better to actually find at least one more user for the stack
>>> > depot in order to understand the requirements, and refactor the code after
>>> > that.
>>
>> I re-think the page owner case and it also may not need refcount.
>> For now, just moving this stuff to /lib would be helpful for other future user.
> I agree this code may need to be moved to /lib someday, but I wouldn't
> hurry with that.
> Right now it is quite KASAN-specific, and it's unclear yet whether
> anyone else is going to use it.
> I suggest we keep it in mm/kasan for now, and factor the common parts
> into /lib when the need arises.
Please consider it one more time. I really have a plan to use it on page owner,
because using page owner requires too many memory for stack trace and
it changes system behaviour a lot.
Page owner uses following structure to store stack trace.
struct page_ext {
unsigned long flags;
#ifdef CONFIG_PAGE_OWNER
unsigned int order;
gfp_t gfp_mask;
unsigned int nr_entries;
int last_migrate_reason;
unsigned long trace_entries[8];
#endif
};
Using stack depot in page owner would be straight forward if stack depot
is in /lib. It is possible to move it when needed but it requires moving
a file and it would not be desirable.
>> BTW, is there any performance number? I guess that it could affect
>> the performance.
> I've compared the performance of KASAN with SLAB allocator on a small
> synthetic benchmark in two modes: with stack depot enabled and with
> kasan_save_stack() unconditionally returning 0.
> In the former case 8% more time was spent in the kernel than in the latter case.
>
> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
> with the slub_debug=UZ boot options) take only 1.5 time, so the
> difference is worth looking into (at least before we switch SLUB to
> stack depot).
Okay.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-02-17 18:29 ` Alexander Potapenko
@ 2016-02-18 8:13 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-18 8:13 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
2016-02-18 3:29 GMT+09:00 Alexander Potapenko <glider@google.com>:
> On Tue, Feb 16, 2016 at 7:37 PM, Alexander Potapenko <glider@google.com> wrote:
>> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>>> >
>>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>>> >>
>>>> >> Hello,
>>>> >>
>>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>>> >>
>>>> >> Looks really nice!
>>>> >>
>>>> >> Could it be more generalized to be used by other feature that need to
>>>> >> store stack trace such as tracepoint or page owner?
>>>> > Certainly yes, but see below.
>>>> >
>>>> >> If it could be, there is one more requirement.
>>>> >> I understand the fact that entry is never removed from depot makes things
>>>> >> very simpler, but, for general usecases, it's better to use reference
>>>> >> count
>>>> >> and allow to remove. Is it possible?
>>>> > For our use case reference counting is not really necessary, and it would
>>>> > introduce unwanted contention.
>>>
>>> Okay.
>>>
>>>> > There are two possible options, each having its advantages and drawbacks: we
>>>> > can let the clients store the refcounters directly in their stacks (more
>>>> > universal, but harder to use for the clients), or keep the counters in the
>>>> > depot but add an API that does not change them (easier for the clients, but
>>>> > potentially error-prone).
>>>> > I'd say it's better to actually find at least one more user for the stack
>>>> > depot in order to understand the requirements, and refactor the code after
>>>> > that.
>>>
>>> I re-think the page owner case and it also may not need refcount.
>>> For now, just moving this stuff to /lib would be helpful for other future user.
>> I agree this code may need to be moved to /lib someday, but I wouldn't
>> hurry with that.
>> Right now it is quite KASAN-specific, and it's unclear yet whether
>> anyone else is going to use it.
>> I suggest we keep it in mm/kasan for now, and factor the common parts
>> into /lib when the need arises.
>>
>>> BTW, is there any performance number? I guess that it could affect
>>> the performance.
>> I've compared the performance of KASAN with SLAB allocator on a small
>> synthetic benchmark in two modes: with stack depot enabled and with
>> kasan_save_stack() unconditionally returning 0.
>> In the former case 8% more time was spent in the kernel than in the latter case.
>>
>> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
>> with the slub_debug=UZ boot options) take only 1.5 time, so the
>> difference is worth looking into (at least before we switch SLUB to
>> stack depot).
>
> I've made additional measurements.
> Previously I had been using a userspace benchmark that created and
> destroyed pipes in a loop
> (https://github.com/google/sanitizers/blob/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c).
>
> Now I've made a kernel module that allocated and deallocated memory
> chunks of different sizes in a loop.
> There were two modes of operation:
> 1) all the allocations were made from the same function, therefore all
> allocation/deallocation stacks were similar and there always was a hit
> in the stackdepot hashtable
> 2) The allocations were made from 2^16 different stacks.
>
> In the first case SLAB+stackdepot turned out to be 13% faster than
> SLUB+slub_debug, in the second SLAB was 11% faster.
I don't know what version of kernel you tested but, until recently,
slub_debug=UZ has a side effect not to using fastpath of SLUB. So,
comparison between them isn't appropriate. Today's linux-next branch
would have some improvements on this area so use it to compare them.
> Note that in both cases and for both allocators most of the time (more
> than 90%) was spent in the x86 stack unwinder, which is common for
> both approaches.
If more than 90% time is spent in stack unwinder which is common for
both cases, how something is better than the other by 13%?
> Yet another observation regarding stackdepot: under a heavy load
> (running Trinity for a hour, 101M allocations) the depot saturates at
> around 20K records with the hashtable miss rate of 0.02%.
> That said, I still cannot justify the results of the userspace
> benchmark, but the slowdown of the stackdepot approach for SLAB sounds
> acceptable, especially given the memory gain compared to SLUB
> bookkeeping (which requires 128 bytes per memory allocation) and the
> fact we'll be dealing with the fast path most of the time.
In fact, I don't have much concern about performance because saving
memory has enough merit to be merged. Anyway, it looks acceptable
even for performance.
> It will certainly be nice to compare SLUB+slub_debug to
> SLUB+stackdepot once we start switching SLUB to stackdepot.
Okay.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-02-18 8:13 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-18 8:13 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
2016-02-18 3:29 GMT+09:00 Alexander Potapenko <glider@google.com>:
> On Tue, Feb 16, 2016 at 7:37 PM, Alexander Potapenko <glider@google.com> wrote:
>> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>>> >
>>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>>> >>
>>>> >> Hello,
>>>> >>
>>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>>> >>
>>>> >> Looks really nice!
>>>> >>
>>>> >> Could it be more generalized to be used by other feature that need to
>>>> >> store stack trace such as tracepoint or page owner?
>>>> > Certainly yes, but see below.
>>>> >
>>>> >> If it could be, there is one more requirement.
>>>> >> I understand the fact that entry is never removed from depot makes things
>>>> >> very simpler, but, for general usecases, it's better to use reference
>>>> >> count
>>>> >> and allow to remove. Is it possible?
>>>> > For our use case reference counting is not really necessary, and it would
>>>> > introduce unwanted contention.
>>>
>>> Okay.
>>>
>>>> > There are two possible options, each having its advantages and drawbacks: we
>>>> > can let the clients store the refcounters directly in their stacks (more
>>>> > universal, but harder to use for the clients), or keep the counters in the
>>>> > depot but add an API that does not change them (easier for the clients, but
>>>> > potentially error-prone).
>>>> > I'd say it's better to actually find at least one more user for the stack
>>>> > depot in order to understand the requirements, and refactor the code after
>>>> > that.
>>>
>>> I re-think the page owner case and it also may not need refcount.
>>> For now, just moving this stuff to /lib would be helpful for other future user.
>> I agree this code may need to be moved to /lib someday, but I wouldn't
>> hurry with that.
>> Right now it is quite KASAN-specific, and it's unclear yet whether
>> anyone else is going to use it.
>> I suggest we keep it in mm/kasan for now, and factor the common parts
>> into /lib when the need arises.
>>
>>> BTW, is there any performance number? I guess that it could affect
>>> the performance.
>> I've compared the performance of KASAN with SLAB allocator on a small
>> synthetic benchmark in two modes: with stack depot enabled and with
>> kasan_save_stack() unconditionally returning 0.
>> In the former case 8% more time was spent in the kernel than in the latter case.
>>
>> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
>> with the slub_debug=UZ boot options) take only 1.5 time, so the
>> difference is worth looking into (at least before we switch SLUB to
>> stack depot).
>
> I've made additional measurements.
> Previously I had been using a userspace benchmark that created and
> destroyed pipes in a loop
> (https://github.com/google/sanitizers/blob/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c).
>
> Now I've made a kernel module that allocated and deallocated memory
> chunks of different sizes in a loop.
> There were two modes of operation:
> 1) all the allocations were made from the same function, therefore all
> allocation/deallocation stacks were similar and there always was a hit
> in the stackdepot hashtable
> 2) The allocations were made from 2^16 different stacks.
>
> In the first case SLAB+stackdepot turned out to be 13% faster than
> SLUB+slub_debug, in the second SLAB was 11% faster.
I don't know what version of kernel you tested but, until recently,
slub_debug=UZ has a side effect not to using fastpath of SLUB. So,
comparison between them isn't appropriate. Today's linux-next branch
would have some improvements on this area so use it to compare them.
> Note that in both cases and for both allocators most of the time (more
> than 90%) was spent in the x86 stack unwinder, which is common for
> both approaches.
If more than 90% time is spent in stack unwinder which is common for
both cases, how something is better than the other by 13%?
> Yet another observation regarding stackdepot: under a heavy load
> (running Trinity for a hour, 101M allocations) the depot saturates at
> around 20K records with the hashtable miss rate of 0.02%.
> That said, I still cannot justify the results of the userspace
> benchmark, but the slowdown of the stackdepot approach for SLAB sounds
> acceptable, especially given the memory gain compared to SLUB
> bookkeeping (which requires 128 bytes per memory allocation) and the
> fact we'll be dealing with the fast path most of the time.
In fact, I don't have much concern about performance because saving
memory has enough merit to be merged. Anyway, it looks acceptable
even for performance.
> It will certainly be nice to compare SLUB+slub_debug to
> SLUB+stackdepot once we start switching SLUB to stackdepot.
Okay.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-02-01 2:15 ` Joonsoo Kim
@ 2016-02-18 12:58 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-18 12:58 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
Steven Rostedt
However this info is meaningless without saved stack traces, which are
only introduced in the stackdepot patch (see "[PATCH v1 5/8] mm,
kasan: Stackdepot implementation. Enable stackdepot for SLAB")
On Mon, Feb 1, 2016 at 3:15 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
>> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>> >
>> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>> >>
>> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>> >> > This patch adds KASAN hooks to SLAB allocator.
>> >> >
>> >> > This patch is based on the "mm: kasan: unified support for SLUB and
>> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>> >> >
>> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
>> >> > ---
>> >> > Documentation/kasan.txt | 5 ++-
>> >>
>> >> ...
>> >>
>> >> > +#ifdef CONFIG_SLAB
>> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>> >> > + const void *object)
>> >> > +{
>> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>> >> > +}
>> >> > +
>> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>> >> > + const void *object)
>> >> > +{
>> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
>> >> > +}
>> >> > +#endif
>> >>
>> >> I cannot find the place to store stack info for free. get_free_info()
>> >> isn't used except print_object(). Plese let me know where.
>> >
>> > This is covered by other patches in this patchset.
>
> This should be covered by this patch. Stroing and printing free_info
> is already done on SLUB and it is meaningful without quarantain.
>
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-02-18 12:58 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-18 12:58 UTC (permalink / raw)
To: Joonsoo Kim
Cc: kasan-dev, Christoph Lameter, linux-kernel, Dmitriy Vyukov,
Andrey Ryabinin, linux-mm, Andrey Konovalov, Andrew Morton,
Steven Rostedt
However this info is meaningless without saved stack traces, which are
only introduced in the stackdepot patch (see "[PATCH v1 5/8] mm,
kasan: Stackdepot implementation. Enable stackdepot for SLAB")
On Mon, Feb 1, 2016 at 3:15 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
>> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>> >
>> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>> >>
>> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>> >> > This patch adds KASAN hooks to SLAB allocator.
>> >> >
>> >> > This patch is based on the "mm: kasan: unified support for SLUB and
>> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>> >> >
>> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
>> >> > ---
>> >> > Documentation/kasan.txt | 5 ++-
>> >>
>> >> ...
>> >>
>> >> > +#ifdef CONFIG_SLAB
>> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>> >> > + const void *object)
>> >> > +{
>> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>> >> > +}
>> >> > +
>> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>> >> > + const void *object)
>> >> > +{
>> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
>> >> > +}
>> >> > +#endif
>> >>
>> >> I cannot find the place to store stack info for free. get_free_info()
>> >> isn't used except print_object(). Plese let me know where.
>> >
>> > This is covered by other patches in this patchset.
>
> This should be covered by this patch. Stroing and printing free_info
> is already done on SLUB and it is meaningful without quarantain.
>
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-02-01 2:47 ` Joonsoo Kim
@ 2016-02-18 14:06 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-18 14:06 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev,
linux-kernel, linux-mm
On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
>> Quarantine isolates freed objects in a separate queue. The objects are
>> returned to the allocator later, which helps to detect use-after-free
>> errors.
>>
>> Freed objects are first added to per-cpu quarantine queues.
>> When a cache is destroyed or memory shrinking is requested, the objects
>> are moved into the global quarantine queue. Whenever a kmalloc call
>> allows memory reclaiming, the oldest objects are popped out of the
>> global queue until the total size of objects in quarantine is less than
>> 3/4 of the maximum quarantine size (which is a fraction of installed
>> physical memory).
>
> Just wondering why not using time based approach rather than size
> based one. In heavy load condition, how much time do the object stay in
> quarantine?
>
>>
>> Right now quarantine support is only enabled in SLAB allocator.
>> Unification of KASAN features in SLAB and SLUB will be done later.
>>
>> This patch is based on the "mm: kasan: quarantine" patch originally
>> prepared by Dmitry Chernenkov.
>>
>> Signed-off-by: Alexander Potapenko <glider@google.com>
>> ---
>> include/linux/kasan.h | 30 ++++--
>> lib/test_kasan.c | 29 ++++++
>> mm/kasan/Makefile | 2 +-
>> mm/kasan/kasan.c | 68 +++++++++++-
>> mm/kasan/kasan.h | 11 +-
>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
>> mm/kasan/report.c | 3 +-
>> mm/mempool.c | 7 +-
>> mm/page_alloc.c | 2 +-
>> mm/slab.c | 12 ++-
>> mm/slab.h | 4 +
>> mm/slab_common.c | 2 +
>> mm/slub.c | 4 +-
>> 13 files changed, 435 insertions(+), 23 deletions(-)
>>
>
> ...
>
>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
>> +{
>> +#ifdef CONFIG_SLAB
>> + /* RCU slabs could be legally used after free within the RCU period */
>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
>> + return false;
>> +
>> + if (likely(cache->flags & SLAB_KASAN)) {
>> + struct kasan_alloc_meta *alloc_info =
>> + get_alloc_info(cache, object);
>> + struct kasan_free_meta *free_info =
>> + get_free_info(cache, object);
>> +
>> + switch (alloc_info->state) {
>> + case KASAN_STATE_ALLOC:
>> + alloc_info->state = KASAN_STATE_QUARANTINE;
>> + quarantine_put(free_info, cache);
>
> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
> although it's not much meaningful without poisoning. But, I have an
> idea to poison object on SLAB_DESTROY_BY_RCU cache.
>
> quarantine_put() moves per cpu list to global queue when
> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
> at that time, after then, we can poison objects. With appropriate size
> setup, it would not be intrusive.
>
Won't this slow the quarantine down unpredictably (e.g. in the case
there're no RCU slabs in quarantine we'll still be waiting for
synchronize_rcu())?
Yet this is something worth looking into. Do you want RCU to be
handled in this patch set?
>> + set_track(&free_info->track, GFP_NOWAIT);
>
> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
Agreed, I can fix that if we decide to handle RCU in this patch
(otherwise it will lead to confusion).
>
>> + kasan_poison_slab_free(cache, object);
>> + return true;
>> + case KASAN_STATE_QUARANTINE:
>> + case KASAN_STATE_FREE:
>> + pr_err("Double free");
>> + dump_stack();
>> + break;
>> + default:
>> + break;
>> + }
>> + }
>> + return false;
>> +#else
>> + kasan_poison_slab_free(cache, object);
>> + return false;
>> +#endif
>> +}
>> +
>
> ...
>
>> +void quarantine_reduce(void)
>> +{
>> + size_t new_quarantine_size;
>> + unsigned long flags;
>> + struct qlist to_free = QLIST_INIT;
>> + size_t size_to_free = 0;
>> + void **last;
>> +
>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
>> + smp_load_acquire(&quarantine_size)))
>> + return;
>> +
>> + spin_lock_irqsave(&quarantine_lock, flags);
>> +
>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
>> + * the installed memory to quarantine minus per-cpu queue limits.
>> + */
>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
>> + QUARANTINE_FRACTION;
>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
>> + smp_store_release(&quarantine_size, new_quarantine_size);
>> +
>> + last = global_quarantine.head;
>> + while (last) {
>> + struct kmem_cache *cache = qlink_to_cache(last);
>> +
>> + size_to_free += cache->size;
>> + if (!*last || size_to_free >
>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
>> + break;
>> + last = (void **) *last;
>> + }
>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
>> +
>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>> +
>> + qlist_free_all(&to_free, NULL);
>> +}
>
> Isn't it better to call quarantine_reduce() in shrink_slab()?
> It will help to maximize quarantine time.
This is true, however if we don't call quarantine_reduce() from
kmalloc()/kfree() the size of the quarantine will be unpredictable.
There's a tradeoff between efficiency and space here, and at least in
some cases we may want to trade efficiency for space.
>
>> +
>> +static inline void qlist_move_cache(struct qlist *from,
>> + struct qlist *to,
>> + struct kmem_cache *cache)
>> +{
>> + void ***prev;
>> +
>> + if (unlikely(empty_qlist(from)))
>> + return;
>> +
>> + prev = &from->head;
>> + while (*prev) {
>> + void **qlink = *prev;
>> + struct kmem_cache *obj_cache = qlink_to_cache(qlink);
>> +
>> + if (obj_cache == cache) {
>> + if (unlikely(from->tail == qlink))
>> + from->tail = (void **) prev;
>> + *prev = (void **) *qlink;
>> + from->bytes -= cache->size;
>> + qlist_put(to, qlink, cache->size);
>> + } else
>> + prev = (void ***) *prev;
>> + }
>> +}
>> +
>> +static void per_cpu_remove_cache(void *arg)
>> +{
>> + struct kmem_cache *cache = arg;
>> + struct qlist to_free = QLIST_INIT;
>> + struct qlist *q;
>> + unsigned long flags;
>> +
>> + local_irq_save(flags);
>> + q = this_cpu_ptr(&cpu_quarantine);
>> + qlist_move_cache(q, &to_free, cache);
>> + local_irq_restore(flags);
>> +
>> + qlist_free_all(&to_free, cache);
>> +}
>> +
>> +void quarantine_remove_cache(struct kmem_cache *cache)
>> +{
>> + unsigned long flags;
>> + struct qlist to_free = QLIST_INIT;
>> +
>> + on_each_cpu(per_cpu_remove_cache, cache, 0);
>
> Should be called with wait = 1.
Agreed, thank you.
>> +
>> + spin_lock_irqsave(&quarantine_lock, flags);
>> + qlist_move_cache(&global_quarantine, &to_free, cache);
>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>> +
>> + qlist_free_all(&to_free, cache);
>> +}
>> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
>> index 6c4afcd..a4dca25 100644
>> --- a/mm/kasan/report.c
>> +++ b/mm/kasan/report.c
>> @@ -148,7 +148,8 @@ static void print_object(struct kmem_cache *cache, void *object)
>> print_track(&alloc_info->track);
>> break;
>> case KASAN_STATE_FREE:
>> - pr_err("Object freed, allocated with size %u bytes\n",
>> + case KASAN_STATE_QUARANTINE:
>> + pr_err("Object freed, allocated with size %lu bytes\n",
>> alloc_info->alloc_size);
>> free_info = get_free_info(cache, object);
>> pr_err("Allocation:\n");
>> diff --git a/mm/mempool.c b/mm/mempool.c
>> index b47c8a7..4beeeef 100644
>> --- a/mm/mempool.c
>> +++ b/mm/mempool.c
>> @@ -105,11 +105,12 @@ static inline void poison_element(mempool_t *pool, void *element)
>> static void kasan_poison_element(mempool_t *pool, void *element)
>> {
>> if (pool->alloc == mempool_alloc_slab)
>> - kasan_slab_free(pool->pool_data, element);
>> + kasan_poison_slab_free(pool->pool_data, element);
>> if (pool->alloc == mempool_kmalloc)
>> - kasan_kfree(element);
>> + kasan_poison_kfree(element);
>> if (pool->alloc == mempool_alloc_pages)
>> - kasan_free_pages(element, (unsigned long)pool->pool_data);
>> + kasan_poison_free_pages(element,
>> + (unsigned long)pool->pool_data);
>> }
>>
>> static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 63358d9..4f65587 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -980,7 +980,7 @@ static bool free_pages_prepare(struct page *page, unsigned int order)
>>
>> trace_mm_page_free(page, order);
>> kmemcheck_free_shadow(page, order);
>> - kasan_free_pages(page, order);
>> + kasan_poison_free_pages(page, order);
>>
>> if (PageAnon(page))
>> page->mapping = NULL;
>> diff --git a/mm/slab.c b/mm/slab.c
>> index 0ec7aa3..e2fac67 100644
>> --- a/mm/slab.c
>> +++ b/mm/slab.c
>> @@ -3374,9 +3374,19 @@ free_done:
>> static inline void __cache_free(struct kmem_cache *cachep, void *objp,
>> unsigned long caller)
>> {
>> +#ifdef CONFIG_KASAN
>> + if (!kasan_slab_free(cachep, objp))
>> + /* The object has been put into the quarantine, don't touch it
>> + * for now.
>> + */
>> + nokasan_free(cachep, objp, caller);
>> +}
>> +
>> +void nokasan_free(struct kmem_cache *cachep, void *objp, unsigned long caller)
>> +{
>> +#endif
>
> It looks not good to me.
> Converting __cache_free() to ____cache_free() and making
> __cache_free() call ____cache_free() if (!kasan_slab_free()) looks
> better to me and less error-prone.
Fixed. Will upload the new patchset soonish.
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-02-18 14:06 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-18 14:06 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev,
linux-kernel, linux-mm
On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
>> Quarantine isolates freed objects in a separate queue. The objects are
>> returned to the allocator later, which helps to detect use-after-free
>> errors.
>>
>> Freed objects are first added to per-cpu quarantine queues.
>> When a cache is destroyed or memory shrinking is requested, the objects
>> are moved into the global quarantine queue. Whenever a kmalloc call
>> allows memory reclaiming, the oldest objects are popped out of the
>> global queue until the total size of objects in quarantine is less than
>> 3/4 of the maximum quarantine size (which is a fraction of installed
>> physical memory).
>
> Just wondering why not using time based approach rather than size
> based one. In heavy load condition, how much time do the object stay in
> quarantine?
>
>>
>> Right now quarantine support is only enabled in SLAB allocator.
>> Unification of KASAN features in SLAB and SLUB will be done later.
>>
>> This patch is based on the "mm: kasan: quarantine" patch originally
>> prepared by Dmitry Chernenkov.
>>
>> Signed-off-by: Alexander Potapenko <glider@google.com>
>> ---
>> include/linux/kasan.h | 30 ++++--
>> lib/test_kasan.c | 29 ++++++
>> mm/kasan/Makefile | 2 +-
>> mm/kasan/kasan.c | 68 +++++++++++-
>> mm/kasan/kasan.h | 11 +-
>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
>> mm/kasan/report.c | 3 +-
>> mm/mempool.c | 7 +-
>> mm/page_alloc.c | 2 +-
>> mm/slab.c | 12 ++-
>> mm/slab.h | 4 +
>> mm/slab_common.c | 2 +
>> mm/slub.c | 4 +-
>> 13 files changed, 435 insertions(+), 23 deletions(-)
>>
>
> ...
>
>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
>> +{
>> +#ifdef CONFIG_SLAB
>> + /* RCU slabs could be legally used after free within the RCU period */
>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
>> + return false;
>> +
>> + if (likely(cache->flags & SLAB_KASAN)) {
>> + struct kasan_alloc_meta *alloc_info =
>> + get_alloc_info(cache, object);
>> + struct kasan_free_meta *free_info =
>> + get_free_info(cache, object);
>> +
>> + switch (alloc_info->state) {
>> + case KASAN_STATE_ALLOC:
>> + alloc_info->state = KASAN_STATE_QUARANTINE;
>> + quarantine_put(free_info, cache);
>
> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
> although it's not much meaningful without poisoning. But, I have an
> idea to poison object on SLAB_DESTROY_BY_RCU cache.
>
> quarantine_put() moves per cpu list to global queue when
> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
> at that time, after then, we can poison objects. With appropriate size
> setup, it would not be intrusive.
>
Won't this slow the quarantine down unpredictably (e.g. in the case
there're no RCU slabs in quarantine we'll still be waiting for
synchronize_rcu())?
Yet this is something worth looking into. Do you want RCU to be
handled in this patch set?
>> + set_track(&free_info->track, GFP_NOWAIT);
>
> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
Agreed, I can fix that if we decide to handle RCU in this patch
(otherwise it will lead to confusion).
>
>> + kasan_poison_slab_free(cache, object);
>> + return true;
>> + case KASAN_STATE_QUARANTINE:
>> + case KASAN_STATE_FREE:
>> + pr_err("Double free");
>> + dump_stack();
>> + break;
>> + default:
>> + break;
>> + }
>> + }
>> + return false;
>> +#else
>> + kasan_poison_slab_free(cache, object);
>> + return false;
>> +#endif
>> +}
>> +
>
> ...
>
>> +void quarantine_reduce(void)
>> +{
>> + size_t new_quarantine_size;
>> + unsigned long flags;
>> + struct qlist to_free = QLIST_INIT;
>> + size_t size_to_free = 0;
>> + void **last;
>> +
>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
>> + smp_load_acquire(&quarantine_size)))
>> + return;
>> +
>> + spin_lock_irqsave(&quarantine_lock, flags);
>> +
>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
>> + * the installed memory to quarantine minus per-cpu queue limits.
>> + */
>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
>> + QUARANTINE_FRACTION;
>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
>> + smp_store_release(&quarantine_size, new_quarantine_size);
>> +
>> + last = global_quarantine.head;
>> + while (last) {
>> + struct kmem_cache *cache = qlink_to_cache(last);
>> +
>> + size_to_free += cache->size;
>> + if (!*last || size_to_free >
>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
>> + break;
>> + last = (void **) *last;
>> + }
>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
>> +
>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>> +
>> + qlist_free_all(&to_free, NULL);
>> +}
>
> Isn't it better to call quarantine_reduce() in shrink_slab()?
> It will help to maximize quarantine time.
This is true, however if we don't call quarantine_reduce() from
kmalloc()/kfree() the size of the quarantine will be unpredictable.
There's a tradeoff between efficiency and space here, and at least in
some cases we may want to trade efficiency for space.
>
>> +
>> +static inline void qlist_move_cache(struct qlist *from,
>> + struct qlist *to,
>> + struct kmem_cache *cache)
>> +{
>> + void ***prev;
>> +
>> + if (unlikely(empty_qlist(from)))
>> + return;
>> +
>> + prev = &from->head;
>> + while (*prev) {
>> + void **qlink = *prev;
>> + struct kmem_cache *obj_cache = qlink_to_cache(qlink);
>> +
>> + if (obj_cache == cache) {
>> + if (unlikely(from->tail == qlink))
>> + from->tail = (void **) prev;
>> + *prev = (void **) *qlink;
>> + from->bytes -= cache->size;
>> + qlist_put(to, qlink, cache->size);
>> + } else
>> + prev = (void ***) *prev;
>> + }
>> +}
>> +
>> +static void per_cpu_remove_cache(void *arg)
>> +{
>> + struct kmem_cache *cache = arg;
>> + struct qlist to_free = QLIST_INIT;
>> + struct qlist *q;
>> + unsigned long flags;
>> +
>> + local_irq_save(flags);
>> + q = this_cpu_ptr(&cpu_quarantine);
>> + qlist_move_cache(q, &to_free, cache);
>> + local_irq_restore(flags);
>> +
>> + qlist_free_all(&to_free, cache);
>> +}
>> +
>> +void quarantine_remove_cache(struct kmem_cache *cache)
>> +{
>> + unsigned long flags;
>> + struct qlist to_free = QLIST_INIT;
>> +
>> + on_each_cpu(per_cpu_remove_cache, cache, 0);
>
> Should be called with wait = 1.
Agreed, thank you.
>> +
>> + spin_lock_irqsave(&quarantine_lock, flags);
>> + qlist_move_cache(&global_quarantine, &to_free, cache);
>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>> +
>> + qlist_free_all(&to_free, cache);
>> +}
>> diff --git a/mm/kasan/report.c b/mm/kasan/report.c
>> index 6c4afcd..a4dca25 100644
>> --- a/mm/kasan/report.c
>> +++ b/mm/kasan/report.c
>> @@ -148,7 +148,8 @@ static void print_object(struct kmem_cache *cache, void *object)
>> print_track(&alloc_info->track);
>> break;
>> case KASAN_STATE_FREE:
>> - pr_err("Object freed, allocated with size %u bytes\n",
>> + case KASAN_STATE_QUARANTINE:
>> + pr_err("Object freed, allocated with size %lu bytes\n",
>> alloc_info->alloc_size);
>> free_info = get_free_info(cache, object);
>> pr_err("Allocation:\n");
>> diff --git a/mm/mempool.c b/mm/mempool.c
>> index b47c8a7..4beeeef 100644
>> --- a/mm/mempool.c
>> +++ b/mm/mempool.c
>> @@ -105,11 +105,12 @@ static inline void poison_element(mempool_t *pool, void *element)
>> static void kasan_poison_element(mempool_t *pool, void *element)
>> {
>> if (pool->alloc == mempool_alloc_slab)
>> - kasan_slab_free(pool->pool_data, element);
>> + kasan_poison_slab_free(pool->pool_data, element);
>> if (pool->alloc == mempool_kmalloc)
>> - kasan_kfree(element);
>> + kasan_poison_kfree(element);
>> if (pool->alloc == mempool_alloc_pages)
>> - kasan_free_pages(element, (unsigned long)pool->pool_data);
>> + kasan_poison_free_pages(element,
>> + (unsigned long)pool->pool_data);
>> }
>>
>> static void kasan_unpoison_element(mempool_t *pool, void *element, gfp_t flags)
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 63358d9..4f65587 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -980,7 +980,7 @@ static bool free_pages_prepare(struct page *page, unsigned int order)
>>
>> trace_mm_page_free(page, order);
>> kmemcheck_free_shadow(page, order);
>> - kasan_free_pages(page, order);
>> + kasan_poison_free_pages(page, order);
>>
>> if (PageAnon(page))
>> page->mapping = NULL;
>> diff --git a/mm/slab.c b/mm/slab.c
>> index 0ec7aa3..e2fac67 100644
>> --- a/mm/slab.c
>> +++ b/mm/slab.c
>> @@ -3374,9 +3374,19 @@ free_done:
>> static inline void __cache_free(struct kmem_cache *cachep, void *objp,
>> unsigned long caller)
>> {
>> +#ifdef CONFIG_KASAN
>> + if (!kasan_slab_free(cachep, objp))
>> + /* The object has been put into the quarantine, don't touch it
>> + * for now.
>> + */
>> + nokasan_free(cachep, objp, caller);
>> +}
>> +
>> +void nokasan_free(struct kmem_cache *cachep, void *objp, unsigned long caller)
>> +{
>> +#endif
>
> It looks not good to me.
> Converting __cache_free() to ____cache_free() and making
> __cache_free() call ____cache_free() if (!kasan_slab_free()) looks
> better to me and less error-prone.
Fixed. Will upload the new patchset soonish.
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
2016-02-18 8:13 ` Joonsoo Kim
@ 2016-02-18 15:01 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-18 15:01 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
On Thu, Feb 18, 2016 at 9:13 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> 2016-02-18 3:29 GMT+09:00 Alexander Potapenko <glider@google.com>:
>> On Tue, Feb 16, 2016 at 7:37 PM, Alexander Potapenko <glider@google.com> wrote:
>>> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>>>> >
>>>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>>>> >>
>>>>> >> Hello,
>>>>> >>
>>>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>>>> >>
>>>>> >> Looks really nice!
>>>>> >>
>>>>> >> Could it be more generalized to be used by other feature that need to
>>>>> >> store stack trace such as tracepoint or page owner?
>>>>> > Certainly yes, but see below.
>>>>> >
>>>>> >> If it could be, there is one more requirement.
>>>>> >> I understand the fact that entry is never removed from depot makes things
>>>>> >> very simpler, but, for general usecases, it's better to use reference
>>>>> >> count
>>>>> >> and allow to remove. Is it possible?
>>>>> > For our use case reference counting is not really necessary, and it would
>>>>> > introduce unwanted contention.
>>>>
>>>> Okay.
>>>>
>>>>> > There are two possible options, each having its advantages and drawbacks: we
>>>>> > can let the clients store the refcounters directly in their stacks (more
>>>>> > universal, but harder to use for the clients), or keep the counters in the
>>>>> > depot but add an API that does not change them (easier for the clients, but
>>>>> > potentially error-prone).
>>>>> > I'd say it's better to actually find at least one more user for the stack
>>>>> > depot in order to understand the requirements, and refactor the code after
>>>>> > that.
>>>>
>>>> I re-think the page owner case and it also may not need refcount.
>>>> For now, just moving this stuff to /lib would be helpful for other future user.
>>> I agree this code may need to be moved to /lib someday, but I wouldn't
>>> hurry with that.
>>> Right now it is quite KASAN-specific, and it's unclear yet whether
>>> anyone else is going to use it.
>>> I suggest we keep it in mm/kasan for now, and factor the common parts
>>> into /lib when the need arises.
>>>
>>>> BTW, is there any performance number? I guess that it could affect
>>>> the performance.
>>> I've compared the performance of KASAN with SLAB allocator on a small
>>> synthetic benchmark in two modes: with stack depot enabled and with
>>> kasan_save_stack() unconditionally returning 0.
>>> In the former case 8% more time was spent in the kernel than in the latter case.
>>>
>>> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
>>> with the slub_debug=UZ boot options) take only 1.5 time, so the
>>> difference is worth looking into (at least before we switch SLUB to
>>> stack depot).
>>
>> I've made additional measurements.
>> Previously I had been using a userspace benchmark that created and
>> destroyed pipes in a loop
>> (https://github.com/google/sanitizers/blob/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c).
>>
>> Now I've made a kernel module that allocated and deallocated memory
>> chunks of different sizes in a loop.
>> There were two modes of operation:
>> 1) all the allocations were made from the same function, therefore all
>> allocation/deallocation stacks were similar and there always was a hit
>> in the stackdepot hashtable
>> 2) The allocations were made from 2^16 different stacks.
>>
>> In the first case SLAB+stackdepot turned out to be 13% faster than
>> SLUB+slub_debug, in the second SLAB was 11% faster.
>
> I don't know what version of kernel you tested but, until recently,
> slub_debug=UZ has a side effect not to using fastpath of SLUB. So,
> comparison between them isn't appropriate. Today's linux-next branch
> would have some improvements on this area so use it to compare them.
>
That's good to know.
I've been using https://github.com/torvalds/linux.git, which probably
didn't have those improvements.
>> Note that in both cases and for both allocators most of the time (more
>> than 90%) was spent in the x86 stack unwinder, which is common for
>> both approaches.
>
> If more than 90% time is spent in stack unwinder which is common for
> both cases, how something is better than the other by 13%?
On the second glance, this number (90%) may be inaccurate, because I
measured the stack unwinding times separately, which could have
introduced deviation (not to mention it was incorrect for SLUB).
Yet we're talking about a significant amount of time spent in the unwinder.
My numbers were 26.111 seconds for 1024K SLAB allocation/deallocation
pairs and 30.278 seconds for 1024K alloc/dealloc pairs with SLUB.
When measured separately in the same routine that did the allocations,
2048K calls to save_stack_trace() took 25.487 seconds.
>> Yet another observation regarding stackdepot: under a heavy load
>> (running Trinity for a hour, 101M allocations) the depot saturates at
>> around 20K records with the hashtable miss rate of 0.02%.
>> That said, I still cannot justify the results of the userspace
>> benchmark, but the slowdown of the stackdepot approach for SLAB sounds
>> acceptable, especially given the memory gain compared to SLUB
>> bookkeeping (which requires 128 bytes per memory allocation) and the
>> fact we'll be dealing with the fast path most of the time.
>
> In fact, I don't have much concern about performance because saving
> memory has enough merit to be merged. Anyway, it looks acceptable
> even for performance.
>
>> It will certainly be nice to compare SLUB+slub_debug to
>> SLUB+stackdepot once we start switching SLUB to stackdepot.
>
> Okay.
>
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB
@ 2016-02-18 15:01 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-18 15:01 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
On Thu, Feb 18, 2016 at 9:13 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> 2016-02-18 3:29 GMT+09:00 Alexander Potapenko <glider@google.com>:
>> On Tue, Feb 16, 2016 at 7:37 PM, Alexander Potapenko <glider@google.com> wrote:
>>> On Mon, Feb 1, 2016 at 3:55 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>>> On Thu, Jan 28, 2016 at 02:27:44PM +0100, Alexander Potapenko wrote:
>>>>> On Thu, Jan 28, 2016 at 1:51 PM, Alexander Potapenko <glider@google.com> wrote:
>>>>> >
>>>>> > On Jan 28, 2016 8:40 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>>>> >>
>>>>> >> Hello,
>>>>> >>
>>>>> >> On Wed, Jan 27, 2016 at 07:25:10PM +0100, Alexander Potapenko wrote:
>>>>> >> > Stack depot will allow KASAN store allocation/deallocation stack traces
>>>>> >> > for memory chunks. The stack traces are stored in a hash table and
>>>>> >> > referenced by handles which reside in the kasan_alloc_meta and
>>>>> >> > kasan_free_meta structures in the allocated memory chunks.
>>>>> >>
>>>>> >> Looks really nice!
>>>>> >>
>>>>> >> Could it be more generalized to be used by other feature that need to
>>>>> >> store stack trace such as tracepoint or page owner?
>>>>> > Certainly yes, but see below.
>>>>> >
>>>>> >> If it could be, there is one more requirement.
>>>>> >> I understand the fact that entry is never removed from depot makes things
>>>>> >> very simpler, but, for general usecases, it's better to use reference
>>>>> >> count
>>>>> >> and allow to remove. Is it possible?
>>>>> > For our use case reference counting is not really necessary, and it would
>>>>> > introduce unwanted contention.
>>>>
>>>> Okay.
>>>>
>>>>> > There are two possible options, each having its advantages and drawbacks: we
>>>>> > can let the clients store the refcounters directly in their stacks (more
>>>>> > universal, but harder to use for the clients), or keep the counters in the
>>>>> > depot but add an API that does not change them (easier for the clients, but
>>>>> > potentially error-prone).
>>>>> > I'd say it's better to actually find at least one more user for the stack
>>>>> > depot in order to understand the requirements, and refactor the code after
>>>>> > that.
>>>>
>>>> I re-think the page owner case and it also may not need refcount.
>>>> For now, just moving this stuff to /lib would be helpful for other future user.
>>> I agree this code may need to be moved to /lib someday, but I wouldn't
>>> hurry with that.
>>> Right now it is quite KASAN-specific, and it's unclear yet whether
>>> anyone else is going to use it.
>>> I suggest we keep it in mm/kasan for now, and factor the common parts
>>> into /lib when the need arises.
>>>
>>>> BTW, is there any performance number? I guess that it could affect
>>>> the performance.
>>> I've compared the performance of KASAN with SLAB allocator on a small
>>> synthetic benchmark in two modes: with stack depot enabled and with
>>> kasan_save_stack() unconditionally returning 0.
>>> In the former case 8% more time was spent in the kernel than in the latter case.
>>>
>>> If I am not mistaking, for SLUB allocator the bookkeeping (enabled
>>> with the slub_debug=UZ boot options) take only 1.5 time, so the
>>> difference is worth looking into (at least before we switch SLUB to
>>> stack depot).
>>
>> I've made additional measurements.
>> Previously I had been using a userspace benchmark that created and
>> destroyed pipes in a loop
>> (https://github.com/google/sanitizers/blob/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c).
>>
>> Now I've made a kernel module that allocated and deallocated memory
>> chunks of different sizes in a loop.
>> There were two modes of operation:
>> 1) all the allocations were made from the same function, therefore all
>> allocation/deallocation stacks were similar and there always was a hit
>> in the stackdepot hashtable
>> 2) The allocations were made from 2^16 different stacks.
>>
>> In the first case SLAB+stackdepot turned out to be 13% faster than
>> SLUB+slub_debug, in the second SLAB was 11% faster.
>
> I don't know what version of kernel you tested but, until recently,
> slub_debug=UZ has a side effect not to using fastpath of SLUB. So,
> comparison between them isn't appropriate. Today's linux-next branch
> would have some improvements on this area so use it to compare them.
>
That's good to know.
I've been using https://github.com/torvalds/linux.git, which probably
didn't have those improvements.
>> Note that in both cases and for both allocators most of the time (more
>> than 90%) was spent in the x86 stack unwinder, which is common for
>> both approaches.
>
> If more than 90% time is spent in stack unwinder which is common for
> both cases, how something is better than the other by 13%?
On the second glance, this number (90%) may be inaccurate, because I
measured the stack unwinding times separately, which could have
introduced deviation (not to mention it was incorrect for SLUB).
Yet we're talking about a significant amount of time spent in the unwinder.
My numbers were 26.111 seconds for 1024K SLAB allocation/deallocation
pairs and 30.278 seconds for 1024K alloc/dealloc pairs with SLUB.
When measured separately in the same routine that did the allocations,
2048K calls to save_stack_trace() took 25.487 seconds.
>> Yet another observation regarding stackdepot: under a heavy load
>> (running Trinity for a hour, 101M allocations) the depot saturates at
>> around 20K records with the hashtable miss rate of 0.02%.
>> That said, I still cannot justify the results of the userspace
>> benchmark, but the slowdown of the stackdepot approach for SLAB sounds
>> acceptable, especially given the memory gain compared to SLUB
>> bookkeeping (which requires 128 bytes per memory allocation) and the
>> fact we'll be dealing with the fast path most of the time.
>
> In fact, I don't have much concern about performance because saving
> memory has enough merit to be merged. Anyway, it looks acceptable
> even for performance.
>
>> It will certainly be nice to compare SLUB+slub_debug to
>> SLUB+stackdepot once we start switching SLUB to stackdepot.
>
> Okay.
>
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-02-18 12:58 ` Alexander Potapenko
@ 2016-02-19 1:41 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-19 1:41 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
> On Mon, Feb 1, 2016 at 3:15 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
>>> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>>> >
>>> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>> >>
>>> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>>> >> > This patch adds KASAN hooks to SLAB allocator.
>>> >> >
>>> >> > This patch is based on the "mm: kasan: unified support for SLUB and
>>> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>>> >> >
>>> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
>>> >> > ---
>>> >> > Documentation/kasan.txt | 5 ++-
>>> >>
>>> >> ...
>>> >>
>>> >> > +#ifdef CONFIG_SLAB
>>> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>>> >> > + const void *object)
>>> >> > +{
>>> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>>> >> > +}
>>> >> > +
>>> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>>> >> > + const void *object)
>>> >> > +{
>>> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
>>> >> > +}
>>> >> > +#endif
>>> >>
>>> >> I cannot find the place to store stack info for free. get_free_info()
>>> >> isn't used except print_object(). Plese let me know where.
>>> >
>>> > This is covered by other patches in this patchset.
>>
>> This should be covered by this patch. Stroing and printing free_info
>> is already done on SLUB and it is meaningful without quarantain.
2016-02-18 21:58 GMT+09:00 Alexander Potapenko <glider@google.com>:
> However this info is meaningless without saved stack traces, which are
> only introduced in the stackdepot patch (see "[PATCH v1 5/8] mm,
> kasan: Stackdepot implementation. Enable stackdepot for SLAB")
Not meaningless. You already did it for allocation caller without saved
stack traces. What makes difference between alloc/free?
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-02-19 1:41 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-19 1:41 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
> On Mon, Feb 1, 2016 at 3:15 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
>>> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>>> >
>>> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>> >>
>>> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>>> >> > This patch adds KASAN hooks to SLAB allocator.
>>> >> >
>>> >> > This patch is based on the "mm: kasan: unified support for SLUB and
>>> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>>> >> >
>>> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
>>> >> > ---
>>> >> > Documentation/kasan.txt | 5 ++-
>>> >>
>>> >> ...
>>> >>
>>> >> > +#ifdef CONFIG_SLAB
>>> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>>> >> > + const void *object)
>>> >> > +{
>>> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>>> >> > +}
>>> >> > +
>>> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>>> >> > + const void *object)
>>> >> > +{
>>> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
>>> >> > +}
>>> >> > +#endif
>>> >>
>>> >> I cannot find the place to store stack info for free. get_free_info()
>>> >> isn't used except print_object(). Plese let me know where.
>>> >
>>> > This is covered by other patches in this patchset.
>>
>> This should be covered by this patch. Stroing and printing free_info
>> is already done on SLUB and it is meaningful without quarantain.
2016-02-18 21:58 GMT+09:00 Alexander Potapenko <glider@google.com>:
> However this info is meaningless without saved stack traces, which are
> only introduced in the stackdepot patch (see "[PATCH v1 5/8] mm,
> kasan: Stackdepot implementation. Enable stackdepot for SLAB")
Not meaningless. You already did it for allocation caller without saved
stack traces. What makes difference between alloc/free?
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-02-18 14:06 ` Alexander Potapenko
@ 2016-02-19 2:11 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-19 2:11 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev, LKML,
Linux Memory Management List
2016-02-18 23:06 GMT+09:00 Alexander Potapenko <glider@google.com>:
> On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
>>> Quarantine isolates freed objects in a separate queue. The objects are
>>> returned to the allocator later, which helps to detect use-after-free
>>> errors.
>>>
>>> Freed objects are first added to per-cpu quarantine queues.
>>> When a cache is destroyed or memory shrinking is requested, the objects
>>> are moved into the global quarantine queue. Whenever a kmalloc call
>>> allows memory reclaiming, the oldest objects are popped out of the
>>> global queue until the total size of objects in quarantine is less than
>>> 3/4 of the maximum quarantine size (which is a fraction of installed
>>> physical memory).
>>
>> Just wondering why not using time based approach rather than size
>> based one. In heavy load condition, how much time do the object stay in
>> quarantine?
>>
>>>
>>> Right now quarantine support is only enabled in SLAB allocator.
>>> Unification of KASAN features in SLAB and SLUB will be done later.
>>>
>>> This patch is based on the "mm: kasan: quarantine" patch originally
>>> prepared by Dmitry Chernenkov.
>>>
>>> Signed-off-by: Alexander Potapenko <glider@google.com>
>>> ---
>>> include/linux/kasan.h | 30 ++++--
>>> lib/test_kasan.c | 29 ++++++
>>> mm/kasan/Makefile | 2 +-
>>> mm/kasan/kasan.c | 68 +++++++++++-
>>> mm/kasan/kasan.h | 11 +-
>>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>> mm/kasan/report.c | 3 +-
>>> mm/mempool.c | 7 +-
>>> mm/page_alloc.c | 2 +-
>>> mm/slab.c | 12 ++-
>>> mm/slab.h | 4 +
>>> mm/slab_common.c | 2 +
>>> mm/slub.c | 4 +-
>>> 13 files changed, 435 insertions(+), 23 deletions(-)
>>>
>>
>> ...
>>
>>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
>>> +{
>>> +#ifdef CONFIG_SLAB
>>> + /* RCU slabs could be legally used after free within the RCU period */
>>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
>>> + return false;
>>> +
>>> + if (likely(cache->flags & SLAB_KASAN)) {
>>> + struct kasan_alloc_meta *alloc_info =
>>> + get_alloc_info(cache, object);
>>> + struct kasan_free_meta *free_info =
>>> + get_free_info(cache, object);
>>> +
>>> + switch (alloc_info->state) {
>>> + case KASAN_STATE_ALLOC:
>>> + alloc_info->state = KASAN_STATE_QUARANTINE;
>>> + quarantine_put(free_info, cache);
>>
>> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
>> although it's not much meaningful without poisoning. But, I have an
>> idea to poison object on SLAB_DESTROY_BY_RCU cache.
>>
>> quarantine_put() moves per cpu list to global queue when
>> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
>> at that time, after then, we can poison objects. With appropriate size
>> setup, it would not be intrusive.
>>
> Won't this slow the quarantine down unpredictably (e.g. in the case
> there're no RCU slabs in quarantine we'll still be waiting for
> synchronize_rcu())?
It could be handled by introducing one cpu variable.
> Yet this is something worth looking into. Do you want RCU to be
> handled in this patch set?
No. It would be future work.
>>> + set_track(&free_info->track, GFP_NOWAIT);
>>
>> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
> Agreed, I can fix that if we decide to handle RCU in this patch
> (otherwise it will lead to confusion).
>
>>
>>> + kasan_poison_slab_free(cache, object);
>>> + return true;
>>> + case KASAN_STATE_QUARANTINE:
>>> + case KASAN_STATE_FREE:
>>> + pr_err("Double free");
>>> + dump_stack();
>>> + break;
>>> + default:
>>> + break;
>>> + }
>>> + }
>>> + return false;
>>> +#else
>>> + kasan_poison_slab_free(cache, object);
>>> + return false;
>>> +#endif
>>> +}
>>> +
>>
>> ...
>>
>>> +void quarantine_reduce(void)
>>> +{
>>> + size_t new_quarantine_size;
>>> + unsigned long flags;
>>> + struct qlist to_free = QLIST_INIT;
>>> + size_t size_to_free = 0;
>>> + void **last;
>>> +
>>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
>>> + smp_load_acquire(&quarantine_size)))
>>> + return;
>>> +
>>> + spin_lock_irqsave(&quarantine_lock, flags);
>>> +
>>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
>>> + * the installed memory to quarantine minus per-cpu queue limits.
>>> + */
>>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
>>> + QUARANTINE_FRACTION;
>>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
>>> + smp_store_release(&quarantine_size, new_quarantine_size);
>>> +
>>> + last = global_quarantine.head;
>>> + while (last) {
>>> + struct kmem_cache *cache = qlink_to_cache(last);
>>> +
>>> + size_to_free += cache->size;
>>> + if (!*last || size_to_free >
>>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
>>> + break;
>>> + last = (void **) *last;
>>> + }
>>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
>>> +
>>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>>> +
>>> + qlist_free_all(&to_free, NULL);
>>> +}
>>
>> Isn't it better to call quarantine_reduce() in shrink_slab()?
>> It will help to maximize quarantine time.
> This is true, however if we don't call quarantine_reduce() from
> kmalloc()/kfree() the size of the quarantine will be unpredictable.
> There's a tradeoff between efficiency and space here, and at least in
> some cases we may want to trade efficiency for space.
size of the quarantine doesn't matter unless there is memory pressure.
If memory pressure, shrink_slab() would be called and we can reduce
size of quarantine. However, I don't think this is show stopper. We can
do it when needed.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-02-19 2:11 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-19 2:11 UTC (permalink / raw)
To: Alexander Potapenko
Cc: Joonsoo Kim, Andrey Konovalov, Christoph Lameter, Dmitriy Vyukov,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev, LKML,
Linux Memory Management List
2016-02-18 23:06 GMT+09:00 Alexander Potapenko <glider@google.com>:
> On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
>>> Quarantine isolates freed objects in a separate queue. The objects are
>>> returned to the allocator later, which helps to detect use-after-free
>>> errors.
>>>
>>> Freed objects are first added to per-cpu quarantine queues.
>>> When a cache is destroyed or memory shrinking is requested, the objects
>>> are moved into the global quarantine queue. Whenever a kmalloc call
>>> allows memory reclaiming, the oldest objects are popped out of the
>>> global queue until the total size of objects in quarantine is less than
>>> 3/4 of the maximum quarantine size (which is a fraction of installed
>>> physical memory).
>>
>> Just wondering why not using time based approach rather than size
>> based one. In heavy load condition, how much time do the object stay in
>> quarantine?
>>
>>>
>>> Right now quarantine support is only enabled in SLAB allocator.
>>> Unification of KASAN features in SLAB and SLUB will be done later.
>>>
>>> This patch is based on the "mm: kasan: quarantine" patch originally
>>> prepared by Dmitry Chernenkov.
>>>
>>> Signed-off-by: Alexander Potapenko <glider@google.com>
>>> ---
>>> include/linux/kasan.h | 30 ++++--
>>> lib/test_kasan.c | 29 ++++++
>>> mm/kasan/Makefile | 2 +-
>>> mm/kasan/kasan.c | 68 +++++++++++-
>>> mm/kasan/kasan.h | 11 +-
>>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>> mm/kasan/report.c | 3 +-
>>> mm/mempool.c | 7 +-
>>> mm/page_alloc.c | 2 +-
>>> mm/slab.c | 12 ++-
>>> mm/slab.h | 4 +
>>> mm/slab_common.c | 2 +
>>> mm/slub.c | 4 +-
>>> 13 files changed, 435 insertions(+), 23 deletions(-)
>>>
>>
>> ...
>>
>>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
>>> +{
>>> +#ifdef CONFIG_SLAB
>>> + /* RCU slabs could be legally used after free within the RCU period */
>>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
>>> + return false;
>>> +
>>> + if (likely(cache->flags & SLAB_KASAN)) {
>>> + struct kasan_alloc_meta *alloc_info =
>>> + get_alloc_info(cache, object);
>>> + struct kasan_free_meta *free_info =
>>> + get_free_info(cache, object);
>>> +
>>> + switch (alloc_info->state) {
>>> + case KASAN_STATE_ALLOC:
>>> + alloc_info->state = KASAN_STATE_QUARANTINE;
>>> + quarantine_put(free_info, cache);
>>
>> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
>> although it's not much meaningful without poisoning. But, I have an
>> idea to poison object on SLAB_DESTROY_BY_RCU cache.
>>
>> quarantine_put() moves per cpu list to global queue when
>> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
>> at that time, after then, we can poison objects. With appropriate size
>> setup, it would not be intrusive.
>>
> Won't this slow the quarantine down unpredictably (e.g. in the case
> there're no RCU slabs in quarantine we'll still be waiting for
> synchronize_rcu())?
It could be handled by introducing one cpu variable.
> Yet this is something worth looking into. Do you want RCU to be
> handled in this patch set?
No. It would be future work.
>>> + set_track(&free_info->track, GFP_NOWAIT);
>>
>> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
> Agreed, I can fix that if we decide to handle RCU in this patch
> (otherwise it will lead to confusion).
>
>>
>>> + kasan_poison_slab_free(cache, object);
>>> + return true;
>>> + case KASAN_STATE_QUARANTINE:
>>> + case KASAN_STATE_FREE:
>>> + pr_err("Double free");
>>> + dump_stack();
>>> + break;
>>> + default:
>>> + break;
>>> + }
>>> + }
>>> + return false;
>>> +#else
>>> + kasan_poison_slab_free(cache, object);
>>> + return false;
>>> +#endif
>>> +}
>>> +
>>
>> ...
>>
>>> +void quarantine_reduce(void)
>>> +{
>>> + size_t new_quarantine_size;
>>> + unsigned long flags;
>>> + struct qlist to_free = QLIST_INIT;
>>> + size_t size_to_free = 0;
>>> + void **last;
>>> +
>>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
>>> + smp_load_acquire(&quarantine_size)))
>>> + return;
>>> +
>>> + spin_lock_irqsave(&quarantine_lock, flags);
>>> +
>>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
>>> + * the installed memory to quarantine minus per-cpu queue limits.
>>> + */
>>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
>>> + QUARANTINE_FRACTION;
>>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
>>> + smp_store_release(&quarantine_size, new_quarantine_size);
>>> +
>>> + last = global_quarantine.head;
>>> + while (last) {
>>> + struct kmem_cache *cache = qlink_to_cache(last);
>>> +
>>> + size_to_free += cache->size;
>>> + if (!*last || size_to_free >
>>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
>>> + break;
>>> + last = (void **) *last;
>>> + }
>>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
>>> +
>>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>>> +
>>> + qlist_free_all(&to_free, NULL);
>>> +}
>>
>> Isn't it better to call quarantine_reduce() in shrink_slab()?
>> It will help to maximize quarantine time.
> This is true, however if we don't call quarantine_reduce() from
> kmalloc()/kfree() the size of the quarantine will be unpredictable.
> There's a tradeoff between efficiency and space here, and at least in
> some cases we may want to trade efficiency for space.
size of the quarantine doesn't matter unless there is memory pressure.
If memory pressure, shrink_slab() would be called and we can reduce
size of quarantine. However, I don't think this is show stopper. We can
do it when needed.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-02-19 2:11 ` Joonsoo Kim
@ 2016-02-19 9:19 ` Dmitry Vyukov
-1 siblings, 0 replies; 78+ messages in thread
From: Dmitry Vyukov @ 2016-02-19 9:19 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Alexander Potapenko, Joonsoo Kim, Andrey Konovalov,
Christoph Lameter, Andrew Morton, Andrey Ryabinin,
Steven Rostedt, kasan-dev, LKML, Linux Memory Management List
On Fri, Feb 19, 2016 at 3:11 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> 2016-02-18 23:06 GMT+09:00 Alexander Potapenko <glider@google.com>:
>> On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
>>>> Quarantine isolates freed objects in a separate queue. The objects are
>>>> returned to the allocator later, which helps to detect use-after-free
>>>> errors.
>>>>
>>>> Freed objects are first added to per-cpu quarantine queues.
>>>> When a cache is destroyed or memory shrinking is requested, the objects
>>>> are moved into the global quarantine queue. Whenever a kmalloc call
>>>> allows memory reclaiming, the oldest objects are popped out of the
>>>> global queue until the total size of objects in quarantine is less than
>>>> 3/4 of the maximum quarantine size (which is a fraction of installed
>>>> physical memory).
>>>
>>> Just wondering why not using time based approach rather than size
>>> based one. In heavy load condition, how much time do the object stay in
>>> quarantine?
>>>
>>>>
>>>> Right now quarantine support is only enabled in SLAB allocator.
>>>> Unification of KASAN features in SLAB and SLUB will be done later.
>>>>
>>>> This patch is based on the "mm: kasan: quarantine" patch originally
>>>> prepared by Dmitry Chernenkov.
>>>>
>>>> Signed-off-by: Alexander Potapenko <glider@google.com>
>>>> ---
>>>> include/linux/kasan.h | 30 ++++--
>>>> lib/test_kasan.c | 29 ++++++
>>>> mm/kasan/Makefile | 2 +-
>>>> mm/kasan/kasan.c | 68 +++++++++++-
>>>> mm/kasan/kasan.h | 11 +-
>>>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> mm/kasan/report.c | 3 +-
>>>> mm/mempool.c | 7 +-
>>>> mm/page_alloc.c | 2 +-
>>>> mm/slab.c | 12 ++-
>>>> mm/slab.h | 4 +
>>>> mm/slab_common.c | 2 +
>>>> mm/slub.c | 4 +-
>>>> 13 files changed, 435 insertions(+), 23 deletions(-)
>>>>
>>>
>>> ...
>>>
>>>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
>>>> +{
>>>> +#ifdef CONFIG_SLAB
>>>> + /* RCU slabs could be legally used after free within the RCU period */
>>>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
>>>> + return false;
>>>> +
>>>> + if (likely(cache->flags & SLAB_KASAN)) {
>>>> + struct kasan_alloc_meta *alloc_info =
>>>> + get_alloc_info(cache, object);
>>>> + struct kasan_free_meta *free_info =
>>>> + get_free_info(cache, object);
>>>> +
>>>> + switch (alloc_info->state) {
>>>> + case KASAN_STATE_ALLOC:
>>>> + alloc_info->state = KASAN_STATE_QUARANTINE;
>>>> + quarantine_put(free_info, cache);
>>>
>>> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
>>> although it's not much meaningful without poisoning. But, I have an
>>> idea to poison object on SLAB_DESTROY_BY_RCU cache.
>>>
>>> quarantine_put() moves per cpu list to global queue when
>>> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
>>> at that time, after then, we can poison objects. With appropriate size
>>> setup, it would not be intrusive.
>>>
>> Won't this slow the quarantine down unpredictably (e.g. in the case
>> there're no RCU slabs in quarantine we'll still be waiting for
>> synchronize_rcu())?
>
> It could be handled by introducing one cpu variable.
>
>> Yet this is something worth looking into. Do you want RCU to be
>> handled in this patch set?
>
> No. It would be future work.
>
>>>> + set_track(&free_info->track, GFP_NOWAIT);
>>>
>>> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
>> Agreed, I can fix that if we decide to handle RCU in this patch
>> (otherwise it will lead to confusion).
>>
>>>
>>>> + kasan_poison_slab_free(cache, object);
>>>> + return true;
>>>> + case KASAN_STATE_QUARANTINE:
>>>> + case KASAN_STATE_FREE:
>>>> + pr_err("Double free");
>>>> + dump_stack();
>>>> + break;
>>>> + default:
>>>> + break;
>>>> + }
>>>> + }
>>>> + return false;
>>>> +#else
>>>> + kasan_poison_slab_free(cache, object);
>>>> + return false;
>>>> +#endif
>>>> +}
>>>> +
>>>
>>> ...
>>>
>>>> +void quarantine_reduce(void)
>>>> +{
>>>> + size_t new_quarantine_size;
>>>> + unsigned long flags;
>>>> + struct qlist to_free = QLIST_INIT;
>>>> + size_t size_to_free = 0;
>>>> + void **last;
>>>> +
>>>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
>>>> + smp_load_acquire(&quarantine_size)))
>>>> + return;
>>>> +
>>>> + spin_lock_irqsave(&quarantine_lock, flags);
>>>> +
>>>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
>>>> + * the installed memory to quarantine minus per-cpu queue limits.
>>>> + */
>>>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
>>>> + QUARANTINE_FRACTION;
>>>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
>>>> + smp_store_release(&quarantine_size, new_quarantine_size);
>>>> +
>>>> + last = global_quarantine.head;
>>>> + while (last) {
>>>> + struct kmem_cache *cache = qlink_to_cache(last);
>>>> +
>>>> + size_to_free += cache->size;
>>>> + if (!*last || size_to_free >
>>>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
>>>> + break;
>>>> + last = (void **) *last;
>>>> + }
>>>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
>>>> +
>>>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>>>> +
>>>> + qlist_free_all(&to_free, NULL);
>>>> +}
>>>
>>> Isn't it better to call quarantine_reduce() in shrink_slab()?
>>> It will help to maximize quarantine time.
>> This is true, however if we don't call quarantine_reduce() from
>> kmalloc()/kfree() the size of the quarantine will be unpredictable.
>> There's a tradeoff between efficiency and space here, and at least in
>> some cases we may want to trade efficiency for space.
>
> size of the quarantine doesn't matter unless there is memory pressure.
> If memory pressure, shrink_slab() would be called and we can reduce
> size of quarantine. However, I don't think this is show stopper. We can
> do it when needed.
No, this does not work. We've tried.
The problem is fragmentation. When all memory is occupied by slab,
it's already too late to reclaim memory. Free objects are randomly
scattered over memory, so if you have just 1% of live objects, the
chances are that you won't be able to reclaim any single page.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-02-19 9:19 ` Dmitry Vyukov
0 siblings, 0 replies; 78+ messages in thread
From: Dmitry Vyukov @ 2016-02-19 9:19 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Alexander Potapenko, Joonsoo Kim, Andrey Konovalov,
Christoph Lameter, Andrew Morton, Andrey Ryabinin,
Steven Rostedt, kasan-dev, LKML, Linux Memory Management List
On Fri, Feb 19, 2016 at 3:11 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> 2016-02-18 23:06 GMT+09:00 Alexander Potapenko <glider@google.com>:
>> On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
>>>> Quarantine isolates freed objects in a separate queue. The objects are
>>>> returned to the allocator later, which helps to detect use-after-free
>>>> errors.
>>>>
>>>> Freed objects are first added to per-cpu quarantine queues.
>>>> When a cache is destroyed or memory shrinking is requested, the objects
>>>> are moved into the global quarantine queue. Whenever a kmalloc call
>>>> allows memory reclaiming, the oldest objects are popped out of the
>>>> global queue until the total size of objects in quarantine is less than
>>>> 3/4 of the maximum quarantine size (which is a fraction of installed
>>>> physical memory).
>>>
>>> Just wondering why not using time based approach rather than size
>>> based one. In heavy load condition, how much time do the object stay in
>>> quarantine?
>>>
>>>>
>>>> Right now quarantine support is only enabled in SLAB allocator.
>>>> Unification of KASAN features in SLAB and SLUB will be done later.
>>>>
>>>> This patch is based on the "mm: kasan: quarantine" patch originally
>>>> prepared by Dmitry Chernenkov.
>>>>
>>>> Signed-off-by: Alexander Potapenko <glider@google.com>
>>>> ---
>>>> include/linux/kasan.h | 30 ++++--
>>>> lib/test_kasan.c | 29 ++++++
>>>> mm/kasan/Makefile | 2 +-
>>>> mm/kasan/kasan.c | 68 +++++++++++-
>>>> mm/kasan/kasan.h | 11 +-
>>>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> mm/kasan/report.c | 3 +-
>>>> mm/mempool.c | 7 +-
>>>> mm/page_alloc.c | 2 +-
>>>> mm/slab.c | 12 ++-
>>>> mm/slab.h | 4 +
>>>> mm/slab_common.c | 2 +
>>>> mm/slub.c | 4 +-
>>>> 13 files changed, 435 insertions(+), 23 deletions(-)
>>>>
>>>
>>> ...
>>>
>>>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
>>>> +{
>>>> +#ifdef CONFIG_SLAB
>>>> + /* RCU slabs could be legally used after free within the RCU period */
>>>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
>>>> + return false;
>>>> +
>>>> + if (likely(cache->flags & SLAB_KASAN)) {
>>>> + struct kasan_alloc_meta *alloc_info =
>>>> + get_alloc_info(cache, object);
>>>> + struct kasan_free_meta *free_info =
>>>> + get_free_info(cache, object);
>>>> +
>>>> + switch (alloc_info->state) {
>>>> + case KASAN_STATE_ALLOC:
>>>> + alloc_info->state = KASAN_STATE_QUARANTINE;
>>>> + quarantine_put(free_info, cache);
>>>
>>> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
>>> although it's not much meaningful without poisoning. But, I have an
>>> idea to poison object on SLAB_DESTROY_BY_RCU cache.
>>>
>>> quarantine_put() moves per cpu list to global queue when
>>> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
>>> at that time, after then, we can poison objects. With appropriate size
>>> setup, it would not be intrusive.
>>>
>> Won't this slow the quarantine down unpredictably (e.g. in the case
>> there're no RCU slabs in quarantine we'll still be waiting for
>> synchronize_rcu())?
>
> It could be handled by introducing one cpu variable.
>
>> Yet this is something worth looking into. Do you want RCU to be
>> handled in this patch set?
>
> No. It would be future work.
>
>>>> + set_track(&free_info->track, GFP_NOWAIT);
>>>
>>> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
>> Agreed, I can fix that if we decide to handle RCU in this patch
>> (otherwise it will lead to confusion).
>>
>>>
>>>> + kasan_poison_slab_free(cache, object);
>>>> + return true;
>>>> + case KASAN_STATE_QUARANTINE:
>>>> + case KASAN_STATE_FREE:
>>>> + pr_err("Double free");
>>>> + dump_stack();
>>>> + break;
>>>> + default:
>>>> + break;
>>>> + }
>>>> + }
>>>> + return false;
>>>> +#else
>>>> + kasan_poison_slab_free(cache, object);
>>>> + return false;
>>>> +#endif
>>>> +}
>>>> +
>>>
>>> ...
>>>
>>>> +void quarantine_reduce(void)
>>>> +{
>>>> + size_t new_quarantine_size;
>>>> + unsigned long flags;
>>>> + struct qlist to_free = QLIST_INIT;
>>>> + size_t size_to_free = 0;
>>>> + void **last;
>>>> +
>>>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
>>>> + smp_load_acquire(&quarantine_size)))
>>>> + return;
>>>> +
>>>> + spin_lock_irqsave(&quarantine_lock, flags);
>>>> +
>>>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
>>>> + * the installed memory to quarantine minus per-cpu queue limits.
>>>> + */
>>>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
>>>> + QUARANTINE_FRACTION;
>>>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
>>>> + smp_store_release(&quarantine_size, new_quarantine_size);
>>>> +
>>>> + last = global_quarantine.head;
>>>> + while (last) {
>>>> + struct kmem_cache *cache = qlink_to_cache(last);
>>>> +
>>>> + size_to_free += cache->size;
>>>> + if (!*last || size_to_free >
>>>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
>>>> + break;
>>>> + last = (void **) *last;
>>>> + }
>>>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
>>>> +
>>>> + spin_unlock_irqrestore(&quarantine_lock, flags);
>>>> +
>>>> + qlist_free_all(&to_free, NULL);
>>>> +}
>>>
>>> Isn't it better to call quarantine_reduce() in shrink_slab()?
>>> It will help to maximize quarantine time.
>> This is true, however if we don't call quarantine_reduce() from
>> kmalloc()/kfree() the size of the quarantine will be unpredictable.
>> There's a tradeoff between efficiency and space here, and at least in
>> some cases we may want to trade efficiency for space.
>
> size of the quarantine doesn't matter unless there is memory pressure.
> If memory pressure, shrink_slab() would be called and we can reduce
> size of quarantine. However, I don't think this is show stopper. We can
> do it when needed.
No, this does not work. We've tried.
The problem is fragmentation. When all memory is occupied by slab,
it's already too late to reclaim memory. Free objects are randomly
scattered over memory, so if you have just 1% of live objects, the
chances are that you won't be able to reclaim any single page.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
2016-02-19 1:41 ` Joonsoo Kim
@ 2016-02-19 12:57 ` Alexander Potapenko
-1 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-19 12:57 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
Ah, yes, I see.
This patch was indeed missing the following bits in kasan_slab_free():
#ifdef CONFIG_SLAB
if (cache->flags & SLAB_KASAN) {
struct kasan_free_meta *free_info =
get_free_info(cache, object);
struct kasan_alloc_meta *alloc_info =
get_alloc_info(cache, object);
alloc_info->state = KASAN_STATE_FREE;
set_track(&free_info->track);
}
#endif
I'll include them in the next round of patches.
On Fri, Feb 19, 2016 at 2:41 AM, Joonsoo Kim <js1304@gmail.com> wrote:
>> On Mon, Feb 1, 2016 at 3:15 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>> On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
>>>> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>>>> >
>>>> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>>> >>
>>>> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>>>> >> > This patch adds KASAN hooks to SLAB allocator.
>>>> >> >
>>>> >> > This patch is based on the "mm: kasan: unified support for SLUB and
>>>> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>>>> >> >
>>>> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
>>>> >> > ---
>>>> >> > Documentation/kasan.txt | 5 ++-
>>>> >>
>>>> >> ...
>>>> >>
>>>> >> > +#ifdef CONFIG_SLAB
>>>> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>>>> >> > + const void *object)
>>>> >> > +{
>>>> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>>>> >> > +}
>>>> >> > +
>>>> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>>>> >> > + const void *object)
>>>> >> > +{
>>>> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
>>>> >> > +}
>>>> >> > +#endif
>>>> >>
>>>> >> I cannot find the place to store stack info for free. get_free_info()
>>>> >> isn't used except print_object(). Plese let me know where.
>>>> >
>>>> > This is covered by other patches in this patchset.
>>>
>>> This should be covered by this patch. Stroing and printing free_info
>>> is already done on SLUB and it is meaningful without quarantain.
>
> 2016-02-18 21:58 GMT+09:00 Alexander Potapenko <glider@google.com>:
>> However this info is meaningless without saved stack traces, which are
>> only introduced in the stackdepot patch (see "[PATCH v1 5/8] mm,
>> kasan: Stackdepot implementation. Enable stackdepot for SLAB")
>
> Not meaningless. You already did it for allocation caller without saved
> stack traces. What makes difference between alloc/free?
>
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 2/8] mm, kasan: SLAB support
@ 2016-02-19 12:57 ` Alexander Potapenko
0 siblings, 0 replies; 78+ messages in thread
From: Alexander Potapenko @ 2016-02-19 12:57 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Joonsoo Kim, kasan-dev, Christoph Lameter, LKML, Dmitriy Vyukov,
Andrey Ryabinin, Linux Memory Management List, Andrey Konovalov,
Andrew Morton, Steven Rostedt
Ah, yes, I see.
This patch was indeed missing the following bits in kasan_slab_free():
#ifdef CONFIG_SLAB
if (cache->flags & SLAB_KASAN) {
struct kasan_free_meta *free_info =
get_free_info(cache, object);
struct kasan_alloc_meta *alloc_info =
get_alloc_info(cache, object);
alloc_info->state = KASAN_STATE_FREE;
set_track(&free_info->track);
}
#endif
I'll include them in the next round of patches.
On Fri, Feb 19, 2016 at 2:41 AM, Joonsoo Kim <js1304@gmail.com> wrote:
>> On Mon, Feb 1, 2016 at 3:15 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>>> On Thu, Jan 28, 2016 at 02:29:42PM +0100, Alexander Potapenko wrote:
>>>> On Thu, Jan 28, 2016 at 1:37 PM, Alexander Potapenko <glider@google.com> wrote:
>>>> >
>>>> > On Jan 28, 2016 8:44 AM, "Joonsoo Kim" <iamjoonsoo.kim@lge.com> wrote:
>>>> >>
>>>> >> On Wed, Jan 27, 2016 at 07:25:07PM +0100, Alexander Potapenko wrote:
>>>> >> > This patch adds KASAN hooks to SLAB allocator.
>>>> >> >
>>>> >> > This patch is based on the "mm: kasan: unified support for SLUB and
>>>> >> > SLAB allocators" patch originally prepared by Dmitry Chernenkov.
>>>> >> >
>>>> >> > Signed-off-by: Alexander Potapenko <glider@google.com>
>>>> >> > ---
>>>> >> > Documentation/kasan.txt | 5 ++-
>>>> >>
>>>> >> ...
>>>> >>
>>>> >> > +#ifdef CONFIG_SLAB
>>>> >> > +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
>>>> >> > + const void *object)
>>>> >> > +{
>>>> >> > + return (void *)object + cache->kasan_info.alloc_meta_offset;
>>>> >> > +}
>>>> >> > +
>>>> >> > +struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
>>>> >> > + const void *object)
>>>> >> > +{
>>>> >> > + return (void *)object + cache->kasan_info.free_meta_offset;
>>>> >> > +}
>>>> >> > +#endif
>>>> >>
>>>> >> I cannot find the place to store stack info for free. get_free_info()
>>>> >> isn't used except print_object(). Plese let me know where.
>>>> >
>>>> > This is covered by other patches in this patchset.
>>>
>>> This should be covered by this patch. Stroing and printing free_info
>>> is already done on SLUB and it is meaningful without quarantain.
>
> 2016-02-18 21:58 GMT+09:00 Alexander Potapenko <glider@google.com>:
>> However this info is meaningless without saved stack traces, which are
>> only introduced in the stackdepot patch (see "[PATCH v1 5/8] mm,
>> kasan: Stackdepot implementation. Enable stackdepot for SLAB")
>
> Not meaningless. You already did it for allocation caller without saved
> stack traces. What makes difference between alloc/free?
>
> Thanks.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den
Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.
This e-mail is confidential. If you are not the right addressee please
do not forward it, please inform the sender, and please erase this
e-mail including any attachments. Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-02-19 9:19 ` Dmitry Vyukov
@ 2016-02-19 15:43 ` Christoph Lameter
-1 siblings, 0 replies; 78+ messages in thread
From: Christoph Lameter @ 2016-02-19 15:43 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Joonsoo Kim, Alexander Potapenko, Joonsoo Kim, Andrey Konovalov,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev, LKML,
Linux Memory Management List
On Fri, 19 Feb 2016, Dmitry Vyukov wrote:
> No, this does not work. We've tried.
> The problem is fragmentation. When all memory is occupied by slab,
> it's already too late to reclaim memory. Free objects are randomly
> scattered over memory, so if you have just 1% of live objects, the
> chances are that you won't be able to reclaim any single page.
Yes that is why slab objects *need* to be *movable*!!!
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-02-19 15:43 ` Christoph Lameter
0 siblings, 0 replies; 78+ messages in thread
From: Christoph Lameter @ 2016-02-19 15:43 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Joonsoo Kim, Alexander Potapenko, Joonsoo Kim, Andrey Konovalov,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev, LKML,
Linux Memory Management List
On Fri, 19 Feb 2016, Dmitry Vyukov wrote:
> No, this does not work. We've tried.
> The problem is fragmentation. When all memory is occupied by slab,
> it's already too late to reclaim memory. Free objects are randomly
> scattered over memory, so if you have just 1% of live objects, the
> chances are that you won't be able to reclaim any single page.
Yes that is why slab objects *need* to be *movable*!!!
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
2016-02-19 9:19 ` Dmitry Vyukov
@ 2016-02-23 7:23 ` Joonsoo Kim
-1 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-23 7:23 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Alexander Potapenko, Andrey Konovalov, Christoph Lameter,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev, LKML,
Linux Memory Management List
On Fri, Feb 19, 2016 at 10:19:48AM +0100, Dmitry Vyukov wrote:
> On Fri, Feb 19, 2016 at 3:11 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> > 2016-02-18 23:06 GMT+09:00 Alexander Potapenko <glider@google.com>:
> >> On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> >>> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
> >>>> Quarantine isolates freed objects in a separate queue. The objects are
> >>>> returned to the allocator later, which helps to detect use-after-free
> >>>> errors.
> >>>>
> >>>> Freed objects are first added to per-cpu quarantine queues.
> >>>> When a cache is destroyed or memory shrinking is requested, the objects
> >>>> are moved into the global quarantine queue. Whenever a kmalloc call
> >>>> allows memory reclaiming, the oldest objects are popped out of the
> >>>> global queue until the total size of objects in quarantine is less than
> >>>> 3/4 of the maximum quarantine size (which is a fraction of installed
> >>>> physical memory).
> >>>
> >>> Just wondering why not using time based approach rather than size
> >>> based one. In heavy load condition, how much time do the object stay in
> >>> quarantine?
> >>>
> >>>>
> >>>> Right now quarantine support is only enabled in SLAB allocator.
> >>>> Unification of KASAN features in SLAB and SLUB will be done later.
> >>>>
> >>>> This patch is based on the "mm: kasan: quarantine" patch originally
> >>>> prepared by Dmitry Chernenkov.
> >>>>
> >>>> Signed-off-by: Alexander Potapenko <glider@google.com>
> >>>> ---
> >>>> include/linux/kasan.h | 30 ++++--
> >>>> lib/test_kasan.c | 29 ++++++
> >>>> mm/kasan/Makefile | 2 +-
> >>>> mm/kasan/kasan.c | 68 +++++++++++-
> >>>> mm/kasan/kasan.h | 11 +-
> >>>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>> mm/kasan/report.c | 3 +-
> >>>> mm/mempool.c | 7 +-
> >>>> mm/page_alloc.c | 2 +-
> >>>> mm/slab.c | 12 ++-
> >>>> mm/slab.h | 4 +
> >>>> mm/slab_common.c | 2 +
> >>>> mm/slub.c | 4 +-
> >>>> 13 files changed, 435 insertions(+), 23 deletions(-)
> >>>>
> >>>
> >>> ...
> >>>
> >>>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
> >>>> +{
> >>>> +#ifdef CONFIG_SLAB
> >>>> + /* RCU slabs could be legally used after free within the RCU period */
> >>>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
> >>>> + return false;
> >>>> +
> >>>> + if (likely(cache->flags & SLAB_KASAN)) {
> >>>> + struct kasan_alloc_meta *alloc_info =
> >>>> + get_alloc_info(cache, object);
> >>>> + struct kasan_free_meta *free_info =
> >>>> + get_free_info(cache, object);
> >>>> +
> >>>> + switch (alloc_info->state) {
> >>>> + case KASAN_STATE_ALLOC:
> >>>> + alloc_info->state = KASAN_STATE_QUARANTINE;
> >>>> + quarantine_put(free_info, cache);
> >>>
> >>> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
> >>> although it's not much meaningful without poisoning. But, I have an
> >>> idea to poison object on SLAB_DESTROY_BY_RCU cache.
> >>>
> >>> quarantine_put() moves per cpu list to global queue when
> >>> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
> >>> at that time, after then, we can poison objects. With appropriate size
> >>> setup, it would not be intrusive.
> >>>
> >> Won't this slow the quarantine down unpredictably (e.g. in the case
> >> there're no RCU slabs in quarantine we'll still be waiting for
> >> synchronize_rcu())?
> >
> > It could be handled by introducing one cpu variable.
> >
> >> Yet this is something worth looking into. Do you want RCU to be
> >> handled in this patch set?
> >
> > No. It would be future work.
> >
> >>>> + set_track(&free_info->track, GFP_NOWAIT);
> >>>
> >>> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
> >> Agreed, I can fix that if we decide to handle RCU in this patch
> >> (otherwise it will lead to confusion).
> >>
> >>>
> >>>> + kasan_poison_slab_free(cache, object);
> >>>> + return true;
> >>>> + case KASAN_STATE_QUARANTINE:
> >>>> + case KASAN_STATE_FREE:
> >>>> + pr_err("Double free");
> >>>> + dump_stack();
> >>>> + break;
> >>>> + default:
> >>>> + break;
> >>>> + }
> >>>> + }
> >>>> + return false;
> >>>> +#else
> >>>> + kasan_poison_slab_free(cache, object);
> >>>> + return false;
> >>>> +#endif
> >>>> +}
> >>>> +
> >>>
> >>> ...
> >>>
> >>>> +void quarantine_reduce(void)
> >>>> +{
> >>>> + size_t new_quarantine_size;
> >>>> + unsigned long flags;
> >>>> + struct qlist to_free = QLIST_INIT;
> >>>> + size_t size_to_free = 0;
> >>>> + void **last;
> >>>> +
> >>>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
> >>>> + smp_load_acquire(&quarantine_size)))
> >>>> + return;
> >>>> +
> >>>> + spin_lock_irqsave(&quarantine_lock, flags);
> >>>> +
> >>>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
> >>>> + * the installed memory to quarantine minus per-cpu queue limits.
> >>>> + */
> >>>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
> >>>> + QUARANTINE_FRACTION;
> >>>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
> >>>> + smp_store_release(&quarantine_size, new_quarantine_size);
> >>>> +
> >>>> + last = global_quarantine.head;
> >>>> + while (last) {
> >>>> + struct kmem_cache *cache = qlink_to_cache(last);
> >>>> +
> >>>> + size_to_free += cache->size;
> >>>> + if (!*last || size_to_free >
> >>>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
> >>>> + break;
> >>>> + last = (void **) *last;
> >>>> + }
> >>>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
> >>>> +
> >>>> + spin_unlock_irqrestore(&quarantine_lock, flags);
> >>>> +
> >>>> + qlist_free_all(&to_free, NULL);
> >>>> +}
> >>>
> >>> Isn't it better to call quarantine_reduce() in shrink_slab()?
> >>> It will help to maximize quarantine time.
> >> This is true, however if we don't call quarantine_reduce() from
> >> kmalloc()/kfree() the size of the quarantine will be unpredictable.
> >> There's a tradeoff between efficiency and space here, and at least in
> >> some cases we may want to trade efficiency for space.
> >
> > size of the quarantine doesn't matter unless there is memory pressure.
> > If memory pressure, shrink_slab() would be called and we can reduce
> > size of quarantine. However, I don't think this is show stopper. We can
> > do it when needed.
>
>
> No, this does not work. We've tried.
> The problem is fragmentation. When all memory is occupied by slab,
> it's already too late to reclaim memory. Free objects are randomly
> scattered over memory, so if you have just 1% of live objects, the
> chances are that you won't be able to reclaim any single page.
Okay. Now, I got it.
Thanks.
^ permalink raw reply [flat|nested] 78+ messages in thread
* Re: [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation
@ 2016-02-23 7:23 ` Joonsoo Kim
0 siblings, 0 replies; 78+ messages in thread
From: Joonsoo Kim @ 2016-02-23 7:23 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Alexander Potapenko, Andrey Konovalov, Christoph Lameter,
Andrew Morton, Andrey Ryabinin, Steven Rostedt, kasan-dev, LKML,
Linux Memory Management List
On Fri, Feb 19, 2016 at 10:19:48AM +0100, Dmitry Vyukov wrote:
> On Fri, Feb 19, 2016 at 3:11 AM, Joonsoo Kim <js1304@gmail.com> wrote:
> > 2016-02-18 23:06 GMT+09:00 Alexander Potapenko <glider@google.com>:
> >> On Mon, Feb 1, 2016 at 3:47 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> >>> On Wed, Jan 27, 2016 at 07:25:13PM +0100, Alexander Potapenko wrote:
> >>>> Quarantine isolates freed objects in a separate queue. The objects are
> >>>> returned to the allocator later, which helps to detect use-after-free
> >>>> errors.
> >>>>
> >>>> Freed objects are first added to per-cpu quarantine queues.
> >>>> When a cache is destroyed or memory shrinking is requested, the objects
> >>>> are moved into the global quarantine queue. Whenever a kmalloc call
> >>>> allows memory reclaiming, the oldest objects are popped out of the
> >>>> global queue until the total size of objects in quarantine is less than
> >>>> 3/4 of the maximum quarantine size (which is a fraction of installed
> >>>> physical memory).
> >>>
> >>> Just wondering why not using time based approach rather than size
> >>> based one. In heavy load condition, how much time do the object stay in
> >>> quarantine?
> >>>
> >>>>
> >>>> Right now quarantine support is only enabled in SLAB allocator.
> >>>> Unification of KASAN features in SLAB and SLUB will be done later.
> >>>>
> >>>> This patch is based on the "mm: kasan: quarantine" patch originally
> >>>> prepared by Dmitry Chernenkov.
> >>>>
> >>>> Signed-off-by: Alexander Potapenko <glider@google.com>
> >>>> ---
> >>>> include/linux/kasan.h | 30 ++++--
> >>>> lib/test_kasan.c | 29 ++++++
> >>>> mm/kasan/Makefile | 2 +-
> >>>> mm/kasan/kasan.c | 68 +++++++++++-
> >>>> mm/kasan/kasan.h | 11 +-
> >>>> mm/kasan/quarantine.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>> mm/kasan/report.c | 3 +-
> >>>> mm/mempool.c | 7 +-
> >>>> mm/page_alloc.c | 2 +-
> >>>> mm/slab.c | 12 ++-
> >>>> mm/slab.h | 4 +
> >>>> mm/slab_common.c | 2 +
> >>>> mm/slub.c | 4 +-
> >>>> 13 files changed, 435 insertions(+), 23 deletions(-)
> >>>>
> >>>
> >>> ...
> >>>
> >>>> +bool kasan_slab_free(struct kmem_cache *cache, void *object)
> >>>> +{
> >>>> +#ifdef CONFIG_SLAB
> >>>> + /* RCU slabs could be legally used after free within the RCU period */
> >>>> + if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
> >>>> + return false;
> >>>> +
> >>>> + if (likely(cache->flags & SLAB_KASAN)) {
> >>>> + struct kasan_alloc_meta *alloc_info =
> >>>> + get_alloc_info(cache, object);
> >>>> + struct kasan_free_meta *free_info =
> >>>> + get_free_info(cache, object);
> >>>> +
> >>>> + switch (alloc_info->state) {
> >>>> + case KASAN_STATE_ALLOC:
> >>>> + alloc_info->state = KASAN_STATE_QUARANTINE;
> >>>> + quarantine_put(free_info, cache);
> >>>
> >>> quarantine_put() can be called regardless of SLAB_DESTROY_BY_RCU,
> >>> although it's not much meaningful without poisoning. But, I have an
> >>> idea to poison object on SLAB_DESTROY_BY_RCU cache.
> >>>
> >>> quarantine_put() moves per cpu list to global queue when
> >>> list size reaches QUARANTINE_PERCPU_SIZE. If we call synchronize_rcu()
> >>> at that time, after then, we can poison objects. With appropriate size
> >>> setup, it would not be intrusive.
> >>>
> >> Won't this slow the quarantine down unpredictably (e.g. in the case
> >> there're no RCU slabs in quarantine we'll still be waiting for
> >> synchronize_rcu())?
> >
> > It could be handled by introducing one cpu variable.
> >
> >> Yet this is something worth looking into. Do you want RCU to be
> >> handled in this patch set?
> >
> > No. It would be future work.
> >
> >>>> + set_track(&free_info->track, GFP_NOWAIT);
> >>>
> >>> set_track() can be called regardless of SLAB_DESTROY_BY_RCU.
> >> Agreed, I can fix that if we decide to handle RCU in this patch
> >> (otherwise it will lead to confusion).
> >>
> >>>
> >>>> + kasan_poison_slab_free(cache, object);
> >>>> + return true;
> >>>> + case KASAN_STATE_QUARANTINE:
> >>>> + case KASAN_STATE_FREE:
> >>>> + pr_err("Double free");
> >>>> + dump_stack();
> >>>> + break;
> >>>> + default:
> >>>> + break;
> >>>> + }
> >>>> + }
> >>>> + return false;
> >>>> +#else
> >>>> + kasan_poison_slab_free(cache, object);
> >>>> + return false;
> >>>> +#endif
> >>>> +}
> >>>> +
> >>>
> >>> ...
> >>>
> >>>> +void quarantine_reduce(void)
> >>>> +{
> >>>> + size_t new_quarantine_size;
> >>>> + unsigned long flags;
> >>>> + struct qlist to_free = QLIST_INIT;
> >>>> + size_t size_to_free = 0;
> >>>> + void **last;
> >>>> +
> >>>> + if (likely(ACCESS_ONCE(global_quarantine.bytes) <=
> >>>> + smp_load_acquire(&quarantine_size)))
> >>>> + return;
> >>>> +
> >>>> + spin_lock_irqsave(&quarantine_lock, flags);
> >>>> +
> >>>> + /* Update quarantine size in case of hotplug. Allocate a fraction of
> >>>> + * the installed memory to quarantine minus per-cpu queue limits.
> >>>> + */
> >>>> + new_quarantine_size = (ACCESS_ONCE(totalram_pages) << PAGE_SHIFT) /
> >>>> + QUARANTINE_FRACTION;
> >>>> + new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
> >>>> + smp_store_release(&quarantine_size, new_quarantine_size);
> >>>> +
> >>>> + last = global_quarantine.head;
> >>>> + while (last) {
> >>>> + struct kmem_cache *cache = qlink_to_cache(last);
> >>>> +
> >>>> + size_to_free += cache->size;
> >>>> + if (!*last || size_to_free >
> >>>> + global_quarantine.bytes - QUARANTINE_LOW_SIZE)
> >>>> + break;
> >>>> + last = (void **) *last;
> >>>> + }
> >>>> + qlist_move(&global_quarantine, last, &to_free, size_to_free);
> >>>> +
> >>>> + spin_unlock_irqrestore(&quarantine_lock, flags);
> >>>> +
> >>>> + qlist_free_all(&to_free, NULL);
> >>>> +}
> >>>
> >>> Isn't it better to call quarantine_reduce() in shrink_slab()?
> >>> It will help to maximize quarantine time.
> >> This is true, however if we don't call quarantine_reduce() from
> >> kmalloc()/kfree() the size of the quarantine will be unpredictable.
> >> There's a tradeoff between efficiency and space here, and at least in
> >> some cases we may want to trade efficiency for space.
> >
> > size of the quarantine doesn't matter unless there is memory pressure.
> > If memory pressure, shrink_slab() would be called and we can reduce
> > size of quarantine. However, I don't think this is show stopper. We can
> > do it when needed.
>
>
> No, this does not work. We've tried.
> The problem is fragmentation. When all memory is occupied by slab,
> it's already too late to reclaim memory. Free objects are randomly
> scattered over memory, so if you have just 1% of live objects, the
> chances are that you won't be able to reclaim any single page.
Okay. Now, I got it.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 78+ messages in thread
end of thread, other threads:[~2016-02-23 7:22 UTC | newest]
Thread overview: 78+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-27 18:25 [PATCH v1 0/8] SLAB support for KASAN Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 1/8] kasan: Change the behavior of kmalloc_large_oob_right test Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-02-02 5:34 ` Andrew Morton
2016-02-02 5:34 ` Andrew Morton
2016-02-02 15:29 ` Andrey Ryabinin
2016-02-02 15:29 ` Andrey Ryabinin
2016-02-02 16:25 ` Alexander Potapenko
2016-02-02 16:25 ` Alexander Potapenko
2016-02-15 14:05 ` Alexander Potapenko
2016-02-15 14:05 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 2/8] mm, kasan: SLAB support Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-28 7:44 ` Joonsoo Kim
2016-01-28 7:44 ` Joonsoo Kim
2016-01-28 12:37 ` Alexander Potapenko
2016-01-28 13:29 ` Alexander Potapenko
2016-01-28 13:29 ` Alexander Potapenko
2016-02-01 2:15 ` Joonsoo Kim
2016-02-01 2:15 ` Joonsoo Kim
2016-02-18 12:58 ` Alexander Potapenko
2016-02-18 12:58 ` Alexander Potapenko
2016-02-19 1:41 ` Joonsoo Kim
2016-02-19 1:41 ` Joonsoo Kim
2016-02-19 12:57 ` Alexander Potapenko
2016-02-19 12:57 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 3/8] mm, kasan: Added GFP flags to KASAN API Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 4/8] arch, ftrace: For KASAN put hard/soft IRQ entries into separate sections Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-28 14:53 ` Steven Rostedt
2016-01-28 14:53 ` Steven Rostedt
2016-01-29 11:33 ` Alexander Potapenko
2016-01-29 11:33 ` Alexander Potapenko
2016-01-29 11:59 ` Alexander Potapenko
2016-01-29 11:59 ` Alexander Potapenko
2016-01-29 14:45 ` Steven Rostedt
2016-01-29 14:45 ` Steven Rostedt
2016-02-16 15:32 ` Alexander Potapenko
2016-02-16 15:32 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 5/8] mm, kasan: Stackdepot implementation. Enable stackdepot for SLAB Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-28 7:40 ` Joonsoo Kim
2016-01-28 7:40 ` Joonsoo Kim
2016-01-28 12:51 ` Alexander Potapenko
2016-01-28 13:27 ` Alexander Potapenko
2016-01-28 13:27 ` Alexander Potapenko
2016-02-01 2:55 ` Joonsoo Kim
2016-02-01 2:55 ` Joonsoo Kim
2016-02-16 18:37 ` Alexander Potapenko
2016-02-16 18:37 ` Alexander Potapenko
2016-02-17 18:29 ` Alexander Potapenko
2016-02-17 18:29 ` Alexander Potapenko
2016-02-18 8:13 ` Joonsoo Kim
2016-02-18 8:13 ` Joonsoo Kim
2016-02-18 15:01 ` Alexander Potapenko
2016-02-18 15:01 ` Alexander Potapenko
2016-02-18 7:58 ` Joonsoo Kim
2016-02-18 7:58 ` Joonsoo Kim
2016-01-27 18:25 ` [PATCH v1 6/8] kasan: Test fix: Warn if the UAF could not be detected in kmalloc_uaf2 Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 7/8] kasan: Changed kmalloc_large_oob_right, added kmalloc_pagealloc_oob_right Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-01-27 18:25 ` [PATCH v1 8/8] mm: kasan: Initial memory quarantine implementation Alexander Potapenko
2016-01-27 18:25 ` Alexander Potapenko
2016-02-01 2:47 ` Joonsoo Kim
2016-02-01 2:47 ` Joonsoo Kim
2016-02-18 14:06 ` Alexander Potapenko
2016-02-18 14:06 ` Alexander Potapenko
2016-02-19 2:11 ` Joonsoo Kim
2016-02-19 2:11 ` Joonsoo Kim
2016-02-19 9:19 ` Dmitry Vyukov
2016-02-19 9:19 ` Dmitry Vyukov
2016-02-19 15:43 ` Christoph Lameter
2016-02-19 15:43 ` Christoph Lameter
2016-02-23 7:23 ` Joonsoo Kim
2016-02-23 7:23 ` Joonsoo Kim
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.