* Issue with kfence and kmemleak @ 2021-03-16 16:42 Luis Henriques 2021-03-16 17:30 ` Marco Elver 0 siblings, 1 reply; 8+ messages in thread From: Luis Henriques @ 2021-03-16 16:42 UTC (permalink / raw) To: Alexander Potapenko, Marco Elver, Dmitry Vyukov, Catalin Marinas Cc: Andrew Morton, kasan-dev, linux-mm, linux-kernel Hi! This is probably a known issue, but just in case: looks like it's not possible to use kmemleak when kfence is enabled: [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 [ 0.272136] Call Trace: [ 0.272136] dump_stack+0x6d/0x89 [ 0.272136] create_object.isra.0.cold+0x40/0x62 [ 0.272136] ? process_one_work+0x5a0/0x5a0 [ 0.272136] ? process_one_work+0x5a0/0x5a0 [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 [ 0.272136] ? process_one_work+0x5a0/0x5a0 [ 0.272136] kthread+0x3f/0x150 [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 [ 0.272136] ? __kthread_bind_mask+0x60/0x60 [ 0.272136] ret_from_fork+0x22/0x30 [ 0.272136] kmemleak: Kernel memory leak detector disabled [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 [ 0.272136] kmemleak: min_count = 0 [ 0.272136] kmemleak: count = 0 [ 0.272136] kmemleak: flags = 0x1 [ 0.272136] kmemleak: checksum = 0 [ 0.272136] kmemleak: backtrace: [ 0.272136] memblock_alloc_internal+0x6d/0xb0 [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a [ 0.272136] kfence_alloc_pool+0x26/0x3f [ 0.272136] start_kernel+0x242/0x548 [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb I've tried the hack below but it didn't really helped. Obviously I don't really understand what's going on ;-) But I think the reason for this patch not working as (I) expected is because kfence is initialised *before* kmemleak. diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 3b8ec938470a..b4ffd7695268 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) if (!__kfence_pool) pr_err("failed to allocate pool\n"); + kmemleak_no_scan(__kfence_pool); } Cheers, -- Luís ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-16 16:42 Issue with kfence and kmemleak Luis Henriques @ 2021-03-16 17:30 ` Marco Elver 2021-03-16 17:42 ` Luis Henriques 2021-03-16 18:19 ` Catalin Marinas 0 siblings, 2 replies; 8+ messages in thread From: Marco Elver @ 2021-03-16 17:30 UTC (permalink / raw) To: Luis Henriques Cc: Alexander Potapenko, Dmitry Vyukov, Catalin Marinas, Andrew Morton, kasan-dev, linux-mm, linux-kernel On Tue, Mar 16, 2021 at 04:42PM +0000, Luis Henriques wrote: > Hi! > > This is probably a known issue, but just in case: looks like it's not > possible to use kmemleak when kfence is enabled: Thanks for spotting this. > [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) > [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 > [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 > [ 0.272136] Call Trace: > [ 0.272136] dump_stack+0x6d/0x89 > [ 0.272136] create_object.isra.0.cold+0x40/0x62 > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > [ 0.272136] kthread+0x3f/0x150 > [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 > [ 0.272136] ? __kthread_bind_mask+0x60/0x60 > [ 0.272136] ret_from_fork+0x22/0x30 > [ 0.272136] kmemleak: Kernel memory leak detector disabled > [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): > [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 > [ 0.272136] kmemleak: min_count = 0 > [ 0.272136] kmemleak: count = 0 > [ 0.272136] kmemleak: flags = 0x1 > [ 0.272136] kmemleak: checksum = 0 > [ 0.272136] kmemleak: backtrace: > [ 0.272136] memblock_alloc_internal+0x6d/0xb0 > [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a > [ 0.272136] kfence_alloc_pool+0x26/0x3f > [ 0.272136] start_kernel+0x242/0x548 > [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb > > I've tried the hack below but it didn't really helped. Obviously I don't > really understand what's going on ;-) But I think the reason for this > patch not working as (I) expected is because kfence is initialised > *before* kmemleak. > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > index 3b8ec938470a..b4ffd7695268 100644 > --- a/mm/kfence/core.c > +++ b/mm/kfence/core.c > @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) > > if (!__kfence_pool) > pr_err("failed to allocate pool\n"); > + kmemleak_no_scan(__kfence_pool); > } Can you try the below patch? Thanks, -- Marco ------ >8 ------ diff --git a/mm/kfence/core.c b/mm/kfence/core.c index f7106f28443d..5891019721f6 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -12,6 +12,7 @@ #include <linux/debugfs.h> #include <linux/kcsan-checks.h> #include <linux/kfence.h> +#include <linux/kmemleak.h> #include <linux/list.h> #include <linux/lockdep.h> #include <linux/memblock.h> @@ -481,6 +482,13 @@ static bool __init kfence_init_pool(void) addr += 2 * PAGE_SIZE; } + /* + * The pool is live and will never be deallocated from this point on; + * tell kmemleak this is now free memory, so that later allocations can + * correctly be tracked. + */ + kmemleak_free_part_phys(__pa(__kfence_pool), KFENCE_POOL_SIZE); + return true; err: ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-16 17:30 ` Marco Elver @ 2021-03-16 17:42 ` Luis Henriques 2021-03-16 18:19 ` Catalin Marinas 1 sibling, 0 replies; 8+ messages in thread From: Luis Henriques @ 2021-03-16 17:42 UTC (permalink / raw) To: Marco Elver Cc: Alexander Potapenko, Dmitry Vyukov, Catalin Marinas, Andrew Morton, kasan-dev, linux-mm, linux-kernel On Tue, Mar 16, 2021 at 06:30:00PM +0100, Marco Elver wrote: > On Tue, Mar 16, 2021 at 04:42PM +0000, Luis Henriques wrote: > > Hi! > > > > This is probably a known issue, but just in case: looks like it's not > > possible to use kmemleak when kfence is enabled: > > Thanks for spotting this. > > > [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) > > [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 > > [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 > > [ 0.272136] Call Trace: > > [ 0.272136] dump_stack+0x6d/0x89 > > [ 0.272136] create_object.isra.0.cold+0x40/0x62 > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > [ 0.272136] kthread+0x3f/0x150 > > [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 > > [ 0.272136] ? __kthread_bind_mask+0x60/0x60 > > [ 0.272136] ret_from_fork+0x22/0x30 > > [ 0.272136] kmemleak: Kernel memory leak detector disabled > > [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): > > [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 > > [ 0.272136] kmemleak: min_count = 0 > > [ 0.272136] kmemleak: count = 0 > > [ 0.272136] kmemleak: flags = 0x1 > > [ 0.272136] kmemleak: checksum = 0 > > [ 0.272136] kmemleak: backtrace: > > [ 0.272136] memblock_alloc_internal+0x6d/0xb0 > > [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a > > [ 0.272136] kfence_alloc_pool+0x26/0x3f > > [ 0.272136] start_kernel+0x242/0x548 > > [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb > > > > I've tried the hack below but it didn't really helped. Obviously I don't > > really understand what's going on ;-) But I think the reason for this > > patch not working as (I) expected is because kfence is initialised > > *before* kmemleak. > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > index 3b8ec938470a..b4ffd7695268 100644 > > --- a/mm/kfence/core.c > > +++ b/mm/kfence/core.c > > @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) > > > > if (!__kfence_pool) > > pr_err("failed to allocate pool\n"); > > + kmemleak_no_scan(__kfence_pool); > > } > > Can you try the below patch? Yep, that seems to fix the issue. Feel free to add my Tested-by. Thanks! Cheers, -- Luís > > Thanks, > -- Marco > > ------ >8 ------ > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > index f7106f28443d..5891019721f6 100644 > --- a/mm/kfence/core.c > +++ b/mm/kfence/core.c > @@ -12,6 +12,7 @@ > #include <linux/debugfs.h> > #include <linux/kcsan-checks.h> > #include <linux/kfence.h> > +#include <linux/kmemleak.h> > #include <linux/list.h> > #include <linux/lockdep.h> > #include <linux/memblock.h> > @@ -481,6 +482,13 @@ static bool __init kfence_init_pool(void) > addr += 2 * PAGE_SIZE; > } > > + /* > + * The pool is live and will never be deallocated from this point on; > + * tell kmemleak this is now free memory, so that later allocations can > + * correctly be tracked. > + */ > + kmemleak_free_part_phys(__pa(__kfence_pool), KFENCE_POOL_SIZE); > + > return true; > > err: ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-16 17:30 ` Marco Elver 2021-03-16 17:42 ` Luis Henriques @ 2021-03-16 18:19 ` Catalin Marinas 2021-03-16 18:47 ` Marco Elver 1 sibling, 1 reply; 8+ messages in thread From: Catalin Marinas @ 2021-03-16 18:19 UTC (permalink / raw) To: Marco Elver Cc: Luis Henriques, Alexander Potapenko, Dmitry Vyukov, Andrew Morton, kasan-dev, linux-mm, linux-kernel On Tue, Mar 16, 2021 at 06:30:00PM +0100, Marco Elver wrote: > On Tue, Mar 16, 2021 at 04:42PM +0000, Luis Henriques wrote: > > This is probably a known issue, but just in case: looks like it's not > > possible to use kmemleak when kfence is enabled: > > > > [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) > > [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 > > [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 > > [ 0.272136] Call Trace: > > [ 0.272136] dump_stack+0x6d/0x89 > > [ 0.272136] create_object.isra.0.cold+0x40/0x62 > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > [ 0.272136] kthread+0x3f/0x150 > > [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 > > [ 0.272136] ? __kthread_bind_mask+0x60/0x60 > > [ 0.272136] ret_from_fork+0x22/0x30 > > [ 0.272136] kmemleak: Kernel memory leak detector disabled > > [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): > > [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 > > [ 0.272136] kmemleak: min_count = 0 > > [ 0.272136] kmemleak: count = 0 > > [ 0.272136] kmemleak: flags = 0x1 > > [ 0.272136] kmemleak: checksum = 0 > > [ 0.272136] kmemleak: backtrace: > > [ 0.272136] memblock_alloc_internal+0x6d/0xb0 > > [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a > > [ 0.272136] kfence_alloc_pool+0x26/0x3f > > [ 0.272136] start_kernel+0x242/0x548 > > [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb > > > > I've tried the hack below but it didn't really helped. Obviously I don't > > really understand what's going on ;-) But I think the reason for this > > patch not working as (I) expected is because kfence is initialised > > *before* kmemleak. > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > index 3b8ec938470a..b4ffd7695268 100644 > > --- a/mm/kfence/core.c > > +++ b/mm/kfence/core.c > > @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) > > > > if (!__kfence_pool) > > pr_err("failed to allocate pool\n"); > > + kmemleak_no_scan(__kfence_pool); > > } > > Can you try the below patch? > > Thanks, > -- Marco > > ------ >8 ------ > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > index f7106f28443d..5891019721f6 100644 > --- a/mm/kfence/core.c > +++ b/mm/kfence/core.c > @@ -12,6 +12,7 @@ > #include <linux/debugfs.h> > #include <linux/kcsan-checks.h> > #include <linux/kfence.h> > +#include <linux/kmemleak.h> > #include <linux/list.h> > #include <linux/lockdep.h> > #include <linux/memblock.h> > @@ -481,6 +482,13 @@ static bool __init kfence_init_pool(void) > addr += 2 * PAGE_SIZE; > } > > + /* > + * The pool is live and will never be deallocated from this point on; > + * tell kmemleak this is now free memory, so that later allocations can > + * correctly be tracked. > + */ > + kmemleak_free_part_phys(__pa(__kfence_pool), KFENCE_POOL_SIZE); I presume this pool does not refer any objects that are only tracked through pool pointers. kmemleak_free() (or *_free_part) should work, no need for the _phys variant (which converts it back with __va). Since we normally use kmemleak_ignore() (or no_scan) for objects we don't care about, I'd expand the comment that this object needs to be removed from the kmemleak object tree as it will overlap with subsequent allocations handled by kfence which return pointers within this range. -- Catalin ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-16 18:19 ` Catalin Marinas @ 2021-03-16 18:47 ` Marco Elver 2021-03-16 19:22 ` Catalin Marinas 2021-03-17 8:41 ` Luis Henriques 0 siblings, 2 replies; 8+ messages in thread From: Marco Elver @ 2021-03-16 18:47 UTC (permalink / raw) To: Catalin Marinas Cc: Luis Henriques, Alexander Potapenko, Dmitry Vyukov, Andrew Morton, kasan-dev, linux-mm, linux-kernel On Tue, Mar 16, 2021 at 06:19PM +0000, Catalin Marinas wrote: > On Tue, Mar 16, 2021 at 06:30:00PM +0100, Marco Elver wrote: > > On Tue, Mar 16, 2021 at 04:42PM +0000, Luis Henriques wrote: > > > This is probably a known issue, but just in case: looks like it's not > > > possible to use kmemleak when kfence is enabled: > > > > > > [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) > > > [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 > > > [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 > > > [ 0.272136] Call Trace: > > > [ 0.272136] dump_stack+0x6d/0x89 > > > [ 0.272136] create_object.isra.0.cold+0x40/0x62 > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > [ 0.272136] kthread+0x3f/0x150 > > > [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 > > > [ 0.272136] ? __kthread_bind_mask+0x60/0x60 > > > [ 0.272136] ret_from_fork+0x22/0x30 > > > [ 0.272136] kmemleak: Kernel memory leak detector disabled > > > [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): > > > [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 > > > [ 0.272136] kmemleak: min_count = 0 > > > [ 0.272136] kmemleak: count = 0 > > > [ 0.272136] kmemleak: flags = 0x1 > > > [ 0.272136] kmemleak: checksum = 0 > > > [ 0.272136] kmemleak: backtrace: > > > [ 0.272136] memblock_alloc_internal+0x6d/0xb0 > > > [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a > > > [ 0.272136] kfence_alloc_pool+0x26/0x3f > > > [ 0.272136] start_kernel+0x242/0x548 > > > [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb > > > > > > I've tried the hack below but it didn't really helped. Obviously I don't > > > really understand what's going on ;-) But I think the reason for this > > > patch not working as (I) expected is because kfence is initialised > > > *before* kmemleak. > > > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > > index 3b8ec938470a..b4ffd7695268 100644 > > > --- a/mm/kfence/core.c > > > +++ b/mm/kfence/core.c > > > @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) > > > > > > if (!__kfence_pool) > > > pr_err("failed to allocate pool\n"); > > > + kmemleak_no_scan(__kfence_pool); > > > } > > > > Can you try the below patch? > > > > Thanks, > > -- Marco > > > > ------ >8 ------ > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > index f7106f28443d..5891019721f6 100644 > > --- a/mm/kfence/core.c > > +++ b/mm/kfence/core.c > > @@ -12,6 +12,7 @@ > > #include <linux/debugfs.h> > > #include <linux/kcsan-checks.h> > > #include <linux/kfence.h> > > +#include <linux/kmemleak.h> > > #include <linux/list.h> > > #include <linux/lockdep.h> > > #include <linux/memblock.h> > > @@ -481,6 +482,13 @@ static bool __init kfence_init_pool(void) > > addr += 2 * PAGE_SIZE; > > } > > > > + /* > > + * The pool is live and will never be deallocated from this point on; > > + * tell kmemleak this is now free memory, so that later allocations can > > + * correctly be tracked. > > + */ > > + kmemleak_free_part_phys(__pa(__kfence_pool), KFENCE_POOL_SIZE); > > I presume this pool does not refer any objects that are only tracked > through pool pointers. No, at this point this memory should not have been touched by anything. > kmemleak_free() (or *_free_part) should work, no need for the _phys > variant (which converts it back with __va). Will fix. > Since we normally use kmemleak_ignore() (or no_scan) for objects we > don't care about, I'd expand the comment that this object needs to be > removed from the kmemleak object tree as it will overlap with subsequent > allocations handled by kfence which return pointers within this range. One thing I've just run into: "BUG: KFENCE: out-of-bounds read in scan_block+0x6b/0x170 mm/kmemleak.c:1244" Probably because kmemleak is passed the rounded size for the size-class, and not the real allocation size. Can this be fixed with kmemleak_ignore() only called on the KFENCE guard pages? I'd like kmemleak to scan the valid portion of an object allocated through KFENCE, but no further than that. Or do we need to fix the size if it's a kfence object: diff --git a/mm/kmemleak.c b/mm/kmemleak.c index c0014d3b91c1..fe6e3ae8e8c6 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -97,6 +97,7 @@ #include <linux/atomic.h> #include <linux/kasan.h> +#include <linux/kfence.h> #include <linux/kmemleak.h> #include <linux/memory_hotplug.h> @@ -589,7 +590,7 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, atomic_set(&object->use_count, 1); object->flags = OBJECT_ALLOCATED; object->pointer = ptr; - object->size = size; + object->size = kfence_ksize((void *)ptr) ?: size; object->excess_ref = 0; object->min_count = min_count; object->count = 0; /* white color initially */ The alternative is to call kfence_ksize() in slab_post_alloc_hook() when calling kmemleak_alloc. Do you have a preference? Thanks, -- Marco ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-16 18:47 ` Marco Elver @ 2021-03-16 19:22 ` Catalin Marinas 2021-03-17 8:41 ` Luis Henriques 1 sibling, 0 replies; 8+ messages in thread From: Catalin Marinas @ 2021-03-16 19:22 UTC (permalink / raw) To: Marco Elver Cc: Luis Henriques, Alexander Potapenko, Dmitry Vyukov, Andrew Morton, kasan-dev, linux-mm, linux-kernel On Tue, Mar 16, 2021 at 07:47:00PM +0100, Marco Elver wrote: > One thing I've just run into: "BUG: KFENCE: out-of-bounds read in > scan_block+0x6b/0x170 mm/kmemleak.c:1244" > > Probably because kmemleak is passed the rounded size for the size-class, > and not the real allocation size. Can this be fixed with > kmemleak_ignore() only called on the KFENCE guard pages? If it's only on the occasional object, you can do a kmemleak_scan_area() but some care needed as this in turn allocates memory for kmemleak internal metadata. > I'd like kmemleak to scan the valid portion of an object allocated > through KFENCE, but no further than that. > > Or do we need to fix the size if it's a kfence object: > > diff --git a/mm/kmemleak.c b/mm/kmemleak.c > index c0014d3b91c1..fe6e3ae8e8c6 100644 > --- a/mm/kmemleak.c > +++ b/mm/kmemleak.c > @@ -97,6 +97,7 @@ > #include <linux/atomic.h> > > #include <linux/kasan.h> > +#include <linux/kfence.h> > #include <linux/kmemleak.h> > #include <linux/memory_hotplug.h> > > @@ -589,7 +590,7 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, > atomic_set(&object->use_count, 1); > object->flags = OBJECT_ALLOCATED; > object->pointer = ptr; > - object->size = size; > + object->size = kfence_ksize((void *)ptr) ?: size; > object->excess_ref = 0; > object->min_count = min_count; > object->count = 0; /* white color initially */ > > The alternative is to call kfence_ksize() in slab_post_alloc_hook() when > calling kmemleak_alloc. One of these is probably the easiest. If kfence only works on slab objects, better to pass the right size from slab_post_alloc_hook(). If you plan to expand it later to vmalloc(), just fix the size in create_object(). -- Catalin ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-16 18:47 ` Marco Elver 2021-03-16 19:22 ` Catalin Marinas @ 2021-03-17 8:41 ` Luis Henriques 2021-03-17 8:48 ` Marco Elver 1 sibling, 1 reply; 8+ messages in thread From: Luis Henriques @ 2021-03-17 8:41 UTC (permalink / raw) To: Marco Elver Cc: Catalin Marinas, Alexander Potapenko, Dmitry Vyukov, Andrew Morton, kasan-dev, linux-mm, linux-kernel On Tue, Mar 16, 2021 at 07:47:00PM +0100, Marco Elver wrote: > On Tue, Mar 16, 2021 at 06:19PM +0000, Catalin Marinas wrote: > > On Tue, Mar 16, 2021 at 06:30:00PM +0100, Marco Elver wrote: > > > On Tue, Mar 16, 2021 at 04:42PM +0000, Luis Henriques wrote: > > > > This is probably a known issue, but just in case: looks like it's not > > > > possible to use kmemleak when kfence is enabled: > > > > > > > > [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) > > > > [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 > > > > [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 > > > > [ 0.272136] Call Trace: > > > > [ 0.272136] dump_stack+0x6d/0x89 > > > > [ 0.272136] create_object.isra.0.cold+0x40/0x62 > > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > > [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 > > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > > [ 0.272136] kthread+0x3f/0x150 > > > > [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 > > > > [ 0.272136] ? __kthread_bind_mask+0x60/0x60 > > > > [ 0.272136] ret_from_fork+0x22/0x30 > > > > [ 0.272136] kmemleak: Kernel memory leak detector disabled > > > > [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): > > > > [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 > > > > [ 0.272136] kmemleak: min_count = 0 > > > > [ 0.272136] kmemleak: count = 0 > > > > [ 0.272136] kmemleak: flags = 0x1 > > > > [ 0.272136] kmemleak: checksum = 0 > > > > [ 0.272136] kmemleak: backtrace: > > > > [ 0.272136] memblock_alloc_internal+0x6d/0xb0 > > > > [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a > > > > [ 0.272136] kfence_alloc_pool+0x26/0x3f > > > > [ 0.272136] start_kernel+0x242/0x548 > > > > [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb > > > > > > > > I've tried the hack below but it didn't really helped. Obviously I don't > > > > really understand what's going on ;-) But I think the reason for this > > > > patch not working as (I) expected is because kfence is initialised > > > > *before* kmemleak. > > > > > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > > > index 3b8ec938470a..b4ffd7695268 100644 > > > > --- a/mm/kfence/core.c > > > > +++ b/mm/kfence/core.c > > > > @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) > > > > > > > > if (!__kfence_pool) > > > > pr_err("failed to allocate pool\n"); > > > > + kmemleak_no_scan(__kfence_pool); > > > > } > > > > > > Can you try the below patch? > > > > > > Thanks, > > > -- Marco > > > > > > ------ >8 ------ > > > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > > index f7106f28443d..5891019721f6 100644 > > > --- a/mm/kfence/core.c > > > +++ b/mm/kfence/core.c > > > @@ -12,6 +12,7 @@ > > > #include <linux/debugfs.h> > > > #include <linux/kcsan-checks.h> > > > #include <linux/kfence.h> > > > +#include <linux/kmemleak.h> > > > #include <linux/list.h> > > > #include <linux/lockdep.h> > > > #include <linux/memblock.h> > > > @@ -481,6 +482,13 @@ static bool __init kfence_init_pool(void) > > > addr += 2 * PAGE_SIZE; > > > } > > > > > > + /* > > > + * The pool is live and will never be deallocated from this point on; > > > + * tell kmemleak this is now free memory, so that later allocations can > > > + * correctly be tracked. > > > + */ > > > + kmemleak_free_part_phys(__pa(__kfence_pool), KFENCE_POOL_SIZE); > > > > I presume this pool does not refer any objects that are only tracked > > through pool pointers. > > No, at this point this memory should not have been touched by anything. > > > kmemleak_free() (or *_free_part) should work, no need for the _phys > > variant (which converts it back with __va). > > Will fix. > > > Since we normally use kmemleak_ignore() (or no_scan) for objects we > > don't care about, I'd expand the comment that this object needs to be > > removed from the kmemleak object tree as it will overlap with subsequent > > allocations handled by kfence which return pointers within this range. > > One thing I've just run into: "BUG: KFENCE: out-of-bounds read in > scan_block+0x6b/0x170 mm/kmemleak.c:1244" FWIW, I just saw this as well. It doesn't happen every time, but yeah I missed it in my initial testing. Cheers, -- Luís > > Probably because kmemleak is passed the rounded size for the size-class, > and not the real allocation size. Can this be fixed with > kmemleak_ignore() only called on the KFENCE guard pages? > > I'd like kmemleak to scan the valid portion of an object allocated > through KFENCE, but no further than that. > > Or do we need to fix the size if it's a kfence object: > > diff --git a/mm/kmemleak.c b/mm/kmemleak.c > index c0014d3b91c1..fe6e3ae8e8c6 100644 > --- a/mm/kmemleak.c > +++ b/mm/kmemleak.c > @@ -97,6 +97,7 @@ > #include <linux/atomic.h> > > #include <linux/kasan.h> > +#include <linux/kfence.h> > #include <linux/kmemleak.h> > #include <linux/memory_hotplug.h> > > @@ -589,7 +590,7 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, > atomic_set(&object->use_count, 1); > object->flags = OBJECT_ALLOCATED; > object->pointer = ptr; > - object->size = size; > + object->size = kfence_ksize((void *)ptr) ?: size; > object->excess_ref = 0; > object->min_count = min_count; > object->count = 0; /* white color initially */ > > > The alternative is to call kfence_ksize() in slab_post_alloc_hook() when > calling kmemleak_alloc. > > Do you have a preference? > > Thanks, > -- Marco ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Issue with kfence and kmemleak 2021-03-17 8:41 ` Luis Henriques @ 2021-03-17 8:48 ` Marco Elver 0 siblings, 0 replies; 8+ messages in thread From: Marco Elver @ 2021-03-17 8:48 UTC (permalink / raw) To: Luis Henriques Cc: Catalin Marinas, Alexander Potapenko, Dmitry Vyukov, Andrew Morton, kasan-dev, Linux Memory Management List, LKML On Wed, 17 Mar 2021 at 09:39, Luis Henriques <lhenriques@suse.de> wrote: > On Tue, Mar 16, 2021 at 07:47:00PM +0100, Marco Elver wrote: > > On Tue, Mar 16, 2021 at 06:19PM +0000, Catalin Marinas wrote: > > > On Tue, Mar 16, 2021 at 06:30:00PM +0100, Marco Elver wrote: > > > > On Tue, Mar 16, 2021 at 04:42PM +0000, Luis Henriques wrote: > > > > > This is probably a known issue, but just in case: looks like it's not > > > > > possible to use kmemleak when kfence is enabled: > > > > > > > > > > [ 0.272136] kmemleak: Cannot insert 0xffff888236e02f00 into the object search tree (overlaps existing) > > > > > [ 0.272136] CPU: 0 PID: 8 Comm: kthreadd Not tainted 5.12.0-rc3+ #92 > > > > > [ 0.272136] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014 > > > > > [ 0.272136] Call Trace: > > > > > [ 0.272136] dump_stack+0x6d/0x89 > > > > > [ 0.272136] create_object.isra.0.cold+0x40/0x62 > > > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > > > [ 0.272136] kmem_cache_alloc_trace+0x110/0x2f0 > > > > > [ 0.272136] ? process_one_work+0x5a0/0x5a0 > > > > > [ 0.272136] kthread+0x3f/0x150 > > > > > [ 0.272136] ? lockdep_hardirqs_on_prepare+0xd4/0x170 > > > > > [ 0.272136] ? __kthread_bind_mask+0x60/0x60 > > > > > [ 0.272136] ret_from_fork+0x22/0x30 > > > > > [ 0.272136] kmemleak: Kernel memory leak detector disabled > > > > > [ 0.272136] kmemleak: Object 0xffff888236e00000 (size 2097152): > > > > > [ 0.272136] kmemleak: comm "swapper", pid 0, jiffies 4294892296 > > > > > [ 0.272136] kmemleak: min_count = 0 > > > > > [ 0.272136] kmemleak: count = 0 > > > > > [ 0.272136] kmemleak: flags = 0x1 > > > > > [ 0.272136] kmemleak: checksum = 0 > > > > > [ 0.272136] kmemleak: backtrace: > > > > > [ 0.272136] memblock_alloc_internal+0x6d/0xb0 > > > > > [ 0.272136] memblock_alloc_try_nid+0x6c/0x8a > > > > > [ 0.272136] kfence_alloc_pool+0x26/0x3f > > > > > [ 0.272136] start_kernel+0x242/0x548 > > > > > [ 0.272136] secondary_startup_64_no_verify+0xb0/0xbb > > > > > > > > > > I've tried the hack below but it didn't really helped. Obviously I don't > > > > > really understand what's going on ;-) But I think the reason for this > > > > > patch not working as (I) expected is because kfence is initialised > > > > > *before* kmemleak. > > > > > > > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > > > > index 3b8ec938470a..b4ffd7695268 100644 > > > > > --- a/mm/kfence/core.c > > > > > +++ b/mm/kfence/core.c > > > > > @@ -631,6 +631,9 @@ void __init kfence_alloc_pool(void) > > > > > > > > > > if (!__kfence_pool) > > > > > pr_err("failed to allocate pool\n"); > > > > > + kmemleak_no_scan(__kfence_pool); > > > > > } > > > > > > > > Can you try the below patch? > > > > > > > > Thanks, > > > > -- Marco > > > > > > > > ------ >8 ------ > > > > > > > > diff --git a/mm/kfence/core.c b/mm/kfence/core.c > > > > index f7106f28443d..5891019721f6 100644 > > > > --- a/mm/kfence/core.c > > > > +++ b/mm/kfence/core.c > > > > @@ -12,6 +12,7 @@ > > > > #include <linux/debugfs.h> > > > > #include <linux/kcsan-checks.h> > > > > #include <linux/kfence.h> > > > > +#include <linux/kmemleak.h> > > > > #include <linux/list.h> > > > > #include <linux/lockdep.h> > > > > #include <linux/memblock.h> > > > > @@ -481,6 +482,13 @@ static bool __init kfence_init_pool(void) > > > > addr += 2 * PAGE_SIZE; > > > > } > > > > > > > > + /* > > > > + * The pool is live and will never be deallocated from this point on; > > > > + * tell kmemleak this is now free memory, so that later allocations can > > > > + * correctly be tracked. > > > > + */ > > > > + kmemleak_free_part_phys(__pa(__kfence_pool), KFENCE_POOL_SIZE); > > > > > > I presume this pool does not refer any objects that are only tracked > > > through pool pointers. > > > > No, at this point this memory should not have been touched by anything. > > > > > kmemleak_free() (or *_free_part) should work, no need for the _phys > > > variant (which converts it back with __va). > > > > Will fix. > > > > > Since we normally use kmemleak_ignore() (or no_scan) for objects we > > > don't care about, I'd expand the comment that this object needs to be > > > removed from the kmemleak object tree as it will overlap with subsequent > > > allocations handled by kfence which return pointers within this range. > > > > One thing I've just run into: "BUG: KFENCE: out-of-bounds read in > > scan_block+0x6b/0x170 mm/kmemleak.c:1244" > > FWIW, I just saw this as well. It doesn't happen every time, but yeah I > missed it in my initial testing. I've just sent the patch, please re-test if you can: https://lkml.kernel.org/r/20210317084740.3099921-1-elver@google.com Thanks! ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-03-17 8:49 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-16 16:42 Issue with kfence and kmemleak Luis Henriques 2021-03-16 17:30 ` Marco Elver 2021-03-16 17:42 ` Luis Henriques 2021-03-16 18:19 ` Catalin Marinas 2021-03-16 18:47 ` Marco Elver 2021-03-16 19:22 ` Catalin Marinas 2021-03-17 8:41 ` Luis Henriques 2021-03-17 8:48 ` Marco Elver
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).