Re: [PATCH v2] mm: kmemleak: Use mempool allocations for kmemleak objects

From: Qian Cai <cai@lca.pw>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Linux-MM <linux-mm@kvack.org>,
	linux-kernel@vger.kernel.org, Michal Hocko <mhocko@kernel.org>,
	Matthew Wilcox <willy@infradead.org>
Subject: Re: [PATCH v2] mm: kmemleak: Use mempool allocations for kmemleak objects
Date: Wed, 31 Jul 2019 08:02:30 -0400	[thread overview]
Message-ID: <C8EF1660-78FF-49E4-B5D7-6B27400F7306@lca.pw> (raw)
In-Reply-To: <20190731095355.GC63307@arrakis.emea.arm.com>

> On Jul 31, 2019, at 5:53 AM, Catalin Marinas <catalin.marinas@arm.com> wrote:
> 
> On Tue, Jul 30, 2019 at 04:22:37PM -0400, Qian Cai wrote:
>> On Tue, 2019-07-30 at 12:57 -0700, Andrew Morton wrote:
>>> On Sat, 27 Jul 2019 14:23:33 +0100 Catalin Marinas <catalin.marinas@arm.com>
>>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>>> @@ -2011,6 +2011,12 @@
>>>>  			Built with CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF=y,
>>>>  			the default is off.
>>>>  
>>>> +	kmemleak.mempool=
>>>> +			[KNL] Boot-time tuning of the minimum kmemleak
>>>> +			metadata pool size.
>>>> +			Format: <int>
>>>> +			Default: NR_CPUS * 4
>>>> +
>> 
>> Catalin, BTW, it is right now unable to handle a large size. I tried to reserve
>> 64M (kmemleak.mempool=67108864),
>> 
>> [    0.039254][    T0] WARNING: CPU: 0 PID: 0 at mm/page_alloc.c:4707 __alloc_pages_nodemask+0x3b8/0x1780
> [...]
>> [    0.039646][    T0] NIP [c000000000395038] __alloc_pages_nodemask+0x3b8/0x1780
>> [    0.039693][    T0] LR [c0000000003d9320] kmalloc_large_node+0x100/0x1a0
>> [    0.039727][    T0] Call Trace:
>> [    0.039795][    T0] [c00000000170fc80] [c0000000003e5080] __kmalloc_node+0x520/0x890
>> [    0.039816][    T0] [c00000000170fd20] [c0000000002e9544] mempool_init_node+0xb4/0x1e0
>> [    0.039836][    T0] [c00000000170fd80] [c0000000002e975c] mempool_create_node+0xcc/0x150
>> [    0.039857][    T0] [c00000000170fdf0] [c000000000b2a730] kmemleak_init+0x16c/0x54c
>> [    0.039878][    T0] [c00000000170fef0] [c000000000ae460c] start_kernel+0x69c/0x7cc
>> [    0.039908][    T0] [c00000000170ff90] [c00000000000a7d4] start_here_common+0x1c/0x434
> [...]
>> [    0.040100][    T0] kmemleak: Kernel memory leak detector disabled
> 
> It looks like the mempool cannot be created. 64M objects means a
> kmalloc(512MB) for the pool array in mempool_init_node(), so that hits
> the MAX_ORDER warning in __alloc_pages_nodemask().
> 
> Maybe the mempool tunable won't help much for your case if you need so
> many objects. It's still worth having a mempool for kmemleak but we
> could look into changing the refill logic while keeping the original
> size constant (say 1024 objects).

Actually, kmemleak.mempool=524288 works quite well on systems I have here. This
is more of making the code robust by error-handling a large value without the
NULL-ptr-deref below. Maybe simply just validate the value by adding upper bound
to not trigger that warning with MAX_ORDER.

> 
>> [   16.192449][    T1] BUG: Unable to handle kernel data access at 0xffffffffffffb2aa
> 
> This doesn't seem kmemleak related from the trace.

This only happens when passing a large kmemleak.mempool, e.g., 64M

[   16.193126][    T1] NIP [c000000000b2a2fc] log_early+0x8/0x160
[   16.193153][    T1] LR [c0000000003e6e48] kmem_cache_free+0x428/0x740
[   16.193190][    T1] Call Trace:
[   16.193213][    T1] [c00000002aaefc60] [0000000000000366] 0x366 (unreliable)
[   16.193243][    T1] [c00000002aaefd00] [c0000000003c9270]
__mpol_put+0x50/0x70
[   16.193272][    T1] [c00000002aaefd20] [c0000000003c9488]
do_set_mempolicy+0x108/0x170
[   16.193314][    T1] [c00000002aaefdb0] [c000000000010434]
kernel_init+0x64/0x150
[   16.193363][    T1] [c00000002aaefe20] [c00000000000b1cc]
ret_from_kernel_thread+0x5c/0x70

# ./scripts/faddr2line vmlinux log_early+0x8/0x160
log_early+0x8/0x160:
log_early at mm/kmemleak.c:859