[PATCH v2] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()

[PATCH v2] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()

[David Hildenbrand]

Without CONFIG_PREEMPT, it can happen that we get soft lockups detected,
e.g., while booting up.

[  105.608900] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [swapper/0:1]
[  105.608933] Modules linked in:
[  105.608933] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-next-20200331+ #4
[  105.608933] Hardware name: Red Hat KVM, BIOS 1.11.1-4.module+el8.1.0+4066+0f1aadab 04/01/2014
[  105.608933] RIP: 0010:__pageblock_pfn_to_page+0x134/0x1c0
[  105.608933] Code: 85 c0 74 71 4a 8b 04 d0 48 85 c0 74 68 48 01 c1 74 63 f6 01 04 74 5e 48 c1 e7 06 4c 8b 05 cc 991
[  105.608933] RSP: 0000:ffffb6d94000fe60 EFLAGS: 00010286 ORIG_RAX: ffffffffffffff13
[  105.608933] RAX: fffff81953250000 RBX: 000000000a4c9600 RCX: ffff8fe9ff7c1990
[  105.608933] RDX: ffff8fe9ff7dab80 RSI: 000000000a4c95ff RDI: 0000000293250000
[  105.608933] RBP: ffff8fe9ff7dab80 R08: fffff816c0000000 R09: 0000000000000008
[  105.608933] R10: 0000000000000014 R11: 0000000000000014 R12: 0000000000000000
[  105.608933] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[  105.608933] FS:  0000000000000000(0000) GS:ffff8fe1ff400000(0000) knlGS:0000000000000000
[  105.608933] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  105.608933] CR2: 000000000f613000 CR3: 00000088cf20a000 CR4: 00000000000006f0
[  105.608933] Call Trace:
[  105.608933]  set_zone_contiguous+0x56/0x70
[  105.608933]  page_alloc_init_late+0x166/0x176
[  105.608933]  kernel_init_freeable+0xfa/0x255
[  105.608933]  ? rest_init+0xaa/0xaa
[  105.608933]  kernel_init+0xa/0x106
[  105.608933]  ret_from_fork+0x35/0x40

The issue becomes visible when having a lot of memory (e.g., 4TB)
assigned to a single NUMA node - a system that can easily be created
using QEMU. Inside VMs on a hypervisor with quite some memory
overcommit, this is fairly easy to trigger.

Reviewed-by: Pavel Tatashin <pasha.tatashin@soleen.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
Reviewed-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Shile Zhang <shile.zhang@linux.alibaba.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
Cc: Shile Zhang <shile.zhang@linux.alibaba.com>
Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Alexander Duyck <alexander.duyck@gmail.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Oscar Salvador <osalvador@suse.de>
Signed-off-by: David Hildenbrand <david@redhat.com>
---

This is just a resend of patch #2 of
https://lkml.kernel.org/r/20200401104156.11564-3-david@redhat.com

Not that patch #1 was replaced by
https://lkml.kernel.org/r/20200403140952.17177-4-pasha.tatashin@soleen.com

---
 mm/page_alloc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1385d786a01a..ae1545246b69 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1607,6 +1607,7 @@ void set_zone_contiguous(struct zone *zone)
 		if (!__pageblock_pfn_to_page(block_start_pfn,
 					     block_end_pfn, zone))
 			return;
+		cond_resched();
 	}
 
 	/* We confirm that there is no hole */
-- 
2.25.1

Re: [PATCH v2] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()

[Andrew Morton]

On Thu, 16 Apr 2020 09:34:17 +0200 David Hildenbrand <david@redhat.com> wrote:

> Without CONFIG_PREEMPT, it can happen that we get soft lockups detected,
> e.g., while booting up.
> 
> ...
> 
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1607,6 +1607,7 @@ void set_zone_contiguous(struct zone *zone)
>  		if (!__pageblock_pfn_to_page(block_start_pfn,
>  					     block_end_pfn, zone))
>  			return;
> +		cond_resched();
>  	}
>  
>  	/* We confirm that there is no hole */

I added cc:stable to this one.  Please let me know if that wasn't a
good idea.

Re: [PATCH v2] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()

[Michal Hocko]

On Fri 17-04-20 15:12:47, Andrew Morton wrote:
> On Thu, 16 Apr 2020 09:34:17 +0200 David Hildenbrand <david@redhat.com> wrote:
> 
> > Without CONFIG_PREEMPT, it can happen that we get soft lockups detected,
> > e.g., while booting up.
> > 
> > ...
> > 
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -1607,6 +1607,7 @@ void set_zone_contiguous(struct zone *zone)
> >  		if (!__pageblock_pfn_to_page(block_start_pfn,
> >  					     block_end_pfn, zone))
> >  			return;
> > +		cond_resched();
> >  	}
> >  
> >  	/* We confirm that there is no hole */
> 
> I added cc:stable to this one.  Please let me know if that wasn't a
> good idea.

Really large memory setups tend to run on distribution kernels so
backporting to old kernels doesn't really harm.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH v2] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous()

[David Hildenbrand]

On 20.04.20 09:34, Michal Hocko wrote:
> On Fri 17-04-20 15:12:47, Andrew Morton wrote:
>> On Thu, 16 Apr 2020 09:34:17 +0200 David Hildenbrand <david@redhat.com> wrote:
>>
>>> Without CONFIG_PREEMPT, it can happen that we get soft lockups detected,
>>> e.g., while booting up.
>>>
>>> ...
>>>
>>> --- a/mm/page_alloc.c
>>> +++ b/mm/page_alloc.c
>>> @@ -1607,6 +1607,7 @@ void set_zone_contiguous(struct zone *zone)
>>>  		if (!__pageblock_pfn_to_page(block_start_pfn,
>>>  					     block_end_pfn, zone))
>>>  			return;
>>> +		cond_resched();
>>>  	}
>>>  
>>>  	/* We confirm that there is no hole */
>>
>> I added cc:stable to this one.  Please let me know if that wasn't a
>> good idea.
> 
> Really large memory setups tend to run on distribution kernels so
> backporting to old kernels doesn't really harm.
> 

Yeah, shouldn't hurt.

-- 
Thanks,

David / dhildenb