Re: [Bug 213299] New: kswapd Regression as of 5.13-rc3, eats 100% of CPU for no reason

Re: [Bug 213299] New: kswapd Regression as of 5.13-rc3, eats 100% of CPU for no reason

[Andrew Morton]

(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Mon, 31 May 2021 18:09:38 +0000 bugzilla-daemon@bugzilla.kernel.org wrote:

> https://bugzilla.kernel.org/show_bug.cgi?id=213299
> 
>             Bug ID: 213299
>            Summary: kswapd Regression as of 5.13-rc3, eats 100% of CPU for
>                     no reason
>            Product: Memory Management
>            Version: 2.5
>     Kernel Version: 5.13-rc3, 5.13-rc4
>           Hardware: All
>                 OS: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: Slab Allocator
>           Assignee: akpm@linux-foundation.org
>           Reporter: gardotd426@gmail.com
>         Regression: No
> 
> I've seen reports of kswapd0 eating up CPU, but this doesn't fit those
> descriptions. 
> 
>  - The bug is not present on 5.12.
> 
>  - It happens immediately on boot, CPU usage goes to 100 on all 24 threads of a
> Ryzen 5900X
> 
>  - The only solution is to run `sudo pkill -9 kswapd0`. swapoff does nothing,
> nor does `echo 1 > /proc/sys/vm/drop_caches`. 
> 
>  - I have a swap partition and 32GB of RAM. Like I said, this happens on boot,
> so nowhere *near* 32GB of RAM is being taken. And no swap is being used. This
> is clearly an error in kswapd, it's like it goes completely bonkers. 5.13-rc2
> didn't have it, nor did rc1, and 5.12 doesn't have it either. 
> 
> 
> System Info:
> 
> Arch Linux
> Kernel: 5.13-rc3, 5.13-rc4
> Ryzen 9 5900X
> 32GB DDR4-3600
> 40GB Swap partition on SSD
> 
> sudo dmesg | grep -i swap only gives these three lines so far (after already
> killing the process and about 5 min. after boot):
> 
> [    0.093277] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user
> pointer sanitization
> [    0.334800] zswap: loaded using pool lz4/z3fold
> [    2.941371] Adding 41502716k swap on /dev/sdb3.  Priority:-2 extents:1
> across:41502716k SSFS
> 
> -- 
> You may reply to this email to add a comment.
> 
> You are receiving this mail because:
> You are the assignee for the bug.

Re: [Bug 213299] New: kswapd Regression as of 5.13-rc3, eats 100% of CPU for no reason

[Johannes Weiner]

On Mon, May 31, 2021 at 02:33:29PM -0700, Andrew Morton wrote:
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> On Mon, 31 May 2021 18:09:38 +0000 bugzilla-daemon@bugzilla.kernel.org wrote:
> 
> > https://bugzilla.kernel.org/show_bug.cgi?id=213299
> > 
> >             Bug ID: 213299
> >            Summary: kswapd Regression as of 5.13-rc3, eats 100% of CPU for
> >                     no reason
> >            Product: Memory Management
> >            Version: 2.5
> >     Kernel Version: 5.13-rc3, 5.13-rc4
> >           Hardware: All
> >                 OS: Linux
> >               Tree: Mainline
> >             Status: NEW
> >           Severity: normal
> >           Priority: P1
> >          Component: Slab Allocator
> >           Assignee: akpm@linux-foundation.org
> >           Reporter: gardotd426@gmail.com
> >         Regression: No
> > 
> > I've seen reports of kswapd0 eating up CPU, but this doesn't fit those
> > descriptions. 
> > 
> >  - The bug is not present on 5.12.
> > 
> >  - It happens immediately on boot, CPU usage goes to 100 on all 24 threads of a
> > Ryzen 5900X
> > 
> >  - The only solution is to run `sudo pkill -9 kswapd0`. swapoff does nothing,
> > nor does `echo 1 > /proc/sys/vm/drop_caches`. 
> > 
> >  - I have a swap partition and 32GB of RAM. Like I said, this happens on boot,
> > so nowhere *near* 32GB of RAM is being taken. And no swap is being used. This
> > is clearly an error in kswapd, it's like it goes completely bonkers. 5.13-rc2
> > didn't have it, nor did rc1, and 5.12 doesn't have it either.

Does it trigger (or not trigger) reliably with these version?

There aren't any obvious mm changes in that window. If it reproduces
reliably during boot, can you please git bisect it?

Re: [Bug 213299] New: kswapd Regression as of 5.13-rc3, eats 100% of CPU for no reason

[Chris Down]

Can you also please show /proc/pid/stack for the offending kswapd, and what 
your watermark settings are?