Please apply to v5.10 stable: 29368e093921 ("x86/smpboot: Move rcu_cpu_starting() earlier")

Please apply to v5.10 stable: 29368e093921 ("x86/smpboot: Move rcu_cpu_starting() earlier")

[Joel Fernandes]

Hello,

Please apply to the stable v5.10 kernel, the commit: 29368e093921
("x86/smpboot:  Move rcu_cpu_starting() earlier").

It made it into the mainline in 5.11.  I am able to reproduce the
following splat without it on v5.10 stable, which is identical to the
one that the commit fixed:

[   42.628511] =============================
[   42.628512] WARNING: suspicious RCU usage
[   42.628513] 5.10.156+ #7 Not tainted
[   42.628514] -----------------------------
[   42.628516] kernel/locking/lockdep.c:3621 RCU-list traversed in
non-reader section!!
[   42.628516]
[   42.628517] other info that might help us debug this:
[   42.628518]
[   42.628519]
[   42.628519] RCU used illegally from offline CPU!
[   42.628520] rcu_scheduler_active = 1, debug_locks = 1
[   42.628521] no locks held by swapper/1/0.
[   42.628522]
[   42.628522] stack backtrace:
[   42.628523] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.10.156+ #7
[   42.628540] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
BIOS 1.16.0-debian-1.16.0-4 04/01/2014
[   42.628541] Call Trace:
[   42.628541]
[   42.628542] =============================
[   42.628543] WARNING: suspicious RCU usage
[   42.628544] 5.10.156+ #7 Not tainted
[   42.628561] -----------------------------
[   42.628563] kernel/kprobes.c:300 RCU-list traversed in non-reader section!!
[   42.628563]
[   42.628564] other info that might help us debug this:
[   42.628565]
[   42.628566]
[   42.628567] RCU used illegally from offline CPU!
[   42.628568] rcu_scheduler_active = 1, debug_locks = 1
[   42.628569] no locks held by swapper/1/0.
[   42.628570]
[   42.628570] stack backtrace:
[   42.628571] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.10.156+ #7
[   42.628573] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
BIOS 1.16.0-debian-1.16.0-4 04/01/2014
[   42.628573] Call Trace:
[   42.628574]  dump_stack+0x77/0x9b
[   42.628575]  __is_insn_slot_addr+0x156/0x170
[   42.628576]  kernel_text_address+0xb1/0xe0
[   42.628577]  ? get_stack_info+0x2b/0x80
[   42.628578]  __kernel_text_address+0x9/0x40
[   42.628578]  show_trace_log_lvl+0x223/0x2f0
[   42.628579]  ? dump_stack+0x77/0x9b
[   42.628580]  dump_stack+0x77/0x9b
[   42.628581]  __lock_acquire.cold+0x326/0x330
[   42.628581]  lock_acquire+0xbd/0x2a0
[   42.628582]  ? vprintk_emit+0x6c/0x310
[   42.628583]  _raw_spin_lock+0x27/0x40
[   42.628584]  ? vprintk_emit+0x6c/0x310
[   42.628584]  vprintk_emit+0x6c/0x310
[   42.628585]  printk+0x63/0x7e
[   42.628586]  start_secondary+0x1c/0xf0
[   42.628587]  secondary_startup_64_no_verify+0xc2/0xcb
[   42.628588]  dump_stack+0x77/0x9b
[   42.628588]  __lock_acquire.cold+0x326/0x330
[   42.628589]  lock_acquire+0xbd/0x2a0
[   42.628590]  ? vprintk_emit+0x6c/0x310
[   42.628591]  _raw_spin_lock+0x27/0x40
[   42.628591]  ? vprintk_emit+0x6c/0x310
[   42.628592]  vprintk_emit+0x6c/0x310

Thanks,

 - Joel

Re: Please apply to v5.10 stable: 29368e093921 ("x86/smpboot: Move rcu_cpu_starting() earlier")

[Joel Fernandes]

On Tue, Dec 13, 2022 at 12:17 PM Joel Fernandes <joel@joelfernandes.org> wrote:
>
> Hello,
>
> Please apply to the stable v5.10 kernel, the commit: 29368e093921
> ("x86/smpboot:  Move rcu_cpu_starting() earlier").

While I am confident this fixes it, I started an overnight test of all
rcutorture scenarios. It cherry-picks cleanly.

I will test 5.4 as well as it applies there.

 - Joel

>
> It made it into the mainline in 5.11.  I am able to reproduce the
> following splat without it on v5.10 stable, which is identical to the
> one that the commit fixed:
>
> [   42.628511] =============================
> [   42.628512] WARNING: suspicious RCU usage
> [   42.628513] 5.10.156+ #7 Not tainted
> [   42.628514] -----------------------------
> [   42.628516] kernel/locking/lockdep.c:3621 RCU-list traversed in
> non-reader section!!
> [   42.628516]
> [   42.628517] other info that might help us debug this:
> [   42.628518]
> [   42.628519]
> [   42.628519] RCU used illegally from offline CPU!
> [   42.628520] rcu_scheduler_active = 1, debug_locks = 1
> [   42.628521] no locks held by swapper/1/0.
> [   42.628522]
> [   42.628522] stack backtrace:
> [   42.628523] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.10.156+ #7
> [   42.628540] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
> BIOS 1.16.0-debian-1.16.0-4 04/01/2014
> [   42.628541] Call Trace:
> [   42.628541]
> [   42.628542] =============================
> [   42.628543] WARNING: suspicious RCU usage
> [   42.628544] 5.10.156+ #7 Not tainted
> [   42.628561] -----------------------------
> [   42.628563] kernel/kprobes.c:300 RCU-list traversed in non-reader section!!
> [   42.628563]
> [   42.628564] other info that might help us debug this:
> [   42.628565]
> [   42.628566]
> [   42.628567] RCU used illegally from offline CPU!
> [   42.628568] rcu_scheduler_active = 1, debug_locks = 1
> [   42.628569] no locks held by swapper/1/0.
> [   42.628570]
> [   42.628570] stack backtrace:
> [   42.628571] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.10.156+ #7
> [   42.628573] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
> BIOS 1.16.0-debian-1.16.0-4 04/01/2014
> [   42.628573] Call Trace:
> [   42.628574]  dump_stack+0x77/0x9b
> [   42.628575]  __is_insn_slot_addr+0x156/0x170
> [   42.628576]  kernel_text_address+0xb1/0xe0
> [   42.628577]  ? get_stack_info+0x2b/0x80
> [   42.628578]  __kernel_text_address+0x9/0x40
> [   42.628578]  show_trace_log_lvl+0x223/0x2f0
> [   42.628579]  ? dump_stack+0x77/0x9b
> [   42.628580]  dump_stack+0x77/0x9b
> [   42.628581]  __lock_acquire.cold+0x326/0x330
> [   42.628581]  lock_acquire+0xbd/0x2a0
> [   42.628582]  ? vprintk_emit+0x6c/0x310
> [   42.628583]  _raw_spin_lock+0x27/0x40
> [   42.628584]  ? vprintk_emit+0x6c/0x310
> [   42.628584]  vprintk_emit+0x6c/0x310
> [   42.628585]  printk+0x63/0x7e
> [   42.628586]  start_secondary+0x1c/0xf0
> [   42.628587]  secondary_startup_64_no_verify+0xc2/0xcb
> [   42.628588]  dump_stack+0x77/0x9b
> [   42.628588]  __lock_acquire.cold+0x326/0x330
> [   42.628589]  lock_acquire+0xbd/0x2a0
> [   42.628590]  ? vprintk_emit+0x6c/0x310
> [   42.628591]  _raw_spin_lock+0x27/0x40
> [   42.628591]  ? vprintk_emit+0x6c/0x310
> [   42.628592]  vprintk_emit+0x6c/0x310
>
> Thanks,
>
>  - Joel

Re: Please apply to v5.10 stable: 29368e093921 ("x86/smpboot: Move rcu_cpu_starting() earlier")

[Greg KH]

On Tue, Dec 13, 2022 at 12:17:20PM -0500, Joel Fernandes wrote:
> Hello,
> 
> Please apply to the stable v5.10 kernel, the commit: 29368e093921
> ("x86/smpboot:  Move rcu_cpu_starting() earlier").
> 
> It made it into the mainline in 5.11.  I am able to reproduce the
> following splat without it on v5.10 stable, which is identical to the
> one that the commit fixed:

Now queued up, thanks.

greg k-h

Re: Please apply to v5.10 stable: 29368e093921 ("x86/smpboot: Move rcu_cpu_starting() earlier")

[Joel Fernandes]

On Wed, Dec 14, 2022 at 11:02 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Tue, Dec 13, 2022 at 12:17:20PM -0500, Joel Fernandes wrote:
> > Hello,
> >
> > Please apply to the stable v5.10 kernel, the commit: 29368e093921
> > ("x86/smpboot:  Move rcu_cpu_starting() earlier").
> >
> > It made it into the mainline in 5.11.  I am able to reproduce the
> > following splat without it on v5.10 stable, which is identical to the
> > one that the commit fixed:
>
> Now queued up, thanks.

Thanks, Greg! I confirmed with overnight testing that the patch fixes the splat.

 - Joel