LinuxPPC-Dev Archive on lore.kernel.org
 help / color / Atom feed
* [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5
       [not found] <bug-195755-206035@https.bugzilla.kernel.org/>
@ 2019-09-23  0:07 ` bugzilla-daemon
  2019-09-30  9:35 ` bugzilla-daemon
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 5+ messages in thread
From: bugzilla-daemon @ 2019-09-23  0:07 UTC (permalink / raw)
  To: linuxppc-dev

https://bugzilla.kernel.org/show_bug.cgi?id=195755

Erhard F. (erhard_f@mailbox.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |CLOSED
         Resolution|---                         |OBSOLETE

--- Comment #26 from Erhard F. (erhard_f@mailbox.org) ---
Have not seen this again on the G5 7,2 since kernel 5.2.x, nor 5.3.x.

Will close the bug for now and hopefully it does not return. ;)

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5
       [not found] <bug-195755-206035@https.bugzilla.kernel.org/>
  2019-09-23  0:07 ` [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5 bugzilla-daemon
@ 2019-09-30  9:35 ` bugzilla-daemon
  2019-09-30 10:50 ` bugzilla-daemon
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 5+ messages in thread
From: bugzilla-daemon @ 2019-09-30  9:35 UTC (permalink / raw)
  To: linuxppc-dev

https://bugzilla.kernel.org/show_bug.cgi?id=195755

Martin Zaharinov (micron10@gmail.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |micron10@gmail.com

--- Comment #27 from Martin Zaharinov (micron10@gmail.com) ---
I see this bug is back in kernel 5.3.x 

Sep 28 15:40:01  [  611.520537][    C0] rcu: INFO: rcu_sched self-detected
stall on CPU
Sep 28 15:40:01  [  611.520992][    C0] rcu:    0-...!: (5998 ticks this GP)
idle=ace/1/0x4000000000000004 softirq=46973/46973 fqs=73
Sep 28 15:40:01  [  611.521721][    C0]         (t=6000 jiffies g=83505 q=2615)
Sep 28 15:40:01  [  611.522076][    C0] rcu: rcu_sched kthread starved for 5853
jiffies! g83505 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
Sep 28 15:40:01  [  611.522858][    C0] rcu: RCU grace-period kthread stack
dump:
Sep 28 15:40:01  [  611.523269][    C0] rcu_sched       I    0    10      2
0x80004000
Sep 28 15:40:01  [  611.523710][    C0] Call Trace:
Sep 28 15:40:01  [  611.523938][    C0]  ? __schedule+0x1db/0x5d0
Sep 28 15:40:01  [  611.524251][    C0]  schedule+0x34/0xa0
Sep 28 15:40:01  [  611.524525][    C0]  schedule_timeout+0x1f7/0x3e0
Sep 28 15:40:01  [  611.524861][    C0]  ? collect_expired_timers+0x270/0x270
Sep 28 15:40:01  [  611.525248][    C0]  rcu_gp_kthread+0x49f/0xea0
Sep 28 15:40:01  [  611.525572][    C0]  kthread+0x11a/0x150
Sep 28 15:40:01  [  611.525851][    C0]  ? rcu_report_qs_rnp+0x160/0x160
Sep 28 15:40:01  [  611.534554][    C0]  ? kthread_park+0x70/0x70
Sep 28 15:40:01  [  611.543250][    C0]  ret_from_fork+0x1f/0x30
Sep 28 15:40:01  [  611.551953][    C0] NMI backtrace for cpu 0
Sep 28 15:40:01  [  611.560402][    C0] CPU: 0 PID: 10870 Comm: ifmgr Tainted:
G           O      5.3.1 #1
Sep 28 15:40:01  [  611.576928][    C0] Hardware name: HP ProLiant ML110 G7,
BIOS J01 08/10/2012
Sep 28 15:40:01  [  611.585641][    C0] Call Trace:
Sep 28 15:40:01  [  611.594279][    C0]  <IRQ>
Sep 28 15:40:01  [  611.602783][    C0]  dump_stack+0x47/0x5a
Sep 28 15:40:01  [  611.611375][    C0]  nmi_cpu_backtrace.cold+0x14/0x53
Sep 28 15:40:01  [  611.620076][    C0]  ? lapic_can_unplug_cpu.cold+0x37/0x37
Sep 28 15:40:01  [  611.628588][    C0] 
nmi_trigger_cpumask_backtrace+0xef/0xf1
Sep 28 15:40:01  [  611.637040][    C0]  rcu_dump_cpu_stacks+0x8d/0xbb
Sep 28 15:40:01  [  611.645425][    C0]  rcu_sched_clock_irq.cold+0x1b1/0x3a6
Sep 28 15:40:01  [  611.653707][    C0]  update_process_times+0x5b/0x90
Sep 28 15:40:01  [  611.661857][    C0]  tick_sched_timer+0x65/0x100
Sep 28 15:40:01  [  611.669733][    C0]  ? get_cpu_iowait_time_us+0x100/0x100
Sep 28 15:40:01  [  611.677571][    C0]  __hrtimer_run_queues+0x105/0x1b0
Sep 28 15:40:01  [  611.685267][    C0]  hrtimer_interrupt+0x10b/0x3d0
Sep 28 15:40:01  [  611.692709][    C0]  smp_apic_timer_interrupt+0x50/0x80
Sep 28 15:40:01  [  611.700044][    C0]  apic_timer_interrupt+0xf/0x20
Sep 28 15:40:01  [  611.707293][    C0] RIP:
0010:native_queued_spin_lock_slowpath+0x63/0x1d0
Sep 28 15:40:01  [  611.714847][    C0] Code: ba 2f 08 0f 92 c0 0f b6 c0 c1 e0
08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 75 1c 85 c0 74 0f 8b 07 84 c0 74 09
0f ae e8 8b 07 <84> c0 75 f7 b8 01 00 00 00 66 89 07 c3 f6 c4 01 75 04 c6 47 01
00
Sep 28 15:40:01  [  611.738876][    C0] RSP: 0000:ffffc90000003ad8 EFLAGS:
00000202 ORIG_RAX: ffffffffffffff13
Sep 28 15:40:01  [  611.755477][    C0] RAX: 0000000000080101 RBX:
0000000000000000 RCX: 0000000000000000
Sep 28 15:40:01  [  611.772993][    C0] RDX: 0000000000000000 RSI:
0000000000000000 RDI: ffff8882dcf72080
Sep 28 15:40:01  [  611.791481][    C0] RBP: ffff8882c7527800 R08:
ffff8882c7765890 R09: 0000000000000000
Sep 28 15:40:01  [  611.810932][    C0] R10: 0000000000000000 R11:
ffffffff817b5d30 R12: ffff8882db3a7000
Sep 28 15:40:01  [  611.830825][    C0] R13: ffff8882dcf72000 R14:
ffff8882dcf71800 R15: ffff8882db3a7000
Sep 28 15:40:01  [  611.851397][    C0]  ? apic_timer_interrupt+0xa/0x20
Sep 28 15:40:01  [  611.862019][    C0]  ? ip_fragment.constprop.0+0x80/0x80
Sep 28 15:40:01  [  611.872654][    C0]  _raw_spin_lock+0x15/0x20
Sep 28 15:40:01  [  611.883227][    C0]  __imq_nf_queue+0x3a5/0x6d0 [imq]
Sep 28 15:40:01  [  611.893786][    C0]  imq_nf_queue+0x326/0x423 [imq]
Sep 28 15:40:01  [  611.904278][    C0]  nf_queue+0x1b8/0x320
Sep 28 15:40:01  [  611.914713][    C0]  nf_hook_slow+0x82/0xe0
Sep 28 15:40:01  [  611.925061][    C0]  ip_output+0xb3/0x280
Sep 28 15:40:01  [  611.935355][    C0]  ? ip_fragment.constprop.0+0x80/0x80
Sep 28 15:40:01  [  611.945745][    C0]  ip_push_pending_frames+0x52/0x80
Sep 28 15:40:01  [  611.956106][    C0]  __icmp_send+0x4ac/0x5a0
Sep 28 15:40:01  [  611.966513][    C0]  ? nf_xfrm_me_harder+0xf4/0x110
[nf_nat]
Sep 28 15:40:02  [  611.976960][    C0]  ? ip_fragment.constprop.0+0x6d/0x80
Sep 28 15:40:02  [  611.987406][    C0]  ip_fragment.constprop.0+0x6d/0x80
Sep 28 15:40:02  [  611.997743][    C0]  nf_reinject+0x15d/0x18f
Sep 28 15:40:02  [  612.007855][    C0]  imq_dev_xmit+0x70/0xb0 [imq]
Sep 28 15:40:02  [  612.017725][    C0]  dev_hard_start_xmit+0x96/0x130
Sep 28 15:40:02  [  612.027513][    C0]  __qdisc_run+0x1bf/0x2d0
Sep 28 15:40:02  [  612.037043][    C0]  net_tx_action+0xd3/0x150
Sep 28 15:40:02  [  612.046310][    C0]  __do_softirq+0xc8/0x206
Sep 28 15:40:02  [  612.055317][    C0]  irq_exit+0xa7/0xf0
Sep 28 15:40:02  [  612.064020][    C0]  smp_apic_timer_interrupt+0x55/0x80
Sep 28 15:40:02  [  612.072615][    C0]  apic_timer_interrupt+0xf/0x20
Sep 28 15:40:02  [  612.080970][    C0]  </IRQ>
Sep 28 15:40:02  [  612.088957][    C0] RIP: 0033:0x55e6e4
Sep 28 15:40:02  [  612.096796][    C0] Code: 48 01 d0 48 89 c2 4c 01 c0 4c 0f
be 47 fb 48 c1 e2 05 48 01 c2 48 89 d0 48 c1 e0 05 4c 01 c2 4c 0f be 47 fc 48
01 d0 48 89 c2 <48> c1 e2 05 4c 01 c0 48 01 d0 48 0f be 57 fd 49 89 c0 49 c1 e0
05
Sep 28 15:40:02  [  612.120192][    C0] RSP: 002b:00007ffc8f1f1fd8 EFLAGS:
00000287 ORIG_RAX: ffffffffffffff13
Sep 28 15:40:02  [  612.135465][    C0] RAX: a888682638287cbb RBX:
00000000000002d0 RCX: 00007fb6b4431f88
Sep 28 15:40:02  [  612.151470][    C0] RDX: a888682638287cbb RSI:
000000000000001a RDI: 00007fb6b4431f80
Sep 28 15:40:02  [  612.168344][    C0] RBP: 0000000000001680 R08:
000000000000004c R09: 000000000000001a
Sep 28 15:40:02  [  612.186162][    C0] R10: 00007fb6b8541948 R11:
00007fb6b8a930a0 R12: 0000000000000090
Sep 28 15:40:02  [  612.205010][    C0] R13: 00007fb6b442daa0 R14:
00007fb6b41044d0 R15: 0000000001619440

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5
       [not found] <bug-195755-206035@https.bugzilla.kernel.org/>
  2019-09-23  0:07 ` [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5 bugzilla-daemon
  2019-09-30  9:35 ` bugzilla-daemon
@ 2019-09-30 10:50 ` bugzilla-daemon
  2020-09-24  7:41 ` bugzilla-daemon
  2020-10-15 18:31 ` bugzilla-daemon
  4 siblings, 0 replies; 5+ messages in thread
From: bugzilla-daemon @ 2019-09-30 10:50 UTC (permalink / raw)
  To: linuxppc-dev

https://bugzilla.kernel.org/show_bug.cgi?id=195755

Erhard F. (erhard_f@mailbox.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |REOPENED
         Resolution|OBSOLETE                    |---

--- Comment #28 from Erhard F. (erhard_f@mailbox.org) ---
Mkay... seems I was only lucky with my top-notch G5 hardware not having seen
this bug for quite some time.

Re-opening 'cause this is already the 2nd report this bug (still) affects other
hardware.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5
       [not found] <bug-195755-206035@https.bugzilla.kernel.org/>
                   ` (2 preceding siblings ...)
  2019-09-30 10:50 ` bugzilla-daemon
@ 2020-09-24  7:41 ` bugzilla-daemon
  2020-10-15 18:31 ` bugzilla-daemon
  4 siblings, 0 replies; 5+ messages in thread
From: bugzilla-daemon @ 2020-09-24  7:41 UTC (permalink / raw)
  To: linuxppc-dev

https://bugzilla.kernel.org/show_bug.cgi?id=195755

Nigel Reed (nigel@nigelreed.net) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nigel@nigelreed.net

--- Comment #29 from Nigel Reed (nigel@nigelreed.net) ---
I know this is old but I have been having some issues for a while, I was
finally able to get something useful:

[165716.089703] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[165716.095949] rcu:    1-...!: (0 ticks this GP) idle=354/0/0x0
softirq=2154363/2154363 fqs=0
[165716.104512] rcu:    3-...!: (0 ticks this GP) idle=29c/0/0x0
softirq=883832/883832 fqs=0
[165716.112873] rcu:    4-...!: (8 GPs behind) idle=ad8/0/0x0
softirq=2165586/2165586 fqs=0
[165716.121179] rcu:    9-...!: (9 GPs behind) idle=acc/0/0x0
softirq=1340600/1340600 fqs=0
[165716.129467] rcu:    11-...!: (2 GPs behind) idle=a18/0/0x0
softirq=4538536/4538537 fqs=0
[165716.137828] rcu:    12-...!: (0 ticks this GP) idle=870/0/0x0
softirq=2158040/2158040 fqs=0
[165775.697763] rcu: rcu_sched kthread starved for 29898 jiffies! g36134941
f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=3
[165775.709013] rcu: RCU grace-period kthread stack dump:
[165837.494623] watchdog: BUG: soft lockup - CPU#6 stuck for 23s!
[(resolved):52315]
[165865.494840] watchdog: BUG: soft lockup - CPU#6 stuck for 23s!
[(resolved):52315]

This happened just after freshclam ran but I don't know if it's related.

This is with a Ryzen 7 1800X CPU.
5.4.0-48-generic #52-Ubuntu

I thought I had sysrq configured but it seems not so I can't really provide any
more information, other than this is driving me crazy.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5
       [not found] <bug-195755-206035@https.bugzilla.kernel.org/>
                   ` (3 preceding siblings ...)
  2020-09-24  7:41 ` bugzilla-daemon
@ 2020-10-15 18:31 ` bugzilla-daemon
  4 siblings, 0 replies; 5+ messages in thread
From: bugzilla-daemon @ 2020-10-15 18:31 UTC (permalink / raw)
  To: linuxppc-dev

https://bugzilla.kernel.org/show_bug.cgi?id=195755

Marco Descher (marco@descher.at) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |marco@descher.at

--- Comment #30 from Marco Descher (marco@descher.at) ---
I experienced this problem today, a freeze on an 

processor       : 3
vendor_id       : AuthenticAMD
cpu family      : 22
model           : 48
model name      : AMD GX-412TC SOC

on Debian 10  4.19.0-11-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64
GNU/Linux

with more and more sysrq messages coming up resulting in the following syslog
entry

Oct 15 11:43:45 gate kernel: [1545118.045973] rcu: INFO: rcu_sched detected
stalls on CPUs/tasks:

leading to the system becoming unreachable. Only after a reboot this works
again.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <bug-195755-206035@https.bugzilla.kernel.org/>
2019-09-23  0:07 ` [Bug 195755] rcu_sched detected stalls on CPUs/tasks: (detected by 0, t=6302 jiffies, g=11405, c=11404, q=1880), ppc64, G5 bugzilla-daemon
2019-09-30  9:35 ` bugzilla-daemon
2019-09-30 10:50 ` bugzilla-daemon
2020-09-24  7:41 ` bugzilla-daemon
2020-10-15 18:31 ` bugzilla-daemon

LinuxPPC-Dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linuxppc-dev/0 linuxppc-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linuxppc-dev linuxppc-dev/ https://lore.kernel.org/linuxppc-dev \
		linuxppc-dev@lists.ozlabs.org linuxppc-dev@ozlabs.org
	public-inbox-index linuxppc-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.ozlabs.lists.linuxppc-dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git