linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* RCU stalls in linux-next
@ 2012-03-13 13:48 Dan Carpenter
  2012-03-13 14:04 ` Dan Carpenter
  2012-03-13 14:33 ` Paul E. McKenney
  0 siblings, 2 replies; 4+ messages in thread
From: Dan Carpenter @ 2012-03-13 13:48 UTC (permalink / raw)
  To: Paul E. McKenney; +Cc: linux-kernel

I've been getting RCU hangs in linux-next.

Also sometimes, when I'm building my smatch database after a kernel
compile, my system hangs.  I'm not certain if the two things are related.

regards,
dan carpenter

Mar 13 14:32:11 elgon kernel: [265405.604199] Pid: 665, comm: kswapd0 Not tainted 3.3.0-rc6-next-20120308+ #141
Mar 13 14:32:11 elgon kernel: [265405.604200] Call Trace:
Mar 13 14:32:11 elgon kernel: [265405.604201]  <IRQ>  [<ffffffff810ab9da>] __rcu_pending+0x19a/0x4d0
Mar 13 14:32:11 elgon kernel: [265405.604208]  [<ffffffff810ac1d0>] rcu_check_callbacks+0xb0/0x1a0
Mar 13 14:32:11 elgon kernel: [265405.604210]  [<ffffffff81044293>] update_process_times+0x43/0x80
Mar 13 14:32:11 elgon kernel: [265405.604220]  [<ffffffff8107eb0f>] tick_sched_timer+0x5f/0xb0
Mar 13 14:32:11 elgon kernel: [265405.604230]  [<ffffffff81058f68>] __run_hrtimer+0x78/0x1d0
Mar 13 14:32:11 elgon kernel: [265405.604232]  [<ffffffff8107eab0>] ? tick_nohz_handler+0xf0/0xf0
Mar 13 14:32:11 elgon kernel: [265405.604234]  [<ffffffff8103ba91>] ? __do_softirq+0xf1/0x210
Mar 13 14:32:11 elgon kernel: [265405.604235]  [<ffffffff81059843>] hrtimer_interrupt+0xe3/0x200
Mar 13 14:32:11 elgon kernel: [265405.604238]  [<ffffffff8170c74c>] ? call_softirq+0x1c/0x30
Mar 13 14:32:11 elgon kernel: [265405.604241]  [<ffffffff8101f7c4>] smp_apic_timer_interrupt+0x64/0xa0
Mar 13 14:32:11 elgon kernel: [265405.604243]  [<ffffffff8170be07>] apic_timer_interrupt+0x67/0x70
Mar 13 14:32:11 elgon kernel: [265405.604244]  <EOI>  [<ffffffff810d87ac>] ? zone_watermark_ok_safe+0x8c/0x170
Mar 13 14:32:11 elgon kernel: [265405.604248]  [<ffffffff810e73c8>] balance_pgdat+0x1a8/0x680
Mar 13 14:32:11 elgon kernel: [265405.604250]  [<ffffffff810e7a08>] kswapd+0x168/0x3f0
Mar 13 14:32:11 elgon kernel: [265405.604253]  [<ffffffff81702916>] ? __schedule+0x3a6/0x750
Mar 13 14:32:11 elgon kernel: [265405.604255]  [<ffffffff810556b0>] ? add_wait_queue+0x60/0x60
Mar 13 14:32:11 elgon kernel: [265405.604256]  [<ffffffff810e78a0>] ? balance_pgdat+0x680/0x680
Mar 13 14:32:11 elgon kernel: [265405.604258]  [<ffffffff81054c7e>] kthread+0x8e/0xa0
Mar 13 14:32:11 elgon kernel: [265405.604260]  [<ffffffff8170c654>] kernel_thread_helper+0x4/0x10
Mar 13 14:32:11 elgon kernel: [265405.604262]  [<ffffffff81054bf0>] ? kthread_freezable_should_stop+0x70/0x70
Mar 13 14:32:11 elgon kernel: [265405.604264]  [<ffffffff8170c650>] ? gs_change+0xb/0xb
Mar 13 14:35:11 elgon kernel: [265585.490971] Pid: 665, comm: kswapd0 Not tainted 3.3.0-rc6-next-20120308+ #141
Mar 13 14:35:11 elgon kernel: [265585.490972] Call Trace:
Mar 13 14:35:11 elgon kernel: [265585.490973]  <IRQ>  [<ffffffff810ab9da>] __rcu_pending+0x19a/0x4d0
Mar 13 14:35:11 elgon kernel: [265585.490987]  [<ffffffff810ac1d0>] rcu_check_callbacks+0xb0/0x1a0
Mar 13 14:35:11 elgon kernel: [265585.490989]  [<ffffffff81044293>] update_process_times+0x43/0x80
Mar 13 14:35:11 elgon kernel: [265585.490991]  [<ffffffff8107eb0f>] tick_sched_timer+0x5f/0xb0
Mar 13 14:35:11 elgon kernel: [265585.490994]  [<ffffffff81058f68>] __run_hrtimer+0x78/0x1d0
Mar 13 14:35:11 elgon kernel: [265585.490995]  [<ffffffff8107eab0>] ? tick_nohz_handler+0xf0/0xf0
Mar 13 14:35:11 elgon kernel: [265585.490997]  [<ffffffff8103ba91>] ? __do_softirq+0xf1/0x210
Mar 13 14:35:11 elgon kernel: [265585.490999]  [<ffffffff81059843>] hrtimer_interrupt+0xe3/0x200
Mar 13 14:35:11 elgon kernel: [265585.491002]  [<ffffffff8170c74c>] ? call_softirq+0x1c/0x30
Mar 13 14:35:11 elgon kernel: [265585.491005]  [<ffffffff8101f7c4>] smp_apic_timer_interrupt+0x64/0xa0
Mar 13 14:35:11 elgon kernel: [265585.491007]  [<ffffffff8170be07>] apic_timer_interrupt+0x67/0x70
Mar 13 14:35:11 elgon kernel: [265585.491008]  <EOI>  [<ffffffff810d876d>] ? zone_watermark_ok_safe+0x4d/0x170
Mar 13 14:35:11 elgon kernel: [265585.491012]  [<ffffffff810e73c8>] balance_pgdat+0x1a8/0x680
Mar 13 14:35:11 elgon kernel: [265585.491014]  [<ffffffff810e7a08>] kswapd+0x168/0x3f0
Mar 13 14:35:11 elgon kernel: [265585.491017]  [<ffffffff81702916>] ? __schedule+0x3a6/0x750
Mar 13 14:35:11 elgon kernel: [265585.491019]  [<ffffffff810556b0>] ? add_wait_queue+0x60/0x60
Mar 13 14:35:11 elgon kernel: [265585.491021]  [<ffffffff810e78a0>] ? balance_pgdat+0x680/0x680
Mar 13 14:35:11 elgon kernel: [265585.491023]  [<ffffffff81054c7e>] kthread+0x8e/0xa0
Mar 13 14:35:11 elgon kernel: [265585.491024]  [<ffffffff8170c654>] kernel_thread_helper+0x4/0x10
Mar 13 14:35:11 elgon kernel: [265585.491026]  [<ffffffff81054bf0>] ? kthread_freezable_should_stop+0x70/0x70
Mar 13 14:35:11 elgon kernel: [265585.491028]  [<ffffffff8170c650>] ? gs_change+0xb/0xb

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: RCU stalls in linux-next
  2012-03-13 13:48 RCU stalls in linux-next Dan Carpenter
@ 2012-03-13 14:04 ` Dan Carpenter
  2012-03-13 14:33 ` Paul E. McKenney
  1 sibling, 0 replies; 4+ messages in thread
From: Dan Carpenter @ 2012-03-13 14:04 UTC (permalink / raw)
  To: Paul E. McKenney; +Cc: linux-kernel

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

On Tue, Mar 13, 2012 at 04:48:23PM +0300, Dan Carpenter wrote:
> I've been getting RCU hangs in linux-next.
> 
> Also sometimes, when I'm building my smatch database after a kernel
> compile, my system hangs.  I'm not certain if the two things are related.

It actually seems to be hanging just after I've built my kernel and
do a:  find -name \*.c.smatch -exec cat \{\} \; > warns.txt

My screen froze but I was able to log in via ssh once and type dmesg
but then I hit tab tab twice to trigger the tab completion and it
hung for good.

I'm still not sure the big hang is related to the RCU stalls...

regards,
dan carpenter

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: RCU stalls in linux-next
  2012-03-13 13:48 RCU stalls in linux-next Dan Carpenter
  2012-03-13 14:04 ` Dan Carpenter
@ 2012-03-13 14:33 ` Paul E. McKenney
  2012-03-14  6:59   ` Dan Carpenter
  1 sibling, 1 reply; 4+ messages in thread
From: Paul E. McKenney @ 2012-03-13 14:33 UTC (permalink / raw)
  To: Dan Carpenter; +Cc: linux-kernel, linux-mm

On Tue, Mar 13, 2012 at 04:48:23PM +0300, Dan Carpenter wrote:
> I've been getting RCU hangs in linux-next.
> 
> Also sometimes, when I'm building my smatch database after a kernel
> compile, my system hangs.  I'm not certain if the two things are related.
> 
> regards,
> dan carpenter
> 
> Mar 13 14:32:11 elgon kernel: [265405.604199] Pid: 665, comm: kswapd0 Not tainted 3.3.0-rc6-next-20120308+ #141
> Mar 13 14:32:11 elgon kernel: [265405.604200] Call Trace:
> Mar 13 14:32:11 elgon kernel: [265405.604201]  <IRQ>  [<ffffffff810ab9da>] __rcu_pending+0x19a/0x4d0
> Mar 13 14:32:11 elgon kernel: [265405.604208]  [<ffffffff810ac1d0>] rcu_check_callbacks+0xb0/0x1a0
> Mar 13 14:32:11 elgon kernel: [265405.604210]  [<ffffffff81044293>] update_process_times+0x43/0x80
> Mar 13 14:32:11 elgon kernel: [265405.604220]  [<ffffffff8107eb0f>] tick_sched_timer+0x5f/0xb0
> Mar 13 14:32:11 elgon kernel: [265405.604230]  [<ffffffff81058f68>] __run_hrtimer+0x78/0x1d0
> Mar 13 14:32:11 elgon kernel: [265405.604232]  [<ffffffff8107eab0>] ? tick_nohz_handler+0xf0/0xf0
> Mar 13 14:32:11 elgon kernel: [265405.604234]  [<ffffffff8103ba91>] ? __do_softirq+0xf1/0x210
> Mar 13 14:32:11 elgon kernel: [265405.604235]  [<ffffffff81059843>] hrtimer_interrupt+0xe3/0x200
> Mar 13 14:32:11 elgon kernel: [265405.604238]  [<ffffffff8170c74c>] ? call_softirq+0x1c/0x30
> Mar 13 14:32:11 elgon kernel: [265405.604241]  [<ffffffff8101f7c4>] smp_apic_timer_interrupt+0x64/0xa0
> Mar 13 14:32:11 elgon kernel: [265405.604243]  [<ffffffff8170be07>] apic_timer_interrupt+0x67/0x70
> Mar 13 14:32:11 elgon kernel: [265405.604244]  <EOI>  [<ffffffff810d87ac>] ? zone_watermark_ok_safe+0x8c/0x170

Looks like kswapd is having a bad hair day, CCing linux-mm to see if they
can help.

							Thanx, Paul

> Mar 13 14:32:11 elgon kernel: [265405.604248]  [<ffffffff810e73c8>] balance_pgdat+0x1a8/0x680
> Mar 13 14:32:11 elgon kernel: [265405.604250]  [<ffffffff810e7a08>] kswapd+0x168/0x3f0
> Mar 13 14:32:11 elgon kernel: [265405.604253]  [<ffffffff81702916>] ? __schedule+0x3a6/0x750
> Mar 13 14:32:11 elgon kernel: [265405.604255]  [<ffffffff810556b0>] ? add_wait_queue+0x60/0x60
> Mar 13 14:32:11 elgon kernel: [265405.604256]  [<ffffffff810e78a0>] ? balance_pgdat+0x680/0x680
> Mar 13 14:32:11 elgon kernel: [265405.604258]  [<ffffffff81054c7e>] kthread+0x8e/0xa0
> Mar 13 14:32:11 elgon kernel: [265405.604260]  [<ffffffff8170c654>] kernel_thread_helper+0x4/0x10
> Mar 13 14:32:11 elgon kernel: [265405.604262]  [<ffffffff81054bf0>] ? kthread_freezable_should_stop+0x70/0x70
> Mar 13 14:32:11 elgon kernel: [265405.604264]  [<ffffffff8170c650>] ? gs_change+0xb/0xb
> Mar 13 14:35:11 elgon kernel: [265585.490971] Pid: 665, comm: kswapd0 Not tainted 3.3.0-rc6-next-20120308+ #141
> Mar 13 14:35:11 elgon kernel: [265585.490972] Call Trace:
> Mar 13 14:35:11 elgon kernel: [265585.490973]  <IRQ>  [<ffffffff810ab9da>] __rcu_pending+0x19a/0x4d0
> Mar 13 14:35:11 elgon kernel: [265585.490987]  [<ffffffff810ac1d0>] rcu_check_callbacks+0xb0/0x1a0
> Mar 13 14:35:11 elgon kernel: [265585.490989]  [<ffffffff81044293>] update_process_times+0x43/0x80
> Mar 13 14:35:11 elgon kernel: [265585.490991]  [<ffffffff8107eb0f>] tick_sched_timer+0x5f/0xb0
> Mar 13 14:35:11 elgon kernel: [265585.490994]  [<ffffffff81058f68>] __run_hrtimer+0x78/0x1d0
> Mar 13 14:35:11 elgon kernel: [265585.490995]  [<ffffffff8107eab0>] ? tick_nohz_handler+0xf0/0xf0
> Mar 13 14:35:11 elgon kernel: [265585.490997]  [<ffffffff8103ba91>] ? __do_softirq+0xf1/0x210
> Mar 13 14:35:11 elgon kernel: [265585.490999]  [<ffffffff81059843>] hrtimer_interrupt+0xe3/0x200
> Mar 13 14:35:11 elgon kernel: [265585.491002]  [<ffffffff8170c74c>] ? call_softirq+0x1c/0x30
> Mar 13 14:35:11 elgon kernel: [265585.491005]  [<ffffffff8101f7c4>] smp_apic_timer_interrupt+0x64/0xa0
> Mar 13 14:35:11 elgon kernel: [265585.491007]  [<ffffffff8170be07>] apic_timer_interrupt+0x67/0x70
> Mar 13 14:35:11 elgon kernel: [265585.491008]  <EOI>  [<ffffffff810d876d>] ? zone_watermark_ok_safe+0x4d/0x170
> Mar 13 14:35:11 elgon kernel: [265585.491012]  [<ffffffff810e73c8>] balance_pgdat+0x1a8/0x680
> Mar 13 14:35:11 elgon kernel: [265585.491014]  [<ffffffff810e7a08>] kswapd+0x168/0x3f0
> Mar 13 14:35:11 elgon kernel: [265585.491017]  [<ffffffff81702916>] ? __schedule+0x3a6/0x750
> Mar 13 14:35:11 elgon kernel: [265585.491019]  [<ffffffff810556b0>] ? add_wait_queue+0x60/0x60
> Mar 13 14:35:11 elgon kernel: [265585.491021]  [<ffffffff810e78a0>] ? balance_pgdat+0x680/0x680
> Mar 13 14:35:11 elgon kernel: [265585.491023]  [<ffffffff81054c7e>] kthread+0x8e/0xa0
> Mar 13 14:35:11 elgon kernel: [265585.491024]  [<ffffffff8170c654>] kernel_thread_helper+0x4/0x10
> Mar 13 14:35:11 elgon kernel: [265585.491026]  [<ffffffff81054bf0>] ? kthread_freezable_should_stop+0x70/0x70
> Mar 13 14:35:11 elgon kernel: [265585.491028]  [<ffffffff8170c650>] ? gs_change+0xb/0xb
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: RCU stalls in linux-next
  2012-03-13 14:33 ` Paul E. McKenney
@ 2012-03-14  6:59   ` Dan Carpenter
  0 siblings, 0 replies; 4+ messages in thread
From: Dan Carpenter @ 2012-03-14  6:59 UTC (permalink / raw)
  To: Paul E. McKenney; +Cc: linux-kernel, linux-mm

[-- Attachment #1: Type: text/plain, Size: 3738 bytes --]

I get these on my netbook as well if I run it for long enough.  I
just read email on that and do the occasional git pull.

regards,
dan carpenter

[ 3906.306118] eth0: no IPv6 routers present
[58395.111474] apt-get used greatest stack depth: 4032 bytes left
[231179.224907] apt-get used greatest stack depth: 3632 bytes left
[491541.321011] INFO: rcu_sched self-detected stall on CPU { 0}  (t=60000 jiffies)
[491541.321011] Pid: 576, comm: kswapd0 Not tainted 3.3.0-rc4-next-20120222+ #129
[491541.321011] Call Trace:
[491541.321011]  <IRQ>  [<ffffffff810ab6ea>] __rcu_pending+0x19a/0x4d0
[491541.321011]  [<ffffffff8106b1dc>] ? trigger_load_balance+0x5c/0x2e0
[491541.321011]  [<ffffffff810abd50>] rcu_check_callbacks+0xb0/0x1a0
[491541.321011]  [<ffffffff81043f73>] update_process_times+0x43/0x80
[491541.321011]  [<ffffffff8107e8cf>] tick_sched_timer+0x5f/0xb0
[491541.321011]  [<ffffffff81058c58>] __run_hrtimer+0x78/0x1d0
[491541.321011]  [<ffffffff8107e870>] ? tick_nohz_handler+0xf0/0xf0
[491541.321011]  [<ffffffff8103baa1>] ? __do_softirq+0xf1/0x210
[491541.321011]  [<ffffffff81059583>] hrtimer_interrupt+0xe3/0x200
[491541.321011]  [<ffffffff8170880c>] ? call_softirq+0x1c/0x30
[491541.321011]  [<ffffffff8101f564>] smp_apic_timer_interrupt+0x64/0xa0
[491541.321011]  [<ffffffff81707ecb>] apic_timer_interrupt+0x6b/0x70
[491541.321011]  <EOI>  [<ffffffff810d8203>] ? zone_watermark_ok_safe+0xe3/0x170
[491541.321011]  [<ffffffff810e6de8>] balance_pgdat+0x1a8/0x690
[491541.321011]  [<ffffffff810e7438>] kswapd+0x168/0x3f0
[491541.321011]  [<ffffffff816fea26>] ? __schedule+0x3a6/0x750
[491541.321011]  [<ffffffff810553a0>] ? add_wait_queue+0x60/0x60
[491541.321011]  [<ffffffff810e72d0>] ? balance_pgdat+0x690/0x690
[491541.321011]  [<ffffffff8105496e>] kthread+0x8e/0xa0
[491541.321011]  [<ffffffff81708714>] kernel_thread_helper+0x4/0x10
[491541.321011]  [<ffffffff810548e0>] ? kthread_freezable_should_stop+0x70/0x70
[491541.321011]  [<ffffffff81708710>] ? gs_change+0xb/0xb
[491721.324004] INFO: rcu_sched self-detected stall on CPU { 0}  (t=240003 jiffies)
[491721.324004] Pid: 576, comm: kswapd0 Not tainted 3.3.0-rc4-next-20120222+ #129
[491721.324004] Call Trace:
[491721.324004]  <IRQ>  [<ffffffff810ab6ea>] __rcu_pending+0x19a/0x4d0
[491721.324004]  [<ffffffff8106b1dc>] ? trigger_load_balance+0x5c/0x2e0
[491721.324004]  [<ffffffff810abd50>] rcu_check_callbacks+0xb0/0x1a0
[491721.324004]  [<ffffffff81043f73>] update_process_times+0x43/0x80
[491721.324004]  [<ffffffff8107e8cf>] tick_sched_timer+0x5f/0xb0
[491721.324004]  [<ffffffff81058c58>] __run_hrtimer+0x78/0x1d0
[491721.324004]  [<ffffffff8107e870>] ? tick_nohz_handler+0xf0/0xf0
[491721.324004]  [<ffffffff8103baa1>] ? __do_softirq+0xf1/0x210
[491721.324004]  [<ffffffff81059583>] hrtimer_interrupt+0xe3/0x200
[491721.324004]  [<ffffffff8170880c>] ? call_softirq+0x1c/0x30
[491721.324004]  [<ffffffff8101f564>] smp_apic_timer_interrupt+0x64/0xa0
[491721.324004]  [<ffffffff81707ecb>] apic_timer_interrupt+0x6b/0x70
[491721.324004]  <EOI>  [<ffffffff810d812d>] ? zone_watermark_ok_safe+0xd/0x170
[491721.324004]  [<ffffffff810e6de8>] balance_pgdat+0x1a8/0x690
[491721.324004]  [<ffffffff810e7438>] kswapd+0x168/0x3f0
[491721.324004]  [<ffffffff816fea26>] ? __schedule+0x3a6/0x750
[491721.324004]  [<ffffffff810553a0>] ? add_wait_queue+0x60/0x60
[491721.324004]  [<ffffffff810e72d0>] ? balance_pgdat+0x690/0x690
[491721.324004]  [<ffffffff8105496e>] kthread+0x8e/0xa0
[491721.324004]  [<ffffffff81708714>] kernel_thread_helper+0x4/0x10
[491721.324004]  [<ffffffff810548e0>] ? kthread_freezable_should_stop+0x70/0x70
[491721.324004]  [<ffffffff81708710>] ? gs_change+0xb/0xb
[491901.327003] INFO: rcu_sched self-detected stall on CPU { 0}  (t=420006 jiffies)


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2012-03-14  6:56 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-03-13 13:48 RCU stalls in linux-next Dan Carpenter
2012-03-13 14:04 ` Dan Carpenter
2012-03-13 14:33 ` Paul E. McKenney
2012-03-14  6:59   ` Dan Carpenter

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).