* [GIT PULL rcu/next] rcu commits for 2.6.40
@ 2011-05-08 15:18 Paul E. McKenney
2011-05-09 7:36 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-08 15:18 UTC (permalink / raw)
To: mingo; +Cc: linux-kernel
Hello, Ingo,
This pull request covers RCU chnages for 2.6.40. The major new features
are RCU priority boosting and the addition of kfree_rcu(), the latter
courtesy of Lai Jiangshan. These two features cover well over half
of the commits. There are a number of smaller features and bug fixes.
All have been sent to LKML in the following batches:
0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
The kfree_rcu() uses in the pull request have Acked-by:s from the
maintainers. I have some additional kfree_rcu() requests that lack
Acked-by:s, and I will deal with these later.
These channges are available in the -rcu git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
------------------>
Eric Dumazet (1):
rcu: optimize rcutiny
Gleb Natapov (1):
rcu: provide rcu_virt_note_context_switch() function.
Lai Jiangshan (40):
rcu: call __rcu_read_unlock() in exit_rcu for tree RCU
rcu: introduce kfree_rcu()
cgroup,rcu: convert call_rcu(free_css_set_rcu) to kfree_rcu()
cgroup,rcu: convert call_rcu(free_cgroup_rcu) to kfree_rcu()
cgroup,rcu: convert call_rcu(__free_css_id_cb) to kfree_rcu()
net,rcu: convert call_rcu(tcf_common_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(tcf_police_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(in6_dev_finish_destroy_rcu) to kfree_rcu()
net,rcu: convert call_rcu(inet6_ifa_finish_destroy_rcu) to kfree_rcu()
net,rcu: convert call_rcu(listeners_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(sctp_local_addr_free) to kfree_rcu()
net,rcu: convert call_rcu(ha_rcu_free) to kfree_rcu()
net,rcu: convert call_rcu(dn_dev_free_ifa_rcu) to kfree_rcu()
net,act_police,rcu: remove rcu_barrier()
security,rcu: convert call_rcu(user_update_rcu_disposal) to kfree_rcu()
net,rcu: convert call_rcu(fc_rport_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(__leaf_info_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(__gen_kill_estimator) to kfree_rcu()
net,rcu: convert call_rcu(ip_mc_list_reclaim) to kfree_rcu()
net,rcu: convert call_rcu(ip_sf_socklist_reclaim) to kfree_rcu()
net,rcu: convert call_rcu(ip_mc_socklist_reclaim) to kfree_rcu()
net,rcu: convert call_rcu(free_dm_hw_stat) to kfree_rcu()
ixgbe,rcu: convert call_rcu(ring_free_rcu) to kfree_rcu()
macvlan,rcu: convert call_rcu(macvlan_port_rcu_free) to kfree_rcu()
net,rcu: convert call_rcu(ipv6_mc_socklist_reclaim) to kfree_rcu()
net,rcu: convert call_rcu(rps_map_release) to kfree_rcu()
net,rcu: convert call_rcu(xps_map_release) to kfree_rcu()
net,rcu: convert call_rcu(xps_dev_maps_release) to kfree_rcu()
security,rcu: convert call_rcu(sel_netif_free) to kfree_rcu()
net,rcu: convert call_rcu(netlbl_unlhsh_free_addr4) to kfree_rcu()
net,rcu: convert call_rcu(netlbl_unlhsh_free_addr6) to kfree_rcu()
net,rcu: convert call_rcu(net_generic_release) to kfree_rcu()
net,rcu: convert call_rcu(__nf_ct_ext_free_rcu) to kfree_rcu()
perf,rcu: convert call_rcu(free_ctx) to kfree_rcu()
perf,rcu: convert call_rcu(swevent_hlist_release_rcu) to kfree_rcu()
net,rcu: convert call_rcu(phonet_device_rcu_free) to kfree_rcu()
net,rcu: convert call_rcu(wq_free_rcu) to kfree_rcu()
net/mac80211,rcu: convert call_rcu(work_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(xt_osf_finger_free_rcu) to kfree_rcu()
net,rcu: convert call_rcu(kfree_tid_tx) to kfree_rcu()
Mathieu Desnoyers (1):
rcu: Enable DEBUG_OBJECTS_RCU_HEAD from !PREEMPT
Paul E. McKenney (34):
rcu: Remove conditional compilation for RCU CPU stall warnings
rcu: Decrease memory-barrier usage based on semi-formal proof
rcu: merge TREE_PREEPT_RCU blocked_tasks[] lists
rcu: move TREE_RCU from softirq to kthread
rcu: priority boosting for TREE_PREEMPT_RCU
rcu: Force per-rcu_node kthreads off of the outgoing CPU
rcu: put per-CPU kthread at non-RT priority during CPU hotplug operations
rcu: avoid hammering sched with yet another bound RT kthread
rcu: eliminate unused boosting statistics
rcu: Add boosting to TREE_PREEMPT_RCU tracing
rcu: Update RCU's trace.txt documentation for new format
rcu: add callback-queue information to rcudata output
rcu: add tracing for RCU's kthread run states.
rcu: make rcutorture version numbers available through debugfs
rcu: update tracing documentation for new rcutorture and rcuboost
rcu: fix tracing bug thinko on boost-balk attribution
rcu: add grace-period age and more kthread state to tracing
rcu: Add forward-progress diagnostic for per-CPU kthreads
rcu: add DEBUG_OBJECTS_RCU_HEAD check for alignment
rcu: mark rcutorture boosting callback as being on-stack
rcu: Use WARN_ON_ONCE for DEBUG_OBJECTS_RCU_HEAD warnings
rcu: Switch to this_cpu() primitives
rcu: code cleanups in TINY_RCU priority boosting.
rcu: remove useless ->boosted_this_gp field
rcu: Converge TINY_RCU expedited and normal boosting
rcu: fix spelling
rcu: further lower priority in rcu_yield()
rcu: prevent call_rcu() from diving into rcu core if irqs disabled
rcu: get rid of signed overflow in check_cpu_stall()
rcu: permit rcu_read_unlock() to be called while holding runqueue locks
batman,rcu: convert call_rcu(gw_node_free_rcu) to kfree_rcu
batman,rcu: convert call_rcu(neigh_node_free_rcu) to kfree()
batman,rcu: convert call_rcu(softif_neigh_free_rcu) to kfree_rcu
net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
Documentation/RCU/00-INDEX | 2 +-
Documentation/RCU/stallwarn.txt | 23 +-
Documentation/RCU/trace.txt | 295 +++++++++++++----
Documentation/filesystems/proc.txt | 1 -
drivers/net/ixgbe/ixgbe_main.c | 7 +-
drivers/net/macvlan.c | 10 +-
include/linux/interrupt.h | 1 -
include/linux/rcupdate.h | 70 ++++-
include/linux/rcutiny.h | 8 +
include/linux/rcutree.h | 13 +
include/net/sctp/sctp.h | 1 -
include/trace/events/irq.h | 3 +-
init/Kconfig | 2 +-
kernel/cgroup.c | 27 +--
kernel/perf_event.c | 20 +-
kernel/rcupdate.c | 32 ++-
kernel/rcutiny.c | 45 ++--
kernel/rcutiny_plugin.h | 203 +++++------
kernel/rcutorture.c | 26 +-
kernel/rcutree.c | 656 ++++++++++++++++++++++++++++-------
kernel/rcutree.h | 113 +++++--
kernel/rcutree_plugin.h | 575 ++++++++++++++++++++++++-------
kernel/rcutree_trace.c | 192 +++++++++--
kernel/softirq.c | 2 +-
lib/Kconfig.debug | 32 +--
net/batman-adv/gateway_client.c | 10 +-
net/batman-adv/originator.c | 10 +-
net/batman-adv/soft-interface.c | 10 +-
net/core/dev_addr_lists.c | 12 +-
net/core/drop_monitor.c | 12 +-
net/core/gen_estimator.c | 9 +-
net/core/net-sysfs.c | 34 +--
net/core/net_namespace.c | 10 +-
net/decnet/dn_dev.c | 7 +-
net/ipv4/fib_semantics.c | 12 +-
net/ipv4/fib_trie.c | 7 +-
net/ipv4/igmp.c | 32 +--
net/ipv6/addrconf.c | 16 +-
net/ipv6/mcast.c | 8 +-
net/ipv6/sit.c | 7 +-
net/mac80211/agg-tx.c | 14 +-
net/mac80211/work.c | 10 +-
net/netfilter/nf_conntrack_extend.c | 8 +-
net/netfilter/xt_osf.c | 11 +-
net/netlabel/netlabel_unlabeled.c | 42 +---
net/netlink/af_netlink.c | 8 +-
net/phonet/pn_dev.c | 10 +-
net/sched/act_api.c | 7 +-
net/sched/act_police.c | 8 +-
net/sctp/bind_addr.c | 2 +-
net/sctp/ipv6.c | 2 +-
net/sctp/protocol.c | 9 +-
net/socket.c | 11 +-
security/keys/user_defined.c | 16 +-
security/selinux/netif.c | 18 +-
tools/perf/util/trace-event-parse.c | 1 -
56 files changed, 1797 insertions(+), 935 deletions(-)
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-08 15:18 [GIT PULL rcu/next] rcu commits for 2.6.40 Paul E. McKenney
@ 2011-05-09 7:36 ` Ingo Molnar
2011-05-09 21:09 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-09 7:36 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> Hello, Ingo,
>
> This pull request covers RCU chnages for 2.6.40. The major new features
> are RCU priority boosting and the addition of kfree_rcu(), the latter
> courtesy of Lai Jiangshan. These two features cover well over half
> of the commits. There are a number of smaller features and bug fixes.
> All have been sent to LKML in the following batches:
>
> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>
> The kfree_rcu() uses in the pull request have Acked-by:s from the
> maintainers. I have some additional kfree_rcu() requests that lack
> Acked-by:s, and I will deal with these later.
>
> These channges are available in the -rcu git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
Pulled, thanks a lot Paul!
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-09 7:36 ` Ingo Molnar
@ 2011-05-09 21:09 ` Yinghai Lu
2011-05-10 8:56 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-09 21:09 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Paul E. McKenney, linux-kernel
On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>
>> Hello, Ingo,
>>
>> This pull request covers RCU chnages for 2.6.40. The major new features
>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>> courtesy of Lai Jiangshan. These two features cover well over half
>> of the commits. There are a number of smaller features and bug fixes.
>> All have been sent to LKML in the following batches:
>>
>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>
>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>> maintainers. I have some additional kfree_rcu() requests that lack
>> Acked-by:s, and I will deal with these later.
>>
>> These channges are available in the -rcu git repository at:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>
> Pulled, thanks a lot Paul!
>
it seems with this one in tip, my 8 sockets test setup will report cpu stall.
after hard code to enable rcu_cpu_stall_suppress
Index: linux-2.6/kernel/rcutree.c
===================================================================
--- linux-2.6.orig/kernel/rcutree.c
+++ linux-2.6/kernel/rcutree.c
@@ -174,7 +174,7 @@ module_param(blimit, int, 0);
module_param(qhimark, int, 0);
module_param(qlowmark, int, 0);
-int rcu_cpu_stall_suppress __read_mostly;
+int rcu_cpu_stall_suppress __read_mostly = 1;
module_param(rcu_cpu_stall_suppress, int, 0644);
static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
will get system hang after pnp ACPI init.
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-09 21:09 ` Yinghai Lu
@ 2011-05-10 8:56 ` Paul E. McKenney
2011-05-10 9:37 ` Ingo Molnar
2011-05-10 18:04 ` Yinghai Lu
0 siblings, 2 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-10 8:56 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
> >
> > * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> >
> >> Hello, Ingo,
> >>
> >> This pull request covers RCU chnages for 2.6.40. The major new features
> >> are RCU priority boosting and the addition of kfree_rcu(), the latter
> >> courtesy of Lai Jiangshan. These two features cover well over half
> >> of the commits. There are a number of smaller features and bug fixes.
> >> All have been sent to LKML in the following batches:
> >>
> >> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> >> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> >> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> >> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
> >>
> >> The kfree_rcu() uses in the pull request have Acked-by:s from the
> >> maintainers. I have some additional kfree_rcu() requests that lack
> >> Acked-by:s, and I will deal with these later.
> >>
> >> These channges are available in the -rcu git repository at:
> >>
> >> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
> >
> > Pulled, thanks a lot Paul!
> >
>
> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>
> after hard code to enable rcu_cpu_stall_suppress
>
> Index: linux-2.6/kernel/rcutree.c
> ===================================================================
> --- linux-2.6.orig/kernel/rcutree.c
> +++ linux-2.6/kernel/rcutree.c
> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
> module_param(qhimark, int, 0);
> module_param(qlowmark, int, 0);
>
> -int rcu_cpu_stall_suppress __read_mostly;
> +int rcu_cpu_stall_suppress __read_mostly = 1;
> module_param(rcu_cpu_stall_suppress, int, 0644);
>
> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>
> will get system hang after pnp ACPI init.
Could you please send the stack traces from the RCU CPU stall? Also,
you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-10 8:56 ` Paul E. McKenney
@ 2011-05-10 9:37 ` Ingo Molnar
2011-05-10 18:04 ` Yinghai Lu
1 sibling, 0 replies; 76+ messages in thread
From: Ingo Molnar @ 2011-05-10 9:37 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
> > On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
> > >
> > > * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> > >
> > >> Hello, Ingo,
> > >>
> > >> This pull request covers RCU chnages for 2.6.40. The major new features
> > >> are RCU priority boosting and the addition of kfree_rcu(), the latter
> > >> courtesy of Lai Jiangshan. These two features cover well over half
> > >> of the commits. There are a number of smaller features and bug fixes.
> > >> All have been sent to LKML in the following batches:
> > >>
> > >> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> > >> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> > >> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> > >> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
> > >>
> > >> The kfree_rcu() uses in the pull request have Acked-by:s from the
> > >> maintainers. I have some additional kfree_rcu() requests that lack
> > >> Acked-by:s, and I will deal with these later.
> > >>
> > >> These channges are available in the -rcu git repository at:
> > >>
> > >> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
> > >
> > > Pulled, thanks a lot Paul!
> > >
> >
> > it seems with this one in tip, my 8 sockets test setup will report cpu stall.
> >
> > after hard code to enable rcu_cpu_stall_suppress
> >
> > Index: linux-2.6/kernel/rcutree.c
> > ===================================================================
> > --- linux-2.6.orig/kernel/rcutree.c
> > +++ linux-2.6/kernel/rcutree.c
> > @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
> > module_param(qhimark, int, 0);
> > module_param(qlowmark, int, 0);
> >
> > -int rcu_cpu_stall_suppress __read_mostly;
> > +int rcu_cpu_stall_suppress __read_mostly = 1;
> > module_param(rcu_cpu_stall_suppress, int, 0644);
> >
> > static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
> >
> > will get system hang after pnp ACPI init.
>
> Could you please send the stack traces from the RCU CPU stall? Also,
> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
if the failure is reproducible then bisection would be helpful as well.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-10 8:56 ` Paul E. McKenney
2011-05-10 9:37 ` Ingo Molnar
@ 2011-05-10 18:04 ` Yinghai Lu
2011-05-10 19:32 ` Paul E. McKenney
1 sibling, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-10 18:04 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>
>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>
>>>> Hello, Ingo,
>>>>
>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>> of the commits. There are a number of smaller features and bug fixes.
>>>> All have been sent to LKML in the following batches:
>>>>
>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>
>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>> Acked-by:s, and I will deal with these later.
>>>>
>>>> These channges are available in the -rcu git repository at:
>>>>
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>
>>> Pulled, thanks a lot Paul!
>>>
>>
>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>
>> after hard code to enable rcu_cpu_stall_suppress
>>
>> Index: linux-2.6/kernel/rcutree.c
>> ===================================================================
>> --- linux-2.6.orig/kernel/rcutree.c
>> +++ linux-2.6/kernel/rcutree.c
>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>> module_param(qhimark, int, 0);
>> module_param(qlowmark, int, 0);
>>
>> -int rcu_cpu_stall_suppress __read_mostly;
>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>
>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>
>> will get system hang after pnp ACPI init.
>
> Could you please send the stack traces from the RCU CPU stall? Also,
> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>
> Thanx, Paul
Do not have time to bisect it at this point.
can you prepare clean revert patch for following commit to test?
commit a00e0d714fbded07a7a2254391ce9ed5a5cb9d82
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Date: Tue Feb 8 17:14:39 2011 -0800
rcu: Remove conditional compilation for RCU CPU stall warnings
Thanks
Yinghai Lu
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-10 18:04 ` Yinghai Lu
@ 2011-05-10 19:32 ` Paul E. McKenney
2011-05-10 20:52 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-10 19:32 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
> > On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
> >> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
> >>>
> >>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> >>>
> >>>> Hello, Ingo,
> >>>>
> >>>> This pull request covers RCU chnages for 2.6.40. The major new features
> >>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
> >>>> courtesy of Lai Jiangshan. These two features cover well over half
> >>>> of the commits. There are a number of smaller features and bug fixes.
> >>>> All have been sent to LKML in the following batches:
> >>>>
> >>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> >>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> >>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> >>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
> >>>>
> >>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
> >>>> maintainers. I have some additional kfree_rcu() requests that lack
> >>>> Acked-by:s, and I will deal with these later.
> >>>>
> >>>> These channges are available in the -rcu git repository at:
> >>>>
> >>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
> >>>
> >>> Pulled, thanks a lot Paul!
> >>>
> >>
> >> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
> >>
> >> after hard code to enable rcu_cpu_stall_suppress
> >>
> >> Index: linux-2.6/kernel/rcutree.c
> >> ===================================================================
> >> --- linux-2.6.orig/kernel/rcutree.c
> >> +++ linux-2.6/kernel/rcutree.c
> >> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
> >> module_param(qhimark, int, 0);
> >> module_param(qlowmark, int, 0);
> >>
> >> -int rcu_cpu_stall_suppress __read_mostly;
> >> +int rcu_cpu_stall_suppress __read_mostly = 1;
> >> module_param(rcu_cpu_stall_suppress, int, 0644);
> >>
> >> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
> >>
> >> will get system hang after pnp ACPI init.
> >
> > Could you please send the stack traces from the RCU CPU stall? Also,
> > you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
> >
> > Thanx, Paul
>
> Do not have time to bisect it at this point.
Could you please send the stack traces from the RCU CPU stall?
Thanx, Paul
> can you prepare clean revert patch for following commit to test?
>
> commit a00e0d714fbded07a7a2254391ce9ed5a5cb9d82
> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Date: Tue Feb 8 17:14:39 2011 -0800
>
> rcu: Remove conditional compilation for RCU CPU stall warnings
>
>
> Thanks
>
> Yinghai Lu
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-10 19:32 ` Paul E. McKenney
@ 2011-05-10 20:52 ` Yinghai Lu
2011-05-11 4:54 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-10 20:52 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>>>
>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>>>
>>>>>> Hello, Ingo,
>>>>>>
>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>>>> of the commits. There are a number of smaller features and bug fixes.
>>>>>> All have been sent to LKML in the following batches:
>>>>>>
>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>>>
>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>>>> Acked-by:s, and I will deal with these later.
>>>>>>
>>>>>> These channges are available in the -rcu git repository at:
>>>>>>
>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>>>
>>>>> Pulled, thanks a lot Paul!
>>>>>
>>>>
>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>>>
>>>> after hard code to enable rcu_cpu_stall_suppress
>>>>
>>>> Index: linux-2.6/kernel/rcutree.c
>>>> ===================================================================
>>>> --- linux-2.6.orig/kernel/rcutree.c
>>>> +++ linux-2.6/kernel/rcutree.c
>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>>>> module_param(qhimark, int, 0);
>>>> module_param(qlowmark, int, 0);
>>>>
>>>> -int rcu_cpu_stall_suppress __read_mostly;
>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>>>
>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>>>
>>>> will get system hang after pnp ACPI init.
>>>
>>> Could you please send the stack traces from the RCU CPU stall? Also,
>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>>>
>>> Thanx, Paul
>>
>> Do not have time to bisect it at this point.
>
> Could you please send the stack traces from the RCU CPU stall?
>
[ 76.553010] calling pnpacpi_init+0x0/0x8c @ 1
[ 76.553236] pnp: PnP ACPI init
[ 76.572833] ACPI: bus type pnp registered
[ 76.573570] pnp 00:00: [bus 00-3f]
[ 76.573766] pnp 00:00: [io 0x0000-0x5fff window]
[ 76.592818] pnp 00:00: [mem 0x000a0000-0x000bffff window]
[ 76.593099] pnp 00:00: [mem 0x000d0000-0x000dffff window]
[ 76.612741] pnp 00:00: [mem 0x00000000-0xffffffff window]
[ 76.613001] pnp 00:00: [mem 0x90000000-0xafffffff window]
[ 76.632696] pnp 00:00: [mem 0xfed40000-0xfed44fff window]
[ 76.632951] pnp 00:00: [mem 0xfc000000000-0xfc07fffffff window]
[ 76.653366] pnp 00:00: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
[ 76.653730] pnp 00:01: [mem 0xfc000000-0xfcffffff]
[ 76.672665] pnp 00:01: [mem 0xfd000000-0xfdffffff]
[ 76.672887] pnp 00:01: [mem 0xfe000000-0xfebfffff]
[ 76.693831] system 00:01: [mem 0xfc000000-0xfcffffff] has been reserved
[ 76.694151] system 00:01: [mem 0xfd000000-0xfdffffff] has been reserved
[ 76.712632] system 00:01: [mem 0xfe000000-0xfebfffff] has been reserved
[ 76.712927] system 00:01: Plug and Play ACPI device, IDs PNP0c01 (active)
[ 136.364832] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 124, t=15002 jiffies)
[ 136.365317] sending NMI to all CPUs:
[ 136.381054] NMI backtrace for cpu 0
[ 136.381364] CPU 0
[ 136.381450] Modules linked in:
[ 136.381638]
[ 136.400938] Pid: 0, comm: swapper Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 136.420934] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 136.421313] RSP: 0018:ffffffff82397ee8 EFLAGS: 00000246
[ 136.440953] RAX: 0000000000000000 RBX: ffffffff82397fd8 RCX: 0000000000000000
[ 136.441256] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff823a5020
[ 136.461059] RBP: ffffffff82397ef8 R08: 0000000000000000 R09: 0000000000000000
[ 136.480842] R10: ffffffffffffffff R11: ffffffff82397ed8 R12: 0000000000000000
[ 136.481133] R13: ffff88807ffff300 R14: ffffffffffffffff R15: 0000000000000000
[ 136.500932] FS: 0000000000000000(0000) GS:ffff88107de00000(0000) knlGS:0000000000000000
[ 136.520779] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 136.521046] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006f0
[ 136.540800] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 136.560608] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 136.560936] Process swapper (pid: 0, threadinfo ffffffff82396000, task ffffffff823a5020)
[ 136.580762] Stack:
[ 136.580852] ffffffff82396000 ffffffff82525380 ffffffff82397f28 ffffffff810391f0
[ 136.600682] 6db6db6db6db6db7 f51568c424a0f956 0000000000000002 6db6db6db6db6db7
[ 136.601087] ffffffff82397f58 ffffffff81bbbb93 ffffffff81bbbadc 0000000000000000
[ 136.620945] Call Trace:
[ 136.621063] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 136.640561] [<ffffffff81bbbb93>] rest_init+0xb7/0xbe
[ 136.640805] [<ffffffff81bbbadc>] ? csum_partial_copy_generic+0x16c/0x16c
[ 136.660553] [<ffffffff826ffb4a>] start_kernel+0x3ba/0x3c5
[ 136.660814] [<ffffffff826ff2a6>] x86_64_start_reservations+0xb6/0xba
[ 136.680553] [<ffffffff826ff39c>] x86_64_start_kernel+0xf2/0xf9
[ 136.680831] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 136.721038] Call Trace:
[ 136.721160] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 136.740386] [<ffffffff81bbbb93>] rest_init+0xb7/0xbe
[ 136.740630] [<ffffffff81bbbadc>] ? csum_partial_copy_generic+0x16c/0x16c
[ 136.760349] [<ffffffff826ffb4a>] start_kernel+0x3ba/0x3c5
[ 136.760626] [<ffffffff826ff2a6>] x86_64_start_reservations+0xb6/0xba
[ 136.780369] [<ffffffff826ff39c>] x86_64_start_kernel+0xf2/0xf9
[ 136.800134] NMI backtrace for cpu 1
[ 136.800321] CPU 1
[ 136.800436] Modules linked in:
[ 136.800616]
[ 136.800703] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 136.839999] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 136.840388] RSP: 0000:ffff88103ee33ee8 EFLAGS: 00000246
[ 136.860052] RAX: 0000000000000000 RBX: ffff88103ee33fd8 RCX: 0000000000000000
[ 136.860373] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ee38000
[ 136.880168] RBP: ffff88103ee33ef8 R08: 0000000000000000 R09: 0000000000000000
[ 136.900089] R10: 0000000000000000 R11: ffff88103ee33ed8 R12: 0000000000000001
[ 136.900416] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 136.920090] FS: 0000000000000000(0000) GS:ffff88107e000000(0000) knlGS:0000000000000000
[ 136.950414] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 136.950684] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 136.970157] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 136.970455] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 136.990120] Process kworker/0:0 (pid: 0, threadinfo ffff88103ee32000, task ffff88103ee38000)
[ 137.009848] Stack:
[ 137.009959] ffff88103ee32000 ffffffff82525380 ffff88103ee33f28 ffffffff810391f0
[ 137.032505] ffff88103ee33f18 2bd3bd551d7037b4 0000000000000000 0000000000000000
[ 137.032879] ffff88103ee33f48 ffffffff81c064f3 0000000000000000 661fe41cd64eec39
[ 137.049938] Call Trace:
[ 137.050080] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 137.069800] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 137.070075] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 137.109943] Call Trace:
[ 137.110080] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 137.129466] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 137.129703] NMI backtrace for cpu 2
[ 137.129877] CPU 2
[ 137.149412] Modules linked in:
[ 137.149603]
[ 137.149705] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 137.169681] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 137.189674] RSP: 0000:ffff88103ee5bee8 EFLAGS: 00000246
[ 137.189933] RAX: 0000000000000000 RBX: ffff88103ee5bfd8 RCX: 0000000000000000
[ 137.209465] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ee60000
[ 137.229230] RBP: ffff88103ee5bef8 R08: 0000000000000000 R09: 0000000000000000
[ 137.229570] R10: 0000000000000000 R11: ffff88103ee5bed8 R12: 0000000000000002
[ 137.249542] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 137.269122] FS: 0000000000000000(0000) GS:ffff88107e200000(0000) knlGS:0000000000000000
[ 137.269486] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 137.289226] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 137.289536] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 137.309303] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 137.329096] Process kworker/0:1 (pid: 0, threadinfo ffff88103ee5a000, task ffff88103ee60000)
[ 137.329443] Stack:
[ 137.349030] ffff88103ee5a000 ffffffff82525380 ffff88103ee5bf28 ffffffff810391f0
[ 137.349444] ffff88103ee5bf18 2a2988cc3681d8f7 0000000000000000 0000000000000000
[ 137.369236] ffff88103ee5bf48 ffffffff81c064f3 0000000000000000 63a332c588e07727
[ 137.388974] Call Trace:
[ 137.389121] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 137.389353] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 137.409021] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 137.448825] Call Trace:
[ 137.448951] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 137.449175] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 137.468814] NMI backtrace for cpu 3
[ 137.468990] CPU 3
[ 137.469084] Modules linked in:
[ 137.469272]
[ 137.488737] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 137.508741] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 137.509123] RSP: 0000:ffff88103ee83ee8 EFLAGS: 00000246
[ 137.528765] RAX: 0000000000000000 RBX: ffff88103ee83fd8 RCX: 0000000000000000
[ 137.529096] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ee88000
[ 137.548872] RBP: ffff88103ee83ef8 R08: 0000000000000000 R09: 0000000000000000
[ 137.568641] R10: 0000000000000000 R11: ffff88103ee83ed8 R12: 0000000000000003
[ 137.568957] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 137.588755] FS: 0000000000000000(0000) GS:ffff88107e400000(0000) knlGS:0000000000000000
[ 137.608591] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 137.608847] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 137.628623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 137.648432] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 137.648728] Process kworker/0:1 (pid: 0, threadinfo ffff88103ee82000, task ffff88103ee88000)
[ 137.670089] Stack:
[ 137.670189] ffff88103ee82000 ffffffff82525380 ffff88103ee83f28 ffffffff810391f0
[ 137.688507] ffff88103ee83f18 cacb1e5ab5a0da93 0000000000000000 0000000000000000
[ 137.708279] ffff88103ee83f48 ffffffff81c064f3 0000000000000000 83c0d3c3e7f0ed35
[ 137.708699] Call Trace:
[ 137.708835] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 137.728388] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 137.728645] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 137.768750] Call Trace:
[ 137.768874] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 137.788207] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 137.788493] NMI backtrace for cpu 4
[ 137.808088] CPU 4
[ 137.808175] Modules linked in:
[ 137.808384]
[ 137.808461] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 137.828362] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 137.848186] RSP: 0000:ffff88103eeadee8 EFLAGS: 00000246
[ 137.848440] RAX: 0000000000000000 RBX: ffff88103eeadfd8 RCX: 0000000000000000
[ 137.868205] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eeb0000
[ 137.887990] RBP: ffff88103eeadef8 R08: 0000000000000000 R09: 0000000000000000
[ 137.888308] R10: 0000000000000000 R11: ffff88103eeaded8 R12: 0000000000000004
[ 137.908084] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 137.928028] FS: 0000000000000000(0000) GS:ffff88107e600000(0000) knlGS:0000000000000000
[ 137.928376] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 137.948182] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 137.968138] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 137.968453] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 137.987930] Process kworker/0:1 (pid: 0, threadinfo ffff88103eeac000, task ffff88103eeb0000)
[ 138.007782] Stack:
[ 138.007905] ffff88103eeac000 ffffffff82525380 ffff88103eeadf28 ffffffff810391f0
[ 138.008306] ffff88103eeadf18 37300352987163c3 0000000000000000 0000000000000000
[ 138.028077] ffff88103eeadf48 ffffffff81c064f3 0000000000000000 f12528251e665927
[ 138.048050] Call Trace:
[ 138.048173] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 138.048386] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 138.067974] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 138.107791] Call Trace:
[ 138.107916] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 138.108150] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 138.127714] NMI backtrace for cpu 5
[ 138.127899] CPU 5
[ 138.127999] Modules linked in:
[ 138.147833]
[ 138.147911] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 138.167633] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 138.187390] RSP: 0000:ffff88103eecfee8 EFLAGS: 00000246
[ 138.187621] RAX: 0000000000000000 RBX: ffff88103eecffd8 RCX: 0000000000000000
[ 138.207291] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eed0000
[ 138.207625] RBP: ffff88103eecfef8 R08: 0000000000000000 R09: 0000000000000000
[ 138.227397] R10: 0000000000000000 R11: ffff88103eecfed8 R12: 0000000000000005
[ 138.247345] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 138.247657] FS: 0000000000000000(0000) GS:ffff88107e800000(0000) knlGS:0000000000000000
[ 138.267316] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 138.267577] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 138.287344] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 138.307936] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 138.308252] Process kworker/0:1 (pid: 0, threadinfo ffff88103eece000, task ffff88103eed0000)
[ 138.327333] Stack:
[ 138.327424] ffff88103eece000 ffffffff82525380 ffff88103eecff28 ffffffff810391f0
[ 138.347307] ffff88103eecff18 5c976e576efe47b3 0000000000000000 0000000000000000
[ 138.367037] ffff88103eecff48 ffffffff81c064f3 0000000000000000 f61e8ac9274555e7
[ 138.367426] Call Trace:
[ 138.386913] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 138.387162] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 138.406850] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 138.427617] Call Trace:
[ 138.427731] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 138.446968] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 138.447219] NMI backtrace for cpu 6
[ 138.466818] CPU 6
[ 138.466913] Modules linked in:
[ 138.467084]
[ 138.467169] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 138.487237] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 138.506894] RSP: 0000:ffff88103eef7ee8 EFLAGS: 00000246
[ 138.526591] RAX: 0000000000000000 RBX: ffff88103eef7fd8 RCX: 0000000000000000
[ 138.526931] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eef8000
[ 138.546705] RBP: ffff88103eef7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 138.566499] R10: 0000000000000000 R11: ffff88103eef7ed8 R12: 0000000000000006
[ 138.566831] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 138.586585] FS: 0000000000000000(0000) GS:ffff88107ea00000(0000) knlGS:0000000000000000
[ 138.606412] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 138.606667] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 138.626666] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 138.626982] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 138.646565] Process kworker/0:1 (pid: 0, threadinfo ffff88103eef6000, task ffff88103eef8000)
[ 138.666398] Stack:
[ 138.666489] ffff88103eef6000 ffffffff82525380 ffff88103eef7f28 ffffffff810391f0
[ 138.686347] ffff88103eef7f18 15e4bcc96f1f15ba 0000000000000000 0000000000000000
[ 138.686762] ffff88103eef7f48 ffffffff81c064f3 0000000000000000 9898fe604c13b8fd
[ 138.706504] Call Trace:
[ 138.706613] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 138.726202] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 138.726447] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 138.766487] Call Trace:
[ 138.766626] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 138.766890] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 138.786275] NMI backtrace for cpu 7
[ 138.786460] CPU 7
[ 138.786556] Modules linked in:
[ 138.806144]
[ 138.806222] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 138.826206] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 138.845997] RSP: 0000:ffff88103ef17ee8 EFLAGS: 00000246
[ 138.846225] RAX: 0000000000000000 RBX: ffff88103ef17fd8 RCX: 0000000000000000
[ 138.866022] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef20000
[ 138.866343] RBP: ffff88103ef17ef8 R08: 0000000000000000 R09: 0000000000000000
[ 138.886108] R10: 0000000000000000 R11: ffff88103ef17ed8 R12: 0000000000000007
[ 138.905915] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 138.906216] FS: 0000000000000000(0000) GS:ffff88107ec00000(0000) knlGS:0000000000000000
[ 138.926104] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 138.945846] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 138.946147] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 138.976047] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 138.995888] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef16000, task ffff88103ef20000)
[ 138.996263] Stack:
[ 139.015720] ffff88103ef16000 ffffffff82525380 ffff88103ef17f28 ffffffff810391f0
[ 139.016116] ffff88103ef17f18 5945687aae80abf9 0000000000000000 0000000000000000
[ 139.035768] ffff88103ef17f48 ffffffff81c064f3 0000000000000000 1e4ee324f58b290c
[ 139.055524] Call Trace:
[ 139.055657] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 139.055904] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 139.075528] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 139.096417] Call Trace:
[ 139.115416] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 139.115626] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 139.135361] NMI backtrace for cpu 8
[ 139.135539] CPU 8
[ 139.135636] Modules linked in:
[ 139.135810]
[ 139.135888] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 139.175277] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 139.175644] RSP: 0000:ffff88103ef2dee8 EFLAGS: 00000246
[ 139.195308] RAX: 0000000000000000 RBX: ffff88103ef2dfd8 RCX: 0000000000000000
[ 139.195607] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef24560
[ 139.215407] RBP: ffff88103ef2def8 R08: 0000000000000000 R09: 0000000000000000
[ 139.235209] R10: 0000000000000000 R11: ffff88103ef2ded8 R12: 0000000000000008
[ 139.235524] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 139.255299] FS: 0000000000000000(0000) GS:ffff88207de00000(0000) knlGS:0000000000000000
[ 139.275144] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 139.275380] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 139.295174] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 139.295480] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 139.315261] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef2c000, task ffff88103ef24560)
[ 139.335121] Stack:
[ 139.335229] ffff88103ef2c000 ffffffff82525380 ffff88103ef2df28 ffffffff810391f0
[ 139.355031] ffff88103ef2df18 eb2a74b30d32a306 0000000000000000 0000000000000000
[ 139.355459] ffff88103ef2df48 ffffffff81c064f3 0000000000000000 27567f4754362612
[ 139.375207] Call Trace:
[ 139.375321] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 139.394922] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 139.395182] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 139.435249] Call Trace:
[ 139.435373] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 139.454753] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 139.455037] NMI backtrace for cpu 9
[ 139.455259] CPU 9
[ 139.474697] Modules linked in:
[ 139.474865]
[ 139.474955] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 139.494927] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 139.514696] RSP: 0000:ffff88103ef43ee8 EFLAGS: 00000246
[ 139.514927] RAX: 0000000000000000 RBX: ffff88103ef43fd8 RCX: 0000000000000000
[ 139.534738] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef3a2b0
[ 139.554512] RBP: ffff88103ef43ef8 R08: 0000000000000000 R09: 0000000000000000
[ 139.554835] R10: 0000000000000000 R11: ffff88103ef43ed8 R12: 0000000000000009
[ 139.574608] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 139.594406] FS: 0000000000000000(0000) GS:ffff88207e000000(0000) knlGS:0000000000000000
[ 139.594741] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 139.615073] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 139.615381] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 139.634602] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 139.654372] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef42000, task ffff88103ef3a2b0)
[ 139.654753] Stack:
[ 139.674315] ffff88103ef42000 ffffffff82525380 ffff88103ef43f28 ffffffff810391f0
[ 139.674694] ffff88103ef43f18 259a5068eb4fd413 0000000000000000 0000000000000000
[ 139.694533] ffff88103ef43f48 ffffffff81c064f3 0000000000000000 239846a0b996998b
[ 139.714264] Call Trace:
[ 139.714401] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 139.714636] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 139.734285] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 139.774083] Call Trace:
[ 139.774217] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 139.774475] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 139.794095] NMI backtrace for cpu 10
[ 139.794275] CPU 10
[ 139.794363] Modules linked in:
[ 139.813951]
[ 139.814034] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 139.834046] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 139.834393] RSP: 0000:ffff88103ef51ee8 EFLAGS: 00000246
[ 139.854066] RAX: 0000000000000000 RBX: ffff88103ef51fd8 RCX: 0000000000000000
[ 139.873832] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef4a2b0
[ 139.874139] RBP: ffff88103ef51ef8 R08: 0000000000000000 R09: 0000000000000000
[ 139.893924] R10: 0000000000000000 R11: ffff88103ef51ed8 R12: 000000000000000a
[ 139.894249] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 139.914130] FS: 0000000000000000(0000) GS:ffff88207e200000(0000) knlGS:0000000000000000
[ 139.934036] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 139.934308] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 139.954045] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 139.973916] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 139.974244] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef50000, task ffff88103ef4a2b0)
[ 139.993995] Stack:
[ 139.994086] ffff88103ef50000 ffffffff82525380 ffff88103ef51f28 ffffffff810391f0
[ 140.013955] ffff88103ef51f18 ecfea0cf87080794 0000000000000000 0000000000000000
[ 140.033587] ffff88103ef51f48 ffffffff81c064f3 0000000000000000 18b78e007f343e26
[ 140.033982] Call Trace:
[ 140.034112] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 140.053671] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 140.053948] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 140.094038] Call Trace:
[ 140.094184] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 140.113500] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 140.113767] NMI backtrace for cpu 11
[ 140.133361] CPU 11
[ 140.133453] Modules linked in:
[ 140.133631]
[ 140.133721] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 140.153658] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 140.173459] RSP: 0000:ffff88103ef69ee8 EFLAGS: 00000246
[ 140.173730] RAX: 0000000000000000 RBX: ffff88103ef69fd8 RCX: 0000000000000000
[ 140.193479] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef5a2b0
[ 140.213268] RBP: ffff88103ef69ef8 R08: 0000000000000000 R09: 0000000000000000
[ 140.213583] R10: 0000000000000000 R11: ffff88103ef69ed8 R12: 000000000000000b
[ 140.233334] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 140.258346] FS: 0000000000000000(0000) GS:ffff88207e400000(0000) knlGS:0000000000000000
[ 140.258715] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 140.273228] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 140.293022] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 140.293328] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 140.313135] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef68000, task ffff88103ef5a2b0)
[ 140.332980] Stack:
[ 140.333075] ffff88103ef68000 ffffffff82525380 ffff88103ef69f28 ffffffff810391f0
[ 140.352854] ffff88103ef69f18 11d983a4aef23a75 0000000000000000 0000000000000000
[ 140.353269] ffff88103ef69f48 ffffffff81c064f3 0000000000000000 32786c201221b6e6
[ 140.373033] Call Trace:
[ 140.373147] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 140.392777] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 140.393028] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 140.432965] Call Trace:
[ 140.433087] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 140.433318] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 140.452856] NMI backtrace for cpu 12
[ 140.453047] CPU 12
[ 140.453152] Modules linked in:
[ 140.472699]
[ 140.472804] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 140.492785] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 140.512550] RSP: 0000:ffff88103ef6fee8 EFLAGS: 00000246
[ 140.512772] RAX: 0000000000000000 RBX: ffff88103ef6ffd8 RCX: 0000000000000000
[ 140.532588] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef7a2b0
[ 140.532919] RBP: ffff88103ef6fef8 R08: 0000000000000000 R09: 0000000000000000
[ 140.552775] R10: 0000000000000000 R11: ffff88103ef6fed8 R12: 000000000000000c
[ 140.572474] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 140.572805] FS: 0000000000000000(0000) GS:ffff88207e600000(0000) knlGS:0000000000000000
[ 140.592615] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 140.612359] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 140.612675] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 140.632533] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 140.632844] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef6e000, task ffff88103ef7a2b0)
[ 140.652591] Stack:
[ 140.652685] ffff88103ef6e000 ffffffff82525380 ffff88103ef6ff28 ffffffff810391f0
[ 140.672580] ffff88103ef6ff18 9d782048b5866d90 0000000000000000 0000000000000000
[ 140.692362] ffff88103ef6ff48 ffffffff81c064f3 0000000000000000 05d404c0ff314e85
[ 140.692756] Call Trace:
[ 140.712335] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 140.712575] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 140.732124] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 140.752935] Call Trace:
[ 140.772011] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 140.772230] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 140.772510] NMI backtrace for cpu 13
[ 140.792102] CPU 13
[ 140.792197] Modules linked in:
[ 140.792371]
[ 140.792458] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 140.812395] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 140.832216] RSP: 0000:ffff88103ef8dee8 EFLAGS: 00000246
[ 140.851940] RAX: 0000000000000000 RBX: ffff88103ef8dfd8 RCX: 0000000000000000
[ 140.852244] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef822b0
[ 140.872040] RBP: ffff88103ef8def8 R08: 0000000000000000 R09: 0000000000000000
[ 140.892636] R10: 0000000000000000 R11: ffff88103ef8ded8 R12: 000000000000000d
[ 140.892947] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 140.911916] FS: 0000000000000000(0000) GS:ffff88207e800000(0000) knlGS:0000000000000000
[ 140.931832] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 140.932117] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 140.952036] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 140.952379] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 140.972012] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef8c000, task ffff88103ef822b0)
[ 140.991960] Stack:
[ 140.992061] ffff88103ef8c000 ffffffff82525380 ffff88103ef8df28 ffffffff810391f0
[ 141.011881] ffff88103ef8df18 1496eb38451a5aa8 0000000000000000 0000000000000000
[ 141.012288] ffff88103ef8df48 ffffffff81c064f3 0000000000000000 1946ee8e3f03f230
[ 141.031949] Call Trace:
[ 141.032057] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 141.051548] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 141.051815] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 141.091816] Call Trace:
[ 141.091945] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 141.111379] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 141.111673] NMI backtrace for cpu 14
[ 141.111839] CPU 14
[ 141.131299] Modules linked in:
[ 141.131492]
[ 141.131579] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 141.151543] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 141.171304] RSP: 0000:ffff88103efa3ee8 EFLAGS: 00000246
[ 141.171553] RAX: 0000000000000000 RBX: ffff88103efa3fd8 RCX: 0000000000000000
[ 141.191319] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef9a2b0
[ 141.211122] RBP: ffff88103efa3ef8 R08: 0000000000000000 R09: 0000000000000000
[ 141.211430] R10: 0000000000000000 R11: ffff88103efa3ed8 R12: 000000000000000e
[ 141.231204] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 141.231532] FS: 0000000000000000(0000) GS:ffff88207ea00000(0000) knlGS:0000000000000000
[ 141.251365] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 141.271218] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 141.271561] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 141.291197] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 141.310971] Process kworker/0:1 (pid: 0, threadinfo ffff88103efa2000, task ffff88103ef9a2b0)
[ 141.311371] Stack:
[ 141.330919] ffff88103efa2000 ffffffff82525380 ffff88103efa3f28 ffffffff810391f0
[ 141.331318] ffff88103efa3f18 0ab959daf09fa942 0000000000000000 0000000000000000
[ 141.351115] ffff88103efa3f48 ffffffff81c064f3 0000000000000000 4298453ed66e2456
[ 141.370848] Call Trace:
[ 141.370981] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 141.371225] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 141.390899] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 141.430647] Call Trace:
[ 141.430774] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 141.431011] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 141.450711] NMI backtrace for cpu 15
[ 141.450896] CPU 15
[ 141.451011] Modules linked in:
[ 141.451189]
[ 141.470621] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 141.490630] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 141.491003] RSP: 0000:ffff88103efb1ee8 EFLAGS: 00000246
[ 141.510686] RAX: 0000000000000000 RBX: ffff88103efb1fd8 RCX: 0000000000000000
[ 141.510979] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efaa2b0
[ 141.530758] RBP: ffff88103efb1ef8 R08: 0000000000000000 R09: 0000000000000000
[ 141.555764] R10: 0000000000000000 R11: ffff88103efb1ed8 R12: 000000000000000f
[ 141.556182] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 141.570663] FS: 0000000000000000(0000) GS:ffff88207ec00000(0000) knlGS:0000000000000000
[ 141.590492] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 141.590742] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 141.610514] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 141.630305] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 141.630631] Process kworker/0:1 (pid: 0, threadinfo ffff88103efb0000, task ffff88103efaa2b0)
[ 141.650438] Stack:
[ 141.650531] ffff88103efb0000 ffffffff82525380 ffff88103efb1f28 ffffffff810391f0
[ 141.670404] ffff88103efb1f18 c089ea705be6a98b 0000000000000000 0000000000000000
[ 141.690174] ffff88103efb1f48 ffffffff81c064f3 0000000000000000 f5949bd6339eff74
[ 141.690594] Call Trace:
[ 141.690713] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 141.710299] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 141.710559] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 141.750698] Call Trace:
[ 141.750815] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 141.770102] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 141.770384] NMI backtrace for cpu 16
[ 141.789970] CPU 16
[ 141.790081] Modules linked in:
[ 141.790289]
[ 141.790389] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 141.810304] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 141.830083] RSP: 0000:ffff88103efd1ee8 EFLAGS: 00000246
[ 141.830332] RAX: 0000000000000000 RBX: ffff88103efd1fd8 RCX: 0000000000000000
[ 141.850103] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efc22b0
[ 141.869892] RBP: ffff88103efd1ef8 R08: 0000000000000000 R09: 0000000000000000
[ 141.870237] R10: 0000000000000000 R11: ffff88103efd1ed8 R12: 0000000000000010
[ 141.889988] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 141.909768] FS: 0000000000000000(0000) GS:ffff88307de00000(0000) knlGS:0000000000000000
[ 141.910145] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 141.930014] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 141.949891] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 141.950219] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 141.969788] Process kworker/0:1 (pid: 0, threadinfo ffff88103efd0000, task ffff88103efc22b0)
[ 141.989625] Stack:
[ 141.989728] ffff88103efd0000 ffffffff82525380 ffff88103efd1f28 ffffffff810391f0
[ 141.990131] ffff88103efd1f18 d308e5f636b1a8a6 0000000000000000 0000000000000000
[ 142.010042] ffff88103efd1f48 ffffffff81c064f3 0000000000000000 6ff58ab7fd41784e
[ 142.029662] Call Trace:
[ 142.029803] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 142.030037] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 142.049772] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 142.089841] Call Trace:
[ 142.089956] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 142.090215] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 142.109532] NMI backtrace for cpu 17
[ 142.109722] CPU 17
[ 142.109821] Modules linked in:
[ 142.129537]
[ 142.129634] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 142.169733] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 142.189467] RSP: 0000:ffff88103efe1ee8 EFLAGS: 00000246
[ 142.189728] RAX: 0000000000000000 RBX: ffff88103efe1fd8 RCX: 0000000000000000
[ 142.214907] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efda2b0
[ 142.215275] RBP: ffff88103efe1ef8 R08: 0000000000000000 R09: 0000000000000000
[ 142.229406] R10: 0000000000000000 R11: ffff88103efe1ed8 R12: 0000000000000011
[ 142.249462] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 142.249788] FS: 0000000000000000(0000) GS:ffff88307e000000(0000) knlGS:0000000000000000
[ 142.279529] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 142.298927] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 142.299266] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 142.319011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 142.319354] Process kworker/0:1 (pid: 0, threadinfo ffff88103efe0000, task ffff88103efda2b0)
[ 142.339157] Stack:
[ 142.339263] ffff88103efe0000 ffffffff82525380 ffff88103efe1f28 ffffffff810391f0
[ 142.359098] ffff88103efe1f18 2cfef7407d13bcdc 0000000000000000 0000000000000000
[ 142.378864] ffff88103efe1f48 ffffffff81c064f3 0000000000000000 5aef71e9d25d79be
[ 142.379280] Call Trace:
[ 142.398788] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 142.399028] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 142.418688] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 142.439455] Call Trace:
[ 142.439602] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 142.458800] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 142.459070] NMI backtrace for cpu 18
[ 142.478648] CPU 18
[ 142.478761] Modules linked in:
[ 142.478954]
[ 142.479053] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 142.498981] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 142.518775] RSP: 0000:ffff88103efe7ee8 EFLAGS: 00000246
[ 142.538455] RAX: 0000000000000000 RBX: ffff88103efe7fd8 RCX: 0000000000000000
[ 142.538785] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efea2b0
[ 142.558556] RBP: ffff88103efe7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 142.578332] R10: 0000000000000000 R11: ffff88103efe7ed8 R12: 0000000000000012
[ 142.578695] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 142.598439] FS: 0000000000000000(0000) GS:ffff88307e200000(0000) knlGS:0000000000000000
[ 142.618278] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 142.618556] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 142.638537] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 142.638904] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 142.658418] Process kworker/0:1 (pid: 0, threadinfo ffff88103efe6000, task ffff88103efea2b0)
[ 142.678265] Stack:
[ 142.678357] ffff88103efe6000 ffffffff82525380 ffff88103efe7f28 ffffffff810391f0
[ 142.698172] ffff88103efe7f18 9996aca01738a61f 0000000000000000 0000000000000000
[ 142.698616] ffff88103efe7f48 ffffffff81c064f3 0000000000000000 40c7708621549c8a
[ 142.718380] Call Trace:
[ 142.718506] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 142.738070] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 142.738368] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 142.778351] Call Trace:
[ 142.778493] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 142.797883] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 142.798156] NMI backtrace for cpu 19
[ 142.798348] CPU 19
[ 142.817835] Modules linked in:
[ 142.818037]
[ 142.818129] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 142.838089] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 142.862710] RSP: 0000:ffff88103e805ee8 EFLAGS: 00000246
[ 142.862964] RAX: 0000000000000000 RBX: ffff88103e805fd8 RCX: 0000000000000000
[ 142.877887] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103effa2b0
[ 142.878204] RBP: ffff88103e805ef8 R08: 0000000000000000 R09: 0000000000000000
[ 142.897981] R10: 0000000000000000 R11: ffff88103e805ed8 R12: 0000000000000013
[ 142.917799] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 142.918130] FS: 0000000000000000(0000) GS:ffff88307e400000(0000) knlGS:0000000000000000
[ 142.939106] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 142.957676] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 142.958011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 142.977809] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 142.997672] Process kworker/0:1 (pid: 0, threadinfo ffff88103e804000, task ffff88103effa2b0)
[ 142.998085] Stack:
[ 143.017695] ffff88103e804000 ffffffff82525380 ffff88103e805f28 ffffffff810391f0
[ 143.018151] ffff88103e805f18 d4ede08c6a7229e8 0000000000000000 0000000000000000
[ 143.037816] ffff88103e805f48 ffffffff81c064f3 0000000000000000 0d5fb9e457f7ceed
[ 143.057424] Call Trace:
[ 143.057539] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 143.057804] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 143.077411] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 143.098284] Call Trace:
[ 143.117331] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 143.117603] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 143.137288] NMI backtrace for cpu 20
[ 143.137473] CPU 20
[ 143.137573] Modules linked in:
[ 143.137783]
[ 143.157190] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 143.177190] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 143.177590] RSP: 0000:ffff88103e825ee8 EFLAGS: 00000246
[ 143.197215] RAX: 0000000000000000 RBX: ffff88103e825fd8 RCX: 0000000000000000
[ 143.197561] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e80a2b0
[ 143.217831] RBP: ffff88103e825ef8 R08: 0000000000000000 R09: 0000000000000000
[ 143.237318] R10: 0000000000000000 R11: ffff88103e825ed8 R12: 0000000000000014
[ 143.237658] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 143.257208] FS: 0000000000000000(0000) GS:ffff88307e600000(0000) knlGS:0000000000000000
[ 143.277028] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 143.277307] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 143.297038] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 143.316856] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 143.317171] Process kworker/0:1 (pid: 0, threadinfo ffff88103e824000, task ffff88103e80a2b0)
[ 143.337024] Stack:
[ 143.337115] ffff88103e824000 ffffffff82525380 ffff88103e825f28 ffffffff810391f0
[ 143.356948] ffff88103e825f18 0744bf4983a87a1b 0000000000000000 0000000000000000
[ 143.357406] ffff88103e825f48 ffffffff81c064f3 0000000000000000 1cb607dd8301e40b
[ 143.377130] Call Trace:
[ 143.377266] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 143.397110] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 143.397386] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 143.437171] Call Trace:
[ 143.437301] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 143.456669] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 143.456963] NMI backtrace for cpu 21
[ 143.476513] CPU 21
[ 143.476611] Modules linked in:
[ 143.476785]
[ 143.476890] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 143.497586] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 143.516629] RSP: 0000:ffff88103e833ee8 EFLAGS: 00000246
[ 143.516873] RAX: 0000000000000000 RBX: ffff88103e833fd8 RCX: 0000000000000000
[ 143.536620] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e82a2b0
[ 143.556431] RBP: ffff88103e833ef8 R08: 0000000000000000 R09: 0000000000000000
[ 143.556789] R10: 0000000000000000 R11: ffff88103e833ed8 R12: 0000000000000015
[ 143.576579] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 143.596325] FS: 0000000000000000(0000) GS:ffff88307e800000(0000) knlGS:0000000000000000
[ 143.596690] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 143.616381] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 143.636171] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 143.636502] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 143.656270] Process kworker/0:1 (pid: 0, threadinfo ffff88103e832000, task ffff88103e82a2b0)
[ 143.676139] Stack:
[ 143.676268] ffff88103e832000 ffffffff82525380 ffff88103e833f28 ffffffff810391f0
[ 143.676696] ffff88103e833f18 9272018b68de3072 0000000000000000 0000000000000000
[ 143.696488] ffff88103e833f48 ffffffff81c064f3 0000000000000000 30c832b73c95166f
[ 143.716202] Call Trace:
[ 143.716330] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 143.716569] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 143.736181] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 143.776036] Call Trace:
[ 143.776163] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 143.776410] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 143.795995] NMI backtrace for cpu 22
[ 143.796190] CPU 22
[ 143.796290] Modules linked in:
[ 143.815859]
[ 143.815964] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 143.835923] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 143.836331] RSP: 0000:ffff88103e849ee8 EFLAGS: 00000246
[ 143.855953] RAX: 0000000000000000 RBX: ffff88103e849fd8 RCX: 0000000000000000
[ 143.875743] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e83a2b0
[ 143.876075] RBP: ffff88103e849ef8 R08: 0000000000000000 R09: 0000000000000000
[ 143.895826] R10: 0000000000000000 R11: ffff88103e849ed8 R12: 0000000000000016
[ 143.915639] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 143.915990] FS: 0000000000000000(0000) GS:ffff88307ea00000(0000) knlGS:0000000000000000
[ 143.935797] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 143.936056] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 143.955929] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 143.975745] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 143.976088] Process kworker/0:1 (pid: 0, threadinfo ffff88103e848000, task ffff88103e83a2b0)
[ 143.996164] Stack:
[ 143.996279] ffff88103e848000 ffffffff82525380 ffff88103e849f28 ffffffff810391f0
[ 144.015921] ffff88103e849f18 55c4bc74ee10c3a9 0000000000000000 0000000000000000
[ 144.035620] ffff88103e849f48 ffffffff81c064f3 0000000000000000 c6af0e0725d8e279
[ 144.036085] Call Trace:
[ 144.055485] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 144.055735] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 144.075270] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 144.096011] Call Trace:
[ 144.096144] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 144.115394] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 144.115679] NMI backtrace for cpu 23
[ 144.135256] CPU 23
[ 144.135358] Modules linked in:
[ 144.135545]
[ 144.135637] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 144.156262] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 144.175570] RSP: 0000:ffff88103e859ee8 EFLAGS: 00000246
[ 144.195054] RAX: 0000000000000000 RBX: ffff88103e859fd8 RCX: 0000000000000000
[ 144.195389] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8522b0
[ 144.215178] RBP: ffff88103e859ef8 R08: 0000000000000000 R09: 0000000000000000
[ 144.234967] R10: 0000000000000000 R11: ffff88103e859ed8 R12: 0000000000000017
[ 144.235325] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 144.255061] FS: 0000000000000000(0000) GS:ffff88307ec00000(0000) knlGS:0000000000000000
[ 144.274868] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 144.275157] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 144.295041] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 144.295381] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 144.315016] Process kworker/0:1 (pid: 0, threadinfo ffff88103e858000, task ffff88103e8522b0)
[ 144.334863] Stack:
[ 144.334969] ffff88103e858000 ffffffff82525380 ffff88103e859f28 ffffffff810391f0
[ 144.354806] ffff88103e859f18 908465f609e5cf69 0000000000000000 0000000000000000
[ 144.355285] ffff88103e859f48 ffffffff81c064f3 0000000000000000 901034fa9e9275cf
[ 144.374955] Call Trace:
[ 144.375070] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 144.394687] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 144.394970] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 144.434889] Call Trace:
[ 144.435031] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 144.435292] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 144.454765] NMI backtrace for cpu 24
[ 144.454967] CPU 24
[ 144.455060] Modules linked in:
[ 144.474733]
[ 144.474821] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 144.494676] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 144.514460] RSP: 0000:ffff88103e85fee8 EFLAGS: 00000246
[ 144.514725] RAX: 0000000000000000 RBX: ffff88103e85ffd8 RCX: 0000000000000000
[ 144.534483] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e864560
[ 144.534810] RBP: ffff88103e85fef8 R08: 0000000000000000 R09: 0000000000000000
[ 144.554590] R10: 0000000000000000 R11: ffff88103e85fed8 R12: 0000000000000018
[ 144.574373] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 144.574733] FS: 0000000000000000(0000) GS:ffff88407de00000(0000) knlGS:0000000000000000
[ 144.594545] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 144.614252] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 144.614576] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 144.634345] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 144.654133] Process kworker/0:1 (pid: 0, threadinfo ffff88103e85e000, task ffff88103e864560)
[ 144.654530] Stack:
[ 144.674053] ffff88103e85e000 ffffffff82525380 ffff88103e85ff28 ffffffff810391f0
[ 144.674514] ffff88103e85ff18 39cac7e094841c82 0000000000000000 0000000000000000
[ 144.694241] ffff88103e85ff48 ffffffff81c064f3 0000000000000000 2dcf15bc6b5f3387
[ 144.714026] Call Trace:
[ 144.714173] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 144.714426] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 144.734049] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 144.754866] Call Trace:
[ 144.773919] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 144.774151] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 144.795051] NMI backtrace for cpu 25
[ 144.795234] CPU 25
[ 144.795340] Modules linked in:
[ 144.795511]
[ 144.813758] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 144.833771] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 144.834146] RSP: 0000:ffff88103e875ee8 EFLAGS: 00000246
[ 144.853849] RAX: 0000000000000000 RBX: ffff88103e875fd8 RCX: 0000000000000000
[ 144.854177] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e884560
[ 144.873888] RBP: ffff88103e875ef8 R08: 0000000000000000 R09: 0000000000000000
[ 144.893689] R10: 0000000000000000 R11: ffff88103e875ed8 R12: 0000000000000019
[ 144.894027] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 144.913789] FS: 0000000000000000(0000) GS:ffff88407e000000(0000) knlGS:0000000000000000
[ 144.933631] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 144.933898] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 144.953718] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 144.973714] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 144.974058] Process kworker/0:1 (pid: 0, threadinfo ffff88103e874000, task ffff88103e884560)
[ 144.993800] Stack:
[ 144.993892] ffff88103e874000 ffffffff82525380 ffff88103e875f28 ffffffff810391f0
[ 145.013686] ffff88103e875f18 0a3cfc4bd45819c1 0000000000000000 0000000000000000
[ 145.014135] ffff88103e875f48 ffffffff81c064f3 0000000000000000 6e7bcc21ed33ee47
[ 145.033735] Call Trace:
[ 145.033868] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 145.053445] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 145.053733] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 145.093757] Call Trace:
[ 145.093889] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 145.113231] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 145.113524] NMI backtrace for cpu 26
[ 145.113697] CPU 26
[ 145.133220] Modules linked in:
[ 145.133387]
[ 145.133489] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 145.153444] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 145.173225] RSP: 0000:ffff88103e893ee8 EFLAGS: 00000246
[ 145.173498] RAX: 0000000000000000 RBX: ffff88103e893fd8 RCX: 0000000000000000
[ 145.193256] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e88c560
[ 145.213024] RBP: ffff88103e893ef8 R08: 0000000000000000 R09: 0000000000000000
[ 145.213354] R10: 0000000000000000 R11: ffff88103e893ed8 R12: 000000000000001a
[ 145.233866] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 145.252944] FS: 0000000000000000(0000) GS:ffff88407e200000(0000) knlGS:0000000000000000
[ 145.253340] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 145.273003] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 145.273344] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 145.293091] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 145.312900] Process kworker/0:1 (pid: 0, threadinfo ffff88103e892000, task ffff88103e88c560)
[ 145.332710] Stack:
[ 145.332810] ffff88103e892000 ffffffff82525380 ffff88103e893f28 ffffffff810391f0
[ 145.333249] ffff88103e893f18 66000d941fa900df 0000000000000000 0000000000000000
[ 145.353045] ffff88103e893f48 ffffffff81c064f3 0000000000000000 fcf39f4b7ffe7411
[ 145.372789] Call Trace:
[ 145.372924] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 145.373167] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 145.413174] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 145.453313] Call Trace:
[ 145.453445] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 145.453693] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 145.472568] NMI backtrace for cpu 27
[ 145.472745] CPU 27
[ 145.472848] Modules linked in:
[ 145.492451]
[ 145.492543] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 145.512476] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 145.512900] RSP: 0000:ffff88103e8a9ee8 EFLAGS: 00000246
[ 145.532517] RAX: 0000000000000000 RBX: ffff88103e8a9fd8 RCX: 0000000000000000
[ 145.552323] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8a4560
[ 145.552650] RBP: ffff88103e8a9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 145.572405] R10: 0000000000000000 R11: ffff88103e8a9ed8 R12: 000000000000001b
[ 145.592190] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 145.592531] FS: 0000000000000000(0000) GS:ffff88407e400000(0000) knlGS:0000000000000000
[ 145.612347] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 145.612595] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 145.632371] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 145.652160] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 145.652497] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8a8000, task ffff88103e8a4560)
[ 145.672331] Stack:
[ 145.672432] ffff88103e8a8000 ffffffff82525380 ffff88103e8a9f28 ffffffff810391f0
[ 145.692272] ffff88103e8a9f18 df03c4b0aef93378 0000000000000000 0000000000000000
[ 145.712012] ffff88103e8a9f48 ffffffff81c064f3 0000000000000000 6f15585bec259797
[ 145.712423] Call Trace:
[ 145.731931] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 145.732172] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 145.732428] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 145.772555] Call Trace:
[ 145.772700] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 145.791961] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 145.792260] NMI backtrace for cpu 28
[ 145.811820] CPU 28
[ 145.811918] Modules linked in:
[ 145.812103]
[ 145.812191] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 145.832145] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 145.851958] RSP: 0000:ffff88103e8dfee8 EFLAGS: 00000246
[ 145.871617] RAX: 0000000000000000 RBX: ffff88103e8dffd8 RCX: 0000000000000000
[ 145.871942] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8bc560
[ 145.891718] RBP: ffff88103e8dfef8 R08: 0000000000000000 R09: 0000000000000000
[ 145.892044] R10: 0000000000000000 R11: ffff88103e8dfed8 R12: 000000000000001c
[ 145.911835] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 145.931622] FS: 0000000000000000(0000) GS:ffff88407e600000(0000) knlGS:0000000000000000
[ 145.931976] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 145.951702] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 145.971542] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 145.971856] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 145.991730] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8de000, task ffff88103e8bc560)
[ 146.011809] Stack:
[ 146.011900] ffff88103e8de000 ffffffff82525380 ffff88103e8dff28 ffffffff810391f0
[ 146.031465] ffff88103e8dff18 413f0ec1ba26191e 0000000000000000 0000000000000000
[ 146.031947] ffff88103e8dff48 ffffffff81c064f3 0000000000000000 a5eb12480065c24e
[ 146.051796] Call Trace:
[ 146.051942] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 146.071240] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 146.071509] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 146.111423] Call Trace:
[ 146.111559] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 146.111807] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 146.131412] NMI backtrace for cpu 29
[ 146.131602] CPU 29
[ 146.131702] Modules linked in:
[ 146.151356]
[ 146.151458] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 146.171256] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 146.191027] RSP: 0000:ffff88103e8e7ee8 EFLAGS: 00000246
[ 146.191302] RAX: 0000000000000000 RBX: ffff88103e8e7fd8 RCX: 0000000000000000
[ 146.211054] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8ec560
[ 146.211390] RBP: ffff88103e8e7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 146.231169] R10: 0000000000000000 R11: ffff88103e8e7ed8 R12: 000000000000001d
[ 146.250948] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 146.251264] FS: 0000000000000000(0000) GS:ffff88407e800000(0000) knlGS:0000000000000000
[ 146.271080] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 146.290841] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 146.291181] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 146.311243] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 146.330731] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8e6000, task ffff88103e8ec560)
[ 146.331132] Stack:
[ 146.331248] ffff88103e8e6000 ffffffff82525380 ffff88103e8e7f28 ffffffff810391f0
[ 146.351311] ffff88103e8e7f18 d32ef0dd85d433ad 0000000000000000 0000000000000000
[ 146.371016] ffff88103e8e7f48 ffffffff81c064f3 0000000000000000 a92aa153b95503a5
[ 146.371414] Call Trace:
[ 146.391290] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 146.391561] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 146.410586] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 146.431456] Call Trace:
[ 146.450599] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 146.450836] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 146.470418] NMI backtrace for cpu 30
[ 146.470593] CPU 30
[ 146.470693] Modules linked in:
[ 146.470901]
[ 146.470976] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 146.510310] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 146.510726] RSP: 0000:ffff88103e8fdee8 EFLAGS: 00000246
[ 146.530366] RAX: 0000000000000000 RBX: ffff88103e8fdfd8 RCX: 0000000000000000
[ 146.530725] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e904560
[ 146.550485] RBP: ffff88103e8fdef8 R08: 0000000000000000 R09: 0000000000000000
[ 146.570257] R10: 0000000000000000 R11: ffff88103e8fded8 R12: 000000000000001e
[ 146.570570] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 146.590387] FS: 0000000000000000(0000) GS:ffff88407ea00000(0000) knlGS:0000000000000000
[ 146.610177] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 146.610447] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 146.630240] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 146.630576] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 146.650412] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8fc000, task ffff88103e904560)
[ 146.670189] Stack:
[ 146.670287] ffff88103e8fc000 ffffffff82525380 ffff88103e8fdf28 ffffffff810391f0
[ 146.690115] ffff88103e8fdf18 913051ccc440aefd 0000000000000000 0000000000000000
[ 146.690513] ffff88103e8fdf48 ffffffff81c064f3 0000000000000000 621a96059901cbc3
[ 146.710278] Call Trace:
[ 146.710398] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 146.730006] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 146.730287] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 146.770294] Call Trace:
[ 146.770412] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 146.789833] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 146.790121] NMI backtrace for cpu 32
[ 146.790346] CPU 32
[ 146.809763] Modules linked in:
[ 146.809964]
[ 146.810054] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 146.830045] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 146.849776] RSP: 0000:ffff88103e933ee8 EFLAGS: 00000246
[ 146.850020] RAX: 0000000000000000 RBX: ffff88103e933fd8 RCX: 0000000000000000
[ 146.869805] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e92c560
[ 146.889563] RBP: ffff88103e933ef8 R08: 0000000000000000 R09: 0000000000000000
[ 146.889911] R10: 0000000000000000 R11: ffff88103e933ed8 R12: 0000000000000020
[ 146.909684] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 146.929468] FS: 0000000000000000(0000) GS:ffff88507de00000(0000) knlGS:0000000000000000
[ 146.929838] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 146.949546] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 146.949896] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 146.971294] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 146.989462] Process kworker/0:1 (pid: 0, threadinfo ffff88103e932000, task ffff88103e92c560)
[ 146.989840] Stack:
[ 147.009427] ffff88103e932000 ffffffff82525380 ffff88103e933f28 ffffffff810391f0
[ 147.009877] ffff88103e933f18 cc93d566d9aab073 0000000000000000 0000000000000000
[ 147.029800] ffff88103e933f48 ffffffff81c064f3 0000000000000000 0892a3361bfa89c4
[ 147.049752] Call Trace:
[ 147.049901] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 147.050162] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 147.069459] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 147.109150] Call Trace:
[ 147.109282] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 147.109531] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 147.129165] NMI backtrace for cpu 33
[ 147.129355] CPU 33
[ 147.129455] Modules linked in:
[ 147.149006]
[ 147.149103] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 147.169122] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 147.169515] RSP: 0000:ffff88103e941ee8 EFLAGS: 00000246
[ 147.189157] RAX: 0000000000000000 RBX: ffff88103e941fd8 RCX: 0000000000000000
[ 147.208902] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e93c560
[ 147.209238] RBP: ffff88103e941ef8 R08: 0000000000000000 R09: 0000000000000000
[ 147.229248] R10: 0000000000000000 R11: ffff88103e941ed8 R12: 0000000000000021
[ 147.229602] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 147.249097] FS: 0000000000000000(0000) GS:ffff88507e000000(0000) knlGS:0000000000000000
[ 147.268935] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 147.269201] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 147.289003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 147.308778] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 147.309104] Process kworker/0:1 (pid: 0, threadinfo ffff88103e940000, task ffff88103e93c560)
[ 147.328926] Stack:
[ 147.329031] ffff88103e940000 ffffffff82525380 ffff88103e941f28 ffffffff810391f0
[ 147.348891] ffff88103e941f18 ab385c0f8fb97b87 0000000000000000 0000000000000000
[ 147.368624] ffff88103e941f48 ffffffff81c064f3 0000000000000000 bbbf54853ffbc8f6
[ 147.369042] Call Trace:
[ 147.369173] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 147.390147] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 147.390442] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 147.429144] Call Trace:
[ 147.429258] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 147.448575] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 147.448889] NMI backtrace for cpu 34
[ 147.468539] CPU 34
[ 147.468637] Modules linked in:
[ 147.468842]
[ 147.468918] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 147.488774] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 147.508554] RSP: 0000:ffff88103e947ee8 EFLAGS: 00000246
[ 147.508813] RAX: 0000000000000000 RBX: ffff88103e947fd8 RCX: 0000000000000000
[ 147.528534] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e954560
[ 147.548340] RBP: ffff88103e947ef8 R08: 0000000000000000 R09: 0000000000000000
[ 147.548688] R10: 0000000000000000 R11: ffff88103e947ed8 R12: 0000000000000022
[ 147.568442] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 147.593543] FS: 0000000000000000(0000) GS:ffff88507e200000(0000) knlGS:0000000000000000
[ 147.593938] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 147.608329] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 147.628094] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 147.628428] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 147.648223] Process kworker/0:1 (pid: 0, threadinfo ffff88103e946000, task ffff88103e954560)
[ 147.668048] Stack:
[ 147.668146] ffff88103e946000 ffffffff82525380 ffff88103e947f28 ffffffff810391f0
[ 147.687912] ffff88103e947f18 c3e1387ef72955fb 0000000000000000 0000000000000000
[ 147.688367] ffff88103e947f48 ffffffff81c064f3 0000000000000000 3b02cb3a56ca3450
[ 147.708128] Call Trace:
[ 147.708264] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 147.727833] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 147.728107] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 147.768011] Call Trace:
[ 147.768133] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 147.768364] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 147.787920] NMI backtrace for cpu 35
[ 147.788127] CPU 35
[ 147.788220] Modules linked in:
[ 147.807761]
[ 147.807868] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 147.827846] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 147.847643] RSP: 0000:ffff88103e95fee8 EFLAGS: 00000246
[ 147.847920] RAX: 0000000000000000 RBX: ffff88103e95ffd8 RCX: 0000000000000000
[ 147.867634] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e964560
[ 147.868005] RBP: ffff88103e95fef8 R08: 0000000000000000 R09: 0000000000000000
[ 147.887786] R10: 0000000000000000 R11: ffff88103e95fed8 R12: 0000000000000023
[ 147.907528] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 147.907897] FS: 0000000000000000(0000) GS:ffff88507e400000(0000) knlGS:0000000000000000
[ 147.927674] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 147.947412] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 147.947790] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 147.967603] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 147.967925] Process kworker/0:1 (pid: 0, threadinfo ffff88103e95e000, task ffff88103e964560)
[ 147.987807] Stack:
[ 147.987908] ffff88103e95e000 ffffffff82525380 ffff88103e95ff28 ffffffff810391f0
[ 148.008237] ffff88103e95ff18 b04a6bcc5d16a033 0000000000000000 0000000000000000
[ 148.028386] ffff88103e95ff48 ffffffff81c064f3 0000000000000000 86562f64e45d9875
[ 148.028826] Call Trace:
[ 148.047569] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 148.047852] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 148.067261] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 148.088104] Call Trace:
[ 148.107087] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 148.107324] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 148.107595] NMI backtrace for cpu 36
[ 148.127204] CPU 36
[ 148.127307] Modules linked in:
[ 148.127492]
[ 148.127592] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 148.147493] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 148.167294] RSP: 0000:ffff88103e96dee8 EFLAGS: 00000246
[ 148.187190] RAX: 0000000000000000 RBX: ffff88103e96dfd8 RCX: 0000000000000000
[ 148.187541] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e974560
[ 148.207048] RBP: ffff88103e96def8 R08: 0000000000000000 R09: 0000000000000000
[ 148.226862] R10: 0000000000000000 R11: ffff88103e96ded8 R12: 0000000000000024
[ 148.227187] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 148.246985] FS: 0000000000000000(0000) GS:ffff88507e600000(0000) knlGS:0000000000000000
[ 148.266789] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 148.267065] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 148.286836] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 148.287182] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 148.306949] Process kworker/0:1 (pid: 0, threadinfo ffff88103e96c000, task ffff88103e974560)
[ 148.326800] Stack:
[ 148.326914] ffff88103e96c000 ffffffff82525380 ffff88103e96df28 ffffffff810391f0
[ 148.346724] ffff88103e96df18 acf4d12aa3bac806 0000000000000000 0000000000000000
[ 148.347157] ffff88103e96df48 ffffffff81c064f3 0000000000000000 f751ed781526a858
[ 148.366914] Call Trace:
[ 148.367029] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 148.386641] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 148.386909] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 148.427152] Call Trace:
[ 148.427274] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 148.446410] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 148.446654] NMI backtrace for cpu 37
[ 148.446833] CPU 37
[ 148.466376] Modules linked in:
[ 148.466589]
[ 148.466691] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 148.486626] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 148.506390] RSP: 0000:ffff88103e993ee8 EFLAGS: 00000246
[ 148.506644] RAX: 0000000000000000 RBX: ffff88103e993fd8 RCX: 0000000000000000
[ 148.526396] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e98c560
[ 148.546191] RBP: ffff88103e993ef8 R08: 0000000000000000 R09: 0000000000000000
[ 148.546529] R10: 0000000000000000 R11: ffff88103e993ed8 R12: 0000000000000025
[ 148.566297] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 148.566608] FS: 0000000000000000(0000) GS:ffff88507e800000(0000) knlGS:0000000000000000
[ 148.586422] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 148.606177] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 148.606523] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 148.626281] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 148.646040] Process kworker/0:1 (pid: 0, threadinfo ffff88103e992000, task ffff88103e98c560)
[ 148.646449] Stack:
[ 148.670787] ffff88103e992000 ffffffff82525380 ffff88103e993f28 ffffffff810391f0
[ 148.671236] ffff88103e993f18 fb85c5af419ff264 0000000000000000 0000000000000000
[ 148.686415] ffff88103e993f48 ffffffff81c064f3 0000000000000000 9331422f9e8ccc3d
[ 148.705894] Call Trace:
[ 148.706023] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 148.706270] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 148.725959] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 148.746866] Call Trace:
[ 148.765852] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 148.766124] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 148.785780] NMI backtrace for cpu 40
[ 148.785966] CPU 40
[ 148.786077] Modules linked in:
[ 148.786246]
[ 148.805683] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 148.825696] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 148.826070] RSP: 0000:ffff88103e9bdee8 EFLAGS: 00000246
[ 148.845734] RAX: 0000000000000000 RBX: ffff88103e9bdfd8 RCX: 0000000000000000
[ 148.846090] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9c8000
[ 148.865826] RBP: ffff88103e9bdef8 R08: 0000000000000000 R09: 0000000000000000
[ 148.885628] R10: 0000000000000000 R11: ffff88103e9bded8 R12: 0000000000000028
[ 148.885953] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 148.905723] FS: 0000000000000000(0000) GS:ffff88607de00000(0000) knlGS:0000000000000000
[ 148.925555] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 148.925826] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 148.945565] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 148.965384] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 148.965728] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9bc000, task ffff88103e9c8000)
[ 148.985555] Stack:
[ 148.985648] ffff88103e9bc000 ffffffff82525380 ffff88103e9bdf28 ffffffff810391f0
[ 149.005650] ffff88103e9bdf18 8884672454ef25f4 0000000000000000 0000000000000000
[ 149.025515] ffff88103e9bdf48 ffffffff81c064f3 0000000000000000 c3c2f6eeaf432544
[ 149.025926] Call Trace:
[ 149.026053] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 149.045493] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 149.045764] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 149.085854] Call Trace:
[ 149.085976] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 149.105177] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 149.105478] NMI backtrace for cpu 42
[ 149.125059] CPU 42
[ 149.125164] Modules linked in:
[ 149.125383]
[ 149.125460] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 149.145540] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 149.165163] RSP: 0000:ffff88103e9ebee8 EFLAGS: 00000246
[ 149.165452] RAX: 0000000000000000 RBX: ffff88103e9ebfd8 RCX: 0000000000000000
[ 149.185144] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9f0000
[ 149.204974] RBP: ffff88103e9ebef8 R08: 0000000000000000 R09: 0000000000000000
[ 149.205299] R10: 0000000000000000 R11: ffff88103e9ebed8 R12: 000000000000002a
[ 149.225235] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 149.244814] FS: 0000000000000000(0000) GS:ffff88607e200000(0000) knlGS:0000000000000000
[ 149.245197] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 149.265453] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 149.284690] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 149.285023] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 149.304772] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9ea000, task ffff88103e9f0000)
[ 149.325516] Stack:
[ 149.325619] ffff88103e9ea000 ffffffff82525380 ffff88103e9ebf28 ffffffff810391f0
[ 149.326088] ffff88103e9ebf18 b7850adb4e3fa3f3 0000000000000000 0000000000000000
[ 149.344913] ffff88103e9ebf48 ffffffff81c064f3 0000000000000000 cba24f187a031774
[ 149.364697] Call Trace:
[ 149.364828] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 149.365069] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 149.384751] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 149.424582] Call Trace:
[ 149.424714] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 149.424975] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 149.444530] NMI backtrace for cpu 41
[ 149.444721] CPU 41
[ 149.444843] Modules linked in:
[ 149.464363]
[ 149.464455] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 149.484442] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 149.504229] RSP: 0000:ffff88103e9d5ee8 EFLAGS: 00000246
[ 149.504490] RAX: 0000000000000000 RBX: ffff88103e9d5fd8 RCX: 0000000000000000
[ 149.524276] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9e0000
[ 149.524620] RBP: ffff88103e9d5ef8 R08: 0000000000000000 R09: 0000000000000000
[ 149.544358] R10: 0000000000000000 R11: ffff88103e9d5ed8 R12: 0000000000000029
[ 149.564177] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 149.564512] FS: 0000000000000000(0000) GS:ffff88607e000000(0000) knlGS:0000000000000000
[ 149.584333] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 149.584601] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 149.604398] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 149.624129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 149.624494] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9d4000, task ffff88103e9e0000)
[ 149.644334] Stack:
[ 149.644449] ffff88103e9d4000 ffffffff82525380 ffff88103e9d5f28 ffffffff810391f0
[ 149.664249] ffff88103e9d5f18 f21ec8d6397b0a89 0000000000000000 0000000000000000
[ 149.684023] ffff88103e9d5f48 ffffffff81c064f3 0000000000000000 37d0a573f1bb7502
[ 149.684455] Call Trace:
[ 149.703863] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 149.704114] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 149.723783] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 149.744613] Call Trace:
[ 149.744743] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 149.763901] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 149.764220] NMI backtrace for cpu 43
[ 149.783770] CPU 43
[ 149.783882] Modules linked in:
[ 149.784092]
[ 149.784184] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 149.804123] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 149.823907] RSP: 0000:ffff88103ea11ee8 EFLAGS: 00000246
[ 149.843590] RAX: 0000000000000000 RBX: ffff88103ea11fd8 RCX: 0000000000000000
[ 149.843928] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea08000
[ 149.863690] RBP: ffff88103ea11ef8 R08: 0000000000000000 R09: 0000000000000000
[ 149.883452] R10: 0000000000000000 R11: ffff88103ea11ed8 R12: 000000000000002b
[ 149.883791] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 149.903583] FS: 0000000000000000(0000) GS:ffff88607e400000(0000) knlGS:0000000000000000
[ 149.923406] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 149.923672] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 149.943429] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 149.943764] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 149.964167] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea10000, task ffff88103ea08000)
[ 149.983386] Stack:
[ 149.983501] ffff88103ea10000 ffffffff82525380 ffff88103ea11f28 ffffffff810391f0
[ 150.003322] ffff88103ea11f18 427e6af2afeb849d 0000000000000000 0000000000000000
[ 150.003762] ffff88103ea11f48 ffffffff81c064f3 0000000000000000 2a7ca1ca1d9c5d76
[ 150.023654] Call Trace:
[ 150.023807] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 150.043573] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 150.043854] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 150.083583] Call Trace:
[ 150.083715] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 150.103023] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 150.103305] NMI backtrace for cpu 31
[ 150.103487] CPU 31
[ 150.122956] Modules linked in:
[ 150.123144]
[ 150.123237] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 150.143189] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 150.163215] RSP: 0000:ffff88103e913ee8 EFLAGS: 00000246
[ 150.163475] RAX: 0000000000000000 RBX: ffff88103e913fd8 RCX: 0000000000000000
[ 150.183032] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e90c560
[ 150.183340] RBP: ffff88103e913ef8 R08: 0000000000000000 R09: 0000000000000000
[ 150.203113] R10: 0000000000000000 R11: ffff88103e913ed8 R12: 000000000000001f
[ 150.222906] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 150.223225] FS: 0000000000000000(0000) GS:ffff88407ec00000(0000) knlGS:0000000000000000
[ 150.243031] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 150.262763] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 150.263096] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 150.282880] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 150.302627] Process kworker/0:1 (pid: 0, threadinfo ffff88103e912000, task ffff88103e90c560)
[ 150.303003] Stack:
[ 150.322564] ffff88103e912000 ffffffff82525380 ffff88103e913f28 ffffffff810391f0
[ 150.322981] ffff88103e913f18 36e0850fd202b94f 0000000000000000 0000000000000000
[ 150.342905] ffff88103e913f48 ffffffff81c064f3 0000000000000000 23640462fccc8760
[ 150.362518] Call Trace:
[ 150.362656] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 150.362927] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 150.382583] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 150.403742] Call Trace:
[ 150.423046] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 150.423294] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 150.442462] NMI backtrace for cpu 48
[ 150.442655] CPU 48
[ 150.442756] Modules linked in:
[ 150.442931]
[ 150.462450] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 150.482459] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 150.482837] RSP: 0000:ffff88103ea6bee8 EFLAGS: 00000246
[ 150.502491] RAX: 0000000000000000 RBX: ffff88103ea6bfd8 RCX: 0000000000000000
[ 150.502796] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea70000
[ 150.522451] RBP: ffff88103ea6bef8 R08: 0000000000000000 R09: 0000000000000000
[ 150.542236] R10: 0000000000000000 R11: ffff88103ea6bed8 R12: 0000000000000030
[ 150.542541] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 150.562303] FS: 0000000000000000(0000) GS:ffff88707de00000(0000) knlGS:0000000000000000
[ 150.582145] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 150.582394] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 150.603461] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 150.621958] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 150.622276] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea6a000, task ffff88103ea70000)
[ 150.642103] Stack:
[ 150.642199] ffff88103ea6a000 ffffffff82525380 ffff88103ea6bf28 ffffffff810391f0
[ 150.662184] ffff88103ea6bf18 0978fc6520c78880 0000000000000000 0000000000000000
[ 150.662561] ffff88103ea6bf48 ffffffff81c064f3 0000000000000000 9353423d8473ae89
[ 150.682209] Call Trace:
[ 150.682348] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 150.701940] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 150.702196] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 150.742272] Call Trace:
[ 150.742395] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 150.761756] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 150.762009] NMI backtrace for cpu 49
[ 150.781726] CPU 49
[ 150.781824] Modules linked in:
[ 150.781995]
[ 150.782082] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 150.801931] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 150.821720] RSP: 0000:ffff88103ea89ee8 EFLAGS: 00000246
[ 150.821962] RAX: 0000000000000000 RBX: ffff88103ea89fd8 RCX: 0000000000000000
[ 150.841744] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea80000
[ 150.861530] RBP: ffff88103ea89ef8 R08: 0000000000000000 R09: 0000000000000000
[ 150.861832] R10: 0000000000000000 R11: ffff88103ea89ed8 R12: 0000000000000031
[ 150.881622] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 150.901428] FS: 0000000000000000(0000) GS:ffff88707e000000(0000) knlGS:0000000000000000
[ 150.901777] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 150.921487] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 150.941299] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 150.941625] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 150.961397] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea88000, task ffff88103ea80000)
[ 150.981223] Stack:
[ 150.981324] ffff88103ea88000 ffffffff82525380 ffff88103ea89f28 ffffffff810391f0
[ 150.981726] ffff88103ea89f18 951b3f639aa8d187 0000000000000000 0000000000000000
[ 151.002912] ffff88103ea89f48 ffffffff81c064f3 0000000000000000 604a3900ef2607fb
[ 151.021327] Call Trace:
[ 151.021454] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 151.021673] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 151.041382] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 151.081354] Call Trace:
[ 151.081467] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 151.081698] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 151.101243] NMI backtrace for cpu 50
[ 151.101407] CPU 50
[ 151.101497] Modules linked in:
[ 151.120977]
[ 151.121058] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 151.141171] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 151.141532] RSP: 0000:ffff88103ea8fee8 EFLAGS: 00000246
[ 151.161045] RAX: 0000000000000000 RBX: ffff88103ea8ffd8 RCX: 0000000000000000
[ 151.180842] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea98000
[ 151.181186] RBP: ffff88103ea8fef8 R08: 0000000000000000 R09: 0000000000000000
[ 151.200975] R10: 0000000000000000 R11: ffff88103ea8fed8 R12: 0000000000000032
[ 151.220978] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 151.221291] FS: 0000000000000000(0000) GS:ffff88707e200000(0000) knlGS:0000000000000000
[ 151.241589] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 151.241838] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 151.260913] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 151.280708] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 151.281016] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea8e000, task ffff88103ea98000)
[ 151.300848] Stack:
[ 151.300944] ffff88103ea8e000 ffffffff82525380 ffff88103ea8ff28 ffffffff810391f0
[ 151.320844] ffff88103ea8ff18 13948b935e8bb551 0000000000000000 0000000000000000
[ 151.340727] ffff88103ea8ff48 ffffffff81c064f3 0000000000000000 799b066110ec0e64
[ 151.341115] Call Trace:
[ 151.360457] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 151.360681] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 151.380646] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 151.401120] Call Trace:
[ 151.401253] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 151.420617] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 151.420870] NMI backtrace for cpu 56
[ 151.440510] CPU 56
[ 151.440610] Modules linked in:
[ 151.440800]
[ 151.440876] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 151.460859] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 151.480498] RSP: 0000:ffff88103eb05ee8 EFLAGS: 00000246
[ 151.500180] RAX: 0000000000000000 RBX: ffff88103eb05fd8 RCX: 0000000000000000
[ 151.500497] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eafc560
[ 151.520264] RBP: ffff88103eb05ef8 R08: 0000000000000000 R09: 0000000000000000
[ 151.520575] R10: 0000000000000000 R11: ffff88103eb05ed8 R12: 0000000000000038
[ 151.540745] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 151.560192] FS: 0000000000000000(0000) GS:ffff88807d800000(0000) knlGS:0000000000000000
[ 151.580000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 151.580255] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 151.600026] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 151.600345] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 151.620157] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb04000, task ffff88103eafc560)
[ 151.640138] Stack:
[ 151.640244] ffff88103eb04000 ffffffff82525380 ffff88103eb05f28 ffffffff810391f0
[ 151.659894] ffff88103eb05f18 1f33aead7ea68430 0000000000000000 0000000000000000
[ 151.660313] ffff88103eb05f48 ffffffff81c064f3 0000000000000000 595592b75fa60d89
[ 151.680074] Call Trace:
[ 151.680190] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 151.699819] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 151.700082] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 151.740150] Call Trace:
[ 151.740272] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 151.740513] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 151.759862] NMI backtrace for cpu 47
[ 151.760045] CPU 47
[ 151.760158] Modules linked in:
[ 151.779745]
[ 151.779852] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 151.799837] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 151.819578] RSP: 0000:ffff88103ea53ee8 EFLAGS: 00000246
[ 151.819830] RAX: 0000000000000000 RBX: ffff88103ea53fd8 RCX: 0000000000000000
[ 151.839604] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea58000
[ 151.839946] RBP: ffff88103ea53ef8 R08: 0000000000000000 R09: 0000000000000000
[ 151.859711] R10: 0000000000000000 R11: ffff88103ea53ed8 R12: 000000000000002f
[ 151.879497] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 151.879830] FS: 0000000000000000(0000) GS:ffff88607ec00000(0000) knlGS:0000000000000000
[ 151.900927] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 151.919469] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 151.919823] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 151.939479] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 151.959236] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea52000, task ffff88103ea58000)
[ 151.959634] Stack:
[ 151.979194] ffff88103ea52000 ffffffff82525380 ffff88103ea53f28 ffffffff810391f0
[ 151.979671] ffff88103ea53f18 50617d206f7b41f7 0000000000000000 0000000000000000
[ 151.999370] ffff88103ea53f48 ffffffff81c064f3 0000000000000000 036c0faac09abc9f
[ 152.019268] Call Trace:
[ 152.019426] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 152.019671] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 152.039785] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 152.060107] Call Trace:
[ 152.079243] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 152.079515] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 152.099144] NMI backtrace for cpu 57
[ 152.099326] CPU 57
[ 152.099427] Modules linked in:
[ 152.099600]
[ 152.099676] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 152.138887] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 152.139266] RSP: 0000:ffff88103eb23ee8 EFLAGS: 00000246
[ 152.158951] RAX: 0000000000000000 RBX: ffff88103eb23fd8 RCX: 0000000000000000
[ 152.159318] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb0a2b0
[ 152.179048] RBP: ffff88103eb23ef8 R08: 0000000000000000 R09: 0000000000000000
[ 152.198914] R10: 0000000000000000 R11: ffff88103eb23ed8 R12: 0000000000000039
[ 152.199247] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 152.218930] FS: 0000000000000000(0000) GS:ffff88807da00000(0000) knlGS:0000000000000000
[ 152.238749] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 152.239019] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 152.258793] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 152.278574] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 152.278911] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb22000, task ffff88103eb0a2b0)
[ 152.298711] Stack:
[ 152.298819] ffff88103eb22000 ffffffff82525380 ffff88103eb23f28 ffffffff810391f0
[ 152.318635] ffff88103eb23f18 cfa00dc14662a13f 0000000000000000 0000000000000000
[ 152.319037] ffff88103eb23f48 ffffffff81c064f3 0000000000000000 7b7fb8c1286f7e8a
[ 152.338824] Call Trace:
[ 152.338960] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 152.358541] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 152.358800] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 152.398851] Call Trace:
[ 152.398966] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 152.418372] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 152.418640] NMI backtrace for cpu 44
[ 152.418833] CPU 44
[ 152.438326] Modules linked in:
[ 152.438524]
[ 152.438622] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 152.458574] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 152.478339] RSP: 0000:ffff88103ea17ee8 EFLAGS: 00000246
[ 152.478586] RAX: 0000000000000000 RBX: ffff88103ea17fd8 RCX: 0000000000000000
[ 152.498355] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea18000
[ 152.518562] RBP: ffff88103ea17ef8 R08: 0000000000000000 R09: 0000000000000000
[ 152.518939] R10: 0000000000000000 R11: ffff88103ea17ed8 R12: 000000000000002c
[ 152.539224] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 152.558012] FS: 0000000000000000(0000) GS:ffff88607e600000(0000) knlGS:0000000000000000
[ 152.558383] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 152.578111] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 152.578441] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 152.598218] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 152.618018] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea16000, task ffff88103ea18000)
[ 152.637851] Stack:
[ 152.637974] ffff88103ea16000 ffffffff82525380 ffff88103ea17f28 ffffffff810391f0
[ 152.638402] ffff88103ea17f18 dc4e2695c0884d50 0000000000000000 0000000000000000
[ 152.658134] ffff88103ea17f48 ffffffff81c064f3 0000000000000000 7517c64c020bf9bd
[ 152.677905] Call Trace:
[ 152.678068] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 152.678326] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 152.697921] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 152.737725] Call Trace:
[ 152.737867] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 152.738091] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 152.757760] NMI backtrace for cpu 52
[ 152.757934] CPU 52
[ 152.758025] Modules linked in:
[ 152.777587]
[ 152.777683] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 152.797670] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 152.798037] RSP: 0000:ffff88103eab5ee8 EFLAGS: 00000246
[ 152.817677] RAX: 0000000000000000 RBX: ffff88103eab5fd8 RCX: 0000000000000000
[ 152.837468] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eac0000
[ 152.837771] RBP: ffff88103eab5ef8 R08: 0000000000000000 R09: 0000000000000000
[ 152.857537] R10: 0000000000000000 R11: ffff88103eab5ed8 R12: 0000000000000034
[ 152.877346] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 152.877674] FS: 0000000000000000(0000) GS:ffff88707e600000(0000) knlGS:0000000000000000
[ 152.897481] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 152.897733] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 152.917526] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 152.937326] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 152.937635] Process kworker/0:1 (pid: 0, threadinfo ffff88103eab4000, task ffff88103eac0000)
[ 152.957451] Stack:
[ 152.957547] ffff88103eab4000 ffffffff82525380 ffff88103eab5f28 ffffffff810391f0
[ 152.977438] ffff88103eab5f18 930c2eb4f563a0aa 0000000000000000 0000000000000000
[ 152.997187] ffff88103eab5f48 ffffffff81c064f3 0000000000000000 19e27f03b5b56431
[ 152.997579] Call Trace:
[ 153.017117] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 153.017338] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 153.017600] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 153.057985] Call Trace:
[ 153.058104] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 153.077322] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 153.077582] NMI backtrace for cpu 51
[ 153.097261] CPU 51
[ 153.097354] Modules linked in:
[ 153.097541]
[ 153.097620] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 153.117429] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 153.137096] RSP: 0000:ffff88103eaa5ee8 EFLAGS: 00000246
[ 153.156777] RAX: 0000000000000000 RBX: ffff88103eaa5fd8 RCX: 0000000000000000
[ 153.157085] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eaa8000
[ 153.178037] RBP: ffff88103eaa5ef8 R08: 0000000000000000 R09: 0000000000000000
[ 153.178355] R10: 0000000000000000 R11: ffff88103eaa5ed8 R12: 0000000000000033
[ 153.196963] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 153.216960] FS: 0000000000000000(0000) GS:ffff88707e400000(0000) knlGS:0000000000000000
[ 153.217319] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 153.236846] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 153.256641] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 153.256962] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 153.276744] Process kworker/0:1 (pid: 0, threadinfo ffff88103eaa4000, task ffff88103eaa8000)
[ 153.297121] Stack:
[ 153.297217] ffff88103eaa4000 ffffffff82525380 ffff88103eaa5f28 ffffffff810391f0
[ 153.316494] ffff88103eaa5f18 6c5150727d6760c1 0000000000000000 0000000000000000
[ 153.316903] ffff88103eaa5f48 ffffffff81c064f3 0000000000000000 a26eef57dafda858
[ 153.336663] Call Trace:
[ 153.336780] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 153.356413] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 153.356674] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 153.396587] Call Trace:
[ 153.396710] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 153.396928] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 153.416571] NMI backtrace for cpu 59
[ 153.416754] CPU 59
[ 153.416860] Modules linked in:
[ 153.436338]
[ 153.436432] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 153.456387] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 153.476180] RSP: 0000:ffff88103eb49ee8 EFLAGS: 00000246
[ 153.476420] RAX: 0000000000000000 RBX: ffff88103eb49fd8 RCX: 0000000000000000
[ 153.496211] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb3a2b0
[ 153.496537] RBP: ffff88103eb49ef8 R08: 0000000000000000 R09: 0000000000000000
[ 153.516302] R10: 0000000000000000 R11: ffff88103eb49ed8 R12: 000000000000003b
[ 153.536107] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 153.536425] FS: 0000000000000000(0000) GS:ffff88807de00000(0000) knlGS:0000000000000000
[ 153.556227] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 153.575968] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 153.576275] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 153.596051] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 153.615844] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb48000, task ffff88103eb3a2b0)
[ 153.616225] Stack:
[ 153.616318] ffff88103eb48000 ffffffff82525380 ffff88103eb49f28 ffffffff810391f0
[ 153.636195] ffff88103eb49f18 faa21ce5a923a46e 0000000000000000 0000000000000000
[ 153.656034] ffff88103eb49f48 ffffffff81c064f3 0000000000000000 37ef54861a9d44d1
[ 153.656443] Call Trace:
[ 153.675826] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 153.676062] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 153.695766] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 153.716540] Call Trace:
[ 153.735645] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 153.735886] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 153.755566] NMI backtrace for cpu 38
[ 153.755747] CPU 38
[ 153.755873] Modules linked in:
[ 153.756045]
[ 153.756125] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 153.776086] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 153.796312] RSP: 0000:ffff88103e9a1ee8 EFLAGS: 00000246
[ 153.816517] RAX: 0000000000000000 RBX: ffff88103e9a1fd8 RCX: 0000000000000000
[ 153.816833] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e99c560
[ 153.835672] RBP: ffff88103e9a1ef8 R08: 0000000000000000 R09: 0000000000000000
[ 153.855425] R10: 0000000000000000 R11: ffff88103e9a1ed8 R12: 0000000000000026
[ 153.855748] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 153.875542] FS: 0000000000000000(0000) GS:ffff88507ea00000(0000) knlGS:0000000000000000
[ 153.895361] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 153.895631] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 153.915395] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 153.915726] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 153.935522] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9a0000, task ffff88103e99c560)
[ 153.955341] Stack:
[ 153.955447] ffff88103e9a0000 ffffffff82525380 ffff88103e9a1f28 ffffffff810391f0
[ 153.975253] ffff88103e9a1f18 9647bc97204c30cb 0000000000000000 0000000000000000
[ 153.975691] ffff88103e9a1f48 ffffffff81c064f3 0000000000000000 dced99c97795ba22
[ 153.995441] Call Trace:
[ 153.995561] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 154.015155] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 154.015436] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 154.055760] Call Trace:
[ 154.055869] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 154.075315] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 154.075587] NMI backtrace for cpu 53
[ 154.075758] CPU 53
[ 154.095150] Modules linked in:
[ 154.095325]
[ 154.095412] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 154.115299] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 154.135163] RSP: 0000:ffff88103ead3ee8 EFLAGS: 00000246
[ 154.135400] RAX: 0000000000000000 RBX: ffff88103ead3fd8 RCX: 0000000000000000
[ 154.154947] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ead8000
[ 154.174742] RBP: ffff88103ead3ef8 R08: 0000000000000000 R09: 0000000000000000
[ 154.175055] R10: 0000000000000000 R11: ffff88103ead3ed8 R12: 0000000000000035
[ 154.194870] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 154.214621] FS: 0000000000000000(0000) GS:ffff88707e800000(0000) knlGS:0000000000000000
[ 154.214972] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 154.234707] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 154.235015] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 154.254816] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 154.274588] Process kworker/0:1 (pid: 0, threadinfo ffff88103ead2000, task ffff88103ead8000)
[ 154.274966] Stack:
[ 154.294531] ffff88103ead2000 ffffffff82525380 ffff88103ead3f28 ffffffff810391f0
[ 154.294930] ffff88103ead3f18 4ad8b2f69adc9d19 0000000000000000 0000000000000000
[ 154.314743] ffff88103ead3f48 ffffffff81c064f3 0000000000000000 39135bbcd7b3444d
[ 154.334507] Call Trace:
[ 154.334634] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 154.334855] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 154.354483] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 154.394279] Call Trace:
[ 154.394408] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 154.394632] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 154.414308] NMI backtrace for cpu 39
[ 154.414511] CPU 39
[ 154.414612] Modules linked in:
[ 154.414789]
[ 154.434266] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 154.454256] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 154.454639] RSP: 0000:ffff88103e9a7ee8 EFLAGS: 00000246
[ 154.475040] RAX: 0000000000000000 RBX: ffff88103e9a7fd8 RCX: 0000000000000000
[ 154.494068] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9b4560
[ 154.494399] RBP: ffff88103e9a7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 154.514242] R10: 0000000000000000 R11: ffff88103e9a7ed8 R12: 0000000000000027
[ 154.514578] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 154.534415] FS: 0000000000000000(0000) GS:ffff88507ec00000(0000) knlGS:0000000000000000
[ 154.554298] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 154.554583] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 154.574564] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 154.594101] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 154.594440] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9a6000, task ffff88103e9b4560)
[ 154.614406] Stack:
[ 154.614512] ffff88103e9a6000 ffffffff82525380 ffff88103e9a7f28 ffffffff810391f0
[ 154.634084] ffff88103e9a7f18 38d6f42eba77c3bb 0000000000000000 0000000000000000
[ 154.653780] ffff88103e9a7f48 ffffffff81c064f3 0000000000000000 755b949bb9c12001
[ 154.654233] Call Trace:
[ 154.654356] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 154.674028] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 154.674284] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 154.714309] Call Trace:
[ 154.714444] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 154.733725] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 154.733995] NMI backtrace for cpu 45
[ 154.753592] CPU 45
[ 154.753695] Modules linked in:
[ 154.753892]
[ 154.753990] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 154.773908] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 154.793770] RSP: 0000:ffff88103ea2dee8 EFLAGS: 00000246
[ 154.794049] RAX: 0000000000000000 RBX: ffff88103ea2dfd8 RCX: 0000000000000000
[ 154.813735] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea30000
[ 154.833523] RBP: ffff88103ea2def8 R08: 0000000000000000 R09: 0000000000000000
[ 154.833866] R10: 0000000000000000 R11: ffff88103ea2ded8 R12: 000000000000002d
[ 154.853602] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 154.873380] FS: 0000000000000000(0000) GS:ffff88607e800000(0000) knlGS:0000000000000000
[ 154.873755] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 154.893466] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 154.913330] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 154.913678] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 154.933355] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea2c000, task ffff88103ea30000)
[ 154.953193] Stack:
[ 154.953286] ffff88103ea2c000 ffffffff82525380 ffff88103ea2df28 ffffffff810391f0
[ 154.953734] ffff88103ea2df18 6270779be7160d7f 0000000000000000 0000000000000000
[ 154.973602] ffff88103ea2df48 ffffffff81c064f3 0000000000000000 3dd00b8e57918ee7
[ 154.993277] Call Trace:
[ 154.993418] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 155.012990] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 155.013263] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 155.053196] Call Trace:
[ 155.053329] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 155.053555] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 155.073623] NMI backtrace for cpu 58
[ 155.073793] CPU 58
[ 155.073885] Modules linked in:
[ 155.093438]
[ 155.093532] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 155.114036] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 155.133133] RSP: 0000:ffff88103eb33ee8 EFLAGS: 00000246
[ 155.133378] RAX: 0000000000000000 RBX: ffff88103eb33fd8 RCX: 0000000000000000
[ 155.152847] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb2a2b0
[ 155.153187] RBP: ffff88103eb33ef8 R08: 0000000000000000 R09: 0000000000000000
[ 155.172903] R10: 0000000000000000 R11: ffff88103eb33ed8 R12: 000000000000003a
[ 155.192709] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 155.193015] FS: 0000000000000000(0000) GS:ffff88807dc00000(0000) knlGS:0000000000000000
[ 155.213094] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 155.232578] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 155.232902] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 155.252653] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 155.252979] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb32000, task ffff88103eb2a2b0)
[ 155.272834] Stack:
[ 155.272930] ffff88103eb32000 ffffffff82525380 ffff88103eb33f28 ffffffff810391f0
[ 155.292791] ffff88103eb33f18 3ec07093f886a24c 0000000000000000 0000000000000000
[ 155.312567] ffff88103eb33f48 ffffffff81c064f3 0000000000000000 51920f6bafeca5f6
[ 155.312947] Call Trace:
[ 155.334676] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 155.334930] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 155.352495] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 155.373375] Call Trace:
[ 155.402529] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 155.402802] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 155.403078] NMI backtrace for cpu 46
[ 155.422302] CPU 46
[ 155.422405] Modules linked in:
[ 155.422613]
[ 155.422700] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 155.442647] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 155.462457] RSP: 0000:ffff88103ea43ee8 EFLAGS: 00000246
[ 155.482116] RAX: 0000000000000000 RBX: ffff88103ea43fd8 RCX: 0000000000000000
[ 155.482467] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea48000
[ 155.502221] RBP: ffff88103ea43ef8 R08: 0000000000000000 R09: 0000000000000000
[ 155.521992] R10: 0000000000000000 R11: ffff88103ea43ed8 R12: 000000000000002e
[ 155.522330] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 155.542121] FS: 0000000000000000(0000) GS:ffff88607ea00000(0000) knlGS:0000000000000000
[ 155.561929] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 155.562187] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 155.581979] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 155.582327] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 155.602060] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea42000, task ffff88103ea48000)
[ 155.621940] Stack:
[ 155.622035] ffff88103ea42000 ffffffff82525380 ffff88103ea43f28 ffffffff810391f0
[ 155.641874] ffff88103ea43f18 703d8aaf380a16b7 0000000000000000 0000000000000000
[ 155.642280] ffff88103ea43f48 ffffffff81c064f3 0000000000000000 378e401c49b042c7
[ 155.662025] Call Trace:
[ 155.662161] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 155.681724] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 155.681979] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 155.721995] Call Trace:
[ 155.722115] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 155.741535] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 155.741804] NMI backtrace for cpu 60
[ 155.742016] CPU 60
[ 155.762251] Modules linked in:
[ 155.762427]
[ 155.762516] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 155.781721] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 155.801501] RSP: 0000:ffff88103eb4fee8 EFLAGS: 00000246
[ 155.801736] RAX: 0000000000000000 RBX: ffff88103eb4ffd8 RCX: 0000000000000000
[ 155.821550] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb522b0
[ 155.841322] RBP: ffff88103eb4fef8 R08: 0000000000000000 R09: 0000000000000000
[ 155.841636] R10: 0000000000000000 R11: ffff88103eb4fed8 R12: 000000000000003c
[ 155.861411] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 155.861738] FS: 0000000000000000(0000) GS:ffff88807e000000(0000) knlGS:0000000000000000
[ 155.881570] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 155.901283] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 155.901605] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 155.921374] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 155.941182] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb4e000, task ffff88103eb522b0)
[ 155.941553] Stack:
[ 155.961097] ffff88103eb4e000 ffffffff82525380 ffff88103eb4ff28 ffffffff810391f0
[ 155.961503] ffff88103eb4ff18 e93585d02d707a09 0000000000000000 0000000000000000
[ 155.981321] ffff88103eb4ff48 ffffffff81c064f3 0000000000000000 8eaacf10758f15f5
[ 156.001050] Call Trace:
[ 156.001182] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 156.001412] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 156.021073] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 156.041987] Call Trace:
[ 156.061141] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 156.061381] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 156.081061] NMI backtrace for cpu 61
[ 156.081225] CPU 61
[ 156.081330] Modules linked in:
[ 156.081504]
[ 156.100974] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 156.121073] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 156.121457] RSP: 0000:ffff88103eb65ee8 EFLAGS: 00000246
[ 156.140977] RAX: 0000000000000000 RBX: ffff88103eb65fd8 RCX: 0000000000000000
[ 156.141324] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb5a2b0
[ 156.160959] RBP: ffff88103eb65ef8 R08: 0000000000000000 R09: 0000000000000000
[ 156.181152] R10: 0000000000000000 R11: ffff88103eb65ed8 R12: 000000000000003d
[ 156.181478] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 156.211235] FS: 0000000000000000(0000) GS:ffff88807e200000(0000) knlGS:0000000000000000
[ 156.231217] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 156.231480] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 156.250814] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 156.270483] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 156.270804] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb64000, task ffff88103eb5a2b0)
[ 156.290623] Stack:
[ 156.290726] ffff88103eb64000 ffffffff82525380 ffff88103eb65f28 ffffffff810391f0
[ 156.310588] ffff88103eb65f18 dc7459780c591b2f 0000000000000000 0000000000000000
[ 156.330348] ffff88103eb65f48 ffffffff81c064f3 0000000000000000 fcd3245bcc0358f7
[ 156.330776] Call Trace:
[ 156.330889] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 156.351067] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 156.351311] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 156.390822] Call Trace:
[ 156.390937] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 156.411673] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 156.411946] NMI backtrace for cpu 63
[ 156.430138] CPU 63
[ 156.430240] Modules linked in:
[ 156.430408]
[ 156.430501] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 156.450462] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 156.470261] RSP: 0000:ffff88103eb91ee8 EFLAGS: 00000246
[ 156.470494] RAX: 0000000000000000 RBX: ffff88103eb91fd8 RCX: 0000000000000000
[ 156.490247] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb8a2b0
[ 156.510060] RBP: ffff88103eb91ef8 R08: 0000000000000000 R09: 0000000000000000
[ 156.510374] R10: 0000000000000000 R11: ffff88103eb91ed8 R12: 000000000000003f
[ 156.530133] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 156.549932] FS: 0000000000000000(0000) GS:ffff88807e600000(0000) knlGS:0000000000000000
[ 156.550274] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 156.570064] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 156.589827] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 156.590146] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 156.609918] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb90000, task ffff88103eb8a2b0)
[ 156.629740] Stack:
[ 156.629845] ffff88103eb90000 ffffffff82525380 ffff88103eb91f28 ffffffff810391f0
[ 156.630227] ffff88103eb91f18 84ed79833eb6a337 0000000000000000 0000000000000000
[ 156.650055] ffff88103eb91f48 ffffffff81c064f3 0000000000000000 44a6529148d78bbf
[ 156.669816] Call Trace:
[ 156.669952] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 156.670175] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 156.689798] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 156.729686] Call Trace:
[ 156.729804] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 156.730036] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 156.749627] NMI backtrace for cpu 54
[ 156.749802] CPU 54
[ 156.749903] Modules linked in:
[ 156.769487]
[ 156.769582] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 156.789570] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 156.809354] RSP: 0000:ffff88103eae9ee8 EFLAGS: 00000246
[ 156.809583] RAX: 0000000000000000 RBX: ffff88103eae9fd8 RCX: 0000000000000000
[ 156.829385] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eae0000
[ 156.829698] RBP: ffff88103eae9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 156.849455] R10: 0000000000000000 R11: ffff88103eae9ed8 R12: 0000000000000036
[ 156.869245] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 156.869559] FS: 0000000000000000(0000) GS:ffff88707ea00000(0000) knlGS:0000000000000000
[ 156.889381] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 156.889640] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 156.909545] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 156.929239] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 156.929560] Process kworker/0:1 (pid: 0, threadinfo ffff88103eae8000, task ffff88103eae0000)
[ 156.949369] Stack:
[ 156.949462] ffff88103eae8000 ffffffff82525380 ffff88103eae9f28 ffffffff810391f0
[ 156.969339] ffff88103eae9f18 8234c77185011823 0000000000000000 0000000000000000
[ 156.989095] ffff88103eae9f48 ffffffff81c064f3 0000000000000000 72311f7455859eb1
[ 156.989512] Call Trace:
[ 157.008980] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 157.009215] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 157.028908] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 157.051654] Call Trace:
[ 157.051780] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 157.074210] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 157.074460] NMI backtrace for cpu 55
[ 157.088911] CPU 55
[ 157.089011] Modules linked in:
[ 157.089197]
[ 157.089273] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 157.109258] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 157.129339] RSP: 0000:ffff88103eaefee8 EFLAGS: 00000246
[ 157.148954] RAX: 0000000000000000 RBX: ffff88103eaeffd8 RCX: 0000000000000000
[ 157.149287] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eaf8000
[ 157.168798] RBP: ffff88103eaefef8 R08: 0000000000000000 R09: 0000000000000000
[ 157.188599] R10: 0000000000000000 R11: ffff88103eaefed8 R12: 0000000000000037
[ 157.188918] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 157.211302] FS: 0000000000000000(0000) GS:ffff88707ec00000(0000) knlGS:0000000000000000
[ 157.228522] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 157.228784] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 157.248559] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 157.248868] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 157.268646] Process kworker/0:1 (pid: 0, threadinfo ffff88103eaee000, task ffff88103eaf8000)
[ 157.288488] Stack:
[ 157.288583] ffff88103eaee000 ffffffff82525380 ffff88103eaeff28 ffffffff810391f0
[ 157.308388] ffff88103eaeff18 9e7c99cf43a1a877 0000000000000000 0000000000000000
[ 157.308799] ffff88103eaeff48 ffffffff81c064f3 0000000000000000 d8c1df20b1dbb268
[ 157.328592] Call Trace:
[ 157.328706] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 157.348307] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 157.348584] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 157.388555] Call Trace:
[ 157.388673] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 157.408129] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 157.408368] NMI backtrace for cpu 62
[ 157.408547] CPU 62
[ 157.428099] Modules linked in:
[ 157.428289]
[ 157.428380] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 157.448323] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 157.468105] RSP: 0000:ffff88103eb83ee8 EFLAGS: 00000246
[ 157.468384] RAX: 0000000000000000 RBX: ffff88103eb83fd8 RCX: 0000000000000000
[ 157.488108] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb722b0
[ 157.488454] RBP: ffff88103eb83ef8 R08: 0000000000000000 R09: 0000000000000000
[ 157.508219] R10: 0000000000000000 R11: ffff88103eb83ed8 R12: 000000000000003e
[ 157.528004] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 157.528368] FS: 0000000000000000(0000) GS:ffff88807e400000(0000) knlGS:0000000000000000
[ 157.548156] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 157.567893] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 157.568206] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 157.587971] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 157.607746] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb82000, task ffff88103eb722b0)
[ 157.608116] Stack:
[ 157.628263] ffff88103eb82000 ffffffff82525380 ffff88103eb83f28 ffffffff810391f0
[ 157.628758] ffff88103eb83f18 4e93b1db6fe4c26f 0000000000000000 0000000000000000
[ 157.647847] ffff88103eb83f48 ffffffff81c064f3 0000000000000000 cc3ee426a493ae7b
[ 157.667637] Call Trace:
[ 157.667758] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 157.668008] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 157.687739] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 157.709742] Call Trace:
[ 157.727548] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 157.727762] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 157.747486] NMI backtrace for cpu 64
[ 157.747681] CPU 64
[ 157.747771] Modules linked in:
[ 157.747974]
[ 157.767397] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 157.787407] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 157.787827] RSP: 0000:ffff88103ebbdee8 EFLAGS: 00000246
[ 157.807451] RAX: 0000000000000000 RBX: ffff88103ebbdfd8 RCX: 0000000000000000
[ 157.807774] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ebc0000
[ 157.827550] RBP: ffff88103ebbdef8 R08: 0000000000000000 R09: 0000000000000000
[ 157.847363] R10: 0000000000000000 R11: ffff88103ebbded8 R12: 0000000000000040
[ 157.847704] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 157.867425] FS: 0000000000000000(0000) GS:ffff88107ee00000(0000) knlGS:0000000000000000
[ 157.887265] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 157.887530] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 157.907307] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 157.927079] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 157.927445] Process kworker/0:1 (pid: 0, threadinfo ffff88103ebbc000, task ffff88103ebc0000)
[ 157.947240] Stack:
[ 157.947345] ffff88103ebbc000 ffffffff82525380 ffff88103ebbdf28 ffffffff810391f0
[ 157.967173] ffff88103ebbdf18 da2a35009143e8db 0000000000000000 0000000000000000
[ 157.967569] ffff88103ebbdf48 ffffffff81c064f3 0000000000000000 e202c1b2447d6be3
[ 157.987308] Call Trace:
[ 157.987435] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 158.007065] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 158.007361] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 158.047366] Call Trace:
[ 158.047483] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 158.067043] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 158.067286] NMI backtrace for cpu 67
[ 158.086766] CPU 67
[ 158.086860] Modules linked in:
[ 158.087027]
[ 158.087130] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 158.107106] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 158.127207] RSP: 0000:ffff88103e44dee8 EFLAGS: 00000246
[ 158.127455] RAX: 0000000000000000 RBX: ffff88103e44dfd8 RCX: 0000000000000000
[ 158.147037] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e450000
[ 158.166786] RBP: ffff88103e44def8 R08: 0000000000000000 R09: 0000000000000000
[ 158.167114] R10: 0000000000000000 R11: ffff88103e44ded8 R12: 0000000000000043
[ 158.186752] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 158.206566] FS: 0000000000000000(0000) GS:ffff88107f400000(0000) knlGS:0000000000000000
[ 158.206952] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 158.226680] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 158.227004] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 158.246752] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 158.266520] Process kworker/0:1 (pid: 0, threadinfo ffff88103e44c000, task ffff88103e450000)
[ 158.286347] Stack:
[ 158.286452] ffff88103e44c000 ffffffff82525380 ffff88103e44df28 ffffffff810391f0
[ 158.286848] ffff88103e44df18 ebf5aa6367d481b1 0000000000000000 0000000000000000
[ 158.306616] ffff88103e44df48 ffffffff81c064f3 0000000000000000 27a37715890bf64b
[ 158.326389] Call Trace:
[ 158.326526] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 158.326802] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 158.348029] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 158.386263] Call Trace:
[ 158.386391] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 158.386626] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 158.406330] NMI backtrace for cpu 65
[ 158.406505] CPU 65
[ 158.406602] Modules linked in:
[ 158.426114]
[ 158.426190] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 158.446186] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 158.446590] RSP: 0000:ffff88103ebe7ee8 EFLAGS: 00000246
[ 158.466219] RAX: 0000000000000000 RBX: ffff88103ebe7fd8 RCX: 0000000000000000
[ 158.485966] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ebe8000
[ 158.486302] RBP: ffff88103ebe7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 158.506073] R10: 0000000000000000 R11: ffff88103ebe7ed8 R12: 0000000000000041
[ 158.525901] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 158.526203] FS: 0000000000000000(0000) GS:ffff88107f000000(0000) knlGS:0000000000000000
[ 158.546011] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 158.546279] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 158.566057] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 158.585859] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 158.586252] Process kworker/0:1 (pid: 0, threadinfo ffff88103ebe6000, task ffff88103ebe8000)
[ 158.606014] Stack:
[ 158.606118] ffff88103ebe6000 ffffffff82525380 ffff88103ebe7f28 ffffffff810391f0
[ 158.636016] ffff88103ebe7f18 26ab9a4774f02d67 0000000000000000 0000000000000000
[ 158.656062] ffff88103ebe7f48 ffffffff81c064f3 0000000000000000 3a137c915fdefa51
[ 158.656439] Call Trace:
[ 158.675780] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 158.676008] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 158.695748] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 158.716263] Call Trace:
[ 158.716384] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 158.735749] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 158.735985] NMI backtrace for cpu 66
[ 158.755480] CPU 66
[ 158.755581] Modules linked in:
[ 158.755756]
[ 158.755846] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 158.775821] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 158.795583] RSP: 0000:ffff88103e42dee8 EFLAGS: 00000246
[ 158.815274] RAX: 0000000000000000 RBX: ffff88103e42dfd8 RCX: 0000000000000000
[ 158.815608] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e430000
[ 158.835420] RBP: ffff88103e42def8 R08: 0000000000000000 R09: 0000000000000000
[ 158.835734] R10: 0000000000000000 R11: ffff88103e42ded8 R12: 0000000000000042
[ 158.855505] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 158.875267] FS: 0000000000000000(0000) GS:ffff88107f200000(0000) knlGS:0000000000000000
[ 158.895089] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 158.895353] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 158.915139] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 158.915459] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 158.935242] Process kworker/0:1 (pid: 0, threadinfo ffff88103e42c000, task ffff88103e430000)
[ 158.955078] Stack:
[ 158.955172] ffff88103e42c000 ffffffff82525380 ffff88103e42df28 ffffffff810391f0
[ 158.974989] ffff88103e42df18 ecd678057994030d 0000000000000000 0000000000000000
[ 158.975385] ffff88103e42df48 ffffffff81c064f3 0000000000000000 6f39f013d9ac3289
[ 158.996217] Call Trace:
[ 158.996338] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.014900] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 159.015180] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 159.055093] Call Trace:
[ 159.055231] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.055486] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 159.075260] NMI backtrace for cpu 69
[ 159.075456] CPU 69
[ 159.075552] Modules linked in:
[ 159.094880]
[ 159.094964] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 159.115131] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 159.134875] RSP: 0000:ffff88103e4a3ee8 EFLAGS: 00000246
[ 159.135116] RAX: 0000000000000000 RBX: ffff88103e4a3fd8 RCX: 0000000000000000
[ 159.154817] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e4a8000
[ 159.155159] RBP: ffff88103e4a3ef8 R08: 0000000000000000 R09: 0000000000000000
[ 159.174816] R10: 0000000000000000 R11: ffff88103e4a3ed8 R12: 0000000000000045
[ 159.194596] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 159.194939] FS: 0000000000000000(0000) GS:ffff88107f800000(0000) knlGS:0000000000000000
[ 159.214755] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 159.234471] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 159.234789] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 159.254603] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 159.274337] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4a2000, task ffff88103e4a8000)
[ 159.274713] Stack:
[ 159.294270] ffff88103e4a2000 ffffffff82525380 ffff88103e4a3f28 ffffffff810391f0
[ 159.294680] ffff88103e4a3f18 12ef20a5c0174c7c 0000000000000000 0000000000000000
[ 159.314468] ffff88103e4a3f48 ffffffff81c064f3 0000000000000000 2874467214a4e39e
[ 159.334199] Call Trace:
[ 159.334330] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.334598] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 159.354245] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 159.375041] Call Trace:
[ 159.394132] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.394358] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 159.414070] NMI backtrace for cpu 71
[ 159.414251] CPU 71
[ 159.414338] Modules linked in:
[ 159.414504]
[ 159.414591] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 159.453983] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 159.454366] RSP: 0000:ffff88103e4edee8 EFLAGS: 00000246
[ 159.474019] RAX: 0000000000000000 RBX: ffff88103e4edfd8 RCX: 0000000000000000
[ 159.474338] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e4f0000
[ 159.494112] RBP: ffff88103e4edef8 R08: 0000000000000000 R09: 0000000000000000
[ 159.513924] R10: 0000000000000000 R11: ffff88103e4eded8 R12: 0000000000000047
[ 159.514269] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 159.534030] FS: 0000000000000000(0000) GS:ffff88107fc00000(0000) knlGS:0000000000000000
[ 159.553842] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 159.554154] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 159.573926] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 159.593668] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 159.594006] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4ec000, task ffff88103e4f0000)
[ 159.613854] Stack:
[ 159.613968] ffff88103e4ec000 ffffffff82525380 ffff88103e4edf28 ffffffff810391f0
[ 159.633749] ffff88103e4edf18 fe58eb5636563f39 0000000000000000 0000000000000000
[ 159.634152] ffff88103e4edf48 ffffffff81c064f3 0000000000000000 dfdfb102a569daf8
[ 159.654863] Call Trace:
[ 159.655000] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.673642] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 159.673921] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 159.713904] Call Trace:
[ 159.714028] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.733478] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 159.733723] NMI backtrace for cpu 68
[ 159.733904] CPU 68
[ 159.753420] Modules linked in:
[ 159.753610]
[ 159.753689] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 159.773674] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 159.793424] RSP: 0000:ffff88103e479ee8 EFLAGS: 00000246
[ 159.793666] RAX: 0000000000000000 RBX: ffff88103e479fd8 RCX: 0000000000000000
[ 159.813458] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e470000
[ 159.833223] RBP: ffff88103e479ef8 R08: 0000000000000000 R09: 0000000000000000
[ 159.833564] R10: 0000000000000000 R11: ffff88103e479ed8 R12: 0000000000000044
[ 159.853338] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 159.873135] FS: 0000000000000000(0000) GS:ffff88107f600000(0000) knlGS:0000000000000000
[ 159.873502] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 159.893207] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 159.893574] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 159.913329] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 159.933094] Process kworker/0:1 (pid: 0, threadinfo ffff88103e478000, task ffff88103e470000)
[ 159.952924] Stack:
[ 159.953033] ffff88103e478000 ffffffff82525380 ffff88103e479f28 ffffffff810391f0
[ 159.953469] ffff88103e479f18 e7234dc0ddbdb12e 0000000000000000 0000000000000000
[ 159.973235] ffff88103e479f48 ffffffff81c064f3 0000000000000000 253694480af14bae
[ 159.992964] Call Trace:
[ 159.993116] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 159.993380] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 160.012990] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 160.052829] Call Trace:
[ 160.052958] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 160.053236] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 160.072848] NMI backtrace for cpu 70
[ 160.073062] CPU 70
[ 160.073154] Modules linked in:
[ 160.092815]
[ 160.092896] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 160.112764] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 160.113213] RSP: 0000:ffff88103e4c5ee8 EFLAGS: 00000246
[ 160.133168] RAX: 0000000000000000 RBX: ffff88103e4c5fd8 RCX: 0000000000000000
[ 160.152740] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e4c8000
[ 160.153090] RBP: ffff88103e4c5ef8 R08: 0000000000000000 R09: 0000000000000000
[ 160.172802] R10: 0000000000000000 R11: ffff88103e4c5ed8 R12: 0000000000000046
[ 160.173147] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 160.192773] FS: 0000000000000000(0000) GS:ffff88107fa00000(0000) knlGS:0000000000000000
[ 160.212598] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 160.212864] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 160.232649] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 160.252423] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 160.252734] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4c4000, task ffff88103e4c8000)
[ 160.272600] Stack:
[ 160.272704] ffff88103e4c4000 ffffffff82525380 ffff88103e4c5f28 ffffffff810391f0
[ 160.293314] ffff88103e4c5f18 6e7cc5eba8cea412 0000000000000000 0000000000000000
[ 160.312278] ffff88103e4c5f48 ffffffff81c064f3 0000000000000000 23ba843c07b63fab
[ 160.312674] Call Trace:
[ 160.332162] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 160.332416] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 160.332714] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 160.372721] Call Trace:
[ 160.372865] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 160.392225] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 160.392540] NMI backtrace for cpu 79
[ 160.412090] CPU 79
[ 160.412182] Modules linked in:
[ 160.412350]
[ 160.412431] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 160.432387] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 160.452168] RSP: 0000:ffff88103e591ee8 EFLAGS: 00000246
[ 160.471869] RAX: 0000000000000000 RBX: ffff88103e591fd8 RCX: 0000000000000000
[ 160.472193] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e588000
[ 160.491987] RBP: ffff88103e591ef8 R08: 0000000000000000 R09: 0000000000000000
[ 160.492302] R10: 0000000000000000 R11: ffff88103e591ed8 R12: 000000000000004f
[ 160.512076] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 160.531890] FS: 0000000000000000(0000) GS:ffff88207fc00000(0000) knlGS:0000000000000000
[ 160.532287] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 160.551961] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 160.571758] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 160.572094] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 160.591836] Process kworker/0:1 (pid: 0, threadinfo ffff88103e590000, task ffff88103e588000)
[ 160.611673] Stack:
[ 160.611779] ffff88103e590000 ffffffff82525380 ffff88103e591f28 ffffffff810391f0
[ 160.631590] ffff88103e591f18 9b70b21e28590e01 0000000000000000 0000000000000000
[ 160.631973] ffff88103e591f48 ffffffff81c064f3 0000000000000000 db865ef5c8a39a71
[ 160.651715] Call Trace:
[ 160.651846] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 160.671492] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 160.671751] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 160.711666] Call Trace:
[ 160.711789] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 160.712020] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 160.731566] NMI backtrace for cpu 73
[ 160.731752] CPU 73
[ 160.731856] Modules linked in:
[ 160.751431]
[ 160.751510] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 160.771486] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 160.791281] RSP: 0018:ffff88103e521ee8 EFLAGS: 00000246
[ 160.791528] RAX: 0000000000000000 RBX: ffff88103e521fd8 RCX: 0000000000000000
[ 160.811393] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e518000
[ 160.811732] RBP: ffff88103e521ef8 R08: 0000000000000000 R09: 0000000000000000
[ 160.831426] R10: 0000000000000000 R11: ffff88103e521ed8 R12: 0000000000000049
[ 160.851216] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 160.851587] FS: 0000000000000000(0000) GS:ffff88207f000000(0000) knlGS:0000000000000000
[ 160.871381] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 160.891082] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 160.891408] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 160.911154] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 160.936751] Process kworker/0:1 (pid: 0, threadinfo ffff88103e520000, task ffff88103e518000)
[ 160.937268] Stack:
[ 160.937369] ffff88103e520000 ffffffff82525380 ffff88103e521f28 ffffffff810391f0
[ 160.951285] ffff88103e521f18 d8b1686f1d48fddc 0000000000000000 0000000000000000
[ 160.971049] ffff88103e521f48 ffffffff81c064f3 0000000000000000 8790dbe0cba0e12f
[ 160.971480] Call Trace:
[ 160.990933] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 160.991198] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.010845] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 161.031582] Call Trace:
[ 161.050750] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 161.050978] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.070683] NMI backtrace for cpu 75
[ 161.070855] CPU 75
[ 161.070954] Modules linked in:
[ 161.071122]
[ 161.071222] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 161.091281] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 161.111280] RSP: 0000:ffff88103e535ee8 EFLAGS: 00000246
[ 161.130841] RAX: 0000000000000000 RBX: ffff88103e535fd8 RCX: 0000000000000000
[ 161.131158] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e538000
[ 161.150892] RBP: ffff88103e535ef8 R08: 0000000000000000 R09: 0000000000000000
[ 161.170698] R10: 0000000000000000 R11: ffff88103e535ed8 R12: 000000000000004b
[ 161.171032] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 161.190633] FS: 0000000000000000(0000) GS:ffff88207f400000(0000) knlGS:0000000000000000
[ 161.210473] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 161.210738] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 161.230638] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 161.230942] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 161.250591] Process kworker/0:1 (pid: 0, threadinfo ffff88103e534000, task ffff88103e538000)
[ 161.270454] Stack:
[ 161.270558] ffff88103e534000 ffffffff82525380 ffff88103e535f28 ffffffff810391f0
[ 161.290363] ffff88103e535f18 cd61f3b922c9e5b4 0000000000000000 0000000000000000
[ 161.290748] ffff88103e535f48 ffffffff81c064f3 0000000000000000 029d491e869419ff
[ 161.310478] Call Trace:
[ 161.310611] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 161.330370] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.330658] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 161.370461] Call Trace:
[ 161.370589] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 161.390052] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.390316] NMI backtrace for cpu 77
[ 161.390486] CPU 77
[ 161.410029] Modules linked in:
[ 161.410209]
[ 161.410297] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 161.430242] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 161.450041] RSP: 0000:ffff88103e55dee8 EFLAGS: 00000246
[ 161.450268] RAX: 0000000000000000 RBX: ffff88103e55dfd8 RCX: 0000000000000000
[ 161.470065] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e560000
[ 161.489845] RBP: ffff88103e55def8 R08: 0000000000000000 R09: 0000000000000000
[ 161.490169] R10: 0000000000000000 R11: ffff88103e55ded8 R12: 000000000000004d
[ 161.509948] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 161.529741] FS: 0000000000000000(0000) GS:ffff88207f800000(0000) knlGS:0000000000000000
[ 161.530113] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 161.549816] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 161.550135] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 161.570786] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 161.589747] Process kworker/0:1 (pid: 0, threadinfo ffff88103e55c000, task ffff88103e560000)
[ 161.590145] Stack:
[ 161.609633] ffff88103e55c000 ffffffff82525380 ffff88103e55df28 ffffffff810391f0
[ 161.610023] ffff88103e55df18 ad33531f6f46c54f 0000000000000000 0000000000000000
[ 161.629816] ffff88103e55df48 ffffffff81c064f3 0000000000000000 d9f51b4f236197d5
[ 161.649593] Call Trace:
[ 161.649705] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 161.649938] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.669586] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 161.709392] Call Trace:
[ 161.709526] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 161.709768] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.729432] NMI backtrace for cpu 72
[ 161.729640] CPU 72
[ 161.729727] Modules linked in:
[ 161.729936]
[ 161.749342] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 161.769384] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 161.769830] RSP: 0000:ffff88103e4fbee8 EFLAGS: 00000246
[ 161.789375] RAX: 0000000000000000 RBX: ffff88103e4fbfd8 RCX: 0000000000000000
[ 161.809156] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e500000
[ 161.809469] RBP: ffff88103e4fbef8 R08: 0000000000000000 R09: 0000000000000000
[ 161.829271] R10: 0000000000000000 R11: ffff88103e4fbed8 R12: 0000000000000048
[ 161.829589] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 161.849394] FS: 0000000000000000(0000) GS:ffff88207ee00000(0000) knlGS:0000000000000000
[ 161.869228] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 161.869494] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 161.889262] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 161.909042] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 161.909363] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4fa000, task ffff88103e500000)
[ 161.929206] Stack:
[ 161.929300] ffff88103e4fa000 ffffffff82525380 ffff88103e4fbf28 ffffffff810391f0
[ 161.949137] ffff88103e4fbf18 ce446216a9327bfa 0000000000000000 0000000000000000
[ 161.968877] ffff88103e4fbf48 ffffffff81c064f3 0000000000000000 35549ebda9dd1599
[ 161.969277] Call Trace:
[ 161.969393] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 161.989001] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 161.989249] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 162.029298] Call Trace:
[ 162.029416] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 162.048827] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 162.049091] NMI backtrace for cpu 78
[ 162.068668] CPU 78
[ 162.068785] Modules linked in:
[ 162.068986]
[ 162.069060] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 162.089028] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 162.108891] RSP: 0000:ffff88103e57bee8 EFLAGS: 00000246
[ 162.109126] RAX: 0000000000000000 RBX: ffff88103e57bfd8 RCX: 0000000000000000
[ 162.129627] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e580000
[ 162.148950] RBP: ffff88103e57bef8 R08: 0000000000000000 R09: 0000000000000000
[ 162.149284] R10: 0000000000000000 R11: ffff88103e57bed8 R12: 000000000000004e
[ 162.168852] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 162.188626] FS: 0000000000000000(0000) GS:ffff88207fa00000(0000) knlGS:0000000000000000
[ 162.188981] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 162.208541] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 162.229972] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 162.230346] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 162.248455] Process kworker/0:1 (pid: 0, threadinfo ffff88103e57a000, task ffff88103e580000)
[ 162.268290] Stack:
[ 162.268385] ffff88103e57a000 ffffffff82525380 ffff88103e57bf28 ffffffff810391f0
[ 162.268801] ffff88103e57bf18 f032e5fdb3adf613 0000000000000000 0000000000000000
[ 162.288547] ffff88103e57bf48 ffffffff81c064f3 0000000000000000 57f6bb23ccbb4736
[ 162.308355] Call Trace:
[ 162.308489] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 162.308746] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 162.328341] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 162.368494] Call Trace:
[ 162.368649] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 162.368860] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 162.388166] NMI backtrace for cpu 76
[ 162.388337] CPU 76
[ 162.388429] Modules linked in:
[ 162.408036]
[ 162.408115] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 162.428130] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 162.447877] RSP: 0000:ffff88103e54fee8 EFLAGS: 00000246
[ 162.448131] RAX: 0000000000000000 RBX: ffff88103e54ffd8 RCX: 0000000000000000
[ 162.467907] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e550000
[ 162.468219] RBP: ffff88103e54fef8 R08: 0000000000000000 R09: 0000000000000000
[ 162.488048] R10: 0000000000000000 R11: ffff88103e54fed8 R12: 000000000000004c
[ 162.507800] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 162.508124] FS: 0000000000000000(0000) GS:ffff88207f600000(0000) knlGS:0000000000000000
[ 162.527928] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 162.547669] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 162.548032] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 162.567780] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 162.568095] Process kworker/0:1 (pid: 0, threadinfo ffff88103e54e000, task ffff88103e550000)
[ 162.587923] Stack:
[ 162.588018] ffff88103e54e000 ffffffff82525380 ffff88103e54ff28 ffffffff810391f0
[ 162.607881] ffff88103e54ff18 cae5dd183ae409a6 0000000000000000 0000000000000000
[ 162.627664] ffff88103e54ff48 ffffffff81c064f3 0000000000000000 17a57ec2fee86728
[ 162.628032] Call Trace:
[ 162.647521] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 162.647750] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 162.667465] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 162.688147] Call Trace:
[ 162.707447] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 162.707667] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 162.707931] NMI backtrace for cpu 74
[ 162.747466] CPU 74
[ 162.747567] Modules linked in:
[ 162.747742]
[ 162.747826] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 162.768064] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 162.787730] RSP: 0018:ffff88103e527ee8 EFLAGS: 00000246
[ 162.807333] RAX: 0000000000000000 RBX: ffff88103e527fd8 RCX: 0000000000000000
[ 162.807667] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e528000
[ 162.827464] RBP: ffff88103e527ef8 R08: 0000000000000000 R09: 0000000000000000
[ 162.847210] R10: 0000000000000000 R11: ffff88103e527ed8 R12: 000000000000004a
[ 162.847530] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 162.867263] FS: 0000000000000000(0000) GS:ffff88207f200000(0000) knlGS:0000000000000000
[ 162.893181] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 162.893478] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 162.907170] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 162.907521] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 162.927151] Process kworker/0:1 (pid: 0, threadinfo ffff88103e526000, task ffff88103e528000)
[ 162.947018] Stack:
[ 162.947132] ffff88103e526000 ffffffff82525380 ffff88103e527f28 ffffffff810391f0
[ 162.966915] ffff88103e527f18 3ceecc1c6f3146ab 0000000000000000 0000000000000000
[ 162.967316] ffff88103e527f48 ffffffff81c064f3 0000000000000000 c6250755f2f9eca7
[ 162.987055] Call Trace:
[ 162.987176] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.006813] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 163.007072] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 163.047044] Call Trace:
[ 163.047157] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.066622] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 163.066876] NMI backtrace for cpu 81
[ 163.067069] CPU 81
[ 163.086579] Modules linked in:
[ 163.086762]
[ 163.086836] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 163.106837] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 163.126616] RSP: 0018:ffff88103e5afee8 EFLAGS: 00000246
[ 163.126895] RAX: 0000000000000000 RBX: ffff88103e5affd8 RCX: 0000000000000000
[ 163.147061] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5b4560
[ 163.166578] RBP: ffff88103e5afef8 R08: 0000000000000000 R09: 0000000000000000
[ 163.166956] R10: 0000000000000000 R11: ffff88103e5afed8 R12: 0000000000000051
[ 163.186810] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 163.187181] FS: 0000000000000000(0000) GS:ffff88307f000000(0000) knlGS:0000000000000000
[ 163.206849] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 163.226392] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 163.226762] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 163.246575] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 163.266265] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5ae000, task ffff88103e5b4560)
[ 163.266723] Stack:
[ 163.286189] ffff88103e5ae000 ffffffff82525380 ffff88103e5aff28 ffffffff810391f0
[ 163.286584] ffff88103e5aff18 7782e4a57542f064 0000000000000000 0000000000000000
[ 163.306399] ffff88103e5aff48 ffffffff81c064f3 0000000000000000 8ba93f0bab920685
[ 163.326120] Call Trace:
[ 163.326252] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.326540] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 163.346197] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 163.366956] Call Trace:
[ 163.386044] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.386320] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 163.405982] NMI backtrace for cpu 85
[ 163.406136] CPU 85
[ 163.406236] Modules linked in:
[ 163.406409]
[ 163.425904] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 163.445897] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 163.446273] RSP: 0000:ffff88103e613ee8 EFLAGS: 00000246
[ 163.466138] RAX: 0000000000000000 RBX: ffff88103e613fd8 RCX: 0000000000000000
[ 163.466548] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e60c560
[ 163.486079] RBP: ffff88103e613ef8 R08: 0000000000000000 R09: 0000000000000000
[ 163.505826] R10: 0000000000000000 R11: ffff88103e613ed8 R12: 0000000000000055
[ 163.506243] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 163.527647] FS: 0000000000000000(0000) GS:ffff88307f800000(0000) knlGS:0000000000000000
[ 163.545778] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 163.546092] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 163.565832] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 163.585568] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 163.585987] Process kworker/0:1 (pid: 0, threadinfo ffff88103e612000, task ffff88103e60c560)
[ 163.605780] Stack:
[ 163.605873] ffff88103e612000 ffffffff82525380 ffff88103e613f28 ffffffff810391f0
[ 163.625647] ffff88103e613f18 1338bcfcabb32e6d 0000000000000000 0000000000000000
[ 163.645439] ffff88103e613f48 ffffffff81c064f3 0000000000000000 71bb112c4d5bc551
[ 163.645899] Call Trace:
[ 163.646032] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.665559] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 163.665813] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 163.705851] Call Trace:
[ 163.705981] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.725406] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 163.725699] NMI backtrace for cpu 87
[ 163.745234] CPU 87
[ 163.745337] Modules linked in:
[ 163.745498]
[ 163.745574] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 163.765557] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 163.785347] RSP: 0000:ffff88103e641ee8 EFLAGS: 00000246
[ 163.785634] RAX: 0000000000000000 RBX: ffff88103e641fd8 RCX: 0000000000000000
[ 163.805440] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e63c560
[ 163.825147] RBP: ffff88103e641ef8 R08: 0000000000000000 R09: 0000000000000000
[ 163.825623] R10: 0000000000000000 R11: ffff88103e641ed8 R12: 0000000000000057
[ 163.845315] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 163.865029] FS: 0000000000000000(0000) GS:ffff88307fc00000(0000) knlGS:0000000000000000
[ 163.865477] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 163.885147] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 163.904917] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 163.905367] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 163.925030] Process kworker/0:1 (pid: 0, threadinfo ffff88103e640000, task ffff88103e63c560)
[ 163.944862] Stack:
[ 163.944954] ffff88103e640000 ffffffff82525380 ffff88103e641f28 ffffffff810391f0
[ 163.945367] ffff88103e641f18 41f11d18f408308f 0000000000000000 0000000000000000
[ 163.965173] ffff88103e641f48 ffffffff81c064f3 0000000000000000 388370f59e8df041
[ 163.984907] Call Trace:
[ 163.985023] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 163.985321] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 164.004921] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 164.044771] Call Trace:
[ 164.044911] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 164.045194] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 164.064719] NMI backtrace for cpu 84
[ 164.064933] CPU 84
[ 164.065020] Modules linked in:
[ 164.084637]
[ 164.084708] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 164.104674] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 164.124627] RSP: 0000:ffff88103e5edee8 EFLAGS: 00000246
[ 164.124880] RAX: 0000000000000000 RBX: ffff88103e5edfd8 RCX: 0000000000000000
[ 164.144688] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5f4560
[ 164.145133] RBP: ffff88103e5edef8 R08: 0000000000000000 R09: 0000000000000000
[ 164.166316] R10: 0000000000000000 R11: ffff88103e5eded8 R12: 0000000000000054
[ 164.184492] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 164.184895] FS: 0000000000000000(0000) GS:ffff88307f600000(0000) knlGS:0000000000000000
[ 164.204826] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 164.205180] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 164.224723] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 164.244472] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 164.244862] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5ec000, task ffff88103e5f4560)
[ 164.264574] Stack:
[ 164.264692] ffff88103e5ec000 ffffffff82525380 ffff88103e5edf28 ffffffff810391f0
[ 164.284428] ffff88103e5edf18 ac950cdde64809da 0000000000000000 0000000000000000
[ 164.304208] ffff88103e5edf48 ffffffff81c064f3 0000000000000000 79cf6d869ee9f9b5
[ 164.304641] Call Trace:
[ 164.324105] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 164.324330] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 164.344004] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 164.364677] Call Trace:
[ 164.364797] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 164.384674] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 164.385034] NMI backtrace for cpu 80
[ 164.403994] CPU 80
[ 164.404084] Modules linked in:
[ 164.404288]
[ 164.404374] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 164.424323] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 164.444136] RSP: 0000:ffff88103e5a9ee8 EFLAGS: 00000246
[ 164.463804] RAX: 0000000000000000 RBX: ffff88103e5a9fd8 RCX: 0000000000000000
[ 164.464126] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5a4560
[ 164.483918] RBP: ffff88103e5a9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 164.503706] R10: 0000000000000000 R11: ffff88103e5a9ed8 R12: 0000000000000050
[ 164.504052] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 164.523786] FS: 0000000000000000(0000) GS:ffff88307ee00000(0000) knlGS:0000000000000000
[ 164.543621] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 164.543938] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 164.563679] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 164.564099] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 164.583788] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5a8000, task ffff88103e5a4560)
[ 164.603591] Stack:
[ 164.603694] ffff88103e5a8000 ffffffff82525380 ffff88103e5a9f28 ffffffff810391f0
[ 164.623514] ffff88103e5a9f18 e1ea2195949618ac 0000000000000000 0000000000000000
[ 164.623940] ffff88103e5a9f48 ffffffff81c064f3 0000000000000000 913d0aa0500edc01
[ 164.643666] Call Trace:
[ 164.643774] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 164.663404] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 164.663733] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 164.703723] Call Trace:
[ 164.703841] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 164.723230] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 164.723575] NMI backtrace for cpu 82
[ 164.723783] CPU 82
[ 164.743189] Modules linked in:
[ 164.743378]
[ 164.743459] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 164.763419] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 164.783196] RSP: 0000:ffff88103e5d9ee8 EFLAGS: 00000246
[ 164.783437] RAX: 0000000000000000 RBX: ffff88103e5d9fd8 RCX: 0000000000000000
[ 164.803280] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5cc560
[ 164.803705] RBP: ffff88103e5d9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 164.825030] R10: 0000000000000000 R11: ffff88103e5d9ed8 R12: 0000000000000052
[ 164.843096] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 164.843567] FS: 0000000000000000(0000) GS:ffff88307f200000(0000) knlGS:0000000000000000
[ 164.863342] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 164.882987] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 164.883337] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 164.903103] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 164.922856] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5d8000, task ffff88103e5cc560)
[ 164.923279] Stack:
[ 164.942787] ffff88103e5d8000 ffffffff82525380 ffff88103e5d9f28 ffffffff810391f0
[ 164.943248] ffff88103e5d9f18 3ca5437b49de96ec 0000000000000000 0000000000000000
[ 164.962981] ffff88103e5d9f48 ffffffff81c064f3 0000000000000000 2b4b5c0dcf3a906f
[ 164.982749] Call Trace:
[ 164.982871] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 164.983154] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.002755] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 165.023581] Call Trace:
[ 165.042671] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 165.042935] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.062603] NMI backtrace for cpu 83
[ 165.062792] CPU 83
[ 165.062913] Modules linked in:
[ 165.063139]
[ 165.082496] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 165.102531] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 165.102959] RSP: 0000:ffff88103e5dfee8 EFLAGS: 00000246
[ 165.122610] RAX: 0000000000000000 RBX: ffff88103e5dffd8 RCX: 0000000000000000
[ 165.123043] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5e4560
[ 165.142773] RBP: ffff88103e5dfef8 R08: 0000000000000000 R09: 0000000000000000
[ 165.162713] R10: 0000000000000000 R11: ffff88103e5dfed8 R12: 0000000000000053
[ 165.163093] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 165.182668] FS: 0000000000000000(0000) GS:ffff88307f400000(0000) knlGS:0000000000000000
[ 165.202485] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 165.202770] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 165.222691] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 165.242184] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 165.242557] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5de000, task ffff88103e5e4560)
[ 165.262353] Stack:
[ 165.262446] ffff88103e5de000 ffffffff82525380 ffff88103e5dff28 ffffffff810391f0
[ 165.282294] ffff88103e5dff18 291962416f9caca0 0000000000000000 0000000000000000
[ 165.282731] ffff88103e5dff48 ffffffff81c064f3 0000000000000000 182eaafab45800c9
[ 165.302476] Call Trace:
[ 165.302606] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 165.322182] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.322495] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 165.362412] Call Trace:
[ 165.362529] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 165.393005] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.393270] NMI backtrace for cpu 86
[ 165.411822] CPU 86
[ 165.411937] Modules linked in:
[ 165.412101]
[ 165.412177] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 165.432223] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 165.451948] RSP: 0000:ffff88103e629ee8 EFLAGS: 00000246
[ 165.452228] RAX: 0000000000000000 RBX: ffff88103e629fd8 RCX: 0000000000000000
[ 165.473706] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e624560
[ 165.491738] RBP: ffff88103e629ef8 R08: 0000000000000000 R09: 0000000000000000
[ 165.492121] R10: 0000000000000000 R11: ffff88103e629ed8 R12: 0000000000000056
[ 165.511828] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 165.531631] FS: 0000000000000000(0000) GS:ffff88307fa00000(0000) knlGS:0000000000000000
[ 165.532019] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 165.551714] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 165.552087] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 165.571840] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 165.591613] Process kworker/0:1 (pid: 0, threadinfo ffff88103e628000, task ffff88103e624560)
[ 165.611425] Stack:
[ 165.611552] ffff88103e628000 ffffffff82525380 ffff88103e629f28 ffffffff810391f0
[ 165.611966] ffff88103e629f18 18b050d3e519eccd 0000000000000000 0000000000000000
[ 165.631721] ffff88103e629f48 ffffffff81c064f3 0000000000000000 e4373b2097d82c28
[ 165.651482] Call Trace:
[ 165.651600] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 165.651845] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.671528] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 165.711543] Call Trace:
[ 165.711662] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 165.711928] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.731348] NMI backtrace for cpu 95
[ 165.731538] CPU 95
[ 165.731648] Modules linked in:
[ 165.751183]
[ 165.751280] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 165.771263] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 165.771659] RSP: 0000:ffff88103e6d5ee8 EFLAGS: 00000246
[ 165.791267] RAX: 0000000000000000 RBX: ffff88103e6d5fd8 RCX: 0000000000000000
[ 165.811054] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6dc560
[ 165.811363] RBP: ffff88103e6d5ef8 R08: 0000000000000000 R09: 0000000000000000
[ 165.831174] R10: 0000000000000000 R11: ffff88103e6d5ed8 R12: 000000000000005f
[ 165.850938] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 165.851262] FS: 0000000000000000(0000) GS:ffff88407fc00000(0000) knlGS:0000000000000000
[ 165.871114] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 165.871382] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 165.891216] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 165.910931] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 165.911268] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6d4000, task ffff88103e6dc560)
[ 165.931078] Stack:
[ 165.931169] ffff88103e6d4000 ffffffff82525380 ffff88103e6d5f28 ffffffff810391f0
[ 165.951035] ffff88103e6d5f18 e9750d6e03254eac 0000000000000000 0000000000000000
[ 165.970777] ffff88103e6d5f48 ffffffff81c064f3 0000000000000000 26997d219409388f
[ 165.971192] Call Trace:
[ 165.990653] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 165.990873] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 165.991119] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 166.031254] Call Trace:
[ 166.031394] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 166.050708] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 166.050969] NMI backtrace for cpu 89
[ 166.070579] CPU 89
[ 166.070699] Modules linked in:
[ 166.070907]
[ 166.070993] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 166.090981] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 166.111649] RSP: 0000:ffff88103e657ee8 EFLAGS: 00000246
[ 166.130385] RAX: 0000000000000000 RBX: ffff88103e657fd8 RCX: 0000000000000000
[ 166.130813] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e65c560
[ 166.151071] RBP: ffff88103e657ef8 R08: 0000000000000000 R09: 0000000000000000
[ 166.151435] R10: 0000000000000000 R11: ffff88103e657ed8 R12: 0000000000000059
[ 166.170873] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 166.190468] FS: 0000000000000000(0000) GS:ffff88407f000000(0000) knlGS:0000000000000000
[ 166.210462] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 166.210829] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 166.230237] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 166.230596] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 166.250492] Process kworker/0:1 (pid: 0, threadinfo ffff88103e656000, task ffff88103e65c560)
[ 166.270185] Stack:
[ 166.270303] ffff88103e656000 ffffffff82525380 ffff88103e657f28 ffffffff810391f0
[ 166.290094] ffff88103e657f18 99a1b0cf9e9d3ebb 0000000000000000 0000000000000000
[ 166.290483] ffff88103e657f48 ffffffff81c064f3 0000000000000000 25270db18ea19940
[ 166.310257] Call Trace:
[ 166.310402] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 166.329985] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 166.330351] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 166.370192] Call Trace:
[ 166.370349] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 166.370593] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 166.390088] NMI backtrace for cpu 92
[ 166.390299] CPU 92
[ 166.390439] Modules linked in:
[ 166.410000]
[ 166.410077] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 166.430006] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 166.449777] RSP: 0000:ffff88103e6a1ee8 EFLAGS: 00000246
[ 166.450048] RAX: 0000000000000000 RBX: ffff88103e6a1fd8 RCX: 0000000000000000
[ 166.469820] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e69c560
[ 166.470176] RBP: ffff88103e6a1ef8 R08: 0000000000000000 R09: 0000000000000000
[ 166.489932] R10: 0000000000000000 R11: ffff88103e6a1ed8 R12: 000000000000005c
[ 166.509692] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 166.510114] FS: 0000000000000000(0000) GS:ffff88407f600000(0000) knlGS:0000000000000000
[ 166.529838] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 166.549585] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 166.549971] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 166.570648] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 166.589433] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6a0000, task ffff88103e69c560)
[ 166.589840] Stack:
[ 166.589953] ffff88103e6a0000 ffffffff82525380 ffff88103e6a1f28 ffffffff810391f0
[ 166.609811] ffff88103e6a1f18 18a3b18c2c15ed65 0000000000000000 0000000000000000
[ 166.629556] ffff88103e6a1f48 ffffffff81c064f3 0000000000000000 6118231e21489cba
[ 166.649294] Call Trace:
[ 166.649445] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 166.649739] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 166.669370] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 166.690108] Call Trace:
[ 166.709251] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 166.709555] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 166.729162] NMI backtrace for cpu 93
[ 166.729336] CPU 93
[ 166.729424] Modules linked in:
[ 166.729601]
[ 166.729704] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 166.769090] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 166.769509] RSP: 0000:ffff88103e6a7ee8 EFLAGS: 00000246
[ 166.789274] RAX: 0000000000000000 RBX: ffff88103e6a7fd8 RCX: 0000000000000000
[ 166.789705] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6ac560
[ 166.809219] RBP: ffff88103e6a7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 166.829023] R10: 0000000000000000 R11: ffff88103e6a7ed8 R12: 000000000000005d
[ 166.829467] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 166.859237] FS: 0000000000000000(0000) GS:ffff88407f800000(0000) knlGS:0000000000000000
[ 166.879116] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 166.879403] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 166.899175] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 166.899576] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 166.919321] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6a6000, task ffff88103e6ac560)
[ 166.949119] Stack:
[ 166.949207] ffff88103e6a6000 ffffffff82525380 ffff88103e6a7f28 ffffffff810391f0
[ 166.968960] ffff88103e6a7f18 12bbac2f2ac6563c 0000000000000000 0000000000000000
[ 166.969388] ffff88103e6a7f48 ffffffff81c064f3 0000000000000000 2c0129df7ca7b3c8
[ 166.989083] Call Trace:
[ 166.989191] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 167.008734] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 167.009075] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 167.049004] Call Trace:
[ 167.049162] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 167.068530] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 167.068821] NMI backtrace for cpu 98
[ 167.069031] CPU 98
[ 167.088467] Modules linked in:
[ 167.088698]
[ 167.088799] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 167.108739] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 167.129860] RSP: 0000:ffff88103e719ee8 EFLAGS: 00000246
[ 167.130128] RAX: 0000000000000000 RBX: ffff88103e719fd8 RCX: 0000000000000000
[ 167.148569] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e70a2b0
[ 167.168308] RBP: ffff88103e719ef8 R08: 0000000000000000 R09: 0000000000000000
[ 167.168679] R10: 0000000000000000 R11: ffff88103e719ed8 R12: 0000000000000062
[ 167.189402] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 167.208443] FS: 0000000000000000(0000) GS:ffff88507f200000(0000) knlGS:0000000000000000
[ 167.208868] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 167.228381] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 167.228772] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 167.252818] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 167.268203] Process kworker/0:1 (pid: 0, threadinfo ffff88103e718000, task ffff88103e70a2b0)
[ 167.268701] Stack:
[ 167.288075] ffff88103e718000 ffffffff82525380 ffff88103e719f28 ffffffff810391f0
[ 167.288543] ffff88103e719f18 c68544f99cc20e07 0000000000000000 0000000000000000
[ 167.308301] ffff88103e719f48 ffffffff81c064f3 0000000000000000 886503d07d16a597
[ 167.328040] Call Trace:
[ 167.328170] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 167.328468] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 167.348094] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 167.387896] Call Trace:
[ 167.388015] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 167.388281] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 167.407899] NMI backtrace for cpu 99
[ 167.408051] CPU 99
[ 167.408148] Modules linked in:
[ 167.428828]
[ 167.428905] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 167.447787] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 167.448164] RSP: 0000:ffff88103e71fee8 EFLAGS: 00000246
[ 167.467839] RAX: 0000000000000000 RBX: ffff88103e71ffd8 RCX: 0000000000000000
[ 167.487619] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e72a2b0
[ 167.488003] RBP: ffff88103e71fef8 R08: 0000000000000000 R09: 0000000000000000
[ 167.507722] R10: 0000000000000000 R11: ffff88103e71fed8 R12: 0000000000000063
[ 167.508045] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 167.527801] FS: 0000000000000000(0000) GS:ffff88507f400000(0000) knlGS:0000000000000000
[ 167.547668] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 167.547980] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 167.567717] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 167.587475] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 167.587866] Process kworker/0:1 (pid: 0, threadinfo ffff88103e71e000, task ffff88103e72a2b0)
[ 167.607715] Stack:
[ 167.607847] ffff88103e71e000 ffffffff82525380 ffff88103e71ff28 ffffffff810391f0
[ 167.627601] ffff88103e71ff18 d1b7ebbe272fa90f 0000000000000000 0000000000000000
[ 167.647336] ffff88103e71ff48 ffffffff81c064f3 0000000000000000 4e968abbf0e34374
[ 167.647776] Call Trace:
[ 167.647891] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 167.667525] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 167.667907] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 167.707784] Call Trace:
[ 167.707902] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 167.727340] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 167.727612] NMI backtrace for cpu 88
[ 167.747140] CPU 88
[ 167.747258] Modules linked in:
[ 167.747441]
[ 167.747520] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 167.767452] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 167.787360] RSP: 0000:ffff88103e651ee8 EFLAGS: 00000246
[ 167.787650] RAX: 0000000000000000 RBX: ffff88103e651fd8 RCX: 0000000000000000
[ 167.807319] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e64c560
[ 167.827053] RBP: ffff88103e651ef8 R08: 0000000000000000 R09: 0000000000000000
[ 167.827415] R10: 0000000000000000 R11: ffff88103e651ed8 R12: 0000000000000058
[ 167.848237] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 167.866944] FS: 0000000000000000(0000) GS:ffff88407ee00000(0000) knlGS:0000000000000000
[ 167.867406] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 167.887050] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 167.906823] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 167.907276] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 167.926921] Process kworker/0:1 (pid: 0, threadinfo ffff88103e650000, task ffff88103e64c560)
[ 167.946733] Stack:
[ 167.946836] ffff88103e650000 ffffffff82525380 ffff88103e651f28 ffffffff810391f0
[ 167.966640] ffff88103e651f18 864a7adc0ed319da 0000000000000000 0000000000000000
[ 167.967117] ffff88103e651f48 ffffffff81c064f3 0000000000000000 43de25d7d634e3af
[ 167.986821] Call Trace:
[ 167.986961] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.006546] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 168.006909] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 168.046737] Call Trace:
[ 168.046907] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.047123] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 168.072366] NMI backtrace for cpu 91
[ 168.072590] CPU 91
[ 168.072697] Modules linked in:
[ 168.086500]
[ 168.086574] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 168.106603] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 168.126361] RSP: 0000:ffff88103e68bee8 EFLAGS: 00000246
[ 168.126617] RAX: 0000000000000000 RBX: ffff88103e68bfd8 RCX: 0000000000000000
[ 168.146509] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e684560
[ 168.146838] RBP: ffff88103e68bef8 R08: 0000000000000000 R09: 0000000000000000
[ 168.166733] R10: 0000000000000000 R11: ffff88103e68bed8 R12: 000000000000005b
[ 168.186481] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 168.186845] FS: 0000000000000000(0000) GS:ffff88407f400000(0000) knlGS:0000000000000000
[ 168.206569] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 168.226269] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 168.226658] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 168.246454] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 168.266014] Process kworker/0:1 (pid: 0, threadinfo ffff88103e68a000, task ffff88103e684560)
[ 168.266386] Stack:
[ 168.266491] ffff88103e68a000 ffffffff82525380 ffff88103e68bf28 ffffffff810391f0
[ 168.286319] ffff88103e68bf18 dacf332056ee6f33 0000000000000000 0000000000000000
[ 168.306112] ffff88103e68bf48 ffffffff81c064f3 0000000000000000 1e902c36a88edc60
[ 168.306508] Call Trace:
[ 168.325977] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.326224] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 168.345962] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 168.366617] Call Trace:
[ 168.385787] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.386052] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 168.405723] NMI backtrace for cpu 94
[ 168.405913] CPU 94
[ 168.406002] Modules linked in:
[ 168.406238]
[ 168.406339] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 168.426256] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 168.446018] RSP: 0000:ffff88103e6b7ee8 EFLAGS: 00000246
[ 168.465683] RAX: 0000000000000000 RBX: ffff88103e6b7fd8 RCX: 0000000000000000
[ 168.466109] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6cc560
[ 168.485868] RBP: ffff88103e6b7ef8 R08: 0000000000000000 R09: 0000000000000000
[ 168.505608] R10: 0000000000000000 R11: ffff88103e6b7ed8 R12: 000000000000005e
[ 168.505956] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 168.525703] FS: 0000000000000000(0000) GS:ffff88407fa00000(0000) knlGS:0000000000000000
[ 168.545537] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 168.545788] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 168.565561] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 168.565992] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 168.585669] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6b6000, task ffff88103e6cc560)
[ 168.605513] Stack:
[ 168.605644] ffff88103e6b6000 ffffffff82525380 ffff88103e6b7f28 ffffffff810391f0
[ 168.625416] ffff88103e6b7f18 a9fa8bc8dff9b80c 0000000000000000 0000000000000000
[ 168.625823] ffff88103e6b7f48 ffffffff81c064f3 0000000000000000 37bc997f211760a8
[ 168.645570] Call Trace:
[ 168.645708] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.665305] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 168.665604] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 168.706857] Call Trace:
[ 168.706977] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.725229] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 168.725492] NMI backtrace for cpu 90
[ 168.725664] CPU 90
[ 168.745087] Modules linked in:
[ 168.745301]
[ 168.745387] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 168.765351] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 168.785094] RSP: 0000:ffff88103e66dee8 EFLAGS: 00000246
[ 168.785415] RAX: 0000000000000000 RBX: ffff88103e66dfd8 RCX: 0000000000000000
[ 168.805151] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e67c560
[ 168.824885] RBP: ffff88103e66def8 R08: 0000000000000000 R09: 0000000000000000
[ 168.825303] R10: 0000000000000000 R11: ffff88103e66ded8 R12: 000000000000005a
[ 168.845106] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 168.845509] FS: 0000000000000000(0000) GS:ffff88407f200000(0000) knlGS:0000000000000000
[ 168.865194] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 168.884911] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 168.885293] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 168.904978] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 168.924769] Process kworker/0:1 (pid: 0, threadinfo ffff88103e66c000, task ffff88103e67c560)
[ 168.925292] Stack:
[ 168.944713] ffff88103e66c000 ffffffff82525380 ffff88103e66df28 ffffffff810391f0
[ 168.945128] ffff88103e66df18 6f19056750a6e749 0000000000000000 0000000000000000
[ 168.964888] ffff88103e66df48 ffffffff81c064f3 0000000000000000 66829d37004aca91
[ 168.984629] Call Trace:
[ 168.984750] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 168.985016] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 169.004665] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 169.044433] Call Trace:
[ 169.044580] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 169.044858] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 169.064498] NMI backtrace for cpu 100
[ 169.064677] CPU 100
[ 169.064810] Modules linked in:
[ 169.065076]
[ 169.084413] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 169.104420] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 169.104894] RSP: 0000:ffff88103e73dee8 EFLAGS: 00000246
[ 169.124467] RAX: 0000000000000000 RBX: ffff88103e73dfd8 RCX: 0000000000000000
[ 169.144339] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7322b0
[ 169.144767] RBP: ffff88103e73def8 R08: 0000000000000000 R09: 0000000000000000
[ 169.164487] R10: 0000000000000000 R11: ffff88103e73ded8 R12: 0000000000000064
[ 169.164878] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 169.184760] FS: 0000000000000000(0000) GS:ffff88507f600000(0000) knlGS:0000000000000000
[ 169.204573] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 169.204908] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 169.224623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 169.244215] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 169.244549] Process kworker/0:1 (pid: 0, threadinfo ffff88103e73c000, task ffff88103e7322b0)
[ 169.264442] Stack:
[ 169.264535] ffff88103e73c000 ffffffff82525380 ffff88103e73df28 ffffffff810391f0
[ 169.284155] ffff88103e73df18 ec3d40b388e1a820 0000000000000000 0000000000000000
[ 169.303948] ffff88103e73df48 ffffffff81c064f3 0000000000000000 7819af42c6415509
[ 169.304385] Call Trace:
[ 169.304503] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 169.324093] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 169.324354] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 169.364412] Call Trace:
[ 169.364559] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 169.384024] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 169.384305] NMI backtrace for cpu 103
[ 169.403770] CPU 103
[ 169.403898] Modules linked in:
[ 169.404109]
[ 169.404195] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 169.424087] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 169.443877] RSP: 0000:ffff88103e781ee8 EFLAGS: 00000246
[ 169.444177] RAX: 0000000000000000 RBX: ffff88103e781fd8 RCX: 0000000000000000
[ 169.463989] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7722b0
[ 169.483686] RBP: ffff88103e781ef8 R08: 0000000000000000 R09: 0000000000000000
[ 169.484057] R10: 0000000000000000 R11: ffff88103e781ed8 R12: 0000000000000067
[ 169.503796] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 169.523581] FS: 0000000000000000(0000) GS:ffff88507fc00000(0000) knlGS:0000000000000000
[ 169.524098] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 169.543656] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 169.563441] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 169.563857] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 169.583539] Process kworker/0:1 (pid: 0, threadinfo ffff88103e780000, task ffff88103e7722b0)
[ 169.603357] Stack:
[ 169.603495] ffff88103e780000 ffffffff82525380 ffff88103e781f28 ffffffff810391f0
[ 169.623253] ffff88103e781f18 1edcc44015d74379 0000000000000000 0000000000000000
[ 169.623711] ffff88103e781f48 ffffffff81c064f3 0000000000000000 308c6b70699dc251
[ 169.643418] Call Trace:
[ 169.643554] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 169.663182] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 169.663444] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 169.703340] Call Trace:
[ 169.703458] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 169.703758] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 169.723298] NMI backtrace for cpu 101
[ 169.723463] CPU 101
[ 169.723565] Modules linked in:
[ 169.743120]
[ 169.743197] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 169.763192] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 169.782980] RSP: 0000:ffff88103e753ee8 EFLAGS: 00000246
[ 169.783243] RAX: 0000000000000000 RBX: ffff88103e753fd8 RCX: 0000000000000000
[ 169.803029] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7422b0
[ 169.803351] RBP: ffff88103e753ef8 R08: 0000000000000000 R09: 0000000000000000
[ 169.823148] R10: 0000000000000000 R11: ffff88103e753ed8 R12: 0000000000000065
[ 169.842867] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 169.843256] FS: 0000000000000000(0000) GS:ffff88507f800000(0000) knlGS:0000000000000000
[ 169.863068] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 169.882739] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 169.883120] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 169.902849] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 169.922642] Process kworker/0:1 (pid: 0, threadinfo ffff88103e752000, task ffff88103e7422b0)
[ 169.923101] Stack:
[ 169.923202] ffff88103e752000 ffffffff82525380 ffff88103e753f28 ffffffff810391f0
[ 169.942961] ffff88103e753f18 e51c0349f6f87d31 0000000000000000 0000000000000000
[ 169.962742] ffff88103e753f48 ffffffff81c064f3 0000000000000000 cf452dd59fbee8f3
[ 169.963148] Call Trace:
[ 169.982623] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 169.982891] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.004075] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 170.023238] Call Trace:
[ 170.042434] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 170.042747] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.062353] NMI backtrace for cpu 97
[ 170.062556] CPU 97
[ 170.062675] Modules linked in:
[ 170.062851]
[ 170.062927] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 170.082837] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 170.102659] RSP: 0000:ffff88103e703ee8 EFLAGS: 00000246
[ 170.122332] RAX: 0000000000000000 RBX: ffff88103e703fd8 RCX: 0000000000000000
[ 170.122652] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6fa2b0
[ 170.142414] RBP: ffff88103e703ef8 R08: 0000000000000000 R09: 0000000000000000
[ 170.162327] R10: 0000000000000000 R11: ffff88103e703ed8 R12: 0000000000000061
[ 170.162671] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 170.182464] FS: 0000000000000000(0000) GS:ffff88507f000000(0000) knlGS:0000000000000000
[ 170.202302] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 170.202604] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 170.222348] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 170.222675] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 170.242449] Process kworker/0:1 (pid: 0, threadinfo ffff88103e702000, task ffff88103e6fa2b0)
[ 170.262272] Stack:
[ 170.262399] ffff88103e702000 ffffffff82525380 ffff88103e703f28 ffffffff810391f0
[ 170.282044] ffff88103e703f18 5b6b76da7595c5ea 0000000000000000 0000000000000000
[ 170.282455] ffff88103e703f48 ffffffff81c064f3 0000000000000000 ef6e93bff8ea7807
[ 170.302166] Call Trace:
[ 170.302286] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 170.321949] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.322203] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 170.362204] Call Trace:
[ 170.362331] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 170.381831] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.382118] NMI backtrace for cpu 96
[ 170.382272] CPU 96
[ 170.401728] Modules linked in:
[ 170.401964]
[ 170.403103] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 170.421996] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 170.441753] RSP: 0000:ffff88103e6edee8 EFLAGS: 00000246
[ 170.442088] RAX: 0000000000000000 RBX: ffff88103e6edfd8 RCX: 0000000000000000
[ 170.461779] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6e4560
[ 170.481516] RBP: ffff88103e6edef8 R08: 0000000000000000 R09: 0000000000000000
[ 170.481904] R10: 0000000000000000 R11: ffff88103e6eded8 R12: 0000000000000060
[ 170.501646] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 170.502047] FS: 0000000000000000(0000) GS:ffff88507ee00000(0000) knlGS:0000000000000000
[ 170.521877] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 170.541472] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 170.541835] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 170.561716] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 170.581394] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6ec000, task ffff88103e6e4560)
[ 170.581929] Stack:
[ 170.601331] ffff88103e6ec000 ffffffff82525380 ffff88103e6edf28 ffffffff810391f0
[ 170.601707] ffff88103e6edf18 38f7cba7c3e92042 0000000000000000 0000000000000000
[ 170.621515] ffff88103e6edf48 ffffffff81c064f3 0000000000000000 4bed50ad4b39cf1b
[ 170.642398] Call Trace:
[ 170.642538] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 170.642787] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.661307] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 170.701074] Call Trace:
[ 170.701196] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 170.701468] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.721107] NMI backtrace for cpu 102
[ 170.721292] CPU 102
[ 170.721387] Modules linked in:
[ 170.721562]
[ 170.741029] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 170.761064] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 170.761494] RSP: 0000:ffff88103e763ee8 EFLAGS: 00000246
[ 170.781065] RAX: 0000000000000000 RBX: ffff88103e763fd8 RCX: 0000000000000000
[ 170.800849] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e75a2b0
[ 170.801247] RBP: ffff88103e763ef8 R08: 0000000000000000 R09: 0000000000000000
[ 170.820994] R10: 0000000000000000 R11: ffff88103e763ed8 R12: 0000000000000066
[ 170.821358] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 170.841164] FS: 0000000000000000(0000) GS:ffff88507fa00000(0000) knlGS:0000000000000000
[ 170.860923] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 170.861258] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 170.880954] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 170.900721] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 170.901102] Process kworker/0:1 (pid: 0, threadinfo ffff88103e762000, task ffff88103e75a2b0)
[ 170.920879] Stack:
[ 170.920990] ffff88103e762000 ffffffff82525380 ffff88103e763f28 ffffffff810391f0
[ 170.940808] ffff88103e763f18 73ced03514241cf0 0000000000000000 0000000000000000
[ 170.970733] ffff88103e763f48 ffffffff81c064f3 0000000000000000 e2284a4bb68b9280
[ 170.971094] Call Trace:
[ 170.971262] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 170.991230] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 170.991552] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 171.031179] Call Trace:
[ 171.031294] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 171.060699] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 171.060986] NMI backtrace for cpu 108
[ 171.080475] CPU 108
[ 171.080595] Modules linked in:
[ 171.080795]
[ 171.080875] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 171.100803] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 171.120445] RSP: 0000:ffff88103e001ee8 EFLAGS: 00000246
[ 171.120747] RAX: 0000000000000000 RBX: ffff88103e001fd8 RCX: 0000000000000000
[ 171.140516] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7d22b0
[ 171.160264] RBP: ffff88103e001ef8 R08: 0000000000000000 R09: 0000000000000000
[ 171.160674] R10: 0000000000000000 R11: ffff88103e001ed8 R12: 000000000000006c
[ 171.180617] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 171.200160] FS: 0000000000000000(0000) GS:ffff88607f600000(0000) knlGS:0000000000000000
[ 171.200691] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 171.220319] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 171.241231] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 171.241544] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 171.270577] Process kworker/0:1 (pid: 0, threadinfo ffff88103e000000, task ffff88103e7d22b0)
[ 171.290096] Stack:
[ 171.290222] ffff88103e000000 ffffffff82525380 ffff88103e001f28 ffffffff810391f0
[ 171.310082] ffff88103e001f18 cf3d086b299fbe80 0000000000000000 0000000000000000
[ 171.310487] ffff88103e001f48 ffffffff81c064f3 0000000000000000 b23c24525439199e
[ 171.330119] Call Trace:
[ 171.330227] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 171.349871] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 171.350165] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 171.389904] Call Trace:
[ 171.390044] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 171.390310] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 171.409838] NMI backtrace for cpu 104
[ 171.410012] CPU 104
[ 171.410110] Modules linked in:
[ 171.429702]
[ 171.429788] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 171.449766] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 171.469562] RSP: 0000:ffff88103e787ee8 EFLAGS: 00000246
[ 171.469810] RAX: 0000000000000000 RBX: ffff88103e787fd8 RCX: 0000000000000000
[ 171.489590] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e78a2b0
[ 171.490008] RBP: ffff88103e787ef8 R08: 0000000000000000 R09: 0000000000000000
[ 171.509654] R10: 0000000000000000 R11: ffff88103e787ed8 R12: 0000000000000068
[ 171.529460] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 171.529840] FS: 0000000000000000(0000) GS:ffff88607ee00000(0000) knlGS:0000000000000000
[ 171.549642] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 171.569323] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 171.569679] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 171.589412] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 171.609206] Process kworker/0:1 (pid: 0, threadinfo ffff88103e786000, task ffff88103e78a2b0)
[ 171.609667] Stack:
[ 171.609759] ffff88103e786000 ffffffff82525380 ffff88103e787f28 ffffffff810391f0
[ 171.629575] ffff88103e787f18 dbaa9279e483322f 0000000000000000 0000000000000000
[ 171.649302] ffff88103e787f48 ffffffff81c064f3 0000000000000000 83aabbf70c51afbb
[ 171.649704] Call Trace:
[ 171.669186] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 171.669449] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 171.689113] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 171.709807] Call Trace:
[ 171.728985] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 171.729222] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 171.748934] NMI backtrace for cpu 106
[ 171.749093] CPU 106
[ 171.749194] Modules linked in:
[ 171.749370]
[ 171.749456] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 171.769411] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 171.789292] RSP: 0000:ffff88103e7b3ee8 EFLAGS: 00000246
[ 171.808904] RAX: 0000000000000000 RBX: ffff88103e7b3fd8 RCX: 0000000000000000
[ 171.809327] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7aa2b0
[ 171.829106] RBP: ffff88103e7b3ef8 R08: 0000000000000000 R09: 0000000000000000
[ 171.848782] R10: 0000000000000000 R11: ffff88103e7b3ed8 R12: 000000000000006a
[ 171.849165] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 171.868898] FS: 0000000000000000(0000) GS:ffff88607f200000(0000) knlGS:0000000000000000
[ 171.888714] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 171.888998] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 171.908746] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 171.909098] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 171.928948] Process kworker/0:1 (pid: 0, threadinfo ffff88103e7b2000, task ffff88103e7aa2b0)
[ 171.948728] Stack:
[ 171.948868] ffff88103e7b2000 ffffffff82525380 ffff88103e7b3f28 ffffffff810391f0
[ 171.968618] ffff88103e7b3f18 f9eadedccb2cfa6c 0000000000000000 0000000000000000
[ 171.969035] ffff88103e7b3f48 ffffffff81c064f3 0000000000000000 8ad0f992b6df189f
[ 171.988770] Call Trace:
[ 171.988923] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 172.008510] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 172.008819] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 172.048800] Call Trace:
[ 172.048950] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 172.068323] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 172.068603] NMI backtrace for cpu 120
[ 172.068801] CPU 120
[ 172.088292] Modules linked in:
[ 172.088468]
[ 172.088556] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 172.108503] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 172.128297] RSP: 0000:ffff88103e0f9ee8 EFLAGS: 00000246
[ 172.128556] RAX: 0000000000000000 RBX: ffff88103e0f9fd8 RCX: 0000000000000000
[ 172.148296] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0f0000
[ 172.169509] RBP: ffff88103e0f9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 172.169837] R10: 0000000000000000 R11: ffff88103e0f9ed8 R12: 0000000000000078
[ 172.188344] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 172.208120] FS: 0000000000000000(0000) GS:ffff88807e800000(0000) knlGS:0000000000000000
[ 172.208485] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 172.228278] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 172.228602] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 172.248367] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 172.268087] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0f8000, task ffff88103e0f0000)
[ 172.268481] Stack:
[ 172.287908] ffff88103e0f8000 ffffffff82525380 ffff88103e0f9f28 ffffffff810391f0
[ 172.288303] ffff88103e0f9f18 9d1fa4a474be1569 0000000000000000 0000000000000000
[ 172.308077] ffff88103e0f9f48 ffffffff81c064f3 0000000000000000 33afc904d5c4ddb1
[ 172.327846] Call Trace:
[ 172.327960] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 172.328222] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 172.347875] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 172.387661] Call Trace:
[ 172.387809] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 172.388043] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 172.407680] NMI backtrace for cpu 122
[ 172.407854] CPU 122
[ 172.407943] Modules linked in:
[ 172.427557]
[ 172.427635] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 172.447626] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 172.447999] RSP: 0000:ffff88103e10dee8 EFLAGS: 00000246
[ 172.467688] RAX: 0000000000000000 RBX: ffff88103e10dfd8 RCX: 0000000000000000
[ 172.487433] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e118000
[ 172.487759] RBP: ffff88103e10def8 R08: 0000000000000000 R09: 0000000000000000
[ 172.507538] R10: 0000000000000000 R11: ffff88103e10ded8 R12: 000000000000007a
[ 172.507849] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 172.527659] FS: 0000000000000000(0000) GS:ffff88807ec00000(0000) knlGS:0000000000000000
[ 172.547478] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 172.547724] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 172.567497] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 172.587308] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 172.587631] Process kworker/0:1 (pid: 0, threadinfo ffff88103e10c000, task ffff88103e118000)
[ 172.607455] Stack:
[ 172.607560] ffff88103e10c000 ffffffff82525380 ffff88103e10df28 ffffffff810391f0
[ 172.627383] ffff88103e10df18 5932e94ddd4561db 0000000000000000 0000000000000000
[ 172.647168] ffff88103e10df48 ffffffff81c064f3 0000000000000000 44418799c393f5ed
[ 172.647585] Call Trace:
[ 172.647693] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 172.667275] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 172.667530] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 172.707611] Call Trace:
[ 172.707731] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 172.727104] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 172.727355] NMI backtrace for cpu 124
[ 172.746972] CPU 124
[ 172.747068] Modules linked in:
[ 172.747246]
[ 172.747339] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 172.767337] RIP: 0010:[<ffffffff810591f0>] [<ffffffff810591f0>] __x2apic_send_IPI_dest+0x1f/0x21
[ 172.787134] RSP: 0000:ffff88807f003d58 EFLAGS: 00000046
[ 172.806808] RAX: 0000000000000c00 RBX: 0000000000000092 RCX: 0000000000000830
[ 172.807113] RDX: 00000000000f0008 RSI: 0000000000000002 RDI: 00000000000f0008
[ 172.826918] RBP: ffff88807f003d58 R08: 0000000000000040 R09: ffffffff82525388
[ 172.846685] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000c7d0
[ 172.847006] R13: 0000000000000002 R14: ffffffff82525380 R15: 000000000000007d
[ 172.866832] FS: 0000000000000000(0000) GS:ffff88807f000000(0000) knlGS:0000000000000000
[ 172.886643] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 172.886914] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 172.906649] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 172.906973] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 172.926749] Process kworker/0:1 (pid: 0, threadinfo ffff88103e142000, task ffff88103e148000)
[ 172.946616] Stack:
[ 172.946709] ffff88807f003d98 ffffffff81059440 ffff88807f003d78 0000000000002710
[ 172.966529] ffffffff823cbf80 0000000000000010 ffffffff823cbf80 7fffffffffffffff
[ 172.966927] ffff88807f003da8 ffffffff8105949a ffff88807f003dc8 ffffffff810560de
[ 172.986675] Call Trace:
[ 172.986810] <IRQ>
[ 172.986920] [<ffffffff81059440>] x2apic_send_IPI_mask+0x52/0x95
[ 173.006568] [<ffffffff8105949a>] x2apic_send_IPI_all+0x17/0x19
[ 173.006841] [<ffffffff810560de>] arch_trigger_all_cpu_backtrace+0x5a/0x8c
[ 173.026610] [<ffffffff810d2940>] print_other_cpu_stall+0x11c/0x12f
[ 173.046349] [<ffffffff810d2a38>] check_cpu_stall+0xe5/0xe9
[ 173.046627] [<ffffffff810d2a64>] __rcu_pending+0x28/0x10f
[ 173.066334] [<ffffffff810d317d>] rcu_check_callbacks+0x8b/0xb8
[ 173.066598] [<ffffffff8108e1e7>] update_process_times+0x41/0x77
[ 173.086305] [<ffffffff810aaa21>] tick_sched_timer+0x77/0x97
[ 173.086584] [<ffffffff8109ff4f>] __run_hrtimer+0xde/0x167
[ 173.106246] [<ffffffff810aa9aa>] ? tick_nohz_handler+0xd6/0xd6
[ 173.106532] [<ffffffff810a08e7>] hrtimer_interrupt+0xd1/0x1bd
[ 173.126284] [<ffffffff81c17b4c>] smp_apic_timer_interrupt+0x88/0x9b
[ 173.126629] [<ffffffff81c16c93>] apic_timer_interrupt+0x13/0x20
[ 173.146261] <EOI>
[ 173.146373] [<ffffffff810409eb>] ? mwait_idle+0xef/0x169
[ 173.166082] [<ffffffff810409e2>] ? mwait_idle+0xe6/0x169
[ 173.166344] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 173.186053] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 173.186318] Code: 66 66 90 48 8b 05 09 5b bd 00 c9 c3 55 48 89 e5 66 66 66 66 90 b9 30 08 00 00 89 f0 09 d0 80 ce 04 83 fe 02 0f 44 c2 89 fa 0f 30 <c9> c3 55 48 89 e5 66 66 66 66 90 b8 01 00 00 00 c9 c3 55 48 89
[ 173.226163] Call Trace:
[ 173.226268] <IRQ> [<ffffffff81059440>] x2apic_send_IPI_mask+0x52/0x95
[ 173.247016] [<ffffffff8105949a>] x2apic_send_IPI_all+0x17/0x19
[ 173.247322] [<ffffffff810560de>] arch_trigger_all_cpu_backtrace+0x5a/0x8c
[ 173.266474] [<ffffffff810d2940>] print_other_cpu_stall+0x11c/0x12f
[ 173.266773] [<ffffffff810d2a38>] check_cpu_stall+0xe5/0xe9
[ 173.286065] [<ffffffff810d2a64>] __rcu_pending+0x28/0x10f
[ 173.286330] [<ffffffff810d317d>] rcu_check_callbacks+0x8b/0xb8
[ 173.305908] [<ffffffff8108e1e7>] update_process_times+0x41/0x77
[ 173.306176] [<ffffffff810aaa21>] tick_sched_timer+0x77/0x97
[ 173.325886] [<ffffffff8109ff4f>] __run_hrtimer+0xde/0x167
[ 173.326138] [<ffffffff810aa9aa>] ? tick_nohz_handler+0xd6/0xd6
[ 173.345862] [<ffffffff810a08e7>] hrtimer_interrupt+0xd1/0x1bd
[ 173.346116] [<ffffffff81c17b4c>] smp_apic_timer_interrupt+0x88/0x9b
[ 173.366028] [<ffffffff81c16c93>] apic_timer_interrupt+0x13/0x20
[ 173.385593] <EOI> [<ffffffff810409eb>] ? mwait_idle+0xef/0x169
[ 173.385898] [<ffffffff810409e2>] ? mwait_idle+0xe6/0x169
[ 173.405586] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 173.405848] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 173.425526] NMI backtrace for cpu 107
[ 173.425699] CPU 107
[ 173.425811] Modules linked in:
[ 173.425975]
[ 173.426053] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 173.465445] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 173.465906] RSP: 0000:ffff88103e7c9ee8 EFLAGS: 00000246
[ 173.485511] RAX: 0000000000000000 RBX: ffff88103e7c9fd8 RCX: 0000000000000000
[ 173.485865] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7c22b0
[ 173.505646] RBP: ffff88103e7c9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 173.525384] R10: 0000000000000000 R11: ffff88103e7c9ed8 R12: 000000000000006b
[ 173.525737] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 173.545617] FS: 0000000000000000(0000) GS:ffff88607f400000(0000) knlGS:0000000000000000
[ 173.565311] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 173.565598] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 173.585404] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 173.585754] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 173.605629] Process kworker/0:1 (pid: 0, threadinfo ffff88103e7c8000, task ffff88103e7c22b0)
[ 173.625290] Stack:
[ 173.625392] ffff88103e7c8000 ffffffff82525380 ffff88103e7c9f28 ffffffff810391f0
[ 173.645191] ffff88103e7c9f18 1eef3661459e59f7 0000000000000000 0000000000000000
[ 173.645606] ffff88103e7c9f48 ffffffff81c064f3 0000000000000000 d110419bdf32a799
[ 173.665391] Call Trace:
[ 173.665508] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 173.685087] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 173.685361] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 173.725355] Call Trace:
[ 173.725467] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 173.744945] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 173.745235] NMI backtrace for cpu 111
[ 173.745414] CPU 111
[ 173.764882] Modules linked in:
[ 173.765068]
[ 173.765154] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 173.785160] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 173.804932] RSP: 0000:ffff88103e03bee8 EFLAGS: 00000246
[ 173.805225] RAX: 0000000000000000 RBX: ffff88103e03bfd8 RCX: 0000000000000000
[ 173.824951] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0322b0
[ 173.844721] RBP: ffff88103e03bef8 R08: 0000000000000000 R09: 0000000000000000
[ 173.845119] R10: 0000000000000000 R11: ffff88103e03bed8 R12: 000000000000006f
[ 173.864820] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 173.884582] FS: 0000000000000000(0000) GS:ffff88607fc00000(0000) knlGS:0000000000000000
[ 173.885062] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 173.904719] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 173.905102] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 173.924773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 173.944577] Process kworker/0:1 (pid: 0, threadinfo ffff88103e03a000, task ffff88103e0322b0)
[ 173.964428] Stack:
[ 173.964536] ffff88103e03a000 ffffffff82525380 ffff88103e03bf28 ffffffff810391f0
[ 173.964963] ffff88103e03bf18 cbecccf87c85ab64 0000000000000000 0000000000000000
[ 173.984657] ffff88103e03bf48 ffffffff81c064f3 0000000000000000 7d8c76c902581a69
[ 174.004435] Call Trace:
[ 174.004556] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 174.004812] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 174.024453] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 174.064286] Call Trace:
[ 174.064421] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 174.064684] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 174.084288] NMI backtrace for cpu 110
[ 174.084454] CPU 110
[ 174.084554] Modules linked in:
[ 174.104155]
[ 174.104246] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 174.124218] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 174.124669] RSP: 0000:ffff88103e01dee8 EFLAGS: 00000246
[ 174.144277] RAX: 0000000000000000 RBX: ffff88103e01dfd8 RCX: 0000000000000000
[ 174.164040] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0122b0
[ 174.164388] RBP: ffff88103e01def8 R08: 0000000000000000 R09: 0000000000000000
[ 174.184207] R10: 0000000000000000 R11: ffff88103e01ded8 R12: 000000000000006e
[ 174.203949] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 174.204325] FS: 0000000000000000(0000) GS:ffff88607fa00000(0000) knlGS:0000000000000000
[ 174.224273] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 174.224543] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 174.244436] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 174.264550] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 174.264896] Process kworker/0:1 (pid: 0, threadinfo ffff88103e01c000, task ffff88103e0122b0)
[ 174.284294] Stack:
[ 174.284387] ffff88103e01c000 ffffffff82525380 ffff88103e01df28 ffffffff810391f0
[ 174.304005] ffff88103e01df18 e1449d45a00f96cd 0000000000000000 0000000000000000
[ 174.323764] ffff88103e01df48 ffffffff81c064f3 0000000000000000 02e0a2113f244cc6
[ 174.324159] Call Trace:
[ 174.343634] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 174.343882] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 174.344158] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 174.384288] Call Trace:
[ 174.384425] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 174.403684] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 174.403923] NMI backtrace for cpu 113
[ 174.423560] CPU 113
[ 174.423663] Modules linked in:
[ 174.423861]
[ 174.423934] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 174.443871] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 174.463695] RSP: 0000:ffff88103e069ee8 EFLAGS: 00000246
[ 174.483352] RAX: 0000000000000000 RBX: ffff88103e069fd8 RCX: 0000000000000000
[ 174.483659] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e060000
[ 174.503485] RBP: ffff88103e069ef8 R08: 0000000000000000 R09: 0000000000000000
[ 174.503801] R10: 0000000000000000 R11: ffff88103e069ed8 R12: 0000000000000071
[ 174.523541] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 174.543378] FS: 0000000000000000(0000) GS:ffff88707f000000(0000) knlGS:0000000000000000
[ 174.543737] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 174.563452] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 174.583232] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 174.583557] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 174.603297] Process kworker/0:1 (pid: 0, threadinfo ffff88103e068000, task ffff88103e060000)
[ 174.623192] Stack:
[ 174.623287] ffff88103e068000 ffffffff82525380 ffff88103e069f28 ffffffff810391f0
[ 174.643073] ffff88103e069f18 d8cc0a5355bbe9f7 0000000000000000 0000000000000000
[ 174.643484] ffff88103e069f48 ffffffff81c064f3 0000000000000000 43f4297694e0e1e5
[ 174.663235] Call Trace:
[ 174.663371] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 174.682994] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 174.683234] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 174.723167] Call Trace:
[ 174.723277] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 174.723498] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 174.743169] NMI backtrace for cpu 109
[ 174.743359] CPU 109
[ 174.743470] Modules linked in:
[ 174.762927]
[ 174.763016] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 174.783023] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 174.802779] RSP: 0000:ffff88103e007ee8 EFLAGS: 00000246
[ 174.803081] RAX: 0000000000000000 RBX: ffff88103e007fd8 RCX: 0000000000000000
[ 174.822839] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e00a2b0
[ 174.823230] RBP: ffff88103e007ef8 R08: 0000000000000000 R09: 0000000000000000
[ 174.843067] R10: 0000000000000000 R11: ffff88103e007ed8 R12: 000000000000006d
[ 174.862777] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 174.863173] FS: 0000000000000000(0000) GS:ffff88607f800000(0000) knlGS:0000000000000000
[ 174.882958] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 174.902533] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 174.902933] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 174.922697] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 174.942450] Process kworker/0:1 (pid: 0, threadinfo ffff88103e006000, task ffff88103e00a2b0)
[ 174.942950] Stack:
[ 174.962373] ffff88103e006000 ffffffff82525380 ffff88103e007f28 ffffffff810391f0
[ 174.962788] ffff88103e007f18 b33cc9080e000452 0000000000000000 0000000000000000
[ 174.982569] ffff88103e007f48 ffffffff81c064f3 0000000000000000 f44d519eb03a804f
[ 175.002295] Call Trace:
[ 175.002431] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 175.002716] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 175.022347] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 175.043133] Call Trace:
[ 175.062263] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 175.062507] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 175.082302] NMI backtrace for cpu 118
[ 175.082476] CPU 118
[ 175.082569] Modules linked in:
[ 175.082750]
[ 175.102358] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 175.122313] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 175.122690] RSP: 0000:ffff88103e0c3ee8 EFLAGS: 00000246
[ 175.142254] RAX: 0000000000000000 RBX: ffff88103e0c3fd8 RCX: 0000000000000000
[ 175.142563] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0c8000
[ 175.162634] RBP: ffff88103e0c3ef8 R08: 0000000000000000 R09: 0000000000000000
[ 175.182892] R10: 0000000000000000 R11: ffff88103e0c3ed8 R12: 0000000000000076
[ 175.183236] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 175.202258] FS: 0000000000000000(0000) GS:ffff88707fa00000(0000) knlGS:0000000000000000
[ 175.222203] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 175.222483] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 175.242067] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 175.261990] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 175.262321] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0c2000, task ffff88103e0c8000)
[ 175.282550] Stack:
[ 175.282651] ffff88103e0c2000 ffffffff82525380 ffff88103e0c3f28 ffffffff810391f0
[ 175.301851] ffff88103e0c3f18 baf4c5c5e3b24d3d 0000000000000000 0000000000000000
[ 175.302273] ffff88103e0c3f48 ffffffff81c064f3 0000000000000000 93acd84e907831b6
[ 175.321996] Call Trace:
[ 175.322117] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 175.341999] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 175.342255] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 175.401990] Call Trace:
[ 175.402123] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 175.421541] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 175.421796] NMI backtrace for cpu 115
[ 175.441389] CPU 115
[ 175.441514] Modules linked in:
[ 175.441702]
[ 175.441786] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 175.461710] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 175.481505] RSP: 0000:ffff88103e07fee8 EFLAGS: 00000246
[ 175.481767] RAX: 0000000000000000 RBX: ffff88103e07ffd8 RCX: 0000000000000000
[ 175.501529] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e090000
[ 175.521310] RBP: ffff88103e07fef8 R08: 0000000000000000 R09: 0000000000000000
[ 175.521652] R10: 0000000000000000 R11: ffff88103e07fed8 R12: 0000000000000073
[ 175.541406] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 175.561195] FS: 0000000000000000(0000) GS:ffff88707f400000(0000) knlGS:0000000000000000
[ 175.561547] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 175.581297] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 175.581618] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 175.601376] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 175.621180] Process kworker/0:1 (pid: 0, threadinfo ffff88103e07e000, task ffff88103e090000)
[ 175.641133] Stack:
[ 175.641236] ffff88103e07e000 ffffffff82525380 ffff88103e07ff28 ffffffff810391f0
[ 175.641635] ffff88103e07ff18 03a98b03e6b043dd 0000000000000000 0000000000000000
[ 175.661262] ffff88103e07ff48 ffffffff81c064f3 0000000000000000 7da074131e177f44
[ 175.681050] Call Trace:
[ 175.681160] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 175.681405] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 175.701054] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 175.740906] Call Trace:
[ 175.741016] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 175.741263] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 175.760890] NMI backtrace for cpu 114
[ 175.761067] CPU 114
[ 175.761164] Modules linked in:
[ 175.780766]
[ 175.780857] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 175.800802] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 175.801175] RSP: 0000:ffff88103e06fee8 EFLAGS: 00000246
[ 175.820949] RAX: 0000000000000000 RBX: ffff88103e06ffd8 RCX: 0000000000000000
[ 175.840618] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e070000
[ 175.840955] RBP: ffff88103e06fef8 R08: 0000000000000000 R09: 0000000000000000
[ 175.860755] R10: 0000000000000000 R11: ffff88103e06fed8 R12: 0000000000000072
[ 175.880628] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 175.880976] FS: 0000000000000000(0000) GS:ffff88707f200000(0000) knlGS:0000000000000000
[ 175.900667] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 175.900937] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 175.920699] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 175.940501] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 175.940824] Process kworker/0:1 (pid: 0, threadinfo ffff88103e06e000, task ffff88103e070000)
[ 175.960649] Stack:
[ 175.960757] ffff88103e06e000 ffffffff82525380 ffff88103e06ff28 ffffffff810391f0
[ 175.980616] ffff88103e06ff18 fd4b0bbf3b80b10e 0000000000000000 0000000000000000
[ 176.000385] ffff88103e06ff48 ffffffff81c064f3 0000000000000000 c5d770f1aa6234e7
[ 176.000798] Call Trace:
[ 176.020263] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 176.020475] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 176.040168] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 176.060840] Call Trace:
[ 176.060957] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 176.080298] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 176.080576] NMI backtrace for cpu 119
[ 176.100160] CPU 119
[ 176.100260] Modules linked in:
[ 176.100446]
[ 176.100549] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 176.120486] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 176.140276] RSP: 0000:ffff88103e0e9ee8 EFLAGS: 00000246
[ 176.159969] RAX: 0000000000000000 RBX: ffff88103e0e9fd8 RCX: 0000000000000000
[ 176.160268] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0e0000
[ 176.180076] RBP: ffff88103e0e9ef8 R08: 0000000000000000 R09: 0000000000000000
[ 176.200108] R10: 0000000000000000 R11: ffff88103e0e9ed8 R12: 0000000000000077
[ 176.200422] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 176.219971] FS: 0000000000000000(0000) GS:ffff88707fc00000(0000) knlGS:0000000000000000
[ 176.240055] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 176.240329] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 176.260001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 176.260323] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 176.280178] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0e8000, task ffff88103e0e0000)
[ 176.299884] Stack:
[ 176.299995] ffff88103e0e8000 ffffffff82525380 ffff88103e0e9f28 ffffffff810391f0
[ 176.319666] ffff88103e0e9f18 6e7842f8b5c2a888 0000000000000000 0000000000000000
[ 176.320047] ffff88103e0e9f48 ffffffff81c064f3 0000000000000000 b8cf95fcbad3979a
[ 176.339828] Call Trace:
[ 176.339961] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 176.359579] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 176.359827] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 176.399776] Call Trace:
[ 176.399884] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 176.400113] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 176.419675] NMI backtrace for cpu 121
[ 176.419851] CPU 121
[ 176.439359] Modules linked in:
[ 176.439537]
[ 176.439615] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 176.459586] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 176.479535] RSP: 0000:ffff88103e0ffee8 EFLAGS: 00000246
[ 176.479766] RAX: 0000000000000000 RBX: ffff88103e0fffd8 RCX: 0000000000000000
[ 176.499399] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e100000
[ 176.499713] RBP: ffff88103e0ffef8 R08: 0000000000000000 R09: 0000000000000000
[ 176.519517] R10: 0000000000000000 R11: ffff88103e0ffed8 R12: 0000000000000079
[ 176.539297] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 176.539598] FS: 0000000000000000(0000) GS:ffff88807ea00000(0000) knlGS:0000000000000000
[ 176.559431] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 176.579153] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 176.579454] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 176.599256] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 176.619029] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0fe000, task ffff88103e100000)
[ 176.619417] Stack:
[ 176.638976] ffff88103e0fe000 ffffffff82525380 ffff88103e0fff28 ffffffff810391f0
[ 176.639365] ffff88103e0fff18 d84b97c200bc65b0 0000000000000000 0000000000000000
[ 176.659164] ffff88103e0fff48 ffffffff81c064f3 0000000000000000 0bb7ccf3334ed6b1
[ 176.678914] Call Trace:
[ 176.679045] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 176.679291] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 176.698947] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 176.719685] Call Trace:
[ 176.738829] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 176.739063] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 176.758748] NMI backtrace for cpu 112
[ 176.758915] CPU 112
[ 176.759030] Modules linked in:
[ 176.759200]
[ 176.778702] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 176.798700] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 176.799094] RSP: 0000:ffff88103e04bee8 EFLAGS: 00000246
[ 176.818735] RAX: 0000000000000000 RBX: ffff88103e04bfd8 RCX: 0000000000000000
[ 176.819047] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e058000
[ 176.838845] RBP: ffff88103e04bef8 R08: 0000000000000000 R09: 0000000000000000
[ 176.858608] R10: 0000000000000000 R11: ffff88103e04bed8 R12: 0000000000000070
[ 176.858935] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 176.878744] FS: 0000000000000000(0000) GS:ffff88707ee00000(0000) knlGS:0000000000000000
[ 176.898581] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 176.898838] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 176.918587] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 176.938368] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 176.938680] Process kworker/0:1 (pid: 0, threadinfo ffff88103e04a000, task ffff88103e058000)
[ 176.958545] Stack:
[ 176.958648] ffff88103e04a000 ffffffff82525380 ffff88103e04bf28 ffffffff810391f0
[ 176.978461] ffff88103e04bf18 eddbed60db20a548 0000000000000000 0000000000000000
[ 176.978874] ffff88103e04bf48 ffffffff81c064f3 0000000000000000 cc0f66974ad24309
[ 176.998604] Call Trace:
[ 176.998722] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 177.018375] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 177.018628] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 177.058662] Call Trace:
[ 177.058783] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 177.078171] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 177.078406] NMI backtrace for cpu 117
[ 177.098030] CPU 117
[ 177.098133] Modules linked in:
[ 177.098321]
[ 177.098413] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 177.118343] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 177.138165] RSP: 0000:ffff88103e0b3ee8 EFLAGS: 00000246
[ 177.138419] RAX: 0000000000000000 RBX: ffff88103e0b3fd8 RCX: 0000000000000000
[ 177.158150] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0b8000
[ 177.177953] RBP: ffff88103e0b3ef8 R08: 0000000000000000 R09: 0000000000000000
[ 177.178278] R10: 0000000000000000 R11: ffff88103e0b3ed8 R12: 0000000000000075
[ 177.198098] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 177.217970] FS: 0000000000000000(0000) GS:ffff88707f800000(0000) knlGS:0000000000000000
[ 177.218347] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 177.238072] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 177.257983] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 177.258296] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 177.277854] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0b2000, task ffff88103e0b8000)
[ 177.297799] Stack:
[ 177.297902] ffff88103e0b2000 ffffffff82525380 ffff88103e0b3f28 ffffffff810391f0
[ 177.298272] ffff88103e0b3f18 3c05f38b456efa63 0000000000000000 0000000000000000
[ 177.317959] ffff88103e0b3f48 ffffffff81c064f3 0000000000000000 c20981672419ff31
[ 177.337683] Call Trace:
[ 177.337806] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 177.338044] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 177.357713] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 177.397558] Call Trace:
[ 177.397668] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 177.397902] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 177.417546] NMI backtrace for cpu 116
[ 177.417702] CPU 116
[ 177.417815] Modules linked in:
[ 177.437415]
[ 177.437495] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 177.457463] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 177.477254] RSP: 0000:ffff88103e09dee8 EFLAGS: 00000246
[ 177.477534] RAX: 0000000000000000 RBX: ffff88103e09dfd8 RCX: 0000000000000000
[ 177.497285] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0a0000
[ 177.497622] RBP: ffff88103e09def8 R08: 0000000000000000 R09: 0000000000000000
[ 177.517372] R10: 0000000000000000 R11: ffff88103e09ded8 R12: 0000000000000074
[ 177.537173] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 177.537477] FS: 0000000000000000(0000) GS:ffff88707f600000(0000) knlGS:0000000000000000
[ 177.557300] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 177.557546] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 177.577367] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 177.597127] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 177.597452] Process kworker/0:1 (pid: 0, threadinfo ffff88103e09c000, task ffff88103e0a0000)
[ 177.617307] Stack:
[ 177.617408] ffff88103e09c000 ffffffff82525380 ffff88103e09df28 ffffffff810391f0
[ 177.637217] ffff88103e09df18 f5a140315aa62005 0000000000000000 0000000000000000
[ 177.656995] ffff88103e09df48 ffffffff81c064f3 0000000000000000 45979eb37cc9aeb0
[ 177.657430] Call Trace:
[ 177.676927] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 177.677154] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 177.696815] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 177.717521] Call Trace:
[ 177.717651] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 177.736943] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 177.737188] NMI backtrace for cpu 125
[ 177.756805] CPU 125
[ 177.756897] Modules linked in:
[ 177.757087]
[ 177.757163] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 177.777127] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 177.796916] RSP: 0000:ffff88103e159ee8 EFLAGS: 00000246
[ 177.816590] RAX: 0000000000000000 RBX: ffff88103e159fd8 RCX: 0000000000000000
[ 177.816924] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e150000
[ 177.836719] RBP: ffff88103e159ef8 R08: 0000000000000000 R09: 0000000000000000
[ 177.856497] R10: 0000000000000000 R11: ffff88103e159ed8 R12: 000000000000007d
[ 177.856810] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 177.876696] FS: 0000000000000000(0000) GS:ffff88807f200000(0000) knlGS:0000000000000000
[ 177.896413] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 177.896670] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 177.916474] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 177.916785] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 177.936571] Process kworker/0:1 (pid: 0, threadinfo ffff88103e158000, task ffff88103e150000)
[ 177.956438] Stack:
[ 177.956528] ffff88103e158000 ffffffff82525380 ffff88103e159f28 ffffffff810391f0
[ 177.976430] ffff88103e159f18 2ccd79cfdd34ac9e 0000000000000000 0000000000000000
[ 177.976840] ffff88103e159f48 ffffffff81c064f3 0000000000000000 10ccf89162e4ca4a
[ 177.996459] Call Trace:
[ 177.996608] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.016238] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 178.016516] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 178.056435] Call Trace:
[ 178.056550] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.076039] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 178.076359] NMI backtrace for cpu 123
[ 178.076517] CPU 123
[ 178.096023] Modules linked in:
[ 178.096213]
[ 178.096289] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 178.116243] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 178.136006] RSP: 0000:ffff88103e12bee8 EFLAGS: 00000246
[ 178.136240] RAX: 0000000000000000 RBX: ffff88103e12bfd8 RCX: 0000000000000000
[ 178.156040] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e130000
[ 178.156368] RBP: ffff88103e12bef8 R08: 0000000000000000 R09: 0000000000000000
[ 178.176117] R10: 0000000000000000 R11: ffff88103e12bed8 R12: 000000000000007b
[ 178.195946] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 178.196293] FS: 0000000000000000(0000) GS:ffff88807ee00000(0000) knlGS:0000000000000000
[ 178.216128] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 178.235947] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 178.236261] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 178.255967] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 178.276291] Process kworker/0:1 (pid: 0, threadinfo ffff88103e12a000, task ffff88103e130000)
[ 178.276664] Stack:
[ 178.295771] ffff88103e12a000 ffffffff82525380 ffff88103e12bf28 ffffffff810391f0
[ 178.296163] ffff88103e12bf18 a4a843542a24023c 0000000000000000 0000000000000000
[ 178.315916] ffff88103e12bf48 ffffffff81c064f3 0000000000000000 e267619a78df27fa
[ 178.335708] Call Trace:
[ 178.335834] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.336074] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 178.355568] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 178.376576] Call Trace:
[ 178.395482] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.395696] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 178.415403] NMI backtrace for cpu 105
[ 178.415563] CPU 105
[ 178.415663] Modules linked in:
[ 178.415851]
[ 178.435334] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 178.455580] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 178.455977] RSP: 0000:ffff88103e79dee8 EFLAGS: 00000246
[ 178.475387] RAX: 0000000000000000 RBX: ffff88103e79dfd8 RCX: 0000000000000000
[ 178.475802] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7922b0
[ 178.495462] RBP: ffff88103e79def8 R08: 0000000000000000 R09: 0000000000000000
[ 178.515274] R10: 0000000000000000 R11: ffff88103e79ded8 R12: 0000000000000069
[ 178.515657] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 178.535391] FS: 0000000000000000(0000) GS:ffff88607f000000(0000) knlGS:0000000000000000
[ 178.555181] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 178.555495] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 178.575260] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 178.595017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 178.595353] Process kworker/0:1 (pid: 0, threadinfo ffff88103e79c000, task ffff88103e7922b0)
[ 178.615200] Stack:
[ 178.615322] ffff88103e79c000 ffffffff82525380 ffff88103e79df28 ffffffff810391f0
[ 178.635134] ffff88103e79df18 7cc4089422c6d371 0000000000000000 0000000000000000
[ 178.654849] ffff88103e79df48 ffffffff81c064f3 0000000000000000 6ca04c2dc270ff18
[ 178.655284] Call Trace:
[ 178.655424] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.674979] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 178.675290] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 178.715289] Call Trace:
[ 178.715406] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.734804] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 178.735144] NMI backtrace for cpu 126
[ 178.754941] CPU 126
[ 178.755032] Modules linked in:
[ 178.755201]
[ 178.755290] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 178.775008] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 178.794775] RSP: 0000:ffff88103e171ee8 EFLAGS: 00000246
[ 178.795025] RAX: 0000000000000000 RBX: ffff88103e171fd8 RCX: 0000000000000000
[ 178.814809] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e168000
[ 178.834586] RBP: ffff88103e171ef8 R08: 0000000000000000 R09: 0000000000000000
[ 178.834896] R10: 0000000000000000 R11: ffff88103e171ed8 R12: 000000000000007e
[ 178.854681] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 178.874557] FS: 0000000000000000(0000) GS:ffff88807f400000(0000) knlGS:0000000000000000
[ 178.874911] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 178.894550] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 178.914336] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 178.914650] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 178.934465] Process kworker/0:1 (pid: 0, threadinfo ffff88103e170000, task ffff88103e168000)
[ 178.954274] Stack:
[ 178.954381] ffff88103e170000 ffffffff82525380 ffff88103e171f28 ffffffff810391f0
[ 178.954773] ffff88103e171f18 5b8357237cb2986a 0000000000000000 0000000000000000
[ 178.974574] ffff88103e171f48 ffffffff81c064f3 0000000000000000 5d4840ef688c0c41
[ 178.994347] Call Trace:
[ 178.994476] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 178.994722] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 179.014351] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 179.054246] Call Trace:
[ 179.054376] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 179.054598] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 179.074151] NMI backtrace for cpu 127
[ 179.074317] CPU 127
[ 179.074407] Modules linked in:
[ 179.094038]
[ 179.094119] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
[ 179.114091] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
[ 179.133888] RSP: 0000:ffff88103e177ee8 EFLAGS: 00000246
[ 179.134130] RAX: 0000000000000000 RBX: ffff88103e177fd8 RCX: 0000000000000000
[ 179.153912] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e188000
[ 179.154239] RBP: ffff88103e177ef8 R08: 0000000000000000 R09: 0000000000000000
[ 179.174044] R10: 0000000000000000 R11: ffff88103e177ed8 R12: 000000000000007f
[ 179.193801] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 179.194108] FS: 0000000000000000(0000) GS:ffff88807f600000(0000) knlGS:0000000000000000
[ 179.213996] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 179.233745] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
[ 179.234072] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 179.254037] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 179.254359] Process kworker/0:1 (pid: 0, threadinfo ffff88103e176000, task ffff88103e188000)
[ 179.273981] Stack:
[ 179.274099] ffff88103e176000 ffffffff82525380 ffff88103e177f28 ffffffff810391f0
[ 179.294154] ffff88103e177f18 23c99be1e1fa11ca 0000000000000000 0000000000000000
[ 179.313859] ffff88103e177f48 ffffffff81c064f3 0000000000000000 75b2778518a8799c
[ 179.314286] Call Trace:
[ 179.333745] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 179.333980] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
[ 179.353757] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
[ 179.374328] Call Trace:
[ 179.374439] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
[ 179.393814] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-10 20:52 ` Yinghai Lu
@ 2011-05-11 4:54 ` Paul E. McKenney
2011-05-11 6:03 ` Yinghai Lu
` (3 more replies)
0 siblings, 4 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-11 4:54 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 281129 bytes --]
On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
> > On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
> >> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
> >>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
> >>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
> >>>>>
> >>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> >>>>>
> >>>>>> Hello, Ingo,
> >>>>>>
> >>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
> >>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
> >>>>>> courtesy of Lai Jiangshan. These two features cover well over half
> >>>>>> of the commits. There are a number of smaller features and bug fixes.
> >>>>>> All have been sent to LKML in the following batches:
> >>>>>>
> >>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> >>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> >>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> >>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
> >>>>>>
> >>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
> >>>>>> maintainers. I have some additional kfree_rcu() requests that lack
> >>>>>> Acked-by:s, and I will deal with these later.
> >>>>>>
> >>>>>> These channges are available in the -rcu git repository at:
> >>>>>>
> >>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
> >>>>>
> >>>>> Pulled, thanks a lot Paul!
> >>>>>
> >>>>
> >>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
> >>>>
> >>>> after hard code to enable rcu_cpu_stall_suppress
> >>>>
> >>>> Index: linux-2.6/kernel/rcutree.c
> >>>> ===================================================================
> >>>> --- linux-2.6.orig/kernel/rcutree.c
> >>>> +++ linux-2.6/kernel/rcutree.c
> >>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
> >>>> module_param(qhimark, int, 0);
> >>>> module_param(qlowmark, int, 0);
> >>>>
> >>>> -int rcu_cpu_stall_suppress __read_mostly;
> >>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
> >>>> module_param(rcu_cpu_stall_suppress, int, 0644);
> >>>>
> >>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
> >>>>
> >>>> will get system hang after pnp ACPI init.
> >>>
> >>> Could you please send the stack traces from the RCU CPU stall? Also,
> >>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
> >>>
> >>> Thanx, Paul
> >>
> >> Do not have time to bisect it at this point.
> >
> > Could you please send the stack traces from the RCU CPU stall?
Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
Everyone is idle, except for CPU 124, which detected the stall, and
possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
that looks like a backtrace error to me. The fact that it hangs if you
disable RCU CPU stall detection leads me to believe that something real
is being detected.
This looks very similar to the situation people were seeing before
ce31332d3c77532d6ea97ddcb475a2b02dd358b4 was applied, so I have attached
the diagnostic script that helped track this down. Could you please
enable CONFIG_RCU_TRACE, mount debugfs, and run the attached script,
and send me the output? Please check to make sure that the script knows
where you mounted debugfs, of course.
Thanx, Paul
> [ 76.553010] calling pnpacpi_init+0x0/0x8c @ 1
> [ 76.553236] pnp: PnP ACPI init
> [ 76.572833] ACPI: bus type pnp registered
> [ 76.573570] pnp 00:00: [bus 00-3f]
> [ 76.573766] pnp 00:00: [io 0x0000-0x5fff window]
> [ 76.592818] pnp 00:00: [mem 0x000a0000-0x000bffff window]
> [ 76.593099] pnp 00:00: [mem 0x000d0000-0x000dffff window]
> [ 76.612741] pnp 00:00: [mem 0x00000000-0xffffffff window]
> [ 76.613001] pnp 00:00: [mem 0x90000000-0xafffffff window]
> [ 76.632696] pnp 00:00: [mem 0xfed40000-0xfed44fff window]
> [ 76.632951] pnp 00:00: [mem 0xfc000000000-0xfc07fffffff window]
> [ 76.653366] pnp 00:00: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
> [ 76.653730] pnp 00:01: [mem 0xfc000000-0xfcffffff]
> [ 76.672665] pnp 00:01: [mem 0xfd000000-0xfdffffff]
> [ 76.672887] pnp 00:01: [mem 0xfe000000-0xfebfffff]
> [ 76.693831] system 00:01: [mem 0xfc000000-0xfcffffff] has been reserved
> [ 76.694151] system 00:01: [mem 0xfd000000-0xfdffffff] has been reserved
> [ 76.712632] system 00:01: [mem 0xfe000000-0xfebfffff] has been reserved
> [ 76.712927] system 00:01: Plug and Play ACPI device, IDs PNP0c01 (active)
> [ 136.364832] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 124, t=15002 jiffies)
> [ 136.365317] sending NMI to all CPUs:
> [ 136.381054] NMI backtrace for cpu 0
> [ 136.381364] CPU 0
> [ 136.381450] Modules linked in:
> [ 136.381638]
> [ 136.400938] Pid: 0, comm: swapper Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 136.420934] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 136.421313] RSP: 0018:ffffffff82397ee8 EFLAGS: 00000246
> [ 136.440953] RAX: 0000000000000000 RBX: ffffffff82397fd8 RCX: 0000000000000000
> [ 136.441256] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff823a5020
> [ 136.461059] RBP: ffffffff82397ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 136.480842] R10: ffffffffffffffff R11: ffffffff82397ed8 R12: 0000000000000000
> [ 136.481133] R13: ffff88807ffff300 R14: ffffffffffffffff R15: 0000000000000000
> [ 136.500932] FS: 0000000000000000(0000) GS:ffff88107de00000(0000) knlGS:0000000000000000
> [ 136.520779] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 136.521046] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006f0
> [ 136.540800] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 136.560608] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 136.560936] Process swapper (pid: 0, threadinfo ffffffff82396000, task ffffffff823a5020)
> [ 136.580762] Stack:
> [ 136.580852] ffffffff82396000 ffffffff82525380 ffffffff82397f28 ffffffff810391f0
> [ 136.600682] 6db6db6db6db6db7 f51568c424a0f956 0000000000000002 6db6db6db6db6db7
> [ 136.601087] ffffffff82397f58 ffffffff81bbbb93 ffffffff81bbbadc 0000000000000000
> [ 136.620945] Call Trace:
> [ 136.621063] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 136.640561] [<ffffffff81bbbb93>] rest_init+0xb7/0xbe
> [ 136.640805] [<ffffffff81bbbadc>] ? csum_partial_copy_generic+0x16c/0x16c
> [ 136.660553] [<ffffffff826ffb4a>] start_kernel+0x3ba/0x3c5
> [ 136.660814] [<ffffffff826ff2a6>] x86_64_start_reservations+0xb6/0xba
> [ 136.680553] [<ffffffff826ff39c>] x86_64_start_kernel+0xf2/0xf9
> [ 136.680831] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 136.721038] Call Trace:
> [ 136.721160] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 136.740386] [<ffffffff81bbbb93>] rest_init+0xb7/0xbe
> [ 136.740630] [<ffffffff81bbbadc>] ? csum_partial_copy_generic+0x16c/0x16c
> [ 136.760349] [<ffffffff826ffb4a>] start_kernel+0x3ba/0x3c5
> [ 136.760626] [<ffffffff826ff2a6>] x86_64_start_reservations+0xb6/0xba
> [ 136.780369] [<ffffffff826ff39c>] x86_64_start_kernel+0xf2/0xf9
> [ 136.800134] NMI backtrace for cpu 1
> [ 136.800321] CPU 1
> [ 136.800436] Modules linked in:
> [ 136.800616]
> [ 136.800703] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 136.839999] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 136.840388] RSP: 0000:ffff88103ee33ee8 EFLAGS: 00000246
> [ 136.860052] RAX: 0000000000000000 RBX: ffff88103ee33fd8 RCX: 0000000000000000
> [ 136.860373] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ee38000
> [ 136.880168] RBP: ffff88103ee33ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 136.900089] R10: 0000000000000000 R11: ffff88103ee33ed8 R12: 0000000000000001
> [ 136.900416] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 136.920090] FS: 0000000000000000(0000) GS:ffff88107e000000(0000) knlGS:0000000000000000
> [ 136.950414] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 136.950684] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 136.970157] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 136.970455] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 136.990120] Process kworker/0:0 (pid: 0, threadinfo ffff88103ee32000, task ffff88103ee38000)
> [ 137.009848] Stack:
> [ 137.009959] ffff88103ee32000 ffffffff82525380 ffff88103ee33f28 ffffffff810391f0
> [ 137.032505] ffff88103ee33f18 2bd3bd551d7037b4 0000000000000000 0000000000000000
> [ 137.032879] ffff88103ee33f48 ffffffff81c064f3 0000000000000000 661fe41cd64eec39
> [ 137.049938] Call Trace:
> [ 137.050080] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 137.069800] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 137.070075] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 137.109943] Call Trace:
> [ 137.110080] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 137.129466] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 137.129703] NMI backtrace for cpu 2
> [ 137.129877] CPU 2
> [ 137.149412] Modules linked in:
> [ 137.149603]
> [ 137.149705] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 137.169681] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 137.189674] RSP: 0000:ffff88103ee5bee8 EFLAGS: 00000246
> [ 137.189933] RAX: 0000000000000000 RBX: ffff88103ee5bfd8 RCX: 0000000000000000
> [ 137.209465] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ee60000
> [ 137.229230] RBP: ffff88103ee5bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 137.229570] R10: 0000000000000000 R11: ffff88103ee5bed8 R12: 0000000000000002
> [ 137.249542] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 137.269122] FS: 0000000000000000(0000) GS:ffff88107e200000(0000) knlGS:0000000000000000
> [ 137.269486] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 137.289226] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 137.289536] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 137.309303] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 137.329096] Process kworker/0:1 (pid: 0, threadinfo ffff88103ee5a000, task ffff88103ee60000)
> [ 137.329443] Stack:
> [ 137.349030] ffff88103ee5a000 ffffffff82525380 ffff88103ee5bf28 ffffffff810391f0
> [ 137.349444] ffff88103ee5bf18 2a2988cc3681d8f7 0000000000000000 0000000000000000
> [ 137.369236] ffff88103ee5bf48 ffffffff81c064f3 0000000000000000 63a332c588e07727
> [ 137.388974] Call Trace:
> [ 137.389121] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 137.389353] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 137.409021] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 137.448825] Call Trace:
> [ 137.448951] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 137.449175] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 137.468814] NMI backtrace for cpu 3
> [ 137.468990] CPU 3
> [ 137.469084] Modules linked in:
> [ 137.469272]
> [ 137.488737] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 137.508741] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 137.509123] RSP: 0000:ffff88103ee83ee8 EFLAGS: 00000246
> [ 137.528765] RAX: 0000000000000000 RBX: ffff88103ee83fd8 RCX: 0000000000000000
> [ 137.529096] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ee88000
> [ 137.548872] RBP: ffff88103ee83ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 137.568641] R10: 0000000000000000 R11: ffff88103ee83ed8 R12: 0000000000000003
> [ 137.568957] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 137.588755] FS: 0000000000000000(0000) GS:ffff88107e400000(0000) knlGS:0000000000000000
> [ 137.608591] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 137.608847] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 137.628623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 137.648432] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 137.648728] Process kworker/0:1 (pid: 0, threadinfo ffff88103ee82000, task ffff88103ee88000)
> [ 137.670089] Stack:
> [ 137.670189] ffff88103ee82000 ffffffff82525380 ffff88103ee83f28 ffffffff810391f0
> [ 137.688507] ffff88103ee83f18 cacb1e5ab5a0da93 0000000000000000 0000000000000000
> [ 137.708279] ffff88103ee83f48 ffffffff81c064f3 0000000000000000 83c0d3c3e7f0ed35
> [ 137.708699] Call Trace:
> [ 137.708835] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 137.728388] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 137.728645] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 137.768750] Call Trace:
> [ 137.768874] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 137.788207] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 137.788493] NMI backtrace for cpu 4
> [ 137.808088] CPU 4
> [ 137.808175] Modules linked in:
> [ 137.808384]
> [ 137.808461] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 137.828362] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 137.848186] RSP: 0000:ffff88103eeadee8 EFLAGS: 00000246
> [ 137.848440] RAX: 0000000000000000 RBX: ffff88103eeadfd8 RCX: 0000000000000000
> [ 137.868205] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eeb0000
> [ 137.887990] RBP: ffff88103eeadef8 R08: 0000000000000000 R09: 0000000000000000
> [ 137.888308] R10: 0000000000000000 R11: ffff88103eeaded8 R12: 0000000000000004
> [ 137.908084] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 137.928028] FS: 0000000000000000(0000) GS:ffff88107e600000(0000) knlGS:0000000000000000
> [ 137.928376] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 137.948182] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 137.968138] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 137.968453] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 137.987930] Process kworker/0:1 (pid: 0, threadinfo ffff88103eeac000, task ffff88103eeb0000)
> [ 138.007782] Stack:
> [ 138.007905] ffff88103eeac000 ffffffff82525380 ffff88103eeadf28 ffffffff810391f0
> [ 138.008306] ffff88103eeadf18 37300352987163c3 0000000000000000 0000000000000000
> [ 138.028077] ffff88103eeadf48 ffffffff81c064f3 0000000000000000 f12528251e665927
> [ 138.048050] Call Trace:
> [ 138.048173] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 138.048386] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 138.067974] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 138.107791] Call Trace:
> [ 138.107916] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 138.108150] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 138.127714] NMI backtrace for cpu 5
> [ 138.127899] CPU 5
> [ 138.127999] Modules linked in:
> [ 138.147833]
> [ 138.147911] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 138.167633] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 138.187390] RSP: 0000:ffff88103eecfee8 EFLAGS: 00000246
> [ 138.187621] RAX: 0000000000000000 RBX: ffff88103eecffd8 RCX: 0000000000000000
> [ 138.207291] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eed0000
> [ 138.207625] RBP: ffff88103eecfef8 R08: 0000000000000000 R09: 0000000000000000
> [ 138.227397] R10: 0000000000000000 R11: ffff88103eecfed8 R12: 0000000000000005
> [ 138.247345] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 138.247657] FS: 0000000000000000(0000) GS:ffff88107e800000(0000) knlGS:0000000000000000
> [ 138.267316] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 138.267577] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 138.287344] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 138.307936] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 138.308252] Process kworker/0:1 (pid: 0, threadinfo ffff88103eece000, task ffff88103eed0000)
> [ 138.327333] Stack:
> [ 138.327424] ffff88103eece000 ffffffff82525380 ffff88103eecff28 ffffffff810391f0
> [ 138.347307] ffff88103eecff18 5c976e576efe47b3 0000000000000000 0000000000000000
> [ 138.367037] ffff88103eecff48 ffffffff81c064f3 0000000000000000 f61e8ac9274555e7
> [ 138.367426] Call Trace:
> [ 138.386913] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 138.387162] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 138.406850] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 138.427617] Call Trace:
> [ 138.427731] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 138.446968] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 138.447219] NMI backtrace for cpu 6
> [ 138.466818] CPU 6
> [ 138.466913] Modules linked in:
> [ 138.467084]
> [ 138.467169] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 138.487237] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 138.506894] RSP: 0000:ffff88103eef7ee8 EFLAGS: 00000246
> [ 138.526591] RAX: 0000000000000000 RBX: ffff88103eef7fd8 RCX: 0000000000000000
> [ 138.526931] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eef8000
> [ 138.546705] RBP: ffff88103eef7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 138.566499] R10: 0000000000000000 R11: ffff88103eef7ed8 R12: 0000000000000006
> [ 138.566831] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 138.586585] FS: 0000000000000000(0000) GS:ffff88107ea00000(0000) knlGS:0000000000000000
> [ 138.606412] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 138.606667] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 138.626666] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 138.626982] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 138.646565] Process kworker/0:1 (pid: 0, threadinfo ffff88103eef6000, task ffff88103eef8000)
> [ 138.666398] Stack:
> [ 138.666489] ffff88103eef6000 ffffffff82525380 ffff88103eef7f28 ffffffff810391f0
> [ 138.686347] ffff88103eef7f18 15e4bcc96f1f15ba 0000000000000000 0000000000000000
> [ 138.686762] ffff88103eef7f48 ffffffff81c064f3 0000000000000000 9898fe604c13b8fd
> [ 138.706504] Call Trace:
> [ 138.706613] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 138.726202] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 138.726447] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 138.766487] Call Trace:
> [ 138.766626] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 138.766890] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 138.786275] NMI backtrace for cpu 7
> [ 138.786460] CPU 7
> [ 138.786556] Modules linked in:
> [ 138.806144]
> [ 138.806222] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 138.826206] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 138.845997] RSP: 0000:ffff88103ef17ee8 EFLAGS: 00000246
> [ 138.846225] RAX: 0000000000000000 RBX: ffff88103ef17fd8 RCX: 0000000000000000
> [ 138.866022] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef20000
> [ 138.866343] RBP: ffff88103ef17ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 138.886108] R10: 0000000000000000 R11: ffff88103ef17ed8 R12: 0000000000000007
> [ 138.905915] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 138.906216] FS: 0000000000000000(0000) GS:ffff88107ec00000(0000) knlGS:0000000000000000
> [ 138.926104] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 138.945846] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 138.946147] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 138.976047] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 138.995888] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef16000, task ffff88103ef20000)
> [ 138.996263] Stack:
> [ 139.015720] ffff88103ef16000 ffffffff82525380 ffff88103ef17f28 ffffffff810391f0
> [ 139.016116] ffff88103ef17f18 5945687aae80abf9 0000000000000000 0000000000000000
> [ 139.035768] ffff88103ef17f48 ffffffff81c064f3 0000000000000000 1e4ee324f58b290c
> [ 139.055524] Call Trace:
> [ 139.055657] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 139.055904] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 139.075528] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 139.096417] Call Trace:
> [ 139.115416] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 139.115626] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 139.135361] NMI backtrace for cpu 8
> [ 139.135539] CPU 8
> [ 139.135636] Modules linked in:
> [ 139.135810]
> [ 139.135888] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 139.175277] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 139.175644] RSP: 0000:ffff88103ef2dee8 EFLAGS: 00000246
> [ 139.195308] RAX: 0000000000000000 RBX: ffff88103ef2dfd8 RCX: 0000000000000000
> [ 139.195607] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef24560
> [ 139.215407] RBP: ffff88103ef2def8 R08: 0000000000000000 R09: 0000000000000000
> [ 139.235209] R10: 0000000000000000 R11: ffff88103ef2ded8 R12: 0000000000000008
> [ 139.235524] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 139.255299] FS: 0000000000000000(0000) GS:ffff88207de00000(0000) knlGS:0000000000000000
> [ 139.275144] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 139.275380] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 139.295174] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 139.295480] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 139.315261] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef2c000, task ffff88103ef24560)
> [ 139.335121] Stack:
> [ 139.335229] ffff88103ef2c000 ffffffff82525380 ffff88103ef2df28 ffffffff810391f0
> [ 139.355031] ffff88103ef2df18 eb2a74b30d32a306 0000000000000000 0000000000000000
> [ 139.355459] ffff88103ef2df48 ffffffff81c064f3 0000000000000000 27567f4754362612
> [ 139.375207] Call Trace:
> [ 139.375321] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 139.394922] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 139.395182] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 139.435249] Call Trace:
> [ 139.435373] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 139.454753] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 139.455037] NMI backtrace for cpu 9
> [ 139.455259] CPU 9
> [ 139.474697] Modules linked in:
> [ 139.474865]
> [ 139.474955] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 139.494927] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 139.514696] RSP: 0000:ffff88103ef43ee8 EFLAGS: 00000246
> [ 139.514927] RAX: 0000000000000000 RBX: ffff88103ef43fd8 RCX: 0000000000000000
> [ 139.534738] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef3a2b0
> [ 139.554512] RBP: ffff88103ef43ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 139.554835] R10: 0000000000000000 R11: ffff88103ef43ed8 R12: 0000000000000009
> [ 139.574608] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 139.594406] FS: 0000000000000000(0000) GS:ffff88207e000000(0000) knlGS:0000000000000000
> [ 139.594741] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 139.615073] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 139.615381] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 139.634602] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 139.654372] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef42000, task ffff88103ef3a2b0)
> [ 139.654753] Stack:
> [ 139.674315] ffff88103ef42000 ffffffff82525380 ffff88103ef43f28 ffffffff810391f0
> [ 139.674694] ffff88103ef43f18 259a5068eb4fd413 0000000000000000 0000000000000000
> [ 139.694533] ffff88103ef43f48 ffffffff81c064f3 0000000000000000 239846a0b996998b
> [ 139.714264] Call Trace:
> [ 139.714401] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 139.714636] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 139.734285] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 139.774083] Call Trace:
> [ 139.774217] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 139.774475] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 139.794095] NMI backtrace for cpu 10
> [ 139.794275] CPU 10
> [ 139.794363] Modules linked in:
> [ 139.813951]
> [ 139.814034] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 139.834046] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 139.834393] RSP: 0000:ffff88103ef51ee8 EFLAGS: 00000246
> [ 139.854066] RAX: 0000000000000000 RBX: ffff88103ef51fd8 RCX: 0000000000000000
> [ 139.873832] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef4a2b0
> [ 139.874139] RBP: ffff88103ef51ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 139.893924] R10: 0000000000000000 R11: ffff88103ef51ed8 R12: 000000000000000a
> [ 139.894249] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 139.914130] FS: 0000000000000000(0000) GS:ffff88207e200000(0000) knlGS:0000000000000000
> [ 139.934036] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 139.934308] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 139.954045] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 139.973916] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 139.974244] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef50000, task ffff88103ef4a2b0)
> [ 139.993995] Stack:
> [ 139.994086] ffff88103ef50000 ffffffff82525380 ffff88103ef51f28 ffffffff810391f0
> [ 140.013955] ffff88103ef51f18 ecfea0cf87080794 0000000000000000 0000000000000000
> [ 140.033587] ffff88103ef51f48 ffffffff81c064f3 0000000000000000 18b78e007f343e26
> [ 140.033982] Call Trace:
> [ 140.034112] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 140.053671] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 140.053948] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 140.094038] Call Trace:
> [ 140.094184] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 140.113500] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 140.113767] NMI backtrace for cpu 11
> [ 140.133361] CPU 11
> [ 140.133453] Modules linked in:
> [ 140.133631]
> [ 140.133721] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 140.153658] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 140.173459] RSP: 0000:ffff88103ef69ee8 EFLAGS: 00000246
> [ 140.173730] RAX: 0000000000000000 RBX: ffff88103ef69fd8 RCX: 0000000000000000
> [ 140.193479] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef5a2b0
> [ 140.213268] RBP: ffff88103ef69ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 140.213583] R10: 0000000000000000 R11: ffff88103ef69ed8 R12: 000000000000000b
> [ 140.233334] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 140.258346] FS: 0000000000000000(0000) GS:ffff88207e400000(0000) knlGS:0000000000000000
> [ 140.258715] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 140.273228] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 140.293022] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 140.293328] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 140.313135] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef68000, task ffff88103ef5a2b0)
> [ 140.332980] Stack:
> [ 140.333075] ffff88103ef68000 ffffffff82525380 ffff88103ef69f28 ffffffff810391f0
> [ 140.352854] ffff88103ef69f18 11d983a4aef23a75 0000000000000000 0000000000000000
> [ 140.353269] ffff88103ef69f48 ffffffff81c064f3 0000000000000000 32786c201221b6e6
> [ 140.373033] Call Trace:
> [ 140.373147] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 140.392777] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 140.393028] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 140.432965] Call Trace:
> [ 140.433087] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 140.433318] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 140.452856] NMI backtrace for cpu 12
> [ 140.453047] CPU 12
> [ 140.453152] Modules linked in:
> [ 140.472699]
> [ 140.472804] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 140.492785] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 140.512550] RSP: 0000:ffff88103ef6fee8 EFLAGS: 00000246
> [ 140.512772] RAX: 0000000000000000 RBX: ffff88103ef6ffd8 RCX: 0000000000000000
> [ 140.532588] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef7a2b0
> [ 140.532919] RBP: ffff88103ef6fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 140.552775] R10: 0000000000000000 R11: ffff88103ef6fed8 R12: 000000000000000c
> [ 140.572474] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 140.572805] FS: 0000000000000000(0000) GS:ffff88207e600000(0000) knlGS:0000000000000000
> [ 140.592615] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 140.612359] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 140.612675] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 140.632533] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 140.632844] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef6e000, task ffff88103ef7a2b0)
> [ 140.652591] Stack:
> [ 140.652685] ffff88103ef6e000 ffffffff82525380 ffff88103ef6ff28 ffffffff810391f0
> [ 140.672580] ffff88103ef6ff18 9d782048b5866d90 0000000000000000 0000000000000000
> [ 140.692362] ffff88103ef6ff48 ffffffff81c064f3 0000000000000000 05d404c0ff314e85
> [ 140.692756] Call Trace:
> [ 140.712335] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 140.712575] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 140.732124] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 140.752935] Call Trace:
> [ 140.772011] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 140.772230] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 140.772510] NMI backtrace for cpu 13
> [ 140.792102] CPU 13
> [ 140.792197] Modules linked in:
> [ 140.792371]
> [ 140.792458] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 140.812395] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 140.832216] RSP: 0000:ffff88103ef8dee8 EFLAGS: 00000246
> [ 140.851940] RAX: 0000000000000000 RBX: ffff88103ef8dfd8 RCX: 0000000000000000
> [ 140.852244] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef822b0
> [ 140.872040] RBP: ffff88103ef8def8 R08: 0000000000000000 R09: 0000000000000000
> [ 140.892636] R10: 0000000000000000 R11: ffff88103ef8ded8 R12: 000000000000000d
> [ 140.892947] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 140.911916] FS: 0000000000000000(0000) GS:ffff88207e800000(0000) knlGS:0000000000000000
> [ 140.931832] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 140.932117] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 140.952036] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 140.952379] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 140.972012] Process kworker/0:1 (pid: 0, threadinfo ffff88103ef8c000, task ffff88103ef822b0)
> [ 140.991960] Stack:
> [ 140.992061] ffff88103ef8c000 ffffffff82525380 ffff88103ef8df28 ffffffff810391f0
> [ 141.011881] ffff88103ef8df18 1496eb38451a5aa8 0000000000000000 0000000000000000
> [ 141.012288] ffff88103ef8df48 ffffffff81c064f3 0000000000000000 1946ee8e3f03f230
> [ 141.031949] Call Trace:
> [ 141.032057] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 141.051548] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 141.051815] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 141.091816] Call Trace:
> [ 141.091945] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 141.111379] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 141.111673] NMI backtrace for cpu 14
> [ 141.111839] CPU 14
> [ 141.131299] Modules linked in:
> [ 141.131492]
> [ 141.131579] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 141.151543] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 141.171304] RSP: 0000:ffff88103efa3ee8 EFLAGS: 00000246
> [ 141.171553] RAX: 0000000000000000 RBX: ffff88103efa3fd8 RCX: 0000000000000000
> [ 141.191319] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ef9a2b0
> [ 141.211122] RBP: ffff88103efa3ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 141.211430] R10: 0000000000000000 R11: ffff88103efa3ed8 R12: 000000000000000e
> [ 141.231204] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 141.231532] FS: 0000000000000000(0000) GS:ffff88207ea00000(0000) knlGS:0000000000000000
> [ 141.251365] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 141.271218] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 141.271561] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 141.291197] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 141.310971] Process kworker/0:1 (pid: 0, threadinfo ffff88103efa2000, task ffff88103ef9a2b0)
> [ 141.311371] Stack:
> [ 141.330919] ffff88103efa2000 ffffffff82525380 ffff88103efa3f28 ffffffff810391f0
> [ 141.331318] ffff88103efa3f18 0ab959daf09fa942 0000000000000000 0000000000000000
> [ 141.351115] ffff88103efa3f48 ffffffff81c064f3 0000000000000000 4298453ed66e2456
> [ 141.370848] Call Trace:
> [ 141.370981] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 141.371225] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 141.390899] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 141.430647] Call Trace:
> [ 141.430774] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 141.431011] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 141.450711] NMI backtrace for cpu 15
> [ 141.450896] CPU 15
> [ 141.451011] Modules linked in:
> [ 141.451189]
> [ 141.470621] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 141.490630] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 141.491003] RSP: 0000:ffff88103efb1ee8 EFLAGS: 00000246
> [ 141.510686] RAX: 0000000000000000 RBX: ffff88103efb1fd8 RCX: 0000000000000000
> [ 141.510979] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efaa2b0
> [ 141.530758] RBP: ffff88103efb1ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 141.555764] R10: 0000000000000000 R11: ffff88103efb1ed8 R12: 000000000000000f
> [ 141.556182] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 141.570663] FS: 0000000000000000(0000) GS:ffff88207ec00000(0000) knlGS:0000000000000000
> [ 141.590492] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 141.590742] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 141.610514] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 141.630305] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 141.630631] Process kworker/0:1 (pid: 0, threadinfo ffff88103efb0000, task ffff88103efaa2b0)
> [ 141.650438] Stack:
> [ 141.650531] ffff88103efb0000 ffffffff82525380 ffff88103efb1f28 ffffffff810391f0
> [ 141.670404] ffff88103efb1f18 c089ea705be6a98b 0000000000000000 0000000000000000
> [ 141.690174] ffff88103efb1f48 ffffffff81c064f3 0000000000000000 f5949bd6339eff74
> [ 141.690594] Call Trace:
> [ 141.690713] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 141.710299] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 141.710559] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 141.750698] Call Trace:
> [ 141.750815] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 141.770102] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 141.770384] NMI backtrace for cpu 16
> [ 141.789970] CPU 16
> [ 141.790081] Modules linked in:
> [ 141.790289]
> [ 141.790389] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 141.810304] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 141.830083] RSP: 0000:ffff88103efd1ee8 EFLAGS: 00000246
> [ 141.830332] RAX: 0000000000000000 RBX: ffff88103efd1fd8 RCX: 0000000000000000
> [ 141.850103] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efc22b0
> [ 141.869892] RBP: ffff88103efd1ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 141.870237] R10: 0000000000000000 R11: ffff88103efd1ed8 R12: 0000000000000010
> [ 141.889988] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 141.909768] FS: 0000000000000000(0000) GS:ffff88307de00000(0000) knlGS:0000000000000000
> [ 141.910145] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 141.930014] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 141.949891] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 141.950219] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 141.969788] Process kworker/0:1 (pid: 0, threadinfo ffff88103efd0000, task ffff88103efc22b0)
> [ 141.989625] Stack:
> [ 141.989728] ffff88103efd0000 ffffffff82525380 ffff88103efd1f28 ffffffff810391f0
> [ 141.990131] ffff88103efd1f18 d308e5f636b1a8a6 0000000000000000 0000000000000000
> [ 142.010042] ffff88103efd1f48 ffffffff81c064f3 0000000000000000 6ff58ab7fd41784e
> [ 142.029662] Call Trace:
> [ 142.029803] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 142.030037] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 142.049772] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 142.089841] Call Trace:
> [ 142.089956] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 142.090215] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 142.109532] NMI backtrace for cpu 17
> [ 142.109722] CPU 17
> [ 142.109821] Modules linked in:
> [ 142.129537]
> [ 142.129634] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 142.169733] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 142.189467] RSP: 0000:ffff88103efe1ee8 EFLAGS: 00000246
> [ 142.189728] RAX: 0000000000000000 RBX: ffff88103efe1fd8 RCX: 0000000000000000
> [ 142.214907] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efda2b0
> [ 142.215275] RBP: ffff88103efe1ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 142.229406] R10: 0000000000000000 R11: ffff88103efe1ed8 R12: 0000000000000011
> [ 142.249462] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 142.249788] FS: 0000000000000000(0000) GS:ffff88307e000000(0000) knlGS:0000000000000000
> [ 142.279529] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 142.298927] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 142.299266] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 142.319011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 142.319354] Process kworker/0:1 (pid: 0, threadinfo ffff88103efe0000, task ffff88103efda2b0)
> [ 142.339157] Stack:
> [ 142.339263] ffff88103efe0000 ffffffff82525380 ffff88103efe1f28 ffffffff810391f0
> [ 142.359098] ffff88103efe1f18 2cfef7407d13bcdc 0000000000000000 0000000000000000
> [ 142.378864] ffff88103efe1f48 ffffffff81c064f3 0000000000000000 5aef71e9d25d79be
> [ 142.379280] Call Trace:
> [ 142.398788] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 142.399028] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 142.418688] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 142.439455] Call Trace:
> [ 142.439602] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 142.458800] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 142.459070] NMI backtrace for cpu 18
> [ 142.478648] CPU 18
> [ 142.478761] Modules linked in:
> [ 142.478954]
> [ 142.479053] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 142.498981] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 142.518775] RSP: 0000:ffff88103efe7ee8 EFLAGS: 00000246
> [ 142.538455] RAX: 0000000000000000 RBX: ffff88103efe7fd8 RCX: 0000000000000000
> [ 142.538785] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103efea2b0
> [ 142.558556] RBP: ffff88103efe7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 142.578332] R10: 0000000000000000 R11: ffff88103efe7ed8 R12: 0000000000000012
> [ 142.578695] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 142.598439] FS: 0000000000000000(0000) GS:ffff88307e200000(0000) knlGS:0000000000000000
> [ 142.618278] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 142.618556] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 142.638537] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 142.638904] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 142.658418] Process kworker/0:1 (pid: 0, threadinfo ffff88103efe6000, task ffff88103efea2b0)
> [ 142.678265] Stack:
> [ 142.678357] ffff88103efe6000 ffffffff82525380 ffff88103efe7f28 ffffffff810391f0
> [ 142.698172] ffff88103efe7f18 9996aca01738a61f 0000000000000000 0000000000000000
> [ 142.698616] ffff88103efe7f48 ffffffff81c064f3 0000000000000000 40c7708621549c8a
> [ 142.718380] Call Trace:
> [ 142.718506] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 142.738070] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 142.738368] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 142.778351] Call Trace:
> [ 142.778493] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 142.797883] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 142.798156] NMI backtrace for cpu 19
> [ 142.798348] CPU 19
> [ 142.817835] Modules linked in:
> [ 142.818037]
> [ 142.818129] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 142.838089] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 142.862710] RSP: 0000:ffff88103e805ee8 EFLAGS: 00000246
> [ 142.862964] RAX: 0000000000000000 RBX: ffff88103e805fd8 RCX: 0000000000000000
> [ 142.877887] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103effa2b0
> [ 142.878204] RBP: ffff88103e805ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 142.897981] R10: 0000000000000000 R11: ffff88103e805ed8 R12: 0000000000000013
> [ 142.917799] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 142.918130] FS: 0000000000000000(0000) GS:ffff88307e400000(0000) knlGS:0000000000000000
> [ 142.939106] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 142.957676] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 142.958011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 142.977809] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 142.997672] Process kworker/0:1 (pid: 0, threadinfo ffff88103e804000, task ffff88103effa2b0)
> [ 142.998085] Stack:
> [ 143.017695] ffff88103e804000 ffffffff82525380 ffff88103e805f28 ffffffff810391f0
> [ 143.018151] ffff88103e805f18 d4ede08c6a7229e8 0000000000000000 0000000000000000
> [ 143.037816] ffff88103e805f48 ffffffff81c064f3 0000000000000000 0d5fb9e457f7ceed
> [ 143.057424] Call Trace:
> [ 143.057539] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 143.057804] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 143.077411] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 143.098284] Call Trace:
> [ 143.117331] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 143.117603] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 143.137288] NMI backtrace for cpu 20
> [ 143.137473] CPU 20
> [ 143.137573] Modules linked in:
> [ 143.137783]
> [ 143.157190] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 143.177190] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 143.177590] RSP: 0000:ffff88103e825ee8 EFLAGS: 00000246
> [ 143.197215] RAX: 0000000000000000 RBX: ffff88103e825fd8 RCX: 0000000000000000
> [ 143.197561] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e80a2b0
> [ 143.217831] RBP: ffff88103e825ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 143.237318] R10: 0000000000000000 R11: ffff88103e825ed8 R12: 0000000000000014
> [ 143.237658] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 143.257208] FS: 0000000000000000(0000) GS:ffff88307e600000(0000) knlGS:0000000000000000
> [ 143.277028] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 143.277307] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 143.297038] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 143.316856] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 143.317171] Process kworker/0:1 (pid: 0, threadinfo ffff88103e824000, task ffff88103e80a2b0)
> [ 143.337024] Stack:
> [ 143.337115] ffff88103e824000 ffffffff82525380 ffff88103e825f28 ffffffff810391f0
> [ 143.356948] ffff88103e825f18 0744bf4983a87a1b 0000000000000000 0000000000000000
> [ 143.357406] ffff88103e825f48 ffffffff81c064f3 0000000000000000 1cb607dd8301e40b
> [ 143.377130] Call Trace:
> [ 143.377266] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 143.397110] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 143.397386] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 143.437171] Call Trace:
> [ 143.437301] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 143.456669] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 143.456963] NMI backtrace for cpu 21
> [ 143.476513] CPU 21
> [ 143.476611] Modules linked in:
> [ 143.476785]
> [ 143.476890] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 143.497586] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 143.516629] RSP: 0000:ffff88103e833ee8 EFLAGS: 00000246
> [ 143.516873] RAX: 0000000000000000 RBX: ffff88103e833fd8 RCX: 0000000000000000
> [ 143.536620] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e82a2b0
> [ 143.556431] RBP: ffff88103e833ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 143.556789] R10: 0000000000000000 R11: ffff88103e833ed8 R12: 0000000000000015
> [ 143.576579] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 143.596325] FS: 0000000000000000(0000) GS:ffff88307e800000(0000) knlGS:0000000000000000
> [ 143.596690] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 143.616381] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 143.636171] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 143.636502] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 143.656270] Process kworker/0:1 (pid: 0, threadinfo ffff88103e832000, task ffff88103e82a2b0)
> [ 143.676139] Stack:
> [ 143.676268] ffff88103e832000 ffffffff82525380 ffff88103e833f28 ffffffff810391f0
> [ 143.676696] ffff88103e833f18 9272018b68de3072 0000000000000000 0000000000000000
> [ 143.696488] ffff88103e833f48 ffffffff81c064f3 0000000000000000 30c832b73c95166f
> [ 143.716202] Call Trace:
> [ 143.716330] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 143.716569] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 143.736181] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 143.776036] Call Trace:
> [ 143.776163] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 143.776410] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 143.795995] NMI backtrace for cpu 22
> [ 143.796190] CPU 22
> [ 143.796290] Modules linked in:
> [ 143.815859]
> [ 143.815964] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 143.835923] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 143.836331] RSP: 0000:ffff88103e849ee8 EFLAGS: 00000246
> [ 143.855953] RAX: 0000000000000000 RBX: ffff88103e849fd8 RCX: 0000000000000000
> [ 143.875743] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e83a2b0
> [ 143.876075] RBP: ffff88103e849ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 143.895826] R10: 0000000000000000 R11: ffff88103e849ed8 R12: 0000000000000016
> [ 143.915639] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 143.915990] FS: 0000000000000000(0000) GS:ffff88307ea00000(0000) knlGS:0000000000000000
> [ 143.935797] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 143.936056] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 143.955929] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 143.975745] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 143.976088] Process kworker/0:1 (pid: 0, threadinfo ffff88103e848000, task ffff88103e83a2b0)
> [ 143.996164] Stack:
> [ 143.996279] ffff88103e848000 ffffffff82525380 ffff88103e849f28 ffffffff810391f0
> [ 144.015921] ffff88103e849f18 55c4bc74ee10c3a9 0000000000000000 0000000000000000
> [ 144.035620] ffff88103e849f48 ffffffff81c064f3 0000000000000000 c6af0e0725d8e279
> [ 144.036085] Call Trace:
> [ 144.055485] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 144.055735] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 144.075270] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 144.096011] Call Trace:
> [ 144.096144] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 144.115394] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 144.115679] NMI backtrace for cpu 23
> [ 144.135256] CPU 23
> [ 144.135358] Modules linked in:
> [ 144.135545]
> [ 144.135637] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 144.156262] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 144.175570] RSP: 0000:ffff88103e859ee8 EFLAGS: 00000246
> [ 144.195054] RAX: 0000000000000000 RBX: ffff88103e859fd8 RCX: 0000000000000000
> [ 144.195389] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8522b0
> [ 144.215178] RBP: ffff88103e859ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 144.234967] R10: 0000000000000000 R11: ffff88103e859ed8 R12: 0000000000000017
> [ 144.235325] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 144.255061] FS: 0000000000000000(0000) GS:ffff88307ec00000(0000) knlGS:0000000000000000
> [ 144.274868] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 144.275157] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 144.295041] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 144.295381] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 144.315016] Process kworker/0:1 (pid: 0, threadinfo ffff88103e858000, task ffff88103e8522b0)
> [ 144.334863] Stack:
> [ 144.334969] ffff88103e858000 ffffffff82525380 ffff88103e859f28 ffffffff810391f0
> [ 144.354806] ffff88103e859f18 908465f609e5cf69 0000000000000000 0000000000000000
> [ 144.355285] ffff88103e859f48 ffffffff81c064f3 0000000000000000 901034fa9e9275cf
> [ 144.374955] Call Trace:
> [ 144.375070] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 144.394687] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 144.394970] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 144.434889] Call Trace:
> [ 144.435031] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 144.435292] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 144.454765] NMI backtrace for cpu 24
> [ 144.454967] CPU 24
> [ 144.455060] Modules linked in:
> [ 144.474733]
> [ 144.474821] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 144.494676] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 144.514460] RSP: 0000:ffff88103e85fee8 EFLAGS: 00000246
> [ 144.514725] RAX: 0000000000000000 RBX: ffff88103e85ffd8 RCX: 0000000000000000
> [ 144.534483] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e864560
> [ 144.534810] RBP: ffff88103e85fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 144.554590] R10: 0000000000000000 R11: ffff88103e85fed8 R12: 0000000000000018
> [ 144.574373] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 144.574733] FS: 0000000000000000(0000) GS:ffff88407de00000(0000) knlGS:0000000000000000
> [ 144.594545] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 144.614252] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 144.614576] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 144.634345] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 144.654133] Process kworker/0:1 (pid: 0, threadinfo ffff88103e85e000, task ffff88103e864560)
> [ 144.654530] Stack:
> [ 144.674053] ffff88103e85e000 ffffffff82525380 ffff88103e85ff28 ffffffff810391f0
> [ 144.674514] ffff88103e85ff18 39cac7e094841c82 0000000000000000 0000000000000000
> [ 144.694241] ffff88103e85ff48 ffffffff81c064f3 0000000000000000 2dcf15bc6b5f3387
> [ 144.714026] Call Trace:
> [ 144.714173] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 144.714426] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 144.734049] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 144.754866] Call Trace:
> [ 144.773919] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 144.774151] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 144.795051] NMI backtrace for cpu 25
> [ 144.795234] CPU 25
> [ 144.795340] Modules linked in:
> [ 144.795511]
> [ 144.813758] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 144.833771] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 144.834146] RSP: 0000:ffff88103e875ee8 EFLAGS: 00000246
> [ 144.853849] RAX: 0000000000000000 RBX: ffff88103e875fd8 RCX: 0000000000000000
> [ 144.854177] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e884560
> [ 144.873888] RBP: ffff88103e875ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 144.893689] R10: 0000000000000000 R11: ffff88103e875ed8 R12: 0000000000000019
> [ 144.894027] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 144.913789] FS: 0000000000000000(0000) GS:ffff88407e000000(0000) knlGS:0000000000000000
> [ 144.933631] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 144.933898] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 144.953718] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 144.973714] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 144.974058] Process kworker/0:1 (pid: 0, threadinfo ffff88103e874000, task ffff88103e884560)
> [ 144.993800] Stack:
> [ 144.993892] ffff88103e874000 ffffffff82525380 ffff88103e875f28 ffffffff810391f0
> [ 145.013686] ffff88103e875f18 0a3cfc4bd45819c1 0000000000000000 0000000000000000
> [ 145.014135] ffff88103e875f48 ffffffff81c064f3 0000000000000000 6e7bcc21ed33ee47
> [ 145.033735] Call Trace:
> [ 145.033868] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 145.053445] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 145.053733] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 145.093757] Call Trace:
> [ 145.093889] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 145.113231] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 145.113524] NMI backtrace for cpu 26
> [ 145.113697] CPU 26
> [ 145.133220] Modules linked in:
> [ 145.133387]
> [ 145.133489] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 145.153444] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 145.173225] RSP: 0000:ffff88103e893ee8 EFLAGS: 00000246
> [ 145.173498] RAX: 0000000000000000 RBX: ffff88103e893fd8 RCX: 0000000000000000
> [ 145.193256] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e88c560
> [ 145.213024] RBP: ffff88103e893ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 145.213354] R10: 0000000000000000 R11: ffff88103e893ed8 R12: 000000000000001a
> [ 145.233866] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 145.252944] FS: 0000000000000000(0000) GS:ffff88407e200000(0000) knlGS:0000000000000000
> [ 145.253340] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 145.273003] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 145.273344] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 145.293091] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 145.312900] Process kworker/0:1 (pid: 0, threadinfo ffff88103e892000, task ffff88103e88c560)
> [ 145.332710] Stack:
> [ 145.332810] ffff88103e892000 ffffffff82525380 ffff88103e893f28 ffffffff810391f0
> [ 145.333249] ffff88103e893f18 66000d941fa900df 0000000000000000 0000000000000000
> [ 145.353045] ffff88103e893f48 ffffffff81c064f3 0000000000000000 fcf39f4b7ffe7411
> [ 145.372789] Call Trace:
> [ 145.372924] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 145.373167] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 145.413174] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 145.453313] Call Trace:
> [ 145.453445] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 145.453693] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 145.472568] NMI backtrace for cpu 27
> [ 145.472745] CPU 27
> [ 145.472848] Modules linked in:
> [ 145.492451]
> [ 145.492543] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 145.512476] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 145.512900] RSP: 0000:ffff88103e8a9ee8 EFLAGS: 00000246
> [ 145.532517] RAX: 0000000000000000 RBX: ffff88103e8a9fd8 RCX: 0000000000000000
> [ 145.552323] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8a4560
> [ 145.552650] RBP: ffff88103e8a9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 145.572405] R10: 0000000000000000 R11: ffff88103e8a9ed8 R12: 000000000000001b
> [ 145.592190] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 145.592531] FS: 0000000000000000(0000) GS:ffff88407e400000(0000) knlGS:0000000000000000
> [ 145.612347] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 145.612595] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 145.632371] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 145.652160] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 145.652497] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8a8000, task ffff88103e8a4560)
> [ 145.672331] Stack:
> [ 145.672432] ffff88103e8a8000 ffffffff82525380 ffff88103e8a9f28 ffffffff810391f0
> [ 145.692272] ffff88103e8a9f18 df03c4b0aef93378 0000000000000000 0000000000000000
> [ 145.712012] ffff88103e8a9f48 ffffffff81c064f3 0000000000000000 6f15585bec259797
> [ 145.712423] Call Trace:
> [ 145.731931] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 145.732172] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 145.732428] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 145.772555] Call Trace:
> [ 145.772700] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 145.791961] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 145.792260] NMI backtrace for cpu 28
> [ 145.811820] CPU 28
> [ 145.811918] Modules linked in:
> [ 145.812103]
> [ 145.812191] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 145.832145] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 145.851958] RSP: 0000:ffff88103e8dfee8 EFLAGS: 00000246
> [ 145.871617] RAX: 0000000000000000 RBX: ffff88103e8dffd8 RCX: 0000000000000000
> [ 145.871942] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8bc560
> [ 145.891718] RBP: ffff88103e8dfef8 R08: 0000000000000000 R09: 0000000000000000
> [ 145.892044] R10: 0000000000000000 R11: ffff88103e8dfed8 R12: 000000000000001c
> [ 145.911835] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 145.931622] FS: 0000000000000000(0000) GS:ffff88407e600000(0000) knlGS:0000000000000000
> [ 145.931976] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 145.951702] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 145.971542] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 145.971856] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 145.991730] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8de000, task ffff88103e8bc560)
> [ 146.011809] Stack:
> [ 146.011900] ffff88103e8de000 ffffffff82525380 ffff88103e8dff28 ffffffff810391f0
> [ 146.031465] ffff88103e8dff18 413f0ec1ba26191e 0000000000000000 0000000000000000
> [ 146.031947] ffff88103e8dff48 ffffffff81c064f3 0000000000000000 a5eb12480065c24e
> [ 146.051796] Call Trace:
> [ 146.051942] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 146.071240] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 146.071509] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 146.111423] Call Trace:
> [ 146.111559] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 146.111807] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 146.131412] NMI backtrace for cpu 29
> [ 146.131602] CPU 29
> [ 146.131702] Modules linked in:
> [ 146.151356]
> [ 146.151458] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 146.171256] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 146.191027] RSP: 0000:ffff88103e8e7ee8 EFLAGS: 00000246
> [ 146.191302] RAX: 0000000000000000 RBX: ffff88103e8e7fd8 RCX: 0000000000000000
> [ 146.211054] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e8ec560
> [ 146.211390] RBP: ffff88103e8e7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 146.231169] R10: 0000000000000000 R11: ffff88103e8e7ed8 R12: 000000000000001d
> [ 146.250948] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 146.251264] FS: 0000000000000000(0000) GS:ffff88407e800000(0000) knlGS:0000000000000000
> [ 146.271080] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 146.290841] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 146.291181] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 146.311243] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 146.330731] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8e6000, task ffff88103e8ec560)
> [ 146.331132] Stack:
> [ 146.331248] ffff88103e8e6000 ffffffff82525380 ffff88103e8e7f28 ffffffff810391f0
> [ 146.351311] ffff88103e8e7f18 d32ef0dd85d433ad 0000000000000000 0000000000000000
> [ 146.371016] ffff88103e8e7f48 ffffffff81c064f3 0000000000000000 a92aa153b95503a5
> [ 146.371414] Call Trace:
> [ 146.391290] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 146.391561] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 146.410586] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 146.431456] Call Trace:
> [ 146.450599] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 146.450836] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 146.470418] NMI backtrace for cpu 30
> [ 146.470593] CPU 30
> [ 146.470693] Modules linked in:
> [ 146.470901]
> [ 146.470976] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 146.510310] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 146.510726] RSP: 0000:ffff88103e8fdee8 EFLAGS: 00000246
> [ 146.530366] RAX: 0000000000000000 RBX: ffff88103e8fdfd8 RCX: 0000000000000000
> [ 146.530725] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e904560
> [ 146.550485] RBP: ffff88103e8fdef8 R08: 0000000000000000 R09: 0000000000000000
> [ 146.570257] R10: 0000000000000000 R11: ffff88103e8fded8 R12: 000000000000001e
> [ 146.570570] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 146.590387] FS: 0000000000000000(0000) GS:ffff88407ea00000(0000) knlGS:0000000000000000
> [ 146.610177] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 146.610447] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 146.630240] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 146.630576] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 146.650412] Process kworker/0:1 (pid: 0, threadinfo ffff88103e8fc000, task ffff88103e904560)
> [ 146.670189] Stack:
> [ 146.670287] ffff88103e8fc000 ffffffff82525380 ffff88103e8fdf28 ffffffff810391f0
> [ 146.690115] ffff88103e8fdf18 913051ccc440aefd 0000000000000000 0000000000000000
> [ 146.690513] ffff88103e8fdf48 ffffffff81c064f3 0000000000000000 621a96059901cbc3
> [ 146.710278] Call Trace:
> [ 146.710398] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 146.730006] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 146.730287] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 146.770294] Call Trace:
> [ 146.770412] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 146.789833] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 146.790121] NMI backtrace for cpu 32
> [ 146.790346] CPU 32
> [ 146.809763] Modules linked in:
> [ 146.809964]
> [ 146.810054] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 146.830045] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 146.849776] RSP: 0000:ffff88103e933ee8 EFLAGS: 00000246
> [ 146.850020] RAX: 0000000000000000 RBX: ffff88103e933fd8 RCX: 0000000000000000
> [ 146.869805] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e92c560
> [ 146.889563] RBP: ffff88103e933ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 146.889911] R10: 0000000000000000 R11: ffff88103e933ed8 R12: 0000000000000020
> [ 146.909684] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 146.929468] FS: 0000000000000000(0000) GS:ffff88507de00000(0000) knlGS:0000000000000000
> [ 146.929838] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 146.949546] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 146.949896] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 146.971294] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 146.989462] Process kworker/0:1 (pid: 0, threadinfo ffff88103e932000, task ffff88103e92c560)
> [ 146.989840] Stack:
> [ 147.009427] ffff88103e932000 ffffffff82525380 ffff88103e933f28 ffffffff810391f0
> [ 147.009877] ffff88103e933f18 cc93d566d9aab073 0000000000000000 0000000000000000
> [ 147.029800] ffff88103e933f48 ffffffff81c064f3 0000000000000000 0892a3361bfa89c4
> [ 147.049752] Call Trace:
> [ 147.049901] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 147.050162] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 147.069459] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 147.109150] Call Trace:
> [ 147.109282] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 147.109531] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 147.129165] NMI backtrace for cpu 33
> [ 147.129355] CPU 33
> [ 147.129455] Modules linked in:
> [ 147.149006]
> [ 147.149103] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 147.169122] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 147.169515] RSP: 0000:ffff88103e941ee8 EFLAGS: 00000246
> [ 147.189157] RAX: 0000000000000000 RBX: ffff88103e941fd8 RCX: 0000000000000000
> [ 147.208902] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e93c560
> [ 147.209238] RBP: ffff88103e941ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 147.229248] R10: 0000000000000000 R11: ffff88103e941ed8 R12: 0000000000000021
> [ 147.229602] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 147.249097] FS: 0000000000000000(0000) GS:ffff88507e000000(0000) knlGS:0000000000000000
> [ 147.268935] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 147.269201] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 147.289003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 147.308778] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 147.309104] Process kworker/0:1 (pid: 0, threadinfo ffff88103e940000, task ffff88103e93c560)
> [ 147.328926] Stack:
> [ 147.329031] ffff88103e940000 ffffffff82525380 ffff88103e941f28 ffffffff810391f0
> [ 147.348891] ffff88103e941f18 ab385c0f8fb97b87 0000000000000000 0000000000000000
> [ 147.368624] ffff88103e941f48 ffffffff81c064f3 0000000000000000 bbbf54853ffbc8f6
> [ 147.369042] Call Trace:
> [ 147.369173] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 147.390147] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 147.390442] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 147.429144] Call Trace:
> [ 147.429258] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 147.448575] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 147.448889] NMI backtrace for cpu 34
> [ 147.468539] CPU 34
> [ 147.468637] Modules linked in:
> [ 147.468842]
> [ 147.468918] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 147.488774] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 147.508554] RSP: 0000:ffff88103e947ee8 EFLAGS: 00000246
> [ 147.508813] RAX: 0000000000000000 RBX: ffff88103e947fd8 RCX: 0000000000000000
> [ 147.528534] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e954560
> [ 147.548340] RBP: ffff88103e947ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 147.548688] R10: 0000000000000000 R11: ffff88103e947ed8 R12: 0000000000000022
> [ 147.568442] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 147.593543] FS: 0000000000000000(0000) GS:ffff88507e200000(0000) knlGS:0000000000000000
> [ 147.593938] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 147.608329] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 147.628094] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 147.628428] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 147.648223] Process kworker/0:1 (pid: 0, threadinfo ffff88103e946000, task ffff88103e954560)
> [ 147.668048] Stack:
> [ 147.668146] ffff88103e946000 ffffffff82525380 ffff88103e947f28 ffffffff810391f0
> [ 147.687912] ffff88103e947f18 c3e1387ef72955fb 0000000000000000 0000000000000000
> [ 147.688367] ffff88103e947f48 ffffffff81c064f3 0000000000000000 3b02cb3a56ca3450
> [ 147.708128] Call Trace:
> [ 147.708264] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 147.727833] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 147.728107] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 147.768011] Call Trace:
> [ 147.768133] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 147.768364] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 147.787920] NMI backtrace for cpu 35
> [ 147.788127] CPU 35
> [ 147.788220] Modules linked in:
> [ 147.807761]
> [ 147.807868] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 147.827846] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 147.847643] RSP: 0000:ffff88103e95fee8 EFLAGS: 00000246
> [ 147.847920] RAX: 0000000000000000 RBX: ffff88103e95ffd8 RCX: 0000000000000000
> [ 147.867634] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e964560
> [ 147.868005] RBP: ffff88103e95fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 147.887786] R10: 0000000000000000 R11: ffff88103e95fed8 R12: 0000000000000023
> [ 147.907528] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 147.907897] FS: 0000000000000000(0000) GS:ffff88507e400000(0000) knlGS:0000000000000000
> [ 147.927674] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 147.947412] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 147.947790] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 147.967603] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 147.967925] Process kworker/0:1 (pid: 0, threadinfo ffff88103e95e000, task ffff88103e964560)
> [ 147.987807] Stack:
> [ 147.987908] ffff88103e95e000 ffffffff82525380 ffff88103e95ff28 ffffffff810391f0
> [ 148.008237] ffff88103e95ff18 b04a6bcc5d16a033 0000000000000000 0000000000000000
> [ 148.028386] ffff88103e95ff48 ffffffff81c064f3 0000000000000000 86562f64e45d9875
> [ 148.028826] Call Trace:
> [ 148.047569] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 148.047852] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 148.067261] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 148.088104] Call Trace:
> [ 148.107087] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 148.107324] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 148.107595] NMI backtrace for cpu 36
> [ 148.127204] CPU 36
> [ 148.127307] Modules linked in:
> [ 148.127492]
> [ 148.127592] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 148.147493] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 148.167294] RSP: 0000:ffff88103e96dee8 EFLAGS: 00000246
> [ 148.187190] RAX: 0000000000000000 RBX: ffff88103e96dfd8 RCX: 0000000000000000
> [ 148.187541] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e974560
> [ 148.207048] RBP: ffff88103e96def8 R08: 0000000000000000 R09: 0000000000000000
> [ 148.226862] R10: 0000000000000000 R11: ffff88103e96ded8 R12: 0000000000000024
> [ 148.227187] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 148.246985] FS: 0000000000000000(0000) GS:ffff88507e600000(0000) knlGS:0000000000000000
> [ 148.266789] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 148.267065] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 148.286836] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 148.287182] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 148.306949] Process kworker/0:1 (pid: 0, threadinfo ffff88103e96c000, task ffff88103e974560)
> [ 148.326800] Stack:
> [ 148.326914] ffff88103e96c000 ffffffff82525380 ffff88103e96df28 ffffffff810391f0
> [ 148.346724] ffff88103e96df18 acf4d12aa3bac806 0000000000000000 0000000000000000
> [ 148.347157] ffff88103e96df48 ffffffff81c064f3 0000000000000000 f751ed781526a858
> [ 148.366914] Call Trace:
> [ 148.367029] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 148.386641] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 148.386909] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 148.427152] Call Trace:
> [ 148.427274] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 148.446410] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 148.446654] NMI backtrace for cpu 37
> [ 148.446833] CPU 37
> [ 148.466376] Modules linked in:
> [ 148.466589]
> [ 148.466691] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 148.486626] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 148.506390] RSP: 0000:ffff88103e993ee8 EFLAGS: 00000246
> [ 148.506644] RAX: 0000000000000000 RBX: ffff88103e993fd8 RCX: 0000000000000000
> [ 148.526396] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e98c560
> [ 148.546191] RBP: ffff88103e993ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 148.546529] R10: 0000000000000000 R11: ffff88103e993ed8 R12: 0000000000000025
> [ 148.566297] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 148.566608] FS: 0000000000000000(0000) GS:ffff88507e800000(0000) knlGS:0000000000000000
> [ 148.586422] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 148.606177] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 148.606523] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 148.626281] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 148.646040] Process kworker/0:1 (pid: 0, threadinfo ffff88103e992000, task ffff88103e98c560)
> [ 148.646449] Stack:
> [ 148.670787] ffff88103e992000 ffffffff82525380 ffff88103e993f28 ffffffff810391f0
> [ 148.671236] ffff88103e993f18 fb85c5af419ff264 0000000000000000 0000000000000000
> [ 148.686415] ffff88103e993f48 ffffffff81c064f3 0000000000000000 9331422f9e8ccc3d
> [ 148.705894] Call Trace:
> [ 148.706023] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 148.706270] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 148.725959] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 148.746866] Call Trace:
> [ 148.765852] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 148.766124] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 148.785780] NMI backtrace for cpu 40
> [ 148.785966] CPU 40
> [ 148.786077] Modules linked in:
> [ 148.786246]
> [ 148.805683] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 148.825696] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 148.826070] RSP: 0000:ffff88103e9bdee8 EFLAGS: 00000246
> [ 148.845734] RAX: 0000000000000000 RBX: ffff88103e9bdfd8 RCX: 0000000000000000
> [ 148.846090] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9c8000
> [ 148.865826] RBP: ffff88103e9bdef8 R08: 0000000000000000 R09: 0000000000000000
> [ 148.885628] R10: 0000000000000000 R11: ffff88103e9bded8 R12: 0000000000000028
> [ 148.885953] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 148.905723] FS: 0000000000000000(0000) GS:ffff88607de00000(0000) knlGS:0000000000000000
> [ 148.925555] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 148.925826] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 148.945565] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 148.965384] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 148.965728] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9bc000, task ffff88103e9c8000)
> [ 148.985555] Stack:
> [ 148.985648] ffff88103e9bc000 ffffffff82525380 ffff88103e9bdf28 ffffffff810391f0
> [ 149.005650] ffff88103e9bdf18 8884672454ef25f4 0000000000000000 0000000000000000
> [ 149.025515] ffff88103e9bdf48 ffffffff81c064f3 0000000000000000 c3c2f6eeaf432544
> [ 149.025926] Call Trace:
> [ 149.026053] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 149.045493] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 149.045764] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 149.085854] Call Trace:
> [ 149.085976] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 149.105177] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 149.105478] NMI backtrace for cpu 42
> [ 149.125059] CPU 42
> [ 149.125164] Modules linked in:
> [ 149.125383]
> [ 149.125460] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 149.145540] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 149.165163] RSP: 0000:ffff88103e9ebee8 EFLAGS: 00000246
> [ 149.165452] RAX: 0000000000000000 RBX: ffff88103e9ebfd8 RCX: 0000000000000000
> [ 149.185144] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9f0000
> [ 149.204974] RBP: ffff88103e9ebef8 R08: 0000000000000000 R09: 0000000000000000
> [ 149.205299] R10: 0000000000000000 R11: ffff88103e9ebed8 R12: 000000000000002a
> [ 149.225235] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 149.244814] FS: 0000000000000000(0000) GS:ffff88607e200000(0000) knlGS:0000000000000000
> [ 149.245197] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 149.265453] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 149.284690] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 149.285023] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 149.304772] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9ea000, task ffff88103e9f0000)
> [ 149.325516] Stack:
> [ 149.325619] ffff88103e9ea000 ffffffff82525380 ffff88103e9ebf28 ffffffff810391f0
> [ 149.326088] ffff88103e9ebf18 b7850adb4e3fa3f3 0000000000000000 0000000000000000
> [ 149.344913] ffff88103e9ebf48 ffffffff81c064f3 0000000000000000 cba24f187a031774
> [ 149.364697] Call Trace:
> [ 149.364828] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 149.365069] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 149.384751] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 149.424582] Call Trace:
> [ 149.424714] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 149.424975] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 149.444530] NMI backtrace for cpu 41
> [ 149.444721] CPU 41
> [ 149.444843] Modules linked in:
> [ 149.464363]
> [ 149.464455] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 149.484442] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 149.504229] RSP: 0000:ffff88103e9d5ee8 EFLAGS: 00000246
> [ 149.504490] RAX: 0000000000000000 RBX: ffff88103e9d5fd8 RCX: 0000000000000000
> [ 149.524276] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9e0000
> [ 149.524620] RBP: ffff88103e9d5ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 149.544358] R10: 0000000000000000 R11: ffff88103e9d5ed8 R12: 0000000000000029
> [ 149.564177] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 149.564512] FS: 0000000000000000(0000) GS:ffff88607e000000(0000) knlGS:0000000000000000
> [ 149.584333] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 149.584601] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 149.604398] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 149.624129] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 149.624494] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9d4000, task ffff88103e9e0000)
> [ 149.644334] Stack:
> [ 149.644449] ffff88103e9d4000 ffffffff82525380 ffff88103e9d5f28 ffffffff810391f0
> [ 149.664249] ffff88103e9d5f18 f21ec8d6397b0a89 0000000000000000 0000000000000000
> [ 149.684023] ffff88103e9d5f48 ffffffff81c064f3 0000000000000000 37d0a573f1bb7502
> [ 149.684455] Call Trace:
> [ 149.703863] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 149.704114] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 149.723783] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 149.744613] Call Trace:
> [ 149.744743] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 149.763901] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 149.764220] NMI backtrace for cpu 43
> [ 149.783770] CPU 43
> [ 149.783882] Modules linked in:
> [ 149.784092]
> [ 149.784184] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 149.804123] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 149.823907] RSP: 0000:ffff88103ea11ee8 EFLAGS: 00000246
> [ 149.843590] RAX: 0000000000000000 RBX: ffff88103ea11fd8 RCX: 0000000000000000
> [ 149.843928] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea08000
> [ 149.863690] RBP: ffff88103ea11ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 149.883452] R10: 0000000000000000 R11: ffff88103ea11ed8 R12: 000000000000002b
> [ 149.883791] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 149.903583] FS: 0000000000000000(0000) GS:ffff88607e400000(0000) knlGS:0000000000000000
> [ 149.923406] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 149.923672] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 149.943429] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 149.943764] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 149.964167] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea10000, task ffff88103ea08000)
> [ 149.983386] Stack:
> [ 149.983501] ffff88103ea10000 ffffffff82525380 ffff88103ea11f28 ffffffff810391f0
> [ 150.003322] ffff88103ea11f18 427e6af2afeb849d 0000000000000000 0000000000000000
> [ 150.003762] ffff88103ea11f48 ffffffff81c064f3 0000000000000000 2a7ca1ca1d9c5d76
> [ 150.023654] Call Trace:
> [ 150.023807] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 150.043573] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 150.043854] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 150.083583] Call Trace:
> [ 150.083715] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 150.103023] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 150.103305] NMI backtrace for cpu 31
> [ 150.103487] CPU 31
> [ 150.122956] Modules linked in:
> [ 150.123144]
> [ 150.123237] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 150.143189] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 150.163215] RSP: 0000:ffff88103e913ee8 EFLAGS: 00000246
> [ 150.163475] RAX: 0000000000000000 RBX: ffff88103e913fd8 RCX: 0000000000000000
> [ 150.183032] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e90c560
> [ 150.183340] RBP: ffff88103e913ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 150.203113] R10: 0000000000000000 R11: ffff88103e913ed8 R12: 000000000000001f
> [ 150.222906] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 150.223225] FS: 0000000000000000(0000) GS:ffff88407ec00000(0000) knlGS:0000000000000000
> [ 150.243031] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 150.262763] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 150.263096] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 150.282880] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 150.302627] Process kworker/0:1 (pid: 0, threadinfo ffff88103e912000, task ffff88103e90c560)
> [ 150.303003] Stack:
> [ 150.322564] ffff88103e912000 ffffffff82525380 ffff88103e913f28 ffffffff810391f0
> [ 150.322981] ffff88103e913f18 36e0850fd202b94f 0000000000000000 0000000000000000
> [ 150.342905] ffff88103e913f48 ffffffff81c064f3 0000000000000000 23640462fccc8760
> [ 150.362518] Call Trace:
> [ 150.362656] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 150.362927] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 150.382583] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 150.403742] Call Trace:
> [ 150.423046] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 150.423294] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 150.442462] NMI backtrace for cpu 48
> [ 150.442655] CPU 48
> [ 150.442756] Modules linked in:
> [ 150.442931]
> [ 150.462450] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 150.482459] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 150.482837] RSP: 0000:ffff88103ea6bee8 EFLAGS: 00000246
> [ 150.502491] RAX: 0000000000000000 RBX: ffff88103ea6bfd8 RCX: 0000000000000000
> [ 150.502796] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea70000
> [ 150.522451] RBP: ffff88103ea6bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 150.542236] R10: 0000000000000000 R11: ffff88103ea6bed8 R12: 0000000000000030
> [ 150.542541] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 150.562303] FS: 0000000000000000(0000) GS:ffff88707de00000(0000) knlGS:0000000000000000
> [ 150.582145] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 150.582394] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 150.603461] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 150.621958] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 150.622276] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea6a000, task ffff88103ea70000)
> [ 150.642103] Stack:
> [ 150.642199] ffff88103ea6a000 ffffffff82525380 ffff88103ea6bf28 ffffffff810391f0
> [ 150.662184] ffff88103ea6bf18 0978fc6520c78880 0000000000000000 0000000000000000
> [ 150.662561] ffff88103ea6bf48 ffffffff81c064f3 0000000000000000 9353423d8473ae89
> [ 150.682209] Call Trace:
> [ 150.682348] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 150.701940] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 150.702196] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 150.742272] Call Trace:
> [ 150.742395] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 150.761756] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 150.762009] NMI backtrace for cpu 49
> [ 150.781726] CPU 49
> [ 150.781824] Modules linked in:
> [ 150.781995]
> [ 150.782082] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 150.801931] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 150.821720] RSP: 0000:ffff88103ea89ee8 EFLAGS: 00000246
> [ 150.821962] RAX: 0000000000000000 RBX: ffff88103ea89fd8 RCX: 0000000000000000
> [ 150.841744] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea80000
> [ 150.861530] RBP: ffff88103ea89ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 150.861832] R10: 0000000000000000 R11: ffff88103ea89ed8 R12: 0000000000000031
> [ 150.881622] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 150.901428] FS: 0000000000000000(0000) GS:ffff88707e000000(0000) knlGS:0000000000000000
> [ 150.901777] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 150.921487] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 150.941299] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 150.941625] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 150.961397] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea88000, task ffff88103ea80000)
> [ 150.981223] Stack:
> [ 150.981324] ffff88103ea88000 ffffffff82525380 ffff88103ea89f28 ffffffff810391f0
> [ 150.981726] ffff88103ea89f18 951b3f639aa8d187 0000000000000000 0000000000000000
> [ 151.002912] ffff88103ea89f48 ffffffff81c064f3 0000000000000000 604a3900ef2607fb
> [ 151.021327] Call Trace:
> [ 151.021454] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 151.021673] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 151.041382] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 151.081354] Call Trace:
> [ 151.081467] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 151.081698] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 151.101243] NMI backtrace for cpu 50
> [ 151.101407] CPU 50
> [ 151.101497] Modules linked in:
> [ 151.120977]
> [ 151.121058] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 151.141171] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 151.141532] RSP: 0000:ffff88103ea8fee8 EFLAGS: 00000246
> [ 151.161045] RAX: 0000000000000000 RBX: ffff88103ea8ffd8 RCX: 0000000000000000
> [ 151.180842] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea98000
> [ 151.181186] RBP: ffff88103ea8fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 151.200975] R10: 0000000000000000 R11: ffff88103ea8fed8 R12: 0000000000000032
> [ 151.220978] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 151.221291] FS: 0000000000000000(0000) GS:ffff88707e200000(0000) knlGS:0000000000000000
> [ 151.241589] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 151.241838] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 151.260913] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 151.280708] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 151.281016] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea8e000, task ffff88103ea98000)
> [ 151.300848] Stack:
> [ 151.300944] ffff88103ea8e000 ffffffff82525380 ffff88103ea8ff28 ffffffff810391f0
> [ 151.320844] ffff88103ea8ff18 13948b935e8bb551 0000000000000000 0000000000000000
> [ 151.340727] ffff88103ea8ff48 ffffffff81c064f3 0000000000000000 799b066110ec0e64
> [ 151.341115] Call Trace:
> [ 151.360457] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 151.360681] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 151.380646] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 151.401120] Call Trace:
> [ 151.401253] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 151.420617] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 151.420870] NMI backtrace for cpu 56
> [ 151.440510] CPU 56
> [ 151.440610] Modules linked in:
> [ 151.440800]
> [ 151.440876] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 151.460859] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 151.480498] RSP: 0000:ffff88103eb05ee8 EFLAGS: 00000246
> [ 151.500180] RAX: 0000000000000000 RBX: ffff88103eb05fd8 RCX: 0000000000000000
> [ 151.500497] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eafc560
> [ 151.520264] RBP: ffff88103eb05ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 151.520575] R10: 0000000000000000 R11: ffff88103eb05ed8 R12: 0000000000000038
> [ 151.540745] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 151.560192] FS: 0000000000000000(0000) GS:ffff88807d800000(0000) knlGS:0000000000000000
> [ 151.580000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 151.580255] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 151.600026] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 151.600345] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 151.620157] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb04000, task ffff88103eafc560)
> [ 151.640138] Stack:
> [ 151.640244] ffff88103eb04000 ffffffff82525380 ffff88103eb05f28 ffffffff810391f0
> [ 151.659894] ffff88103eb05f18 1f33aead7ea68430 0000000000000000 0000000000000000
> [ 151.660313] ffff88103eb05f48 ffffffff81c064f3 0000000000000000 595592b75fa60d89
> [ 151.680074] Call Trace:
> [ 151.680190] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 151.699819] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 151.700082] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 151.740150] Call Trace:
> [ 151.740272] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 151.740513] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 151.759862] NMI backtrace for cpu 47
> [ 151.760045] CPU 47
> [ 151.760158] Modules linked in:
> [ 151.779745]
> [ 151.779852] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 151.799837] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 151.819578] RSP: 0000:ffff88103ea53ee8 EFLAGS: 00000246
> [ 151.819830] RAX: 0000000000000000 RBX: ffff88103ea53fd8 RCX: 0000000000000000
> [ 151.839604] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea58000
> [ 151.839946] RBP: ffff88103ea53ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 151.859711] R10: 0000000000000000 R11: ffff88103ea53ed8 R12: 000000000000002f
> [ 151.879497] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 151.879830] FS: 0000000000000000(0000) GS:ffff88607ec00000(0000) knlGS:0000000000000000
> [ 151.900927] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 151.919469] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 151.919823] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 151.939479] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 151.959236] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea52000, task ffff88103ea58000)
> [ 151.959634] Stack:
> [ 151.979194] ffff88103ea52000 ffffffff82525380 ffff88103ea53f28 ffffffff810391f0
> [ 151.979671] ffff88103ea53f18 50617d206f7b41f7 0000000000000000 0000000000000000
> [ 151.999370] ffff88103ea53f48 ffffffff81c064f3 0000000000000000 036c0faac09abc9f
> [ 152.019268] Call Trace:
> [ 152.019426] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 152.019671] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 152.039785] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 152.060107] Call Trace:
> [ 152.079243] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 152.079515] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 152.099144] NMI backtrace for cpu 57
> [ 152.099326] CPU 57
> [ 152.099427] Modules linked in:
> [ 152.099600]
> [ 152.099676] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 152.138887] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 152.139266] RSP: 0000:ffff88103eb23ee8 EFLAGS: 00000246
> [ 152.158951] RAX: 0000000000000000 RBX: ffff88103eb23fd8 RCX: 0000000000000000
> [ 152.159318] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb0a2b0
> [ 152.179048] RBP: ffff88103eb23ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 152.198914] R10: 0000000000000000 R11: ffff88103eb23ed8 R12: 0000000000000039
> [ 152.199247] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 152.218930] FS: 0000000000000000(0000) GS:ffff88807da00000(0000) knlGS:0000000000000000
> [ 152.238749] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 152.239019] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 152.258793] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 152.278574] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 152.278911] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb22000, task ffff88103eb0a2b0)
> [ 152.298711] Stack:
> [ 152.298819] ffff88103eb22000 ffffffff82525380 ffff88103eb23f28 ffffffff810391f0
> [ 152.318635] ffff88103eb23f18 cfa00dc14662a13f 0000000000000000 0000000000000000
> [ 152.319037] ffff88103eb23f48 ffffffff81c064f3 0000000000000000 7b7fb8c1286f7e8a
> [ 152.338824] Call Trace:
> [ 152.338960] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 152.358541] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 152.358800] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 152.398851] Call Trace:
> [ 152.398966] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 152.418372] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 152.418640] NMI backtrace for cpu 44
> [ 152.418833] CPU 44
> [ 152.438326] Modules linked in:
> [ 152.438524]
> [ 152.438622] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 152.458574] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 152.478339] RSP: 0000:ffff88103ea17ee8 EFLAGS: 00000246
> [ 152.478586] RAX: 0000000000000000 RBX: ffff88103ea17fd8 RCX: 0000000000000000
> [ 152.498355] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea18000
> [ 152.518562] RBP: ffff88103ea17ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 152.518939] R10: 0000000000000000 R11: ffff88103ea17ed8 R12: 000000000000002c
> [ 152.539224] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 152.558012] FS: 0000000000000000(0000) GS:ffff88607e600000(0000) knlGS:0000000000000000
> [ 152.558383] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 152.578111] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 152.578441] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 152.598218] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 152.618018] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea16000, task ffff88103ea18000)
> [ 152.637851] Stack:
> [ 152.637974] ffff88103ea16000 ffffffff82525380 ffff88103ea17f28 ffffffff810391f0
> [ 152.638402] ffff88103ea17f18 dc4e2695c0884d50 0000000000000000 0000000000000000
> [ 152.658134] ffff88103ea17f48 ffffffff81c064f3 0000000000000000 7517c64c020bf9bd
> [ 152.677905] Call Trace:
> [ 152.678068] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 152.678326] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 152.697921] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 152.737725] Call Trace:
> [ 152.737867] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 152.738091] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 152.757760] NMI backtrace for cpu 52
> [ 152.757934] CPU 52
> [ 152.758025] Modules linked in:
> [ 152.777587]
> [ 152.777683] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 152.797670] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 152.798037] RSP: 0000:ffff88103eab5ee8 EFLAGS: 00000246
> [ 152.817677] RAX: 0000000000000000 RBX: ffff88103eab5fd8 RCX: 0000000000000000
> [ 152.837468] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eac0000
> [ 152.837771] RBP: ffff88103eab5ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 152.857537] R10: 0000000000000000 R11: ffff88103eab5ed8 R12: 0000000000000034
> [ 152.877346] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 152.877674] FS: 0000000000000000(0000) GS:ffff88707e600000(0000) knlGS:0000000000000000
> [ 152.897481] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 152.897733] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 152.917526] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 152.937326] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 152.937635] Process kworker/0:1 (pid: 0, threadinfo ffff88103eab4000, task ffff88103eac0000)
> [ 152.957451] Stack:
> [ 152.957547] ffff88103eab4000 ffffffff82525380 ffff88103eab5f28 ffffffff810391f0
> [ 152.977438] ffff88103eab5f18 930c2eb4f563a0aa 0000000000000000 0000000000000000
> [ 152.997187] ffff88103eab5f48 ffffffff81c064f3 0000000000000000 19e27f03b5b56431
> [ 152.997579] Call Trace:
> [ 153.017117] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 153.017338] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 153.017600] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 153.057985] Call Trace:
> [ 153.058104] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 153.077322] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 153.077582] NMI backtrace for cpu 51
> [ 153.097261] CPU 51
> [ 153.097354] Modules linked in:
> [ 153.097541]
> [ 153.097620] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 153.117429] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 153.137096] RSP: 0000:ffff88103eaa5ee8 EFLAGS: 00000246
> [ 153.156777] RAX: 0000000000000000 RBX: ffff88103eaa5fd8 RCX: 0000000000000000
> [ 153.157085] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eaa8000
> [ 153.178037] RBP: ffff88103eaa5ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 153.178355] R10: 0000000000000000 R11: ffff88103eaa5ed8 R12: 0000000000000033
> [ 153.196963] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 153.216960] FS: 0000000000000000(0000) GS:ffff88707e400000(0000) knlGS:0000000000000000
> [ 153.217319] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 153.236846] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 153.256641] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 153.256962] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 153.276744] Process kworker/0:1 (pid: 0, threadinfo ffff88103eaa4000, task ffff88103eaa8000)
> [ 153.297121] Stack:
> [ 153.297217] ffff88103eaa4000 ffffffff82525380 ffff88103eaa5f28 ffffffff810391f0
> [ 153.316494] ffff88103eaa5f18 6c5150727d6760c1 0000000000000000 0000000000000000
> [ 153.316903] ffff88103eaa5f48 ffffffff81c064f3 0000000000000000 a26eef57dafda858
> [ 153.336663] Call Trace:
> [ 153.336780] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 153.356413] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 153.356674] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 153.396587] Call Trace:
> [ 153.396710] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 153.396928] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 153.416571] NMI backtrace for cpu 59
> [ 153.416754] CPU 59
> [ 153.416860] Modules linked in:
> [ 153.436338]
> [ 153.436432] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 153.456387] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 153.476180] RSP: 0000:ffff88103eb49ee8 EFLAGS: 00000246
> [ 153.476420] RAX: 0000000000000000 RBX: ffff88103eb49fd8 RCX: 0000000000000000
> [ 153.496211] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb3a2b0
> [ 153.496537] RBP: ffff88103eb49ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 153.516302] R10: 0000000000000000 R11: ffff88103eb49ed8 R12: 000000000000003b
> [ 153.536107] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 153.536425] FS: 0000000000000000(0000) GS:ffff88807de00000(0000) knlGS:0000000000000000
> [ 153.556227] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 153.575968] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 153.576275] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 153.596051] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 153.615844] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb48000, task ffff88103eb3a2b0)
> [ 153.616225] Stack:
> [ 153.616318] ffff88103eb48000 ffffffff82525380 ffff88103eb49f28 ffffffff810391f0
> [ 153.636195] ffff88103eb49f18 faa21ce5a923a46e 0000000000000000 0000000000000000
> [ 153.656034] ffff88103eb49f48 ffffffff81c064f3 0000000000000000 37ef54861a9d44d1
> [ 153.656443] Call Trace:
> [ 153.675826] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 153.676062] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 153.695766] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 153.716540] Call Trace:
> [ 153.735645] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 153.735886] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 153.755566] NMI backtrace for cpu 38
> [ 153.755747] CPU 38
> [ 153.755873] Modules linked in:
> [ 153.756045]
> [ 153.756125] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 153.776086] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 153.796312] RSP: 0000:ffff88103e9a1ee8 EFLAGS: 00000246
> [ 153.816517] RAX: 0000000000000000 RBX: ffff88103e9a1fd8 RCX: 0000000000000000
> [ 153.816833] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e99c560
> [ 153.835672] RBP: ffff88103e9a1ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 153.855425] R10: 0000000000000000 R11: ffff88103e9a1ed8 R12: 0000000000000026
> [ 153.855748] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 153.875542] FS: 0000000000000000(0000) GS:ffff88507ea00000(0000) knlGS:0000000000000000
> [ 153.895361] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 153.895631] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 153.915395] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 153.915726] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 153.935522] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9a0000, task ffff88103e99c560)
> [ 153.955341] Stack:
> [ 153.955447] ffff88103e9a0000 ffffffff82525380 ffff88103e9a1f28 ffffffff810391f0
> [ 153.975253] ffff88103e9a1f18 9647bc97204c30cb 0000000000000000 0000000000000000
> [ 153.975691] ffff88103e9a1f48 ffffffff81c064f3 0000000000000000 dced99c97795ba22
> [ 153.995441] Call Trace:
> [ 153.995561] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 154.015155] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 154.015436] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 154.055760] Call Trace:
> [ 154.055869] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 154.075315] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 154.075587] NMI backtrace for cpu 53
> [ 154.075758] CPU 53
> [ 154.095150] Modules linked in:
> [ 154.095325]
> [ 154.095412] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 154.115299] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 154.135163] RSP: 0000:ffff88103ead3ee8 EFLAGS: 00000246
> [ 154.135400] RAX: 0000000000000000 RBX: ffff88103ead3fd8 RCX: 0000000000000000
> [ 154.154947] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ead8000
> [ 154.174742] RBP: ffff88103ead3ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 154.175055] R10: 0000000000000000 R11: ffff88103ead3ed8 R12: 0000000000000035
> [ 154.194870] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 154.214621] FS: 0000000000000000(0000) GS:ffff88707e800000(0000) knlGS:0000000000000000
> [ 154.214972] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 154.234707] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 154.235015] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 154.254816] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 154.274588] Process kworker/0:1 (pid: 0, threadinfo ffff88103ead2000, task ffff88103ead8000)
> [ 154.274966] Stack:
> [ 154.294531] ffff88103ead2000 ffffffff82525380 ffff88103ead3f28 ffffffff810391f0
> [ 154.294930] ffff88103ead3f18 4ad8b2f69adc9d19 0000000000000000 0000000000000000
> [ 154.314743] ffff88103ead3f48 ffffffff81c064f3 0000000000000000 39135bbcd7b3444d
> [ 154.334507] Call Trace:
> [ 154.334634] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 154.334855] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 154.354483] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 154.394279] Call Trace:
> [ 154.394408] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 154.394632] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 154.414308] NMI backtrace for cpu 39
> [ 154.414511] CPU 39
> [ 154.414612] Modules linked in:
> [ 154.414789]
> [ 154.434266] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 154.454256] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 154.454639] RSP: 0000:ffff88103e9a7ee8 EFLAGS: 00000246
> [ 154.475040] RAX: 0000000000000000 RBX: ffff88103e9a7fd8 RCX: 0000000000000000
> [ 154.494068] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e9b4560
> [ 154.494399] RBP: ffff88103e9a7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 154.514242] R10: 0000000000000000 R11: ffff88103e9a7ed8 R12: 0000000000000027
> [ 154.514578] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 154.534415] FS: 0000000000000000(0000) GS:ffff88507ec00000(0000) knlGS:0000000000000000
> [ 154.554298] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 154.554583] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 154.574564] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 154.594101] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 154.594440] Process kworker/0:1 (pid: 0, threadinfo ffff88103e9a6000, task ffff88103e9b4560)
> [ 154.614406] Stack:
> [ 154.614512] ffff88103e9a6000 ffffffff82525380 ffff88103e9a7f28 ffffffff810391f0
> [ 154.634084] ffff88103e9a7f18 38d6f42eba77c3bb 0000000000000000 0000000000000000
> [ 154.653780] ffff88103e9a7f48 ffffffff81c064f3 0000000000000000 755b949bb9c12001
> [ 154.654233] Call Trace:
> [ 154.654356] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 154.674028] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 154.674284] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 154.714309] Call Trace:
> [ 154.714444] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 154.733725] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 154.733995] NMI backtrace for cpu 45
> [ 154.753592] CPU 45
> [ 154.753695] Modules linked in:
> [ 154.753892]
> [ 154.753990] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 154.773908] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 154.793770] RSP: 0000:ffff88103ea2dee8 EFLAGS: 00000246
> [ 154.794049] RAX: 0000000000000000 RBX: ffff88103ea2dfd8 RCX: 0000000000000000
> [ 154.813735] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea30000
> [ 154.833523] RBP: ffff88103ea2def8 R08: 0000000000000000 R09: 0000000000000000
> [ 154.833866] R10: 0000000000000000 R11: ffff88103ea2ded8 R12: 000000000000002d
> [ 154.853602] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 154.873380] FS: 0000000000000000(0000) GS:ffff88607e800000(0000) knlGS:0000000000000000
> [ 154.873755] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 154.893466] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 154.913330] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 154.913678] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 154.933355] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea2c000, task ffff88103ea30000)
> [ 154.953193] Stack:
> [ 154.953286] ffff88103ea2c000 ffffffff82525380 ffff88103ea2df28 ffffffff810391f0
> [ 154.953734] ffff88103ea2df18 6270779be7160d7f 0000000000000000 0000000000000000
> [ 154.973602] ffff88103ea2df48 ffffffff81c064f3 0000000000000000 3dd00b8e57918ee7
> [ 154.993277] Call Trace:
> [ 154.993418] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 155.012990] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 155.013263] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 155.053196] Call Trace:
> [ 155.053329] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 155.053555] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 155.073623] NMI backtrace for cpu 58
> [ 155.073793] CPU 58
> [ 155.073885] Modules linked in:
> [ 155.093438]
> [ 155.093532] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 155.114036] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 155.133133] RSP: 0000:ffff88103eb33ee8 EFLAGS: 00000246
> [ 155.133378] RAX: 0000000000000000 RBX: ffff88103eb33fd8 RCX: 0000000000000000
> [ 155.152847] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb2a2b0
> [ 155.153187] RBP: ffff88103eb33ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 155.172903] R10: 0000000000000000 R11: ffff88103eb33ed8 R12: 000000000000003a
> [ 155.192709] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 155.193015] FS: 0000000000000000(0000) GS:ffff88807dc00000(0000) knlGS:0000000000000000
> [ 155.213094] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 155.232578] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 155.232902] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 155.252653] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 155.252979] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb32000, task ffff88103eb2a2b0)
> [ 155.272834] Stack:
> [ 155.272930] ffff88103eb32000 ffffffff82525380 ffff88103eb33f28 ffffffff810391f0
> [ 155.292791] ffff88103eb33f18 3ec07093f886a24c 0000000000000000 0000000000000000
> [ 155.312567] ffff88103eb33f48 ffffffff81c064f3 0000000000000000 51920f6bafeca5f6
> [ 155.312947] Call Trace:
> [ 155.334676] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 155.334930] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 155.352495] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 155.373375] Call Trace:
> [ 155.402529] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 155.402802] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 155.403078] NMI backtrace for cpu 46
> [ 155.422302] CPU 46
> [ 155.422405] Modules linked in:
> [ 155.422613]
> [ 155.422700] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 155.442647] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 155.462457] RSP: 0000:ffff88103ea43ee8 EFLAGS: 00000246
> [ 155.482116] RAX: 0000000000000000 RBX: ffff88103ea43fd8 RCX: 0000000000000000
> [ 155.482467] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ea48000
> [ 155.502221] RBP: ffff88103ea43ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 155.521992] R10: 0000000000000000 R11: ffff88103ea43ed8 R12: 000000000000002e
> [ 155.522330] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 155.542121] FS: 0000000000000000(0000) GS:ffff88607ea00000(0000) knlGS:0000000000000000
> [ 155.561929] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 155.562187] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 155.581979] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 155.582327] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 155.602060] Process kworker/0:1 (pid: 0, threadinfo ffff88103ea42000, task ffff88103ea48000)
> [ 155.621940] Stack:
> [ 155.622035] ffff88103ea42000 ffffffff82525380 ffff88103ea43f28 ffffffff810391f0
> [ 155.641874] ffff88103ea43f18 703d8aaf380a16b7 0000000000000000 0000000000000000
> [ 155.642280] ffff88103ea43f48 ffffffff81c064f3 0000000000000000 378e401c49b042c7
> [ 155.662025] Call Trace:
> [ 155.662161] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 155.681724] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 155.681979] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 155.721995] Call Trace:
> [ 155.722115] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 155.741535] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 155.741804] NMI backtrace for cpu 60
> [ 155.742016] CPU 60
> [ 155.762251] Modules linked in:
> [ 155.762427]
> [ 155.762516] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 155.781721] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 155.801501] RSP: 0000:ffff88103eb4fee8 EFLAGS: 00000246
> [ 155.801736] RAX: 0000000000000000 RBX: ffff88103eb4ffd8 RCX: 0000000000000000
> [ 155.821550] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb522b0
> [ 155.841322] RBP: ffff88103eb4fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 155.841636] R10: 0000000000000000 R11: ffff88103eb4fed8 R12: 000000000000003c
> [ 155.861411] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 155.861738] FS: 0000000000000000(0000) GS:ffff88807e000000(0000) knlGS:0000000000000000
> [ 155.881570] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 155.901283] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 155.901605] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 155.921374] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 155.941182] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb4e000, task ffff88103eb522b0)
> [ 155.941553] Stack:
> [ 155.961097] ffff88103eb4e000 ffffffff82525380 ffff88103eb4ff28 ffffffff810391f0
> [ 155.961503] ffff88103eb4ff18 e93585d02d707a09 0000000000000000 0000000000000000
> [ 155.981321] ffff88103eb4ff48 ffffffff81c064f3 0000000000000000 8eaacf10758f15f5
> [ 156.001050] Call Trace:
> [ 156.001182] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 156.001412] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 156.021073] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 156.041987] Call Trace:
> [ 156.061141] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 156.061381] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 156.081061] NMI backtrace for cpu 61
> [ 156.081225] CPU 61
> [ 156.081330] Modules linked in:
> [ 156.081504]
> [ 156.100974] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 156.121073] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 156.121457] RSP: 0000:ffff88103eb65ee8 EFLAGS: 00000246
> [ 156.140977] RAX: 0000000000000000 RBX: ffff88103eb65fd8 RCX: 0000000000000000
> [ 156.141324] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb5a2b0
> [ 156.160959] RBP: ffff88103eb65ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 156.181152] R10: 0000000000000000 R11: ffff88103eb65ed8 R12: 000000000000003d
> [ 156.181478] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 156.211235] FS: 0000000000000000(0000) GS:ffff88807e200000(0000) knlGS:0000000000000000
> [ 156.231217] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 156.231480] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 156.250814] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 156.270483] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 156.270804] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb64000, task ffff88103eb5a2b0)
> [ 156.290623] Stack:
> [ 156.290726] ffff88103eb64000 ffffffff82525380 ffff88103eb65f28 ffffffff810391f0
> [ 156.310588] ffff88103eb65f18 dc7459780c591b2f 0000000000000000 0000000000000000
> [ 156.330348] ffff88103eb65f48 ffffffff81c064f3 0000000000000000 fcd3245bcc0358f7
> [ 156.330776] Call Trace:
> [ 156.330889] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 156.351067] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 156.351311] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 156.390822] Call Trace:
> [ 156.390937] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 156.411673] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 156.411946] NMI backtrace for cpu 63
> [ 156.430138] CPU 63
> [ 156.430240] Modules linked in:
> [ 156.430408]
> [ 156.430501] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 156.450462] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 156.470261] RSP: 0000:ffff88103eb91ee8 EFLAGS: 00000246
> [ 156.470494] RAX: 0000000000000000 RBX: ffff88103eb91fd8 RCX: 0000000000000000
> [ 156.490247] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb8a2b0
> [ 156.510060] RBP: ffff88103eb91ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 156.510374] R10: 0000000000000000 R11: ffff88103eb91ed8 R12: 000000000000003f
> [ 156.530133] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 156.549932] FS: 0000000000000000(0000) GS:ffff88807e600000(0000) knlGS:0000000000000000
> [ 156.550274] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 156.570064] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 156.589827] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 156.590146] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 156.609918] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb90000, task ffff88103eb8a2b0)
> [ 156.629740] Stack:
> [ 156.629845] ffff88103eb90000 ffffffff82525380 ffff88103eb91f28 ffffffff810391f0
> [ 156.630227] ffff88103eb91f18 84ed79833eb6a337 0000000000000000 0000000000000000
> [ 156.650055] ffff88103eb91f48 ffffffff81c064f3 0000000000000000 44a6529148d78bbf
> [ 156.669816] Call Trace:
> [ 156.669952] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 156.670175] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 156.689798] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 156.729686] Call Trace:
> [ 156.729804] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 156.730036] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 156.749627] NMI backtrace for cpu 54
> [ 156.749802] CPU 54
> [ 156.749903] Modules linked in:
> [ 156.769487]
> [ 156.769582] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 156.789570] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 156.809354] RSP: 0000:ffff88103eae9ee8 EFLAGS: 00000246
> [ 156.809583] RAX: 0000000000000000 RBX: ffff88103eae9fd8 RCX: 0000000000000000
> [ 156.829385] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eae0000
> [ 156.829698] RBP: ffff88103eae9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 156.849455] R10: 0000000000000000 R11: ffff88103eae9ed8 R12: 0000000000000036
> [ 156.869245] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 156.869559] FS: 0000000000000000(0000) GS:ffff88707ea00000(0000) knlGS:0000000000000000
> [ 156.889381] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 156.889640] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 156.909545] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 156.929239] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 156.929560] Process kworker/0:1 (pid: 0, threadinfo ffff88103eae8000, task ffff88103eae0000)
> [ 156.949369] Stack:
> [ 156.949462] ffff88103eae8000 ffffffff82525380 ffff88103eae9f28 ffffffff810391f0
> [ 156.969339] ffff88103eae9f18 8234c77185011823 0000000000000000 0000000000000000
> [ 156.989095] ffff88103eae9f48 ffffffff81c064f3 0000000000000000 72311f7455859eb1
> [ 156.989512] Call Trace:
> [ 157.008980] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 157.009215] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 157.028908] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 157.051654] Call Trace:
> [ 157.051780] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 157.074210] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 157.074460] NMI backtrace for cpu 55
> [ 157.088911] CPU 55
> [ 157.089011] Modules linked in:
> [ 157.089197]
> [ 157.089273] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 157.109258] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 157.129339] RSP: 0000:ffff88103eaefee8 EFLAGS: 00000246
> [ 157.148954] RAX: 0000000000000000 RBX: ffff88103eaeffd8 RCX: 0000000000000000
> [ 157.149287] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eaf8000
> [ 157.168798] RBP: ffff88103eaefef8 R08: 0000000000000000 R09: 0000000000000000
> [ 157.188599] R10: 0000000000000000 R11: ffff88103eaefed8 R12: 0000000000000037
> [ 157.188918] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 157.211302] FS: 0000000000000000(0000) GS:ffff88707ec00000(0000) knlGS:0000000000000000
> [ 157.228522] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 157.228784] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 157.248559] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 157.248868] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 157.268646] Process kworker/0:1 (pid: 0, threadinfo ffff88103eaee000, task ffff88103eaf8000)
> [ 157.288488] Stack:
> [ 157.288583] ffff88103eaee000 ffffffff82525380 ffff88103eaeff28 ffffffff810391f0
> [ 157.308388] ffff88103eaeff18 9e7c99cf43a1a877 0000000000000000 0000000000000000
> [ 157.308799] ffff88103eaeff48 ffffffff81c064f3 0000000000000000 d8c1df20b1dbb268
> [ 157.328592] Call Trace:
> [ 157.328706] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 157.348307] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 157.348584] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 157.388555] Call Trace:
> [ 157.388673] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 157.408129] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 157.408368] NMI backtrace for cpu 62
> [ 157.408547] CPU 62
> [ 157.428099] Modules linked in:
> [ 157.428289]
> [ 157.428380] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 157.448323] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 157.468105] RSP: 0000:ffff88103eb83ee8 EFLAGS: 00000246
> [ 157.468384] RAX: 0000000000000000 RBX: ffff88103eb83fd8 RCX: 0000000000000000
> [ 157.488108] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103eb722b0
> [ 157.488454] RBP: ffff88103eb83ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 157.508219] R10: 0000000000000000 R11: ffff88103eb83ed8 R12: 000000000000003e
> [ 157.528004] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 157.528368] FS: 0000000000000000(0000) GS:ffff88807e400000(0000) knlGS:0000000000000000
> [ 157.548156] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 157.567893] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 157.568206] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 157.587971] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 157.607746] Process kworker/0:1 (pid: 0, threadinfo ffff88103eb82000, task ffff88103eb722b0)
> [ 157.608116] Stack:
> [ 157.628263] ffff88103eb82000 ffffffff82525380 ffff88103eb83f28 ffffffff810391f0
> [ 157.628758] ffff88103eb83f18 4e93b1db6fe4c26f 0000000000000000 0000000000000000
> [ 157.647847] ffff88103eb83f48 ffffffff81c064f3 0000000000000000 cc3ee426a493ae7b
> [ 157.667637] Call Trace:
> [ 157.667758] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 157.668008] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 157.687739] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 157.709742] Call Trace:
> [ 157.727548] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 157.727762] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 157.747486] NMI backtrace for cpu 64
> [ 157.747681] CPU 64
> [ 157.747771] Modules linked in:
> [ 157.747974]
> [ 157.767397] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 157.787407] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 157.787827] RSP: 0000:ffff88103ebbdee8 EFLAGS: 00000246
> [ 157.807451] RAX: 0000000000000000 RBX: ffff88103ebbdfd8 RCX: 0000000000000000
> [ 157.807774] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ebc0000
> [ 157.827550] RBP: ffff88103ebbdef8 R08: 0000000000000000 R09: 0000000000000000
> [ 157.847363] R10: 0000000000000000 R11: ffff88103ebbded8 R12: 0000000000000040
> [ 157.847704] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 157.867425] FS: 0000000000000000(0000) GS:ffff88107ee00000(0000) knlGS:0000000000000000
> [ 157.887265] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 157.887530] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 157.907307] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 157.927079] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 157.927445] Process kworker/0:1 (pid: 0, threadinfo ffff88103ebbc000, task ffff88103ebc0000)
> [ 157.947240] Stack:
> [ 157.947345] ffff88103ebbc000 ffffffff82525380 ffff88103ebbdf28 ffffffff810391f0
> [ 157.967173] ffff88103ebbdf18 da2a35009143e8db 0000000000000000 0000000000000000
> [ 157.967569] ffff88103ebbdf48 ffffffff81c064f3 0000000000000000 e202c1b2447d6be3
> [ 157.987308] Call Trace:
> [ 157.987435] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 158.007065] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 158.007361] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 158.047366] Call Trace:
> [ 158.047483] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 158.067043] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 158.067286] NMI backtrace for cpu 67
> [ 158.086766] CPU 67
> [ 158.086860] Modules linked in:
> [ 158.087027]
> [ 158.087130] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 158.107106] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 158.127207] RSP: 0000:ffff88103e44dee8 EFLAGS: 00000246
> [ 158.127455] RAX: 0000000000000000 RBX: ffff88103e44dfd8 RCX: 0000000000000000
> [ 158.147037] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e450000
> [ 158.166786] RBP: ffff88103e44def8 R08: 0000000000000000 R09: 0000000000000000
> [ 158.167114] R10: 0000000000000000 R11: ffff88103e44ded8 R12: 0000000000000043
> [ 158.186752] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 158.206566] FS: 0000000000000000(0000) GS:ffff88107f400000(0000) knlGS:0000000000000000
> [ 158.206952] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 158.226680] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 158.227004] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 158.246752] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 158.266520] Process kworker/0:1 (pid: 0, threadinfo ffff88103e44c000, task ffff88103e450000)
> [ 158.286347] Stack:
> [ 158.286452] ffff88103e44c000 ffffffff82525380 ffff88103e44df28 ffffffff810391f0
> [ 158.286848] ffff88103e44df18 ebf5aa6367d481b1 0000000000000000 0000000000000000
> [ 158.306616] ffff88103e44df48 ffffffff81c064f3 0000000000000000 27a37715890bf64b
> [ 158.326389] Call Trace:
> [ 158.326526] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 158.326802] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 158.348029] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 158.386263] Call Trace:
> [ 158.386391] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 158.386626] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 158.406330] NMI backtrace for cpu 65
> [ 158.406505] CPU 65
> [ 158.406602] Modules linked in:
> [ 158.426114]
> [ 158.426190] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 158.446186] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 158.446590] RSP: 0000:ffff88103ebe7ee8 EFLAGS: 00000246
> [ 158.466219] RAX: 0000000000000000 RBX: ffff88103ebe7fd8 RCX: 0000000000000000
> [ 158.485966] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103ebe8000
> [ 158.486302] RBP: ffff88103ebe7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 158.506073] R10: 0000000000000000 R11: ffff88103ebe7ed8 R12: 0000000000000041
> [ 158.525901] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 158.526203] FS: 0000000000000000(0000) GS:ffff88107f000000(0000) knlGS:0000000000000000
> [ 158.546011] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 158.546279] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 158.566057] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 158.585859] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 158.586252] Process kworker/0:1 (pid: 0, threadinfo ffff88103ebe6000, task ffff88103ebe8000)
> [ 158.606014] Stack:
> [ 158.606118] ffff88103ebe6000 ffffffff82525380 ffff88103ebe7f28 ffffffff810391f0
> [ 158.636016] ffff88103ebe7f18 26ab9a4774f02d67 0000000000000000 0000000000000000
> [ 158.656062] ffff88103ebe7f48 ffffffff81c064f3 0000000000000000 3a137c915fdefa51
> [ 158.656439] Call Trace:
> [ 158.675780] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 158.676008] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 158.695748] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 158.716263] Call Trace:
> [ 158.716384] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 158.735749] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 158.735985] NMI backtrace for cpu 66
> [ 158.755480] CPU 66
> [ 158.755581] Modules linked in:
> [ 158.755756]
> [ 158.755846] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 158.775821] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 158.795583] RSP: 0000:ffff88103e42dee8 EFLAGS: 00000246
> [ 158.815274] RAX: 0000000000000000 RBX: ffff88103e42dfd8 RCX: 0000000000000000
> [ 158.815608] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e430000
> [ 158.835420] RBP: ffff88103e42def8 R08: 0000000000000000 R09: 0000000000000000
> [ 158.835734] R10: 0000000000000000 R11: ffff88103e42ded8 R12: 0000000000000042
> [ 158.855505] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 158.875267] FS: 0000000000000000(0000) GS:ffff88107f200000(0000) knlGS:0000000000000000
> [ 158.895089] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 158.895353] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 158.915139] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 158.915459] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 158.935242] Process kworker/0:1 (pid: 0, threadinfo ffff88103e42c000, task ffff88103e430000)
> [ 158.955078] Stack:
> [ 158.955172] ffff88103e42c000 ffffffff82525380 ffff88103e42df28 ffffffff810391f0
> [ 158.974989] ffff88103e42df18 ecd678057994030d 0000000000000000 0000000000000000
> [ 158.975385] ffff88103e42df48 ffffffff81c064f3 0000000000000000 6f39f013d9ac3289
> [ 158.996217] Call Trace:
> [ 158.996338] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.014900] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 159.015180] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 159.055093] Call Trace:
> [ 159.055231] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.055486] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 159.075260] NMI backtrace for cpu 69
> [ 159.075456] CPU 69
> [ 159.075552] Modules linked in:
> [ 159.094880]
> [ 159.094964] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 159.115131] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 159.134875] RSP: 0000:ffff88103e4a3ee8 EFLAGS: 00000246
> [ 159.135116] RAX: 0000000000000000 RBX: ffff88103e4a3fd8 RCX: 0000000000000000
> [ 159.154817] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e4a8000
> [ 159.155159] RBP: ffff88103e4a3ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 159.174816] R10: 0000000000000000 R11: ffff88103e4a3ed8 R12: 0000000000000045
> [ 159.194596] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 159.194939] FS: 0000000000000000(0000) GS:ffff88107f800000(0000) knlGS:0000000000000000
> [ 159.214755] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 159.234471] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 159.234789] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 159.254603] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 159.274337] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4a2000, task ffff88103e4a8000)
> [ 159.274713] Stack:
> [ 159.294270] ffff88103e4a2000 ffffffff82525380 ffff88103e4a3f28 ffffffff810391f0
> [ 159.294680] ffff88103e4a3f18 12ef20a5c0174c7c 0000000000000000 0000000000000000
> [ 159.314468] ffff88103e4a3f48 ffffffff81c064f3 0000000000000000 2874467214a4e39e
> [ 159.334199] Call Trace:
> [ 159.334330] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.334598] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 159.354245] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 159.375041] Call Trace:
> [ 159.394132] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.394358] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 159.414070] NMI backtrace for cpu 71
> [ 159.414251] CPU 71
> [ 159.414338] Modules linked in:
> [ 159.414504]
> [ 159.414591] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 159.453983] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 159.454366] RSP: 0000:ffff88103e4edee8 EFLAGS: 00000246
> [ 159.474019] RAX: 0000000000000000 RBX: ffff88103e4edfd8 RCX: 0000000000000000
> [ 159.474338] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e4f0000
> [ 159.494112] RBP: ffff88103e4edef8 R08: 0000000000000000 R09: 0000000000000000
> [ 159.513924] R10: 0000000000000000 R11: ffff88103e4eded8 R12: 0000000000000047
> [ 159.514269] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 159.534030] FS: 0000000000000000(0000) GS:ffff88107fc00000(0000) knlGS:0000000000000000
> [ 159.553842] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 159.554154] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 159.573926] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 159.593668] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 159.594006] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4ec000, task ffff88103e4f0000)
> [ 159.613854] Stack:
> [ 159.613968] ffff88103e4ec000 ffffffff82525380 ffff88103e4edf28 ffffffff810391f0
> [ 159.633749] ffff88103e4edf18 fe58eb5636563f39 0000000000000000 0000000000000000
> [ 159.634152] ffff88103e4edf48 ffffffff81c064f3 0000000000000000 dfdfb102a569daf8
> [ 159.654863] Call Trace:
> [ 159.655000] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.673642] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 159.673921] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 159.713904] Call Trace:
> [ 159.714028] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.733478] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 159.733723] NMI backtrace for cpu 68
> [ 159.733904] CPU 68
> [ 159.753420] Modules linked in:
> [ 159.753610]
> [ 159.753689] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 159.773674] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 159.793424] RSP: 0000:ffff88103e479ee8 EFLAGS: 00000246
> [ 159.793666] RAX: 0000000000000000 RBX: ffff88103e479fd8 RCX: 0000000000000000
> [ 159.813458] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e470000
> [ 159.833223] RBP: ffff88103e479ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 159.833564] R10: 0000000000000000 R11: ffff88103e479ed8 R12: 0000000000000044
> [ 159.853338] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 159.873135] FS: 0000000000000000(0000) GS:ffff88107f600000(0000) knlGS:0000000000000000
> [ 159.873502] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 159.893207] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 159.893574] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 159.913329] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 159.933094] Process kworker/0:1 (pid: 0, threadinfo ffff88103e478000, task ffff88103e470000)
> [ 159.952924] Stack:
> [ 159.953033] ffff88103e478000 ffffffff82525380 ffff88103e479f28 ffffffff810391f0
> [ 159.953469] ffff88103e479f18 e7234dc0ddbdb12e 0000000000000000 0000000000000000
> [ 159.973235] ffff88103e479f48 ffffffff81c064f3 0000000000000000 253694480af14bae
> [ 159.992964] Call Trace:
> [ 159.993116] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 159.993380] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 160.012990] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 160.052829] Call Trace:
> [ 160.052958] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 160.053236] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 160.072848] NMI backtrace for cpu 70
> [ 160.073062] CPU 70
> [ 160.073154] Modules linked in:
> [ 160.092815]
> [ 160.092896] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 160.112764] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 160.113213] RSP: 0000:ffff88103e4c5ee8 EFLAGS: 00000246
> [ 160.133168] RAX: 0000000000000000 RBX: ffff88103e4c5fd8 RCX: 0000000000000000
> [ 160.152740] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e4c8000
> [ 160.153090] RBP: ffff88103e4c5ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 160.172802] R10: 0000000000000000 R11: ffff88103e4c5ed8 R12: 0000000000000046
> [ 160.173147] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 160.192773] FS: 0000000000000000(0000) GS:ffff88107fa00000(0000) knlGS:0000000000000000
> [ 160.212598] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 160.212864] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 160.232649] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 160.252423] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 160.252734] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4c4000, task ffff88103e4c8000)
> [ 160.272600] Stack:
> [ 160.272704] ffff88103e4c4000 ffffffff82525380 ffff88103e4c5f28 ffffffff810391f0
> [ 160.293314] ffff88103e4c5f18 6e7cc5eba8cea412 0000000000000000 0000000000000000
> [ 160.312278] ffff88103e4c5f48 ffffffff81c064f3 0000000000000000 23ba843c07b63fab
> [ 160.312674] Call Trace:
> [ 160.332162] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 160.332416] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 160.332714] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 160.372721] Call Trace:
> [ 160.372865] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 160.392225] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 160.392540] NMI backtrace for cpu 79
> [ 160.412090] CPU 79
> [ 160.412182] Modules linked in:
> [ 160.412350]
> [ 160.412431] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 160.432387] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 160.452168] RSP: 0000:ffff88103e591ee8 EFLAGS: 00000246
> [ 160.471869] RAX: 0000000000000000 RBX: ffff88103e591fd8 RCX: 0000000000000000
> [ 160.472193] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e588000
> [ 160.491987] RBP: ffff88103e591ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 160.492302] R10: 0000000000000000 R11: ffff88103e591ed8 R12: 000000000000004f
> [ 160.512076] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 160.531890] FS: 0000000000000000(0000) GS:ffff88207fc00000(0000) knlGS:0000000000000000
> [ 160.532287] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 160.551961] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 160.571758] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 160.572094] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 160.591836] Process kworker/0:1 (pid: 0, threadinfo ffff88103e590000, task ffff88103e588000)
> [ 160.611673] Stack:
> [ 160.611779] ffff88103e590000 ffffffff82525380 ffff88103e591f28 ffffffff810391f0
> [ 160.631590] ffff88103e591f18 9b70b21e28590e01 0000000000000000 0000000000000000
> [ 160.631973] ffff88103e591f48 ffffffff81c064f3 0000000000000000 db865ef5c8a39a71
> [ 160.651715] Call Trace:
> [ 160.651846] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 160.671492] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 160.671751] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 160.711666] Call Trace:
> [ 160.711789] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 160.712020] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 160.731566] NMI backtrace for cpu 73
> [ 160.731752] CPU 73
> [ 160.731856] Modules linked in:
> [ 160.751431]
> [ 160.751510] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 160.771486] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 160.791281] RSP: 0018:ffff88103e521ee8 EFLAGS: 00000246
> [ 160.791528] RAX: 0000000000000000 RBX: ffff88103e521fd8 RCX: 0000000000000000
> [ 160.811393] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e518000
> [ 160.811732] RBP: ffff88103e521ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 160.831426] R10: 0000000000000000 R11: ffff88103e521ed8 R12: 0000000000000049
> [ 160.851216] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 160.851587] FS: 0000000000000000(0000) GS:ffff88207f000000(0000) knlGS:0000000000000000
> [ 160.871381] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 160.891082] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 160.891408] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 160.911154] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 160.936751] Process kworker/0:1 (pid: 0, threadinfo ffff88103e520000, task ffff88103e518000)
> [ 160.937268] Stack:
> [ 160.937369] ffff88103e520000 ffffffff82525380 ffff88103e521f28 ffffffff810391f0
> [ 160.951285] ffff88103e521f18 d8b1686f1d48fddc 0000000000000000 0000000000000000
> [ 160.971049] ffff88103e521f48 ffffffff81c064f3 0000000000000000 8790dbe0cba0e12f
> [ 160.971480] Call Trace:
> [ 160.990933] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 160.991198] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.010845] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 161.031582] Call Trace:
> [ 161.050750] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 161.050978] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.070683] NMI backtrace for cpu 75
> [ 161.070855] CPU 75
> [ 161.070954] Modules linked in:
> [ 161.071122]
> [ 161.071222] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 161.091281] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 161.111280] RSP: 0000:ffff88103e535ee8 EFLAGS: 00000246
> [ 161.130841] RAX: 0000000000000000 RBX: ffff88103e535fd8 RCX: 0000000000000000
> [ 161.131158] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e538000
> [ 161.150892] RBP: ffff88103e535ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 161.170698] R10: 0000000000000000 R11: ffff88103e535ed8 R12: 000000000000004b
> [ 161.171032] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 161.190633] FS: 0000000000000000(0000) GS:ffff88207f400000(0000) knlGS:0000000000000000
> [ 161.210473] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 161.210738] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 161.230638] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 161.230942] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 161.250591] Process kworker/0:1 (pid: 0, threadinfo ffff88103e534000, task ffff88103e538000)
> [ 161.270454] Stack:
> [ 161.270558] ffff88103e534000 ffffffff82525380 ffff88103e535f28 ffffffff810391f0
> [ 161.290363] ffff88103e535f18 cd61f3b922c9e5b4 0000000000000000 0000000000000000
> [ 161.290748] ffff88103e535f48 ffffffff81c064f3 0000000000000000 029d491e869419ff
> [ 161.310478] Call Trace:
> [ 161.310611] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 161.330370] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.330658] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 161.370461] Call Trace:
> [ 161.370589] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 161.390052] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.390316] NMI backtrace for cpu 77
> [ 161.390486] CPU 77
> [ 161.410029] Modules linked in:
> [ 161.410209]
> [ 161.410297] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 161.430242] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 161.450041] RSP: 0000:ffff88103e55dee8 EFLAGS: 00000246
> [ 161.450268] RAX: 0000000000000000 RBX: ffff88103e55dfd8 RCX: 0000000000000000
> [ 161.470065] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e560000
> [ 161.489845] RBP: ffff88103e55def8 R08: 0000000000000000 R09: 0000000000000000
> [ 161.490169] R10: 0000000000000000 R11: ffff88103e55ded8 R12: 000000000000004d
> [ 161.509948] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 161.529741] FS: 0000000000000000(0000) GS:ffff88207f800000(0000) knlGS:0000000000000000
> [ 161.530113] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 161.549816] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 161.550135] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 161.570786] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 161.589747] Process kworker/0:1 (pid: 0, threadinfo ffff88103e55c000, task ffff88103e560000)
> [ 161.590145] Stack:
> [ 161.609633] ffff88103e55c000 ffffffff82525380 ffff88103e55df28 ffffffff810391f0
> [ 161.610023] ffff88103e55df18 ad33531f6f46c54f 0000000000000000 0000000000000000
> [ 161.629816] ffff88103e55df48 ffffffff81c064f3 0000000000000000 d9f51b4f236197d5
> [ 161.649593] Call Trace:
> [ 161.649705] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 161.649938] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.669586] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 161.709392] Call Trace:
> [ 161.709526] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 161.709768] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.729432] NMI backtrace for cpu 72
> [ 161.729640] CPU 72
> [ 161.729727] Modules linked in:
> [ 161.729936]
> [ 161.749342] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 161.769384] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 161.769830] RSP: 0000:ffff88103e4fbee8 EFLAGS: 00000246
> [ 161.789375] RAX: 0000000000000000 RBX: ffff88103e4fbfd8 RCX: 0000000000000000
> [ 161.809156] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e500000
> [ 161.809469] RBP: ffff88103e4fbef8 R08: 0000000000000000 R09: 0000000000000000
> [ 161.829271] R10: 0000000000000000 R11: ffff88103e4fbed8 R12: 0000000000000048
> [ 161.829589] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 161.849394] FS: 0000000000000000(0000) GS:ffff88207ee00000(0000) knlGS:0000000000000000
> [ 161.869228] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 161.869494] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 161.889262] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 161.909042] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 161.909363] Process kworker/0:1 (pid: 0, threadinfo ffff88103e4fa000, task ffff88103e500000)
> [ 161.929206] Stack:
> [ 161.929300] ffff88103e4fa000 ffffffff82525380 ffff88103e4fbf28 ffffffff810391f0
> [ 161.949137] ffff88103e4fbf18 ce446216a9327bfa 0000000000000000 0000000000000000
> [ 161.968877] ffff88103e4fbf48 ffffffff81c064f3 0000000000000000 35549ebda9dd1599
> [ 161.969277] Call Trace:
> [ 161.969393] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 161.989001] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 161.989249] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 162.029298] Call Trace:
> [ 162.029416] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 162.048827] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 162.049091] NMI backtrace for cpu 78
> [ 162.068668] CPU 78
> [ 162.068785] Modules linked in:
> [ 162.068986]
> [ 162.069060] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 162.089028] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 162.108891] RSP: 0000:ffff88103e57bee8 EFLAGS: 00000246
> [ 162.109126] RAX: 0000000000000000 RBX: ffff88103e57bfd8 RCX: 0000000000000000
> [ 162.129627] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e580000
> [ 162.148950] RBP: ffff88103e57bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 162.149284] R10: 0000000000000000 R11: ffff88103e57bed8 R12: 000000000000004e
> [ 162.168852] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 162.188626] FS: 0000000000000000(0000) GS:ffff88207fa00000(0000) knlGS:0000000000000000
> [ 162.188981] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 162.208541] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 162.229972] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 162.230346] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 162.248455] Process kworker/0:1 (pid: 0, threadinfo ffff88103e57a000, task ffff88103e580000)
> [ 162.268290] Stack:
> [ 162.268385] ffff88103e57a000 ffffffff82525380 ffff88103e57bf28 ffffffff810391f0
> [ 162.268801] ffff88103e57bf18 f032e5fdb3adf613 0000000000000000 0000000000000000
> [ 162.288547] ffff88103e57bf48 ffffffff81c064f3 0000000000000000 57f6bb23ccbb4736
> [ 162.308355] Call Trace:
> [ 162.308489] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 162.308746] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 162.328341] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 162.368494] Call Trace:
> [ 162.368649] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 162.368860] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 162.388166] NMI backtrace for cpu 76
> [ 162.388337] CPU 76
> [ 162.388429] Modules linked in:
> [ 162.408036]
> [ 162.408115] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 162.428130] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 162.447877] RSP: 0000:ffff88103e54fee8 EFLAGS: 00000246
> [ 162.448131] RAX: 0000000000000000 RBX: ffff88103e54ffd8 RCX: 0000000000000000
> [ 162.467907] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e550000
> [ 162.468219] RBP: ffff88103e54fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 162.488048] R10: 0000000000000000 R11: ffff88103e54fed8 R12: 000000000000004c
> [ 162.507800] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 162.508124] FS: 0000000000000000(0000) GS:ffff88207f600000(0000) knlGS:0000000000000000
> [ 162.527928] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 162.547669] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 162.548032] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 162.567780] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 162.568095] Process kworker/0:1 (pid: 0, threadinfo ffff88103e54e000, task ffff88103e550000)
> [ 162.587923] Stack:
> [ 162.588018] ffff88103e54e000 ffffffff82525380 ffff88103e54ff28 ffffffff810391f0
> [ 162.607881] ffff88103e54ff18 cae5dd183ae409a6 0000000000000000 0000000000000000
> [ 162.627664] ffff88103e54ff48 ffffffff81c064f3 0000000000000000 17a57ec2fee86728
> [ 162.628032] Call Trace:
> [ 162.647521] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 162.647750] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 162.667465] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 162.688147] Call Trace:
> [ 162.707447] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 162.707667] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 162.707931] NMI backtrace for cpu 74
> [ 162.747466] CPU 74
> [ 162.747567] Modules linked in:
> [ 162.747742]
> [ 162.747826] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 162.768064] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 162.787730] RSP: 0018:ffff88103e527ee8 EFLAGS: 00000246
> [ 162.807333] RAX: 0000000000000000 RBX: ffff88103e527fd8 RCX: 0000000000000000
> [ 162.807667] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e528000
> [ 162.827464] RBP: ffff88103e527ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 162.847210] R10: 0000000000000000 R11: ffff88103e527ed8 R12: 000000000000004a
> [ 162.847530] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 162.867263] FS: 0000000000000000(0000) GS:ffff88207f200000(0000) knlGS:0000000000000000
> [ 162.893181] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 162.893478] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 162.907170] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 162.907521] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 162.927151] Process kworker/0:1 (pid: 0, threadinfo ffff88103e526000, task ffff88103e528000)
> [ 162.947018] Stack:
> [ 162.947132] ffff88103e526000 ffffffff82525380 ffff88103e527f28 ffffffff810391f0
> [ 162.966915] ffff88103e527f18 3ceecc1c6f3146ab 0000000000000000 0000000000000000
> [ 162.967316] ffff88103e527f48 ffffffff81c064f3 0000000000000000 c6250755f2f9eca7
> [ 162.987055] Call Trace:
> [ 162.987176] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.006813] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 163.007072] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 163.047044] Call Trace:
> [ 163.047157] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.066622] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 163.066876] NMI backtrace for cpu 81
> [ 163.067069] CPU 81
> [ 163.086579] Modules linked in:
> [ 163.086762]
> [ 163.086836] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 163.106837] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 163.126616] RSP: 0018:ffff88103e5afee8 EFLAGS: 00000246
> [ 163.126895] RAX: 0000000000000000 RBX: ffff88103e5affd8 RCX: 0000000000000000
> [ 163.147061] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5b4560
> [ 163.166578] RBP: ffff88103e5afef8 R08: 0000000000000000 R09: 0000000000000000
> [ 163.166956] R10: 0000000000000000 R11: ffff88103e5afed8 R12: 0000000000000051
> [ 163.186810] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 163.187181] FS: 0000000000000000(0000) GS:ffff88307f000000(0000) knlGS:0000000000000000
> [ 163.206849] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 163.226392] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 163.226762] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 163.246575] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 163.266265] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5ae000, task ffff88103e5b4560)
> [ 163.266723] Stack:
> [ 163.286189] ffff88103e5ae000 ffffffff82525380 ffff88103e5aff28 ffffffff810391f0
> [ 163.286584] ffff88103e5aff18 7782e4a57542f064 0000000000000000 0000000000000000
> [ 163.306399] ffff88103e5aff48 ffffffff81c064f3 0000000000000000 8ba93f0bab920685
> [ 163.326120] Call Trace:
> [ 163.326252] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.326540] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 163.346197] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 163.366956] Call Trace:
> [ 163.386044] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.386320] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 163.405982] NMI backtrace for cpu 85
> [ 163.406136] CPU 85
> [ 163.406236] Modules linked in:
> [ 163.406409]
> [ 163.425904] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 163.445897] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 163.446273] RSP: 0000:ffff88103e613ee8 EFLAGS: 00000246
> [ 163.466138] RAX: 0000000000000000 RBX: ffff88103e613fd8 RCX: 0000000000000000
> [ 163.466548] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e60c560
> [ 163.486079] RBP: ffff88103e613ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 163.505826] R10: 0000000000000000 R11: ffff88103e613ed8 R12: 0000000000000055
> [ 163.506243] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 163.527647] FS: 0000000000000000(0000) GS:ffff88307f800000(0000) knlGS:0000000000000000
> [ 163.545778] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 163.546092] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 163.565832] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 163.585568] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 163.585987] Process kworker/0:1 (pid: 0, threadinfo ffff88103e612000, task ffff88103e60c560)
> [ 163.605780] Stack:
> [ 163.605873] ffff88103e612000 ffffffff82525380 ffff88103e613f28 ffffffff810391f0
> [ 163.625647] ffff88103e613f18 1338bcfcabb32e6d 0000000000000000 0000000000000000
> [ 163.645439] ffff88103e613f48 ffffffff81c064f3 0000000000000000 71bb112c4d5bc551
> [ 163.645899] Call Trace:
> [ 163.646032] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.665559] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 163.665813] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 163.705851] Call Trace:
> [ 163.705981] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.725406] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 163.725699] NMI backtrace for cpu 87
> [ 163.745234] CPU 87
> [ 163.745337] Modules linked in:
> [ 163.745498]
> [ 163.745574] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 163.765557] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 163.785347] RSP: 0000:ffff88103e641ee8 EFLAGS: 00000246
> [ 163.785634] RAX: 0000000000000000 RBX: ffff88103e641fd8 RCX: 0000000000000000
> [ 163.805440] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e63c560
> [ 163.825147] RBP: ffff88103e641ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 163.825623] R10: 0000000000000000 R11: ffff88103e641ed8 R12: 0000000000000057
> [ 163.845315] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 163.865029] FS: 0000000000000000(0000) GS:ffff88307fc00000(0000) knlGS:0000000000000000
> [ 163.865477] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 163.885147] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 163.904917] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 163.905367] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 163.925030] Process kworker/0:1 (pid: 0, threadinfo ffff88103e640000, task ffff88103e63c560)
> [ 163.944862] Stack:
> [ 163.944954] ffff88103e640000 ffffffff82525380 ffff88103e641f28 ffffffff810391f0
> [ 163.945367] ffff88103e641f18 41f11d18f408308f 0000000000000000 0000000000000000
> [ 163.965173] ffff88103e641f48 ffffffff81c064f3 0000000000000000 388370f59e8df041
> [ 163.984907] Call Trace:
> [ 163.985023] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 163.985321] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 164.004921] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 164.044771] Call Trace:
> [ 164.044911] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 164.045194] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 164.064719] NMI backtrace for cpu 84
> [ 164.064933] CPU 84
> [ 164.065020] Modules linked in:
> [ 164.084637]
> [ 164.084708] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 164.104674] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 164.124627] RSP: 0000:ffff88103e5edee8 EFLAGS: 00000246
> [ 164.124880] RAX: 0000000000000000 RBX: ffff88103e5edfd8 RCX: 0000000000000000
> [ 164.144688] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5f4560
> [ 164.145133] RBP: ffff88103e5edef8 R08: 0000000000000000 R09: 0000000000000000
> [ 164.166316] R10: 0000000000000000 R11: ffff88103e5eded8 R12: 0000000000000054
> [ 164.184492] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 164.184895] FS: 0000000000000000(0000) GS:ffff88307f600000(0000) knlGS:0000000000000000
> [ 164.204826] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 164.205180] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 164.224723] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 164.244472] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 164.244862] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5ec000, task ffff88103e5f4560)
> [ 164.264574] Stack:
> [ 164.264692] ffff88103e5ec000 ffffffff82525380 ffff88103e5edf28 ffffffff810391f0
> [ 164.284428] ffff88103e5edf18 ac950cdde64809da 0000000000000000 0000000000000000
> [ 164.304208] ffff88103e5edf48 ffffffff81c064f3 0000000000000000 79cf6d869ee9f9b5
> [ 164.304641] Call Trace:
> [ 164.324105] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 164.324330] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 164.344004] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 164.364677] Call Trace:
> [ 164.364797] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 164.384674] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 164.385034] NMI backtrace for cpu 80
> [ 164.403994] CPU 80
> [ 164.404084] Modules linked in:
> [ 164.404288]
> [ 164.404374] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 164.424323] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 164.444136] RSP: 0000:ffff88103e5a9ee8 EFLAGS: 00000246
> [ 164.463804] RAX: 0000000000000000 RBX: ffff88103e5a9fd8 RCX: 0000000000000000
> [ 164.464126] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5a4560
> [ 164.483918] RBP: ffff88103e5a9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 164.503706] R10: 0000000000000000 R11: ffff88103e5a9ed8 R12: 0000000000000050
> [ 164.504052] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 164.523786] FS: 0000000000000000(0000) GS:ffff88307ee00000(0000) knlGS:0000000000000000
> [ 164.543621] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 164.543938] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 164.563679] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 164.564099] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 164.583788] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5a8000, task ffff88103e5a4560)
> [ 164.603591] Stack:
> [ 164.603694] ffff88103e5a8000 ffffffff82525380 ffff88103e5a9f28 ffffffff810391f0
> [ 164.623514] ffff88103e5a9f18 e1ea2195949618ac 0000000000000000 0000000000000000
> [ 164.623940] ffff88103e5a9f48 ffffffff81c064f3 0000000000000000 913d0aa0500edc01
> [ 164.643666] Call Trace:
> [ 164.643774] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 164.663404] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 164.663733] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 164.703723] Call Trace:
> [ 164.703841] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 164.723230] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 164.723575] NMI backtrace for cpu 82
> [ 164.723783] CPU 82
> [ 164.743189] Modules linked in:
> [ 164.743378]
> [ 164.743459] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 164.763419] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 164.783196] RSP: 0000:ffff88103e5d9ee8 EFLAGS: 00000246
> [ 164.783437] RAX: 0000000000000000 RBX: ffff88103e5d9fd8 RCX: 0000000000000000
> [ 164.803280] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5cc560
> [ 164.803705] RBP: ffff88103e5d9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 164.825030] R10: 0000000000000000 R11: ffff88103e5d9ed8 R12: 0000000000000052
> [ 164.843096] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 164.843567] FS: 0000000000000000(0000) GS:ffff88307f200000(0000) knlGS:0000000000000000
> [ 164.863342] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 164.882987] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 164.883337] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 164.903103] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 164.922856] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5d8000, task ffff88103e5cc560)
> [ 164.923279] Stack:
> [ 164.942787] ffff88103e5d8000 ffffffff82525380 ffff88103e5d9f28 ffffffff810391f0
> [ 164.943248] ffff88103e5d9f18 3ca5437b49de96ec 0000000000000000 0000000000000000
> [ 164.962981] ffff88103e5d9f48 ffffffff81c064f3 0000000000000000 2b4b5c0dcf3a906f
> [ 164.982749] Call Trace:
> [ 164.982871] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 164.983154] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.002755] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 165.023581] Call Trace:
> [ 165.042671] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 165.042935] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.062603] NMI backtrace for cpu 83
> [ 165.062792] CPU 83
> [ 165.062913] Modules linked in:
> [ 165.063139]
> [ 165.082496] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 165.102531] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 165.102959] RSP: 0000:ffff88103e5dfee8 EFLAGS: 00000246
> [ 165.122610] RAX: 0000000000000000 RBX: ffff88103e5dffd8 RCX: 0000000000000000
> [ 165.123043] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e5e4560
> [ 165.142773] RBP: ffff88103e5dfef8 R08: 0000000000000000 R09: 0000000000000000
> [ 165.162713] R10: 0000000000000000 R11: ffff88103e5dfed8 R12: 0000000000000053
> [ 165.163093] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 165.182668] FS: 0000000000000000(0000) GS:ffff88307f400000(0000) knlGS:0000000000000000
> [ 165.202485] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 165.202770] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 165.222691] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 165.242184] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 165.242557] Process kworker/0:1 (pid: 0, threadinfo ffff88103e5de000, task ffff88103e5e4560)
> [ 165.262353] Stack:
> [ 165.262446] ffff88103e5de000 ffffffff82525380 ffff88103e5dff28 ffffffff810391f0
> [ 165.282294] ffff88103e5dff18 291962416f9caca0 0000000000000000 0000000000000000
> [ 165.282731] ffff88103e5dff48 ffffffff81c064f3 0000000000000000 182eaafab45800c9
> [ 165.302476] Call Trace:
> [ 165.302606] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 165.322182] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.322495] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 165.362412] Call Trace:
> [ 165.362529] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 165.393005] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.393270] NMI backtrace for cpu 86
> [ 165.411822] CPU 86
> [ 165.411937] Modules linked in:
> [ 165.412101]
> [ 165.412177] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 165.432223] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 165.451948] RSP: 0000:ffff88103e629ee8 EFLAGS: 00000246
> [ 165.452228] RAX: 0000000000000000 RBX: ffff88103e629fd8 RCX: 0000000000000000
> [ 165.473706] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e624560
> [ 165.491738] RBP: ffff88103e629ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 165.492121] R10: 0000000000000000 R11: ffff88103e629ed8 R12: 0000000000000056
> [ 165.511828] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 165.531631] FS: 0000000000000000(0000) GS:ffff88307fa00000(0000) knlGS:0000000000000000
> [ 165.532019] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 165.551714] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 165.552087] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 165.571840] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 165.591613] Process kworker/0:1 (pid: 0, threadinfo ffff88103e628000, task ffff88103e624560)
> [ 165.611425] Stack:
> [ 165.611552] ffff88103e628000 ffffffff82525380 ffff88103e629f28 ffffffff810391f0
> [ 165.611966] ffff88103e629f18 18b050d3e519eccd 0000000000000000 0000000000000000
> [ 165.631721] ffff88103e629f48 ffffffff81c064f3 0000000000000000 e4373b2097d82c28
> [ 165.651482] Call Trace:
> [ 165.651600] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 165.651845] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.671528] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 165.711543] Call Trace:
> [ 165.711662] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 165.711928] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.731348] NMI backtrace for cpu 95
> [ 165.731538] CPU 95
> [ 165.731648] Modules linked in:
> [ 165.751183]
> [ 165.751280] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 165.771263] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 165.771659] RSP: 0000:ffff88103e6d5ee8 EFLAGS: 00000246
> [ 165.791267] RAX: 0000000000000000 RBX: ffff88103e6d5fd8 RCX: 0000000000000000
> [ 165.811054] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6dc560
> [ 165.811363] RBP: ffff88103e6d5ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 165.831174] R10: 0000000000000000 R11: ffff88103e6d5ed8 R12: 000000000000005f
> [ 165.850938] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 165.851262] FS: 0000000000000000(0000) GS:ffff88407fc00000(0000) knlGS:0000000000000000
> [ 165.871114] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 165.871382] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 165.891216] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 165.910931] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 165.911268] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6d4000, task ffff88103e6dc560)
> [ 165.931078] Stack:
> [ 165.931169] ffff88103e6d4000 ffffffff82525380 ffff88103e6d5f28 ffffffff810391f0
> [ 165.951035] ffff88103e6d5f18 e9750d6e03254eac 0000000000000000 0000000000000000
> [ 165.970777] ffff88103e6d5f48 ffffffff81c064f3 0000000000000000 26997d219409388f
> [ 165.971192] Call Trace:
> [ 165.990653] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 165.990873] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 165.991119] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 166.031254] Call Trace:
> [ 166.031394] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 166.050708] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 166.050969] NMI backtrace for cpu 89
> [ 166.070579] CPU 89
> [ 166.070699] Modules linked in:
> [ 166.070907]
> [ 166.070993] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 166.090981] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 166.111649] RSP: 0000:ffff88103e657ee8 EFLAGS: 00000246
> [ 166.130385] RAX: 0000000000000000 RBX: ffff88103e657fd8 RCX: 0000000000000000
> [ 166.130813] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e65c560
> [ 166.151071] RBP: ffff88103e657ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 166.151435] R10: 0000000000000000 R11: ffff88103e657ed8 R12: 0000000000000059
> [ 166.170873] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 166.190468] FS: 0000000000000000(0000) GS:ffff88407f000000(0000) knlGS:0000000000000000
> [ 166.210462] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 166.210829] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 166.230237] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 166.230596] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 166.250492] Process kworker/0:1 (pid: 0, threadinfo ffff88103e656000, task ffff88103e65c560)
> [ 166.270185] Stack:
> [ 166.270303] ffff88103e656000 ffffffff82525380 ffff88103e657f28 ffffffff810391f0
> [ 166.290094] ffff88103e657f18 99a1b0cf9e9d3ebb 0000000000000000 0000000000000000
> [ 166.290483] ffff88103e657f48 ffffffff81c064f3 0000000000000000 25270db18ea19940
> [ 166.310257] Call Trace:
> [ 166.310402] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 166.329985] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 166.330351] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 166.370192] Call Trace:
> [ 166.370349] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 166.370593] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 166.390088] NMI backtrace for cpu 92
> [ 166.390299] CPU 92
> [ 166.390439] Modules linked in:
> [ 166.410000]
> [ 166.410077] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 166.430006] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 166.449777] RSP: 0000:ffff88103e6a1ee8 EFLAGS: 00000246
> [ 166.450048] RAX: 0000000000000000 RBX: ffff88103e6a1fd8 RCX: 0000000000000000
> [ 166.469820] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e69c560
> [ 166.470176] RBP: ffff88103e6a1ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 166.489932] R10: 0000000000000000 R11: ffff88103e6a1ed8 R12: 000000000000005c
> [ 166.509692] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 166.510114] FS: 0000000000000000(0000) GS:ffff88407f600000(0000) knlGS:0000000000000000
> [ 166.529838] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 166.549585] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 166.549971] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 166.570648] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 166.589433] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6a0000, task ffff88103e69c560)
> [ 166.589840] Stack:
> [ 166.589953] ffff88103e6a0000 ffffffff82525380 ffff88103e6a1f28 ffffffff810391f0
> [ 166.609811] ffff88103e6a1f18 18a3b18c2c15ed65 0000000000000000 0000000000000000
> [ 166.629556] ffff88103e6a1f48 ffffffff81c064f3 0000000000000000 6118231e21489cba
> [ 166.649294] Call Trace:
> [ 166.649445] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 166.649739] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 166.669370] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 166.690108] Call Trace:
> [ 166.709251] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 166.709555] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 166.729162] NMI backtrace for cpu 93
> [ 166.729336] CPU 93
> [ 166.729424] Modules linked in:
> [ 166.729601]
> [ 166.729704] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 166.769090] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 166.769509] RSP: 0000:ffff88103e6a7ee8 EFLAGS: 00000246
> [ 166.789274] RAX: 0000000000000000 RBX: ffff88103e6a7fd8 RCX: 0000000000000000
> [ 166.789705] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6ac560
> [ 166.809219] RBP: ffff88103e6a7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 166.829023] R10: 0000000000000000 R11: ffff88103e6a7ed8 R12: 000000000000005d
> [ 166.829467] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 166.859237] FS: 0000000000000000(0000) GS:ffff88407f800000(0000) knlGS:0000000000000000
> [ 166.879116] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 166.879403] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 166.899175] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 166.899576] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 166.919321] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6a6000, task ffff88103e6ac560)
> [ 166.949119] Stack:
> [ 166.949207] ffff88103e6a6000 ffffffff82525380 ffff88103e6a7f28 ffffffff810391f0
> [ 166.968960] ffff88103e6a7f18 12bbac2f2ac6563c 0000000000000000 0000000000000000
> [ 166.969388] ffff88103e6a7f48 ffffffff81c064f3 0000000000000000 2c0129df7ca7b3c8
> [ 166.989083] Call Trace:
> [ 166.989191] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 167.008734] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 167.009075] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 167.049004] Call Trace:
> [ 167.049162] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 167.068530] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 167.068821] NMI backtrace for cpu 98
> [ 167.069031] CPU 98
> [ 167.088467] Modules linked in:
> [ 167.088698]
> [ 167.088799] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 167.108739] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 167.129860] RSP: 0000:ffff88103e719ee8 EFLAGS: 00000246
> [ 167.130128] RAX: 0000000000000000 RBX: ffff88103e719fd8 RCX: 0000000000000000
> [ 167.148569] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e70a2b0
> [ 167.168308] RBP: ffff88103e719ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 167.168679] R10: 0000000000000000 R11: ffff88103e719ed8 R12: 0000000000000062
> [ 167.189402] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 167.208443] FS: 0000000000000000(0000) GS:ffff88507f200000(0000) knlGS:0000000000000000
> [ 167.208868] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 167.228381] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 167.228772] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 167.252818] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 167.268203] Process kworker/0:1 (pid: 0, threadinfo ffff88103e718000, task ffff88103e70a2b0)
> [ 167.268701] Stack:
> [ 167.288075] ffff88103e718000 ffffffff82525380 ffff88103e719f28 ffffffff810391f0
> [ 167.288543] ffff88103e719f18 c68544f99cc20e07 0000000000000000 0000000000000000
> [ 167.308301] ffff88103e719f48 ffffffff81c064f3 0000000000000000 886503d07d16a597
> [ 167.328040] Call Trace:
> [ 167.328170] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 167.328468] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 167.348094] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 167.387896] Call Trace:
> [ 167.388015] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 167.388281] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 167.407899] NMI backtrace for cpu 99
> [ 167.408051] CPU 99
> [ 167.408148] Modules linked in:
> [ 167.428828]
> [ 167.428905] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 167.447787] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 167.448164] RSP: 0000:ffff88103e71fee8 EFLAGS: 00000246
> [ 167.467839] RAX: 0000000000000000 RBX: ffff88103e71ffd8 RCX: 0000000000000000
> [ 167.487619] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e72a2b0
> [ 167.488003] RBP: ffff88103e71fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 167.507722] R10: 0000000000000000 R11: ffff88103e71fed8 R12: 0000000000000063
> [ 167.508045] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 167.527801] FS: 0000000000000000(0000) GS:ffff88507f400000(0000) knlGS:0000000000000000
> [ 167.547668] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 167.547980] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 167.567717] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 167.587475] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 167.587866] Process kworker/0:1 (pid: 0, threadinfo ffff88103e71e000, task ffff88103e72a2b0)
> [ 167.607715] Stack:
> [ 167.607847] ffff88103e71e000 ffffffff82525380 ffff88103e71ff28 ffffffff810391f0
> [ 167.627601] ffff88103e71ff18 d1b7ebbe272fa90f 0000000000000000 0000000000000000
> [ 167.647336] ffff88103e71ff48 ffffffff81c064f3 0000000000000000 4e968abbf0e34374
> [ 167.647776] Call Trace:
> [ 167.647891] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 167.667525] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 167.667907] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 167.707784] Call Trace:
> [ 167.707902] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 167.727340] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 167.727612] NMI backtrace for cpu 88
> [ 167.747140] CPU 88
> [ 167.747258] Modules linked in:
> [ 167.747441]
> [ 167.747520] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 167.767452] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 167.787360] RSP: 0000:ffff88103e651ee8 EFLAGS: 00000246
> [ 167.787650] RAX: 0000000000000000 RBX: ffff88103e651fd8 RCX: 0000000000000000
> [ 167.807319] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e64c560
> [ 167.827053] RBP: ffff88103e651ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 167.827415] R10: 0000000000000000 R11: ffff88103e651ed8 R12: 0000000000000058
> [ 167.848237] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 167.866944] FS: 0000000000000000(0000) GS:ffff88407ee00000(0000) knlGS:0000000000000000
> [ 167.867406] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 167.887050] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 167.906823] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 167.907276] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 167.926921] Process kworker/0:1 (pid: 0, threadinfo ffff88103e650000, task ffff88103e64c560)
> [ 167.946733] Stack:
> [ 167.946836] ffff88103e650000 ffffffff82525380 ffff88103e651f28 ffffffff810391f0
> [ 167.966640] ffff88103e651f18 864a7adc0ed319da 0000000000000000 0000000000000000
> [ 167.967117] ffff88103e651f48 ffffffff81c064f3 0000000000000000 43de25d7d634e3af
> [ 167.986821] Call Trace:
> [ 167.986961] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.006546] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 168.006909] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 168.046737] Call Trace:
> [ 168.046907] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.047123] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 168.072366] NMI backtrace for cpu 91
> [ 168.072590] CPU 91
> [ 168.072697] Modules linked in:
> [ 168.086500]
> [ 168.086574] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 168.106603] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 168.126361] RSP: 0000:ffff88103e68bee8 EFLAGS: 00000246
> [ 168.126617] RAX: 0000000000000000 RBX: ffff88103e68bfd8 RCX: 0000000000000000
> [ 168.146509] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e684560
> [ 168.146838] RBP: ffff88103e68bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 168.166733] R10: 0000000000000000 R11: ffff88103e68bed8 R12: 000000000000005b
> [ 168.186481] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 168.186845] FS: 0000000000000000(0000) GS:ffff88407f400000(0000) knlGS:0000000000000000
> [ 168.206569] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 168.226269] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 168.226658] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 168.246454] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 168.266014] Process kworker/0:1 (pid: 0, threadinfo ffff88103e68a000, task ffff88103e684560)
> [ 168.266386] Stack:
> [ 168.266491] ffff88103e68a000 ffffffff82525380 ffff88103e68bf28 ffffffff810391f0
> [ 168.286319] ffff88103e68bf18 dacf332056ee6f33 0000000000000000 0000000000000000
> [ 168.306112] ffff88103e68bf48 ffffffff81c064f3 0000000000000000 1e902c36a88edc60
> [ 168.306508] Call Trace:
> [ 168.325977] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.326224] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 168.345962] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 168.366617] Call Trace:
> [ 168.385787] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.386052] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 168.405723] NMI backtrace for cpu 94
> [ 168.405913] CPU 94
> [ 168.406002] Modules linked in:
> [ 168.406238]
> [ 168.406339] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 168.426256] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 168.446018] RSP: 0000:ffff88103e6b7ee8 EFLAGS: 00000246
> [ 168.465683] RAX: 0000000000000000 RBX: ffff88103e6b7fd8 RCX: 0000000000000000
> [ 168.466109] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6cc560
> [ 168.485868] RBP: ffff88103e6b7ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 168.505608] R10: 0000000000000000 R11: ffff88103e6b7ed8 R12: 000000000000005e
> [ 168.505956] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 168.525703] FS: 0000000000000000(0000) GS:ffff88407fa00000(0000) knlGS:0000000000000000
> [ 168.545537] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 168.545788] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 168.565561] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 168.565992] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 168.585669] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6b6000, task ffff88103e6cc560)
> [ 168.605513] Stack:
> [ 168.605644] ffff88103e6b6000 ffffffff82525380 ffff88103e6b7f28 ffffffff810391f0
> [ 168.625416] ffff88103e6b7f18 a9fa8bc8dff9b80c 0000000000000000 0000000000000000
> [ 168.625823] ffff88103e6b7f48 ffffffff81c064f3 0000000000000000 37bc997f211760a8
> [ 168.645570] Call Trace:
> [ 168.645708] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.665305] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 168.665604] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 168.706857] Call Trace:
> [ 168.706977] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.725229] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 168.725492] NMI backtrace for cpu 90
> [ 168.725664] CPU 90
> [ 168.745087] Modules linked in:
> [ 168.745301]
> [ 168.745387] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 168.765351] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 168.785094] RSP: 0000:ffff88103e66dee8 EFLAGS: 00000246
> [ 168.785415] RAX: 0000000000000000 RBX: ffff88103e66dfd8 RCX: 0000000000000000
> [ 168.805151] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e67c560
> [ 168.824885] RBP: ffff88103e66def8 R08: 0000000000000000 R09: 0000000000000000
> [ 168.825303] R10: 0000000000000000 R11: ffff88103e66ded8 R12: 000000000000005a
> [ 168.845106] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 168.845509] FS: 0000000000000000(0000) GS:ffff88407f200000(0000) knlGS:0000000000000000
> [ 168.865194] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 168.884911] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 168.885293] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 168.904978] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 168.924769] Process kworker/0:1 (pid: 0, threadinfo ffff88103e66c000, task ffff88103e67c560)
> [ 168.925292] Stack:
> [ 168.944713] ffff88103e66c000 ffffffff82525380 ffff88103e66df28 ffffffff810391f0
> [ 168.945128] ffff88103e66df18 6f19056750a6e749 0000000000000000 0000000000000000
> [ 168.964888] ffff88103e66df48 ffffffff81c064f3 0000000000000000 66829d37004aca91
> [ 168.984629] Call Trace:
> [ 168.984750] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 168.985016] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 169.004665] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 169.044433] Call Trace:
> [ 169.044580] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 169.044858] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 169.064498] NMI backtrace for cpu 100
> [ 169.064677] CPU 100
> [ 169.064810] Modules linked in:
> [ 169.065076]
> [ 169.084413] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 169.104420] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 169.104894] RSP: 0000:ffff88103e73dee8 EFLAGS: 00000246
> [ 169.124467] RAX: 0000000000000000 RBX: ffff88103e73dfd8 RCX: 0000000000000000
> [ 169.144339] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7322b0
> [ 169.144767] RBP: ffff88103e73def8 R08: 0000000000000000 R09: 0000000000000000
> [ 169.164487] R10: 0000000000000000 R11: ffff88103e73ded8 R12: 0000000000000064
> [ 169.164878] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 169.184760] FS: 0000000000000000(0000) GS:ffff88507f600000(0000) knlGS:0000000000000000
> [ 169.204573] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 169.204908] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 169.224623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 169.244215] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 169.244549] Process kworker/0:1 (pid: 0, threadinfo ffff88103e73c000, task ffff88103e7322b0)
> [ 169.264442] Stack:
> [ 169.264535] ffff88103e73c000 ffffffff82525380 ffff88103e73df28 ffffffff810391f0
> [ 169.284155] ffff88103e73df18 ec3d40b388e1a820 0000000000000000 0000000000000000
> [ 169.303948] ffff88103e73df48 ffffffff81c064f3 0000000000000000 7819af42c6415509
> [ 169.304385] Call Trace:
> [ 169.304503] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 169.324093] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 169.324354] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 169.364412] Call Trace:
> [ 169.364559] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 169.384024] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 169.384305] NMI backtrace for cpu 103
> [ 169.403770] CPU 103
> [ 169.403898] Modules linked in:
> [ 169.404109]
> [ 169.404195] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 169.424087] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 169.443877] RSP: 0000:ffff88103e781ee8 EFLAGS: 00000246
> [ 169.444177] RAX: 0000000000000000 RBX: ffff88103e781fd8 RCX: 0000000000000000
> [ 169.463989] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7722b0
> [ 169.483686] RBP: ffff88103e781ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 169.484057] R10: 0000000000000000 R11: ffff88103e781ed8 R12: 0000000000000067
> [ 169.503796] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 169.523581] FS: 0000000000000000(0000) GS:ffff88507fc00000(0000) knlGS:0000000000000000
> [ 169.524098] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 169.543656] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 169.563441] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 169.563857] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 169.583539] Process kworker/0:1 (pid: 0, threadinfo ffff88103e780000, task ffff88103e7722b0)
> [ 169.603357] Stack:
> [ 169.603495] ffff88103e780000 ffffffff82525380 ffff88103e781f28 ffffffff810391f0
> [ 169.623253] ffff88103e781f18 1edcc44015d74379 0000000000000000 0000000000000000
> [ 169.623711] ffff88103e781f48 ffffffff81c064f3 0000000000000000 308c6b70699dc251
> [ 169.643418] Call Trace:
> [ 169.643554] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 169.663182] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 169.663444] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 169.703340] Call Trace:
> [ 169.703458] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 169.703758] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 169.723298] NMI backtrace for cpu 101
> [ 169.723463] CPU 101
> [ 169.723565] Modules linked in:
> [ 169.743120]
> [ 169.743197] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 169.763192] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 169.782980] RSP: 0000:ffff88103e753ee8 EFLAGS: 00000246
> [ 169.783243] RAX: 0000000000000000 RBX: ffff88103e753fd8 RCX: 0000000000000000
> [ 169.803029] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7422b0
> [ 169.803351] RBP: ffff88103e753ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 169.823148] R10: 0000000000000000 R11: ffff88103e753ed8 R12: 0000000000000065
> [ 169.842867] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 169.843256] FS: 0000000000000000(0000) GS:ffff88507f800000(0000) knlGS:0000000000000000
> [ 169.863068] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 169.882739] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 169.883120] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 169.902849] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 169.922642] Process kworker/0:1 (pid: 0, threadinfo ffff88103e752000, task ffff88103e7422b0)
> [ 169.923101] Stack:
> [ 169.923202] ffff88103e752000 ffffffff82525380 ffff88103e753f28 ffffffff810391f0
> [ 169.942961] ffff88103e753f18 e51c0349f6f87d31 0000000000000000 0000000000000000
> [ 169.962742] ffff88103e753f48 ffffffff81c064f3 0000000000000000 cf452dd59fbee8f3
> [ 169.963148] Call Trace:
> [ 169.982623] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 169.982891] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.004075] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 170.023238] Call Trace:
> [ 170.042434] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 170.042747] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.062353] NMI backtrace for cpu 97
> [ 170.062556] CPU 97
> [ 170.062675] Modules linked in:
> [ 170.062851]
> [ 170.062927] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 170.082837] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 170.102659] RSP: 0000:ffff88103e703ee8 EFLAGS: 00000246
> [ 170.122332] RAX: 0000000000000000 RBX: ffff88103e703fd8 RCX: 0000000000000000
> [ 170.122652] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6fa2b0
> [ 170.142414] RBP: ffff88103e703ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 170.162327] R10: 0000000000000000 R11: ffff88103e703ed8 R12: 0000000000000061
> [ 170.162671] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 170.182464] FS: 0000000000000000(0000) GS:ffff88507f000000(0000) knlGS:0000000000000000
> [ 170.202302] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 170.202604] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 170.222348] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 170.222675] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 170.242449] Process kworker/0:1 (pid: 0, threadinfo ffff88103e702000, task ffff88103e6fa2b0)
> [ 170.262272] Stack:
> [ 170.262399] ffff88103e702000 ffffffff82525380 ffff88103e703f28 ffffffff810391f0
> [ 170.282044] ffff88103e703f18 5b6b76da7595c5ea 0000000000000000 0000000000000000
> [ 170.282455] ffff88103e703f48 ffffffff81c064f3 0000000000000000 ef6e93bff8ea7807
> [ 170.302166] Call Trace:
> [ 170.302286] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 170.321949] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.322203] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 170.362204] Call Trace:
> [ 170.362331] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 170.381831] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.382118] NMI backtrace for cpu 96
> [ 170.382272] CPU 96
> [ 170.401728] Modules linked in:
> [ 170.401964]
> [ 170.403103] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 170.421996] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 170.441753] RSP: 0000:ffff88103e6edee8 EFLAGS: 00000246
> [ 170.442088] RAX: 0000000000000000 RBX: ffff88103e6edfd8 RCX: 0000000000000000
> [ 170.461779] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e6e4560
> [ 170.481516] RBP: ffff88103e6edef8 R08: 0000000000000000 R09: 0000000000000000
> [ 170.481904] R10: 0000000000000000 R11: ffff88103e6eded8 R12: 0000000000000060
> [ 170.501646] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 170.502047] FS: 0000000000000000(0000) GS:ffff88507ee00000(0000) knlGS:0000000000000000
> [ 170.521877] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 170.541472] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 170.541835] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 170.561716] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 170.581394] Process kworker/0:1 (pid: 0, threadinfo ffff88103e6ec000, task ffff88103e6e4560)
> [ 170.581929] Stack:
> [ 170.601331] ffff88103e6ec000 ffffffff82525380 ffff88103e6edf28 ffffffff810391f0
> [ 170.601707] ffff88103e6edf18 38f7cba7c3e92042 0000000000000000 0000000000000000
> [ 170.621515] ffff88103e6edf48 ffffffff81c064f3 0000000000000000 4bed50ad4b39cf1b
> [ 170.642398] Call Trace:
> [ 170.642538] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 170.642787] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.661307] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 170.701074] Call Trace:
> [ 170.701196] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 170.701468] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.721107] NMI backtrace for cpu 102
> [ 170.721292] CPU 102
> [ 170.721387] Modules linked in:
> [ 170.721562]
> [ 170.741029] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 170.761064] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 170.761494] RSP: 0000:ffff88103e763ee8 EFLAGS: 00000246
> [ 170.781065] RAX: 0000000000000000 RBX: ffff88103e763fd8 RCX: 0000000000000000
> [ 170.800849] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e75a2b0
> [ 170.801247] RBP: ffff88103e763ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 170.820994] R10: 0000000000000000 R11: ffff88103e763ed8 R12: 0000000000000066
> [ 170.821358] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 170.841164] FS: 0000000000000000(0000) GS:ffff88507fa00000(0000) knlGS:0000000000000000
> [ 170.860923] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 170.861258] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 170.880954] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 170.900721] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 170.901102] Process kworker/0:1 (pid: 0, threadinfo ffff88103e762000, task ffff88103e75a2b0)
> [ 170.920879] Stack:
> [ 170.920990] ffff88103e762000 ffffffff82525380 ffff88103e763f28 ffffffff810391f0
> [ 170.940808] ffff88103e763f18 73ced03514241cf0 0000000000000000 0000000000000000
> [ 170.970733] ffff88103e763f48 ffffffff81c064f3 0000000000000000 e2284a4bb68b9280
> [ 170.971094] Call Trace:
> [ 170.971262] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 170.991230] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 170.991552] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 171.031179] Call Trace:
> [ 171.031294] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 171.060699] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 171.060986] NMI backtrace for cpu 108
> [ 171.080475] CPU 108
> [ 171.080595] Modules linked in:
> [ 171.080795]
> [ 171.080875] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 171.100803] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 171.120445] RSP: 0000:ffff88103e001ee8 EFLAGS: 00000246
> [ 171.120747] RAX: 0000000000000000 RBX: ffff88103e001fd8 RCX: 0000000000000000
> [ 171.140516] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7d22b0
> [ 171.160264] RBP: ffff88103e001ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 171.160674] R10: 0000000000000000 R11: ffff88103e001ed8 R12: 000000000000006c
> [ 171.180617] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 171.200160] FS: 0000000000000000(0000) GS:ffff88607f600000(0000) knlGS:0000000000000000
> [ 171.200691] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 171.220319] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 171.241231] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 171.241544] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 171.270577] Process kworker/0:1 (pid: 0, threadinfo ffff88103e000000, task ffff88103e7d22b0)
> [ 171.290096] Stack:
> [ 171.290222] ffff88103e000000 ffffffff82525380 ffff88103e001f28 ffffffff810391f0
> [ 171.310082] ffff88103e001f18 cf3d086b299fbe80 0000000000000000 0000000000000000
> [ 171.310487] ffff88103e001f48 ffffffff81c064f3 0000000000000000 b23c24525439199e
> [ 171.330119] Call Trace:
> [ 171.330227] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 171.349871] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 171.350165] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 171.389904] Call Trace:
> [ 171.390044] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 171.390310] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 171.409838] NMI backtrace for cpu 104
> [ 171.410012] CPU 104
> [ 171.410110] Modules linked in:
> [ 171.429702]
> [ 171.429788] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 171.449766] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 171.469562] RSP: 0000:ffff88103e787ee8 EFLAGS: 00000246
> [ 171.469810] RAX: 0000000000000000 RBX: ffff88103e787fd8 RCX: 0000000000000000
> [ 171.489590] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e78a2b0
> [ 171.490008] RBP: ffff88103e787ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 171.509654] R10: 0000000000000000 R11: ffff88103e787ed8 R12: 0000000000000068
> [ 171.529460] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 171.529840] FS: 0000000000000000(0000) GS:ffff88607ee00000(0000) knlGS:0000000000000000
> [ 171.549642] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 171.569323] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 171.569679] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 171.589412] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 171.609206] Process kworker/0:1 (pid: 0, threadinfo ffff88103e786000, task ffff88103e78a2b0)
> [ 171.609667] Stack:
> [ 171.609759] ffff88103e786000 ffffffff82525380 ffff88103e787f28 ffffffff810391f0
> [ 171.629575] ffff88103e787f18 dbaa9279e483322f 0000000000000000 0000000000000000
> [ 171.649302] ffff88103e787f48 ffffffff81c064f3 0000000000000000 83aabbf70c51afbb
> [ 171.649704] Call Trace:
> [ 171.669186] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 171.669449] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 171.689113] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 171.709807] Call Trace:
> [ 171.728985] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 171.729222] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 171.748934] NMI backtrace for cpu 106
> [ 171.749093] CPU 106
> [ 171.749194] Modules linked in:
> [ 171.749370]
> [ 171.749456] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 171.769411] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 171.789292] RSP: 0000:ffff88103e7b3ee8 EFLAGS: 00000246
> [ 171.808904] RAX: 0000000000000000 RBX: ffff88103e7b3fd8 RCX: 0000000000000000
> [ 171.809327] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7aa2b0
> [ 171.829106] RBP: ffff88103e7b3ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 171.848782] R10: 0000000000000000 R11: ffff88103e7b3ed8 R12: 000000000000006a
> [ 171.849165] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 171.868898] FS: 0000000000000000(0000) GS:ffff88607f200000(0000) knlGS:0000000000000000
> [ 171.888714] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 171.888998] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 171.908746] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 171.909098] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 171.928948] Process kworker/0:1 (pid: 0, threadinfo ffff88103e7b2000, task ffff88103e7aa2b0)
> [ 171.948728] Stack:
> [ 171.948868] ffff88103e7b2000 ffffffff82525380 ffff88103e7b3f28 ffffffff810391f0
> [ 171.968618] ffff88103e7b3f18 f9eadedccb2cfa6c 0000000000000000 0000000000000000
> [ 171.969035] ffff88103e7b3f48 ffffffff81c064f3 0000000000000000 8ad0f992b6df189f
> [ 171.988770] Call Trace:
> [ 171.988923] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 172.008510] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 172.008819] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 172.048800] Call Trace:
> [ 172.048950] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 172.068323] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 172.068603] NMI backtrace for cpu 120
> [ 172.068801] CPU 120
> [ 172.088292] Modules linked in:
> [ 172.088468]
> [ 172.088556] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 172.108503] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 172.128297] RSP: 0000:ffff88103e0f9ee8 EFLAGS: 00000246
> [ 172.128556] RAX: 0000000000000000 RBX: ffff88103e0f9fd8 RCX: 0000000000000000
> [ 172.148296] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0f0000
> [ 172.169509] RBP: ffff88103e0f9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 172.169837] R10: 0000000000000000 R11: ffff88103e0f9ed8 R12: 0000000000000078
> [ 172.188344] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 172.208120] FS: 0000000000000000(0000) GS:ffff88807e800000(0000) knlGS:0000000000000000
> [ 172.208485] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 172.228278] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 172.228602] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 172.248367] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 172.268087] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0f8000, task ffff88103e0f0000)
> [ 172.268481] Stack:
> [ 172.287908] ffff88103e0f8000 ffffffff82525380 ffff88103e0f9f28 ffffffff810391f0
> [ 172.288303] ffff88103e0f9f18 9d1fa4a474be1569 0000000000000000 0000000000000000
> [ 172.308077] ffff88103e0f9f48 ffffffff81c064f3 0000000000000000 33afc904d5c4ddb1
> [ 172.327846] Call Trace:
> [ 172.327960] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 172.328222] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 172.347875] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 172.387661] Call Trace:
> [ 172.387809] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 172.388043] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 172.407680] NMI backtrace for cpu 122
> [ 172.407854] CPU 122
> [ 172.407943] Modules linked in:
> [ 172.427557]
> [ 172.427635] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 172.447626] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 172.447999] RSP: 0000:ffff88103e10dee8 EFLAGS: 00000246
> [ 172.467688] RAX: 0000000000000000 RBX: ffff88103e10dfd8 RCX: 0000000000000000
> [ 172.487433] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e118000
> [ 172.487759] RBP: ffff88103e10def8 R08: 0000000000000000 R09: 0000000000000000
> [ 172.507538] R10: 0000000000000000 R11: ffff88103e10ded8 R12: 000000000000007a
> [ 172.507849] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 172.527659] FS: 0000000000000000(0000) GS:ffff88807ec00000(0000) knlGS:0000000000000000
> [ 172.547478] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 172.547724] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 172.567497] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 172.587308] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 172.587631] Process kworker/0:1 (pid: 0, threadinfo ffff88103e10c000, task ffff88103e118000)
> [ 172.607455] Stack:
> [ 172.607560] ffff88103e10c000 ffffffff82525380 ffff88103e10df28 ffffffff810391f0
> [ 172.627383] ffff88103e10df18 5932e94ddd4561db 0000000000000000 0000000000000000
> [ 172.647168] ffff88103e10df48 ffffffff81c064f3 0000000000000000 44418799c393f5ed
> [ 172.647585] Call Trace:
> [ 172.647693] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 172.667275] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 172.667530] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 172.707611] Call Trace:
> [ 172.707731] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 172.727104] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 172.727355] NMI backtrace for cpu 124
> [ 172.746972] CPU 124
> [ 172.747068] Modules linked in:
> [ 172.747246]
> [ 172.747339] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 172.767337] RIP: 0010:[<ffffffff810591f0>] [<ffffffff810591f0>] __x2apic_send_IPI_dest+0x1f/0x21
> [ 172.787134] RSP: 0000:ffff88807f003d58 EFLAGS: 00000046
> [ 172.806808] RAX: 0000000000000c00 RBX: 0000000000000092 RCX: 0000000000000830
> [ 172.807113] RDX: 00000000000f0008 RSI: 0000000000000002 RDI: 00000000000f0008
> [ 172.826918] RBP: ffff88807f003d58 R08: 0000000000000040 R09: ffffffff82525388
> [ 172.846685] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000c7d0
> [ 172.847006] R13: 0000000000000002 R14: ffffffff82525380 R15: 000000000000007d
> [ 172.866832] FS: 0000000000000000(0000) GS:ffff88807f000000(0000) knlGS:0000000000000000
> [ 172.886643] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 172.886914] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 172.906649] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 172.906973] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 172.926749] Process kworker/0:1 (pid: 0, threadinfo ffff88103e142000, task ffff88103e148000)
> [ 172.946616] Stack:
> [ 172.946709] ffff88807f003d98 ffffffff81059440 ffff88807f003d78 0000000000002710
> [ 172.966529] ffffffff823cbf80 0000000000000010 ffffffff823cbf80 7fffffffffffffff
> [ 172.966927] ffff88807f003da8 ffffffff8105949a ffff88807f003dc8 ffffffff810560de
> [ 172.986675] Call Trace:
> [ 172.986810] <IRQ>
> [ 172.986920] [<ffffffff81059440>] x2apic_send_IPI_mask+0x52/0x95
> [ 173.006568] [<ffffffff8105949a>] x2apic_send_IPI_all+0x17/0x19
> [ 173.006841] [<ffffffff810560de>] arch_trigger_all_cpu_backtrace+0x5a/0x8c
> [ 173.026610] [<ffffffff810d2940>] print_other_cpu_stall+0x11c/0x12f
> [ 173.046349] [<ffffffff810d2a38>] check_cpu_stall+0xe5/0xe9
> [ 173.046627] [<ffffffff810d2a64>] __rcu_pending+0x28/0x10f
> [ 173.066334] [<ffffffff810d317d>] rcu_check_callbacks+0x8b/0xb8
> [ 173.066598] [<ffffffff8108e1e7>] update_process_times+0x41/0x77
> [ 173.086305] [<ffffffff810aaa21>] tick_sched_timer+0x77/0x97
> [ 173.086584] [<ffffffff8109ff4f>] __run_hrtimer+0xde/0x167
> [ 173.106246] [<ffffffff810aa9aa>] ? tick_nohz_handler+0xd6/0xd6
> [ 173.106532] [<ffffffff810a08e7>] hrtimer_interrupt+0xd1/0x1bd
> [ 173.126284] [<ffffffff81c17b4c>] smp_apic_timer_interrupt+0x88/0x9b
> [ 173.126629] [<ffffffff81c16c93>] apic_timer_interrupt+0x13/0x20
> [ 173.146261] <EOI>
> [ 173.146373] [<ffffffff810409eb>] ? mwait_idle+0xef/0x169
> [ 173.166082] [<ffffffff810409e2>] ? mwait_idle+0xe6/0x169
> [ 173.166344] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 173.186053] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 173.186318] Code: 66 66 90 48 8b 05 09 5b bd 00 c9 c3 55 48 89 e5 66 66 66 66 90 b9 30 08 00 00 89 f0 09 d0 80 ce 04 83 fe 02 0f 44 c2 89 fa 0f 30 <c9> c3 55 48 89 e5 66 66 66 66 90 b8 01 00 00 00 c9 c3 55 48 89
> [ 173.226163] Call Trace:
> [ 173.226268] <IRQ> [<ffffffff81059440>] x2apic_send_IPI_mask+0x52/0x95
> [ 173.247016] [<ffffffff8105949a>] x2apic_send_IPI_all+0x17/0x19
> [ 173.247322] [<ffffffff810560de>] arch_trigger_all_cpu_backtrace+0x5a/0x8c
> [ 173.266474] [<ffffffff810d2940>] print_other_cpu_stall+0x11c/0x12f
> [ 173.266773] [<ffffffff810d2a38>] check_cpu_stall+0xe5/0xe9
> [ 173.286065] [<ffffffff810d2a64>] __rcu_pending+0x28/0x10f
> [ 173.286330] [<ffffffff810d317d>] rcu_check_callbacks+0x8b/0xb8
> [ 173.305908] [<ffffffff8108e1e7>] update_process_times+0x41/0x77
> [ 173.306176] [<ffffffff810aaa21>] tick_sched_timer+0x77/0x97
> [ 173.325886] [<ffffffff8109ff4f>] __run_hrtimer+0xde/0x167
> [ 173.326138] [<ffffffff810aa9aa>] ? tick_nohz_handler+0xd6/0xd6
> [ 173.345862] [<ffffffff810a08e7>] hrtimer_interrupt+0xd1/0x1bd
> [ 173.346116] [<ffffffff81c17b4c>] smp_apic_timer_interrupt+0x88/0x9b
> [ 173.366028] [<ffffffff81c16c93>] apic_timer_interrupt+0x13/0x20
> [ 173.385593] <EOI> [<ffffffff810409eb>] ? mwait_idle+0xef/0x169
> [ 173.385898] [<ffffffff810409e2>] ? mwait_idle+0xe6/0x169
> [ 173.405586] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 173.405848] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 173.425526] NMI backtrace for cpu 107
> [ 173.425699] CPU 107
> [ 173.425811] Modules linked in:
> [ 173.425975]
> [ 173.426053] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 173.465445] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 173.465906] RSP: 0000:ffff88103e7c9ee8 EFLAGS: 00000246
> [ 173.485511] RAX: 0000000000000000 RBX: ffff88103e7c9fd8 RCX: 0000000000000000
> [ 173.485865] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7c22b0
> [ 173.505646] RBP: ffff88103e7c9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 173.525384] R10: 0000000000000000 R11: ffff88103e7c9ed8 R12: 000000000000006b
> [ 173.525737] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 173.545617] FS: 0000000000000000(0000) GS:ffff88607f400000(0000) knlGS:0000000000000000
> [ 173.565311] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 173.565598] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 173.585404] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 173.585754] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 173.605629] Process kworker/0:1 (pid: 0, threadinfo ffff88103e7c8000, task ffff88103e7c22b0)
> [ 173.625290] Stack:
> [ 173.625392] ffff88103e7c8000 ffffffff82525380 ffff88103e7c9f28 ffffffff810391f0
> [ 173.645191] ffff88103e7c9f18 1eef3661459e59f7 0000000000000000 0000000000000000
> [ 173.645606] ffff88103e7c9f48 ffffffff81c064f3 0000000000000000 d110419bdf32a799
> [ 173.665391] Call Trace:
> [ 173.665508] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 173.685087] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 173.685361] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 173.725355] Call Trace:
> [ 173.725467] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 173.744945] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 173.745235] NMI backtrace for cpu 111
> [ 173.745414] CPU 111
> [ 173.764882] Modules linked in:
> [ 173.765068]
> [ 173.765154] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 173.785160] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 173.804932] RSP: 0000:ffff88103e03bee8 EFLAGS: 00000246
> [ 173.805225] RAX: 0000000000000000 RBX: ffff88103e03bfd8 RCX: 0000000000000000
> [ 173.824951] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0322b0
> [ 173.844721] RBP: ffff88103e03bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 173.845119] R10: 0000000000000000 R11: ffff88103e03bed8 R12: 000000000000006f
> [ 173.864820] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 173.884582] FS: 0000000000000000(0000) GS:ffff88607fc00000(0000) knlGS:0000000000000000
> [ 173.885062] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 173.904719] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 173.905102] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 173.924773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 173.944577] Process kworker/0:1 (pid: 0, threadinfo ffff88103e03a000, task ffff88103e0322b0)
> [ 173.964428] Stack:
> [ 173.964536] ffff88103e03a000 ffffffff82525380 ffff88103e03bf28 ffffffff810391f0
> [ 173.964963] ffff88103e03bf18 cbecccf87c85ab64 0000000000000000 0000000000000000
> [ 173.984657] ffff88103e03bf48 ffffffff81c064f3 0000000000000000 7d8c76c902581a69
> [ 174.004435] Call Trace:
> [ 174.004556] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 174.004812] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 174.024453] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 174.064286] Call Trace:
> [ 174.064421] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 174.064684] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 174.084288] NMI backtrace for cpu 110
> [ 174.084454] CPU 110
> [ 174.084554] Modules linked in:
> [ 174.104155]
> [ 174.104246] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 174.124218] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 174.124669] RSP: 0000:ffff88103e01dee8 EFLAGS: 00000246
> [ 174.144277] RAX: 0000000000000000 RBX: ffff88103e01dfd8 RCX: 0000000000000000
> [ 174.164040] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0122b0
> [ 174.164388] RBP: ffff88103e01def8 R08: 0000000000000000 R09: 0000000000000000
> [ 174.184207] R10: 0000000000000000 R11: ffff88103e01ded8 R12: 000000000000006e
> [ 174.203949] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 174.204325] FS: 0000000000000000(0000) GS:ffff88607fa00000(0000) knlGS:0000000000000000
> [ 174.224273] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 174.224543] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 174.244436] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 174.264550] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 174.264896] Process kworker/0:1 (pid: 0, threadinfo ffff88103e01c000, task ffff88103e0122b0)
> [ 174.284294] Stack:
> [ 174.284387] ffff88103e01c000 ffffffff82525380 ffff88103e01df28 ffffffff810391f0
> [ 174.304005] ffff88103e01df18 e1449d45a00f96cd 0000000000000000 0000000000000000
> [ 174.323764] ffff88103e01df48 ffffffff81c064f3 0000000000000000 02e0a2113f244cc6
> [ 174.324159] Call Trace:
> [ 174.343634] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 174.343882] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 174.344158] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 174.384288] Call Trace:
> [ 174.384425] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 174.403684] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 174.403923] NMI backtrace for cpu 113
> [ 174.423560] CPU 113
> [ 174.423663] Modules linked in:
> [ 174.423861]
> [ 174.423934] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 174.443871] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 174.463695] RSP: 0000:ffff88103e069ee8 EFLAGS: 00000246
> [ 174.483352] RAX: 0000000000000000 RBX: ffff88103e069fd8 RCX: 0000000000000000
> [ 174.483659] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e060000
> [ 174.503485] RBP: ffff88103e069ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 174.503801] R10: 0000000000000000 R11: ffff88103e069ed8 R12: 0000000000000071
> [ 174.523541] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 174.543378] FS: 0000000000000000(0000) GS:ffff88707f000000(0000) knlGS:0000000000000000
> [ 174.543737] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 174.563452] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 174.583232] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 174.583557] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 174.603297] Process kworker/0:1 (pid: 0, threadinfo ffff88103e068000, task ffff88103e060000)
> [ 174.623192] Stack:
> [ 174.623287] ffff88103e068000 ffffffff82525380 ffff88103e069f28 ffffffff810391f0
> [ 174.643073] ffff88103e069f18 d8cc0a5355bbe9f7 0000000000000000 0000000000000000
> [ 174.643484] ffff88103e069f48 ffffffff81c064f3 0000000000000000 43f4297694e0e1e5
> [ 174.663235] Call Trace:
> [ 174.663371] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 174.682994] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 174.683234] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 174.723167] Call Trace:
> [ 174.723277] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 174.723498] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 174.743169] NMI backtrace for cpu 109
> [ 174.743359] CPU 109
> [ 174.743470] Modules linked in:
> [ 174.762927]
> [ 174.763016] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 174.783023] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 174.802779] RSP: 0000:ffff88103e007ee8 EFLAGS: 00000246
> [ 174.803081] RAX: 0000000000000000 RBX: ffff88103e007fd8 RCX: 0000000000000000
> [ 174.822839] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e00a2b0
> [ 174.823230] RBP: ffff88103e007ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 174.843067] R10: 0000000000000000 R11: ffff88103e007ed8 R12: 000000000000006d
> [ 174.862777] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 174.863173] FS: 0000000000000000(0000) GS:ffff88607f800000(0000) knlGS:0000000000000000
> [ 174.882958] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 174.902533] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 174.902933] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 174.922697] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 174.942450] Process kworker/0:1 (pid: 0, threadinfo ffff88103e006000, task ffff88103e00a2b0)
> [ 174.942950] Stack:
> [ 174.962373] ffff88103e006000 ffffffff82525380 ffff88103e007f28 ffffffff810391f0
> [ 174.962788] ffff88103e007f18 b33cc9080e000452 0000000000000000 0000000000000000
> [ 174.982569] ffff88103e007f48 ffffffff81c064f3 0000000000000000 f44d519eb03a804f
> [ 175.002295] Call Trace:
> [ 175.002431] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 175.002716] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 175.022347] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 175.043133] Call Trace:
> [ 175.062263] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 175.062507] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 175.082302] NMI backtrace for cpu 118
> [ 175.082476] CPU 118
> [ 175.082569] Modules linked in:
> [ 175.082750]
> [ 175.102358] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 175.122313] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 175.122690] RSP: 0000:ffff88103e0c3ee8 EFLAGS: 00000246
> [ 175.142254] RAX: 0000000000000000 RBX: ffff88103e0c3fd8 RCX: 0000000000000000
> [ 175.142563] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0c8000
> [ 175.162634] RBP: ffff88103e0c3ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 175.182892] R10: 0000000000000000 R11: ffff88103e0c3ed8 R12: 0000000000000076
> [ 175.183236] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 175.202258] FS: 0000000000000000(0000) GS:ffff88707fa00000(0000) knlGS:0000000000000000
> [ 175.222203] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 175.222483] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 175.242067] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 175.261990] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 175.262321] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0c2000, task ffff88103e0c8000)
> [ 175.282550] Stack:
> [ 175.282651] ffff88103e0c2000 ffffffff82525380 ffff88103e0c3f28 ffffffff810391f0
> [ 175.301851] ffff88103e0c3f18 baf4c5c5e3b24d3d 0000000000000000 0000000000000000
> [ 175.302273] ffff88103e0c3f48 ffffffff81c064f3 0000000000000000 93acd84e907831b6
> [ 175.321996] Call Trace:
> [ 175.322117] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 175.341999] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 175.342255] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 175.401990] Call Trace:
> [ 175.402123] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 175.421541] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 175.421796] NMI backtrace for cpu 115
> [ 175.441389] CPU 115
> [ 175.441514] Modules linked in:
> [ 175.441702]
> [ 175.441786] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 175.461710] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 175.481505] RSP: 0000:ffff88103e07fee8 EFLAGS: 00000246
> [ 175.481767] RAX: 0000000000000000 RBX: ffff88103e07ffd8 RCX: 0000000000000000
> [ 175.501529] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e090000
> [ 175.521310] RBP: ffff88103e07fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 175.521652] R10: 0000000000000000 R11: ffff88103e07fed8 R12: 0000000000000073
> [ 175.541406] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 175.561195] FS: 0000000000000000(0000) GS:ffff88707f400000(0000) knlGS:0000000000000000
> [ 175.561547] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 175.581297] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 175.581618] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 175.601376] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 175.621180] Process kworker/0:1 (pid: 0, threadinfo ffff88103e07e000, task ffff88103e090000)
> [ 175.641133] Stack:
> [ 175.641236] ffff88103e07e000 ffffffff82525380 ffff88103e07ff28 ffffffff810391f0
> [ 175.641635] ffff88103e07ff18 03a98b03e6b043dd 0000000000000000 0000000000000000
> [ 175.661262] ffff88103e07ff48 ffffffff81c064f3 0000000000000000 7da074131e177f44
> [ 175.681050] Call Trace:
> [ 175.681160] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 175.681405] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 175.701054] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 175.740906] Call Trace:
> [ 175.741016] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 175.741263] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 175.760890] NMI backtrace for cpu 114
> [ 175.761067] CPU 114
> [ 175.761164] Modules linked in:
> [ 175.780766]
> [ 175.780857] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 175.800802] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 175.801175] RSP: 0000:ffff88103e06fee8 EFLAGS: 00000246
> [ 175.820949] RAX: 0000000000000000 RBX: ffff88103e06ffd8 RCX: 0000000000000000
> [ 175.840618] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e070000
> [ 175.840955] RBP: ffff88103e06fef8 R08: 0000000000000000 R09: 0000000000000000
> [ 175.860755] R10: 0000000000000000 R11: ffff88103e06fed8 R12: 0000000000000072
> [ 175.880628] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 175.880976] FS: 0000000000000000(0000) GS:ffff88707f200000(0000) knlGS:0000000000000000
> [ 175.900667] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 175.900937] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 175.920699] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 175.940501] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 175.940824] Process kworker/0:1 (pid: 0, threadinfo ffff88103e06e000, task ffff88103e070000)
> [ 175.960649] Stack:
> [ 175.960757] ffff88103e06e000 ffffffff82525380 ffff88103e06ff28 ffffffff810391f0
> [ 175.980616] ffff88103e06ff18 fd4b0bbf3b80b10e 0000000000000000 0000000000000000
> [ 176.000385] ffff88103e06ff48 ffffffff81c064f3 0000000000000000 c5d770f1aa6234e7
> [ 176.000798] Call Trace:
> [ 176.020263] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 176.020475] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 176.040168] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 176.060840] Call Trace:
> [ 176.060957] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 176.080298] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 176.080576] NMI backtrace for cpu 119
> [ 176.100160] CPU 119
> [ 176.100260] Modules linked in:
> [ 176.100446]
> [ 176.100549] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 176.120486] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 176.140276] RSP: 0000:ffff88103e0e9ee8 EFLAGS: 00000246
> [ 176.159969] RAX: 0000000000000000 RBX: ffff88103e0e9fd8 RCX: 0000000000000000
> [ 176.160268] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0e0000
> [ 176.180076] RBP: ffff88103e0e9ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 176.200108] R10: 0000000000000000 R11: ffff88103e0e9ed8 R12: 0000000000000077
> [ 176.200422] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 176.219971] FS: 0000000000000000(0000) GS:ffff88707fc00000(0000) knlGS:0000000000000000
> [ 176.240055] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 176.240329] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 176.260001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 176.260323] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 176.280178] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0e8000, task ffff88103e0e0000)
> [ 176.299884] Stack:
> [ 176.299995] ffff88103e0e8000 ffffffff82525380 ffff88103e0e9f28 ffffffff810391f0
> [ 176.319666] ffff88103e0e9f18 6e7842f8b5c2a888 0000000000000000 0000000000000000
> [ 176.320047] ffff88103e0e9f48 ffffffff81c064f3 0000000000000000 b8cf95fcbad3979a
> [ 176.339828] Call Trace:
> [ 176.339961] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 176.359579] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 176.359827] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 176.399776] Call Trace:
> [ 176.399884] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 176.400113] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 176.419675] NMI backtrace for cpu 121
> [ 176.419851] CPU 121
> [ 176.439359] Modules linked in:
> [ 176.439537]
> [ 176.439615] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 176.459586] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 176.479535] RSP: 0000:ffff88103e0ffee8 EFLAGS: 00000246
> [ 176.479766] RAX: 0000000000000000 RBX: ffff88103e0fffd8 RCX: 0000000000000000
> [ 176.499399] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e100000
> [ 176.499713] RBP: ffff88103e0ffef8 R08: 0000000000000000 R09: 0000000000000000
> [ 176.519517] R10: 0000000000000000 R11: ffff88103e0ffed8 R12: 0000000000000079
> [ 176.539297] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 176.539598] FS: 0000000000000000(0000) GS:ffff88807ea00000(0000) knlGS:0000000000000000
> [ 176.559431] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 176.579153] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 176.579454] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 176.599256] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 176.619029] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0fe000, task ffff88103e100000)
> [ 176.619417] Stack:
> [ 176.638976] ffff88103e0fe000 ffffffff82525380 ffff88103e0fff28 ffffffff810391f0
> [ 176.639365] ffff88103e0fff18 d84b97c200bc65b0 0000000000000000 0000000000000000
> [ 176.659164] ffff88103e0fff48 ffffffff81c064f3 0000000000000000 0bb7ccf3334ed6b1
> [ 176.678914] Call Trace:
> [ 176.679045] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 176.679291] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 176.698947] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 176.719685] Call Trace:
> [ 176.738829] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 176.739063] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 176.758748] NMI backtrace for cpu 112
> [ 176.758915] CPU 112
> [ 176.759030] Modules linked in:
> [ 176.759200]
> [ 176.778702] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 176.798700] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 176.799094] RSP: 0000:ffff88103e04bee8 EFLAGS: 00000246
> [ 176.818735] RAX: 0000000000000000 RBX: ffff88103e04bfd8 RCX: 0000000000000000
> [ 176.819047] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e058000
> [ 176.838845] RBP: ffff88103e04bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 176.858608] R10: 0000000000000000 R11: ffff88103e04bed8 R12: 0000000000000070
> [ 176.858935] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 176.878744] FS: 0000000000000000(0000) GS:ffff88707ee00000(0000) knlGS:0000000000000000
> [ 176.898581] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 176.898838] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 176.918587] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 176.938368] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 176.938680] Process kworker/0:1 (pid: 0, threadinfo ffff88103e04a000, task ffff88103e058000)
> [ 176.958545] Stack:
> [ 176.958648] ffff88103e04a000 ffffffff82525380 ffff88103e04bf28 ffffffff810391f0
> [ 176.978461] ffff88103e04bf18 eddbed60db20a548 0000000000000000 0000000000000000
> [ 176.978874] ffff88103e04bf48 ffffffff81c064f3 0000000000000000 cc0f66974ad24309
> [ 176.998604] Call Trace:
> [ 176.998722] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 177.018375] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 177.018628] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 177.058662] Call Trace:
> [ 177.058783] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 177.078171] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 177.078406] NMI backtrace for cpu 117
> [ 177.098030] CPU 117
> [ 177.098133] Modules linked in:
> [ 177.098321]
> [ 177.098413] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 177.118343] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 177.138165] RSP: 0000:ffff88103e0b3ee8 EFLAGS: 00000246
> [ 177.138419] RAX: 0000000000000000 RBX: ffff88103e0b3fd8 RCX: 0000000000000000
> [ 177.158150] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0b8000
> [ 177.177953] RBP: ffff88103e0b3ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 177.178278] R10: 0000000000000000 R11: ffff88103e0b3ed8 R12: 0000000000000075
> [ 177.198098] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 177.217970] FS: 0000000000000000(0000) GS:ffff88707f800000(0000) knlGS:0000000000000000
> [ 177.218347] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 177.238072] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 177.257983] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 177.258296] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 177.277854] Process kworker/0:1 (pid: 0, threadinfo ffff88103e0b2000, task ffff88103e0b8000)
> [ 177.297799] Stack:
> [ 177.297902] ffff88103e0b2000 ffffffff82525380 ffff88103e0b3f28 ffffffff810391f0
> [ 177.298272] ffff88103e0b3f18 3c05f38b456efa63 0000000000000000 0000000000000000
> [ 177.317959] ffff88103e0b3f48 ffffffff81c064f3 0000000000000000 c20981672419ff31
> [ 177.337683] Call Trace:
> [ 177.337806] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 177.338044] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 177.357713] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 177.397558] Call Trace:
> [ 177.397668] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 177.397902] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 177.417546] NMI backtrace for cpu 116
> [ 177.417702] CPU 116
> [ 177.417815] Modules linked in:
> [ 177.437415]
> [ 177.437495] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 177.457463] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 177.477254] RSP: 0000:ffff88103e09dee8 EFLAGS: 00000246
> [ 177.477534] RAX: 0000000000000000 RBX: ffff88103e09dfd8 RCX: 0000000000000000
> [ 177.497285] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e0a0000
> [ 177.497622] RBP: ffff88103e09def8 R08: 0000000000000000 R09: 0000000000000000
> [ 177.517372] R10: 0000000000000000 R11: ffff88103e09ded8 R12: 0000000000000074
> [ 177.537173] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 177.537477] FS: 0000000000000000(0000) GS:ffff88707f600000(0000) knlGS:0000000000000000
> [ 177.557300] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 177.557546] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 177.577367] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 177.597127] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 177.597452] Process kworker/0:1 (pid: 0, threadinfo ffff88103e09c000, task ffff88103e0a0000)
> [ 177.617307] Stack:
> [ 177.617408] ffff88103e09c000 ffffffff82525380 ffff88103e09df28 ffffffff810391f0
> [ 177.637217] ffff88103e09df18 f5a140315aa62005 0000000000000000 0000000000000000
> [ 177.656995] ffff88103e09df48 ffffffff81c064f3 0000000000000000 45979eb37cc9aeb0
> [ 177.657430] Call Trace:
> [ 177.676927] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 177.677154] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 177.696815] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 177.717521] Call Trace:
> [ 177.717651] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 177.736943] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 177.737188] NMI backtrace for cpu 125
> [ 177.756805] CPU 125
> [ 177.756897] Modules linked in:
> [ 177.757087]
> [ 177.757163] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 177.777127] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 177.796916] RSP: 0000:ffff88103e159ee8 EFLAGS: 00000246
> [ 177.816590] RAX: 0000000000000000 RBX: ffff88103e159fd8 RCX: 0000000000000000
> [ 177.816924] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e150000
> [ 177.836719] RBP: ffff88103e159ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 177.856497] R10: 0000000000000000 R11: ffff88103e159ed8 R12: 000000000000007d
> [ 177.856810] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 177.876696] FS: 0000000000000000(0000) GS:ffff88807f200000(0000) knlGS:0000000000000000
> [ 177.896413] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 177.896670] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 177.916474] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 177.916785] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 177.936571] Process kworker/0:1 (pid: 0, threadinfo ffff88103e158000, task ffff88103e150000)
> [ 177.956438] Stack:
> [ 177.956528] ffff88103e158000 ffffffff82525380 ffff88103e159f28 ffffffff810391f0
> [ 177.976430] ffff88103e159f18 2ccd79cfdd34ac9e 0000000000000000 0000000000000000
> [ 177.976840] ffff88103e159f48 ffffffff81c064f3 0000000000000000 10ccf89162e4ca4a
> [ 177.996459] Call Trace:
> [ 177.996608] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.016238] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 178.016516] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 178.056435] Call Trace:
> [ 178.056550] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.076039] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 178.076359] NMI backtrace for cpu 123
> [ 178.076517] CPU 123
> [ 178.096023] Modules linked in:
> [ 178.096213]
> [ 178.096289] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 178.116243] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 178.136006] RSP: 0000:ffff88103e12bee8 EFLAGS: 00000246
> [ 178.136240] RAX: 0000000000000000 RBX: ffff88103e12bfd8 RCX: 0000000000000000
> [ 178.156040] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e130000
> [ 178.156368] RBP: ffff88103e12bef8 R08: 0000000000000000 R09: 0000000000000000
> [ 178.176117] R10: 0000000000000000 R11: ffff88103e12bed8 R12: 000000000000007b
> [ 178.195946] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 178.196293] FS: 0000000000000000(0000) GS:ffff88807ee00000(0000) knlGS:0000000000000000
> [ 178.216128] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 178.235947] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 178.236261] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 178.255967] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 178.276291] Process kworker/0:1 (pid: 0, threadinfo ffff88103e12a000, task ffff88103e130000)
> [ 178.276664] Stack:
> [ 178.295771] ffff88103e12a000 ffffffff82525380 ffff88103e12bf28 ffffffff810391f0
> [ 178.296163] ffff88103e12bf18 a4a843542a24023c 0000000000000000 0000000000000000
> [ 178.315916] ffff88103e12bf48 ffffffff81c064f3 0000000000000000 e267619a78df27fa
> [ 178.335708] Call Trace:
> [ 178.335834] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.336074] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 178.355568] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 178.376576] Call Trace:
> [ 178.395482] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.395696] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 178.415403] NMI backtrace for cpu 105
> [ 178.415563] CPU 105
> [ 178.415663] Modules linked in:
> [ 178.415851]
> [ 178.435334] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 178.455580] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 178.455977] RSP: 0000:ffff88103e79dee8 EFLAGS: 00000246
> [ 178.475387] RAX: 0000000000000000 RBX: ffff88103e79dfd8 RCX: 0000000000000000
> [ 178.475802] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e7922b0
> [ 178.495462] RBP: ffff88103e79def8 R08: 0000000000000000 R09: 0000000000000000
> [ 178.515274] R10: 0000000000000000 R11: ffff88103e79ded8 R12: 0000000000000069
> [ 178.515657] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 178.535391] FS: 0000000000000000(0000) GS:ffff88607f000000(0000) knlGS:0000000000000000
> [ 178.555181] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 178.555495] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 178.575260] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 178.595017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 178.595353] Process kworker/0:1 (pid: 0, threadinfo ffff88103e79c000, task ffff88103e7922b0)
> [ 178.615200] Stack:
> [ 178.615322] ffff88103e79c000 ffffffff82525380 ffff88103e79df28 ffffffff810391f0
> [ 178.635134] ffff88103e79df18 7cc4089422c6d371 0000000000000000 0000000000000000
> [ 178.654849] ffff88103e79df48 ffffffff81c064f3 0000000000000000 6ca04c2dc270ff18
> [ 178.655284] Call Trace:
> [ 178.655424] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.674979] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 178.675290] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 178.715289] Call Trace:
> [ 178.715406] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.734804] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 178.735144] NMI backtrace for cpu 126
> [ 178.754941] CPU 126
> [ 178.755032] Modules linked in:
> [ 178.755201]
> [ 178.755290] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 178.775008] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 178.794775] RSP: 0000:ffff88103e171ee8 EFLAGS: 00000246
> [ 178.795025] RAX: 0000000000000000 RBX: ffff88103e171fd8 RCX: 0000000000000000
> [ 178.814809] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e168000
> [ 178.834586] RBP: ffff88103e171ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 178.834896] R10: 0000000000000000 R11: ffff88103e171ed8 R12: 000000000000007e
> [ 178.854681] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 178.874557] FS: 0000000000000000(0000) GS:ffff88807f400000(0000) knlGS:0000000000000000
> [ 178.874911] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 178.894550] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 178.914336] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 178.914650] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 178.934465] Process kworker/0:1 (pid: 0, threadinfo ffff88103e170000, task ffff88103e168000)
> [ 178.954274] Stack:
> [ 178.954381] ffff88103e170000 ffffffff82525380 ffff88103e171f28 ffffffff810391f0
> [ 178.954773] ffff88103e171f18 5b8357237cb2986a 0000000000000000 0000000000000000
> [ 178.974574] ffff88103e171f48 ffffffff81c064f3 0000000000000000 5d4840ef688c0c41
> [ 178.994347] Call Trace:
> [ 178.994476] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 178.994722] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 179.014351] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 179.054246] Call Trace:
> [ 179.054376] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 179.054598] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 179.074151] NMI backtrace for cpu 127
> [ 179.074317] CPU 127
> [ 179.074407] Modules linked in:
> [ 179.094038]
> [ 179.094119] Pid: 0, comm: kworker/0:1 Not tainted 2.6.39-rc6-tip-yh-03030-gfb0b6a9 #985 Oracle Corporation Sun Fire X4800/
> [ 179.114091] RIP: 0010:[<ffffffff810409eb>] [<ffffffff810409eb>] mwait_idle+0xef/0x169
> [ 179.133888] RSP: 0000:ffff88103e177ee8 EFLAGS: 00000246
> [ 179.134130] RAX: 0000000000000000 RBX: ffff88103e177fd8 RCX: 0000000000000000
> [ 179.153912] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88103e188000
> [ 179.154239] RBP: ffff88103e177ef8 R08: 0000000000000000 R09: 0000000000000000
> [ 179.174044] R10: 0000000000000000 R11: ffff88103e177ed8 R12: 000000000000007f
> [ 179.193801] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 179.194108] FS: 0000000000000000(0000) GS:ffff88807f600000(0000) knlGS:0000000000000000
> [ 179.213996] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 179.233745] CR2: 0000000000000000 CR3: 000000000239d000 CR4: 00000000000006e0
> [ 179.234072] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 179.254037] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 179.254359] Process kworker/0:1 (pid: 0, threadinfo ffff88103e176000, task ffff88103e188000)
> [ 179.273981] Stack:
> [ 179.274099] ffff88103e176000 ffffffff82525380 ffff88103e177f28 ffffffff810391f0
> [ 179.294154] ffff88103e177f18 23c99be1e1fa11ca 0000000000000000 0000000000000000
> [ 179.313859] ffff88103e177f48 ffffffff81c064f3 0000000000000000 75b2778518a8799c
> [ 179.314286] Call Trace:
> [ 179.333745] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 179.333980] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
> [ 179.353757] Code: 00 00 48 89 d1 48 8d 83 38 e0 ff ff 0f 01 c8 0f ae f0 48 8b 83 38 e0 ff ff a8 08 75 10 e8 ec fd 06 00 31 c0 48 89 c1 fb 0f 01 c9 <eb> 06 e8 dc fd 06 00 fb 8b 05 0f 0a 4e 01 65 44 8b 24 25 30 c7
> [ 179.374328] Call Trace:
> [ 179.374439] [<ffffffff810391f0>] cpu_idle+0xae/0xcc
> [ 179.393814] [<ffffffff81c064f3>] start_secondary+0xbd/0xbf
>
[-- Attachment #2: collectdebugfs.sh --]
[-- Type: application/x-sh, Size: 1715 bytes --]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 4:54 ` Paul E. McKenney
@ 2011-05-11 6:03 ` Yinghai Lu
2011-05-11 6:42 ` Yinghai Lu
` (2 subsequent siblings)
3 siblings, 0 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 6:03 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/10/2011 09:54 PM, Paul E. McKenney wrote:
> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>>> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>>>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>>>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>>>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>>>>>
>>>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>>>>>
>>>>>>>> Hello, Ingo,
>>>>>>>>
>>>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>>>>>> of the commits. There are a number of smaller features and bug fixes.
>>>>>>>> All have been sent to LKML in the following batches:
>>>>>>>>
>>>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>>>>>
>>>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>>>>>> Acked-by:s, and I will deal with these later.
>>>>>>>>
>>>>>>>> These channges are available in the -rcu git repository at:
>>>>>>>>
>>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>>>>>
>>>>>>> Pulled, thanks a lot Paul!
>>>>>>>
>>>>>>
>>>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>>>>>
>>>>>> after hard code to enable rcu_cpu_stall_suppress
>>>>>>
>>>>>> Index: linux-2.6/kernel/rcutree.c
>>>>>> ===================================================================
>>>>>> --- linux-2.6.orig/kernel/rcutree.c
>>>>>> +++ linux-2.6/kernel/rcutree.c
>>>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>>>>>> module_param(qhimark, int, 0);
>>>>>> module_param(qlowmark, int, 0);
>>>>>>
>>>>>> -int rcu_cpu_stall_suppress __read_mostly;
>>>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>>>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>>>>>
>>>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>>>>>
>>>>>> will get system hang after pnp ACPI init.
>>>>>
>>>>> Could you please send the stack traces from the RCU CPU stall? Also,
>>>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>>>>>
>>>>> Thanx, Paul
>>>>
>>>> Do not have time to bisect it at this point.
>>>
>>> Could you please send the stack traces from the RCU CPU stall?
>
> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
> Everyone is idle, except for CPU 124, which detected the stall, and
> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
> that looks like a backtrace error to me. The fact that it hangs if you
> disable RCU CPU stall detection leads me to believe that something real
> is being detected.
>
> This looks very similar to the situation people were seeing before
> ce31332d3c77532d6ea97ddcb475a2b02dd358b4 was applied, so I have attached
> the diagnostic script that helped track this down. Could you please
> enable CONFIG_RCU_TRACE, mount debugfs, and run the attached script,
> and send me the output? Please check to make sure that the script knows
> where you mounted debugfs, of course.
>
so which kernel that i should boot?
that is during boot stage.
assume at least i need to boot to shell to run your scripts.
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 4:54 ` Paul E. McKenney
2011-05-11 6:03 ` Yinghai Lu
@ 2011-05-11 6:42 ` Yinghai Lu
2011-05-11 20:13 ` Paul E. McKenney
2011-05-11 16:54 ` Yinghai Lu
2011-05-11 16:56 ` Yinghai Lu
3 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 6:42 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/10/2011 09:54 PM, Paul E. McKenney wrote:
> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>>> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>>>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>>>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>>>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>>>>>
>>>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>>>>>
>>>>>>>> Hello, Ingo,
>>>>>>>>
>>>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>>>>>> of the commits. There are a number of smaller features and bug fixes.
>>>>>>>> All have been sent to LKML in the following batches:
>>>>>>>>
>>>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>>>>>
>>>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>>>>>> Acked-by:s, and I will deal with these later.
>>>>>>>>
>>>>>>>> These channges are available in the -rcu git repository at:
>>>>>>>>
>>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>>>>>
>>>>>>> Pulled, thanks a lot Paul!
>>>>>>>
>>>>>>
>>>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>>>>>
>>>>>> after hard code to enable rcu_cpu_stall_suppress
>>>>>>
>>>>>> Index: linux-2.6/kernel/rcutree.c
>>>>>> ===================================================================
>>>>>> --- linux-2.6.orig/kernel/rcutree.c
>>>>>> +++ linux-2.6/kernel/rcutree.c
>>>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>>>>>> module_param(qhimark, int, 0);
>>>>>> module_param(qlowmark, int, 0);
>>>>>>
>>>>>> -int rcu_cpu_stall_suppress __read_mostly;
>>>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>>>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>>>>>
>>>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>>>>>
>>>>>> will get system hang after pnp ACPI init.
>>>>>
>>>>> Could you please send the stack traces from the RCU CPU stall? Also,
>>>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>>>>>
>>>>> Thanx, Paul
>>>>
>>>> Do not have time to bisect it at this point.
>>>
>>> Could you please send the stack traces from the RCU CPU stall?
>
> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
> Everyone is idle, except for CPU 124, which detected the stall, and
> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
> that looks like a backtrace error to me. The fact that it hangs if you
> disable RCU CPU stall detection leads me to believe that something real
> is being detected.
>
> This looks very similar to the situation people were seeing before
> ce31332d3c77532d6ea97ddcb475a2b02dd358b4 was applied, so I have attached
> the diagnostic script that helped track this down. Could you please
> enable CONFIG_RCU_TRACE, mount debugfs, and run the attached script,
> and send me the output? Please check to make sure that the script knows
> where you mounted debugfs, of course.
Wed May 11 14:33:35 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4164 gpnum=4165
rcu_bh: completed=-282 gpnum=18446744073709551334
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4164 g=4165 s=2 jfq=0 j=92fc nfqs=6792/nfqsng=0(6792) fqlh=737
ff/ff ..>.. 0:254 ^0
fdff/ffff ..>.. 0:15 ^0 fffc/ffff ..>.. 16:31 ^1 ffff/ffff ..>.. 32:47 ^2 ffff/ffff ..>.. 48:63 ^3 fdff/ffff ..>.. 64:79 ^4 ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
rcu_bh:
c=18446744073709551334 g=18446744073709551334 s=0 jfq=-1351 j=92fc nfqs=21/nfqsng=0(21) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2 0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>.. 80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=6072/72 dn=4 df=2570 of=0 ri=601 ql=0 b=10 ci=6119 co=0 ca=0
1 c=4163 g=4164 pq=0 pqc=4162 qp=1 dt=12470/0 dn=0 df=2722 of=0 ri=13 ql=0 b=10 ci=11239 co=0 ca=0
2 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=11168/0 dn=2 df=3326 of=0 ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1694/0 dn=0 df=3570 of=0 ri=6 ql=0 b=10 ci=123 co=0 ca=0
4 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1538/0 dn=2 df=3580 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
5 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1608/0 dn=0 df=3597 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
6 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1598/0 dn=2 df=3597 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
7 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1646/0 dn=2 df=3597 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
8 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=9158/0 dn=2 df=2368 of=0 ri=760 ql=0 b=10 ci=7223 co=0 ca=0
9 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=3131/1 dn=0 df=2685 of=0 ri=1 ql=2 b=10 ci=13631 co=0 ca=0
10 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=2020/0 dn=0 df=3444 of=0 ri=1 ql=0 b=10 ci=702 co=0 ca=0
11 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1760/0 dn=2 df=3590 of=0 ri=1 ql=0 b=10 ci=25 co=0 ca=0
12 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1800/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
13 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=3594/0 dn=2 df=3598 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
14 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1704/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
15 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1648/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
16 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=18235/1 dn=0 df=2734 of=0 ri=103 ql=2 b=10 ci=3533 co=0 ca=0
17 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=1273/1 dn=0 df=2736 of=0 ri=1 ql=11 b=10 ci=11685 co=0 ca=0
18 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1678/0 dn=0 df=3444 of=0 ri=1 ql=0 b=10 ci=665 co=0 ca=0
19 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=2044/0 dn=0 df=3568 of=0 ri=1 ql=0 b=10 ci=121 co=0 ca=0
20 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=3666/0 dn=0 df=3596 of=0 ri=1 ql=0 b=10 ci=24 co=0 ca=0
21 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1788/0 dn=2 df=3598 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
22 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1726/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
23 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1720/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
24 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=14333/1 dn=0 df=2865 of=0 ri=1 ql=2 b=10 ci=3337 co=0 ca=0
25 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=1593/1 dn=0 df=2707 of=0 ri=1 ql=9 b=10 ci=19146 co=0 ca=0
26 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1786/0 dn=2 df=3427 of=0 ri=1 ql=0 b=10 ci=1035 co=0 ca=0
27 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1586/0 dn=2 df=3571 of=0 ri=1 ql=0 b=10 ci=157 co=0 ca=0
28 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1582/0 dn=0 df=3569 of=0 ri=1 ql=0 b=10 ci=92 co=0 ca=0
29 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1550/0 dn=2 df=3594 of=0 ri=1 ql=0 b=10 ci=14 co=0 ca=0
30 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1650/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
31 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1662/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
32 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=5537/1 dn=0 df=2756 of=0 ri=97 ql=2 b=10 ci=3209 co=0 ca=0
33 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=1367/1 dn=0 df=2725 of=0 ri=1 ql=9 b=10 ci=17809 co=0 ca=0
34 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1784/0 dn=0 df=3474 of=0 ri=1 ql=0 b=10 ci=538 co=0 ca=0
35 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1670/0 dn=2 df=3577 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
36 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1682/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
37 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1664/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
38 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1622/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
39 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1664/0 dn=2 df=3598 of=0 ri=1 ql=0 b=10 ci=18 co=0 ca=0
40 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1846/0 dn=0 df=2958 of=0 ri=1 ql=0 b=10 ci=10276 co=0 ca=0
41 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1621/1 dn=0 df=2831 of=0 ri=1 ql=3 b=10 ci=16207 co=0 ca=0
42 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1764/0 dn=0 df=3503 of=0 ri=1 ql=0 b=10 ci=176 co=0 ca=0
43 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1706/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
44 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1654/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
45 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=2048/0 dn=0 df=3600 of=0 ri=1 ql=0 b=10 ci=12 co=0 ca=0
46 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1670/0 dn=2 df=3599 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
47 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1694/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
48 c=4151 g=4152 pq=1 pqc=4151 qp=0 dt=35230/0 dn=2 df=2620 of=0 ri=161 ql=0 b=10 ci=20305 co=0 ca=0
49 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1534/0 dn=0 df=2847 of=0 ri=15 ql=0 b=10 ci=28443 co=0 ca=0
50 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1910/0 dn=0 df=3461 of=0 ri=1 ql=0 b=10 ci=537 co=0 ca=0
51 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1994/0 dn=0 df=3564 of=0 ri=1 ql=0 b=10 ci=102 co=0 ca=0
52 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1834/0 dn=2 df=3584 of=0 ri=1 ql=0 b=10 ci=57 co=0 ca=0
53 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1630/0 dn=0 df=3596 of=0 ri=1 ql=0 b=10 ci=15 co=0 ca=0
54 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1608/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
55 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1680/0 dn=2 df=3599 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
56 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1650/0 dn=2 df=2887 of=0 ri=1 ql=0 b=10 ci=2645 co=0 ca=0
57 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1284/0 dn=0 df=2776 of=0 ri=1 ql=0 b=10 ci=11764 co=0 ca=0
58 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2056/0 dn=0 df=3462 of=0 ri=1 ql=0 b=10 ci=650 co=0 ca=0
59 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2004/0 dn=2 df=3591 of=0 ri=1 ql=0 b=10 ci=40 co=0 ca=0
60 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1686/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
61 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1598/0 dn=2 df=3599 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
62 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1686/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
63 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=6030/0 dn=0 df=3414 of=0 ri=30 ql=0 b=10 ci=150 co=0 ca=0
64 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=8338/0 dn=2 df=3180 of=0 ri=274 ql=0 b=10 ci=1436 co=0 ca=0
65 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2888/0 dn=0 df=3076 of=0 ri=6 ql=0 b=10 ci=5781 co=0 ca=0
66 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=3958/0 dn=0 df=3549 of=0 ri=6 ql=0 b=10 ci=131 co=0 ca=0
67 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1616/0 dn=0 df=3593 of=0 ri=1 ql=0 b=10 ci=26 co=0 ca=0
68 c=4155 g=4156 pq=1 pqc=4155 qp=0 dt=1622/0 dn=0 df=3597 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
69 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1672/0 dn=2 df=3597 of=0 ri=6 ql=0 b=10 ci=11 co=0 ca=0
70 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1656/0 dn=2 df=3598 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
71 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1588/0 dn=2 df=3598 of=0 ri=1 ql=0 b=10 ci=12 co=0 ca=0
72 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=6142/0 dn=4 df=3388 of=0 ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=7465/1 dn=0 df=2868 of=0 ri=3 ql=32 b=10 ci=72222 co=0 ca=0
74 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1690/0 dn=2 df=3570 of=0 ri=1 ql=0 b=10 ci=125 co=0 ca=0
75 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1680/0 dn=2 df=3597 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
76 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1678/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
77 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1666/0 dn=0 df=3597 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
78 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1714/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
79 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1620/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
80 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=2154/0 dn=2 df=3331 of=0 ri=1 ql=0 b=10 ci=590 co=0 ca=0
81 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=3648/0 dn=0 df=3130 of=0 ri=1 ql=0 b=10 ci=5193 co=0 ca=0
82 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1626/0 dn=2 df=3571 of=0 ri=1 ql=0 b=10 ci=106 co=0 ca=0
83 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1672/0 dn=2 df=3594 of=0 ri=1 ql=0 b=10 ci=22 co=0 ca=0
84 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1610/0 dn=2 df=3599 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
85 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1640/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
86 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1644/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
87 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1672/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
88 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1852/0 dn=0 df=3349 of=0 ri=1 ql=0 b=10 ci=884 co=0 ca=0
89 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=2814/0 dn=0 df=3230 of=0 ri=1 ql=0 b=10 ci=2572 co=0 ca=0
90 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1542/0 dn=0 df=3554 of=0 ri=1 ql=0 b=10 ci=154 co=0 ca=0
91 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1770/0 dn=2 df=3593 of=0 ri=1 ql=0 b=10 ci=20 co=0 ca=0
92 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1728/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
93 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1718/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
94 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1716/0 dn=2 df=3602 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
95 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1608/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
96 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1718/0 dn=2 df=3372 of=0 ri=1 ql=0 b=10 ci=493 co=0 ca=0
97 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2656/0 dn=0 df=3278 of=0 ri=1 ql=0 b=10 ci=1015 co=0 ca=0
98 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1720/0 dn=2 df=3592 of=0 ri=1 ql=0 b=10 ci=27 co=0 ca=0
99 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1640/0 dn=2 df=3599 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
100 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1728/0 dn=2 df=3598 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
101 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1704/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
102 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1690/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
103 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1696/0 dn=2 df=3597 of=0 ri=1 ql=0 b=10 ci=1923 co=0 ca=0
104 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1694/0 dn=2 df=3406 of=0 ri=1 ql=0 b=10 ci=387 co=0 ca=0
105 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2726/0 dn=0 df=3306 of=0 ri=1 ql=0 b=10 ci=20365 co=0 ca=0
106 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1814/0 dn=2 df=3589 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
107 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1722/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
108 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1650/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
109 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1662/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
110 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1714/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
111 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1626/0 dn=2 df=3599 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
112 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=7810/0 dn=0 df=3325 of=0 ri=36 ql=0 b=10 ci=613 co=0 ca=0
113 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=2392/0 dn=0 df=3277 of=0 ri=1 ql=0 b=10 ci=5505 co=0 ca=0
114 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1654/0 dn=0 df=3518 of=0 ri=1 ql=0 b=10 ci=497 co=0 ca=0
115 c=4138 g=4138 pq=1 pqc=4137 qp=0 dt=1566/0 dn=2 df=3589 of=0 ri=1 ql=0 b=10 ci=38 co=0 ca=0
116 c=4139 g=4140 pq=1 pqc=4139 qp=0 dt=1736/0 dn=2 df=3596 of=0 ri=1 ql=0 b=10 ci=34 co=0 ca=0
117 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1596/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=10 co=0 ca=0
118 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1638/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
119 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=2682/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
120 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=2234/0 dn=0 df=3397 of=0 ri=1 ql=0 b=10 ci=393 co=0 ca=0
121 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=2980/0 dn=0 df=2005 of=0 ri=1 ql=0 b=10 ci=48877 co=0 ca=0
122 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1554/0 dn=0 df=3527 of=0 ri=1 ql=0 b=10 ci=333 co=0 ca=0
123 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1630/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
124 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1682/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
125 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1650/0 dn=2 df=3600 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
126 c=4141 g=4142 pq=1 pqc=4141 qp=0 dt=2466/0 dn=2 df=3595 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
127 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1720/0 dn=2 df=3601 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=6072/72 dn=4 df=7 of=0 ri=0 ql=0 b=10 ci=6 co=0 ca=0
1 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=12470/0 dn=0 df=12 of=0 ri=0 ql=0 b=10 ci=4 co=0 ca=0
2 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=11168/0 dn=2 df=13 of=0 ri=0 ql=0 b=10 ci=11 co=0 ca=0
3 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1694/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
4 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1538/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
5 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1608/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
6 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1598/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
7 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1646/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
8 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=9158/0 dn=2 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
9 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=3131/1 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
10 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2020/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
11 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1760/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
12 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1800/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
13 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=3597/1 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
14 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1704/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
15 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1648/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
16 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=18235/1 dn=0 df=13 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
17 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1273/1 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
18 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1678/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
19 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2044/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
20 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=3666/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
21 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1788/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
22 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1726/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
23 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1720/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
24 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=14333/1 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
25 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1593/1 dn=0 df=15 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
26 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1786/0 dn=2 df=16 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
27 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1586/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
28 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1582/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
29 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1550/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
30 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
31 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1662/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
32 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=5537/1 dn=0 df=12 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
33 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1367/1 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
34 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1784/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
35 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1670/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
36 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1682/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
37 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1664/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
38 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1622/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
39 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1664/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
40 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1846/0 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
41 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1621/1 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
42 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1764/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
43 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1706/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
44 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1654/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
45 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2048/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
46 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1670/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
47 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1694/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
48 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=35230/0 dn=2 df=12 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
49 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1534/0 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
50 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1910/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
51 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1994/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
52 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1834/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
53 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1630/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
54 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1608/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
55 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
56 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
57 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1284/0 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
58 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2056/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
59 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2004/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
60 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1686/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
61 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1598/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
62 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1686/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
63 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=6030/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
64 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=8338/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
65 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2888/0 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
66 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=3958/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
67 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1616/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
68 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1622/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
69 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1672/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
70 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1656/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
71 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1588/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
72 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=6142/0 dn=4 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
73 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=7465/1 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
74 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1690/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
75 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
76 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1678/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
77 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1666/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
78 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1714/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
79 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1620/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
80 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2154/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
81 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=3648/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
82 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1626/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
83 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1672/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
84 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1610/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
85 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1640/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
86 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1644/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
87 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1672/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
88 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1852/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
89 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2814/0 dn=0 df=15 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
90 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1542/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
91 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1770/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
92 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1728/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
93 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
94 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1716/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
95 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1608/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
96 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
97 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=2656/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
98 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1720/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
99 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1640/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
100 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1728/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
101 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1704/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
102 c=18446744073709551332 g=18446744073709551332 pq=1 pqc=18446744073709551331 qp=0 dt=1690/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
103 c=18446744073709551332 g=18446744073709551332 pq=1 pqc=18446744073709551331 qp=0 dt=1696/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
104 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1694/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
105 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=2728/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
106 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1814/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
107 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1722/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
108 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1650/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
109 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1662/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
110 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1714/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
111 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1626/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
112 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=7810/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
113 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=2392/0 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
114 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1654/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
115 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1566/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
116 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1736/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
117 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1596/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
118 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1638/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
119 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2682/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
120 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2234/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
121 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2980/0 dn=0 df=14 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
122 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1554/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
123 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1630/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
124 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1682/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
125 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
126 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2466/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
127 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1720/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=24806 qsp=1521 rpq=6020 cbr=1 cng=1254 gpc=1128 gps=131 nf=683 nn=15589
1 np=26594 qsp=60 rpq=8004 cbr=5 cng=44 gpc=1620 gps=748 nf=430 nn=15743
2 np=23556 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1101 gps=715 nf=180 nn=14668
3 np=21600 qsp=29 rpq=6997 cbr=0 cng=5 gpc=909 gps=708 nf=96 nn=12885
4 np=21508 qsp=9 rpq=6943 cbr=0 cng=5 gpc=983 gps=729 nf=100 nn=12748
5 np=21376 qsp=28 rpq=6885 cbr=0 cng=2 gpc=875 gps=683 nf=109 nn=12822
6 np=21338 qsp=30 rpq=6845 cbr=0 cng=2 gpc=874 gps=689 nf=104 nn=12824
7 np=21298 qsp=32 rpq=6806 cbr=0 cng=2 gpc=981 gps=700 nf=107 nn=12702
8 np=30293 qsp=840 rpq=8100 cbr=1 cng=45 gpc=1657 gps=779 nf=590 nn=19121
9 np=25376 qsp=32 rpq=7677 cbr=1 cng=22 gpc=1742 gps=783 nf=171 nn=14980
10 np=21235 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1269 gps=806 nf=15 nn=12285
11 np=20409 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1063 gps=807 nf=5 nn=11903
12 np=20307 qsp=9 rpq=6574 cbr=0 cng=1 gpc=856 gps=755 nf=2 nn=12119
13 np=20785 qsp=9 rpq=6541 cbr=0 cng=1 gpc=909 gps=735 nf=4 nn=12595
14 np=20213 qsp=7 rpq=6482 cbr=0 cng=0 gpc=775 gps=758 nf=2 nn=12196
15 np=20176 qsp=9 rpq=6443 cbr=0 cng=0 gpc=870 gps=757 nf=3 nn=12103
16 np=28721 qsp=173 rpq=7569 cbr=0 cng=27 gpc=1742 gps=748 nf=262 nn=18373
17 np=25111 qsp=26 rpq=7251 cbr=1 cng=11 gpc=1709 gps=698 nf=205 nn=15236
18 np=20875 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1027 gps=681 nf=26 nn=12653
19 np=20173 qsp=11 rpq=6309 cbr=0 cng=1 gpc=895 gps=692 nf=8 nn=12268
20 np=19991 qsp=7 rpq=6224 cbr=0 cng=0 gpc=876 gps=671 nf=5 nn=12215
21 np=19928 qsp=9 rpq=6173 cbr=0 cng=0 gpc=869 gps=697 nf=5 nn=12184
22 np=19876 qsp=7 rpq=6122 cbr=0 cng=0 gpc=875 gps=696 nf=1 nn=12182
23 np=19829 qsp=7 rpq=6077 cbr=0 cng=0 gpc=873 gps=696 nf=2 nn=12181
24 np=23838 qsp=19 rpq=6839 cbr=0 cng=18 gpc=1548 gps=685 nf=167 nn=14581
25 np=24901 qsp=96 rpq=6902 cbr=12 cng=15 gpc=1692 gps=698 nf=261 nn=15321
26 np=20559 qsp=15 rpq=6148 cbr=0 cng=2 gpc=1044 gps=692 nf=14 nn=12659
27 np=19784 qsp=10 rpq=5927 cbr=0 cng=1 gpc=906 gps=691 nf=13 nn=12246
28 np=19762 qsp=7 rpq=5886 cbr=0 cng=1 gpc=905 gps=677 nf=9 nn=12284
29 np=19579 qsp=7 rpq=5813 cbr=0 cng=1 gpc=886 gps=685 nf=8 nn=12186
30 np=19504 qsp=6 rpq=5760 cbr=0 cng=1 gpc=873 gps=717 nf=2 nn=12151
31 np=19463 qsp=8 rpq=5714 cbr=0 cng=1 gpc=873 gps=700 nf=1 nn=12174
32 np=24626 qsp=164 rpq=6629 cbr=0 cng=17 gpc=1707 gps=741 nf=131 nn=15401
33 np=23699 qsp=37 rpq=6514 cbr=5 cng=7 gpc=1707 gps=733 nf=118 nn=14615
34 np=19922 qsp=8 rpq=5728 cbr=0 cng=4 gpc=994 gps=729 nf=14 nn=12453
35 np=19378 qsp=9 rpq=5558 cbr=0 cng=0 gpc=889 gps=727 nf=9 nn=12195
36 np=19221 qsp=8 rpq=5488 cbr=0 cng=0 gpc=875 gps=716 nf=6 nn=12136
37 np=19176 qsp=8 rpq=5442 cbr=0 cng=0 gpc=869 gps=721 nf=2 nn=12142
38 np=19130 qsp=7 rpq=5399 cbr=0 cng=0 gpc=873 gps=722 nf=4 nn=12132
39 np=19098 qsp=9 rpq=5357 cbr=0 cng=0 gpc=873 gps=720 nf=5 nn=12143
40 np=22183 qsp=29 rpq=6020 cbr=1 cng=7 gpc=1469 gps=784 nf=83 nn=13819
41 np=22788 qsp=66 rpq=6047 cbr=13 cng=13 gpc=1670 gps=661 nf=88 nn=14296
42 np=19432 qsp=8 rpq=5349 cbr=0 cng=0 gpc=1001 gps=793 nf=4 nn=12285
43 np=18929 qsp=8 rpq=5179 cbr=0 cng=0 gpc=950 gps=797 nf=5 nn=11998
44 np=18883 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1206 gps=767 nf=1 nn=11774
45 np=18844 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1199 gps=737 nf=3 nn=11813
46 np=18799 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1315 gps=801 nf=3 nn=11636
47 np=18746 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1050 gps=653 nf=3 nn=12042
48 np=25095 qsp=297 rpq=6163 cbr=3 cng=14 gpc=1687 gps=740 nf=503 nn=15985
49 np=22838 qsp=68 rpq=5689 cbr=3 cng=12 gpc=1596 gps=695 nf=116 nn=14727
50 np=19790 qsp=9 rpq=5021 cbr=0 cng=0 gpc=1017 gps=686 nf=96 nn=12970
51 np=19209 qsp=8 rpq=4858 cbr=0 cng=0 gpc=910 gps=679 nf=63 nn=12699
52 np=19011 qsp=8 rpq=4788 cbr=0 cng=0 gpc=889 gps=680 nf=48 nn=12606
53 np=19120 qsp=7 rpq=4730 cbr=0 cng=0 gpc=874 gps=682 nf=60 nn=12774
54 np=18397 qsp=8 rpq=4682 cbr=0 cng=0 gpc=869 gps=682 nf=9 nn=12155
55 np=18350 qsp=9 rpq=4635 cbr=0 cng=0 gpc=871 gps=678 nf=5 nn=12161
56 np=21837 qsp=11 rpq=5384 cbr=0 cng=10 gpc=1575 gps=701 nf=91 nn=14076
57 np=22322 qsp=27 rpq=5397 cbr=2 cng=11 gpc=1821 gps=707 nf=98 nn=14286
58 np=18955 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1217 gps=712 nf=21 nn=12346
59 np=18242 qsp=8 rpq=4469 cbr=0 cng=0 gpc=907 gps=706 nf=4 nn=12156
60 np=18162 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1030 gps=741 nf=6 nn=11973
61 np=18117 qsp=8 rpq=4369 cbr=0 cng=0 gpc=950 gps=688 nf=4 nn=12106
62 np=18075 qsp=7 rpq=4330 cbr=0 cng=0 gpc=872 gps=662 nf=7 nn=12204
63 np=19170 qsp=44 rpq=4583 cbr=0 cng=0 gpc=1019 gps=671 nf=133 nn=12764
64 np=22024 qsp=234 rpq=4773 cbr=0 cng=10 gpc=1344 gps=684 nf=67 nn=15146
65 np=20692 qsp=33 rpq=4833 cbr=0 cng=9 gpc=1402 gps=686 nf=192 nn=13570
66 np=18466 qsp=32 rpq=4222 cbr=0 cng=4 gpc=923 gps=636 nf=28 nn=12653
67 np=18101 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1149 gps=633 nf=25 nn=12172
68 np=18042 qsp=8 rpq=4072 cbr=0 cng=3 gpc=882 gps=666 nf=23 nn=12396
69 np=17993 qsp=30 rpq=4025 cbr=0 cng=2 gpc=919 gps=683 nf=29 nn=12335
70 np=17951 qsp=32 rpq=3979 cbr=0 cng=3 gpc=878 gps=676 nf=21 nn=12394
71 np=17901 qsp=8 rpq=3935 cbr=0 cng=3 gpc=931 gps=733 nf=33 nn=12266
72 np=20530 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1106 gps=628 nf=28 nn=14623
73 np=25529 qsp=10 rpq=4626 cbr=1 cng=311 gpc=1244 gps=606 nf=1401 nn=17340
74 np=17670 qsp=7 rpq=3829 cbr=0 cng=3 gpc=900 gps=775 nf=7 nn=12156
75 np=17494 qsp=8 rpq=3753 cbr=0 cng=1 gpc=878 gps=770 nf=4 nn=12088
76 np=17429 qsp=8 rpq=3703 cbr=0 cng=0 gpc=872 gps=766 nf=3 nn=12085
77 np=17399 qsp=8 rpq=3660 cbr=0 cng=1 gpc=878 gps=772 nf=3 nn=12085
78 np=17334 qsp=7 rpq=3608 cbr=0 cng=1 gpc=873 gps=764 nf=1 nn=12087
79 np=17288 qsp=9 rpq=3563 cbr=0 cng=1 gpc=871 gps=758 nf=2 nn=12093
80 np=19879 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1129 gps=700 nf=116 nn=14085
81 np=21160 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1317 gps=704 nf=315 nn=14712
82 np=18105 qsp=7 rpq=3461 cbr=0 cng=0 gpc=907 gps=699 nf=150 nn=12888
83 np=17157 qsp=9 rpq=3391 cbr=0 cng=0 gpc=886 gps=740 nf=5 nn=12135
84 np=17085 qsp=8 rpq=3336 cbr=0 cng=0 gpc=880 gps=756 nf=7 nn=12106
85 np=17039 qsp=9 rpq=3293 cbr=0 cng=0 gpc=875 gps=749 nf=5 nn=12117
86 np=16993 qsp=7 rpq=3248 cbr=0 cng=0 gpc=877 gps=747 nf=3 nn=12118
87 np=16948 qsp=8 rpq=3201 cbr=0 cng=0 gpc=879 gps=749 nf=4 nn=12115
88 np=18331 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1147 gps=718 nf=38 nn=12975
89 np=19383 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1286 gps=715 nf=165 nn=13632
90 np=17241 qsp=9 rpq=3121 cbr=0 cng=2 gpc=928 gps=710 nf=38 nn=12442
91 np=16803 qsp=8 rpq=3031 cbr=0 cng=1 gpc=895 gps=698 nf=2 nn=12176
92 np=16724 qsp=9 rpq=2976 cbr=0 cng=0 gpc=878 gps=695 nf=8 nn=12167
93 np=16678 qsp=8 rpq=2932 cbr=0 cng=1 gpc=884 gps=694 nf=3 nn=12164
94 np=16624 qsp=8 rpq=2886 cbr=0 cng=1 gpc=875 gps=698 nf=1 nn=12163
95 np=16586 qsp=7 rpq=2842 cbr=0 cng=0 gpc=878 gps=687 nf=3 nn=12176
96 np=17817 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1114 gps=710 nf=24 nn=12916
97 np=19239 qsp=8 rpq=3182 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16473 qsp=9 rpq=2713 cbr=0 cng=0 gpc=882 gps=710 nf=3 nn=12165
99 np=16407 qsp=8 rpq=2663 cbr=0 cng=0 gpc=875 gps=719 nf=4 nn=12146
100 np=16350 qsp=8 rpq=2609 cbr=0 cng=0 gpc=876 gps=699 nf=1 nn=12165
101 np=16293 qsp=8 rpq=2562 cbr=0 cng=1 gpc=874 gps=692 nf=3 nn=12161
102 np=16251 qsp=8 rpq=2518 cbr=0 cng=0 gpc=877 gps=683 nf=0 nn=12173
103 np=16225 qsp=11 rpq=2475 cbr=0 cng=0 gpc=877 gps=689 nf=6 nn=12178
104 np=17119 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1089 gps=768 nf=13 nn=12597
105 np=17595 qsp=67 rpq=2711 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16158 qsp=9 rpq=2364 cbr=0 cng=0 gpc=887 gps=770 nf=5 nn=12132
107 np=16048 qsp=8 rpq=2296 cbr=0 cng=0 gpc=875 gps=770 nf=2 nn=12105
108 np=16006 qsp=7 rpq=2252 cbr=0 cng=0 gpc=881 gps=774 nf=7 nn=12092
109 np=15958 qsp=8 rpq=2206 cbr=0 cng=0 gpc=881 gps=773 nf=3 nn=12095
110 np=15911 qsp=7 rpq=2160 cbr=0 cng=0 gpc=881 gps=767 nf=4 nn=12099
111 np=15866 qsp=7 rpq=2117 cbr=0 cng=0 gpc=881 gps=772 nf=3 nn=12093
112 np=17863 qsp=27 rpq=2417 cbr=0 cng=4 gpc=1163 gps=701 nf=82 nn=13496
113 np=17613 qsp=12 rpq=2430 cbr=1 cng=2 gpc=1214 gps=722 nf=97 nn=13147
114 np=16333 qsp=8 rpq=2072 cbr=0 cng=0 gpc=957 gps=725 nf=25 nn=12554
115 np=15914 qsp=7 rpq=1947 cbr=0 cng=0 gpc=891 gps=711 nf=14 nn=12351
116 np=15840 qsp=8 rpq=1894 cbr=0 cng=0 gpc=881 gps=724 nf=15 nn=12326
117 np=15776 qsp=8 rpq=1845 cbr=0 cng=1 gpc=879 gps=721 nf=5 nn=12325
118 np=15514 qsp=7 rpq=1798 cbr=0 cng=0 gpc=881 gps=717 nf=2 nn=12116
119 np=15472 qsp=8 rpq=1755 cbr=0 cng=0 gpc=881 gps=720 nf=3 nn=12113
120 np=16459 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1103 gps=662 nf=9 nn=12742
121 np=27487 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1560 gps=706 nf=2320 nn=19383
122 np=15730 qsp=7 rpq=1699 cbr=0 cng=2 gpc=954 gps=668 nf=16 nn=12391
123 np=15325 qsp=8 rpq=1577 cbr=0 cng=0 gpc=885 gps=663 nf=5 nn=12195
124 np=15281 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1044 gps=698 nf=5 nn=12003
125 np=15228 qsp=8 rpq=1488 cbr=0 cng=0 gpc=925 gps=686 nf=2 nn=12127
126 np=15223 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1034 gps=724 nf=5 nn=12014
127 np=15142 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1003 gps=669 nf=3 nn=12072
rcu_bh:
0 np=15589 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3305
1 np=15743 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=2946
2 np=14668 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1159
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19121 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2466
9 np=14980 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2492
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12119 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
13 np=12595 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=469
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18373 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3187
17 np=15236 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2411
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14581 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1923
25 np=15321 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2471
26 np=12659 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=477
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15401 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2303
33 np=14615 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2472
34 np=12453 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=333
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13819 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1724
41 np=14296 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2166
42 np=12285 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=243
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=15985 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2286
49 np=14727 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2110
50 np=12970 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=369
51 np=12699 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=85
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14076 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1895
57 np=14286 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2257
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12764 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=110
64 np=15146 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=956
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12266 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17340 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=861
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13496 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=647
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
Wed May 11 14:33:58 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4217 gpnum=4218
rcu_bh: completed=-278 gpnum=18446744073709551338
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4217 g=4218 s=2 jfq=1 j=a9d7 nfqs=6853/nfqsng=0(6853) fqlh=738
ff/ff ..>.. 0:254 ^0
ffff/ffff ..>.. 0:15 ^0 ffff/ffff ..>.. 16:31 ^1 ffff/ffff ..>.. 32:47 ^2 fcfc/ffff ..>.. 48:63 ^3 ffff/ffff ..>.. 64:79 ^4 ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
rcu_bh:
c=18446744073709551338 g=18446744073709551338 s=0 jfq=-5462 j=a9d7 nfqs=26/nfqsng=0(26) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2 0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>.. 80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=6377/73 dn=4 df=2610 of=0 ri=601 ql=2 b=10 ci=6137 co=0 ca=0
1 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=12539/1 dn=0 df=2761 of=0 ri=13 ql=9 b=10 ci=11255 co=0 ca=0
2 c=4182 g=4182 pq=1 pqc=4181 qp=0 dt=11202/0 dn=2 df=3378 of=0 ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4190 g=4191 pq=1 pqc=4190 qp=1 dt=1716/0 dn=0 df=3622 of=0 ri=6 ql=0 b=10 ci=123 co=0 ca=0
4 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1580/0 dn=2 df=3632 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
5 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1634/0 dn=0 df=3649 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
6 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1632/0 dn=2 df=3649 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
7 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1680/0 dn=2 df=3649 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
8 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=9281/1 dn=2 df=2418 of=0 ri=760 ql=2 b=10 ci=7225 co=0 ca=0
9 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=3181/1 dn=0 df=2729 of=0 ri=1 ql=9 b=10 ci=13650 co=0 ca=0
10 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2054/0 dn=0 df=3496 of=0 ri=1 ql=0 b=10 ci=702 co=0 ca=0
11 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1802/0 dn=2 df=3642 of=0 ri=1 ql=0 b=10 ci=25 co=0 ca=0
12 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1844/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
13 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=3972/0 dn=2 df=3650 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
14 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1746/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
15 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1674/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
16 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=18828/0 dn=0 df=2779 of=0 ri=103 ql=0 b=10 ci=3539 co=0 ca=0
17 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=1311/1 dn=0 df=2777 of=0 ri=1 ql=3 b=10 ci=11712 co=0 ca=0
18 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1712/0 dn=0 df=3496 of=0 ri=1 ql=0 b=10 ci=665 co=0 ca=0
19 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2078/0 dn=0 df=3620 of=0 ri=1 ql=0 b=10 ci=121 co=0 ca=0
20 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=3688/0 dn=0 df=3648 of=0 ri=1 ql=0 b=10 ci=24 co=0 ca=0
21 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1810/0 dn=2 df=3650 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
22 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1752/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
23 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1762/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
24 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=14418/0 dn=0 df=2910 of=0 ri=1 ql=0 b=10 ci=3348 co=0 ca=0
25 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1630/0 dn=0 df=2740 of=0 ri=1 ql=0 b=10 ci=22440 co=0 ca=0
26 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1824/0 dn=2 df=3477 of=0 ri=1 ql=0 b=10 ci=1047 co=0 ca=0
27 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1608/0 dn=2 df=3623 of=0 ri=1 ql=0 b=10 ci=157 co=0 ca=0
28 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1616/0 dn=0 df=3621 of=0 ri=1 ql=0 b=10 ci=92 co=0 ca=0
29 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1572/0 dn=2 df=3646 of=0 ri=1 ql=0 b=10 ci=14 co=0 ca=0
30 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1672/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
31 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1696/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
32 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=5680/0 dn=0 df=2801 of=0 ri=97 ql=0 b=10 ci=3220 co=0 ca=0
33 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1408/0 dn=0 df=2757 of=0 ri=1 ql=0 b=10 ci=21106 co=0 ca=0
34 c=4196 g=4197 pq=1 pqc=4196 qp=1 dt=1812/0 dn=0 df=3524 of=0 ri=1 ql=0 b=10 ci=550 co=0 ca=0
35 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1682/0 dn=2 df=3629 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
36 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1694/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
37 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1680/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
38 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1634/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
39 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1700/0 dn=2 df=3650 of=0 ri=1 ql=0 b=10 ci=18 co=0 ca=0
40 c=4208 g=4209 pq=1 pqc=4208 qp=1 dt=1918/0 dn=0 df=3004 of=0 ri=1 ql=0 b=10 ci=10285 co=0 ca=0
41 c=4210 g=4211 pq=1 pqc=4210 qp=1 dt=1656/0 dn=0 df=2864 of=0 ri=1 ql=0 b=10 ci=19494 co=0 ca=0
42 c=4204 g=4205 pq=1 pqc=4204 qp=1 dt=1788/0 dn=0 df=3553 of=0 ri=1 ql=0 b=10 ci=188 co=0 ca=0
43 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1718/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
44 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1666/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
45 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=2060/0 dn=0 df=3652 of=0 ri=1 ql=0 b=10 ci=12 co=0 ca=0
46 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1710/0 dn=2 df=3651 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
47 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1730/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
48 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=35267/1 dn=2 df=2672 of=0 ri=161 ql=2 b=10 ci=20305 co=0 ca=0
49 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=1551/1 dn=0 df=2895 of=0 ri=15 ql=7 b=10 ci=28449 co=0 ca=0
50 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1940/0 dn=0 df=3513 of=0 ri=1 ql=0 b=10 ci=537 co=0 ca=0
51 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=2000/0 dn=0 df=3616 of=0 ri=1 ql=0 b=10 ci=102 co=0 ca=0
52 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1864/0 dn=2 df=3636 of=0 ri=1 ql=0 b=10 ci=57 co=0 ca=0
53 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1660/0 dn=0 df=3648 of=0 ri=1 ql=0 b=10 ci=15 co=0 ca=0
54 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1614/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
55 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1692/0 dn=2 df=3651 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
56 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=1663/1 dn=2 df=2939 of=0 ri=1 ql=2 b=10 ci=2645 co=0 ca=0
57 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=1297/1 dn=0 df=2828 of=0 ri=1 ql=11 b=10 ci=11764 co=0 ca=0
58 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2088/0 dn=0 df=3514 of=0 ri=1 ql=0 b=10 ci=650 co=0 ca=0
59 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2020/0 dn=2 df=3643 of=0 ri=1 ql=0 b=10 ci=40 co=0 ca=0
60 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1718/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
61 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1610/0 dn=2 df=3651 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
62 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1706/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
63 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=6180/0 dn=0 df=3466 of=0 ri=30 ql=0 b=10 ci=150 co=0 ca=0
64 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=8396/0 dn=2 df=3232 of=0 ri=274 ql=0 b=10 ci=1436 co=0 ca=0
65 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=2912/0 dn=0 df=3128 of=0 ri=6 ql=0 b=10 ci=5781 co=0 ca=0
66 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=3994/0 dn=0 df=3601 of=0 ri=6 ql=0 b=10 ci=131 co=0 ca=0
67 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1678/0 dn=0 df=3645 of=0 ri=1 ql=0 b=10 ci=26 co=0 ca=0
68 c=4155 g=4156 pq=1 pqc=4155 qp=0 dt=1634/0 dn=0 df=3649 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
69 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1688/0 dn=2 df=3649 of=0 ri=6 ql=0 b=10 ci=11 co=0 ca=0
70 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1692/0 dn=2 df=3650 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
71 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1624/0 dn=2 df=3650 of=0 ri=1 ql=0 b=10 ci=12 co=0 ca=0
72 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=6178/0 dn=4 df=3440 of=0 ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=7498/0 dn=0 df=2919 of=0 ri=3 ql=0 b=10 ci=72254 co=0 ca=0
74 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1726/0 dn=2 df=3622 of=0 ri=1 ql=0 b=10 ci=125 co=0 ca=0
75 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1716/0 dn=2 df=3649 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
76 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1714/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
77 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1678/0 dn=0 df=3649 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
78 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1730/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
79 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1656/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
80 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=2190/0 dn=2 df=3383 of=0 ri=1 ql=0 b=10 ci=590 co=0 ca=0
81 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=3668/0 dn=0 df=3182 of=0 ri=1 ql=0 b=10 ci=5193 co=0 ca=0
82 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1662/0 dn=2 df=3623 of=0 ri=1 ql=0 b=10 ci=106 co=0 ca=0
83 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1708/0 dn=2 df=3646 of=0 ri=1 ql=0 b=10 ci=22 co=0 ca=0
84 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1638/0 dn=2 df=3651 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
85 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1652/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
86 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1680/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
87 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1688/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
88 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1888/0 dn=0 df=3401 of=0 ri=1 ql=0 b=10 ci=884 co=0 ca=0
89 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=2830/0 dn=0 df=3282 of=0 ri=1 ql=0 b=10 ci=2572 co=0 ca=0
90 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1578/0 dn=0 df=3606 of=0 ri=1 ql=0 b=10 ci=154 co=0 ca=0
91 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1788/0 dn=2 df=3645 of=0 ri=1 ql=0 b=10 ci=20 co=0 ca=0
92 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1758/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
93 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1742/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
94 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1748/0 dn=2 df=3654 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
95 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1628/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
96 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1742/0 dn=2 df=3424 of=0 ri=1 ql=0 b=10 ci=493 co=0 ca=0
97 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2676/0 dn=0 df=3330 of=0 ri=1 ql=0 b=10 ci=1015 co=0 ca=0
98 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1744/0 dn=2 df=3644 of=0 ri=1 ql=0 b=10 ci=27 co=0 ca=0
99 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1676/0 dn=2 df=3651 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
100 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1764/0 dn=2 df=3650 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
101 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1724/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
102 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1726/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
103 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1708/0 dn=2 df=3649 of=0 ri=1 ql=0 b=10 ci=1923 co=0 ca=0
104 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1726/0 dn=2 df=3458 of=0 ri=1 ql=0 b=10 ci=387 co=0 ca=0
105 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2748/0 dn=0 df=3359 of=0 ri=1 ql=0 b=10 ci=20365 co=0 ca=0
106 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1850/0 dn=2 df=3641 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
107 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1740/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
108 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1686/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
109 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1698/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
110 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1734/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
111 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1662/0 dn=2 df=3651 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
112 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=7844/0 dn=0 df=3377 of=0 ri=36 ql=0 b=10 ci=613 co=0 ca=0
113 c=4202 g=4203 pq=1 pqc=4202 qp=1 dt=2416/0 dn=0 df=3329 of=0 ri=1 ql=0 b=10 ci=5505 co=0 ca=0
114 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1686/0 dn=0 df=3570 of=0 ri=1 ql=0 b=10 ci=497 co=0 ca=0
115 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1604/0 dn=2 df=3641 of=0 ri=1 ql=0 b=10 ci=38 co=0 ca=0
116 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1776/0 dn=2 df=3648 of=0 ri=1 ql=0 b=10 ci=34 co=0 ca=0
117 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1638/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=10 co=0 ca=0
118 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1680/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
119 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2724/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
120 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2276/0 dn=0 df=3449 of=0 ri=1 ql=0 b=10 ci=393 co=0 ca=0
121 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=3026/0 dn=0 df=2057 of=0 ri=1 ql=0 b=10 ci=48877 co=0 ca=0
122 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1592/0 dn=0 df=3579 of=0 ri=1 ql=0 b=10 ci=333 co=0 ca=0
123 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1668/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
124 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1724/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
125 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1688/0 dn=2 df=3652 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
126 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=2504/0 dn=2 df=3647 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
127 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1758/0 dn=2 df=3653 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6377/73 dn=4 df=9 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
1 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=12539/1 dn=0 df=15 of=0 ri=0 ql=0 b=10 ci=4 co=0 ca=0
2 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=11202/0 dn=2 df=17 of=0 ri=0 ql=0 b=10 ci=11 co=0 ca=0
3 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1716/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
4 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1580/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
5 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1634/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
6 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1632/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
7 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
8 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=9281/1 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
9 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3181/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
10 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2054/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
11 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1802/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
12 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1844/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
13 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3972/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
14 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
15 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1674/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
16 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=18828/0 dn=0 df=17 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
17 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1311/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
18 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1712/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
19 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2078/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
20 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3688/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
21 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1810/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
22 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1752/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
23 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1762/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
24 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=14418/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
25 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1630/0 dn=0 df=19 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
26 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1824/0 dn=2 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
27 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1608/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
28 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1616/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
29 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1572/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
30 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1672/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
31 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1696/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
32 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=5680/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
33 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1408/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
34 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1812/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
35 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1682/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
36 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1694/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
37 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
38 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1634/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
39 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1700/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
40 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1918/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
41 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1656/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
42 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1788/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
43 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
44 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1666/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
45 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2060/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
46 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1710/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
47 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
48 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=35267/2 dn=2 df=16 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
49 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1551/2 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
50 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1940/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
51 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2000/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
52 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1864/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
53 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1660/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
54 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1614/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
55 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1692/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
56 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1663/1 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
57 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1297/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
58 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2088/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
59 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2020/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
60 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
61 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1610/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
62 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1706/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
63 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6180/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
64 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=8396/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
65 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2912/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
66 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=3994/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
67 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1678/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
68 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1634/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
69 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
70 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1692/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
71 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1624/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
72 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=6178/0 dn=4 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
73 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=7498/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
74 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
75 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1716/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
76 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1714/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
77 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1678/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
78 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
79 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1656/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
80 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2190/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
81 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=3668/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
82 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1662/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
83 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
84 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1638/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
85 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1652/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
86 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
87 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
88 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1888/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
89 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2830/0 dn=0 df=19 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
90 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1578/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
91 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1788/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
92 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
93 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1742/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
94 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1748/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
95 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1628/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
96 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1742/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
97 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=2676/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
98 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1744/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
99 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1676/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
100 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
101 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
102 c=18446744073709551332 g=18446744073709551332 pq=1 pqc=18446744073709551331 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
103 c=18446744073709551332 g=18446744073709551332 pq=1 pqc=18446744073709551331 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
104 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
105 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=2748/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
106 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1850/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
107 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1740/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
108 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1686/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
109 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
110 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
111 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1662/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
112 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=7844/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
113 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2416/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
114 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1686/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
115 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1604/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
116 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1776/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
117 c=18446744073709551335 g=18446744073709551336 pq=1 pqc=18446744073709551335 qp=1 dt=1638/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
118 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
119 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
120 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2276/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
121 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3026/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
122 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1592/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
123 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1668/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
124 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
125 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
126 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2504/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
127 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=25023 qsp=1521 rpq=6044 cbr=1 cng=1255 gpc=1142 gps=133 nf=695 nn=15753
1 np=26683 qsp=60 rpq=8032 cbr=5 cng=45 gpc=1628 gps=749 nf=441 nn=15783
2 np=23557 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1102 gps=715 nf=180 nn=14668
3 np=21601 qsp=29 rpq=6997 cbr=0 cng=5 gpc=910 gps=708 nf=96 nn=12885
4 np=21509 qsp=9 rpq=6943 cbr=0 cng=5 gpc=984 gps=729 nf=100 nn=12748
5 np=21377 qsp=28 rpq=6885 cbr=0 cng=2 gpc=876 gps=683 nf=109 nn=12822
6 np=21339 qsp=30 rpq=6845 cbr=0 cng=2 gpc=875 gps=689 nf=104 nn=12824
7 np=21299 qsp=32 rpq=6806 cbr=0 cng=2 gpc=982 gps=700 nf=107 nn=12702
8 np=30347 qsp=840 rpq=8110 cbr=1 cng=46 gpc=1670 gps=781 nf=590 nn=19149
9 np=25433 qsp=32 rpq=7693 cbr=1 cng=22 gpc=1752 gps=783 nf=177 nn=15005
10 np=21236 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1270 gps=806 nf=15 nn=12285
11 np=20410 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1064 gps=807 nf=5 nn=11903
12 np=20308 qsp=9 rpq=6574 cbr=0 cng=1 gpc=857 gps=755 nf=2 nn=12119
13 np=20874 qsp=9 rpq=6543 cbr=0 cng=1 gpc=917 gps=736 nf=4 nn=12673
14 np=20214 qsp=7 rpq=6482 cbr=0 cng=0 gpc=776 gps=758 nf=2 nn=12196
15 np=20177 qsp=9 rpq=6443 cbr=0 cng=0 gpc=871 gps=757 nf=3 nn=12103
16 np=28829 qsp=173 rpq=7583 cbr=0 cng=28 gpc=1745 gps=754 nf=262 nn=18457
17 np=25146 qsp=26 rpq=7261 cbr=1 cng=12 gpc=1717 gps=699 nf=205 nn=15251
18 np=20876 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1028 gps=681 nf=26 nn=12653
19 np=20174 qsp=11 rpq=6309 cbr=0 cng=1 gpc=896 gps=692 nf=8 nn=12268
20 np=19992 qsp=7 rpq=6224 cbr=0 cng=0 gpc=877 gps=671 nf=5 nn=12215
21 np=19929 qsp=9 rpq=6173 cbr=0 cng=0 gpc=870 gps=697 nf=5 nn=12184
22 np=19877 qsp=7 rpq=6122 cbr=0 cng=0 gpc=876 gps=696 nf=1 nn=12182
23 np=19830 qsp=7 rpq=6077 cbr=0 cng=0 gpc=874 gps=696 nf=2 nn=12181
24 np=23873 qsp=19 rpq=6847 cbr=0 cng=18 gpc=1554 gps=685 nf=168 nn=14601
25 np=24970 qsp=103 rpq=6912 cbr=16 cng=15 gpc=1701 gps=698 nf=262 nn=15366
26 np=20576 qsp=15 rpq=6151 cbr=0 cng=2 gpc=1047 gps=692 nf=14 nn=12670
27 np=19785 qsp=10 rpq=5927 cbr=0 cng=1 gpc=907 gps=691 nf=13 nn=12246
28 np=19763 qsp=7 rpq=5886 cbr=0 cng=1 gpc=906 gps=677 nf=9 nn=12284
29 np=19580 qsp=7 rpq=5813 cbr=0 cng=1 gpc=887 gps=685 nf=8 nn=12186
30 np=19505 qsp=6 rpq=5760 cbr=0 cng=1 gpc=874 gps=717 nf=2 nn=12151
31 np=19464 qsp=8 rpq=5714 cbr=0 cng=1 gpc=874 gps=700 nf=1 nn=12174
32 np=24658 qsp=164 rpq=6636 cbr=0 cng=17 gpc=1719 gps=741 nf=131 nn=15414
33 np=23772 qsp=42 rpq=6526 cbr=8 cng=7 gpc=1717 gps=733 nf=120 nn=14661
34 np=19929 qsp=8 rpq=5731 cbr=0 cng=4 gpc=996 gps=729 nf=14 nn=12455
35 np=19379 qsp=9 rpq=5558 cbr=0 cng=0 gpc=890 gps=727 nf=9 nn=12195
36 np=19222 qsp=8 rpq=5488 cbr=0 cng=0 gpc=876 gps=716 nf=6 nn=12136
37 np=19177 qsp=8 rpq=5442 cbr=0 cng=0 gpc=870 gps=721 nf=2 nn=12142
38 np=19131 qsp=7 rpq=5399 cbr=0 cng=0 gpc=874 gps=722 nf=4 nn=12132
39 np=19099 qsp=9 rpq=5357 cbr=0 cng=0 gpc=874 gps=720 nf=5 nn=12143
40 np=22208 qsp=29 rpq=6027 cbr=1 cng=7 gpc=1470 gps=784 nf=89 nn=13830
41 np=22867 qsp=70 rpq=6061 cbr=14 cng=18 gpc=1674 gps=663 nf=96 nn=14341
42 np=19441 qsp=8 rpq=5352 cbr=0 cng=0 gpc=1004 gps=793 nf=4 nn=12288
43 np=18930 qsp=8 rpq=5179 cbr=0 cng=0 gpc=951 gps=797 nf=5 nn=11998
44 np=18884 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1207 gps=767 nf=1 nn=11774
45 np=18844 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1199 gps=737 nf=3 nn=11813
46 np=18799 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1315 gps=801 nf=3 nn=11636
47 np=18746 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1050 gps=653 nf=3 nn=12042
48 np=25120 qsp=297 rpq=6169 cbr=3 cng=15 gpc=1693 gps=740 nf=503 nn=15997
49 np=22872 qsp=68 rpq=5700 cbr=3 cng=12 gpc=1606 gps=695 nf=116 nn=14740
50 np=19790 qsp=9 rpq=5021 cbr=0 cng=0 gpc=1017 gps=686 nf=96 nn=12970
51 np=19209 qsp=8 rpq=4858 cbr=0 cng=0 gpc=910 gps=679 nf=63 nn=12699
52 np=19011 qsp=8 rpq=4788 cbr=0 cng=0 gpc=889 gps=680 nf=48 nn=12606
53 np=19120 qsp=7 rpq=4730 cbr=0 cng=0 gpc=874 gps=682 nf=60 nn=12774
54 np=18397 qsp=8 rpq=4682 cbr=0 cng=0 gpc=869 gps=682 nf=9 nn=12155
55 np=18350 qsp=9 rpq=4635 cbr=0 cng=0 gpc=871 gps=678 nf=5 nn=12161
56 np=21845 qsp=11 rpq=5386 cbr=0 cng=10 gpc=1578 gps=701 nf=91 nn=14079
57 np=22346 qsp=27 rpq=5403 cbr=2 cng=11 gpc=1828 gps=707 nf=98 nn=14297
58 np=18955 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1217 gps=712 nf=21 nn=12346
59 np=18242 qsp=8 rpq=4469 cbr=0 cng=0 gpc=907 gps=706 nf=4 nn=12156
60 np=18162 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1030 gps=741 nf=6 nn=11973
61 np=18117 qsp=8 rpq=4369 cbr=0 cng=0 gpc=950 gps=688 nf=4 nn=12106
62 np=18075 qsp=7 rpq=4330 cbr=0 cng=0 gpc=872 gps=662 nf=7 nn=12204
63 np=19194 qsp=44 rpq=4585 cbr=0 cng=0 gpc=1027 gps=671 nf=133 nn=12778
64 np=22024 qsp=234 rpq=4773 cbr=0 cng=10 gpc=1344 gps=684 nf=67 nn=15146
65 np=20693 qsp=33 rpq=4833 cbr=0 cng=9 gpc=1403 gps=686 nf=192 nn=13570
66 np=18466 qsp=32 rpq=4222 cbr=0 cng=4 gpc=923 gps=636 nf=28 nn=12653
67 np=18101 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1149 gps=633 nf=25 nn=12172
68 np=18042 qsp=8 rpq=4072 cbr=0 cng=3 gpc=882 gps=666 nf=23 nn=12396
69 np=17993 qsp=30 rpq=4025 cbr=0 cng=2 gpc=919 gps=683 nf=29 nn=12335
70 np=17951 qsp=32 rpq=3979 cbr=0 cng=3 gpc=878 gps=676 nf=21 nn=12394
71 np=17901 qsp=8 rpq=3935 cbr=0 cng=3 gpc=931 gps=733 nf=33 nn=12266
72 np=20530 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1106 gps=628 nf=28 nn=14623
73 np=25530 qsp=10 rpq=4627 cbr=1 cng=311 gpc=1244 gps=606 nf=1401 nn=17340
74 np=17670 qsp=7 rpq=3829 cbr=0 cng=3 gpc=900 gps=775 nf=7 nn=12156
75 np=17494 qsp=8 rpq=3753 cbr=0 cng=1 gpc=878 gps=770 nf=4 nn=12088
76 np=17429 qsp=8 rpq=3703 cbr=0 cng=0 gpc=872 gps=766 nf=3 nn=12085
77 np=17399 qsp=8 rpq=3660 cbr=0 cng=1 gpc=878 gps=772 nf=3 nn=12085
78 np=17334 qsp=7 rpq=3608 cbr=0 cng=1 gpc=873 gps=764 nf=1 nn=12087
79 np=17288 qsp=9 rpq=3563 cbr=0 cng=1 gpc=871 gps=758 nf=2 nn=12093
80 np=19879 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1129 gps=700 nf=116 nn=14085
81 np=21160 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1317 gps=704 nf=315 nn=14712
82 np=18105 qsp=7 rpq=3461 cbr=0 cng=0 gpc=907 gps=699 nf=150 nn=12888
83 np=17157 qsp=9 rpq=3391 cbr=0 cng=0 gpc=886 gps=740 nf=5 nn=12135
84 np=17085 qsp=8 rpq=3336 cbr=0 cng=0 gpc=880 gps=756 nf=7 nn=12106
85 np=17039 qsp=9 rpq=3293 cbr=0 cng=0 gpc=875 gps=749 nf=5 nn=12117
86 np=16993 qsp=7 rpq=3248 cbr=0 cng=0 gpc=877 gps=747 nf=3 nn=12118
87 np=16948 qsp=8 rpq=3201 cbr=0 cng=0 gpc=879 gps=749 nf=4 nn=12115
88 np=18331 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1147 gps=718 nf=38 nn=12975
89 np=19383 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1286 gps=715 nf=165 nn=13632
90 np=17241 qsp=9 rpq=3121 cbr=0 cng=2 gpc=928 gps=710 nf=38 nn=12442
91 np=16803 qsp=8 rpq=3031 cbr=0 cng=1 gpc=895 gps=698 nf=2 nn=12176
92 np=16724 qsp=9 rpq=2976 cbr=0 cng=0 gpc=878 gps=695 nf=8 nn=12167
93 np=16678 qsp=8 rpq=2932 cbr=0 cng=1 gpc=884 gps=694 nf=3 nn=12164
94 np=16624 qsp=8 rpq=2886 cbr=0 cng=1 gpc=875 gps=698 nf=1 nn=12163
95 np=16586 qsp=7 rpq=2842 cbr=0 cng=0 gpc=878 gps=687 nf=3 nn=12176
96 np=17817 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1114 gps=710 nf=24 nn=12916
97 np=19239 qsp=8 rpq=3182 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16473 qsp=9 rpq=2713 cbr=0 cng=0 gpc=882 gps=710 nf=3 nn=12165
99 np=16407 qsp=8 rpq=2663 cbr=0 cng=0 gpc=875 gps=719 nf=4 nn=12146
100 np=16350 qsp=8 rpq=2609 cbr=0 cng=0 gpc=876 gps=699 nf=1 nn=12165
101 np=16293 qsp=8 rpq=2562 cbr=0 cng=1 gpc=874 gps=692 nf=3 nn=12161
102 np=16251 qsp=8 rpq=2518 cbr=0 cng=0 gpc=877 gps=683 nf=0 nn=12173
103 np=16225 qsp=11 rpq=2475 cbr=0 cng=0 gpc=877 gps=689 nf=6 nn=12178
104 np=17119 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1089 gps=768 nf=13 nn=12597
105 np=17595 qsp=67 rpq=2711 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16158 qsp=9 rpq=2364 cbr=0 cng=0 gpc=887 gps=770 nf=5 nn=12132
107 np=16048 qsp=8 rpq=2296 cbr=0 cng=0 gpc=875 gps=770 nf=2 nn=12105
108 np=16006 qsp=7 rpq=2252 cbr=0 cng=0 gpc=881 gps=774 nf=7 nn=12092
109 np=15958 qsp=8 rpq=2206 cbr=0 cng=0 gpc=881 gps=773 nf=3 nn=12095
110 np=15911 qsp=7 rpq=2160 cbr=0 cng=0 gpc=881 gps=767 nf=4 nn=12099
111 np=15866 qsp=7 rpq=2117 cbr=0 cng=0 gpc=881 gps=772 nf=3 nn=12093
112 np=17863 qsp=27 rpq=2417 cbr=0 cng=4 gpc=1163 gps=701 nf=82 nn=13496
113 np=17614 qsp=12 rpq=2430 cbr=1 cng=2 gpc=1215 gps=722 nf=97 nn=13147
114 np=16333 qsp=8 rpq=2072 cbr=0 cng=0 gpc=957 gps=725 nf=25 nn=12554
115 np=15914 qsp=7 rpq=1947 cbr=0 cng=0 gpc=891 gps=711 nf=14 nn=12351
116 np=15840 qsp=8 rpq=1894 cbr=0 cng=0 gpc=881 gps=724 nf=15 nn=12326
117 np=15776 qsp=8 rpq=1845 cbr=0 cng=1 gpc=879 gps=721 nf=5 nn=12325
118 np=15514 qsp=7 rpq=1798 cbr=0 cng=0 gpc=881 gps=717 nf=2 nn=12116
119 np=15472 qsp=8 rpq=1755 cbr=0 cng=0 gpc=881 gps=720 nf=3 nn=12113
120 np=16459 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1103 gps=662 nf=9 nn=12742
121 np=27488 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1561 gps=706 nf=2320 nn=19383
122 np=15730 qsp=7 rpq=1699 cbr=0 cng=2 gpc=954 gps=668 nf=16 nn=12391
123 np=15325 qsp=8 rpq=1577 cbr=0 cng=0 gpc=885 gps=663 nf=5 nn=12195
124 np=15281 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1044 gps=698 nf=5 nn=12003
125 np=15229 qsp=8 rpq=1488 cbr=0 cng=0 gpc=926 gps=686 nf=2 nn=12127
126 np=15224 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1035 gps=724 nf=5 nn=12014
127 np=15143 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1004 gps=669 nf=3 nn=12072
rcu_bh:
0 np=15753 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3469
1 np=15783 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=2986
2 np=14668 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1159
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19149 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2494
9 np=15005 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2517
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12119 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
13 np=12673 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=547
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18457 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3271
17 np=15251 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2426
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14601 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1943
25 np=15366 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2516
26 np=12670 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=488
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15414 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2316
33 np=14661 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2518
34 np=12455 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=335
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13830 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1735
41 np=14341 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2211
42 np=12288 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=246
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=15997 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2298
49 np=14740 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2123
50 np=12970 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=369
51 np=12699 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=85
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14079 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1898
57 np=14297 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2268
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12778 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=124
64 np=15146 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=956
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12266 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17340 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=861
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13496 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=647
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
Wed May 11 14:34:21 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4303 gpnum=4304
rcu_bh: completed=-278 gpnum=18446744073709551338
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4303 g=4304 s=2 jfq=2 j=c086 nfqs=6951/nfqsng=0(6951) fqlh=739
ff/ff ..>.. 0:254 ^0
ffff/ffff ..>.. 0:15 ^0 ffff/ffff ..>.. 16:31 ^1 ffff/ffff ..>.. 32:47 ^2 ffff/ffff ..>.. 48:63 ^3 ffff/ffff ..>.. 64:79 ^4 ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
rcu_bh:
c=18446744073709551338 g=18446744073709551338 s=0 jfq=-11269 j=c086 nfqs=26/nfqsng=0(26) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2 0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>.. 80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=6809/73 dn=4 df=2679 of=0 ri=601 ql=2 b=10 ci=6159 co=0 ca=0
1 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=12597/1 dn=0 df=2827 of=0 ri=13 ql=18 b=10 ci=11282 co=0 ca=0
2 c=4182 g=4182 pq=1 pqc=4181 qp=0 dt=11214/0 dn=2 df=3464 of=0 ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4190 g=4191 pq=1 pqc=4190 qp=1 dt=1728/0 dn=0 df=3708 of=0 ri=6 ql=0 b=10 ci=123 co=0 ca=0
4 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1600/0 dn=2 df=3718 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
5 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1646/0 dn=0 df=3735 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
6 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1668/0 dn=2 df=3735 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
7 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1712/0 dn=2 df=3735 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
8 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=9384/0 dn=2 df=2496 of=0 ri=760 ql=0 b=10 ci=7231 co=0 ca=0
9 c=4303 g=4304 pq=0 pqc=4239 qp=1 dt=3201/1 dn=0 df=2801 of=0 ri=1 ql=3 b=10 ci=13678 co=0 ca=0
10 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2074/0 dn=0 df=3582 of=0 ri=1 ql=0 b=10 ci=702 co=0 ca=0
11 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1822/0 dn=2 df=3728 of=0 ri=1 ql=0 b=10 ci=25 co=0 ca=0
12 c=4267 g=4268 pq=1 pqc=4267 qp=1 dt=1874/0 dn=2 df=3732 of=0 ri=1 ql=0 b=10 ci=27 co=0 ca=0
13 c=4291 g=4292 pq=1 pqc=4291 qp=1 dt=4206/0 dn=2 df=3734 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
14 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1760/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
15 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1710/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
16 c=4301 g=4302 pq=1 pqc=4301 qp=1 dt=19402/0 dn=0 df=2862 of=0 ri=103 ql=0 b=10 ci=3541 co=0 ca=0
17 c=4267 g=4268 pq=1 pqc=4267 qp=1 dt=1336/0 dn=0 df=2849 of=0 ri=1 ql=0 b=10 ci=11747 co=0 ca=0
18 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1748/0 dn=0 df=3582 of=0 ri=1 ql=0 b=10 ci=665 co=0 ca=0
19 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2102/0 dn=0 df=3706 of=0 ri=1 ql=0 b=10 ci=121 co=0 ca=0
20 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=3712/0 dn=0 df=3734 of=0 ri=1 ql=0 b=10 ci=24 co=0 ca=0
21 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1834/0 dn=2 df=3736 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
22 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1768/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
23 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1786/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
24 c=4245 g=4246 pq=1 pqc=4245 qp=1 dt=14498/0 dn=0 df=2990 of=0 ri=1 ql=0 b=10 ci=3357 co=0 ca=0
25 c=4247 g=4248 pq=1 pqc=4247 qp=1 dt=1654/0 dn=0 df=2812 of=0 ri=1 ql=0 b=10 ci=25721 co=0 ca=0
26 c=4241 g=4242 pq=1 pqc=4241 qp=1 dt=1848/0 dn=2 df=3561 of=0 ri=1 ql=0 b=10 ci=1059 co=0 ca=0
27 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1644/0 dn=2 df=3709 of=0 ri=1 ql=0 b=10 ci=157 co=0 ca=0
28 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1652/0 dn=0 df=3707 of=0 ri=1 ql=0 b=10 ci=92 co=0 ca=0
29 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1584/0 dn=2 df=3732 of=0 ri=1 ql=0 b=10 ci=14 co=0 ca=0
30 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1698/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
31 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1732/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
32 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=5810/0 dn=0 df=2881 of=0 ri=97 ql=0 b=10 ci=3229 co=0 ca=0
33 c=4253 g=4254 pq=1 pqc=4253 qp=1 dt=1436/0 dn=0 df=2828 of=0 ri=1 ql=0 b=10 ci=24390 co=0 ca=0
34 c=4247 g=4248 pq=1 pqc=4247 qp=1 dt=1842/0 dn=0 df=3608 of=0 ri=1 ql=0 b=10 ci=562 co=0 ca=0
35 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1712/0 dn=2 df=3715 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
36 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1736/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
37 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1716/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
38 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1672/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
39 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1742/0 dn=2 df=3736 of=0 ri=1 ql=0 b=10 ci=18 co=0 ca=0
40 c=4259 g=4260 pq=1 pqc=4259 qp=1 dt=2004/0 dn=0 df=3084 of=0 ri=1 ql=0 b=10 ci=10294 co=0 ca=0
41 c=4260 g=4261 pq=1 pqc=4260 qp=1 dt=1700/0 dn=0 df=2934 of=0 ri=1 ql=0 b=10 ci=22778 co=0 ca=0
42 c=4255 g=4256 pq=1 pqc=4255 qp=1 dt=1816/0 dn=0 df=3637 of=0 ri=1 ql=0 b=10 ci=200 co=0 ca=0
43 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1750/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
44 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1698/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
45 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=2092/0 dn=0 df=3738 of=0 ri=1 ql=0 b=10 ci=12 co=0 ca=0
46 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1744/0 dn=2 df=3737 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
47 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1766/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
48 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=35311/1 dn=2 df=2735 of=0 ri=161 ql=2 b=10 ci=20577 co=0 ca=0
49 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=1593/1 dn=0 df=2964 of=0 ri=15 ql=7 b=10 ci=28689 co=0 ca=0
50 c=4268 g=4269 pq=0 pqc=4267 qp=1 dt=1976/0 dn=0 df=3592 of=0 ri=1 ql=0 b=10 ci=722 co=0 ca=0
51 c=4268 g=4269 pq=0 pqc=4267 qp=1 dt=2034/0 dn=0 df=3695 of=0 ri=1 ql=0 b=10 ci=265 co=0 ca=0
52 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1902/0 dn=2 df=3722 of=0 ri=1 ql=0 b=10 ci=57 co=0 ca=0
53 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1698/0 dn=0 df=3734 of=0 ri=1 ql=0 b=10 ci=15 co=0 ca=0
54 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1644/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
55 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1722/0 dn=2 df=3737 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
56 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=1697/1 dn=2 df=3023 of=0 ri=1 ql=4 b=10 ci=2647 co=0 ca=0
57 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=1331/1 dn=0 df=2908 of=0 ri=1 ql=11 b=10 ci=11779 co=0 ca=0
58 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2122/0 dn=0 df=3600 of=0 ri=1 ql=0 b=10 ci=650 co=0 ca=0
59 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2046/0 dn=2 df=3729 of=0 ri=1 ql=0 b=10 ci=40 co=0 ca=0
60 c=4268 g=4269 pq=1 pqc=4268 qp=0 dt=1764/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
61 c=4273 g=4274 pq=1 pqc=4273 qp=1 dt=1632/0 dn=2 df=3737 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
62 c=4281 g=4281 pq=1 pqc=4280 qp=0 dt=1736/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
63 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=6340/0 dn=0 df=3552 of=0 ri=30 ql=0 b=10 ci=150 co=0 ca=0
64 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=8448/0 dn=2 df=3318 of=0 ri=274 ql=0 b=10 ci=1436 co=0 ca=0
65 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=2942/0 dn=0 df=3214 of=0 ri=6 ql=0 b=10 ci=5781 co=0 ca=0
66 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=4036/0 dn=0 df=3687 of=0 ri=6 ql=0 b=10 ci=131 co=0 ca=0
67 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1738/0 dn=0 df=3731 of=0 ri=1 ql=0 b=10 ci=26 co=0 ca=0
68 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1660/0 dn=0 df=3735 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
69 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1718/0 dn=2 df=3735 of=0 ri=6 ql=0 b=10 ci=11 co=0 ca=0
70 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1734/0 dn=2 df=3736 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
71 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1666/0 dn=2 df=3730 of=0 ri=1 ql=0 b=10 ci=32 co=0 ca=0
72 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=6224/0 dn=4 df=3526 of=0 ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=7544/0 dn=0 df=2999 of=0 ri=3 ql=0 b=10 ci=72272 co=0 ca=0
74 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1764/0 dn=2 df=3708 of=0 ri=1 ql=0 b=10 ci=125 co=0 ca=0
75 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1754/0 dn=2 df=3735 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
76 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1756/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
77 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1700/0 dn=0 df=3735 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
78 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1756/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
79 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1698/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
80 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=2230/0 dn=2 df=3469 of=0 ri=1 ql=0 b=10 ci=590 co=0 ca=0
81 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=3684/0 dn=0 df=3268 of=0 ri=1 ql=0 b=10 ci=5193 co=0 ca=0
82 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1698/0 dn=2 df=3709 of=0 ri=1 ql=0 b=10 ci=106 co=0 ca=0
83 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1736/0 dn=2 df=3732 of=0 ri=1 ql=0 b=10 ci=22 co=0 ca=0
84 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1650/0 dn=2 df=3737 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
85 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1664/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
86 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1708/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
87 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1704/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
88 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1908/0 dn=0 df=3487 of=0 ri=1 ql=0 b=10 ci=884 co=0 ca=0
89 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=2846/0 dn=0 df=3368 of=0 ri=1 ql=0 b=10 ci=2572 co=0 ca=0
90 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1598/0 dn=0 df=3692 of=0 ri=1 ql=0 b=10 ci=154 co=0 ca=0
91 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1800/0 dn=2 df=3731 of=0 ri=1 ql=0 b=10 ci=20 co=0 ca=0
92 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1770/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
93 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1754/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
94 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1784/0 dn=2 df=3740 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
95 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1640/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
96 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1758/0 dn=2 df=3510 of=0 ri=1 ql=0 b=10 ci=493 co=0 ca=0
97 c=4239 g=4240 pq=1 pqc=4239 qp=1 dt=2696/0 dn=0 df=3416 of=0 ri=1 ql=0 b=10 ci=1015 co=0 ca=0
98 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1756/0 dn=2 df=3730 of=0 ri=1 ql=0 b=10 ci=27 co=0 ca=0
99 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1712/0 dn=2 df=3737 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
100 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1784/0 dn=2 df=3736 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
101 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1736/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
102 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1748/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
103 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1720/0 dn=2 df=3735 of=0 ri=1 ql=0 b=10 ci=1923 co=0 ca=0
104 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1762/0 dn=2 df=3544 of=0 ri=1 ql=0 b=10 ci=387 co=0 ca=0
105 c=4239 g=4240 pq=1 pqc=4239 qp=1 dt=2768/0 dn=0 df=3445 of=0 ri=1 ql=0 b=10 ci=20365 co=0 ca=0
106 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1890/0 dn=2 df=3727 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
107 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1752/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
108 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1706/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
109 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1726/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
110 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1746/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
111 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1684/0 dn=2 df=3737 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
112 c=4298 g=4299 pq=1 pqc=4298 qp=1 dt=7876/0 dn=0 df=3441 of=0 ri=36 ql=0 b=10 ci=694 co=0 ca=0
113 c=4202 g=4203 pq=1 pqc=4202 qp=1 dt=2436/0 dn=0 df=3415 of=0 ri=1 ql=0 b=10 ci=5505 co=0 ca=0
114 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1722/0 dn=0 df=3656 of=0 ri=1 ql=0 b=10 ci=497 co=0 ca=0
115 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1632/0 dn=2 df=3727 of=0 ri=1 ql=0 b=10 ci=38 co=0 ca=0
116 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1792/0 dn=2 df=3734 of=0 ri=1 ql=0 b=10 ci=34 co=0 ca=0
117 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1674/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=10 co=0 ca=0
118 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1716/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
119 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2752/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
120 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2312/0 dn=0 df=3535 of=0 ri=1 ql=0 b=10 ci=393 co=0 ca=0
121 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=3044/0 dn=0 df=2143 of=0 ri=1 ql=0 b=10 ci=48877 co=0 ca=0
122 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1604/0 dn=0 df=3665 of=0 ri=1 ql=0 b=10 ci=333 co=0 ca=0
123 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1684/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
124 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1744/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
125 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1708/0 dn=2 df=3738 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
126 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=2524/0 dn=2 df=3733 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
127 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1774/0 dn=2 df=3739 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6809/73 dn=4 df=9 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
1 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=12597/1 dn=0 df=15 of=0 ri=0 ql=0 b=10 ci=4 co=0 ca=0
2 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=11214/0 dn=2 df=17 of=0 ri=0 ql=0 b=10 ci=11 co=0 ca=0
3 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1728/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
4 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1600/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
5 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1646/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
6 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1668/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
7 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
8 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=9384/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
9 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3201/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
10 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2074/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
11 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1822/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
12 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1874/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
13 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=4206/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
14 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1760/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
15 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1710/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
16 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=19402/0 dn=0 df=17 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
17 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1336/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
18 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1748/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
19 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2102/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
20 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3712/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
21 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1834/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
22 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1768/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
23 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1786/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
24 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=14498/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
25 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1654/0 dn=0 df=19 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
26 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1848/0 dn=2 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
27 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1644/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
28 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1652/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
29 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1584/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
30 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
31 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1732/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
32 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=5810/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
33 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1436/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
34 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1842/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
35 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
36 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
37 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1716/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
38 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1672/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
39 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1742/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
40 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2004/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
41 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1700/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
42 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1816/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
43 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1750/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
44 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
45 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2092/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
46 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1744/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
47 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1766/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
48 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=35311/1 dn=2 df=16 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
49 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1593/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
50 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1976/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
51 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2034/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
52 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1902/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
53 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1698/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
54 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1644/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
55 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1722/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
56 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1697/1 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
57 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1331/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
58 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2122/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
59 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2046/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
60 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
61 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1632/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
62 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
63 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6340/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
64 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=8448/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
65 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2942/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
66 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=4036/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
67 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1738/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
68 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1660/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
69 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1718/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
70 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
71 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1666/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
72 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6224/0 dn=4 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
73 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=7544/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
74 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
75 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1754/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
76 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
77 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1700/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
78 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
79 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
80 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2230/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
81 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=3684/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
82 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
83 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
84 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
85 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1664/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
86 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
87 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1704/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
88 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1908/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
89 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=2846/0 dn=0 df=19 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
90 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1598/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
91 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1800/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
92 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1770/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
93 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1754/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
94 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
95 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1640/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
96 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
97 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2696/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
98 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
99 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
100 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
101 c=18446744073709551330 g=18446744073709551330 pq=1 pqc=18446744073709551329 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
102 c=18446744073709551332 g=18446744073709551332 pq=1 pqc=18446744073709551331 qp=0 dt=1748/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
103 c=18446744073709551332 g=18446744073709551332 pq=1 pqc=18446744073709551331 qp=0 dt=1720/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
104 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1762/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
105 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2768/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
106 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1890/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
107 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1752/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
108 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1706/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
109 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
110 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
111 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1684/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
112 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=7876/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
113 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2436/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
114 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1722/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
115 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1632/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
116 c=18446744073709551334 g=18446744073709551334 pq=1 pqc=18446744073709551333 qp=0 dt=1792/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
117 c=18446744073709551335 g=18446744073709551336 pq=1 pqc=18446744073709551335 qp=1 dt=1674/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
118 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1716/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
119 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2752/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
120 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2312/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
121 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3044/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
122 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1604/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
123 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1684/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
124 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1744/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
125 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
126 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2524/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
127 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1774/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=25285 qsp=1521 rpq=6066 cbr=1 cng=1256 gpc=1169 gps=134 nf=695 nn=15964
1 np=26771 qsp=60 rpq=8060 cbr=5 cng=45 gpc=1638 gps=749 nf=452 nn=15822
2 np=23557 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1102 gps=715 nf=180 nn=14668
3 np=21601 qsp=29 rpq=6997 cbr=0 cng=5 gpc=910 gps=708 nf=96 nn=12885
4 np=21509 qsp=9 rpq=6943 cbr=0 cng=5 gpc=984 gps=729 nf=100 nn=12748
5 np=21377 qsp=28 rpq=6885 cbr=0 cng=2 gpc=876 gps=683 nf=109 nn=12822
6 np=21339 qsp=30 rpq=6845 cbr=0 cng=2 gpc=875 gps=689 nf=104 nn=12824
7 np=21299 qsp=32 rpq=6806 cbr=0 cng=2 gpc=982 gps=700 nf=107 nn=12702
8 np=30373 qsp=840 rpq=8114 cbr=1 cng=46 gpc=1676 gps=782 nf=590 nn=19164
9 np=25478 qsp=32 rpq=7706 cbr=1 cng=22 gpc=1757 gps=783 nf=183 nn=15026
10 np=21236 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1270 gps=806 nf=15 nn=12285
11 np=20410 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1064 gps=807 nf=5 nn=11903
12 np=20332 qsp=9 rpq=6580 cbr=0 cng=2 gpc=858 gps=755 nf=7 nn=12130
13 np=20911 qsp=9 rpq=6547 cbr=0 cng=1 gpc=924 gps=736 nf=4 nn=12699
14 np=20214 qsp=7 rpq=6482 cbr=0 cng=0 gpc=776 gps=758 nf=2 nn=12196
15 np=20177 qsp=9 rpq=6443 cbr=0 cng=0 gpc=871 gps=757 nf=3 nn=12103
16 np=28931 qsp=173 rpq=7597 cbr=0 cng=29 gpc=1751 gps=758 nf=262 nn=18534
17 np=25198 qsp=26 rpq=7275 cbr=1 cng=15 gpc=1726 gps=699 nf=205 nn=15277
18 np=20876 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1028 gps=681 nf=26 nn=12653
19 np=20174 qsp=11 rpq=6309 cbr=0 cng=1 gpc=896 gps=692 nf=8 nn=12268
20 np=19992 qsp=7 rpq=6224 cbr=0 cng=0 gpc=877 gps=671 nf=5 nn=12215
21 np=19929 qsp=9 rpq=6173 cbr=0 cng=0 gpc=870 gps=697 nf=5 nn=12184
22 np=19877 qsp=7 rpq=6122 cbr=0 cng=0 gpc=876 gps=696 nf=1 nn=12182
23 np=19830 qsp=7 rpq=6077 cbr=0 cng=0 gpc=874 gps=696 nf=2 nn=12181
24 np=23902 qsp=19 rpq=6853 cbr=0 cng=18 gpc=1560 gps=685 nf=169 nn=14617
25 np=25038 qsp=108 rpq=6919 cbr=18 cng=16 gpc=1713 gps=698 nf=263 nn=15411
26 np=20585 qsp=15 rpq=6153 cbr=0 cng=2 gpc=1050 gps=692 nf=14 nn=12674
27 np=19785 qsp=10 rpq=5927 cbr=0 cng=1 gpc=907 gps=691 nf=13 nn=12246
28 np=19763 qsp=7 rpq=5886 cbr=0 cng=1 gpc=906 gps=677 nf=9 nn=12284
29 np=19580 qsp=7 rpq=5813 cbr=0 cng=1 gpc=887 gps=685 nf=8 nn=12186
30 np=19505 qsp=6 rpq=5760 cbr=0 cng=1 gpc=874 gps=717 nf=2 nn=12151
31 np=19464 qsp=8 rpq=5714 cbr=0 cng=1 gpc=874 gps=700 nf=1 nn=12174
32 np=24711 qsp=164 rpq=6649 cbr=0 cng=18 gpc=1730 gps=741 nf=136 nn=15437
33 np=23840 qsp=47 rpq=6535 cbr=12 cng=7 gpc=1729 gps=733 nf=121 nn=14703
34 np=19935 qsp=8 rpq=5733 cbr=0 cng=4 gpc=998 gps=729 nf=14 nn=12457
35 np=19379 qsp=9 rpq=5558 cbr=0 cng=0 gpc=890 gps=727 nf=9 nn=12195
36 np=19222 qsp=8 rpq=5488 cbr=0 cng=0 gpc=876 gps=716 nf=6 nn=12136
37 np=19177 qsp=8 rpq=5442 cbr=0 cng=0 gpc=870 gps=721 nf=2 nn=12142
38 np=19131 qsp=7 rpq=5399 cbr=0 cng=0 gpc=874 gps=722 nf=4 nn=12132
39 np=19099 qsp=9 rpq=5357 cbr=0 cng=0 gpc=874 gps=720 nf=5 nn=12143
40 np=22232 qsp=29 rpq=6033 cbr=1 cng=7 gpc=1471 gps=784 nf=95 nn=13841
41 np=22933 qsp=77 rpq=6072 cbr=18 cng=22 gpc=1680 gps=664 nf=99 nn=14378
42 np=19449 qsp=8 rpq=5354 cbr=0 cng=0 gpc=1007 gps=793 nf=4 nn=12291
43 np=18930 qsp=8 rpq=5179 cbr=0 cng=0 gpc=951 gps=797 nf=5 nn=11998
44 np=18884 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1207 gps=767 nf=1 nn=11774
45 np=18845 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1200 gps=737 nf=3 nn=11813
46 np=18800 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1316 gps=801 nf=3 nn=11636
47 np=18747 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1051 gps=653 nf=3 nn=12042
48 np=25202 qsp=297 rpq=6190 cbr=3 cng=15 gpc=1702 gps=740 nf=514 nn=16038
49 np=22939 qsp=68 rpq=5720 cbr=3 cng=12 gpc=1623 gps=695 nf=116 nn=14770
50 np=19819 qsp=9 rpq=5028 cbr=0 cng=0 gpc=1025 gps=686 nf=96 nn=12984
51 np=19238 qsp=8 rpq=4864 cbr=0 cng=0 gpc=918 gps=680 nf=63 nn=12713
52 np=19012 qsp=8 rpq=4788 cbr=0 cng=0 gpc=890 gps=680 nf=48 nn=12606
53 np=19121 qsp=7 rpq=4730 cbr=0 cng=0 gpc=875 gps=682 nf=60 nn=12774
54 np=18398 qsp=8 rpq=4682 cbr=0 cng=0 gpc=870 gps=682 nf=9 nn=12155
55 np=18351 qsp=9 rpq=4635 cbr=0 cng=0 gpc=872 gps=678 nf=5 nn=12161
56 np=21872 qsp=11 rpq=5393 cbr=0 cng=10 gpc=1585 gps=701 nf=91 nn=14092
57 np=22373 qsp=27 rpq=5410 cbr=2 cng=11 gpc=1830 gps=707 nf=104 nn=14309
58 np=18956 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1218 gps=712 nf=21 nn=12346
59 np=18243 qsp=8 rpq=4469 cbr=0 cng=0 gpc=908 gps=706 nf=4 nn=12156
60 np=18163 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1031 gps=741 nf=6 nn=11973
61 np=18118 qsp=8 rpq=4369 cbr=0 cng=0 gpc=951 gps=688 nf=4 nn=12106
62 np=18076 qsp=7 rpq=4330 cbr=0 cng=0 gpc=873 gps=662 nf=7 nn=12204
63 np=19218 qsp=44 rpq=4589 cbr=0 cng=0 gpc=1034 gps=671 nf=133 nn=12791
64 np=22026 qsp=234 rpq=4773 cbr=0 cng=11 gpc=1345 gps=684 nf=67 nn=15146
65 np=20694 qsp=33 rpq=4834 cbr=0 cng=9 gpc=1403 gps=686 nf=192 nn=13570
66 np=18467 qsp=32 rpq=4222 cbr=0 cng=4 gpc=924 gps=636 nf=28 nn=12653
67 np=18102 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1150 gps=633 nf=25 nn=12172
68 np=18043 qsp=8 rpq=4072 cbr=0 cng=3 gpc=883 gps=666 nf=23 nn=12396
69 np=17994 qsp=30 rpq=4025 cbr=0 cng=2 gpc=920 gps=683 nf=29 nn=12335
70 np=17952 qsp=32 rpq=3979 cbr=0 cng=3 gpc=879 gps=676 nf=21 nn=12394
71 np=17926 qsp=8 rpq=3942 cbr=0 cng=3 gpc=938 gps=733 nf=33 nn=12277
72 np=20531 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1107 gps=628 nf=28 nn=14623
73 np=25555 qsp=10 rpq=4635 cbr=1 cng=311 gpc=1250 gps=606 nf=1401 nn=17351
74 np=17671 qsp=7 rpq=3829 cbr=0 cng=3 gpc=901 gps=775 nf=7 nn=12156
75 np=17495 qsp=8 rpq=3753 cbr=0 cng=1 gpc=879 gps=770 nf=4 nn=12088
76 np=17430 qsp=8 rpq=3703 cbr=0 cng=0 gpc=873 gps=766 nf=3 nn=12085
77 np=17400 qsp=8 rpq=3660 cbr=0 cng=1 gpc=879 gps=772 nf=3 nn=12085
78 np=17335 qsp=7 rpq=3608 cbr=0 cng=1 gpc=874 gps=764 nf=1 nn=12087
79 np=17289 qsp=9 rpq=3563 cbr=0 cng=1 gpc=872 gps=758 nf=2 nn=12093
80 np=19880 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1130 gps=700 nf=116 nn=14085
81 np=21161 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1318 gps=704 nf=315 nn=14712
82 np=18106 qsp=7 rpq=3461 cbr=0 cng=0 gpc=908 gps=699 nf=150 nn=12888
83 np=17158 qsp=9 rpq=3391 cbr=0 cng=0 gpc=887 gps=740 nf=5 nn=12135
84 np=17086 qsp=8 rpq=3336 cbr=0 cng=0 gpc=881 gps=756 nf=7 nn=12106
85 np=17040 qsp=9 rpq=3293 cbr=0 cng=0 gpc=876 gps=749 nf=5 nn=12117
86 np=16994 qsp=7 rpq=3248 cbr=0 cng=0 gpc=878 gps=747 nf=3 nn=12118
87 np=16949 qsp=8 rpq=3201 cbr=0 cng=0 gpc=880 gps=749 nf=4 nn=12115
88 np=18332 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1148 gps=718 nf=38 nn=12975
89 np=19384 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1287 gps=715 nf=165 nn=13632
90 np=17242 qsp=9 rpq=3121 cbr=0 cng=2 gpc=929 gps=710 nf=38 nn=12442
91 np=16804 qsp=8 rpq=3031 cbr=0 cng=1 gpc=896 gps=698 nf=2 nn=12176
92 np=16724 qsp=9 rpq=2976 cbr=0 cng=0 gpc=878 gps=695 nf=8 nn=12167
93 np=16678 qsp=8 rpq=2932 cbr=0 cng=1 gpc=884 gps=694 nf=3 nn=12164
94 np=16624 qsp=8 rpq=2886 cbr=0 cng=1 gpc=875 gps=698 nf=1 nn=12163
95 np=16586 qsp=7 rpq=2842 cbr=0 cng=0 gpc=878 gps=687 nf=3 nn=12176
96 np=17817 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1114 gps=710 nf=24 nn=12916
97 np=19240 qsp=8 rpq=3183 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16473 qsp=9 rpq=2713 cbr=0 cng=0 gpc=882 gps=710 nf=3 nn=12165
99 np=16407 qsp=8 rpq=2663 cbr=0 cng=0 gpc=875 gps=719 nf=4 nn=12146
100 np=16350 qsp=8 rpq=2609 cbr=0 cng=0 gpc=876 gps=699 nf=1 nn=12165
101 np=16293 qsp=8 rpq=2562 cbr=0 cng=1 gpc=874 gps=692 nf=3 nn=12161
102 np=16251 qsp=8 rpq=2518 cbr=0 cng=0 gpc=877 gps=683 nf=0 nn=12173
103 np=16225 qsp=11 rpq=2475 cbr=0 cng=0 gpc=877 gps=689 nf=6 nn=12178
104 np=17119 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1089 gps=768 nf=13 nn=12597
105 np=17596 qsp=67 rpq=2712 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16158 qsp=9 rpq=2364 cbr=0 cng=0 gpc=887 gps=770 nf=5 nn=12132
107 np=16048 qsp=8 rpq=2296 cbr=0 cng=0 gpc=875 gps=770 nf=2 nn=12105
108 np=16006 qsp=7 rpq=2252 cbr=0 cng=0 gpc=881 gps=774 nf=7 nn=12092
109 np=15958 qsp=8 rpq=2206 cbr=0 cng=0 gpc=881 gps=773 nf=3 nn=12095
110 np=15911 qsp=7 rpq=2160 cbr=0 cng=0 gpc=881 gps=767 nf=4 nn=12099
111 np=15866 qsp=7 rpq=2117 cbr=0 cng=0 gpc=881 gps=772 nf=3 nn=12093
112 np=17959 qsp=27 rpq=2442 cbr=0 cng=4 gpc=1170 gps=701 nf=97 nn=13545
113 np=17614 qsp=12 rpq=2430 cbr=1 cng=2 gpc=1215 gps=722 nf=97 nn=13147
114 np=16333 qsp=8 rpq=2072 cbr=0 cng=0 gpc=957 gps=725 nf=25 nn=12554
115 np=15914 qsp=7 rpq=1947 cbr=0 cng=0 gpc=891 gps=711 nf=14 nn=12351
116 np=15840 qsp=8 rpq=1894 cbr=0 cng=0 gpc=881 gps=724 nf=15 nn=12326
117 np=15776 qsp=8 rpq=1845 cbr=0 cng=1 gpc=879 gps=721 nf=5 nn=12325
118 np=15514 qsp=7 rpq=1798 cbr=0 cng=0 gpc=881 gps=717 nf=2 nn=12116
119 np=15472 qsp=8 rpq=1755 cbr=0 cng=0 gpc=881 gps=720 nf=3 nn=12113
120 np=16459 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1103 gps=662 nf=9 nn=12742
121 np=27488 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1561 gps=706 nf=2320 nn=19383
122 np=15730 qsp=7 rpq=1699 cbr=0 cng=2 gpc=954 gps=668 nf=16 nn=12391
123 np=15325 qsp=8 rpq=1577 cbr=0 cng=0 gpc=885 gps=663 nf=5 nn=12195
124 np=15281 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1044 gps=698 nf=5 nn=12003
125 np=15229 qsp=8 rpq=1488 cbr=0 cng=0 gpc=926 gps=686 nf=2 nn=12127
126 np=15224 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1035 gps=724 nf=5 nn=12014
127 np=15143 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1004 gps=669 nf=3 nn=12072
rcu_bh:
0 np=15964 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3680
1 np=15822 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=3025
2 np=14668 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1159
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19164 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2509
9 np=15026 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2538
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12130 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
13 np=12699 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=573
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18534 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3348
17 np=15277 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2452
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14617 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1959
25 np=15411 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2561
26 np=12674 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=492
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15437 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2339
33 np=14703 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2560
34 np=12457 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=337
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13841 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1746
41 np=14378 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2248
42 np=12291 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=249
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=16038 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2339
49 np=14770 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2153
50 np=12984 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=383
51 np=12713 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14092 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1911
57 np=14309 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2280
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12791 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=137
64 np=15146 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=956
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12277 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17351 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=872
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13545 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=696
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
Wed May 11 14:34:45 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4365 gpnum=4366
rcu_bh: completed=-278 gpnum=18446744073709551338
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4365 g=4366 s=2 jfq=2 j=d7dd nfqs=7021/nfqsng=0(7021) fqlh=739
ff/ff ..>.. 0:254 ^0
ffff/ffff ..>.. 0:15 ^0 ffff/ffff ..>.. 16:31 ^1 ffff/ffff ..>.. 32:47 ^2 fdfe/ffff ..>.. 48:63 ^3 ffff/ffff ..>.. 64:79 ^4 ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
rcu_bh:
c=18446744073709551338 g=18446744073709551338 s=0 jfq=-17244 j=d7dd nfqs=26/nfqsng=0(26) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2 0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>.. 80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=7237/73 dn=4 df=2718 of=0 ri=601 ql=4 b=10 ci=6199 co=0 ca=0
1 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=12649/1 dn=0 df=2878 of=0 ri=13 ql=9 b=10 ci=11312 co=0 ca=0
2 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=11246/0 dn=2 df=3526 of=0 ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4190 g=4191 pq=1 pqc=4190 qp=1 dt=1740/0 dn=0 df=3770 of=0 ri=6 ql=0 b=10 ci=123 co=0 ca=0
4 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1612/0 dn=2 df=3780 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
5 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1658/0 dn=0 df=3797 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
6 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1700/0 dn=2 df=3797 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
7 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1728/0 dn=2 df=3797 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
8 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=9484/0 dn=2 df=2552 of=0 ri=760 ql=0 b=10 ci=7235 co=0 ca=0
9 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=3221/1 dn=0 df=2853 of=0 ri=1 ql=12 b=10 ci=13696 co=0 ca=0
10 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2086/0 dn=0 df=3644 of=0 ri=1 ql=0 b=10 ci=702 co=0 ca=0
11 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1834/0 dn=2 df=3790 of=0 ri=1 ql=0 b=10 ci=25 co=0 ca=0
12 c=4267 g=4268 pq=1 pqc=4267 qp=1 dt=1890/0 dn=2 df=3794 of=0 ri=1 ql=0 b=10 ci=27 co=0 ca=0
13 c=4291 g=4292 pq=1 pqc=4291 qp=1 dt=4218/0 dn=2 df=3796 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
14 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1772/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
15 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1746/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
16 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=19988/0 dn=0 df=2922 of=0 ri=103 ql=0 b=10 ci=3543 co=0 ca=0
17 c=4331 g=4332 pq=1 pqc=4331 qp=1 dt=1352/0 dn=0 df=2905 of=0 ri=1 ql=0 b=10 ci=11761 co=0 ca=0
18 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1768/0 dn=0 df=3644 of=0 ri=1 ql=0 b=10 ci=665 co=0 ca=0
19 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2114/0 dn=0 df=3768 of=0 ri=1 ql=0 b=10 ci=121 co=0 ca=0
20 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=3724/0 dn=0 df=3796 of=0 ri=1 ql=0 b=10 ci=24 co=0 ca=0
21 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1846/0 dn=2 df=3798 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
22 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1784/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
23 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1798/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
24 c=4337 g=4338 pq=1 pqc=4337 qp=1 dt=14566/0 dn=0 df=3046 of=0 ri=1 ql=0 b=10 ci=3366 co=0 ca=0
25 c=4339 g=4340 pq=1 pqc=4339 qp=1 dt=1670/0 dn=0 df=2860 of=0 ri=1 ql=0 b=10 ci=28852 co=0 ca=0
26 c=4333 g=4334 pq=1 pqc=4333 qp=1 dt=1864/0 dn=2 df=3621 of=0 ri=1 ql=0 b=10 ci=1071 co=0 ca=0
27 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1680/0 dn=2 df=3771 of=0 ri=1 ql=0 b=10 ci=157 co=0 ca=0
28 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1688/0 dn=0 df=3769 of=0 ri=1 ql=0 b=10 ci=92 co=0 ca=0
29 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1596/0 dn=2 df=3794 of=0 ri=1 ql=0 b=10 ci=14 co=0 ca=0
30 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1710/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
31 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1764/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
32 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=5938/0 dn=0 df=2932 of=0 ri=97 ql=0 b=10 ci=3256 co=0 ca=0
33 c=4343 g=4344 pq=1 pqc=4343 qp=1 dt=1456/0 dn=0 df=2876 of=0 ri=1 ql=0 b=10 ci=27522 co=0 ca=0
34 c=4343 g=4344 pq=1 pqc=4343 qp=1 dt=1858/0 dn=0 df=3664 of=0 ri=1 ql=0 b=10 ci=576 co=0 ca=0
35 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1724/0 dn=2 df=3777 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
36 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1772/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
37 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1728/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
38 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1684/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
39 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1770/0 dn=2 df=3798 of=0 ri=1 ql=0 b=10 ci=18 co=0 ca=0
40 c=4350 g=4351 pq=1 pqc=4350 qp=1 dt=2074/0 dn=0 df=3140 of=0 ri=1 ql=0 b=10 ci=10303 co=0 ca=0
41 c=4352 g=4353 pq=1 pqc=4352 qp=1 dt=1734/0 dn=0 df=2979 of=0 ri=1 ql=0 b=10 ci=25912 co=0 ca=0
42 c=4346 g=4347 pq=1 pqc=4346 qp=1 dt=1828/0 dn=0 df=3697 of=0 ri=1 ql=0 b=10 ci=212 co=0 ca=0
43 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1786/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
44 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1734/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
45 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=2128/0 dn=0 df=3800 of=0 ri=1 ql=0 b=10 ci=12 co=0 ca=0
46 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1780/0 dn=2 df=3799 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
47 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1802/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
48 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=35327/1 dn=2 df=2796 of=0 ri=161 ql=2 b=10 ci=20579 co=0 ca=0
49 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=1619/1 dn=0 df=3017 of=0 ri=15 ql=7 b=10 ci=28706 co=0 ca=0
50 c=4268 g=4269 pq=1 pqc=4268 qp=1 dt=1992/0 dn=0 df=3654 of=0 ri=1 ql=0 b=10 ci=722 co=0 ca=0
51 c=4268 g=4269 pq=1 pqc=4268 qp=1 dt=2048/0 dn=0 df=3757 of=0 ri=1 ql=0 b=10 ci=265 co=0 ca=0
52 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1938/0 dn=2 df=3784 of=0 ri=1 ql=0 b=10 ci=57 co=0 ca=0
53 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1714/0 dn=0 df=3796 of=0 ri=1 ql=0 b=10 ci=15 co=0 ca=0
54 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1668/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
55 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1746/0 dn=2 df=3799 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
56 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=1713/1 dn=2 df=3079 of=0 ri=1 ql=2 b=10 ci=2655 co=0 ca=0
57 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=1347/1 dn=0 df=2964 of=0 ri=1 ql=11 b=10 ci=11792 co=0 ca=0
58 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2158/0 dn=0 df=3662 of=0 ri=1 ql=0 b=10 ci=650 co=0 ca=0
59 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2062/0 dn=2 df=3791 of=0 ri=1 ql=0 b=10 ci=40 co=0 ca=0
60 c=4268 g=4269 pq=1 pqc=4268 qp=0 dt=1800/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
61 c=4273 g=4274 pq=1 pqc=4273 qp=1 dt=1644/0 dn=2 df=3799 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
62 c=4281 g=4281 pq=1 pqc=4280 qp=0 dt=1756/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
63 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=6496/0 dn=0 df=3614 of=0 ri=30 ql=0 b=10 ci=150 co=0 ca=0
64 c=4327 g=4328 pq=1 pqc=4327 qp=1 dt=8498/0 dn=2 df=3372 of=0 ri=274 ql=0 b=10 ci=1442 co=0 ca=0
65 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=2958/0 dn=0 df=3276 of=0 ri=6 ql=0 b=10 ci=5781 co=0 ca=0
66 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=4094/0 dn=0 df=3749 of=0 ri=6 ql=0 b=10 ci=131 co=0 ca=0
67 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1756/0 dn=0 df=3793 of=0 ri=1 ql=0 b=10 ci=26 co=0 ca=0
68 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1696/0 dn=0 df=3797 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
69 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1758/0 dn=2 df=3797 of=0 ri=6 ql=0 b=10 ci=11 co=0 ca=0
70 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1750/0 dn=2 df=3798 of=0 ri=6 ql=0 b=10 ci=10 co=0 ca=0
71 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1702/0 dn=2 df=3792 of=0 ri=1 ql=0 b=10 ci=32 co=0 ca=0
72 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=6248/0 dn=4 df=3588 of=0 ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=7578/0 dn=0 df=3061 of=0 ri=3 ql=0 b=10 ci=72272 co=0 ca=0
74 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1800/0 dn=2 df=3770 of=0 ri=1 ql=0 b=10 ci=125 co=0 ca=0
75 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1790/0 dn=2 df=3797 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
76 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1792/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
77 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1736/0 dn=0 df=3797 of=0 ri=1 ql=0 b=10 ci=111 co=0 ca=0
78 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1778/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
79 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1734/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
80 c=4307 g=4308 pq=1 pqc=4307 qp=1 dt=2248/0 dn=2 df=3531 of=0 ri=1 ql=0 b=10 ci=590 co=0 ca=0
81 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=3718/0 dn=0 df=3330 of=0 ri=1 ql=0 b=10 ci=5193 co=0 ca=0
82 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1740/0 dn=2 df=3771 of=0 ri=1 ql=0 b=10 ci=106 co=0 ca=0
83 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1778/0 dn=2 df=3794 of=0 ri=1 ql=0 b=10 ci=22 co=0 ca=0
84 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1688/0 dn=2 df=3799 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
85 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1702/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
86 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1746/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
87 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1730/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
88 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1934/0 dn=0 df=3549 of=0 ri=1 ql=0 b=10 ci=884 co=0 ca=0
89 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=2872/0 dn=0 df=3430 of=0 ri=1 ql=0 b=10 ci=2572 co=0 ca=0
90 c=4318 g=4318 pq=1 pqc=4317 qp=0 dt=1620/0 dn=0 df=3754 of=0 ri=1 ql=0 b=10 ci=154 co=0 ca=0
91 c=4318 g=4318 pq=1 pqc=4317 qp=0 dt=1838/0 dn=2 df=3793 of=0 ri=1 ql=0 b=10 ci=20 co=0 ca=0
92 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1808/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
93 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1788/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
94 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1826/0 dn=2 df=3802 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
95 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1662/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
96 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1784/0 dn=2 df=3572 of=0 ri=1 ql=0 b=10 ci=493 co=0 ca=0
97 c=4326 g=4327 pq=1 pqc=4326 qp=1 dt=2726/0 dn=0 df=3478 of=0 ri=1 ql=0 b=10 ci=1015 co=0 ca=0
98 c=4352 g=4353 pq=1 pqc=4352 qp=0 dt=1786/0 dn=2 df=3792 of=0 ri=1 ql=0 b=10 ci=27 co=0 ca=0
99 c=4353 g=4353 pq=1 pqc=4352 qp=0 dt=1754/0 dn=2 df=3799 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
100 c=4354 g=4355 pq=1 pqc=4354 qp=0 dt=1818/0 dn=2 df=3798 of=0 ri=1 ql=0 b=10 ci=11 co=0 ca=0
101 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1770/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
102 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1782/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
103 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1750/0 dn=2 df=3797 of=0 ri=1 ql=0 b=10 ci=1923 co=0 ca=0
104 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1804/0 dn=2 df=3606 of=0 ri=1 ql=0 b=10 ci=387 co=0 ca=0
105 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=2806/0 dn=0 df=3507 of=0 ri=1 ql=0 b=10 ci=20365 co=0 ca=0
106 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1928/0 dn=2 df=3789 of=0 ri=1 ql=0 b=10 ci=50 co=0 ca=0
107 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1782/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
108 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1734/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
109 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1756/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
110 c=4356 g=4357 pq=1 pqc=4356 qp=0 dt=1776/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
111 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1714/0 dn=2 df=3799 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
112 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=7918/0 dn=0 df=3503 of=0 ri=36 ql=0 b=10 ci=694 co=0 ca=0
113 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=2470/0 dn=0 df=3477 of=0 ri=1 ql=0 b=10 ci=5505 co=0 ca=0
114 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1764/0 dn=0 df=3718 of=0 ri=1 ql=0 b=10 ci=497 co=0 ca=0
115 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1658/0 dn=2 df=3789 of=0 ri=1 ql=0 b=10 ci=38 co=0 ca=0
116 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1820/0 dn=2 df=3796 of=0 ri=1 ql=0 b=10 ci=34 co=0 ca=0
117 c=4363 g=4363 pq=1 pqc=4362 qp=0 dt=1712/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=10 co=0 ca=0
118 c=4363 g=4363 pq=1 pqc=4362 qp=0 dt=1758/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
119 c=4363 g=4363 pq=1 pqc=4362 qp=0 dt=2778/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
120 c=4364 g=4365 pq=1 pqc=4364 qp=0 dt=2334/0 dn=0 df=3597 of=0 ri=1 ql=0 b=10 ci=393 co=0 ca=0
121 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=3070/0 dn=0 df=2205 of=0 ri=1 ql=0 b=10 ci=48877 co=0 ca=0
122 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1626/0 dn=0 df=3727 of=0 ri=1 ql=0 b=10 ci=333 co=0 ca=0
123 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1706/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
124 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1766/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
125 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1730/0 dn=2 df=3800 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
126 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=2546/0 dn=2 df=3795 of=0 ri=1 ql=0 b=10 ci=21 co=0 ca=0
127 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1796/0 dn=2 df=3801 of=0 ri=1 ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=7237/73 dn=4 df=9 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
1 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=12649/1 dn=0 df=15 of=0 ri=0 ql=0 b=10 ci=4 co=0 ca=0
2 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=11246/0 dn=2 df=17 of=0 ri=0 ql=0 b=10 ci=11 co=0 ca=0
3 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1740/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
4 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1612/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
5 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1658/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
6 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1700/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
7 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1728/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
8 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=9484/0 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
9 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3221/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
10 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2086/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
11 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1834/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
12 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1890/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
13 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=4218/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
14 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1772/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
15 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
16 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=19988/0 dn=0 df=17 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
17 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1352/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=8 co=0 ca=0
18 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1768/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
19 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2114/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
20 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3724/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
21 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1846/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
22 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
23 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1798/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
24 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=14566/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
25 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1670/0 dn=0 df=19 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
26 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1864/0 dn=2 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
27 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
28 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1688/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
29 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1596/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
30 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1710/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
31 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
32 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=5938/0 dn=0 df=16 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
33 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1456/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
34 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1858/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
35 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
36 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1772/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
37 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1728/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
38 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1684/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
39 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1770/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
40 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2074/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
41 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1734/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
42 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1828/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
43 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1786/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
44 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
45 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2128/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
46 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1780/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
47 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1802/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
48 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=35327/1 dn=2 df=16 of=0 ri=0 ql=0 b=10 ci=2 co=0 ca=0
49 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1619/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
50 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1992/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
51 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2048/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
52 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1938/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
53 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1714/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
54 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1668/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
55 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
56 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1713/1 dn=2 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
57 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1347/1 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
58 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2158/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
59 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2062/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
60 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1800/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
61 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1644/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
62 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
63 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6496/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
64 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=8498/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
65 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2958/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
66 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=4094/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
67 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1756/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
68 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1696/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
69 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
70 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1750/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
71 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1702/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
72 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=6248/0 dn=4 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
73 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=7578/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
74 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1800/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
75 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1790/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
76 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1792/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
77 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1736/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
78 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1778/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
79 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
80 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2248/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
81 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3718/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
82 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1740/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
83 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1778/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
84 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
85 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1702/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
86 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
87 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
88 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1934/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
89 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2872/0 dn=0 df=19 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
90 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1620/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
91 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1838/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
92 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1808/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
93 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1788/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
94 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1826/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
95 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1662/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
96 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
97 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2726/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
98 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1786/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
99 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1754/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
100 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1818/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
101 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1770/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
102 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1782/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
103 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1750/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
104 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1804/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
105 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2806/0 dn=0 df=20 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
106 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1928/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
107 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1782/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
108 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
109 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
110 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1776/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
111 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1714/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
112 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=7918/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
113 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2470/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
114 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1764/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
115 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1658/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
116 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1820/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
117 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
118 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
119 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2778/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
120 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2334/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
121 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=3070/0 dn=0 df=18 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
122 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1626/0 dn=0 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
123 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1706/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
124 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1766/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
125 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
126 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=2546/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
127 c=18446744073709551338 g=18446744073709551338 pq=1 pqc=18446744073709551337 qp=0 dt=1796/0 dn=2 df=22 of=0 ri=0 ql=0 b=10 ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=25632 qsp=1521 rpq=6099 cbr=1 cng=1258 gpc=1188 gps=137 nf=703 nn=16246
1 np=26817 qsp=60 rpq=8075 cbr=5 cng=46 gpc=1644 gps=749 nf=456 nn=15842
2 np=23561 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1104 gps=715 nf=180 nn=14670
3 np=21602 qsp=29 rpq=6998 cbr=0 cng=5 gpc=910 gps=708 nf=96 nn=12885
4 np=21510 qsp=9 rpq=6943 cbr=0 cng=5 gpc=985 gps=729 nf=100 nn=12748
5 np=21378 qsp=28 rpq=6885 cbr=0 cng=2 gpc=877 gps=683 nf=109 nn=12822
6 np=21340 qsp=30 rpq=6845 cbr=0 cng=2 gpc=876 gps=689 nf=104 nn=12824
7 np=21300 qsp=32 rpq=6806 cbr=0 cng=2 gpc=983 gps=700 nf=107 nn=12702
8 np=30419 qsp=840 rpq=8121 cbr=1 cng=46 gpc=1688 gps=783 nf=590 nn=19190
9 np=25540 qsp=32 rpq=7723 cbr=1 cng=22 gpc=1766 gps=784 nf=191 nn=15053
10 np=21237 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1271 gps=806 nf=15 nn=12285
11 np=20410 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1064 gps=807 nf=5 nn=11903
12 np=20332 qsp=9 rpq=6580 cbr=0 cng=2 gpc=858 gps=755 nf=7 nn=12130
13 np=20911 qsp=9 rpq=6547 cbr=0 cng=1 gpc=924 gps=736 nf=4 nn=12699
14 np=20214 qsp=7 rpq=6482 cbr=0 cng=0 gpc=776 gps=758 nf=2 nn=12196
15 np=20177 qsp=9 rpq=6443 cbr=0 cng=0 gpc=871 gps=757 nf=3 nn=12103
16 np=29031 qsp=173 rpq=7610 cbr=0 cng=29 gpc=1756 gps=763 nf=262 nn=18611
17 np=25227 qsp=26 rpq=7282 cbr=1 cng=15 gpc=1732 gps=699 nf=205 nn=15293
18 np=20876 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1028 gps=681 nf=26 nn=12653
19 np=20174 qsp=11 rpq=6309 cbr=0 cng=1 gpc=896 gps=692 nf=8 nn=12268
20 np=19992 qsp=7 rpq=6224 cbr=0 cng=0 gpc=877 gps=671 nf=5 nn=12215
21 np=19929 qsp=9 rpq=6173 cbr=0 cng=0 gpc=870 gps=697 nf=5 nn=12184
22 np=19877 qsp=7 rpq=6122 cbr=0 cng=0 gpc=876 gps=696 nf=1 nn=12182
23 np=19830 qsp=7 rpq=6077 cbr=0 cng=0 gpc=874 gps=696 nf=2 nn=12181
24 np=23932 qsp=19 rpq=6860 cbr=0 cng=18 gpc=1565 gps=685 nf=170 nn=14634
25 np=25105 qsp=115 rpq=6927 cbr=20 cng=16 gpc=1723 gps=698 nf=264 nn=15457
26 np=20599 qsp=15 rpq=6156 cbr=0 cng=2 gpc=1052 gps=692 nf=14 nn=12683
27 np=19785 qsp=10 rpq=5927 cbr=0 cng=1 gpc=907 gps=691 nf=13 nn=12246
28 np=19763 qsp=7 rpq=5886 cbr=0 cng=1 gpc=906 gps=677 nf=9 nn=12284
29 np=19580 qsp=7 rpq=5813 cbr=0 cng=1 gpc=887 gps=685 nf=8 nn=12186
30 np=19505 qsp=6 rpq=5760 cbr=0 cng=1 gpc=874 gps=717 nf=2 nn=12151
31 np=19464 qsp=8 rpq=5714 cbr=0 cng=1 gpc=874 gps=700 nf=1 nn=12174
32 np=24744 qsp=164 rpq=6657 cbr=0 cng=18 gpc=1743 gps=741 nf=136 nn=15449
33 np=23901 qsp=50 rpq=6547 cbr=15 cng=7 gpc=1738 gps=733 nf=121 nn=14740
34 np=19959 qsp=8 rpq=5740 cbr=0 cng=4 gpc=1004 gps=729 nf=14 nn=12468
35 np=19379 qsp=9 rpq=5558 cbr=0 cng=0 gpc=890 gps=727 nf=9 nn=12195
36 np=19222 qsp=8 rpq=5488 cbr=0 cng=0 gpc=876 gps=716 nf=6 nn=12136
37 np=19177 qsp=8 rpq=5442 cbr=0 cng=0 gpc=870 gps=721 nf=2 nn=12142
38 np=19131 qsp=7 rpq=5399 cbr=0 cng=0 gpc=874 gps=722 nf=4 nn=12132
39 np=19099 qsp=9 rpq=5357 cbr=0 cng=0 gpc=874 gps=720 nf=5 nn=12143
40 np=22259 qsp=29 rpq=6040 cbr=1 cng=7 gpc=1471 gps=784 nf=101 nn=13855
41 np=23011 qsp=81 rpq=6085 cbr=22 cng=26 gpc=1685 gps=664 nf=102 nn=14427
42 np=19460 qsp=8 rpq=5357 cbr=0 cng=0 gpc=1009 gps=793 nf=4 nn=12297
43 np=18930 qsp=8 rpq=5179 cbr=0 cng=0 gpc=951 gps=797 nf=5 nn=11998
44 np=18884 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1207 gps=767 nf=1 nn=11774
45 np=18845 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1200 gps=737 nf=3 nn=11813
46 np=18800 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1316 gps=801 nf=3 nn=11636
47 np=18747 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1051 gps=653 nf=3 nn=12042
48 np=25212 qsp=297 rpq=6193 cbr=3 cng=15 gpc=1704 gps=740 nf=514 nn=16043
49 np=22984 qsp=68 rpq=5734 cbr=3 cng=12 gpc=1633 gps=695 nf=116 nn=14791
50 np=19819 qsp=9 rpq=5028 cbr=0 cng=0 gpc=1025 gps=686 nf=96 nn=12984
51 np=19238 qsp=8 rpq=4864 cbr=0 cng=0 gpc=918 gps=680 nf=63 nn=12713
52 np=19012 qsp=8 rpq=4788 cbr=0 cng=0 gpc=890 gps=680 nf=48 nn=12606
53 np=19121 qsp=7 rpq=4730 cbr=0 cng=0 gpc=875 gps=682 nf=60 nn=12774
54 np=18398 qsp=8 rpq=4682 cbr=0 cng=0 gpc=870 gps=682 nf=9 nn=12155
55 np=18351 qsp=9 rpq=4635 cbr=0 cng=0 gpc=872 gps=678 nf=5 nn=12161
56 np=21897 qsp=11 rpq=5400 cbr=0 cng=10 gpc=1591 gps=701 nf=91 nn=14104
57 np=22398 qsp=27 rpq=5417 cbr=2 cng=11 gpc=1830 gps=707 nf=110 nn=14321
58 np=18956 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1218 gps=712 nf=21 nn=12346
59 np=18243 qsp=8 rpq=4469 cbr=0 cng=0 gpc=908 gps=706 nf=4 nn=12156
60 np=18163 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1031 gps=741 nf=6 nn=11973
61 np=18118 qsp=8 rpq=4369 cbr=0 cng=0 gpc=951 gps=688 nf=4 nn=12106
62 np=18076 qsp=7 rpq=4330 cbr=0 cng=0 gpc=873 gps=662 nf=7 nn=12204
63 np=19242 qsp=44 rpq=4594 cbr=0 cng=0 gpc=1040 gps=671 nf=133 nn=12804
64 np=22060 qsp=234 rpq=4783 cbr=0 cng=11 gpc=1352 gps=684 nf=71 nn=15159
65 np=20694 qsp=33 rpq=4834 cbr=0 cng=9 gpc=1403 gps=686 nf=192 nn=13570
66 np=18467 qsp=32 rpq=4222 cbr=0 cng=4 gpc=924 gps=636 nf=28 nn=12653
67 np=18102 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1150 gps=633 nf=25 nn=12172
68 np=18043 qsp=8 rpq=4072 cbr=0 cng=3 gpc=883 gps=666 nf=23 nn=12396
69 np=17994 qsp=30 rpq=4025 cbr=0 cng=2 gpc=920 gps=683 nf=29 nn=12335
70 np=17952 qsp=32 rpq=3979 cbr=0 cng=3 gpc=879 gps=676 nf=21 nn=12394
71 np=17926 qsp=8 rpq=3942 cbr=0 cng=3 gpc=938 gps=733 nf=33 nn=12277
72 np=20531 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1107 gps=628 nf=28 nn=14623
73 np=25555 qsp=10 rpq=4635 cbr=1 cng=311 gpc=1250 gps=606 nf=1401 nn=17351
74 np=17671 qsp=7 rpq=3829 cbr=0 cng=3 gpc=901 gps=775 nf=7 nn=12156
75 np=17495 qsp=8 rpq=3753 cbr=0 cng=1 gpc=879 gps=770 nf=4 nn=12088
76 np=17430 qsp=8 rpq=3703 cbr=0 cng=0 gpc=873 gps=766 nf=3 nn=12085
77 np=17400 qsp=8 rpq=3660 cbr=0 cng=1 gpc=879 gps=772 nf=3 nn=12085
78 np=17335 qsp=7 rpq=3608 cbr=0 cng=1 gpc=874 gps=764 nf=1 nn=12087
79 np=17289 qsp=9 rpq=3563 cbr=0 cng=1 gpc=872 gps=758 nf=2 nn=12093
80 np=19880 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1130 gps=700 nf=116 nn=14085
81 np=21161 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1318 gps=704 nf=315 nn=14712
82 np=18106 qsp=7 rpq=3461 cbr=0 cng=0 gpc=908 gps=699 nf=150 nn=12888
83 np=17158 qsp=9 rpq=3391 cbr=0 cng=0 gpc=887 gps=740 nf=5 nn=12135
84 np=17086 qsp=8 rpq=3336 cbr=0 cng=0 gpc=881 gps=756 nf=7 nn=12106
85 np=17040 qsp=9 rpq=3293 cbr=0 cng=0 gpc=876 gps=749 nf=5 nn=12117
86 np=16994 qsp=7 rpq=3248 cbr=0 cng=0 gpc=878 gps=747 nf=3 nn=12118
87 np=16949 qsp=8 rpq=3201 cbr=0 cng=0 gpc=880 gps=749 nf=4 nn=12115
88 np=18332 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1148 gps=718 nf=38 nn=12975
89 np=19384 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1287 gps=715 nf=165 nn=13632
90 np=17242 qsp=9 rpq=3121 cbr=0 cng=2 gpc=929 gps=710 nf=38 nn=12442
91 np=16804 qsp=8 rpq=3031 cbr=0 cng=1 gpc=896 gps=698 nf=2 nn=12176
92 np=16725 qsp=9 rpq=2976 cbr=0 cng=0 gpc=879 gps=695 nf=8 nn=12167
93 np=16679 qsp=8 rpq=2932 cbr=0 cng=1 gpc=885 gps=694 nf=3 nn=12164
94 np=16625 qsp=8 rpq=2886 cbr=0 cng=1 gpc=876 gps=698 nf=1 nn=12163
95 np=16587 qsp=7 rpq=2842 cbr=0 cng=0 gpc=879 gps=687 nf=3 nn=12176
96 np=17818 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1115 gps=710 nf=24 nn=12916
97 np=19241 qsp=8 rpq=3184 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16474 qsp=9 rpq=2713 cbr=0 cng=0 gpc=883 gps=710 nf=3 nn=12165
99 np=16408 qsp=8 rpq=2663 cbr=0 cng=0 gpc=876 gps=719 nf=4 nn=12146
100 np=16351 qsp=8 rpq=2609 cbr=0 cng=0 gpc=877 gps=699 nf=1 nn=12165
101 np=16294 qsp=8 rpq=2562 cbr=0 cng=1 gpc=875 gps=692 nf=3 nn=12161
102 np=16252 qsp=8 rpq=2518 cbr=0 cng=0 gpc=878 gps=683 nf=0 nn=12173
103 np=16226 qsp=11 rpq=2475 cbr=0 cng=0 gpc=878 gps=689 nf=6 nn=12178
104 np=17120 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1090 gps=768 nf=13 nn=12597
105 np=17597 qsp=67 rpq=2713 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16159 qsp=9 rpq=2364 cbr=0 cng=0 gpc=888 gps=770 nf=5 nn=12132
107 np=16049 qsp=8 rpq=2296 cbr=0 cng=0 gpc=876 gps=770 nf=2 nn=12105
108 np=16007 qsp=7 rpq=2252 cbr=0 cng=0 gpc=882 gps=774 nf=7 nn=12092
109 np=15959 qsp=8 rpq=2206 cbr=0 cng=0 gpc=882 gps=773 nf=3 nn=12095
110 np=15912 qsp=7 rpq=2160 cbr=0 cng=0 gpc=882 gps=767 nf=4 nn=12099
111 np=15867 qsp=7 rpq=2117 cbr=0 cng=0 gpc=882 gps=772 nf=3 nn=12093
112 np=17960 qsp=27 rpq=2443 cbr=0 cng=4 gpc=1170 gps=701 nf=97 nn=13545
113 np=17615 qsp=12 rpq=2431 cbr=1 cng=2 gpc=1215 gps=722 nf=97 nn=13147
114 np=16334 qsp=8 rpq=2072 cbr=0 cng=0 gpc=958 gps=725 nf=25 nn=12554
115 np=15915 qsp=7 rpq=1947 cbr=0 cng=0 gpc=892 gps=711 nf=14 nn=12351
116 np=15841 qsp=8 rpq=1894 cbr=0 cng=0 gpc=882 gps=724 nf=15 nn=12326
117 np=15777 qsp=8 rpq=1845 cbr=0 cng=1 gpc=880 gps=721 nf=5 nn=12325
118 np=15515 qsp=7 rpq=1798 cbr=0 cng=0 gpc=882 gps=717 nf=2 nn=12116
119 np=15473 qsp=8 rpq=1755 cbr=0 cng=0 gpc=882 gps=720 nf=3 nn=12113
120 np=16460 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1104 gps=662 nf=9 nn=12742
121 np=27489 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1562 gps=706 nf=2320 nn=19383
122 np=15731 qsp=7 rpq=1699 cbr=0 cng=2 gpc=955 gps=668 nf=16 nn=12391
123 np=15326 qsp=8 rpq=1577 cbr=0 cng=0 gpc=886 gps=663 nf=5 nn=12195
124 np=15282 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1045 gps=698 nf=5 nn=12003
125 np=15230 qsp=8 rpq=1488 cbr=0 cng=0 gpc=927 gps=686 nf=2 nn=12127
126 np=15225 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1036 gps=724 nf=5 nn=12014
127 np=15144 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1005 gps=669 nf=3 nn=12072
rcu_bh:
0 np=16246 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3962
1 np=15842 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=3045
2 np=14670 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1161
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19190 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2535
9 np=15053 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2565
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12130 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
13 np=12699 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=573
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18611 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3425
17 np=15293 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2468
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14634 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1976
25 np=15457 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2607
26 np=12683 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=501
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15449 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2351
33 np=14740 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2597
34 np=12468 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=348
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13855 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1760
41 np=14427 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2297
42 np=12297 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=255
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=16043 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2344
49 np=14791 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2174
50 np=12984 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=383
51 np=12713 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14104 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1923
57 np=14321 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2292
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12804 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=150
64 np=15159 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=969
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12277 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17351 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=872
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13545 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=696
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 4:54 ` Paul E. McKenney
2011-05-11 6:03 ` Yinghai Lu
2011-05-11 6:42 ` Yinghai Lu
@ 2011-05-11 16:54 ` Yinghai Lu
2011-05-11 16:56 ` Yinghai Lu
3 siblings, 0 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 16:54 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On Tue, May 10, 2011 at 9:54 PM, Paul E. McKenney
<paulmck@linux.vnet.ibm.com> wrote:
> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>> > On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>> >> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>> >>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>> >>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>> >>>>>
>> >>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>> >>>>>
>> >>>>>> Hello, Ingo,
>> >>>>>>
>> >>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>> >>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>> >>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>> >>>>>> of the commits. There are a number of smaller features and bug fixes.
>> >>>>>> All have been sent to LKML in the following batches:
>> >>>>>>
>> >>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>> >>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>> >>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>> >>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>> >>>>>>
>> >>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>> >>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>> >>>>>> Acked-by:s, and I will deal with these later.
>> >>>>>>
>> >>>>>> These channges are available in the -rcu git repository at:
>> >>>>>>
>> >>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>> >>>>>
>> >>>>> Pulled, thanks a lot Paul!
>> >>>>>
>> >>>>
>> >>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>> >>>>
>> >>>> after hard code to enable rcu_cpu_stall_suppress
>> >>>>
>> >>>> Index: linux-2.6/kernel/rcutree.c
>> >>>> ===================================================================
>> >>>> --- linux-2.6.orig/kernel/rcutree.c
>> >>>> +++ linux-2.6/kernel/rcutree.c
>> >>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>> >>>> module_param(qhimark, int, 0);
>> >>>> module_param(qlowmark, int, 0);
>> >>>>
>> >>>> -int rcu_cpu_stall_suppress __read_mostly;
>> >>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>> >>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>> >>>>
>> >>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>> >>>>
>> >>>> will get system hang after pnp ACPI init.
>> >>>
>> >>> Could you please send the stack traces from the RCU CPU stall? Also,
>> >>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>> >>>
>> >>> Thanx, Paul
>> >>
>> >> Do not have time to bisect it at this point.
>> >
>> > Could you please send the stack traces from the RCU CPU stall?
>
> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
> Everyone is idle, except for CPU 124, which detected the stall, and
> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
> that looks like a backtrace error to me. The fact that it hangs if you
> disable RCU CPU stall detection leads me to believe that something real
> is being detected.
>
> This looks very similar to the situation people were seeing before
> ce31332d3c77532d6ea97ddcb475a2b02dd358b4 was applied, so I have attached
> the diagnostic script that helped track this down. Could you please
> enable CONFIG_RCU_TRACE, mount debugfs, and run the attached script,
> and send me the output? Please check to make sure that the script knows
> where you mounted debugfs, of course.
>
Wed May 11 14:33:35 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4164 gpnum=4165
rcu_bh: completed=-282 gpnum=18446744073709551334
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4164 g=4165 s=2 jfq=0 j=92fc nfqs=6792/nfqsng=0(6792) fqlh=737
ff/ff ..>.. 0:254 ^0
fdff/ffff ..>.. 0:15 ^0 fffc/ffff ..>.. 16:31 ^1 ffff/ffff ..>..
32:47 ^2 ffff/ffff ..>.. 48:63 ^3 fdff/ffff ..>.. 64:79 ^4
ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff
..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9
0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207
^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>..
240:254 ^15
rcu_bh:
c=18446744073709551334 g=18446744073709551334 s=0 jfq=-1351 j=92fc
nfqs=21/nfqsng=0(21) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2
0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>..
80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0
..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10
0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223
^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=6072/72 dn=4 df=2570 of=0
ri=601 ql=0 b=10 ci=6119 co=0 ca=0
1 c=4163 g=4164 pq=0 pqc=4162 qp=1 dt=12470/0 dn=0 df=2722 of=0
ri=13 ql=0 b=10 ci=11239 co=0 ca=0
2 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=11168/0 dn=2 df=3326 of=0
ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1694/0 dn=0 df=3570 of=0 ri=6
ql=0 b=10 ci=123 co=0 ca=0
4 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1538/0 dn=2 df=3580 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
5 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1608/0 dn=0 df=3597 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
6 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1598/0 dn=2 df=3597 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
7 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1646/0 dn=2 df=3597 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
8 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=9158/0 dn=2 df=2368 of=0
ri=760 ql=0 b=10 ci=7223 co=0 ca=0
9 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=3131/1 dn=0 df=2685 of=0 ri=1
ql=2 b=10 ci=13631 co=0 ca=0
10 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=2020/0 dn=0 df=3444 of=0 ri=1
ql=0 b=10 ci=702 co=0 ca=0
11 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1760/0 dn=2 df=3590 of=0 ri=1
ql=0 b=10 ci=25 co=0 ca=0
12 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1800/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
13 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=3594/0 dn=2 df=3598 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
14 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1704/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
15 c=4144 g=4144 pq=1 pqc=4143 qp=0 dt=1648/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
16 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=18235/1 dn=0 df=2734 of=0
ri=103 ql=2 b=10 ci=3533 co=0 ca=0
17 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=1273/1 dn=0 df=2736 of=0 ri=1
ql=11 b=10 ci=11685 co=0 ca=0
18 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1678/0 dn=0 df=3444 of=0 ri=1
ql=0 b=10 ci=665 co=0 ca=0
19 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=2044/0 dn=0 df=3568 of=0 ri=1
ql=0 b=10 ci=121 co=0 ca=0
20 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=3666/0 dn=0 df=3596 of=0 ri=1
ql=0 b=10 ci=24 co=0 ca=0
21 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1788/0 dn=2 df=3598 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
22 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1726/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
23 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1720/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
24 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=14333/1 dn=0 df=2865 of=0 ri=1
ql=2 b=10 ci=3337 co=0 ca=0
25 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=1593/1 dn=0 df=2707 of=0 ri=1
ql=9 b=10 ci=19146 co=0 ca=0
26 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1786/0 dn=2 df=3427 of=0 ri=1
ql=0 b=10 ci=1035 co=0 ca=0
27 c=4146 g=4146 pq=1 pqc=4145 qp=0 dt=1586/0 dn=2 df=3571 of=0 ri=1
ql=0 b=10 ci=157 co=0 ca=0
28 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1582/0 dn=0 df=3569 of=0 ri=1
ql=0 b=10 ci=92 co=0 ca=0
29 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1550/0 dn=2 df=3594 of=0 ri=1
ql=0 b=10 ci=14 co=0 ca=0
30 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1650/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
31 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1662/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
32 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=5537/1 dn=0 df=2756 of=0 ri=97
ql=2 b=10 ci=3209 co=0 ca=0
33 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=1367/1 dn=0 df=2725 of=0 ri=1
ql=9 b=10 ci=17809 co=0 ca=0
34 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1784/0 dn=0 df=3474 of=0 ri=1
ql=0 b=10 ci=538 co=0 ca=0
35 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1670/0 dn=2 df=3577 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
36 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1682/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
37 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1664/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
38 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1622/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
39 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1664/0 dn=2 df=3598 of=0 ri=1
ql=0 b=10 ci=18 co=0 ca=0
40 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1846/0 dn=0 df=2958 of=0 ri=1
ql=0 b=10 ci=10276 co=0 ca=0
41 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1621/1 dn=0 df=2831 of=0 ri=1
ql=3 b=10 ci=16207 co=0 ca=0
42 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1764/0 dn=0 df=3503 of=0 ri=1
ql=0 b=10 ci=176 co=0 ca=0
43 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1706/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
44 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1654/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
45 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=2048/0 dn=0 df=3600 of=0 ri=1
ql=0 b=10 ci=12 co=0 ca=0
46 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1670/0 dn=2 df=3599 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
47 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1694/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
48 c=4151 g=4152 pq=1 pqc=4151 qp=0 dt=35230/0 dn=2 df=2620 of=0
ri=161 ql=0 b=10 ci=20305 co=0 ca=0
49 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1534/0 dn=0 df=2847 of=0 ri=15
ql=0 b=10 ci=28443 co=0 ca=0
50 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1910/0 dn=0 df=3461 of=0 ri=1
ql=0 b=10 ci=537 co=0 ca=0
51 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1994/0 dn=0 df=3564 of=0 ri=1
ql=0 b=10 ci=102 co=0 ca=0
52 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1834/0 dn=2 df=3584 of=0 ri=1
ql=0 b=10 ci=57 co=0 ca=0
53 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1630/0 dn=0 df=3596 of=0 ri=1
ql=0 b=10 ci=15 co=0 ca=0
54 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1608/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
55 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1680/0 dn=2 df=3599 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
56 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1650/0 dn=2 df=2887 of=0 ri=1
ql=0 b=10 ci=2645 co=0 ca=0
57 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1284/0 dn=0 df=2776 of=0 ri=1
ql=0 b=10 ci=11764 co=0 ca=0
58 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2056/0 dn=0 df=3462 of=0 ri=1
ql=0 b=10 ci=650 co=0 ca=0
59 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2004/0 dn=2 df=3591 of=0 ri=1
ql=0 b=10 ci=40 co=0 ca=0
60 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1686/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
61 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1598/0 dn=2 df=3599 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
62 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1686/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
63 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=6030/0 dn=0 df=3414 of=0 ri=30
ql=0 b=10 ci=150 co=0 ca=0
64 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=8338/0 dn=2 df=3180 of=0
ri=274 ql=0 b=10 ci=1436 co=0 ca=0
65 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2888/0 dn=0 df=3076 of=0 ri=6
ql=0 b=10 ci=5781 co=0 ca=0
66 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=3958/0 dn=0 df=3549 of=0 ri=6
ql=0 b=10 ci=131 co=0 ca=0
67 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1616/0 dn=0 df=3593 of=0 ri=1
ql=0 b=10 ci=26 co=0 ca=0
68 c=4155 g=4156 pq=1 pqc=4155 qp=0 dt=1622/0 dn=0 df=3597 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
69 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1672/0 dn=2 df=3597 of=0 ri=6
ql=0 b=10 ci=11 co=0 ca=0
70 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1656/0 dn=2 df=3598 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
71 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1588/0 dn=2 df=3598 of=0 ri=1
ql=0 b=10 ci=12 co=0 ca=0
72 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=6142/0 dn=4 df=3388 of=0
ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4164 g=4165 pq=1 pqc=4164 qp=0 dt=7465/1 dn=0 df=2868 of=0 ri=3
ql=32 b=10 ci=72222 co=0 ca=0
74 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1690/0 dn=2 df=3570 of=0 ri=1
ql=0 b=10 ci=125 co=0 ca=0
75 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1680/0 dn=2 df=3597 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
76 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1678/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
77 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1666/0 dn=0 df=3597 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
78 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1714/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
79 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1620/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
80 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=2154/0 dn=2 df=3331 of=0 ri=1
ql=0 b=10 ci=590 co=0 ca=0
81 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=3648/0 dn=0 df=3130 of=0 ri=1
ql=0 b=10 ci=5193 co=0 ca=0
82 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1626/0 dn=2 df=3571 of=0 ri=1
ql=0 b=10 ci=106 co=0 ca=0
83 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1672/0 dn=2 df=3594 of=0 ri=1
ql=0 b=10 ci=22 co=0 ca=0
84 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1610/0 dn=2 df=3599 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
85 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1640/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
86 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1644/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
87 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1672/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
88 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1852/0 dn=0 df=3349 of=0 ri=1
ql=0 b=10 ci=884 co=0 ca=0
89 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=2814/0 dn=0 df=3230 of=0 ri=1
ql=0 b=10 ci=2572 co=0 ca=0
90 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1542/0 dn=0 df=3554 of=0 ri=1
ql=0 b=10 ci=154 co=0 ca=0
91 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1770/0 dn=2 df=3593 of=0 ri=1
ql=0 b=10 ci=20 co=0 ca=0
92 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1728/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
93 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1718/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
94 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1716/0 dn=2 df=3602 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
95 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1608/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
96 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1718/0 dn=2 df=3372 of=0 ri=1
ql=0 b=10 ci=493 co=0 ca=0
97 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2656/0 dn=0 df=3278 of=0 ri=1
ql=0 b=10 ci=1015 co=0 ca=0
98 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1720/0 dn=2 df=3592 of=0 ri=1
ql=0 b=10 ci=27 co=0 ca=0
99 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1640/0 dn=2 df=3599 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
100 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1728/0 dn=2 df=3598 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
101 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1704/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
102 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1690/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
103 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1696/0 dn=2 df=3597 of=0 ri=1
ql=0 b=10 ci=1923 co=0 ca=0
104 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1694/0 dn=2 df=3406 of=0 ri=1
ql=0 b=10 ci=387 co=0 ca=0
105 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2726/0 dn=0 df=3306 of=0 ri=1
ql=0 b=10 ci=20365 co=0 ca=0
106 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1814/0 dn=2 df=3589 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
107 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1722/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
108 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1650/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
109 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1662/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
110 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1714/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
111 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1626/0 dn=2 df=3599 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
112 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=7810/0 dn=0 df=3325 of=0 ri=36
ql=0 b=10 ci=613 co=0 ca=0
113 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=2392/0 dn=0 df=3277 of=0 ri=1
ql=0 b=10 ci=5505 co=0 ca=0
114 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1654/0 dn=0 df=3518 of=0 ri=1
ql=0 b=10 ci=497 co=0 ca=0
115 c=4138 g=4138 pq=1 pqc=4137 qp=0 dt=1566/0 dn=2 df=3589 of=0 ri=1
ql=0 b=10 ci=38 co=0 ca=0
116 c=4139 g=4140 pq=1 pqc=4139 qp=0 dt=1736/0 dn=2 df=3596 of=0 ri=1
ql=0 b=10 ci=34 co=0 ca=0
117 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1596/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=10 co=0 ca=0
118 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1638/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
119 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=2682/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
120 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=2234/0 dn=0 df=3397 of=0 ri=1
ql=0 b=10 ci=393 co=0 ca=0
121 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=2980/0 dn=0 df=2005 of=0 ri=1
ql=0 b=10 ci=48877 co=0 ca=0
122 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1554/0 dn=0 df=3527 of=0 ri=1
ql=0 b=10 ci=333 co=0 ca=0
123 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1630/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
124 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1682/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
125 c=4140 g=4140 pq=1 pqc=4139 qp=0 dt=1650/0 dn=2 df=3600 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
126 c=4141 g=4142 pq=1 pqc=4141 qp=0 dt=2466/0 dn=2 df=3595 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
127 c=4142 g=4142 pq=1 pqc=4141 qp=0 dt=1720/0 dn=2 df=3601 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=6072/72 dn=4 df=7 of=0 ri=0 ql=0 b=10
ci=6 co=0 ca=0
1 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=12470/0 dn=0 df=12 of=0 ri=0 ql=0
b=10 ci=4 co=0 ca=0
2 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=11168/0 dn=2 df=13 of=0 ri=0 ql=0
b=10 ci=11 co=0 ca=0
3 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1694/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
4 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1538/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
5 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1608/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
6 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1598/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
7 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1646/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
8 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=9158/0 dn=2 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
9 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=3131/1 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
10 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2020/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
11 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1760/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
12 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1800/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
13 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=3597/1 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
14 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1704/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
15 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1648/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
16 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=18235/1 dn=0 df=13 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
17 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1273/1 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
18 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1678/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
19 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2044/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
20 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=3666/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
21 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1788/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
22 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1726/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
23 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1720/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
24 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=14333/1 dn=0 df=14 of=0 ri=0 ql=0
b=10 ci=0 co=0 ca=0
25 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1593/1 dn=0 df=15 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
26 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1786/0 dn=2 df=16 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
27 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1586/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
28 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1582/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
29 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1550/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
30 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
31 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1662/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
32 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=5537/1 dn=0 df=12 of=0 ri=0 ql=0 b=10
ci=2 co=0 ca=0
33 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1367/1 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
34 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1784/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
35 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1670/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
36 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1682/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
37 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1664/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
38 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1622/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
39 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1664/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
40 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1846/0 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
41 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1621/1 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
42 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1764/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
43 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1706/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
44 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1654/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
45 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2048/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
46 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1670/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
47 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1694/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
48 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=35230/0 dn=2 df=12 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
49 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1534/0 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
50 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1910/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
51 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1994/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
52 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1834/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
53 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1630/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
54 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1608/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
55 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
56 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
57 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1284/0 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
58 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2056/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
59 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2004/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
60 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1686/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
61 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1598/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
62 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1686/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
63 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=6030/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
64 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=8338/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
65 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2888/0 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
66 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=3958/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
67 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1616/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
68 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1622/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
69 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1672/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
70 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1656/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
71 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1588/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
72 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=6142/0 dn=4 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
73 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=7465/1 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
74 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1690/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
75 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
76 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1678/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
77 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1666/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
78 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1714/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
79 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1620/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
80 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2154/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
81 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=3648/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
82 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1626/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
83 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1672/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
84 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1610/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
85 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1640/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
86 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1644/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
87 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1672/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
88 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1852/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
89 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2814/0 dn=0 df=15 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
90 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1542/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
91 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1770/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
92 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1728/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
93 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
94 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1716/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
95 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1608/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
96 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
97 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=2656/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
98 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1720/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
99 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1640/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
100 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1728/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
101 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1704/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
102 c=18446744073709551332 g=18446744073709551332 pq=1
pqc=18446744073709551331 qp=0 dt=1690/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
103 c=18446744073709551332 g=18446744073709551332 pq=1
pqc=18446744073709551331 qp=0 dt=1696/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
104 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1694/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
105 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=2728/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
106 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1814/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
107 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1722/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
108 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1650/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
109 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1662/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
110 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1714/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
111 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1626/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
112 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=7810/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
113 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=2392/0 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
114 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1654/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
115 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1566/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
116 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1736/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
117 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1596/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
118 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1638/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
119 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2682/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
120 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2234/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
121 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2980/0 dn=0 df=14 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
122 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1554/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
123 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1630/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
124 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1682/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
125 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
126 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2466/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
127 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1720/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=24806 qsp=1521 rpq=6020 cbr=1 cng=1254 gpc=1128 gps=131 nf=683 nn=15589
1 np=26594 qsp=60 rpq=8004 cbr=5 cng=44 gpc=1620 gps=748 nf=430 nn=15743
2 np=23556 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1101 gps=715 nf=180 nn=14668
3 np=21600 qsp=29 rpq=6997 cbr=0 cng=5 gpc=909 gps=708 nf=96 nn=12885
4 np=21508 qsp=9 rpq=6943 cbr=0 cng=5 gpc=983 gps=729 nf=100 nn=12748
5 np=21376 qsp=28 rpq=6885 cbr=0 cng=2 gpc=875 gps=683 nf=109 nn=12822
6 np=21338 qsp=30 rpq=6845 cbr=0 cng=2 gpc=874 gps=689 nf=104 nn=12824
7 np=21298 qsp=32 rpq=6806 cbr=0 cng=2 gpc=981 gps=700 nf=107 nn=12702
8 np=30293 qsp=840 rpq=8100 cbr=1 cng=45 gpc=1657 gps=779 nf=590 nn=19121
9 np=25376 qsp=32 rpq=7677 cbr=1 cng=22 gpc=1742 gps=783 nf=171 nn=14980
10 np=21235 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1269 gps=806 nf=15 nn=12285
11 np=20409 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1063 gps=807 nf=5 nn=11903
12 np=20307 qsp=9 rpq=6574 cbr=0 cng=1 gpc=856 gps=755 nf=2 nn=12119
13 np=20785 qsp=9 rpq=6541 cbr=0 cng=1 gpc=909 gps=735 nf=4 nn=12595
14 np=20213 qsp=7 rpq=6482 cbr=0 cng=0 gpc=775 gps=758 nf=2 nn=12196
15 np=20176 qsp=9 rpq=6443 cbr=0 cng=0 gpc=870 gps=757 nf=3 nn=12103
16 np=28721 qsp=173 rpq=7569 cbr=0 cng=27 gpc=1742 gps=748 nf=262 nn=18373
17 np=25111 qsp=26 rpq=7251 cbr=1 cng=11 gpc=1709 gps=698 nf=205 nn=15236
18 np=20875 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1027 gps=681 nf=26 nn=12653
19 np=20173 qsp=11 rpq=6309 cbr=0 cng=1 gpc=895 gps=692 nf=8 nn=12268
20 np=19991 qsp=7 rpq=6224 cbr=0 cng=0 gpc=876 gps=671 nf=5 nn=12215
21 np=19928 qsp=9 rpq=6173 cbr=0 cng=0 gpc=869 gps=697 nf=5 nn=12184
22 np=19876 qsp=7 rpq=6122 cbr=0 cng=0 gpc=875 gps=696 nf=1 nn=12182
23 np=19829 qsp=7 rpq=6077 cbr=0 cng=0 gpc=873 gps=696 nf=2 nn=12181
24 np=23838 qsp=19 rpq=6839 cbr=0 cng=18 gpc=1548 gps=685 nf=167 nn=14581
25 np=24901 qsp=96 rpq=6902 cbr=12 cng=15 gpc=1692 gps=698 nf=261 nn=15321
26 np=20559 qsp=15 rpq=6148 cbr=0 cng=2 gpc=1044 gps=692 nf=14 nn=12659
27 np=19784 qsp=10 rpq=5927 cbr=0 cng=1 gpc=906 gps=691 nf=13 nn=12246
28 np=19762 qsp=7 rpq=5886 cbr=0 cng=1 gpc=905 gps=677 nf=9 nn=12284
29 np=19579 qsp=7 rpq=5813 cbr=0 cng=1 gpc=886 gps=685 nf=8 nn=12186
30 np=19504 qsp=6 rpq=5760 cbr=0 cng=1 gpc=873 gps=717 nf=2 nn=12151
31 np=19463 qsp=8 rpq=5714 cbr=0 cng=1 gpc=873 gps=700 nf=1 nn=12174
32 np=24626 qsp=164 rpq=6629 cbr=0 cng=17 gpc=1707 gps=741 nf=131 nn=15401
33 np=23699 qsp=37 rpq=6514 cbr=5 cng=7 gpc=1707 gps=733 nf=118 nn=14615
34 np=19922 qsp=8 rpq=5728 cbr=0 cng=4 gpc=994 gps=729 nf=14 nn=12453
35 np=19378 qsp=9 rpq=5558 cbr=0 cng=0 gpc=889 gps=727 nf=9 nn=12195
36 np=19221 qsp=8 rpq=5488 cbr=0 cng=0 gpc=875 gps=716 nf=6 nn=12136
37 np=19176 qsp=8 rpq=5442 cbr=0 cng=0 gpc=869 gps=721 nf=2 nn=12142
38 np=19130 qsp=7 rpq=5399 cbr=0 cng=0 gpc=873 gps=722 nf=4 nn=12132
39 np=19098 qsp=9 rpq=5357 cbr=0 cng=0 gpc=873 gps=720 nf=5 nn=12143
40 np=22183 qsp=29 rpq=6020 cbr=1 cng=7 gpc=1469 gps=784 nf=83 nn=13819
41 np=22788 qsp=66 rpq=6047 cbr=13 cng=13 gpc=1670 gps=661 nf=88 nn=14296
42 np=19432 qsp=8 rpq=5349 cbr=0 cng=0 gpc=1001 gps=793 nf=4 nn=12285
43 np=18929 qsp=8 rpq=5179 cbr=0 cng=0 gpc=950 gps=797 nf=5 nn=11998
44 np=18883 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1206 gps=767 nf=1 nn=11774
45 np=18844 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1199 gps=737 nf=3 nn=11813
46 np=18799 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1315 gps=801 nf=3 nn=11636
47 np=18746 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1050 gps=653 nf=3 nn=12042
48 np=25095 qsp=297 rpq=6163 cbr=3 cng=14 gpc=1687 gps=740 nf=503 nn=15985
49 np=22838 qsp=68 rpq=5689 cbr=3 cng=12 gpc=1596 gps=695 nf=116 nn=14727
50 np=19790 qsp=9 rpq=5021 cbr=0 cng=0 gpc=1017 gps=686 nf=96 nn=12970
51 np=19209 qsp=8 rpq=4858 cbr=0 cng=0 gpc=910 gps=679 nf=63 nn=12699
52 np=19011 qsp=8 rpq=4788 cbr=0 cng=0 gpc=889 gps=680 nf=48 nn=12606
53 np=19120 qsp=7 rpq=4730 cbr=0 cng=0 gpc=874 gps=682 nf=60 nn=12774
54 np=18397 qsp=8 rpq=4682 cbr=0 cng=0 gpc=869 gps=682 nf=9 nn=12155
55 np=18350 qsp=9 rpq=4635 cbr=0 cng=0 gpc=871 gps=678 nf=5 nn=12161
56 np=21837 qsp=11 rpq=5384 cbr=0 cng=10 gpc=1575 gps=701 nf=91 nn=14076
57 np=22322 qsp=27 rpq=5397 cbr=2 cng=11 gpc=1821 gps=707 nf=98 nn=14286
58 np=18955 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1217 gps=712 nf=21 nn=12346
59 np=18242 qsp=8 rpq=4469 cbr=0 cng=0 gpc=907 gps=706 nf=4 nn=12156
60 np=18162 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1030 gps=741 nf=6 nn=11973
61 np=18117 qsp=8 rpq=4369 cbr=0 cng=0 gpc=950 gps=688 nf=4 nn=12106
62 np=18075 qsp=7 rpq=4330 cbr=0 cng=0 gpc=872 gps=662 nf=7 nn=12204
63 np=19170 qsp=44 rpq=4583 cbr=0 cng=0 gpc=1019 gps=671 nf=133 nn=12764
64 np=22024 qsp=234 rpq=4773 cbr=0 cng=10 gpc=1344 gps=684 nf=67 nn=15146
65 np=20692 qsp=33 rpq=4833 cbr=0 cng=9 gpc=1402 gps=686 nf=192 nn=13570
66 np=18466 qsp=32 rpq=4222 cbr=0 cng=4 gpc=923 gps=636 nf=28 nn=12653
67 np=18101 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1149 gps=633 nf=25 nn=12172
68 np=18042 qsp=8 rpq=4072 cbr=0 cng=3 gpc=882 gps=666 nf=23 nn=12396
69 np=17993 qsp=30 rpq=4025 cbr=0 cng=2 gpc=919 gps=683 nf=29 nn=12335
70 np=17951 qsp=32 rpq=3979 cbr=0 cng=3 gpc=878 gps=676 nf=21 nn=12394
71 np=17901 qsp=8 rpq=3935 cbr=0 cng=3 gpc=931 gps=733 nf=33 nn=12266
72 np=20530 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1106 gps=628 nf=28 nn=14623
73 np=25529 qsp=10 rpq=4626 cbr=1 cng=311 gpc=1244 gps=606 nf=1401 nn=17340
74 np=17670 qsp=7 rpq=3829 cbr=0 cng=3 gpc=900 gps=775 nf=7 nn=12156
75 np=17494 qsp=8 rpq=3753 cbr=0 cng=1 gpc=878 gps=770 nf=4 nn=12088
76 np=17429 qsp=8 rpq=3703 cbr=0 cng=0 gpc=872 gps=766 nf=3 nn=12085
77 np=17399 qsp=8 rpq=3660 cbr=0 cng=1 gpc=878 gps=772 nf=3 nn=12085
78 np=17334 qsp=7 rpq=3608 cbr=0 cng=1 gpc=873 gps=764 nf=1 nn=12087
79 np=17288 qsp=9 rpq=3563 cbr=0 cng=1 gpc=871 gps=758 nf=2 nn=12093
80 np=19879 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1129 gps=700 nf=116 nn=14085
81 np=21160 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1317 gps=704 nf=315 nn=14712
82 np=18105 qsp=7 rpq=3461 cbr=0 cng=0 gpc=907 gps=699 nf=150 nn=12888
83 np=17157 qsp=9 rpq=3391 cbr=0 cng=0 gpc=886 gps=740 nf=5 nn=12135
84 np=17085 qsp=8 rpq=3336 cbr=0 cng=0 gpc=880 gps=756 nf=7 nn=12106
85 np=17039 qsp=9 rpq=3293 cbr=0 cng=0 gpc=875 gps=749 nf=5 nn=12117
86 np=16993 qsp=7 rpq=3248 cbr=0 cng=0 gpc=877 gps=747 nf=3 nn=12118
87 np=16948 qsp=8 rpq=3201 cbr=0 cng=0 gpc=879 gps=749 nf=4 nn=12115
88 np=18331 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1147 gps=718 nf=38 nn=12975
89 np=19383 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1286 gps=715 nf=165 nn=13632
90 np=17241 qsp=9 rpq=3121 cbr=0 cng=2 gpc=928 gps=710 nf=38 nn=12442
91 np=16803 qsp=8 rpq=3031 cbr=0 cng=1 gpc=895 gps=698 nf=2 nn=12176
92 np=16724 qsp=9 rpq=2976 cbr=0 cng=0 gpc=878 gps=695 nf=8 nn=12167
93 np=16678 qsp=8 rpq=2932 cbr=0 cng=1 gpc=884 gps=694 nf=3 nn=12164
94 np=16624 qsp=8 rpq=2886 cbr=0 cng=1 gpc=875 gps=698 nf=1 nn=12163
95 np=16586 qsp=7 rpq=2842 cbr=0 cng=0 gpc=878 gps=687 nf=3 nn=12176
96 np=17817 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1114 gps=710 nf=24 nn=12916
97 np=19239 qsp=8 rpq=3182 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16473 qsp=9 rpq=2713 cbr=0 cng=0 gpc=882 gps=710 nf=3 nn=12165
99 np=16407 qsp=8 rpq=2663 cbr=0 cng=0 gpc=875 gps=719 nf=4 nn=12146
100 np=16350 qsp=8 rpq=2609 cbr=0 cng=0 gpc=876 gps=699 nf=1 nn=12165
101 np=16293 qsp=8 rpq=2562 cbr=0 cng=1 gpc=874 gps=692 nf=3 nn=12161
102 np=16251 qsp=8 rpq=2518 cbr=0 cng=0 gpc=877 gps=683 nf=0 nn=12173
103 np=16225 qsp=11 rpq=2475 cbr=0 cng=0 gpc=877 gps=689 nf=6 nn=12178
104 np=17119 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1089 gps=768 nf=13 nn=12597
105 np=17595 qsp=67 rpq=2711 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16158 qsp=9 rpq=2364 cbr=0 cng=0 gpc=887 gps=770 nf=5 nn=12132
107 np=16048 qsp=8 rpq=2296 cbr=0 cng=0 gpc=875 gps=770 nf=2 nn=12105
108 np=16006 qsp=7 rpq=2252 cbr=0 cng=0 gpc=881 gps=774 nf=7 nn=12092
109 np=15958 qsp=8 rpq=2206 cbr=0 cng=0 gpc=881 gps=773 nf=3 nn=12095
110 np=15911 qsp=7 rpq=2160 cbr=0 cng=0 gpc=881 gps=767 nf=4 nn=12099
111 np=15866 qsp=7 rpq=2117 cbr=0 cng=0 gpc=881 gps=772 nf=3 nn=12093
112 np=17863 qsp=27 rpq=2417 cbr=0 cng=4 gpc=1163 gps=701 nf=82 nn=13496
113 np=17613 qsp=12 rpq=2430 cbr=1 cng=2 gpc=1214 gps=722 nf=97 nn=13147
114 np=16333 qsp=8 rpq=2072 cbr=0 cng=0 gpc=957 gps=725 nf=25 nn=12554
115 np=15914 qsp=7 rpq=1947 cbr=0 cng=0 gpc=891 gps=711 nf=14 nn=12351
116 np=15840 qsp=8 rpq=1894 cbr=0 cng=0 gpc=881 gps=724 nf=15 nn=12326
117 np=15776 qsp=8 rpq=1845 cbr=0 cng=1 gpc=879 gps=721 nf=5 nn=12325
118 np=15514 qsp=7 rpq=1798 cbr=0 cng=0 gpc=881 gps=717 nf=2 nn=12116
119 np=15472 qsp=8 rpq=1755 cbr=0 cng=0 gpc=881 gps=720 nf=3 nn=12113
120 np=16459 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1103 gps=662 nf=9 nn=12742
121 np=27487 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1560 gps=706 nf=2320 nn=19383
122 np=15730 qsp=7 rpq=1699 cbr=0 cng=2 gpc=954 gps=668 nf=16 nn=12391
123 np=15325 qsp=8 rpq=1577 cbr=0 cng=0 gpc=885 gps=663 nf=5 nn=12195
124 np=15281 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1044 gps=698 nf=5 nn=12003
125 np=15228 qsp=8 rpq=1488 cbr=0 cng=0 gpc=925 gps=686 nf=2 nn=12127
126 np=15223 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1034 gps=724 nf=5 nn=12014
127 np=15142 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1003 gps=669 nf=3 nn=12072
rcu_bh:
0 np=15589 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3305
1 np=15743 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=2946
2 np=14668 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1159
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19121 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2466
9 np=14980 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2492
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12119 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
13 np=12595 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=469
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18373 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3187
17 np=15236 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2411
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14581 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1923
25 np=15321 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2471
26 np=12659 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=477
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15401 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2303
33 np=14615 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2472
34 np=12453 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=333
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13819 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1724
41 np=14296 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2166
42 np=12285 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=243
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=15985 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2286
49 np=14727 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2110
50 np=12970 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=369
51 np=12699 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=85
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14076 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1895
57 np=14286 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2257
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12764 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=110
64 np=15146 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=956
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12266 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17340 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=861
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13496 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=647
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
Wed May 11 14:33:58 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4217 gpnum=4218
rcu_bh: completed=-278 gpnum=18446744073709551338
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4217 g=4218 s=2 jfq=1 j=a9d7 nfqs=6853/nfqsng=0(6853) fqlh=738
ff/ff ..>.. 0:254 ^0
ffff/ffff ..>.. 0:15 ^0 ffff/ffff ..>.. 16:31 ^1 ffff/ffff ..>..
32:47 ^2 fcfc/ffff ..>.. 48:63 ^3 ffff/ffff ..>.. 64:79 ^4
ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff
..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9
0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207
^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>..
240:254 ^15
rcu_bh:
c=18446744073709551338 g=18446744073709551338 s=0 jfq=-5462 j=a9d7
nfqs=26/nfqsng=0(26) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2
0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>..
80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0
..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10
0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223
^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=6377/73 dn=4 df=2610 of=0
ri=601 ql=2 b=10 ci=6137 co=0 ca=0
1 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=12539/1 dn=0 df=2761 of=0
ri=13 ql=9 b=10 ci=11255 co=0 ca=0
2 c=4182 g=4182 pq=1 pqc=4181 qp=0 dt=11202/0 dn=2 df=3378 of=0
ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4190 g=4191 pq=1 pqc=4190 qp=1 dt=1716/0 dn=0 df=3622 of=0 ri=6
ql=0 b=10 ci=123 co=0 ca=0
4 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1580/0 dn=2 df=3632 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
5 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1634/0 dn=0 df=3649 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
6 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1632/0 dn=2 df=3649 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
7 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1680/0 dn=2 df=3649 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
8 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=9281/1 dn=2 df=2418 of=0
ri=760 ql=2 b=10 ci=7225 co=0 ca=0
9 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=3181/1 dn=0 df=2729 of=0 ri=1
ql=9 b=10 ci=13650 co=0 ca=0
10 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2054/0 dn=0 df=3496 of=0 ri=1
ql=0 b=10 ci=702 co=0 ca=0
11 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1802/0 dn=2 df=3642 of=0 ri=1
ql=0 b=10 ci=25 co=0 ca=0
12 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1844/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
13 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=3972/0 dn=2 df=3650 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
14 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1746/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
15 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1674/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
16 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=18828/0 dn=0 df=2779 of=0
ri=103 ql=0 b=10 ci=3539 co=0 ca=0
17 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=1311/1 dn=0 df=2777 of=0 ri=1
ql=3 b=10 ci=11712 co=0 ca=0
18 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1712/0 dn=0 df=3496 of=0 ri=1
ql=0 b=10 ci=665 co=0 ca=0
19 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2078/0 dn=0 df=3620 of=0 ri=1
ql=0 b=10 ci=121 co=0 ca=0
20 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=3688/0 dn=0 df=3648 of=0 ri=1
ql=0 b=10 ci=24 co=0 ca=0
21 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1810/0 dn=2 df=3650 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
22 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1752/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
23 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1762/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
24 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=14418/0 dn=0 df=2910 of=0 ri=1
ql=0 b=10 ci=3348 co=0 ca=0
25 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1630/0 dn=0 df=2740 of=0 ri=1
ql=0 b=10 ci=22440 co=0 ca=0
26 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1824/0 dn=2 df=3477 of=0 ri=1
ql=0 b=10 ci=1047 co=0 ca=0
27 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1608/0 dn=2 df=3623 of=0 ri=1
ql=0 b=10 ci=157 co=0 ca=0
28 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1616/0 dn=0 df=3621 of=0 ri=1
ql=0 b=10 ci=92 co=0 ca=0
29 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1572/0 dn=2 df=3646 of=0 ri=1
ql=0 b=10 ci=14 co=0 ca=0
30 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1672/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
31 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1696/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
32 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=5680/0 dn=0 df=2801 of=0 ri=97
ql=0 b=10 ci=3220 co=0 ca=0
33 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1408/0 dn=0 df=2757 of=0 ri=1
ql=0 b=10 ci=21106 co=0 ca=0
34 c=4196 g=4197 pq=1 pqc=4196 qp=1 dt=1812/0 dn=0 df=3524 of=0 ri=1
ql=0 b=10 ci=550 co=0 ca=0
35 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1682/0 dn=2 df=3629 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
36 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1694/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
37 c=4148 g=4148 pq=1 pqc=4147 qp=0 dt=1680/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
38 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1634/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
39 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1700/0 dn=2 df=3650 of=0 ri=1
ql=0 b=10 ci=18 co=0 ca=0
40 c=4208 g=4209 pq=1 pqc=4208 qp=1 dt=1918/0 dn=0 df=3004 of=0 ri=1
ql=0 b=10 ci=10285 co=0 ca=0
41 c=4210 g=4211 pq=1 pqc=4210 qp=1 dt=1656/0 dn=0 df=2864 of=0 ri=1
ql=0 b=10 ci=19494 co=0 ca=0
42 c=4204 g=4205 pq=1 pqc=4204 qp=1 dt=1788/0 dn=0 df=3553 of=0 ri=1
ql=0 b=10 ci=188 co=0 ca=0
43 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1718/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
44 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1666/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
45 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=2060/0 dn=0 df=3652 of=0 ri=1
ql=0 b=10 ci=12 co=0 ca=0
46 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1710/0 dn=2 df=3651 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
47 c=4150 g=4150 pq=1 pqc=4149 qp=0 dt=1730/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
48 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=35267/1 dn=2 df=2672 of=0
ri=161 ql=2 b=10 ci=20305 co=0 ca=0
49 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=1551/1 dn=0 df=2895 of=0 ri=15
ql=7 b=10 ci=28449 co=0 ca=0
50 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1940/0 dn=0 df=3513 of=0 ri=1
ql=0 b=10 ci=537 co=0 ca=0
51 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=2000/0 dn=0 df=3616 of=0 ri=1
ql=0 b=10 ci=102 co=0 ca=0
52 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1864/0 dn=2 df=3636 of=0 ri=1
ql=0 b=10 ci=57 co=0 ca=0
53 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1660/0 dn=0 df=3648 of=0 ri=1
ql=0 b=10 ci=15 co=0 ca=0
54 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1614/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
55 c=4152 g=4152 pq=1 pqc=4151 qp=0 dt=1692/0 dn=2 df=3651 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
56 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=1663/1 dn=2 df=2939 of=0 ri=1
ql=2 b=10 ci=2645 co=0 ca=0
57 c=4217 g=4218 pq=1 pqc=4217 qp=0 dt=1297/1 dn=0 df=2828 of=0 ri=1
ql=11 b=10 ci=11764 co=0 ca=0
58 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2088/0 dn=0 df=3514 of=0 ri=1
ql=0 b=10 ci=650 co=0 ca=0
59 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=2020/0 dn=2 df=3643 of=0 ri=1
ql=0 b=10 ci=40 co=0 ca=0
60 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1718/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
61 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1610/0 dn=2 df=3651 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
62 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1706/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
63 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=6180/0 dn=0 df=3466 of=0 ri=30
ql=0 b=10 ci=150 co=0 ca=0
64 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=8396/0 dn=2 df=3232 of=0
ri=274 ql=0 b=10 ci=1436 co=0 ca=0
65 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=2912/0 dn=0 df=3128 of=0 ri=6
ql=0 b=10 ci=5781 co=0 ca=0
66 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=3994/0 dn=0 df=3601 of=0 ri=6
ql=0 b=10 ci=131 co=0 ca=0
67 c=4154 g=4154 pq=1 pqc=4153 qp=0 dt=1678/0 dn=0 df=3645 of=0 ri=1
ql=0 b=10 ci=26 co=0 ca=0
68 c=4155 g=4156 pq=1 pqc=4155 qp=0 dt=1634/0 dn=0 df=3649 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
69 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1688/0 dn=2 df=3649 of=0 ri=6
ql=0 b=10 ci=11 co=0 ca=0
70 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1692/0 dn=2 df=3650 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
71 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1624/0 dn=2 df=3650 of=0 ri=1
ql=0 b=10 ci=12 co=0 ca=0
72 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=6178/0 dn=4 df=3440 of=0
ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4217 g=4218 pq=1 pqc=4217 qp=1 dt=7498/0 dn=0 df=2919 of=0 ri=3
ql=0 b=10 ci=72254 co=0 ca=0
74 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1726/0 dn=2 df=3622 of=0 ri=1
ql=0 b=10 ci=125 co=0 ca=0
75 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1716/0 dn=2 df=3649 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
76 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1714/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
77 c=4156 g=4156 pq=1 pqc=4155 qp=0 dt=1678/0 dn=0 df=3649 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
78 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1730/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
79 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1656/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
80 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=2190/0 dn=2 df=3383 of=0 ri=1
ql=0 b=10 ci=590 co=0 ca=0
81 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=3668/0 dn=0 df=3182 of=0 ri=1
ql=0 b=10 ci=5193 co=0 ca=0
82 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1662/0 dn=2 df=3623 of=0 ri=1
ql=0 b=10 ci=106 co=0 ca=0
83 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1708/0 dn=2 df=3646 of=0 ri=1
ql=0 b=10 ci=22 co=0 ca=0
84 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1638/0 dn=2 df=3651 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
85 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1652/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
86 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1680/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
87 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1688/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
88 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1888/0 dn=0 df=3401 of=0 ri=1
ql=0 b=10 ci=884 co=0 ca=0
89 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=2830/0 dn=0 df=3282 of=0 ri=1
ql=0 b=10 ci=2572 co=0 ca=0
90 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1578/0 dn=0 df=3606 of=0 ri=1
ql=0 b=10 ci=154 co=0 ca=0
91 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1788/0 dn=2 df=3645 of=0 ri=1
ql=0 b=10 ci=20 co=0 ca=0
92 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1758/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
93 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1742/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
94 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1748/0 dn=2 df=3654 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
95 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1628/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
96 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1742/0 dn=2 df=3424 of=0 ri=1
ql=0 b=10 ci=493 co=0 ca=0
97 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2676/0 dn=0 df=3330 of=0 ri=1
ql=0 b=10 ci=1015 co=0 ca=0
98 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1744/0 dn=2 df=3644 of=0 ri=1
ql=0 b=10 ci=27 co=0 ca=0
99 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1676/0 dn=2 df=3651 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
100 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1764/0 dn=2 df=3650 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
101 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1724/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
102 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1726/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
103 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1708/0 dn=2 df=3649 of=0 ri=1
ql=0 b=10 ci=1923 co=0 ca=0
104 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1726/0 dn=2 df=3458 of=0 ri=1
ql=0 b=10 ci=387 co=0 ca=0
105 c=4164 g=4165 pq=1 pqc=4164 qp=1 dt=2748/0 dn=0 df=3359 of=0 ri=1
ql=0 b=10 ci=20365 co=0 ca=0
106 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1850/0 dn=2 df=3641 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
107 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1740/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
108 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1686/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
109 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1698/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
110 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1734/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
111 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1662/0 dn=2 df=3651 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
112 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=7844/0 dn=0 df=3377 of=0 ri=36
ql=0 b=10 ci=613 co=0 ca=0
113 c=4202 g=4203 pq=1 pqc=4202 qp=1 dt=2416/0 dn=0 df=3329 of=0 ri=1
ql=0 b=10 ci=5505 co=0 ca=0
114 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1686/0 dn=0 df=3570 of=0 ri=1
ql=0 b=10 ci=497 co=0 ca=0
115 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1604/0 dn=2 df=3641 of=0 ri=1
ql=0 b=10 ci=38 co=0 ca=0
116 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1776/0 dn=2 df=3648 of=0 ri=1
ql=0 b=10 ci=34 co=0 ca=0
117 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1638/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=10 co=0 ca=0
118 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1680/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
119 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2724/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
120 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2276/0 dn=0 df=3449 of=0 ri=1
ql=0 b=10 ci=393 co=0 ca=0
121 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=3026/0 dn=0 df=2057 of=0 ri=1
ql=0 b=10 ci=48877 co=0 ca=0
122 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1592/0 dn=0 df=3579 of=0 ri=1
ql=0 b=10 ci=333 co=0 ca=0
123 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1668/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
124 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1724/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
125 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1688/0 dn=2 df=3652 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
126 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=2504/0 dn=2 df=3647 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
127 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1758/0 dn=2 df=3653 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6377/73 dn=4 df=9 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
1 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=12539/1 dn=0 df=15 of=0 ri=0 ql=0
b=10 ci=4 co=0 ca=0
2 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=11202/0 dn=2 df=17 of=0 ri=0 ql=0
b=10 ci=11 co=0 ca=0
3 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1716/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
4 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1580/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
5 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1634/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
6 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1632/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
7 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
8 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=9281/1 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
9 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3181/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
10 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2054/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
11 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1802/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
12 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1844/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
13 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3972/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
14 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
15 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1674/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
16 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=18828/0 dn=0 df=17 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
17 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1311/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
18 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1712/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
19 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2078/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
20 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3688/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
21 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1810/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
22 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1752/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
23 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1762/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
24 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=14418/0 dn=0 df=18 of=0 ri=0 ql=0
b=10 ci=0 co=0 ca=0
25 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1630/0 dn=0 df=19 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
26 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1824/0 dn=2 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
27 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1608/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
28 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1616/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
29 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1572/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
30 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1672/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
31 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1696/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
32 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=5680/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=2 co=0 ca=0
33 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1408/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
34 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1812/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
35 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1682/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
36 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1694/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
37 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
38 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1634/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
39 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1700/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
40 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1918/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
41 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1656/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
42 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1788/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
43 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
44 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1666/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
45 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2060/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
46 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1710/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
47 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
48 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=35267/2 dn=2 df=16 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
49 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1551/2 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
50 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1940/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
51 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2000/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
52 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1864/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
53 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1660/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
54 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1614/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
55 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1692/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
56 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1663/1 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
57 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1297/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
58 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2088/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
59 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2020/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
60 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1718/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
61 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1610/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
62 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1706/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
63 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6180/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
64 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=8396/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
65 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2912/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
66 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=3994/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
67 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1678/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
68 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1634/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
69 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
70 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1692/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
71 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1624/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
72 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=6178/0 dn=4 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
73 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=7498/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
74 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
75 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1716/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
76 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1714/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
77 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1678/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
78 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
79 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1656/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
80 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2190/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
81 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=3668/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
82 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1662/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
83 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
84 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1638/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
85 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1652/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
86 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
87 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
88 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1888/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
89 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2830/0 dn=0 df=19 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
90 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1578/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
91 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1788/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
92 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
93 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1742/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
94 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1748/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
95 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1628/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
96 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1742/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
97 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=2676/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
98 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1744/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
99 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1676/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
100 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
101 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
102 c=18446744073709551332 g=18446744073709551332 pq=1
pqc=18446744073709551331 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
103 c=18446744073709551332 g=18446744073709551332 pq=1
pqc=18446744073709551331 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
104 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
105 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=2748/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
106 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1850/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
107 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1740/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
108 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1686/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
109 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
110 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
111 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1662/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
112 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=7844/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
113 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2416/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
114 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1686/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
115 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1604/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
116 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1776/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
117 c=18446744073709551335 g=18446744073709551336 pq=1
pqc=18446744073709551335 qp=1 dt=1638/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
118 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
119 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
120 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2276/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
121 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3026/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
122 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1592/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
123 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1668/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
124 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
125 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
126 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2504/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
127 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=25023 qsp=1521 rpq=6044 cbr=1 cng=1255 gpc=1142 gps=133 nf=695 nn=15753
1 np=26683 qsp=60 rpq=8032 cbr=5 cng=45 gpc=1628 gps=749 nf=441 nn=15783
2 np=23557 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1102 gps=715 nf=180 nn=14668
3 np=21601 qsp=29 rpq=6997 cbr=0 cng=5 gpc=910 gps=708 nf=96 nn=12885
4 np=21509 qsp=9 rpq=6943 cbr=0 cng=5 gpc=984 gps=729 nf=100 nn=12748
5 np=21377 qsp=28 rpq=6885 cbr=0 cng=2 gpc=876 gps=683 nf=109 nn=12822
6 np=21339 qsp=30 rpq=6845 cbr=0 cng=2 gpc=875 gps=689 nf=104 nn=12824
7 np=21299 qsp=32 rpq=6806 cbr=0 cng=2 gpc=982 gps=700 nf=107 nn=12702
8 np=30347 qsp=840 rpq=8110 cbr=1 cng=46 gpc=1670 gps=781 nf=590 nn=19149
9 np=25433 qsp=32 rpq=7693 cbr=1 cng=22 gpc=1752 gps=783 nf=177 nn=15005
10 np=21236 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1270 gps=806 nf=15 nn=12285
11 np=20410 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1064 gps=807 nf=5 nn=11903
12 np=20308 qsp=9 rpq=6574 cbr=0 cng=1 gpc=857 gps=755 nf=2 nn=12119
13 np=20874 qsp=9 rpq=6543 cbr=0 cng=1 gpc=917 gps=736 nf=4 nn=12673
14 np=20214 qsp=7 rpq=6482 cbr=0 cng=0 gpc=776 gps=758 nf=2 nn=12196
15 np=20177 qsp=9 rpq=6443 cbr=0 cng=0 gpc=871 gps=757 nf=3 nn=12103
16 np=28829 qsp=173 rpq=7583 cbr=0 cng=28 gpc=1745 gps=754 nf=262 nn=18457
17 np=25146 qsp=26 rpq=7261 cbr=1 cng=12 gpc=1717 gps=699 nf=205 nn=15251
18 np=20876 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1028 gps=681 nf=26 nn=12653
19 np=20174 qsp=11 rpq=6309 cbr=0 cng=1 gpc=896 gps=692 nf=8 nn=12268
20 np=19992 qsp=7 rpq=6224 cbr=0 cng=0 gpc=877 gps=671 nf=5 nn=12215
21 np=19929 qsp=9 rpq=6173 cbr=0 cng=0 gpc=870 gps=697 nf=5 nn=12184
22 np=19877 qsp=7 rpq=6122 cbr=0 cng=0 gpc=876 gps=696 nf=1 nn=12182
23 np=19830 qsp=7 rpq=6077 cbr=0 cng=0 gpc=874 gps=696 nf=2 nn=12181
24 np=23873 qsp=19 rpq=6847 cbr=0 cng=18 gpc=1554 gps=685 nf=168 nn=14601
25 np=24970 qsp=103 rpq=6912 cbr=16 cng=15 gpc=1701 gps=698 nf=262 nn=15366
26 np=20576 qsp=15 rpq=6151 cbr=0 cng=2 gpc=1047 gps=692 nf=14 nn=12670
27 np=19785 qsp=10 rpq=5927 cbr=0 cng=1 gpc=907 gps=691 nf=13 nn=12246
28 np=19763 qsp=7 rpq=5886 cbr=0 cng=1 gpc=906 gps=677 nf=9 nn=12284
29 np=19580 qsp=7 rpq=5813 cbr=0 cng=1 gpc=887 gps=685 nf=8 nn=12186
30 np=19505 qsp=6 rpq=5760 cbr=0 cng=1 gpc=874 gps=717 nf=2 nn=12151
31 np=19464 qsp=8 rpq=5714 cbr=0 cng=1 gpc=874 gps=700 nf=1 nn=12174
32 np=24658 qsp=164 rpq=6636 cbr=0 cng=17 gpc=1719 gps=741 nf=131 nn=15414
33 np=23772 qsp=42 rpq=6526 cbr=8 cng=7 gpc=1717 gps=733 nf=120 nn=14661
34 np=19929 qsp=8 rpq=5731 cbr=0 cng=4 gpc=996 gps=729 nf=14 nn=12455
35 np=19379 qsp=9 rpq=5558 cbr=0 cng=0 gpc=890 gps=727 nf=9 nn=12195
36 np=19222 qsp=8 rpq=5488 cbr=0 cng=0 gpc=876 gps=716 nf=6 nn=12136
37 np=19177 qsp=8 rpq=5442 cbr=0 cng=0 gpc=870 gps=721 nf=2 nn=12142
38 np=19131 qsp=7 rpq=5399 cbr=0 cng=0 gpc=874 gps=722 nf=4 nn=12132
39 np=19099 qsp=9 rpq=5357 cbr=0 cng=0 gpc=874 gps=720 nf=5 nn=12143
40 np=22208 qsp=29 rpq=6027 cbr=1 cng=7 gpc=1470 gps=784 nf=89 nn=13830
41 np=22867 qsp=70 rpq=6061 cbr=14 cng=18 gpc=1674 gps=663 nf=96 nn=14341
42 np=19441 qsp=8 rpq=5352 cbr=0 cng=0 gpc=1004 gps=793 nf=4 nn=12288
43 np=18930 qsp=8 rpq=5179 cbr=0 cng=0 gpc=951 gps=797 nf=5 nn=11998
44 np=18884 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1207 gps=767 nf=1 nn=11774
45 np=18844 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1199 gps=737 nf=3 nn=11813
46 np=18799 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1315 gps=801 nf=3 nn=11636
47 np=18746 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1050 gps=653 nf=3 nn=12042
48 np=25120 qsp=297 rpq=6169 cbr=3 cng=15 gpc=1693 gps=740 nf=503 nn=15997
49 np=22872 qsp=68 rpq=5700 cbr=3 cng=12 gpc=1606 gps=695 nf=116 nn=14740
50 np=19790 qsp=9 rpq=5021 cbr=0 cng=0 gpc=1017 gps=686 nf=96 nn=12970
51 np=19209 qsp=8 rpq=4858 cbr=0 cng=0 gpc=910 gps=679 nf=63 nn=12699
52 np=19011 qsp=8 rpq=4788 cbr=0 cng=0 gpc=889 gps=680 nf=48 nn=12606
53 np=19120 qsp=7 rpq=4730 cbr=0 cng=0 gpc=874 gps=682 nf=60 nn=12774
54 np=18397 qsp=8 rpq=4682 cbr=0 cng=0 gpc=869 gps=682 nf=9 nn=12155
55 np=18350 qsp=9 rpq=4635 cbr=0 cng=0 gpc=871 gps=678 nf=5 nn=12161
56 np=21845 qsp=11 rpq=5386 cbr=0 cng=10 gpc=1578 gps=701 nf=91 nn=14079
57 np=22346 qsp=27 rpq=5403 cbr=2 cng=11 gpc=1828 gps=707 nf=98 nn=14297
58 np=18955 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1217 gps=712 nf=21 nn=12346
59 np=18242 qsp=8 rpq=4469 cbr=0 cng=0 gpc=907 gps=706 nf=4 nn=12156
60 np=18162 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1030 gps=741 nf=6 nn=11973
61 np=18117 qsp=8 rpq=4369 cbr=0 cng=0 gpc=950 gps=688 nf=4 nn=12106
62 np=18075 qsp=7 rpq=4330 cbr=0 cng=0 gpc=872 gps=662 nf=7 nn=12204
63 np=19194 qsp=44 rpq=4585 cbr=0 cng=0 gpc=1027 gps=671 nf=133 nn=12778
64 np=22024 qsp=234 rpq=4773 cbr=0 cng=10 gpc=1344 gps=684 nf=67 nn=15146
65 np=20693 qsp=33 rpq=4833 cbr=0 cng=9 gpc=1403 gps=686 nf=192 nn=13570
66 np=18466 qsp=32 rpq=4222 cbr=0 cng=4 gpc=923 gps=636 nf=28 nn=12653
67 np=18101 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1149 gps=633 nf=25 nn=12172
68 np=18042 qsp=8 rpq=4072 cbr=0 cng=3 gpc=882 gps=666 nf=23 nn=12396
69 np=17993 qsp=30 rpq=4025 cbr=0 cng=2 gpc=919 gps=683 nf=29 nn=12335
70 np=17951 qsp=32 rpq=3979 cbr=0 cng=3 gpc=878 gps=676 nf=21 nn=12394
71 np=17901 qsp=8 rpq=3935 cbr=0 cng=3 gpc=931 gps=733 nf=33 nn=12266
72 np=20530 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1106 gps=628 nf=28 nn=14623
73 np=25530 qsp=10 rpq=4627 cbr=1 cng=311 gpc=1244 gps=606 nf=1401 nn=17340
74 np=17670 qsp=7 rpq=3829 cbr=0 cng=3 gpc=900 gps=775 nf=7 nn=12156
75 np=17494 qsp=8 rpq=3753 cbr=0 cng=1 gpc=878 gps=770 nf=4 nn=12088
76 np=17429 qsp=8 rpq=3703 cbr=0 cng=0 gpc=872 gps=766 nf=3 nn=12085
77 np=17399 qsp=8 rpq=3660 cbr=0 cng=1 gpc=878 gps=772 nf=3 nn=12085
78 np=17334 qsp=7 rpq=3608 cbr=0 cng=1 gpc=873 gps=764 nf=1 nn=12087
79 np=17288 qsp=9 rpq=3563 cbr=0 cng=1 gpc=871 gps=758 nf=2 nn=12093
80 np=19879 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1129 gps=700 nf=116 nn=14085
81 np=21160 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1317 gps=704 nf=315 nn=14712
82 np=18105 qsp=7 rpq=3461 cbr=0 cng=0 gpc=907 gps=699 nf=150 nn=12888
83 np=17157 qsp=9 rpq=3391 cbr=0 cng=0 gpc=886 gps=740 nf=5 nn=12135
84 np=17085 qsp=8 rpq=3336 cbr=0 cng=0 gpc=880 gps=756 nf=7 nn=12106
85 np=17039 qsp=9 rpq=3293 cbr=0 cng=0 gpc=875 gps=749 nf=5 nn=12117
86 np=16993 qsp=7 rpq=3248 cbr=0 cng=0 gpc=877 gps=747 nf=3 nn=12118
87 np=16948 qsp=8 rpq=3201 cbr=0 cng=0 gpc=879 gps=749 nf=4 nn=12115
88 np=18331 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1147 gps=718 nf=38 nn=12975
89 np=19383 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1286 gps=715 nf=165 nn=13632
90 np=17241 qsp=9 rpq=3121 cbr=0 cng=2 gpc=928 gps=710 nf=38 nn=12442
91 np=16803 qsp=8 rpq=3031 cbr=0 cng=1 gpc=895 gps=698 nf=2 nn=12176
92 np=16724 qsp=9 rpq=2976 cbr=0 cng=0 gpc=878 gps=695 nf=8 nn=12167
93 np=16678 qsp=8 rpq=2932 cbr=0 cng=1 gpc=884 gps=694 nf=3 nn=12164
94 np=16624 qsp=8 rpq=2886 cbr=0 cng=1 gpc=875 gps=698 nf=1 nn=12163
95 np=16586 qsp=7 rpq=2842 cbr=0 cng=0 gpc=878 gps=687 nf=3 nn=12176
96 np=17817 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1114 gps=710 nf=24 nn=12916
97 np=19239 qsp=8 rpq=3182 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16473 qsp=9 rpq=2713 cbr=0 cng=0 gpc=882 gps=710 nf=3 nn=12165
99 np=16407 qsp=8 rpq=2663 cbr=0 cng=0 gpc=875 gps=719 nf=4 nn=12146
100 np=16350 qsp=8 rpq=2609 cbr=0 cng=0 gpc=876 gps=699 nf=1 nn=12165
101 np=16293 qsp=8 rpq=2562 cbr=0 cng=1 gpc=874 gps=692 nf=3 nn=12161
102 np=16251 qsp=8 rpq=2518 cbr=0 cng=0 gpc=877 gps=683 nf=0 nn=12173
103 np=16225 qsp=11 rpq=2475 cbr=0 cng=0 gpc=877 gps=689 nf=6 nn=12178
104 np=17119 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1089 gps=768 nf=13 nn=12597
105 np=17595 qsp=67 rpq=2711 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16158 qsp=9 rpq=2364 cbr=0 cng=0 gpc=887 gps=770 nf=5 nn=12132
107 np=16048 qsp=8 rpq=2296 cbr=0 cng=0 gpc=875 gps=770 nf=2 nn=12105
108 np=16006 qsp=7 rpq=2252 cbr=0 cng=0 gpc=881 gps=774 nf=7 nn=12092
109 np=15958 qsp=8 rpq=2206 cbr=0 cng=0 gpc=881 gps=773 nf=3 nn=12095
110 np=15911 qsp=7 rpq=2160 cbr=0 cng=0 gpc=881 gps=767 nf=4 nn=12099
111 np=15866 qsp=7 rpq=2117 cbr=0 cng=0 gpc=881 gps=772 nf=3 nn=12093
112 np=17863 qsp=27 rpq=2417 cbr=0 cng=4 gpc=1163 gps=701 nf=82 nn=13496
113 np=17614 qsp=12 rpq=2430 cbr=1 cng=2 gpc=1215 gps=722 nf=97 nn=13147
114 np=16333 qsp=8 rpq=2072 cbr=0 cng=0 gpc=957 gps=725 nf=25 nn=12554
115 np=15914 qsp=7 rpq=1947 cbr=0 cng=0 gpc=891 gps=711 nf=14 nn=12351
116 np=15840 qsp=8 rpq=1894 cbr=0 cng=0 gpc=881 gps=724 nf=15 nn=12326
117 np=15776 qsp=8 rpq=1845 cbr=0 cng=1 gpc=879 gps=721 nf=5 nn=12325
118 np=15514 qsp=7 rpq=1798 cbr=0 cng=0 gpc=881 gps=717 nf=2 nn=12116
119 np=15472 qsp=8 rpq=1755 cbr=0 cng=0 gpc=881 gps=720 nf=3 nn=12113
120 np=16459 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1103 gps=662 nf=9 nn=12742
121 np=27488 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1561 gps=706 nf=2320 nn=19383
122 np=15730 qsp=7 rpq=1699 cbr=0 cng=2 gpc=954 gps=668 nf=16 nn=12391
123 np=15325 qsp=8 rpq=1577 cbr=0 cng=0 gpc=885 gps=663 nf=5 nn=12195
124 np=15281 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1044 gps=698 nf=5 nn=12003
125 np=15229 qsp=8 rpq=1488 cbr=0 cng=0 gpc=926 gps=686 nf=2 nn=12127
126 np=15224 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1035 gps=724 nf=5 nn=12014
127 np=15143 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1004 gps=669 nf=3 nn=12072
rcu_bh:
0 np=15753 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3469
1 np=15783 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=2986
2 np=14668 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1159
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19149 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2494
9 np=15005 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2517
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12119 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
13 np=12673 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=547
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18457 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3271
17 np=15251 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2426
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14601 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1943
25 np=15366 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2516
26 np=12670 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=488
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15414 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2316
33 np=14661 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2518
34 np=12455 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=335
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13830 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1735
41 np=14341 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2211
42 np=12288 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=246
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=15997 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2298
49 np=14740 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2123
50 np=12970 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=369
51 np=12699 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=85
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14079 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1898
57 np=14297 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2268
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12778 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=124
64 np=15146 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=956
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12266 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17340 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=861
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13496 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=647
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
Wed May 11 14:34:21 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4303 gpnum=4304
rcu_bh: completed=-278 gpnum=18446744073709551338
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4303 g=4304 s=2 jfq=2 j=c086 nfqs=6951/nfqsng=0(6951) fqlh=739
ff/ff ..>.. 0:254 ^0
ffff/ffff ..>.. 0:15 ^0 ffff/ffff ..>.. 16:31 ^1 ffff/ffff ..>..
32:47 ^2 ffff/ffff ..>.. 48:63 ^3 ffff/ffff ..>.. 64:79 ^4
ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff
..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9
0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207
^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>..
240:254 ^15
rcu_bh:
c=18446744073709551338 g=18446744073709551338 s=0 jfq=-11269 j=c086
nfqs=26/nfqsng=0(26) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2
0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>..
80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0
..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10
0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223
^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=6809/73 dn=4 df=2679 of=0
ri=601 ql=2 b=10 ci=6159 co=0 ca=0
1 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=12597/1 dn=0 df=2827 of=0
ri=13 ql=18 b=10 ci=11282 co=0 ca=0
2 c=4182 g=4182 pq=1 pqc=4181 qp=0 dt=11214/0 dn=2 df=3464 of=0
ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4190 g=4191 pq=1 pqc=4190 qp=1 dt=1728/0 dn=0 df=3708 of=0 ri=6
ql=0 b=10 ci=123 co=0 ca=0
4 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1600/0 dn=2 df=3718 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
5 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1646/0 dn=0 df=3735 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
6 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1668/0 dn=2 df=3735 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
7 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1712/0 dn=2 df=3735 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
8 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=9384/0 dn=2 df=2496 of=0
ri=760 ql=0 b=10 ci=7231 co=0 ca=0
9 c=4303 g=4304 pq=0 pqc=4239 qp=1 dt=3201/1 dn=0 df=2801 of=0 ri=1
ql=3 b=10 ci=13678 co=0 ca=0
10 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2074/0 dn=0 df=3582 of=0 ri=1
ql=0 b=10 ci=702 co=0 ca=0
11 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1822/0 dn=2 df=3728 of=0 ri=1
ql=0 b=10 ci=25 co=0 ca=0
12 c=4267 g=4268 pq=1 pqc=4267 qp=1 dt=1874/0 dn=2 df=3732 of=0 ri=1
ql=0 b=10 ci=27 co=0 ca=0
13 c=4291 g=4292 pq=1 pqc=4291 qp=1 dt=4206/0 dn=2 df=3734 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
14 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1760/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
15 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1710/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
16 c=4301 g=4302 pq=1 pqc=4301 qp=1 dt=19402/0 dn=0 df=2862 of=0
ri=103 ql=0 b=10 ci=3541 co=0 ca=0
17 c=4267 g=4268 pq=1 pqc=4267 qp=1 dt=1336/0 dn=0 df=2849 of=0 ri=1
ql=0 b=10 ci=11747 co=0 ca=0
18 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1748/0 dn=0 df=3582 of=0 ri=1
ql=0 b=10 ci=665 co=0 ca=0
19 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2102/0 dn=0 df=3706 of=0 ri=1
ql=0 b=10 ci=121 co=0 ca=0
20 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=3712/0 dn=0 df=3734 of=0 ri=1
ql=0 b=10 ci=24 co=0 ca=0
21 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1834/0 dn=2 df=3736 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
22 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1768/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
23 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1786/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
24 c=4245 g=4246 pq=1 pqc=4245 qp=1 dt=14498/0 dn=0 df=2990 of=0 ri=1
ql=0 b=10 ci=3357 co=0 ca=0
25 c=4247 g=4248 pq=1 pqc=4247 qp=1 dt=1654/0 dn=0 df=2812 of=0 ri=1
ql=0 b=10 ci=25721 co=0 ca=0
26 c=4241 g=4242 pq=1 pqc=4241 qp=1 dt=1848/0 dn=2 df=3561 of=0 ri=1
ql=0 b=10 ci=1059 co=0 ca=0
27 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1644/0 dn=2 df=3709 of=0 ri=1
ql=0 b=10 ci=157 co=0 ca=0
28 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1652/0 dn=0 df=3707 of=0 ri=1
ql=0 b=10 ci=92 co=0 ca=0
29 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1584/0 dn=2 df=3732 of=0 ri=1
ql=0 b=10 ci=14 co=0 ca=0
30 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1698/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
31 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1732/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
32 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=5810/0 dn=0 df=2881 of=0 ri=97
ql=0 b=10 ci=3229 co=0 ca=0
33 c=4253 g=4254 pq=1 pqc=4253 qp=1 dt=1436/0 dn=0 df=2828 of=0 ri=1
ql=0 b=10 ci=24390 co=0 ca=0
34 c=4247 g=4248 pq=1 pqc=4247 qp=1 dt=1842/0 dn=0 df=3608 of=0 ri=1
ql=0 b=10 ci=562 co=0 ca=0
35 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1712/0 dn=2 df=3715 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
36 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1736/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
37 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1716/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
38 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1672/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
39 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1742/0 dn=2 df=3736 of=0 ri=1
ql=0 b=10 ci=18 co=0 ca=0
40 c=4259 g=4260 pq=1 pqc=4259 qp=1 dt=2004/0 dn=0 df=3084 of=0 ri=1
ql=0 b=10 ci=10294 co=0 ca=0
41 c=4260 g=4261 pq=1 pqc=4260 qp=1 dt=1700/0 dn=0 df=2934 of=0 ri=1
ql=0 b=10 ci=22778 co=0 ca=0
42 c=4255 g=4256 pq=1 pqc=4255 qp=1 dt=1816/0 dn=0 df=3637 of=0 ri=1
ql=0 b=10 ci=200 co=0 ca=0
43 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1750/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
44 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1698/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
45 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=2092/0 dn=0 df=3738 of=0 ri=1
ql=0 b=10 ci=12 co=0 ca=0
46 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1744/0 dn=2 df=3737 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
47 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1766/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
48 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=35311/1 dn=2 df=2735 of=0
ri=161 ql=2 b=10 ci=20577 co=0 ca=0
49 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=1593/1 dn=0 df=2964 of=0 ri=15
ql=7 b=10 ci=28689 co=0 ca=0
50 c=4268 g=4269 pq=0 pqc=4267 qp=1 dt=1976/0 dn=0 df=3592 of=0 ri=1
ql=0 b=10 ci=722 co=0 ca=0
51 c=4268 g=4269 pq=0 pqc=4267 qp=1 dt=2034/0 dn=0 df=3695 of=0 ri=1
ql=0 b=10 ci=265 co=0 ca=0
52 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1902/0 dn=2 df=3722 of=0 ri=1
ql=0 b=10 ci=57 co=0 ca=0
53 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1698/0 dn=0 df=3734 of=0 ri=1
ql=0 b=10 ci=15 co=0 ca=0
54 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1644/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
55 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1722/0 dn=2 df=3737 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
56 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=1697/1 dn=2 df=3023 of=0 ri=1
ql=4 b=10 ci=2647 co=0 ca=0
57 c=4303 g=4304 pq=1 pqc=4303 qp=0 dt=1331/1 dn=0 df=2908 of=0 ri=1
ql=11 b=10 ci=11779 co=0 ca=0
58 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2122/0 dn=0 df=3600 of=0 ri=1
ql=0 b=10 ci=650 co=0 ca=0
59 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2046/0 dn=2 df=3729 of=0 ri=1
ql=0 b=10 ci=40 co=0 ca=0
60 c=4268 g=4269 pq=1 pqc=4268 qp=0 dt=1764/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
61 c=4273 g=4274 pq=1 pqc=4273 qp=1 dt=1632/0 dn=2 df=3737 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
62 c=4281 g=4281 pq=1 pqc=4280 qp=0 dt=1736/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
63 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=6340/0 dn=0 df=3552 of=0 ri=30
ql=0 b=10 ci=150 co=0 ca=0
64 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=8448/0 dn=2 df=3318 of=0
ri=274 ql=0 b=10 ci=1436 co=0 ca=0
65 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=2942/0 dn=0 df=3214 of=0 ri=6
ql=0 b=10 ci=5781 co=0 ca=0
66 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=4036/0 dn=0 df=3687 of=0 ri=6
ql=0 b=10 ci=131 co=0 ca=0
67 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1738/0 dn=0 df=3731 of=0 ri=1
ql=0 b=10 ci=26 co=0 ca=0
68 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1660/0 dn=0 df=3735 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
69 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1718/0 dn=2 df=3735 of=0 ri=6
ql=0 b=10 ci=11 co=0 ca=0
70 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1734/0 dn=2 df=3736 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
71 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1666/0 dn=2 df=3730 of=0 ri=1
ql=0 b=10 ci=32 co=0 ca=0
72 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=6224/0 dn=4 df=3526 of=0
ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=7544/0 dn=0 df=2999 of=0 ri=3
ql=0 b=10 ci=72272 co=0 ca=0
74 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1764/0 dn=2 df=3708 of=0 ri=1
ql=0 b=10 ci=125 co=0 ca=0
75 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1754/0 dn=2 df=3735 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
76 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1756/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
77 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1700/0 dn=0 df=3735 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
78 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1756/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
79 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1698/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
80 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=2230/0 dn=2 df=3469 of=0 ri=1
ql=0 b=10 ci=590 co=0 ca=0
81 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=3684/0 dn=0 df=3268 of=0 ri=1
ql=0 b=10 ci=5193 co=0 ca=0
82 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1698/0 dn=2 df=3709 of=0 ri=1
ql=0 b=10 ci=106 co=0 ca=0
83 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1736/0 dn=2 df=3732 of=0 ri=1
ql=0 b=10 ci=22 co=0 ca=0
84 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1650/0 dn=2 df=3737 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
85 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1664/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
86 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1708/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
87 c=4158 g=4158 pq=1 pqc=4157 qp=0 dt=1704/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
88 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1908/0 dn=0 df=3487 of=0 ri=1
ql=0 b=10 ci=884 co=0 ca=0
89 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=2846/0 dn=0 df=3368 of=0 ri=1
ql=0 b=10 ci=2572 co=0 ca=0
90 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1598/0 dn=0 df=3692 of=0 ri=1
ql=0 b=10 ci=154 co=0 ca=0
91 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1800/0 dn=2 df=3731 of=0 ri=1
ql=0 b=10 ci=20 co=0 ca=0
92 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1770/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
93 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1754/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
94 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1784/0 dn=2 df=3740 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
95 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1640/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
96 c=4160 g=4160 pq=1 pqc=4159 qp=0 dt=1758/0 dn=2 df=3510 of=0 ri=1
ql=0 b=10 ci=493 co=0 ca=0
97 c=4239 g=4240 pq=1 pqc=4239 qp=1 dt=2696/0 dn=0 df=3416 of=0 ri=1
ql=0 b=10 ci=1015 co=0 ca=0
98 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1756/0 dn=2 df=3730 of=0 ri=1
ql=0 b=10 ci=27 co=0 ca=0
99 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1712/0 dn=2 df=3737 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
100 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1784/0 dn=2 df=3736 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
101 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1736/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
102 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1748/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
103 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1720/0 dn=2 df=3735 of=0 ri=1
ql=0 b=10 ci=1923 co=0 ca=0
104 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1762/0 dn=2 df=3544 of=0 ri=1
ql=0 b=10 ci=387 co=0 ca=0
105 c=4239 g=4240 pq=1 pqc=4239 qp=1 dt=2768/0 dn=0 df=3445 of=0 ri=1
ql=0 b=10 ci=20365 co=0 ca=0
106 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1890/0 dn=2 df=3727 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
107 c=4162 g=4162 pq=1 pqc=4161 qp=0 dt=1752/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
108 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1706/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
109 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1726/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
110 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1746/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
111 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1684/0 dn=2 df=3737 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
112 c=4298 g=4299 pq=1 pqc=4298 qp=1 dt=7876/0 dn=0 df=3441 of=0 ri=36
ql=0 b=10 ci=694 co=0 ca=0
113 c=4202 g=4203 pq=1 pqc=4202 qp=1 dt=2436/0 dn=0 df=3415 of=0 ri=1
ql=0 b=10 ci=5505 co=0 ca=0
114 c=4164 g=4164 pq=1 pqc=4163 qp=0 dt=1722/0 dn=0 df=3656 of=0 ri=1
ql=0 b=10 ci=497 co=0 ca=0
115 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1632/0 dn=2 df=3727 of=0 ri=1
ql=0 b=10 ci=38 co=0 ca=0
116 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1792/0 dn=2 df=3734 of=0 ri=1
ql=0 b=10 ci=34 co=0 ca=0
117 c=4171 g=4171 pq=1 pqc=4170 qp=0 dt=1674/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=10 co=0 ca=0
118 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1716/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
119 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2752/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
120 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=2312/0 dn=0 df=3535 of=0 ri=1
ql=0 b=10 ci=393 co=0 ca=0
121 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=3044/0 dn=0 df=2143 of=0 ri=1
ql=0 b=10 ci=48877 co=0 ca=0
122 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1604/0 dn=0 df=3665 of=0 ri=1
ql=0 b=10 ci=333 co=0 ca=0
123 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1684/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
124 c=4173 g=4173 pq=1 pqc=4172 qp=0 dt=1744/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
125 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1708/0 dn=2 df=3738 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
126 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=2524/0 dn=2 df=3733 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
127 c=4180 g=4180 pq=1 pqc=4179 qp=0 dt=1774/0 dn=2 df=3739 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6809/73 dn=4 df=9 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
1 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=12597/1 dn=0 df=15 of=0 ri=0 ql=0
b=10 ci=4 co=0 ca=0
2 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=11214/0 dn=2 df=17 of=0 ri=0 ql=0
b=10 ci=11 co=0 ca=0
3 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1728/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
4 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1600/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
5 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1646/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
6 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1668/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
7 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
8 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=9384/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
9 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3201/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
10 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2074/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
11 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1822/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
12 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1874/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
13 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=4206/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
14 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1760/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
15 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1710/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
16 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=19402/0 dn=0 df=17 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
17 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1336/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
18 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1748/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
19 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2102/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
20 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3712/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
21 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1834/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
22 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1768/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
23 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1786/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
24 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=14498/0 dn=0 df=18 of=0 ri=0 ql=0
b=10 ci=0 co=0 ca=0
25 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1654/0 dn=0 df=19 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
26 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1848/0 dn=2 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
27 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1644/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
28 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1652/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
29 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1584/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
30 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
31 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1732/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
32 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=5810/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=2 co=0 ca=0
33 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1436/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
34 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1842/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
35 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
36 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
37 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1716/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
38 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1672/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
39 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1742/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
40 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2004/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
41 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1700/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
42 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1816/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
43 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1750/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
44 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
45 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2092/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
46 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1744/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
47 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1766/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
48 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=35311/1 dn=2 df=16 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
49 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1593/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
50 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1976/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
51 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2034/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
52 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1902/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
53 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1698/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
54 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1644/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
55 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1722/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
56 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1697/1 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
57 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1331/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
58 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2122/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
59 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2046/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
60 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
61 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1632/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
62 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
63 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6340/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
64 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=8448/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
65 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2942/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
66 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=4036/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
67 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1738/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
68 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1660/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
69 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1718/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
70 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
71 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1666/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
72 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6224/0 dn=4 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
73 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=7544/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
74 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
75 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1754/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
76 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
77 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1700/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
78 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
79 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
80 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2230/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
81 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=3684/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
82 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1698/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
83 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
84 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1650/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
85 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1664/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
86 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
87 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1704/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
88 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1908/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
89 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=2846/0 dn=0 df=19 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
90 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1598/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
91 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1800/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
92 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1770/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
93 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1754/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
94 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
95 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1640/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
96 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
97 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2696/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
98 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
99 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
100 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
101 c=18446744073709551330 g=18446744073709551330 pq=1
pqc=18446744073709551329 qp=0 dt=1736/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
102 c=18446744073709551332 g=18446744073709551332 pq=1
pqc=18446744073709551331 qp=0 dt=1748/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
103 c=18446744073709551332 g=18446744073709551332 pq=1
pqc=18446744073709551331 qp=0 dt=1720/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
104 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1762/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
105 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2768/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
106 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1890/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
107 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1752/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
108 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1706/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
109 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1726/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
110 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
111 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1684/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
112 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=7876/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
113 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2436/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
114 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1722/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
115 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1632/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
116 c=18446744073709551334 g=18446744073709551334 pq=1
pqc=18446744073709551333 qp=0 dt=1792/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
117 c=18446744073709551335 g=18446744073709551336 pq=1
pqc=18446744073709551335 qp=1 dt=1674/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
118 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1716/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
119 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2752/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
120 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2312/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
121 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3044/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
122 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1604/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
123 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1684/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
124 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1744/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
125 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1708/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
126 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2524/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
127 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1774/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=25285 qsp=1521 rpq=6066 cbr=1 cng=1256 gpc=1169 gps=134 nf=695 nn=15964
1 np=26771 qsp=60 rpq=8060 cbr=5 cng=45 gpc=1638 gps=749 nf=452 nn=15822
2 np=23557 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1102 gps=715 nf=180 nn=14668
3 np=21601 qsp=29 rpq=6997 cbr=0 cng=5 gpc=910 gps=708 nf=96 nn=12885
4 np=21509 qsp=9 rpq=6943 cbr=0 cng=5 gpc=984 gps=729 nf=100 nn=12748
5 np=21377 qsp=28 rpq=6885 cbr=0 cng=2 gpc=876 gps=683 nf=109 nn=12822
6 np=21339 qsp=30 rpq=6845 cbr=0 cng=2 gpc=875 gps=689 nf=104 nn=12824
7 np=21299 qsp=32 rpq=6806 cbr=0 cng=2 gpc=982 gps=700 nf=107 nn=12702
8 np=30373 qsp=840 rpq=8114 cbr=1 cng=46 gpc=1676 gps=782 nf=590 nn=19164
9 np=25478 qsp=32 rpq=7706 cbr=1 cng=22 gpc=1757 gps=783 nf=183 nn=15026
10 np=21236 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1270 gps=806 nf=15 nn=12285
11 np=20410 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1064 gps=807 nf=5 nn=11903
12 np=20332 qsp=9 rpq=6580 cbr=0 cng=2 gpc=858 gps=755 nf=7 nn=12130
13 np=20911 qsp=9 rpq=6547 cbr=0 cng=1 gpc=924 gps=736 nf=4 nn=12699
14 np=20214 qsp=7 rpq=6482 cbr=0 cng=0 gpc=776 gps=758 nf=2 nn=12196
15 np=20177 qsp=9 rpq=6443 cbr=0 cng=0 gpc=871 gps=757 nf=3 nn=12103
16 np=28931 qsp=173 rpq=7597 cbr=0 cng=29 gpc=1751 gps=758 nf=262 nn=18534
17 np=25198 qsp=26 rpq=7275 cbr=1 cng=15 gpc=1726 gps=699 nf=205 nn=15277
18 np=20876 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1028 gps=681 nf=26 nn=12653
19 np=20174 qsp=11 rpq=6309 cbr=0 cng=1 gpc=896 gps=692 nf=8 nn=12268
20 np=19992 qsp=7 rpq=6224 cbr=0 cng=0 gpc=877 gps=671 nf=5 nn=12215
21 np=19929 qsp=9 rpq=6173 cbr=0 cng=0 gpc=870 gps=697 nf=5 nn=12184
22 np=19877 qsp=7 rpq=6122 cbr=0 cng=0 gpc=876 gps=696 nf=1 nn=12182
23 np=19830 qsp=7 rpq=6077 cbr=0 cng=0 gpc=874 gps=696 nf=2 nn=12181
24 np=23902 qsp=19 rpq=6853 cbr=0 cng=18 gpc=1560 gps=685 nf=169 nn=14617
25 np=25038 qsp=108 rpq=6919 cbr=18 cng=16 gpc=1713 gps=698 nf=263 nn=15411
26 np=20585 qsp=15 rpq=6153 cbr=0 cng=2 gpc=1050 gps=692 nf=14 nn=12674
27 np=19785 qsp=10 rpq=5927 cbr=0 cng=1 gpc=907 gps=691 nf=13 nn=12246
28 np=19763 qsp=7 rpq=5886 cbr=0 cng=1 gpc=906 gps=677 nf=9 nn=12284
29 np=19580 qsp=7 rpq=5813 cbr=0 cng=1 gpc=887 gps=685 nf=8 nn=12186
30 np=19505 qsp=6 rpq=5760 cbr=0 cng=1 gpc=874 gps=717 nf=2 nn=12151
31 np=19464 qsp=8 rpq=5714 cbr=0 cng=1 gpc=874 gps=700 nf=1 nn=12174
32 np=24711 qsp=164 rpq=6649 cbr=0 cng=18 gpc=1730 gps=741 nf=136 nn=15437
33 np=23840 qsp=47 rpq=6535 cbr=12 cng=7 gpc=1729 gps=733 nf=121 nn=14703
34 np=19935 qsp=8 rpq=5733 cbr=0 cng=4 gpc=998 gps=729 nf=14 nn=12457
35 np=19379 qsp=9 rpq=5558 cbr=0 cng=0 gpc=890 gps=727 nf=9 nn=12195
36 np=19222 qsp=8 rpq=5488 cbr=0 cng=0 gpc=876 gps=716 nf=6 nn=12136
37 np=19177 qsp=8 rpq=5442 cbr=0 cng=0 gpc=870 gps=721 nf=2 nn=12142
38 np=19131 qsp=7 rpq=5399 cbr=0 cng=0 gpc=874 gps=722 nf=4 nn=12132
39 np=19099 qsp=9 rpq=5357 cbr=0 cng=0 gpc=874 gps=720 nf=5 nn=12143
40 np=22232 qsp=29 rpq=6033 cbr=1 cng=7 gpc=1471 gps=784 nf=95 nn=13841
41 np=22933 qsp=77 rpq=6072 cbr=18 cng=22 gpc=1680 gps=664 nf=99 nn=14378
42 np=19449 qsp=8 rpq=5354 cbr=0 cng=0 gpc=1007 gps=793 nf=4 nn=12291
43 np=18930 qsp=8 rpq=5179 cbr=0 cng=0 gpc=951 gps=797 nf=5 nn=11998
44 np=18884 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1207 gps=767 nf=1 nn=11774
45 np=18845 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1200 gps=737 nf=3 nn=11813
46 np=18800 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1316 gps=801 nf=3 nn=11636
47 np=18747 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1051 gps=653 nf=3 nn=12042
48 np=25202 qsp=297 rpq=6190 cbr=3 cng=15 gpc=1702 gps=740 nf=514 nn=16038
49 np=22939 qsp=68 rpq=5720 cbr=3 cng=12 gpc=1623 gps=695 nf=116 nn=14770
50 np=19819 qsp=9 rpq=5028 cbr=0 cng=0 gpc=1025 gps=686 nf=96 nn=12984
51 np=19238 qsp=8 rpq=4864 cbr=0 cng=0 gpc=918 gps=680 nf=63 nn=12713
52 np=19012 qsp=8 rpq=4788 cbr=0 cng=0 gpc=890 gps=680 nf=48 nn=12606
53 np=19121 qsp=7 rpq=4730 cbr=0 cng=0 gpc=875 gps=682 nf=60 nn=12774
54 np=18398 qsp=8 rpq=4682 cbr=0 cng=0 gpc=870 gps=682 nf=9 nn=12155
55 np=18351 qsp=9 rpq=4635 cbr=0 cng=0 gpc=872 gps=678 nf=5 nn=12161
56 np=21872 qsp=11 rpq=5393 cbr=0 cng=10 gpc=1585 gps=701 nf=91 nn=14092
57 np=22373 qsp=27 rpq=5410 cbr=2 cng=11 gpc=1830 gps=707 nf=104 nn=14309
58 np=18956 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1218 gps=712 nf=21 nn=12346
59 np=18243 qsp=8 rpq=4469 cbr=0 cng=0 gpc=908 gps=706 nf=4 nn=12156
60 np=18163 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1031 gps=741 nf=6 nn=11973
61 np=18118 qsp=8 rpq=4369 cbr=0 cng=0 gpc=951 gps=688 nf=4 nn=12106
62 np=18076 qsp=7 rpq=4330 cbr=0 cng=0 gpc=873 gps=662 nf=7 nn=12204
63 np=19218 qsp=44 rpq=4589 cbr=0 cng=0 gpc=1034 gps=671 nf=133 nn=12791
64 np=22026 qsp=234 rpq=4773 cbr=0 cng=11 gpc=1345 gps=684 nf=67 nn=15146
65 np=20694 qsp=33 rpq=4834 cbr=0 cng=9 gpc=1403 gps=686 nf=192 nn=13570
66 np=18467 qsp=32 rpq=4222 cbr=0 cng=4 gpc=924 gps=636 nf=28 nn=12653
67 np=18102 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1150 gps=633 nf=25 nn=12172
68 np=18043 qsp=8 rpq=4072 cbr=0 cng=3 gpc=883 gps=666 nf=23 nn=12396
69 np=17994 qsp=30 rpq=4025 cbr=0 cng=2 gpc=920 gps=683 nf=29 nn=12335
70 np=17952 qsp=32 rpq=3979 cbr=0 cng=3 gpc=879 gps=676 nf=21 nn=12394
71 np=17926 qsp=8 rpq=3942 cbr=0 cng=3 gpc=938 gps=733 nf=33 nn=12277
72 np=20531 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1107 gps=628 nf=28 nn=14623
73 np=25555 qsp=10 rpq=4635 cbr=1 cng=311 gpc=1250 gps=606 nf=1401 nn=17351
74 np=17671 qsp=7 rpq=3829 cbr=0 cng=3 gpc=901 gps=775 nf=7 nn=12156
75 np=17495 qsp=8 rpq=3753 cbr=0 cng=1 gpc=879 gps=770 nf=4 nn=12088
76 np=17430 qsp=8 rpq=3703 cbr=0 cng=0 gpc=873 gps=766 nf=3 nn=12085
77 np=17400 qsp=8 rpq=3660 cbr=0 cng=1 gpc=879 gps=772 nf=3 nn=12085
78 np=17335 qsp=7 rpq=3608 cbr=0 cng=1 gpc=874 gps=764 nf=1 nn=12087
79 np=17289 qsp=9 rpq=3563 cbr=0 cng=1 gpc=872 gps=758 nf=2 nn=12093
80 np=19880 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1130 gps=700 nf=116 nn=14085
81 np=21161 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1318 gps=704 nf=315 nn=14712
82 np=18106 qsp=7 rpq=3461 cbr=0 cng=0 gpc=908 gps=699 nf=150 nn=12888
83 np=17158 qsp=9 rpq=3391 cbr=0 cng=0 gpc=887 gps=740 nf=5 nn=12135
84 np=17086 qsp=8 rpq=3336 cbr=0 cng=0 gpc=881 gps=756 nf=7 nn=12106
85 np=17040 qsp=9 rpq=3293 cbr=0 cng=0 gpc=876 gps=749 nf=5 nn=12117
86 np=16994 qsp=7 rpq=3248 cbr=0 cng=0 gpc=878 gps=747 nf=3 nn=12118
87 np=16949 qsp=8 rpq=3201 cbr=0 cng=0 gpc=880 gps=749 nf=4 nn=12115
88 np=18332 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1148 gps=718 nf=38 nn=12975
89 np=19384 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1287 gps=715 nf=165 nn=13632
90 np=17242 qsp=9 rpq=3121 cbr=0 cng=2 gpc=929 gps=710 nf=38 nn=12442
91 np=16804 qsp=8 rpq=3031 cbr=0 cng=1 gpc=896 gps=698 nf=2 nn=12176
92 np=16724 qsp=9 rpq=2976 cbr=0 cng=0 gpc=878 gps=695 nf=8 nn=12167
93 np=16678 qsp=8 rpq=2932 cbr=0 cng=1 gpc=884 gps=694 nf=3 nn=12164
94 np=16624 qsp=8 rpq=2886 cbr=0 cng=1 gpc=875 gps=698 nf=1 nn=12163
95 np=16586 qsp=7 rpq=2842 cbr=0 cng=0 gpc=878 gps=687 nf=3 nn=12176
96 np=17817 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1114 gps=710 nf=24 nn=12916
97 np=19240 qsp=8 rpq=3183 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16473 qsp=9 rpq=2713 cbr=0 cng=0 gpc=882 gps=710 nf=3 nn=12165
99 np=16407 qsp=8 rpq=2663 cbr=0 cng=0 gpc=875 gps=719 nf=4 nn=12146
100 np=16350 qsp=8 rpq=2609 cbr=0 cng=0 gpc=876 gps=699 nf=1 nn=12165
101 np=16293 qsp=8 rpq=2562 cbr=0 cng=1 gpc=874 gps=692 nf=3 nn=12161
102 np=16251 qsp=8 rpq=2518 cbr=0 cng=0 gpc=877 gps=683 nf=0 nn=12173
103 np=16225 qsp=11 rpq=2475 cbr=0 cng=0 gpc=877 gps=689 nf=6 nn=12178
104 np=17119 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1089 gps=768 nf=13 nn=12597
105 np=17596 qsp=67 rpq=2712 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16158 qsp=9 rpq=2364 cbr=0 cng=0 gpc=887 gps=770 nf=5 nn=12132
107 np=16048 qsp=8 rpq=2296 cbr=0 cng=0 gpc=875 gps=770 nf=2 nn=12105
108 np=16006 qsp=7 rpq=2252 cbr=0 cng=0 gpc=881 gps=774 nf=7 nn=12092
109 np=15958 qsp=8 rpq=2206 cbr=0 cng=0 gpc=881 gps=773 nf=3 nn=12095
110 np=15911 qsp=7 rpq=2160 cbr=0 cng=0 gpc=881 gps=767 nf=4 nn=12099
111 np=15866 qsp=7 rpq=2117 cbr=0 cng=0 gpc=881 gps=772 nf=3 nn=12093
112 np=17959 qsp=27 rpq=2442 cbr=0 cng=4 gpc=1170 gps=701 nf=97 nn=13545
113 np=17614 qsp=12 rpq=2430 cbr=1 cng=2 gpc=1215 gps=722 nf=97 nn=13147
114 np=16333 qsp=8 rpq=2072 cbr=0 cng=0 gpc=957 gps=725 nf=25 nn=12554
115 np=15914 qsp=7 rpq=1947 cbr=0 cng=0 gpc=891 gps=711 nf=14 nn=12351
116 np=15840 qsp=8 rpq=1894 cbr=0 cng=0 gpc=881 gps=724 nf=15 nn=12326
117 np=15776 qsp=8 rpq=1845 cbr=0 cng=1 gpc=879 gps=721 nf=5 nn=12325
118 np=15514 qsp=7 rpq=1798 cbr=0 cng=0 gpc=881 gps=717 nf=2 nn=12116
119 np=15472 qsp=8 rpq=1755 cbr=0 cng=0 gpc=881 gps=720 nf=3 nn=12113
120 np=16459 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1103 gps=662 nf=9 nn=12742
121 np=27488 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1561 gps=706 nf=2320 nn=19383
122 np=15730 qsp=7 rpq=1699 cbr=0 cng=2 gpc=954 gps=668 nf=16 nn=12391
123 np=15325 qsp=8 rpq=1577 cbr=0 cng=0 gpc=885 gps=663 nf=5 nn=12195
124 np=15281 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1044 gps=698 nf=5 nn=12003
125 np=15229 qsp=8 rpq=1488 cbr=0 cng=0 gpc=926 gps=686 nf=2 nn=12127
126 np=15224 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1035 gps=724 nf=5 nn=12014
127 np=15143 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1004 gps=669 nf=3 nn=12072
rcu_bh:
0 np=15964 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3680
1 np=15822 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=3025
2 np=14668 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1159
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19164 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2509
9 np=15026 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2538
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12130 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
13 np=12699 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=573
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18534 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3348
17 np=15277 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2452
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14617 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1959
25 np=15411 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2561
26 np=12674 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=492
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15437 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2339
33 np=14703 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2560
34 np=12457 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=337
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13841 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1746
41 np=14378 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2248
42 np=12291 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=249
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=16038 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2339
49 np=14770 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2153
50 np=12984 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=383
51 np=12713 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14092 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1911
57 np=14309 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2280
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12791 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=137
64 np=15146 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=956
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12277 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17351 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=872
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13545 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=696
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
Wed May 11 14:34:45 UTC 2011
/sys/kernel/debug/rcu/rcugp:
rcu_sched: completed=4365 gpnum=4366
rcu_bh: completed=-278 gpnum=18446744073709551338
/sys/kernel/debug/rcu/rcuhier:
rcu_sched:
c=4365 g=4366 s=2 jfq=2 j=d7dd nfqs=7021/nfqsng=0(7021) fqlh=739
ff/ff ..>.. 0:254 ^0
ffff/ffff ..>.. 0:15 ^0 ffff/ffff ..>.. 16:31 ^1 ffff/ffff ..>..
32:47 ^2 fdfe/ffff ..>.. 48:63 ^3 ffff/ffff ..>.. 64:79 ^4
ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff
..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9
0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207
^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>..
240:254 ^15
rcu_bh:
c=18446744073709551338 g=18446744073709551338 s=0 jfq=-17244 j=d7dd
nfqs=26/nfqsng=0(26) fqlh=2
0/ff ..>.. 0:254 ^0
0/ffff ..>.. 0:15 ^0 0/ffff ..>.. 16:31 ^1 0/ffff ..>.. 32:47 ^2
0/ffff ..>.. 48:63 ^3 0/ffff ..>.. 64:79 ^4 0/ffff ..>..
80:95 ^5 0/ffff ..>.. 96:111 ^6 0/ffff ..>.. 112:127 ^7 0/0
..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10
0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223
^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
/sys/kernel/debug/rcu/rcudata:
rcu_sched:
0 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=7237/73 dn=4 df=2718 of=0
ri=601 ql=4 b=10 ci=6199 co=0 ca=0
1 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=12649/1 dn=0 df=2878 of=0
ri=13 ql=9 b=10 ci=11312 co=0 ca=0
2 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=11246/0 dn=2 df=3526 of=0
ri=149 ql=0 b=10 ci=2170 co=0 ca=0
3 c=4190 g=4191 pq=1 pqc=4190 qp=1 dt=1740/0 dn=0 df=3770 of=0 ri=6
ql=0 b=10 ci=123 co=0 ca=0
4 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1612/0 dn=2 df=3780 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
5 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1658/0 dn=0 df=3797 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
6 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1700/0 dn=2 df=3797 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
7 c=4211 g=4211 pq=1 pqc=4210 qp=0 dt=1728/0 dn=2 df=3797 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
8 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=9484/0 dn=2 df=2552 of=0
ri=760 ql=0 b=10 ci=7235 co=0 ca=0
9 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=3221/1 dn=0 df=2853 of=0 ri=1
ql=12 b=10 ci=13696 co=0 ca=0
10 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2086/0 dn=0 df=3644 of=0 ri=1
ql=0 b=10 ci=702 co=0 ca=0
11 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1834/0 dn=2 df=3790 of=0 ri=1
ql=0 b=10 ci=25 co=0 ca=0
12 c=4267 g=4268 pq=1 pqc=4267 qp=1 dt=1890/0 dn=2 df=3794 of=0 ri=1
ql=0 b=10 ci=27 co=0 ca=0
13 c=4291 g=4292 pq=1 pqc=4291 qp=1 dt=4218/0 dn=2 df=3796 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
14 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1772/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
15 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1746/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
16 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=19988/0 dn=0 df=2922 of=0
ri=103 ql=0 b=10 ci=3543 co=0 ca=0
17 c=4331 g=4332 pq=1 pqc=4331 qp=1 dt=1352/0 dn=0 df=2905 of=0 ri=1
ql=0 b=10 ci=11761 co=0 ca=0
18 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=1768/0 dn=0 df=3644 of=0 ri=1
ql=0 b=10 ci=665 co=0 ca=0
19 c=4213 g=4213 pq=1 pqc=4212 qp=0 dt=2114/0 dn=0 df=3768 of=0 ri=1
ql=0 b=10 ci=121 co=0 ca=0
20 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=3724/0 dn=0 df=3796 of=0 ri=1
ql=0 b=10 ci=24 co=0 ca=0
21 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1846/0 dn=2 df=3798 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
22 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1784/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
23 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1798/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
24 c=4337 g=4338 pq=1 pqc=4337 qp=1 dt=14566/0 dn=0 df=3046 of=0 ri=1
ql=0 b=10 ci=3366 co=0 ca=0
25 c=4339 g=4340 pq=1 pqc=4339 qp=1 dt=1670/0 dn=0 df=2860 of=0 ri=1
ql=0 b=10 ci=28852 co=0 ca=0
26 c=4333 g=4334 pq=1 pqc=4333 qp=1 dt=1864/0 dn=2 df=3621 of=0 ri=1
ql=0 b=10 ci=1071 co=0 ca=0
27 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1680/0 dn=2 df=3771 of=0 ri=1
ql=0 b=10 ci=157 co=0 ca=0
28 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1688/0 dn=0 df=3769 of=0 ri=1
ql=0 b=10 ci=92 co=0 ca=0
29 c=4215 g=4215 pq=1 pqc=4214 qp=0 dt=1596/0 dn=2 df=3794 of=0 ri=1
ql=0 b=10 ci=14 co=0 ca=0
30 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1710/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
31 c=4217 g=4217 pq=1 pqc=4216 qp=0 dt=1764/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
32 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=5938/0 dn=0 df=2932 of=0 ri=97
ql=0 b=10 ci=3256 co=0 ca=0
33 c=4343 g=4344 pq=1 pqc=4343 qp=1 dt=1456/0 dn=0 df=2876 of=0 ri=1
ql=0 b=10 ci=27522 co=0 ca=0
34 c=4343 g=4344 pq=1 pqc=4343 qp=1 dt=1858/0 dn=0 df=3664 of=0 ri=1
ql=0 b=10 ci=576 co=0 ca=0
35 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1724/0 dn=2 df=3777 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
36 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1772/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
37 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1728/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
38 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1684/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
39 c=4224 g=4224 pq=1 pqc=4223 qp=0 dt=1770/0 dn=2 df=3798 of=0 ri=1
ql=0 b=10 ci=18 co=0 ca=0
40 c=4350 g=4351 pq=1 pqc=4350 qp=1 dt=2074/0 dn=0 df=3140 of=0 ri=1
ql=0 b=10 ci=10303 co=0 ca=0
41 c=4352 g=4353 pq=1 pqc=4352 qp=1 dt=1734/0 dn=0 df=2979 of=0 ri=1
ql=0 b=10 ci=25912 co=0 ca=0
42 c=4346 g=4347 pq=1 pqc=4346 qp=1 dt=1828/0 dn=0 df=3697 of=0 ri=1
ql=0 b=10 ci=212 co=0 ca=0
43 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1786/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
44 c=4226 g=4226 pq=1 pqc=4225 qp=0 dt=1734/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
45 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=2128/0 dn=0 df=3800 of=0 ri=1
ql=0 b=10 ci=12 co=0 ca=0
46 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1780/0 dn=2 df=3799 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
47 c=4233 g=4233 pq=1 pqc=4232 qp=0 dt=1802/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
48 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=35327/1 dn=2 df=2796 of=0
ri=161 ql=2 b=10 ci=20579 co=0 ca=0
49 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=1619/1 dn=0 df=3017 of=0 ri=15
ql=7 b=10 ci=28706 co=0 ca=0
50 c=4268 g=4269 pq=1 pqc=4268 qp=1 dt=1992/0 dn=0 df=3654 of=0 ri=1
ql=0 b=10 ci=722 co=0 ca=0
51 c=4268 g=4269 pq=1 pqc=4268 qp=1 dt=2048/0 dn=0 df=3757 of=0 ri=1
ql=0 b=10 ci=265 co=0 ca=0
52 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1938/0 dn=2 df=3784 of=0 ri=1
ql=0 b=10 ci=57 co=0 ca=0
53 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1714/0 dn=0 df=3796 of=0 ri=1
ql=0 b=10 ci=15 co=0 ca=0
54 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1668/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
55 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=1746/0 dn=2 df=3799 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
56 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=1713/1 dn=2 df=3079 of=0 ri=1
ql=2 b=10 ci=2655 co=0 ca=0
57 c=4365 g=4366 pq=1 pqc=4365 qp=0 dt=1347/1 dn=0 df=2964 of=0 ri=1
ql=11 b=10 ci=11792 co=0 ca=0
58 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2158/0 dn=0 df=3662 of=0 ri=1
ql=0 b=10 ci=650 co=0 ca=0
59 c=4261 g=4261 pq=1 pqc=4260 qp=0 dt=2062/0 dn=2 df=3791 of=0 ri=1
ql=0 b=10 ci=40 co=0 ca=0
60 c=4268 g=4269 pq=1 pqc=4268 qp=0 dt=1800/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
61 c=4273 g=4274 pq=1 pqc=4273 qp=1 dt=1644/0 dn=2 df=3799 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
62 c=4281 g=4281 pq=1 pqc=4280 qp=0 dt=1756/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
63 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=6496/0 dn=0 df=3614 of=0 ri=30
ql=0 b=10 ci=150 co=0 ca=0
64 c=4327 g=4328 pq=1 pqc=4327 qp=1 dt=8498/0 dn=2 df=3372 of=0
ri=274 ql=0 b=10 ci=1442 co=0 ca=0
65 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=2958/0 dn=0 df=3276 of=0 ri=6
ql=0 b=10 ci=5781 co=0 ca=0
66 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=4094/0 dn=0 df=3749 of=0 ri=6
ql=0 b=10 ci=131 co=0 ca=0
67 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1756/0 dn=0 df=3793 of=0 ri=1
ql=0 b=10 ci=26 co=0 ca=0
68 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1696/0 dn=0 df=3797 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
69 c=4299 g=4299 pq=1 pqc=4298 qp=0 dt=1758/0 dn=2 df=3797 of=0 ri=6
ql=0 b=10 ci=11 co=0 ca=0
70 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1750/0 dn=2 df=3798 of=0 ri=6
ql=0 b=10 ci=10 co=0 ca=0
71 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1702/0 dn=2 df=3792 of=0 ri=1
ql=0 b=10 ci=32 co=0 ca=0
72 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=6248/0 dn=4 df=3588 of=0
ri=112 ql=0 b=10 ci=493 co=0 ca=0
73 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=7578/0 dn=0 df=3061 of=0 ri=3
ql=0 b=10 ci=72272 co=0 ca=0
74 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1800/0 dn=2 df=3770 of=0 ri=1
ql=0 b=10 ci=125 co=0 ca=0
75 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1790/0 dn=2 df=3797 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
76 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1792/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
77 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1736/0 dn=0 df=3797 of=0 ri=1
ql=0 b=10 ci=111 co=0 ca=0
78 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1778/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
79 c=4301 g=4301 pq=1 pqc=4300 qp=0 dt=1734/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
80 c=4307 g=4308 pq=1 pqc=4307 qp=1 dt=2248/0 dn=2 df=3531 of=0 ri=1
ql=0 b=10 ci=590 co=0 ca=0
81 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=3718/0 dn=0 df=3330 of=0 ri=1
ql=0 b=10 ci=5193 co=0 ca=0
82 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1740/0 dn=2 df=3771 of=0 ri=1
ql=0 b=10 ci=106 co=0 ca=0
83 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1778/0 dn=2 df=3794 of=0 ri=1
ql=0 b=10 ci=22 co=0 ca=0
84 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1688/0 dn=2 df=3799 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
85 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1702/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
86 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1746/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
87 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1730/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
88 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=1934/0 dn=0 df=3549 of=0 ri=1
ql=0 b=10 ci=884 co=0 ca=0
89 c=4310 g=4310 pq=1 pqc=4309 qp=0 dt=2872/0 dn=0 df=3430 of=0 ri=1
ql=0 b=10 ci=2572 co=0 ca=0
90 c=4318 g=4318 pq=1 pqc=4317 qp=0 dt=1620/0 dn=0 df=3754 of=0 ri=1
ql=0 b=10 ci=154 co=0 ca=0
91 c=4318 g=4318 pq=1 pqc=4317 qp=0 dt=1838/0 dn=2 df=3793 of=0 ri=1
ql=0 b=10 ci=20 co=0 ca=0
92 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1808/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
93 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1788/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
94 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1826/0 dn=2 df=3802 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
95 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1662/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
96 c=4325 g=4325 pq=1 pqc=4324 qp=0 dt=1784/0 dn=2 df=3572 of=0 ri=1
ql=0 b=10 ci=493 co=0 ca=0
97 c=4326 g=4327 pq=1 pqc=4326 qp=1 dt=2726/0 dn=0 df=3478 of=0 ri=1
ql=0 b=10 ci=1015 co=0 ca=0
98 c=4352 g=4353 pq=1 pqc=4352 qp=0 dt=1786/0 dn=2 df=3792 of=0 ri=1
ql=0 b=10 ci=27 co=0 ca=0
99 c=4353 g=4353 pq=1 pqc=4352 qp=0 dt=1754/0 dn=2 df=3799 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
100 c=4354 g=4355 pq=1 pqc=4354 qp=0 dt=1818/0 dn=2 df=3798 of=0 ri=1
ql=0 b=10 ci=11 co=0 ca=0
101 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1770/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
102 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1782/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
103 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1750/0 dn=2 df=3797 of=0 ri=1
ql=0 b=10 ci=1923 co=0 ca=0
104 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1804/0 dn=2 df=3606 of=0 ri=1
ql=0 b=10 ci=387 co=0 ca=0
105 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=2806/0 dn=0 df=3507 of=0 ri=1
ql=0 b=10 ci=20365 co=0 ca=0
106 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1928/0 dn=2 df=3789 of=0 ri=1
ql=0 b=10 ci=50 co=0 ca=0
107 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1782/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
108 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1734/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
109 c=4355 g=4355 pq=1 pqc=4354 qp=0 dt=1756/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
110 c=4356 g=4357 pq=1 pqc=4356 qp=0 dt=1776/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
111 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1714/0 dn=2 df=3799 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
112 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=7918/0 dn=0 df=3503 of=0 ri=36
ql=0 b=10 ci=694 co=0 ca=0
113 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=2470/0 dn=0 df=3477 of=0 ri=1
ql=0 b=10 ci=5505 co=0 ca=0
114 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1764/0 dn=0 df=3718 of=0 ri=1
ql=0 b=10 ci=497 co=0 ca=0
115 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1658/0 dn=2 df=3789 of=0 ri=1
ql=0 b=10 ci=38 co=0 ca=0
116 c=4357 g=4357 pq=1 pqc=4356 qp=0 dt=1820/0 dn=2 df=3796 of=0 ri=1
ql=0 b=10 ci=34 co=0 ca=0
117 c=4363 g=4363 pq=1 pqc=4362 qp=0 dt=1712/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=10 co=0 ca=0
118 c=4363 g=4363 pq=1 pqc=4362 qp=0 dt=1758/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
119 c=4363 g=4363 pq=1 pqc=4362 qp=0 dt=2778/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
120 c=4364 g=4365 pq=1 pqc=4364 qp=0 dt=2334/0 dn=0 df=3597 of=0 ri=1
ql=0 b=10 ci=393 co=0 ca=0
121 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=3070/0 dn=0 df=2205 of=0 ri=1
ql=0 b=10 ci=48877 co=0 ca=0
122 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1626/0 dn=0 df=3727 of=0 ri=1
ql=0 b=10 ci=333 co=0 ca=0
123 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1706/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
124 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1766/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
125 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1730/0 dn=2 df=3800 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
126 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=2546/0 dn=2 df=3795 of=0 ri=1
ql=0 b=10 ci=21 co=0 ca=0
127 c=4365 g=4365 pq=1 pqc=4364 qp=0 dt=1796/0 dn=2 df=3801 of=0 ri=1
ql=0 b=10 ci=9 co=0 ca=0
rcu_bh:
0 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=7237/73 dn=4 df=9 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
1 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=12649/1 dn=0 df=15 of=0 ri=0 ql=0
b=10 ci=4 co=0 ca=0
2 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=11246/0 dn=2 df=17 of=0 ri=0 ql=0
b=10 ci=11 co=0 ca=0
3 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1740/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
4 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1612/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
5 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1658/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
6 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1700/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
7 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1728/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
8 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=9484/0 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
9 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3221/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
10 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2086/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
11 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1834/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
12 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1890/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
13 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=4218/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
14 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1772/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
15 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
16 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=19988/0 dn=0 df=17 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
17 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1352/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=8 co=0 ca=0
18 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1768/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
19 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2114/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
20 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3724/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
21 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1846/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
22 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
23 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1798/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
24 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=14566/0 dn=0 df=18 of=0 ri=0 ql=0
b=10 ci=0 co=0 ca=0
25 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1670/0 dn=0 df=19 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
26 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1864/0 dn=2 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
27 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1680/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
28 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1688/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
29 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1596/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
30 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1710/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
31 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1764/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
32 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=5938/0 dn=0 df=16 of=0 ri=0 ql=0 b=10
ci=2 co=0 ca=0
33 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1456/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
34 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1858/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
35 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1724/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
36 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1772/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
37 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1728/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
38 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1684/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
39 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1770/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
40 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2074/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
41 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1734/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
42 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1828/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
43 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1786/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
44 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
45 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2128/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
46 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1780/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
47 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1802/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
48 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=35327/1 dn=2 df=16 of=0 ri=0 ql=0
b=10 ci=2 co=0 ca=0
49 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1619/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
50 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1992/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
51 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2048/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
52 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1938/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
53 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1714/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
54 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1668/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
55 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
56 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1713/1 dn=2 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
57 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1347/1 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
58 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2158/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
59 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2062/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
60 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1800/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
61 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1644/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
62 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
63 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6496/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
64 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=8498/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
65 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2958/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
66 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=4094/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
67 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1756/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
68 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1696/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
69 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
70 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1750/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
71 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1702/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
72 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=6248/0 dn=4 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
73 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=7578/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
74 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1800/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
75 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1790/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
76 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1792/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
77 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1736/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
78 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1778/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
79 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
80 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2248/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
81 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3718/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
82 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1740/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
83 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1778/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
84 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1688/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
85 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1702/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
86 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1746/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
87 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
88 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1934/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
89 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2872/0 dn=0 df=19 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
90 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1620/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
91 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1838/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
92 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1808/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
93 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1788/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
94 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1826/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
95 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1662/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
96 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1784/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
97 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2726/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
98 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1786/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
99 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1754/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
100 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1818/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
101 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1770/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
102 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1782/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
103 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1750/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
104 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1804/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
105 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2806/0 dn=0 df=20 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
106 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1928/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
107 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1782/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
108 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1734/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
109 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1756/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
110 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1776/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
111 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1714/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
112 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=7918/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
113 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2470/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
114 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1764/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
115 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1658/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
116 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1820/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
117 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1712/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
118 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1758/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
119 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2778/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
120 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2334/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
121 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=3070/0 dn=0 df=18 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
122 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1626/0 dn=0 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
123 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1706/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
124 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1766/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
125 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1730/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
126 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=2546/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
127 c=18446744073709551338 g=18446744073709551338 pq=1
pqc=18446744073709551337 qp=0 dt=1796/0 dn=2 df=22 of=0 ri=0 ql=0 b=10
ci=0 co=0 ca=0
/sys/kernel/debug/rcu/rcu_pending:
rcu_sched:
0 np=25632 qsp=1521 rpq=6099 cbr=1 cng=1258 gpc=1188 gps=137 nf=703 nn=16246
1 np=26817 qsp=60 rpq=8075 cbr=5 cng=46 gpc=1644 gps=749 nf=456 nn=15842
2 np=23561 qsp=164 rpq=6864 cbr=0 cng=28 gpc=1104 gps=715 nf=180 nn=14670
3 np=21602 qsp=29 rpq=6998 cbr=0 cng=5 gpc=910 gps=708 nf=96 nn=12885
4 np=21510 qsp=9 rpq=6943 cbr=0 cng=5 gpc=985 gps=729 nf=100 nn=12748
5 np=21378 qsp=28 rpq=6885 cbr=0 cng=2 gpc=877 gps=683 nf=109 nn=12822
6 np=21340 qsp=30 rpq=6845 cbr=0 cng=2 gpc=876 gps=689 nf=104 nn=12824
7 np=21300 qsp=32 rpq=6806 cbr=0 cng=2 gpc=983 gps=700 nf=107 nn=12702
8 np=30419 qsp=840 rpq=8121 cbr=1 cng=46 gpc=1688 gps=783 nf=590 nn=19190
9 np=25540 qsp=32 rpq=7723 cbr=1 cng=22 gpc=1766 gps=784 nf=191 nn=15053
10 np=21237 qsp=7 rpq=6852 cbr=0 cng=8 gpc=1271 gps=806 nf=15 nn=12285
11 np=20410 qsp=9 rpq=6630 cbr=0 cng=1 gpc=1064 gps=807 nf=5 nn=11903
12 np=20332 qsp=9 rpq=6580 cbr=0 cng=2 gpc=858 gps=755 nf=7 nn=12130
13 np=20911 qsp=9 rpq=6547 cbr=0 cng=1 gpc=924 gps=736 nf=4 nn=12699
14 np=20214 qsp=7 rpq=6482 cbr=0 cng=0 gpc=776 gps=758 nf=2 nn=12196
15 np=20177 qsp=9 rpq=6443 cbr=0 cng=0 gpc=871 gps=757 nf=3 nn=12103
16 np=29031 qsp=173 rpq=7610 cbr=0 cng=29 gpc=1756 gps=763 nf=262 nn=18611
17 np=25227 qsp=26 rpq=7282 cbr=1 cng=15 gpc=1732 gps=699 nf=205 nn=15293
18 np=20876 qsp=9 rpq=6484 cbr=0 cng=4 gpc=1028 gps=681 nf=26 nn=12653
19 np=20174 qsp=11 rpq=6309 cbr=0 cng=1 gpc=896 gps=692 nf=8 nn=12268
20 np=19992 qsp=7 rpq=6224 cbr=0 cng=0 gpc=877 gps=671 nf=5 nn=12215
21 np=19929 qsp=9 rpq=6173 cbr=0 cng=0 gpc=870 gps=697 nf=5 nn=12184
22 np=19877 qsp=7 rpq=6122 cbr=0 cng=0 gpc=876 gps=696 nf=1 nn=12182
23 np=19830 qsp=7 rpq=6077 cbr=0 cng=0 gpc=874 gps=696 nf=2 nn=12181
24 np=23932 qsp=19 rpq=6860 cbr=0 cng=18 gpc=1565 gps=685 nf=170 nn=14634
25 np=25105 qsp=115 rpq=6927 cbr=20 cng=16 gpc=1723 gps=698 nf=264 nn=15457
26 np=20599 qsp=15 rpq=6156 cbr=0 cng=2 gpc=1052 gps=692 nf=14 nn=12683
27 np=19785 qsp=10 rpq=5927 cbr=0 cng=1 gpc=907 gps=691 nf=13 nn=12246
28 np=19763 qsp=7 rpq=5886 cbr=0 cng=1 gpc=906 gps=677 nf=9 nn=12284
29 np=19580 qsp=7 rpq=5813 cbr=0 cng=1 gpc=887 gps=685 nf=8 nn=12186
30 np=19505 qsp=6 rpq=5760 cbr=0 cng=1 gpc=874 gps=717 nf=2 nn=12151
31 np=19464 qsp=8 rpq=5714 cbr=0 cng=1 gpc=874 gps=700 nf=1 nn=12174
32 np=24744 qsp=164 rpq=6657 cbr=0 cng=18 gpc=1743 gps=741 nf=136 nn=15449
33 np=23901 qsp=50 rpq=6547 cbr=15 cng=7 gpc=1738 gps=733 nf=121 nn=14740
34 np=19959 qsp=8 rpq=5740 cbr=0 cng=4 gpc=1004 gps=729 nf=14 nn=12468
35 np=19379 qsp=9 rpq=5558 cbr=0 cng=0 gpc=890 gps=727 nf=9 nn=12195
36 np=19222 qsp=8 rpq=5488 cbr=0 cng=0 gpc=876 gps=716 nf=6 nn=12136
37 np=19177 qsp=8 rpq=5442 cbr=0 cng=0 gpc=870 gps=721 nf=2 nn=12142
38 np=19131 qsp=7 rpq=5399 cbr=0 cng=0 gpc=874 gps=722 nf=4 nn=12132
39 np=19099 qsp=9 rpq=5357 cbr=0 cng=0 gpc=874 gps=720 nf=5 nn=12143
40 np=22259 qsp=29 rpq=6040 cbr=1 cng=7 gpc=1471 gps=784 nf=101 nn=13855
41 np=23011 qsp=81 rpq=6085 cbr=22 cng=26 gpc=1685 gps=664 nf=102 nn=14427
42 np=19460 qsp=8 rpq=5357 cbr=0 cng=0 gpc=1009 gps=793 nf=4 nn=12297
43 np=18930 qsp=8 rpq=5179 cbr=0 cng=0 gpc=951 gps=797 nf=5 nn=11998
44 np=18884 qsp=7 rpq=5135 cbr=0 cng=0 gpc=1207 gps=767 nf=1 nn=11774
45 np=18845 qsp=8 rpq=5092 cbr=0 cng=0 gpc=1200 gps=737 nf=3 nn=11813
46 np=18800 qsp=7 rpq=5044 cbr=0 cng=0 gpc=1316 gps=801 nf=3 nn=11636
47 np=18747 qsp=8 rpq=4998 cbr=0 cng=0 gpc=1051 gps=653 nf=3 nn=12042
48 np=25212 qsp=297 rpq=6193 cbr=3 cng=15 gpc=1704 gps=740 nf=514 nn=16043
49 np=22984 qsp=68 rpq=5734 cbr=3 cng=12 gpc=1633 gps=695 nf=116 nn=14791
50 np=19819 qsp=9 rpq=5028 cbr=0 cng=0 gpc=1025 gps=686 nf=96 nn=12984
51 np=19238 qsp=8 rpq=4864 cbr=0 cng=0 gpc=918 gps=680 nf=63 nn=12713
52 np=19012 qsp=8 rpq=4788 cbr=0 cng=0 gpc=890 gps=680 nf=48 nn=12606
53 np=19121 qsp=7 rpq=4730 cbr=0 cng=0 gpc=875 gps=682 nf=60 nn=12774
54 np=18398 qsp=8 rpq=4682 cbr=0 cng=0 gpc=870 gps=682 nf=9 nn=12155
55 np=18351 qsp=9 rpq=4635 cbr=0 cng=0 gpc=872 gps=678 nf=5 nn=12161
56 np=21897 qsp=11 rpq=5400 cbr=0 cng=10 gpc=1591 gps=701 nf=91 nn=14104
57 np=22398 qsp=27 rpq=5417 cbr=2 cng=11 gpc=1830 gps=707 nf=110 nn=14321
58 np=18956 qsp=10 rpq=4658 cbr=0 cng=1 gpc=1218 gps=712 nf=21 nn=12346
59 np=18243 qsp=8 rpq=4469 cbr=0 cng=0 gpc=908 gps=706 nf=4 nn=12156
60 np=18163 qsp=8 rpq=4412 cbr=0 cng=0 gpc=1031 gps=741 nf=6 nn=11973
61 np=18118 qsp=8 rpq=4369 cbr=0 cng=0 gpc=951 gps=688 nf=4 nn=12106
62 np=18076 qsp=7 rpq=4330 cbr=0 cng=0 gpc=873 gps=662 nf=7 nn=12204
63 np=19242 qsp=44 rpq=4594 cbr=0 cng=0 gpc=1040 gps=671 nf=133 nn=12804
64 np=22060 qsp=234 rpq=4783 cbr=0 cng=11 gpc=1352 gps=684 nf=71 nn=15159
65 np=20694 qsp=33 rpq=4834 cbr=0 cng=9 gpc=1403 gps=686 nf=192 nn=13570
66 np=18467 qsp=32 rpq=4222 cbr=0 cng=4 gpc=924 gps=636 nf=28 nn=12653
67 np=18102 qsp=8 rpq=4119 cbr=0 cng=3 gpc=1150 gps=633 nf=25 nn=12172
68 np=18043 qsp=8 rpq=4072 cbr=0 cng=3 gpc=883 gps=666 nf=23 nn=12396
69 np=17994 qsp=30 rpq=4025 cbr=0 cng=2 gpc=920 gps=683 nf=29 nn=12335
70 np=17952 qsp=32 rpq=3979 cbr=0 cng=3 gpc=879 gps=676 nf=21 nn=12394
71 np=17926 qsp=8 rpq=3942 cbr=0 cng=3 gpc=938 gps=733 nf=33 nn=12277
72 np=20531 qsp=129 rpq=4144 cbr=0 cng=1 gpc=1107 gps=628 nf=28 nn=14623
73 np=25555 qsp=10 rpq=4635 cbr=1 cng=311 gpc=1250 gps=606 nf=1401 nn=17351
74 np=17671 qsp=7 rpq=3829 cbr=0 cng=3 gpc=901 gps=775 nf=7 nn=12156
75 np=17495 qsp=8 rpq=3753 cbr=0 cng=1 gpc=879 gps=770 nf=4 nn=12088
76 np=17430 qsp=8 rpq=3703 cbr=0 cng=0 gpc=873 gps=766 nf=3 nn=12085
77 np=17400 qsp=8 rpq=3660 cbr=0 cng=1 gpc=879 gps=772 nf=3 nn=12085
78 np=17335 qsp=7 rpq=3608 cbr=0 cng=1 gpc=874 gps=764 nf=1 nn=12087
79 np=17289 qsp=9 rpq=3563 cbr=0 cng=1 gpc=872 gps=758 nf=2 nn=12093
80 np=19880 qsp=9 rpq=3836 cbr=0 cng=13 gpc=1130 gps=700 nf=116 nn=14085
81 np=21161 qsp=7 rpq=4050 cbr=0 cng=62 gpc=1318 gps=704 nf=315 nn=14712
82 np=18106 qsp=7 rpq=3461 cbr=0 cng=0 gpc=908 gps=699 nf=150 nn=12888
83 np=17158 qsp=9 rpq=3391 cbr=0 cng=0 gpc=887 gps=740 nf=5 nn=12135
84 np=17086 qsp=8 rpq=3336 cbr=0 cng=0 gpc=881 gps=756 nf=7 nn=12106
85 np=17040 qsp=9 rpq=3293 cbr=0 cng=0 gpc=876 gps=749 nf=5 nn=12117
86 np=16994 qsp=7 rpq=3248 cbr=0 cng=0 gpc=878 gps=747 nf=3 nn=12118
87 np=16949 qsp=8 rpq=3201 cbr=0 cng=0 gpc=880 gps=749 nf=4 nn=12115
88 np=18332 qsp=20 rpq=3450 cbr=0 cng=3 gpc=1148 gps=718 nf=38 nn=12975
89 np=19384 qsp=12 rpq=3582 cbr=0 cng=3 gpc=1287 gps=715 nf=165 nn=13632
90 np=17242 qsp=9 rpq=3121 cbr=0 cng=2 gpc=929 gps=710 nf=38 nn=12442
91 np=16804 qsp=8 rpq=3031 cbr=0 cng=1 gpc=896 gps=698 nf=2 nn=12176
92 np=16725 qsp=9 rpq=2976 cbr=0 cng=0 gpc=879 gps=695 nf=8 nn=12167
93 np=16679 qsp=8 rpq=2932 cbr=0 cng=1 gpc=885 gps=694 nf=3 nn=12164
94 np=16625 qsp=8 rpq=2886 cbr=0 cng=1 gpc=876 gps=698 nf=1 nn=12163
95 np=16587 qsp=7 rpq=2842 cbr=0 cng=0 gpc=879 gps=687 nf=3 nn=12176
96 np=17818 qsp=8 rpq=3050 cbr=0 cng=3 gpc=1115 gps=710 nf=24 nn=12916
97 np=19241 qsp=8 rpq=3184 cbr=0 cng=3 gpc=1223 gps=714 nf=127 nn=13990
98 np=16474 qsp=9 rpq=2713 cbr=0 cng=0 gpc=883 gps=710 nf=3 nn=12165
99 np=16408 qsp=8 rpq=2663 cbr=0 cng=0 gpc=876 gps=719 nf=4 nn=12146
100 np=16351 qsp=8 rpq=2609 cbr=0 cng=0 gpc=877 gps=699 nf=1 nn=12165
101 np=16294 qsp=8 rpq=2562 cbr=0 cng=1 gpc=875 gps=692 nf=3 nn=12161
102 np=16252 qsp=8 rpq=2518 cbr=0 cng=0 gpc=878 gps=683 nf=0 nn=12173
103 np=16226 qsp=11 rpq=2475 cbr=0 cng=0 gpc=878 gps=689 nf=6 nn=12178
104 np=17120 qsp=9 rpq=2651 cbr=0 cng=1 gpc=1090 gps=768 nf=13 nn=12597
105 np=17597 qsp=67 rpq=2713 cbr=2 cng=3 gpc=1140 gps=771 nf=129 nn=12839
106 np=16159 qsp=9 rpq=2364 cbr=0 cng=0 gpc=888 gps=770 nf=5 nn=12132
107 np=16049 qsp=8 rpq=2296 cbr=0 cng=0 gpc=876 gps=770 nf=2 nn=12105
108 np=16007 qsp=7 rpq=2252 cbr=0 cng=0 gpc=882 gps=774 nf=7 nn=12092
109 np=15959 qsp=8 rpq=2206 cbr=0 cng=0 gpc=882 gps=773 nf=3 nn=12095
110 np=15912 qsp=7 rpq=2160 cbr=0 cng=0 gpc=882 gps=767 nf=4 nn=12099
111 np=15867 qsp=7 rpq=2117 cbr=0 cng=0 gpc=882 gps=772 nf=3 nn=12093
112 np=17960 qsp=27 rpq=2443 cbr=0 cng=4 gpc=1170 gps=701 nf=97 nn=13545
113 np=17615 qsp=12 rpq=2431 cbr=1 cng=2 gpc=1215 gps=722 nf=97 nn=13147
114 np=16334 qsp=8 rpq=2072 cbr=0 cng=0 gpc=958 gps=725 nf=25 nn=12554
115 np=15915 qsp=7 rpq=1947 cbr=0 cng=0 gpc=892 gps=711 nf=14 nn=12351
116 np=15841 qsp=8 rpq=1894 cbr=0 cng=0 gpc=882 gps=724 nf=15 nn=12326
117 np=15777 qsp=8 rpq=1845 cbr=0 cng=1 gpc=880 gps=721 nf=5 nn=12325
118 np=15515 qsp=7 rpq=1798 cbr=0 cng=0 gpc=882 gps=717 nf=2 nn=12116
119 np=15473 qsp=8 rpq=1755 cbr=0 cng=0 gpc=882 gps=720 nf=3 nn=12113
120 np=16460 qsp=9 rpq=1943 cbr=0 cng=0 gpc=1104 gps=662 nf=9 nn=12742
121 np=27489 qsp=10 rpq=3390 cbr=1 cng=127 gpc=1562 gps=706 nf=2320 nn=19383
122 np=15731 qsp=7 rpq=1699 cbr=0 cng=2 gpc=955 gps=668 nf=16 nn=12391
123 np=15326 qsp=8 rpq=1577 cbr=0 cng=0 gpc=886 gps=663 nf=5 nn=12195
124 np=15282 qsp=7 rpq=1531 cbr=0 cng=0 gpc=1045 gps=698 nf=5 nn=12003
125 np=15230 qsp=8 rpq=1488 cbr=0 cng=0 gpc=927 gps=686 nf=2 nn=12127
126 np=15225 qsp=9 rpq=1446 cbr=0 cng=0 gpc=1036 gps=724 nf=5 nn=12014
127 np=15144 qsp=8 rpq=1395 cbr=0 cng=0 gpc=1005 gps=669 nf=3 nn=12072
rcu_bh:
0 np=16246 qsp=0 rpq=12266 cbr=0 cng=6 gpc=3 gps=2 nf=7 nn=3962
1 np=15842 qsp=0 rpq=12789 cbr=0 cng=1 gpc=4 gps=1 nf=2 nn=3045
2 np=14670 qsp=0 rpq=13505 cbr=0 cng=1 gpc=1 gps=0 nf=2 nn=1161
3 np=12885 qsp=0 rpq=12815 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=70
4 np=12748 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=50
5 np=12822 qsp=0 rpq=12818 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
6 np=12824 qsp=0 rpq=12820 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
7 np=12702 qsp=0 rpq=12698 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
8 np=19190 qsp=0 rpq=16647 cbr=0 cng=0 gpc=6 gps=2 nf=0 nn=2535
9 np=15053 qsp=0 rpq=12483 cbr=0 cng=0 gpc=3 gps=2 nf=0 nn=2565
10 np=12285 qsp=0 rpq=11814 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=471
11 np=11903 qsp=0 rpq=11873 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=30
12 np=12130 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
13 np=12699 qsp=0 rpq=12123 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=573
14 np=12196 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
15 np=12103 qsp=0 rpq=12099 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
16 np=18611 qsp=0 rpq=15177 cbr=0 cng=0 gpc=6 gps=1 nf=2 nn=3425
17 np=15293 qsp=0 rpq=12820 cbr=0 cng=2 gpc=2 gps=1 nf=0 nn=2468
18 np=12653 qsp=0 rpq=12195 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=458
19 np=12268 qsp=0 rpq=12184 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=84
20 np=12215 qsp=0 rpq=12198 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=17
21 np=12184 qsp=0 rpq=12176 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=8
22 np=12182 qsp=0 rpq=12178 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
23 np=12181 qsp=0 rpq=12177 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
24 np=14634 qsp=0 rpq=12655 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=1976
25 np=15457 qsp=0 rpq=12848 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=2607
26 np=12683 qsp=0 rpq=12180 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=501
27 np=12246 qsp=0 rpq=12167 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
28 np=12284 qsp=0 rpq=12185 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
29 np=12186 qsp=0 rpq=12166 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=20
30 np=12151 qsp=0 rpq=12147 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
31 np=12174 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
32 np=15449 qsp=0 rpq=13091 cbr=0 cng=1 gpc=5 gps=1 nf=0 nn=2351
33 np=14740 qsp=0 rpq=12140 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2597
34 np=12468 qsp=0 rpq=12120 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=348
35 np=12195 qsp=0 rpq=12133 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=62
36 np=12136 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
37 np=12142 qsp=0 rpq=12138 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
38 np=12132 qsp=0 rpq=12128 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
39 np=12143 qsp=0 rpq=12132 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
40 np=13855 qsp=0 rpq=12093 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1760
41 np=14427 qsp=0 rpq=12126 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2297
42 np=12297 qsp=0 rpq=12042 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=255
43 np=11998 qsp=0 rpq=11995 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
44 np=11774 qsp=0 rpq=11770 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
45 np=11813 qsp=0 rpq=11807 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=6
46 np=11636 qsp=0 rpq=11633 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
47 np=12042 qsp=0 rpq=12039 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
48 np=16043 qsp=0 rpq=13693 cbr=0 cng=1 gpc=3 gps=2 nf=0 nn=2344
49 np=14791 qsp=0 rpq=12614 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=2174
50 np=12984 qsp=0 rpq=12601 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=383
51 np=12713 qsp=0 rpq=12614 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
52 np=12606 qsp=0 rpq=12563 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=43
53 np=12774 qsp=0 rpq=12764 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
54 np=12155 qsp=0 rpq=12152 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
55 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
56 np=14104 qsp=0 rpq=12179 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=1923
57 np=14321 qsp=0 rpq=12025 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=2292
58 np=12346 qsp=0 rpq=11955 cbr=0 cng=0 gpc=1 gps=1 nf=0 nn=389
59 np=12156 qsp=0 rpq=12134 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
60 np=11973 qsp=0 rpq=11970 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
61 np=12106 qsp=0 rpq=12102 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=3
62 np=12204 qsp=0 rpq=12200 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
63 np=12804 qsp=0 rpq=12651 cbr=0 cng=0 gpc=3 gps=0 nf=0 nn=150
64 np=15159 qsp=0 rpq=14190 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=969
65 np=13570 qsp=0 rpq=12478 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1088
66 np=12653 qsp=0 rpq=12470 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=183
67 np=12172 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
68 np=12396 qsp=0 rpq=12392 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
69 np=12335 qsp=0 rpq=12334 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
70 np=12394 qsp=0 rpq=12390 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
71 np=12277 qsp=0 rpq=12262 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
72 np=14623 qsp=0 rpq=14140 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=483
73 np=17351 qsp=0 rpq=16476 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=872
74 np=12156 qsp=0 rpq=12077 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=79
75 np=12088 qsp=0 rpq=12078 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
76 np=12085 qsp=0 rpq=12083 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
77 np=12085 qsp=0 rpq=12073 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=12
78 np=12087 qsp=0 rpq=12084 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
79 np=12093 qsp=0 rpq=12090 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
80 np=14085 qsp=0 rpq=12836 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1249
81 np=14712 qsp=0 rpq=13675 cbr=0 cng=0 gpc=3 gps=1 nf=0 nn=1033
82 np=12888 qsp=0 rpq=12827 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=61
83 np=12135 qsp=0 rpq=12122 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=13
84 np=12106 qsp=0 rpq=12103 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
85 np=12117 qsp=0 rpq=12114 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
86 np=12118 qsp=0 rpq=12115 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
87 np=12115 qsp=0 rpq=12111 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
88 np=12975 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=642
89 np=13632 qsp=0 rpq=12832 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=797
90 np=12442 qsp=0 rpq=12343 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=99
91 np=12176 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=16
92 np=12167 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=9
93 np=12164 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
94 np=12163 qsp=0 rpq=12160 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
95 np=12176 qsp=0 rpq=12172 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
96 np=12916 qsp=0 rpq=12333 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=583
97 np=13990 qsp=0 rpq=13175 cbr=0 cng=0 gpc=2 gps=1 nf=0 nn=812
98 np=12165 qsp=0 rpq=12143 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=22
99 np=12146 qsp=0 rpq=12135 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
100 np=12165 qsp=0 rpq=12155 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=10
101 np=12161 qsp=0 rpq=12158 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
102 np=12173 qsp=0 rpq=12170 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
103 np=12178 qsp=0 rpq=12163 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=15
104 np=12597 qsp=0 rpq=12106 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=491
105 np=12839 qsp=0 rpq=12100 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=739
106 np=12132 qsp=0 rpq=12098 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=34
107 np=12105 qsp=0 rpq=12101 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
108 np=12092 qsp=0 rpq=12089 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
109 np=12095 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=4
110 np=12099 qsp=0 rpq=12096 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
111 np=12093 qsp=0 rpq=12091 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=2
112 np=13545 qsp=0 rpq=12849 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=696
113 np=13147 qsp=0 rpq=12321 cbr=0 cng=0 gpc=1 gps=1 nf=1 nn=823
114 np=12554 qsp=0 rpq=12330 cbr=0 cng=0 gpc=1 gps=0 nf=0 nn=223
115 np=12351 qsp=0 rpq=12325 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=26
116 np=12326 qsp=0 rpq=12315 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=11
117 np=12325 qsp=0 rpq=12322 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
118 np=12116 qsp=0 rpq=12113 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
119 np=12113 qsp=0 rpq=12112 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=1
120 np=12742 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=541
121 np=19383 qsp=0 rpq=18247 cbr=0 cng=0 gpc=2 gps=1 nf=1 nn=1132
122 np=12391 qsp=0 rpq=12201 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=190
123 np=12195 qsp=0 rpq=12192 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
124 np=12003 qsp=0 rpq=12000 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
125 np=12127 qsp=0 rpq=12124 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
126 np=12014 qsp=0 rpq=11983 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=31
127 np=12072 qsp=0 rpq=12069 cbr=0 cng=0 gpc=0 gps=0 nf=0 nn=3
/sys/kernel/debug/rcu/rcutorture:
cat: /sys/kernel/debug/rcu/rcutorture: No such file or directory
no /sys/kernel/debug/rcu/rcutorture
/sys/kernel/debug/rcu/rcuboost:
cat: /sys/kernel/debug/rcu/rcuboost: No such file or directory
no /sys/kernel/debug/rcu/rcuboost
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 4:54 ` Paul E. McKenney
` (2 preceding siblings ...)
2011-05-11 16:54 ` Yinghai Lu
@ 2011-05-11 16:56 ` Yinghai Lu
2011-05-11 20:18 ` Paul E. McKenney
3 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 16:56 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On Tue, May 10, 2011 at 9:54 PM, Paul E. McKenney
<paulmck@linux.vnet.ibm.com> wrote:
> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>> > On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>> >> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>> >>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>> >>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>> >>>>>
>> >>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>> >>>>>
>> >>>>>> Hello, Ingo,
>> >>>>>>
>> >>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>> >>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>> >>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>> >>>>>> of the commits. There are a number of smaller features and bug fixes.
>> >>>>>> All have been sent to LKML in the following batches:
>> >>>>>>
>> >>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>> >>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>> >>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>> >>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>> >>>>>>
>> >>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>> >>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>> >>>>>> Acked-by:s, and I will deal with these later.
>> >>>>>>
>> >>>>>> These channges are available in the -rcu git repository at:
>> >>>>>>
>> >>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>> >>>>>
>> >>>>> Pulled, thanks a lot Paul!
>> >>>>>
>> >>>>
>> >>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>> >>>>
>> >>>> after hard code to enable rcu_cpu_stall_suppress
>> >>>>
>> >>>> Index: linux-2.6/kernel/rcutree.c
>> >>>> ===================================================================
>> >>>> --- linux-2.6.orig/kernel/rcutree.c
>> >>>> +++ linux-2.6/kernel/rcutree.c
>> >>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>> >>>> module_param(qhimark, int, 0);
>> >>>> module_param(qlowmark, int, 0);
>> >>>>
>> >>>> -int rcu_cpu_stall_suppress __read_mostly;
>> >>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>> >>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>> >>>>
>> >>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>> >>>>
>> >>>> will get system hang after pnp ACPI init.
>> >>>
>> >>> Could you please send the stack traces from the RCU CPU stall? Also,
>> >>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>> >>>
>> >>> Thanx, Paul
>> >>
>> >> Do not have time to bisect it at this point.
>> >
>> > Could you please send the stack traces from the RCU CPU stall?
>
> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
> Everyone is idle, except for CPU 124, which detected the stall, and
> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
> that looks like a backtrace error to me. The fact that it hangs if you
> disable RCU CPU stall detection leads me to believe that something real
> is being detected.
the problem is that now I can not disable RCU CPU stall detection any more.
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 6:42 ` Yinghai Lu
@ 2011-05-11 20:13 ` Paul E. McKenney
0 siblings, 0 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-11 20:13 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Tue, May 10, 2011 at 11:42:24PM -0700, Yinghai Lu wrote:
> On 05/10/2011 09:54 PM, Paul E. McKenney wrote:
> > On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
> >> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
> >>> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
> >>>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
> >>>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
> >>>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
> >>>>>>>
> >>>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> >>>>>>>
> >>>>>>>> Hello, Ingo,
> >>>>>>>>
> >>>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
> >>>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
> >>>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
> >>>>>>>> of the commits. There are a number of smaller features and bug fixes.
> >>>>>>>> All have been sent to LKML in the following batches:
> >>>>>>>>
> >>>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> >>>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> >>>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> >>>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
> >>>>>>>>
> >>>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
> >>>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
> >>>>>>>> Acked-by:s, and I will deal with these later.
> >>>>>>>>
> >>>>>>>> These channges are available in the -rcu git repository at:
> >>>>>>>>
> >>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
> >>>>>>>
> >>>>>>> Pulled, thanks a lot Paul!
> >>>>>>>
> >>>>>>
> >>>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
> >>>>>>
> >>>>>> after hard code to enable rcu_cpu_stall_suppress
> >>>>>>
> >>>>>> Index: linux-2.6/kernel/rcutree.c
> >>>>>> ===================================================================
> >>>>>> --- linux-2.6.orig/kernel/rcutree.c
> >>>>>> +++ linux-2.6/kernel/rcutree.c
> >>>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
> >>>>>> module_param(qhimark, int, 0);
> >>>>>> module_param(qlowmark, int, 0);
> >>>>>>
> >>>>>> -int rcu_cpu_stall_suppress __read_mostly;
> >>>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
> >>>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
> >>>>>>
> >>>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
> >>>>>>
> >>>>>> will get system hang after pnp ACPI init.
> >>>>>
> >>>>> Could you please send the stack traces from the RCU CPU stall? Also,
> >>>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
> >>>>>
> >>>>> Thanx, Paul
> >>>>
> >>>> Do not have time to bisect it at this point.
> >>>
> >>> Could you please send the stack traces from the RCU CPU stall?
> >
> > Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
> > Everyone is idle, except for CPU 124, which detected the stall, and
> > possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
> > that looks like a backtrace error to me. The fact that it hangs if you
> > disable RCU CPU stall detection leads me to believe that something real
> > is being detected.
> >
> > This looks very similar to the situation people were seeing before
> > ce31332d3c77532d6ea97ddcb475a2b02dd358b4 was applied, so I have attached
> > the diagnostic script that helped track this down. Could you please
> > enable CONFIG_RCU_TRACE, mount debugfs, and run the attached script,
> > and send me the output? Please check to make sure that the script knows
> > where you mounted debugfs, of course.
And this data shows RCU running normally, which goes along with your
saying that this was happening only at boot time.
Thank you for collecting it.
So help me understand what is happening. You get the stall warning
during boot, then sometimes a hang? Or does disabling the stall warnings
prevent the boot-time hang?
Thanx, Paul
> Wed May 11 14:33:35 UTC 2011
> /sys/kernel/debug/rcu/rcugp:
> rcu_sched: completed=4164 gpnum=4165
> rcu_bh: completed=-282 gpnum=18446744073709551334
> /sys/kernel/debug/rcu/rcuhier:
> rcu_sched:
> c=4164 g=4165 s=2 jfq=0 j=92fc nfqs=6792/nfqsng=0(6792) fqlh=737
> ff/ff ..>.. 0:254 ^0
> fdff/ffff ..>.. 0:15 ^0 fffc/ffff ..>.. 16:31 ^1 ffff/ffff ..>.. 32:47 ^2 ffff/ffff ..>.. 48:63 ^3 fdff/ffff ..>.. 64:79 ^4 ffff/ffff ..>.. 80:95 ^5 ffff/ffff ..>.. 96:111 ^6 ffff/ffff ..>.. 112:127 ^7 0/0 ..>.. 128:143 ^8 0/0 ..>.. 144:159 ^9 0/0 ..>.. 160:175 ^10 0/0 ..>.. 176:191 ^11 0/0 ..>.. 192:207 ^12 0/0 ..>.. 208:223 ^13 0/0 ..>.. 224:239 ^14 0/0 ..>.. 240:254 ^15
...
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 16:56 ` Yinghai Lu
@ 2011-05-11 20:18 ` Paul E. McKenney
2011-05-11 20:59 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-11 20:18 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Wed, May 11, 2011 at 09:56:35AM -0700, Yinghai Lu wrote:
> On Tue, May 10, 2011 at 9:54 PM, Paul E. McKenney
> <paulmck@linux.vnet.ibm.com> wrote:
> > On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
> >> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
> >> > On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
> >> >> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
> >> >>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
> >> >>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
> >> >>>>>
> >> >>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> >> >>>>>
> >> >>>>>> Hello, Ingo,
> >> >>>>>>
> >> >>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
> >> >>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
> >> >>>>>> courtesy of Lai Jiangshan. These two features cover well over half
> >> >>>>>> of the commits. There are a number of smaller features and bug fixes.
> >> >>>>>> All have been sent to LKML in the following batches:
> >> >>>>>>
> >> >>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
> >> >>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
> >> >>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
> >> >>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
> >> >>>>>>
> >> >>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
> >> >>>>>> maintainers. I have some additional kfree_rcu() requests that lack
> >> >>>>>> Acked-by:s, and I will deal with these later.
> >> >>>>>>
> >> >>>>>> These channges are available in the -rcu git repository at:
> >> >>>>>>
> >> >>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
> >> >>>>>
> >> >>>>> Pulled, thanks a lot Paul!
> >> >>>>>
> >> >>>>
> >> >>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
> >> >>>>
> >> >>>> after hard code to enable rcu_cpu_stall_suppress
> >> >>>>
> >> >>>> Index: linux-2.6/kernel/rcutree.c
> >> >>>> ===================================================================
> >> >>>> --- linux-2.6.orig/kernel/rcutree.c
> >> >>>> +++ linux-2.6/kernel/rcutree.c
> >> >>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
> >> >>>> module_param(qhimark, int, 0);
> >> >>>> module_param(qlowmark, int, 0);
> >> >>>>
> >> >>>> -int rcu_cpu_stall_suppress __read_mostly;
> >> >>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
> >> >>>> module_param(rcu_cpu_stall_suppress, int, 0644);
> >> >>>>
> >> >>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
> >> >>>>
> >> >>>> will get system hang after pnp ACPI init.
> >> >>>
> >> >>> Could you please send the stack traces from the RCU CPU stall? Also,
> >> >>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
> >> >>>
> >> >>> Thanx, Paul
> >> >>
> >> >> Do not have time to bisect it at this point.
> >> >
> >> > Could you please send the stack traces from the RCU CPU stall?
> >
> > Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
> > Everyone is idle, except for CPU 124, which detected the stall, and
> > possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
> > that looks like a backtrace error to me. The fact that it hangs if you
> > disable RCU CPU stall detection leads me to believe that something real
> > is being detected.
>
> the problem is that now I can not disable RCU CPU stall detection any more.
There is a rcu_cpu_stall_suppress module parameter, and you should be
able to pass in rcu_cpu_stall_suppress=1 as a boot parameter. However,
I did produce a patch that reverts the change, please see below.
I would be surprised if this did anything different than your change
that initializes rcu_cpu_stall_suppress to 1. If this patch somehow
does make a difference, please let me know.
Thanx, Paul
------------------------------------------------------------------------
diff --git a/Documentation/RCU/00-INDEX b/Documentation/RCU/00-INDEX
index 1d7a885..71b6f50 100644
--- a/Documentation/RCU/00-INDEX
+++ b/Documentation/RCU/00-INDEX
@@ -21,7 +21,7 @@ rcu.txt
RTFP.txt
- List of RCU papers (bibliography) going back to 1980.
stallwarn.txt
- - RCU CPU stall warnings (module parameter rcu_cpu_stall_suppress)
+ - RCU CPU stall warnings (CONFIG_RCU_CPU_STALL_DETECTOR)
torture.txt
- RCU Torture Test Operation (CONFIG_RCU_TORTURE_TEST)
trace.txt
diff --git a/Documentation/RCU/stallwarn.txt b/Documentation/RCU/stallwarn.txt
index 4e95920..862c08e 100644
--- a/Documentation/RCU/stallwarn.txt
+++ b/Documentation/RCU/stallwarn.txt
@@ -1,25 +1,22 @@
Using RCU's CPU Stall Detector
-The rcu_cpu_stall_suppress module parameter enables RCU's CPU stall
-detector, which detects conditions that unduly delay RCU grace periods.
-This module parameter enables CPU stall detection by default, but
-may be overridden via boot-time parameter or at runtime via sysfs.
-The stall detector's idea of what constitutes "unduly delayed" is
-controlled by a set of kernel configuration variables and cpp macros:
+The CONFIG_RCU_CPU_STALL_DETECTOR kernel config parameter enables
+RCU's CPU stall detector, which detects conditions that unduly delay
+RCU grace periods. The stall detector's idea of what constitutes
+"unduly delayed" is controlled by a set of C preprocessor macros:
-CONFIG_RCU_CPU_STALL_TIMEOUT
+RCU_SECONDS_TILL_STALL_CHECK
- This kernel configuration parameter defines the period of time
- that RCU will wait from the beginning of a grace period until it
- issues an RCU CPU stall warning. This time period is normally
- ten seconds.
+ This macro defines the period of time that RCU will wait from
+ the beginning of a grace period until it issues an RCU CPU
+ stall warning. This time period is normally ten seconds.
RCU_SECONDS_TILL_STALL_RECHECK
This macro defines the period of time that RCU will wait after
issuing a stall warning until it issues another stall warning
- for the same stall. This time period is normally set to three
- times the check interval plus thirty seconds.
+ for the same stall. This time period is normally set to thirty
+ seconds.
RCU_STALL_RAT_DELAY
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 86f44a3..2e8fbed 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -174,8 +174,10 @@ module_param(blimit, int, 0);
module_param(qhimark, int, 0);
module_param(qlowmark, int, 0);
-int rcu_cpu_stall_suppress __read_mostly;
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
+int rcu_cpu_stall_suppress __read_mostly = RCU_CPU_STALL_SUPPRESS_INIT;
module_param(rcu_cpu_stall_suppress, int, 0644);
+#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
static int rcu_pending(int cpu);
@@ -497,6 +499,8 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
#endif /* #else #ifdef CONFIG_NO_HZ */
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
+
int rcu_cpu_stall_suppress __read_mostly;
static void record_gp_stall_check_time(struct rcu_state *rsp)
@@ -635,6 +639,26 @@ static void __init check_cpu_stall_init(void)
atomic_notifier_chain_register(&panic_notifier_list, &rcu_panic_block);
}
+#else /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
+
+static void record_gp_stall_check_time(struct rcu_state *rsp)
+{
+}
+
+static void check_cpu_stall(struct rcu_state *rsp, struct rcu_data *rdp)
+{
+}
+
+void rcu_cpu_stall_reset(void)
+{
+}
+
+static void __init check_cpu_stall_init(void)
+{
+}
+
+#endif /* #else #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
+
/*
* Update CPU-local rcu_data state to record the newly noticed grace period.
* This is used both when we started the grace period and when we notice
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index 93d4a1c..c8e5bf4 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -317,6 +317,7 @@ struct rcu_data {
#endif /* #else #ifdef CONFIG_NO_HZ */
#define RCU_JIFFIES_TILL_FORCE_QS 3 /* for rsp->jiffies_force_qs */
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
#ifdef CONFIG_PROVE_RCU
#define RCU_STALL_DELAY_DELTA (5 * HZ)
@@ -334,6 +335,13 @@ struct rcu_data {
/* scheduling clock irq */
/* before ratting on them. */
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR_RUNNABLE
+#define RCU_CPU_STALL_SUPPRESS_INIT 0
+#else
+#define RCU_CPU_STALL_SUPPRESS_INIT 1
+#endif
+
+#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
/*
* RCU global state, including node hierarchy. This hierarchy is
@@ -382,8 +390,10 @@ struct rcu_state {
/* due to no GP active. */
unsigned long gp_start; /* Time at which GP started, */
/* but in jiffies. */
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
unsigned long jiffies_stall; /* Time at which to check */
/* for CPU stalls. */
+#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
unsigned long gp_max; /* Maximum GP duration in */
/* jiffies. */
char *name; /* Name of structure. */
@@ -421,9 +431,11 @@ static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp);
static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp,
unsigned long flags);
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
static void rcu_print_detail_task_stall(struct rcu_state *rsp);
static void rcu_print_task_stall(struct rcu_node *rnp);
static void rcu_preempt_stall_reset(void);
+#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp);
#ifdef CONFIG_HOTPLUG_CPU
static int rcu_preempt_offline_tasks(struct rcu_state *rsp,
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index ed339702..f77bc10 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -54,6 +54,10 @@ static void __init rcu_bootup_announce_oddness(void)
#ifdef CONFIG_RCU_TORTURE_TEST_RUNNABLE
printk(KERN_INFO "\tRCU torture testing starts during boot.\n");
#endif
+#ifndef CONFIG_RCU_CPU_STALL_DETECTOR
+ printk(KERN_INFO
+ "\tRCU-based detection of stalled CPUs is disabled.\n");
+#endif
#if defined(CONFIG_TREE_PREEMPT_RCU) && !defined(CONFIG_RCU_CPU_STALL_VERBOSE)
printk(KERN_INFO "\tVerbose stalled-CPUs detection is disabled.\n");
#endif
@@ -398,6 +402,8 @@ void __rcu_read_unlock(void)
}
EXPORT_SYMBOL_GPL(__rcu_read_unlock);
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
+
#ifdef CONFIG_RCU_CPU_STALL_VERBOSE
/*
@@ -466,6 +472,8 @@ static void rcu_preempt_stall_reset(void)
rcu_preempt_state.jiffies_stall = jiffies + ULONG_MAX / 2;
}
+#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
+
/*
* Check that the list of blocked tasks for the newly completed grace
* period is in fact empty. It is a serious bug to complete a grace
@@ -922,6 +930,8 @@ static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags)
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR
+
/*
* Because preemptible RCU does not exist, we never have to check for
* tasks blocked within RCU read-side critical sections.
@@ -946,6 +956,8 @@ static void rcu_preempt_stall_reset(void)
{
}
+#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
+
/*
* Because there is no preemptible RCU, there can be no readers blocked,
* so there is no need to check for blocked tasks. So check only for
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 3aa2780..a863e35 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -875,9 +875,22 @@ config RCU_TORTURE_TEST_RUNNABLE
Say N here if you want the RCU torture tests to start only
after being manually enabled via /proc.
+config RCU_CPU_STALL_DETECTOR
+ bool "Check for stalled CPUs delaying RCU grace periods"
+ depends on TREE_RCU || TREE_PREEMPT_RCU
+ default y
+ help
+ This option causes RCU to printk information on which
+ CPUs are delaying the current grace period, but only when
+ the grace period extends for excessive time periods.
+
+ Say N if you want to disable such checks.
+
+ Say Y if you are unsure.
+
config RCU_CPU_STALL_TIMEOUT
int "RCU CPU stall timeout in seconds"
- depends on TREE_RCU || TREE_PREEMPT_RCU
+ depends on RCU_CPU_STALL_DETECTOR
range 3 300
default 60
help
@@ -886,9 +899,22 @@ config RCU_CPU_STALL_TIMEOUT
RCU grace period persists, additional CPU stall warnings are
printed at more widely spaced intervals.
+config RCU_CPU_STALL_DETECTOR_RUNNABLE
+ bool "RCU CPU stall checking starts automatically at boot"
+ depends on RCU_CPU_STALL_DETECTOR
+ default y
+ help
+ If set, start checking for RCU CPU stalls immediately on
+ boot. Otherwise, RCU CPU stall checking must be manually
+ enabled.
+
+ Say Y if you are unsure.
+
+ Say N if you wish to suppress RCU CPU stall checking during boot.
+
config RCU_CPU_STALL_VERBOSE
bool "Print additional per-task information for RCU_CPU_STALL_DETECTOR"
- depends on TREE_PREEMPT_RCU
+ depends on RCU_CPU_STALL_DETECTOR && TREE_PREEMPT_RCU
default y
help
This option causes RCU to printk detailed per-task information
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 20:18 ` Paul E. McKenney
@ 2011-05-11 20:59 ` Yinghai Lu
2011-05-11 21:30 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 20:59 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel, Len Brown
On 05/11/2011 01:18 PM, Paul E. McKenney wrote:
> On Wed, May 11, 2011 at 09:56:35AM -0700, Yinghai Lu wrote:
>> On Tue, May 10, 2011 at 9:54 PM, Paul E. McKenney
>> <paulmck@linux.vnet.ibm.com> wrote:
>>> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>>>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>>>>> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>>>>>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>>>>>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>>>>>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>>>>>>>
>>>>>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello, Ingo,
>>>>>>>>>>
>>>>>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>>>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>>>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>>>>>>>> of the commits. There are a number of smaller features and bug fixes.
>>>>>>>>>> All have been sent to LKML in the following batches:
>>>>>>>>>>
>>>>>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>>>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>>>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>>>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>>>>>>>
>>>>>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>>>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>>>>>>>> Acked-by:s, and I will deal with these later.
>>>>>>>>>>
>>>>>>>>>> These channges are available in the -rcu git repository at:
>>>>>>>>>>
>>>>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>>>>>>>
>>>>>>>>> Pulled, thanks a lot Paul!
>>>>>>>>>
>>>>>>>>
>>>>>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>>>>>>>
>>>>>>>> after hard code to enable rcu_cpu_stall_suppress
>>>>>>>>
>>>>>>>> Index: linux-2.6/kernel/rcutree.c
>>>>>>>> ===================================================================
>>>>>>>> --- linux-2.6.orig/kernel/rcutree.c
>>>>>>>> +++ linux-2.6/kernel/rcutree.c
>>>>>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>>>>>>>> module_param(qhimark, int, 0);
>>>>>>>> module_param(qlowmark, int, 0);
>>>>>>>>
>>>>>>>> -int rcu_cpu_stall_suppress __read_mostly;
>>>>>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>>>>>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>>>>>>>
>>>>>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>>>>>>>
>>>>>>>> will get system hang after pnp ACPI init.
>>>>>>>
>>>>>>> Could you please send the stack traces from the RCU CPU stall? Also,
>>>>>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>>>>>>>
>>>>>>> Thanx, Paul
>>>>>>
>>>>>> Do not have time to bisect it at this point.
>>>>>
>>>>> Could you please send the stack traces from the RCU CPU stall?
>>>
>>> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
>>> Everyone is idle, except for CPU 124, which detected the stall, and
>>> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
>>> that looks like a backtrace error to me. The fact that it hangs if you
>>> disable RCU CPU stall detection leads me to believe that something real
>>> is being detected.
>>
>> the problem is that now I can not disable RCU CPU stall detection any more.
>
> There is a rcu_cpu_stall_suppress module parameter, and you should be
> able to pass in rcu_cpu_stall_suppress=1 as a boot parameter. However,
> I did produce a patch that reverts the change, please see below.
> I would be surprised if this did anything different than your change
> that initializes rcu_cpu_stall_suppress to 1. If this patch somehow
> does make a difference, please let me know.
the same.
looks like other commit in the pull cause the delay...of acpi related stuff.
[ 22.255535] calling pnpacpi_init+0x0/0x8c @ 1
[ 22.260001] pnp: PnP ACPI init
[ 22.263125] ACPI: bus type pnp registered
[ 603.121084] pnp 00:00: [bus 00-7f]
...
[ 603.130564] pnp 00:0c: Plug and Play ACPI device, IDs PNP0103 (active)
[ 1202.948788] pnp 00:0d: [bus 80-ff]
[ 1202.952187] pnp 00:0d: [io 0x0000 window]
[ 1202.956316] pnp 00:0d: [io 0xc000-0xffff window]
[ 1202.961056] pnp 00:0d: [mem 0x00000000 window]
[ 1202.965533] pnp 00:0d: [mem 0xe0000000-0xfbffffff window]
[ 1202.970965] pnp 00:0d: [mem 0x00000000 window]
[ 1202.975934] pnp 00:0d: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
[ 1202.983823] system 00:0e: Plug and Play ACPI device, IDs PNP0c01 (active)
[ 1202.991093] pnp: PnP ACPI: found 15 devices
[ 1202.995310] ACPI: ACPI bus type pnp unregistered
[ 1202.999963] initcall pnpacpi_init+0x0/0x8c returned 0 after 1153066591 usecs
...
[ 1206.094838] calling shpcd_init+0x0/0xe8 @ 1
[ 1206.095194] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 1206.095199] initcall shpcd_init+0x0/0xe8 returned 0 after 347 usecs
[ 1206.095202] calling acpiphp_init+0x0/0x5f @ 1
[ 1206.095206] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 1302.573367] initcall acpiphp_init+0x0/0x5f returned -19 after 94216997 usecs
[ 1302.580449] calling ibm_acpiphp_init+0x0/0x175 @ 1
[ 1398.573429] acpiphp_ibm: ibm_acpiphp_init: acpi_walk_namespace failed
[ 1398.579896] initcall ibm_acpiphp_init+0x0/0x175 returned -19 after 93744718 usecs
...
[ 1398.739321] calling acpi_pci_slot_init+0x0/0x20 @ 1
[ 1494.572598] initcall acpi_pci_slot_init+0x0/0x20 returned 0 after 93582352 usecs
[ 1494.580019] calling acpi_processor_init+0x0/0xcd @ 1
[ 1494.585119] ACPI: acpi_idle registered with cpuidle
[ 1494.622886] initcall acpi_processor_init+0x0/0xcd returned 0 after 36896 usecs
[ 1494.630135] calling acpi_container_init+0x0/0x4a @ 1
[ 1734.573253] initcall acpi_container_init+0x0/0x4a returned 0 after 234314615 usecs
[ 1734.580854] calling acpi_thermal_init+0x0/0x42 @ 1
[ 1734.586032] initcall acpi_thermal_init+0x0/0x42 returned 0 after 275 usecs
[ 1734.592926] calling acpi_memory_device_init+0x0/0x87 @ 1
[ 1974.573125] initcall acpi_memory_device_init+0x0/0x87 returned 0 after 234350484 usecs
[ 1974.581069] calling acpi_battery_init+0x0/0x16 @ 1
[ 1974.586009] initcall acpi_battery_init+0x0/0x16 returned 0 after 34 usecs
[ 1974.586112] calling 1_acpi_battery_init_async+0x0/0x20 @ 5
[ 1974.586636] initcall 1_acpi_battery_init_async+0x0/0x20 returned 0 after 505 usecs
[ 1974.605962] calling acpi_power_meter_init+0x0/0x32 @ 1
[ 1974.611505] initcall acpi_power_meter_init+0x0/0x32 returned 0 after 286 usecs
[ 1974.618748] calling acpi_hed_init+0x0/0x26 @ 1
[ 1974.623565] initcall acpi_hed_init+0x0/0x26 returned 0 after 255 usecs
[ 1974.630115] calling acpi_pad_init+0x0/0x26 @ 1
[ 1974.634932] initcall acpi_pad_init+0x0/0x26 returned 0 after 254 usecs
[ 1974.641475] calling erst_init+0x0/0x2bb @ 1
[ 1974.645956] ERST: Can not request iomem region <0x 3f-0x 3f> for ERST.
[ 2070.572123] initcall erst_init+0x0/0x2bb returned -5 after 93678128 usecs
...
[ 2070.792625] calling serial8250_init+0x0/0x18e @ 1
[ 2070.797433] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 2070.824642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 2534.572086] initcall serial8250_init+0x0/0x18e returned 0 after 452905158 usecs
[ 2534.579451] calling serial8250_pnp_init+0x0/0x12 @ 1
[ 2662.572104] initcall serial8250_pnp_init+0x0/0x12 returned 0 after 124987929 usecs
[ 2662.579749] calling serial8250_pci_init+0x0/0x1b @ 1
[ 2662.585088] initcall serial8250_pci_init+0x0/0x1b returned 0 after 261 usecs
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 20:59 ` Yinghai Lu
@ 2011-05-11 21:30 ` Yinghai Lu
2011-05-11 23:02 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 21:30 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel, Len Brown
On 05/11/2011 01:59 PM, Yinghai Lu wrote:
> On 05/11/2011 01:18 PM, Paul E. McKenney wrote:
>> On Wed, May 11, 2011 at 09:56:35AM -0700, Yinghai Lu wrote:
>>> On Tue, May 10, 2011 at 9:54 PM, Paul E. McKenney
>>> <paulmck@linux.vnet.ibm.com> wrote:
>>>> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>>>>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>>>>>> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>>>>>>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>>>>>>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>>>>>>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>>>>>>>>
>>>>>>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello, Ingo,
>>>>>>>>>>>
>>>>>>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>>>>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>>>>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>>>>>>>>> of the commits. There are a number of smaller features and bug fixes.
>>>>>>>>>>> All have been sent to LKML in the following batches:
>>>>>>>>>>>
>>>>>>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>>>>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>>>>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>>>>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>>>>>>>>
>>>>>>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>>>>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>>>>>>>>> Acked-by:s, and I will deal with these later.
>>>>>>>>>>>
>>>>>>>>>>> These channges are available in the -rcu git repository at:
>>>>>>>>>>>
>>>>>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>>>>>>>>
>>>>>>>>>> Pulled, thanks a lot Paul!
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>>>>>>>>
>>>>>>>>> after hard code to enable rcu_cpu_stall_suppress
>>>>>>>>>
>>>>>>>>> Index: linux-2.6/kernel/rcutree.c
>>>>>>>>> ===================================================================
>>>>>>>>> --- linux-2.6.orig/kernel/rcutree.c
>>>>>>>>> +++ linux-2.6/kernel/rcutree.c
>>>>>>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>>>>>>>>> module_param(qhimark, int, 0);
>>>>>>>>> module_param(qlowmark, int, 0);
>>>>>>>>>
>>>>>>>>> -int rcu_cpu_stall_suppress __read_mostly;
>>>>>>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>>>>>>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>>>>>>>>
>>>>>>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>>>>>>>>
>>>>>>>>> will get system hang after pnp ACPI init.
>>>>>>>>
>>>>>>>> Could you please send the stack traces from the RCU CPU stall? Also,
>>>>>>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>>>>>>>>
>>>>>>>> Thanx, Paul
>>>>>>>
>>>>>>> Do not have time to bisect it at this point.
>>>>>>
>>>>>> Could you please send the stack traces from the RCU CPU stall?
>>>>
>>>> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
>>>> Everyone is idle, except for CPU 124, which detected the stall, and
>>>> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
>>>> that looks like a backtrace error to me. The fact that it hangs if you
>>>> disable RCU CPU stall detection leads me to believe that something real
>>>> is being detected.
>>>
>>> the problem is that now I can not disable RCU CPU stall detection any more.
>>
>> There is a rcu_cpu_stall_suppress module parameter, and you should be
>> able to pass in rcu_cpu_stall_suppress=1 as a boot parameter. However,
>> I did produce a patch that reverts the change, please see below.
>> I would be surprised if this did anything different than your change
>> that initializes rcu_cpu_stall_suppress to 1. If this patch somehow
>> does make a difference, please let me know.
>
> the same.
>
> looks like other commit in the pull cause the delay...of acpi related stuff.
>
> [ 22.255535] calling pnpacpi_init+0x0/0x8c @ 1
> [ 22.260001] pnp: PnP ACPI init
> [ 22.263125] ACPI: bus type pnp registered
> [ 603.121084] pnp 00:00: [bus 00-7f]
> ...
> [ 603.130564] pnp 00:0c: Plug and Play ACPI device, IDs PNP0103 (active)
> [ 1202.948788] pnp 00:0d: [bus 80-ff]
> [ 1202.952187] pnp 00:0d: [io 0x0000 window]
> [ 1202.956316] pnp 00:0d: [io 0xc000-0xffff window]
> [ 1202.961056] pnp 00:0d: [mem 0x00000000 window]
> [ 1202.965533] pnp 00:0d: [mem 0xe0000000-0xfbffffff window]
> [ 1202.970965] pnp 00:0d: [mem 0x00000000 window]
> [ 1202.975934] pnp 00:0d: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
> [ 1202.983823] system 00:0e: Plug and Play ACPI device, IDs PNP0c01 (active)
> [ 1202.991093] pnp: PnP ACPI: found 15 devices
> [ 1202.995310] ACPI: ACPI bus type pnp unregistered
> [ 1202.999963] initcall pnpacpi_init+0x0/0x8c returned 0 after 1153066591 usecs
> ...
> [ 1206.094838] calling shpcd_init+0x0/0xe8 @ 1
> [ 1206.095194] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
> [ 1206.095199] initcall shpcd_init+0x0/0xe8 returned 0 after 347 usecs
> [ 1206.095202] calling acpiphp_init+0x0/0x5f @ 1
> [ 1206.095206] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
> [ 1302.573367] initcall acpiphp_init+0x0/0x5f returned -19 after 94216997 usecs
> [ 1302.580449] calling ibm_acpiphp_init+0x0/0x175 @ 1
> [ 1398.573429] acpiphp_ibm: ibm_acpiphp_init: acpi_walk_namespace failed
> [ 1398.579896] initcall ibm_acpiphp_init+0x0/0x175 returned -19 after 93744718 usecs
> ...
> [ 1398.739321] calling acpi_pci_slot_init+0x0/0x20 @ 1
> [ 1494.572598] initcall acpi_pci_slot_init+0x0/0x20 returned 0 after 93582352 usecs
> [ 1494.580019] calling acpi_processor_init+0x0/0xcd @ 1
> [ 1494.585119] ACPI: acpi_idle registered with cpuidle
> [ 1494.622886] initcall acpi_processor_init+0x0/0xcd returned 0 after 36896 usecs
> [ 1494.630135] calling acpi_container_init+0x0/0x4a @ 1
> [ 1734.573253] initcall acpi_container_init+0x0/0x4a returned 0 after 234314615 usecs
> [ 1734.580854] calling acpi_thermal_init+0x0/0x42 @ 1
> [ 1734.586032] initcall acpi_thermal_init+0x0/0x42 returned 0 after 275 usecs
> [ 1734.592926] calling acpi_memory_device_init+0x0/0x87 @ 1
> [ 1974.573125] initcall acpi_memory_device_init+0x0/0x87 returned 0 after 234350484 usecs
> [ 1974.581069] calling acpi_battery_init+0x0/0x16 @ 1
> [ 1974.586009] initcall acpi_battery_init+0x0/0x16 returned 0 after 34 usecs
> [ 1974.586112] calling 1_acpi_battery_init_async+0x0/0x20 @ 5
> [ 1974.586636] initcall 1_acpi_battery_init_async+0x0/0x20 returned 0 after 505 usecs
> [ 1974.605962] calling acpi_power_meter_init+0x0/0x32 @ 1
> [ 1974.611505] initcall acpi_power_meter_init+0x0/0x32 returned 0 after 286 usecs
> [ 1974.618748] calling acpi_hed_init+0x0/0x26 @ 1
> [ 1974.623565] initcall acpi_hed_init+0x0/0x26 returned 0 after 255 usecs
> [ 1974.630115] calling acpi_pad_init+0x0/0x26 @ 1
> [ 1974.634932] initcall acpi_pad_init+0x0/0x26 returned 0 after 254 usecs
> [ 1974.641475] calling erst_init+0x0/0x2bb @ 1
> [ 1974.645956] ERST: Can not request iomem region <0x 3f-0x 3f> for ERST.
> [ 2070.572123] initcall erst_init+0x0/0x2bb returned -5 after 93678128 usecs
> ...
> [ 2070.792625] calling serial8250_init+0x0/0x18e @ 1
> [ 2070.797433] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> [ 2070.824642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> [ 2534.572086] initcall serial8250_init+0x0/0x18e returned 0 after 452905158 usecs
> [ 2534.579451] calling serial8250_pnp_init+0x0/0x12 @ 1
> [ 2662.572104] initcall serial8250_pnp_init+0x0/0x12 returned 0 after 124987929 usecs
> [ 2662.579749] calling serial8250_pci_init+0x0/0x1b @ 1
> [ 2662.585088] initcall serial8250_pci_init+0x0/0x1b returned 0 after 261 usecs
>
without this pull:
[ 18.733956] calling pnpacpi_init+0x0/0x8c @ 1
[ 18.738400] pnp: PnP ACPI init
[ 18.741484] ACPI: bus type pnp registered
[ 18.798240] pnp 00:00: [bus 00-7f]
[ 18.801638] pnp 00:00: [io 0x0cf8-0x0cff]
...
[ 19.302232] pnp 00:0c: Plug and Play ACPI device, IDs PNP0103 (active)
[ 19.381017] pnp 00:0d: [bus 80-ff]
[ 19.384410] pnp 00:0d: [io 0x0000 window]
[ 19.388502] pnp 00:0d: [io 0xc000-0xffff window]
[ 19.393205] pnp 00:0d: [mem 0x00000000 window]
[ 19.397639] pnp 00:0d: [mem 0xe0000000-0xfbffffff window]
[ 19.403031] pnp 00:0d: [mem 0x00000000 window]
[ 19.407874] pnp 00:0d: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
[ 19.415591] system 00:0e: Plug and Play ACPI device, IDs PNP0c01 (active)
[ 19.422747] pnp: PnP ACPI: found 15 devices
[ 19.426924] ACPI: ACPI bus type pnp unregistered
[ 19.431538] initcall pnpacpi_init+0x0/0x8c returned 0 after 678214 usecs
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 21:30 ` Yinghai Lu
@ 2011-05-11 23:02 ` Yinghai Lu
2011-05-12 6:03 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-11 23:02 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel, Len Brown
On 05/11/2011 02:30 PM, Yinghai Lu wrote:
> On 05/11/2011 01:59 PM, Yinghai Lu wrote:
>> On 05/11/2011 01:18 PM, Paul E. McKenney wrote:
>>> On Wed, May 11, 2011 at 09:56:35AM -0700, Yinghai Lu wrote:
>>>> On Tue, May 10, 2011 at 9:54 PM, Paul E. McKenney
>>>> <paulmck@linux.vnet.ibm.com> wrote:
>>>>> On Tue, May 10, 2011 at 01:52:52PM -0700, Yinghai Lu wrote:
>>>>>> On 05/10/2011 12:32 PM, Paul E. McKenney wrote:
>>>>>>> On Tue, May 10, 2011 at 11:04:57AM -0700, Yinghai Lu wrote:
>>>>>>>> On 05/10/2011 01:56 AM, Paul E. McKenney wrote:
>>>>>>>>> On Mon, May 09, 2011 at 02:09:21PM -0700, Yinghai Lu wrote:
>>>>>>>>>> On Mon, May 9, 2011 at 12:36 AM, Ingo Molnar <mingo@elte.hu> wrote:
>>>>>>>>>>>
>>>>>>>>>>> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello, Ingo,
>>>>>>>>>>>>
>>>>>>>>>>>> This pull request covers RCU chnages for 2.6.40. The major new features
>>>>>>>>>>>> are RCU priority boosting and the addition of kfree_rcu(), the latter
>>>>>>>>>>>> courtesy of Lai Jiangshan. These two features cover well over half
>>>>>>>>>>>> of the commits. There are a number of smaller features and bug fixes.
>>>>>>>>>>>> All have been sent to LKML in the following batches:
>>>>>>>>>>>>
>>>>>>>>>>>> 0. https://lkml.org/lkml/2011/2/22/660: RCU priority boosting preview
>>>>>>>>>>>> 1. https://lkml.org/lkml/2011/5/1/19: RCU priority boosting, kfree_rcu()
>>>>>>>>>>>> 2. https://lkml.org/lkml/2011/5/2/40: More uses of kfree_rcu()
>>>>>>>>>>>> 3. https://lkml.org/lkml/2011/5/8/60: miscellaneous
>>>>>>>>>>>>
>>>>>>>>>>>> The kfree_rcu() uses in the pull request have Acked-by:s from the
>>>>>>>>>>>> maintainers. I have some additional kfree_rcu() requests that lack
>>>>>>>>>>>> Acked-by:s, and I will deal with these later.
>>>>>>>>>>>>
>>>>>>>>>>>> These channges are available in the -rcu git repository at:
>>>>>>>>>>>>
>>>>>>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/next
>>>>>>>>>>>
>>>>>>>>>>> Pulled, thanks a lot Paul!
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> it seems with this one in tip, my 8 sockets test setup will report cpu stall.
>>>>>>>>>>
>>>>>>>>>> after hard code to enable rcu_cpu_stall_suppress
>>>>>>>>>>
>>>>>>>>>> Index: linux-2.6/kernel/rcutree.c
>>>>>>>>>> ===================================================================
>>>>>>>>>> --- linux-2.6.orig/kernel/rcutree.c
>>>>>>>>>> +++ linux-2.6/kernel/rcutree.c
>>>>>>>>>> @@ -174,7 +174,7 @@ module_param(blimit, int, 0);
>>>>>>>>>> module_param(qhimark, int, 0);
>>>>>>>>>> module_param(qlowmark, int, 0);
>>>>>>>>>>
>>>>>>>>>> -int rcu_cpu_stall_suppress __read_mostly;
>>>>>>>>>> +int rcu_cpu_stall_suppress __read_mostly = 1;
>>>>>>>>>> module_param(rcu_cpu_stall_suppress, int, 0644);
>>>>>>>>>>
>>>>>>>>>> static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
>>>>>>>>>>
>>>>>>>>>> will get system hang after pnp ACPI init.
>>>>>>>>>
>>>>>>>>> Could you please send the stack traces from the RCU CPU stall? Also,
>>>>>>>>> you do have ce31332d3c77532d6ea97ddcb475a2b02dd358b4 applied, correct?
>>>>>>>>>
>>>>>>>>> Thanx, Paul
>>>>>>>>
>>>>>>>> Do not have time to bisect it at this point.
>>>>>>>
>>>>>>> Could you please send the stack traces from the RCU CPU stall?
>>>>>
>>>>> Thank you! OK, so CPU 0 has not been responding, despite resched IPIs.
>>>>> Everyone is idle, except for CPU 124, which detected the stall, and
>>>>> possibly CPU 0, which has csum_partial_copy_generic() on the stack, though
>>>>> that looks like a backtrace error to me. The fact that it hangs if you
>>>>> disable RCU CPU stall detection leads me to believe that something real
>>>>> is being detected.
>>>>
>>>> the problem is that now I can not disable RCU CPU stall detection any more.
>>>
>>> There is a rcu_cpu_stall_suppress module parameter, and you should be
>>> able to pass in rcu_cpu_stall_suppress=1 as a boot parameter. However,
>>> I did produce a patch that reverts the change, please see below.
>>> I would be surprised if this did anything different than your change
>>> that initializes rcu_cpu_stall_suppress to 1. If this patch somehow
>>> does make a difference, please let me know.
>>
>> the same.
>>
>> looks like other commit in the pull cause the delay...of acpi related stuff.
>>
>> [ 22.255535] calling pnpacpi_init+0x0/0x8c @ 1
>> [ 22.260001] pnp: PnP ACPI init
>> [ 22.263125] ACPI: bus type pnp registered
>> [ 603.121084] pnp 00:00: [bus 00-7f]
>> ...
>> [ 603.130564] pnp 00:0c: Plug and Play ACPI device, IDs PNP0103 (active)
>> [ 1202.948788] pnp 00:0d: [bus 80-ff]
>> [ 1202.952187] pnp 00:0d: [io 0x0000 window]
>> [ 1202.956316] pnp 00:0d: [io 0xc000-0xffff window]
>> [ 1202.961056] pnp 00:0d: [mem 0x00000000 window]
>> [ 1202.965533] pnp 00:0d: [mem 0xe0000000-0xfbffffff window]
>> [ 1202.970965] pnp 00:0d: [mem 0x00000000 window]
>> [ 1202.975934] pnp 00:0d: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
>> [ 1202.983823] system 00:0e: Plug and Play ACPI device, IDs PNP0c01 (active)
>> [ 1202.991093] pnp: PnP ACPI: found 15 devices
>> [ 1202.995310] ACPI: ACPI bus type pnp unregistered
>> [ 1202.999963] initcall pnpacpi_init+0x0/0x8c returned 0 after 1153066591 usecs
>> ...
>> [ 1206.094838] calling shpcd_init+0x0/0xe8 @ 1
>> [ 1206.095194] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
>> [ 1206.095199] initcall shpcd_init+0x0/0xe8 returned 0 after 347 usecs
>> [ 1206.095202] calling acpiphp_init+0x0/0x5f @ 1
>> [ 1206.095206] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
>> [ 1302.573367] initcall acpiphp_init+0x0/0x5f returned -19 after 94216997 usecs
>> [ 1302.580449] calling ibm_acpiphp_init+0x0/0x175 @ 1
>> [ 1398.573429] acpiphp_ibm: ibm_acpiphp_init: acpi_walk_namespace failed
>> [ 1398.579896] initcall ibm_acpiphp_init+0x0/0x175 returned -19 after 93744718 usecs
>> ...
>> [ 1398.739321] calling acpi_pci_slot_init+0x0/0x20 @ 1
>> [ 1494.572598] initcall acpi_pci_slot_init+0x0/0x20 returned 0 after 93582352 usecs
>> [ 1494.580019] calling acpi_processor_init+0x0/0xcd @ 1
>> [ 1494.585119] ACPI: acpi_idle registered with cpuidle
>> [ 1494.622886] initcall acpi_processor_init+0x0/0xcd returned 0 after 36896 usecs
>> [ 1494.630135] calling acpi_container_init+0x0/0x4a @ 1
>> [ 1734.573253] initcall acpi_container_init+0x0/0x4a returned 0 after 234314615 usecs
>> [ 1734.580854] calling acpi_thermal_init+0x0/0x42 @ 1
>> [ 1734.586032] initcall acpi_thermal_init+0x0/0x42 returned 0 after 275 usecs
>> [ 1734.592926] calling acpi_memory_device_init+0x0/0x87 @ 1
>> [ 1974.573125] initcall acpi_memory_device_init+0x0/0x87 returned 0 after 234350484 usecs
>> [ 1974.581069] calling acpi_battery_init+0x0/0x16 @ 1
>> [ 1974.586009] initcall acpi_battery_init+0x0/0x16 returned 0 after 34 usecs
>> [ 1974.586112] calling 1_acpi_battery_init_async+0x0/0x20 @ 5
>> [ 1974.586636] initcall 1_acpi_battery_init_async+0x0/0x20 returned 0 after 505 usecs
>> [ 1974.605962] calling acpi_power_meter_init+0x0/0x32 @ 1
>> [ 1974.611505] initcall acpi_power_meter_init+0x0/0x32 returned 0 after 286 usecs
>> [ 1974.618748] calling acpi_hed_init+0x0/0x26 @ 1
>> [ 1974.623565] initcall acpi_hed_init+0x0/0x26 returned 0 after 255 usecs
>> [ 1974.630115] calling acpi_pad_init+0x0/0x26 @ 1
>> [ 1974.634932] initcall acpi_pad_init+0x0/0x26 returned 0 after 254 usecs
>> [ 1974.641475] calling erst_init+0x0/0x2bb @ 1
>> [ 1974.645956] ERST: Can not request iomem region <0x 3f-0x 3f> for ERST.
>> [ 2070.572123] initcall erst_init+0x0/0x2bb returned -5 after 93678128 usecs
>> ...
>> [ 2070.792625] calling serial8250_init+0x0/0x18e @ 1
>> [ 2070.797433] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
>> [ 2070.824642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
>> [ 2534.572086] initcall serial8250_init+0x0/0x18e returned 0 after 452905158 usecs
>> [ 2534.579451] calling serial8250_pnp_init+0x0/0x12 @ 1
>> [ 2662.572104] initcall serial8250_pnp_init+0x0/0x12 returned 0 after 124987929 usecs
>> [ 2662.579749] calling serial8250_pci_init+0x0/0x1b @ 1
>> [ 2662.585088] initcall serial8250_pci_init+0x0/0x1b returned 0 after 261 usecs
>>
>
> without this pull:
>
> [ 18.733956] calling pnpacpi_init+0x0/0x8c @ 1
> [ 18.738400] pnp: PnP ACPI init
> [ 18.741484] ACPI: bus type pnp registered
> [ 18.798240] pnp 00:00: [bus 00-7f]
> [ 18.801638] pnp 00:00: [io 0x0cf8-0x0cff]
> ...
> [ 19.302232] pnp 00:0c: Plug and Play ACPI device, IDs PNP0103 (active)
> [ 19.381017] pnp 00:0d: [bus 80-ff]
> [ 19.384410] pnp 00:0d: [io 0x0000 window]
> [ 19.388502] pnp 00:0d: [io 0xc000-0xffff window]
> [ 19.393205] pnp 00:0d: [mem 0x00000000 window]
> [ 19.397639] pnp 00:0d: [mem 0xe0000000-0xfbffffff window]
> [ 19.403031] pnp 00:0d: [mem 0x00000000 window]
> [ 19.407874] pnp 00:0d: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
> [ 19.415591] system 00:0e: Plug and Play ACPI device, IDs PNP0c01 (active)
> [ 19.422747] pnp: PnP ACPI: found 15 devices
> [ 19.426924] ACPI: ACPI bus type pnp unregistered
> [ 19.431538] initcall pnpacpi_init+0x0/0x8c returned 0 after 678214 usecs
>
e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Date: Tue Sep 7 10:38:22 2010 -0700
rcu: Decrease memory-barrier usage based on semi-formal proof
Commit d09b62d fixed grace-period synchronization, but left some smp_mb()
invocations in rcu_process_callbacks() that are no longer needed, but
sheer paranoia prevented them from being removed. This commit removes
them and provides a proof of correctness in their absence. It also adds
a memory barrier to rcu_report_qs_rsp() immediately before the update to
rsp->completed in order to handle the theoretical possibility that the
compiler or CPU might move massive quantities of code into a lock-based
critical section. This also proves that the sheer paranoia was not
entirely unjustified, at least from a theoretical point of view.
In addition, the old dyntick-idle synchronization depended on the fact
that grace periods were many milliseconds in duration, so that it could
be assumed that no dyntick-idle CPU could reorder a memory reference
across an entire grace period. Unfortunately for this design, the
addition of expedited grace periods breaks this assumption, which has
the unfortunate side-effect of requiring atomic operations in the
functions that track dyntick-idle state for RCU. (There is some hope
that the algorithms used in user-level RCU might be applied here, but
some work is required to handle the NMIs that user-space applications
can happily ignore. For the short term, better safe than sorry.)
This proof assumes that neither compiler nor CPU will allow a lock
acquisition and release to be reordered, as doing so can result in
deadlock. The proof is as follows:
1. A given CPU declares a quiescent state under the protection of
its leaf rcu_node's lock.
2. If there is more than one level of rcu_node hierarchy, the
last CPU to declare a quiescent state will also acquire the
->lock of the next rcu_node up in the hierarchy, but only
after releasing the lower level's lock. The acquisition of this
lock clearly cannot occur prior to the acquisition of the leaf
node's lock.
3. Step 2 repeats until we reach the root rcu_node structure.
Please note again that only one lock is held at a time through
this process. The acquisition of the root rcu_node's ->lock
must occur after the release of that of the leaf rcu_node.
4. At this point, we set the ->completed field in the rcu_state
structure in rcu_report_qs_rsp(). However, if the rcu_node
hierarchy contains only one rcu_node, then in theory the code
preceding the quiescent state could leak into the critical
section. We therefore precede the update of ->completed with a
memory barrier. All CPUs will therefore agree that any updates
preceding any report of a quiescent state will have happened
before the update of ->completed.
5. Regardless of whether a new grace period is needed, rcu_start_gp()
will propagate the new value of ->completed to all of the leaf
rcu_node structures, under the protection of each rcu_node's ->lock.
If a new grace period is needed immediately, this propagation
will occur in the same critical section that ->completed was
set in, but courtesy of the memory barrier in #4 above, is still
seen to follow any pre-quiescent-state activity.
6. When a given CPU invokes __rcu_process_gp_end(), it becomes
aware of the end of the old grace period and therefore makes
any RCU callbacks that were waiting on that grace period eligible
for invocation.
If this CPU is the same one that detected the end of the grace
period, and if there is but a single rcu_node in the hierarchy,
we will still be in the single critical section. In this case,
the memory barrier in step #4 guarantees that all callbacks will
be seen to execute after each CPU's quiescent state.
On the other hand, if this is a different CPU, it will acquire
the leaf rcu_node's ->lock, and will again be serialized after
each CPU's quiescent state for the old grace period.
On the strength of this proof, this commit therefore removes the memory
barriers from rcu_process_callbacks() and adds one to rcu_report_qs_rsp().
The effect is to reduce the number of memory barriers by one and to
reduce the frequency of execution from about once per scheduling tick
per CPU to once per grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
:040000 040000 cb1295bf2b408fc7a060a54985936376b646028d 1e70d95defc274757b58b80dfdcf681a1595a3ec M Documentation
:040000 040000 c326c2a8b90d257a47359c20f25cd9eac27b421d 79c31032c17e87226a23dc69ae0f8f6bc84fda5c M kernel
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-11 23:02 ` Yinghai Lu
@ 2011-05-12 6:03 ` Ingo Molnar
2011-05-12 7:27 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-12 6:03 UTC (permalink / raw)
To: Yinghai Lu; +Cc: paulmck, linux-kernel
* Yinghai Lu <yinghai@kernel.org> wrote:
> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Date: Tue Sep 7 10:38:22 2010 -0700
>
> rcu: Decrease memory-barrier usage based on semi-formal proof
Find below an (untested!) attempt at reverting it for debugging purposes: could
you please try it, does your system now boot up fine?
Thanks,
Ingo
----------------------------->
>From bce6fbbda040d4b0783cf59b5a9b7601bdf011bd Mon Sep 17 00:00:00 2001
From: Ingo Molnar <mingo@elte.hu>
Date: Thu, 12 May 2011 08:02:05 +0200
Subject: [PATCH] Revert "rcu: Decrease memory-barrier usage based on semi-formal proof"
This reverts commit e59fb3120becfb36b22ddb8bd27d065d3cdca499.
Conflicts:
Documentation/RCU/trace.txt
kernel/rcutree.c
---
Documentation/RCU/trace.txt | 78 ++++++++++++--------------
kernel/rcutree.c | 130 ++++++++++++++++++++++++------------------
kernel/rcutree.h | 9 ++-
kernel/rcutree_plugin.h | 7 +-
kernel/rcutree_trace.c | 12 ++--
5 files changed, 126 insertions(+), 110 deletions(-)
diff --git a/Documentation/RCU/trace.txt b/Documentation/RCU/trace.txt
index 8173cec..2b4d887 100644
--- a/Documentation/RCU/trace.txt
+++ b/Documentation/RCU/trace.txt
@@ -10,46 +10,34 @@ for rcutree and next for rcutiny.
CONFIG_TREE_RCU and CONFIG_TREE_PREEMPT_RCU debugfs Files and Formats
-These implementations of RCU provides several debugfs files under the
-top-level directory "rcu":
-
-rcu/rcudata:
- Displays fields in struct rcu_data.
-rcu/rcudata.csv:
- Comma-separated values spreadsheet version of rcudata.
-rcu/rcugp:
- Displays grace-period counters.
-rcu/rcuhier:
- Displays the struct rcu_node hierarchy.
-rcu/rcu_pending:
- Displays counts of the reasons rcu_pending() decided that RCU had
- work to do.
-rcu/rcutorture:
- Displays rcutorture test progress.
-rcu/rcuboost:
- Displays RCU boosting statistics. Only present if
- CONFIG_RCU_BOOST=y.
+These implementations of RCU provides five debugfs files under the
+top-level directory RCU: rcu/rcudata (which displays fields in struct
+rcu_data), rcu/rcudata.csv (which is a .csv spreadsheet version of
+rcu/rcudata), rcu/rcugp (which displays grace-period counters),
+rcu/rcuhier (which displays the struct rcu_node hierarchy), and
+rcu/rcu_pending (which displays counts of the reasons that the
+rcu_pending() function decided that there was core RCU work to do).
The output of "cat rcu/rcudata" looks as follows:
rcu_sched:
- 0 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=545/1/0 df=50 of=0 ri=0 ql=163 qs=NRW. kt=0/W/0 ktl=ebc3 b=10 ci=153737 co=0 ca=0
- 1 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=967/1/0 df=58 of=0 ri=0 ql=634 qs=NRW. kt=0/W/1 ktl=58c b=10 ci=191037 co=0 ca=0
- 2 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=1081/1/0 df=175 of=0 ri=0 ql=74 qs=N.W. kt=0/W/2 ktl=da94 b=10 ci=75991 co=0 ca=0
- 3 c=20942 g=20943 pq=1 pqc=20942 qp=1 dt=1846/0/0 df=404 of=0 ri=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=72261 co=0 ca=0
- 4 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=369/1/0 df=83 of=0 ri=0 ql=48 qs=N.W. kt=0/W/4 ktl=e0e7 b=10 ci=128365 co=0 ca=0
- 5 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=381/1/0 df=64 of=0 ri=0 ql=169 qs=NRW. kt=0/W/5 ktl=fb2f b=10 ci=164360 co=0 ca=0
- 6 c=20972 g=20973 pq=1 pqc=20972 qp=0 dt=1037/1/0 df=183 of=0 ri=0 ql=62 qs=N.W. kt=0/W/6 ktl=d2ad b=10 ci=65663 co=0 ca=0
- 7 c=20897 g=20897 pq=1 pqc=20896 qp=0 dt=1572/0/0 df=382 of=0 ri=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=75006 co=0 ca=0
+ 0 c=17829 g=17829 pq=1 pqc=17829 qp=0 dt=10951/1 dn=0 df=1101 of=0 ri=36 ql=0 b=10
+ 1 c=17829 g=17829 pq=1 pqc=17829 qp=0 dt=16117/1 dn=0 df=1015 of=0 ri=0 ql=0 b=10
+ 2 c=17829 g=17829 pq=1 pqc=17829 qp=0 dt=1445/1 dn=0 df=1839 of=0 ri=0 ql=0 b=10
+ 3 c=17829 g=17829 pq=1 pqc=17829 qp=0 dt=6681/1 dn=0 df=1545 of=0 ri=0 ql=0 b=10
+ 4 c=17829 g=17829 pq=1 pqc=17829 qp=0 dt=1003/1 dn=0 df=1992 of=0 ri=0 ql=0 b=10
+ 5 c=17829 g=17830 pq=1 pqc=17829 qp=1 dt=3887/1 dn=0 df=3331 of=0 ri=4 ql=2 b=10
+ 6 c=17829 g=17829 pq=1 pqc=17829 qp=0 dt=859/1 dn=0 df=3224 of=0 ri=0 ql=0 b=10
+ 7 c=17829 g=17830 pq=0 pqc=17829 qp=1 dt=3761/1 dn=0 df=1818 of=0 ri=0 ql=2 b=10
rcu_bh:
- 0 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=545/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/0 ktl=ebc3 b=10 ci=0 co=0 ca=0
- 1 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=967/1/0 df=3 of=0 ri=1 ql=0 qs=.... kt=0/W/1 ktl=58c b=10 ci=151 co=0 ca=0
- 2 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=1081/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/2 ktl=da94 b=10 ci=0 co=0 ca=0
- 3 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=1846/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=0 co=0 ca=0
- 4 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=369/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/4 ktl=e0e7 b=10 ci=0 co=0 ca=0
- 5 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=381/1/0 df=4 of=0 ri=1 ql=0 qs=.... kt=0/W/5 ktl=fb2f b=10 ci=0 co=0 ca=0
- 6 c=1480 g=1480 pq=1 pqc=1479 qp=0 dt=1037/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/6 ktl=d2ad b=10 ci=0 co=0 ca=0
- 7 c=1474 g=1474 pq=1 pqc=1473 qp=0 dt=1572/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=0 co=0 ca=0
+ 0 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=10951/1 dn=0 df=0 of=0 ri=0 ql=0 b=10
+ 1 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=16117/1 dn=0 df=13 of=0 ri=0 ql=0 b=10
+ 2 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=1445/1 dn=0 df=15 of=0 ri=0 ql=0 b=10
+ 3 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=6681/1 dn=0 df=9 of=0 ri=0 ql=0 b=10
+ 4 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=1003/1 dn=0 df=15 of=0 ri=0 ql=0 b=10
+ 5 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=3887/1 dn=0 df=15 of=0 ri=0 ql=0 b=10
+ 6 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=859/1 dn=0 df=15 of=0 ri=0 ql=0 b=10
+ 7 c=-275 g=-275 pq=1 pqc=-275 qp=0 dt=3761/1 dn=0 df=15 of=0 ri=0 ql=0 b=10
The first section lists the rcu_data structures for rcu_sched, the second
for rcu_bh. Note that CONFIG_TREE_PREEMPT_RCU kernels will have an
@@ -94,16 +82,22 @@ o "pqc" indicates which grace period the last-observed quiescent
the next grace period!
o "qp" indicates that RCU still expects a quiescent state from
- this CPU. Offlined CPUs and CPUs in dyntick idle mode might
- well have qp=1, which is OK: RCU is still ignoring them.
+ this CPU.
o "dt" is the current value of the dyntick counter that is incremented
when entering or leaving dynticks idle state, either by the
- scheduler or by irq. This number is even if the CPU is in
- dyntick idle mode and odd otherwise. The number after the first
- "/" is the interrupt nesting depth when in dyntick-idle state,
- or one greater than the interrupt-nesting depth otherwise.
- The number after the second "/" is the NMI nesting depth.
+ scheduler or by irq. The number after the "/" is the interrupt
+ nesting depth when in dyntick-idle state, or one greater than
+ the interrupt-nesting depth otherwise.
+
+ This field is displayed only for CONFIG_NO_HZ kernels.
+
+o "dn" is the current value of the dyntick counter that is incremented
+ when entering or leaving dynticks idle state via NMI. If both
+ the "dt" and "dn" values are even, then this CPU is in dynticks
+ idle mode and may be ignored by RCU. If either of these two
+ counters is odd, then RCU must be alert to the possibility of
+ an RCU read-side critical section running on this CPU.
This field is displayed only for CONFIG_NO_HZ kernels.
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 5616b17..e486f7c 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -162,7 +162,7 @@ EXPORT_SYMBOL_GPL(rcu_note_context_switch);
#ifdef CONFIG_NO_HZ
DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
.dynticks_nesting = 1,
- .dynticks = ATOMIC_INIT(1),
+ .dynticks = 1,
};
#endif /* #ifdef CONFIG_NO_HZ */
@@ -321,25 +321,13 @@ void rcu_enter_nohz(void)
unsigned long flags;
struct rcu_dynticks *rdtp;
+ smp_mb(); /* CPUs seeing ++ must see prior RCU read-side crit sects */
local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
- if (--rdtp->dynticks_nesting) {
- local_irq_restore(flags);
- return;
- }
- /* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
- smp_mb__before_atomic_inc(); /* See above. */
- atomic_inc(&rdtp->dynticks);
- smp_mb__after_atomic_inc(); /* Force ordering with next sojourn. */
- WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
+ rdtp->dynticks++;
+ rdtp->dynticks_nesting--;
+ WARN_ON_ONCE(rdtp->dynticks & 0x1);
local_irq_restore(flags);
-
- /* If the interrupt queued a callback, get out of dyntick mode. */
- if (in_irq() &&
- (__get_cpu_var(rcu_sched_data).nxtlist ||
- __get_cpu_var(rcu_bh_data).nxtlist ||
- rcu_preempt_needs_cpu(smp_processor_id())))
- set_need_resched();
}
/*
@@ -355,16 +343,11 @@ void rcu_exit_nohz(void)
local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
- if (rdtp->dynticks_nesting++) {
- local_irq_restore(flags);
- return;
- }
- smp_mb__before_atomic_inc(); /* Force ordering w/previous sojourn. */
- atomic_inc(&rdtp->dynticks);
- /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
- smp_mb__after_atomic_inc(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ rdtp->dynticks++;
+ rdtp->dynticks_nesting++;
+ WARN_ON_ONCE(!(rdtp->dynticks & 0x1));
local_irq_restore(flags);
+ smp_mb(); /* CPUs seeing ++ must see later RCU read-side crit sects */
}
/**
@@ -378,15 +361,11 @@ void rcu_nmi_enter(void)
{
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
- if (rdtp->dynticks_nmi_nesting == 0 &&
- (atomic_read(&rdtp->dynticks) & 0x1))
+ if (rdtp->dynticks & 0x1)
return;
- rdtp->dynticks_nmi_nesting++;
- smp_mb__before_atomic_inc(); /* Force delay from prior write. */
- atomic_inc(&rdtp->dynticks);
- /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
- smp_mb__after_atomic_inc(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ rdtp->dynticks_nmi++;
+ WARN_ON_ONCE(!(rdtp->dynticks_nmi & 0x1));
+ smp_mb(); /* CPUs seeing ++ must see later RCU read-side crit sects */
}
/**
@@ -400,14 +379,11 @@ void rcu_nmi_exit(void)
{
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
- if (rdtp->dynticks_nmi_nesting == 0 ||
- --rdtp->dynticks_nmi_nesting != 0)
+ if (rdtp->dynticks & 0x1)
return;
- /* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
- smp_mb__before_atomic_inc(); /* See above. */
- atomic_inc(&rdtp->dynticks);
- smp_mb__after_atomic_inc(); /* Force delay to next write. */
- WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
+ smp_mb(); /* CPUs seeing ++ must see prior RCU read-side crit sects */
+ rdtp->dynticks_nmi++;
+ WARN_ON_ONCE(rdtp->dynticks_nmi & 0x1);
}
/**
@@ -418,7 +394,13 @@ void rcu_nmi_exit(void)
*/
void rcu_irq_enter(void)
{
- rcu_exit_nohz();
+ struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
+
+ if (rdtp->dynticks_nesting++)
+ return;
+ rdtp->dynticks++;
+ WARN_ON_ONCE(!(rdtp->dynticks & 0x1));
+ smp_mb(); /* CPUs seeing ++ must see later RCU read-side crit sects */
}
/**
@@ -430,7 +412,18 @@ void rcu_irq_enter(void)
*/
void rcu_irq_exit(void)
{
- rcu_enter_nohz();
+ struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
+
+ if (--rdtp->dynticks_nesting)
+ return;
+ smp_mb(); /* CPUs seeing ++ must see prior RCU read-side crit sects */
+ rdtp->dynticks++;
+ WARN_ON_ONCE(rdtp->dynticks & 0x1);
+
+ /* If the interrupt queued a callback, get out of dyntick mode. */
+ if (__this_cpu_read(rcu_sched_data.nxtlist) ||
+ __this_cpu_read(rcu_bh_data.nxtlist))
+ set_need_resched();
}
#ifdef CONFIG_SMP
@@ -442,8 +435,19 @@ void rcu_irq_exit(void)
*/
static int dyntick_save_progress_counter(struct rcu_data *rdp)
{
- rdp->dynticks_snap = atomic_add_return(0, &rdp->dynticks->dynticks);
- return 0;
+ int ret;
+ int snap;
+ int snap_nmi;
+
+ snap = rdp->dynticks->dynticks;
+ snap_nmi = rdp->dynticks->dynticks_nmi;
+ smp_mb(); /* Order sampling of snap with end of grace period. */
+ rdp->dynticks_snap = snap;
+ rdp->dynticks_nmi_snap = snap_nmi;
+ ret = ((snap & 0x1) == 0) && ((snap_nmi & 0x1) == 0);
+ if (ret)
+ rdp->dynticks_fqs++;
+ return ret;
}
/*
@@ -454,11 +458,16 @@ static int dyntick_save_progress_counter(struct rcu_data *rdp)
*/
static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
{
- unsigned long curr;
- unsigned long snap;
+ long curr;
+ long curr_nmi;
+ long snap;
+ long snap_nmi;
- curr = (unsigned long)atomic_add_return(0, &rdp->dynticks->dynticks);
- snap = (unsigned long)rdp->dynticks_snap;
+ curr = rdp->dynticks->dynticks;
+ snap = rdp->dynticks_snap;
+ curr_nmi = rdp->dynticks->dynticks_nmi;
+ snap_nmi = rdp->dynticks_nmi_snap;
+ smp_mb(); /* force ordering with cpu entering/leaving dynticks. */
/*
* If the CPU passed through or entered a dynticks idle phase with
@@ -468,7 +477,8 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
* read-side critical section that started before the beginning
* of the current RCU grace period.
*/
- if ((curr & 0x1) == 0 || ULONG_CMP_GE(curr, snap + 2)) {
+ if ((curr != snap || (curr & 0x1) == 0) &&
+ (curr_nmi != snap_nmi || (curr_nmi & 0x1) == 0)) {
rdp->dynticks_fqs++;
return 1;
}
@@ -897,12 +907,6 @@ static void rcu_report_qs_rsp(struct rcu_state *rsp, unsigned long flags)
unsigned long gp_duration;
WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
-
- /*
- * Ensure that all grace-period and pre-grace-period activity
- * is seen before the assignment to rsp->completed.
- */
- smp_mb(); /* See above block comment. */
gp_duration = jiffies - rsp->gp_start;
if (gp_duration > rsp->gp_max)
rsp->gp_max = gp_duration;
@@ -1450,11 +1454,25 @@ __rcu_process_callbacks(struct rcu_state *rsp, struct rcu_data *rdp)
*/
static void rcu_process_callbacks(void)
{
+ /*
+ * Memory references from any prior RCU read-side critical sections
+ * executed by the interrupted code must be seen before any RCU
+ * grace-period manipulations below.
+ */
+ smp_mb(); /* See above block comment. */
+
__rcu_process_callbacks(&rcu_sched_state,
&__get_cpu_var(rcu_sched_data));
__rcu_process_callbacks(&rcu_bh_state, &__get_cpu_var(rcu_bh_data));
rcu_preempt_process_callbacks();
+ /*
+ * Memory references from any later RCU read-side critical sections
+ * executed by the interrupted code must be seen after any RCU
+ * grace-period manipulations above.
+ */
+ smp_mb(); /* See above block comment. */
+
/* If we are last CPU on way to dyntick-idle mode, accelerate it. */
rcu_needs_cpu_flush();
}
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index 93d4a1c..2576648 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -84,9 +84,11 @@
* Dynticks per-CPU state.
*/
struct rcu_dynticks {
- int dynticks_nesting; /* Track irq/process nesting level. */
- int dynticks_nmi_nesting; /* Track NMI nesting level. */
- atomic_t dynticks; /* Even value for dynticks-idle, else odd. */
+ int dynticks_nesting; /* Track nesting level, sort of. */
+ int dynticks; /* Even value for dynticks-idle, else odd. */
+ int dynticks_nmi; /* Even value for either dynticks-idle or */
+ /* not in nmi handler, else odd. So this */
+ /* remains even for nmi from irq handler. */
};
/* RCU's kthread states for tracing. */
@@ -282,6 +284,7 @@ struct rcu_data {
/* 3) dynticks interface. */
struct rcu_dynticks *dynticks; /* Shared per-CPU dynticks state. */
int dynticks_snap; /* Per-GP tracking for dynticks. */
+ int dynticks_nmi_snap; /* Per-GP tracking for dynticks_nmi. */
#endif /* #ifdef CONFIG_NO_HZ */
/* 4) reasons this CPU needed to be kicked by force_quiescent_state */
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index ed339702..3f6559a 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -1520,6 +1520,7 @@ int rcu_needs_cpu(int cpu)
{
int c = 0;
int snap;
+ int snap_nmi;
int thatcpu;
/* Check for being in the holdoff period. */
@@ -1530,10 +1531,10 @@ int rcu_needs_cpu(int cpu)
for_each_online_cpu(thatcpu) {
if (thatcpu == cpu)
continue;
- snap = atomic_add_return(0, &per_cpu(rcu_dynticks,
- thatcpu).dynticks);
+ snap = per_cpu(rcu_dynticks, thatcpu).dynticks;
+ snap_nmi = per_cpu(rcu_dynticks, thatcpu).dynticks_nmi;
smp_mb(); /* Order sampling of snap with end of grace period. */
- if ((snap & 0x1) != 0) {
+ if (((snap & 0x1) != 0) || ((snap_nmi & 0x1) != 0)) {
per_cpu(rcu_dyntick_drain, cpu) = 0;
per_cpu(rcu_dyntick_holdoff, cpu) = jiffies - 1;
return rcu_needs_cpu_quick_check(cpu);
diff --git a/kernel/rcutree_trace.c b/kernel/rcutree_trace.c
index 9678cc3..aa0fd72 100644
--- a/kernel/rcutree_trace.c
+++ b/kernel/rcutree_trace.c
@@ -69,10 +69,10 @@ static void print_one_rcu_data(struct seq_file *m, struct rcu_data *rdp)
rdp->passed_quiesc, rdp->passed_quiesc_completed,
rdp->qs_pending);
#ifdef CONFIG_NO_HZ
- seq_printf(m, " dt=%d/%d/%d df=%lu",
- atomic_read(&rdp->dynticks->dynticks),
+ seq_printf(m, " dt=%d/%d dn=%d df=%lu",
+ rdp->dynticks->dynticks,
rdp->dynticks->dynticks_nesting,
- rdp->dynticks->dynticks_nmi_nesting,
+ rdp->dynticks->dynticks_nmi,
rdp->dynticks_fqs);
#endif /* #ifdef CONFIG_NO_HZ */
seq_printf(m, " of=%lu ri=%lu", rdp->offline_fqs, rdp->resched_ipi);
@@ -141,9 +141,9 @@ static void print_one_rcu_data_csv(struct seq_file *m, struct rcu_data *rdp)
rdp->qs_pending);
#ifdef CONFIG_NO_HZ
seq_printf(m, ",%d,%d,%d,%lu",
- atomic_read(&rdp->dynticks->dynticks),
+ rdp->dynticks->dynticks,
rdp->dynticks->dynticks_nesting,
- rdp->dynticks->dynticks_nmi_nesting,
+ rdp->dynticks->dynticks_nmi,
rdp->dynticks_fqs);
#endif /* #ifdef CONFIG_NO_HZ */
seq_printf(m, ",%lu,%lu", rdp->offline_fqs, rdp->resched_ipi);
@@ -167,7 +167,7 @@ static int show_rcudata_csv(struct seq_file *m, void *unused)
{
seq_puts(m, "\"CPU\",\"Online?\",\"c\",\"g\",\"pq\",\"pqc\",\"pq\",");
#ifdef CONFIG_NO_HZ
- seq_puts(m, "\"dt\",\"dt nesting\",\"dt NMI nesting\",\"df\",");
+ seq_puts(m, "\"dt\",\"dt nesting\",\"dn\",\"df\",");
#endif /* #ifdef CONFIG_NO_HZ */
seq_puts(m, "\"of\",\"ri\",\"ql\",\"b\",\"ci\",\"co\",\"ca\"\n");
#ifdef CONFIG_TREE_PREEMPT_RCU
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 6:03 ` Ingo Molnar
@ 2011-05-12 7:27 ` Yinghai Lu
2011-05-12 7:42 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-12 7:27 UTC (permalink / raw)
To: Ingo Molnar; +Cc: paulmck, linux-kernel
On 05/11/2011 11:03 PM, Ingo Molnar wrote:
>
> * Yinghai Lu <yinghai@kernel.org> wrote:
>
>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>> Date: Tue Sep 7 10:38:22 2010 -0700
>>
>> rcu: Decrease memory-barrier usage based on semi-formal proof
>
> Find below an (untested!) attempt at reverting it for debugging purposes: could
> you please try it, does your system now boot up fine?
>
> Thanks,
>
> Ingo
>
yes, reverted manually that commit fix the problem.
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 5616b17..ed59987 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -162,7 +162,7 @@ EXPORT_SYMBOL_GPL(rcu_note_context_switch);
#ifdef CONFIG_NO_HZ
DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
.dynticks_nesting = 1,
- .dynticks = ATOMIC_INIT(1),
+ .dynticks = 1,
};
#endif /* #ifdef CONFIG_NO_HZ */
@@ -321,25 +321,13 @@ void rcu_enter_nohz(void)
unsigned long flags;
struct rcu_dynticks *rdtp;
+ smp_mb(); /* CPUs seeing ++ must see prior RCU read-side crit sects */
local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
- if (--rdtp->dynticks_nesting) {
- local_irq_restore(flags);
- return;
- }
- /* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
- smp_mb__before_atomic_inc(); /* See above. */
- atomic_inc(&rdtp->dynticks);
- smp_mb__after_atomic_inc(); /* Force ordering with next sojourn. */
- WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
+ rdtp->dynticks++;
+ rdtp->dynticks_nesting--;
+ WARN_ON_ONCE(rdtp->dynticks & 0x1);
local_irq_restore(flags);
-
- /* If the interrupt queued a callback, get out of dyntick mode. */
- if (in_irq() &&
- (__get_cpu_var(rcu_sched_data).nxtlist ||
- __get_cpu_var(rcu_bh_data).nxtlist ||
- rcu_preempt_needs_cpu(smp_processor_id())))
- set_need_resched();
}
/*
@@ -355,16 +343,11 @@ void rcu_exit_nohz(void)
local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
- if (rdtp->dynticks_nesting++) {
- local_irq_restore(flags);
- return;
- }
- smp_mb__before_atomic_inc(); /* Force ordering w/previous sojourn. */
- atomic_inc(&rdtp->dynticks);
- /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
- smp_mb__after_atomic_inc(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ rdtp->dynticks++;
+ rdtp->dynticks_nesting++;
+ WARN_ON_ONCE(!(rdtp->dynticks & 0x1));
local_irq_restore(flags);
+ smp_mb(); /* CPUs seeing ++ must see later RCU read-side crit sects */
}
/**
@@ -378,15 +361,11 @@ void rcu_nmi_enter(void)
{
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
- if (rdtp->dynticks_nmi_nesting == 0 &&
- (atomic_read(&rdtp->dynticks) & 0x1))
+ if (rdtp->dynticks & 0x1)
return;
- rdtp->dynticks_nmi_nesting++;
- smp_mb__before_atomic_inc(); /* Force delay from prior write. */
- atomic_inc(&rdtp->dynticks);
- /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
- smp_mb__after_atomic_inc(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ rdtp->dynticks_nmi++;
+ WARN_ON_ONCE(!(rdtp->dynticks_nmi & 0x1));
+ smp_mb(); /* CPUs seeing ++ must see later RCU read-side crit sects */
}
/**
@@ -400,14 +379,11 @@ void rcu_nmi_exit(void)
{
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
- if (rdtp->dynticks_nmi_nesting == 0 ||
- --rdtp->dynticks_nmi_nesting != 0)
+ if (rdtp->dynticks & 0x1)
return;
- /* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
- smp_mb__before_atomic_inc(); /* See above. */
- atomic_inc(&rdtp->dynticks);
- smp_mb__after_atomic_inc(); /* Force delay to next write. */
- WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
+ smp_mb(); /* CPUs seeing ++ must see prior RCU read-side crit sects */
+ rdtp->dynticks_nmi++;
+ WARN_ON_ONCE(rdtp->dynticks_nmi & 0x1);
}
/**
@@ -418,7 +394,13 @@ void rcu_nmi_exit(void)
*/
void rcu_irq_enter(void)
{
- rcu_exit_nohz();
+ struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
+
+ if (rdtp->dynticks_nesting++)
+ return;
+ rdtp->dynticks++;
+ WARN_ON_ONCE(!(rdtp->dynticks & 0x1));
+ smp_mb(); /* CPUs seeing ++ must see later RCU read-side crit sects */
}
/**
@@ -430,7 +412,18 @@ void rcu_irq_enter(void)
*/
void rcu_irq_exit(void)
{
- rcu_enter_nohz();
+ struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
+
+ if (--rdtp->dynticks_nesting)
+ return;
+ smp_mb(); /* CPUs seeing ++ must see prior RCU read-side crit sects */
+ rdtp->dynticks++;
+ WARN_ON_ONCE(rdtp->dynticks & 0x1);
+
+ /* If the interrupt queued a callback, get out of dyntick mode. */
+ if (__this_cpu_read(rcu_sched_data.nxtlist) ||
+ __this_cpu_read(rcu_bh_data.nxtlist))
+ set_need_resched();
}
#ifdef CONFIG_SMP
@@ -442,8 +435,19 @@ void rcu_irq_exit(void)
*/
static int dyntick_save_progress_counter(struct rcu_data *rdp)
{
- rdp->dynticks_snap = atomic_add_return(0, &rdp->dynticks->dynticks);
- return 0;
+ int ret;
+ int snap;
+ int snap_nmi;
+
+ snap = rdp->dynticks->dynticks;
+ snap_nmi = rdp->dynticks->dynticks_nmi;
+ smp_mb(); /* Order sampling of snap with end of grace period. */
+ rdp->dynticks_snap = snap;
+ rdp->dynticks_nmi_snap = snap_nmi;
+ ret = ((snap & 0x1) == 0) && ((snap_nmi & 0x1) == 0);
+ if (ret)
+ rdp->dynticks_fqs++;
+ return ret;
}
/*
@@ -454,11 +458,16 @@ static int dyntick_save_progress_counter(struct rcu_data *rdp)
*/
static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
{
- unsigned long curr;
- unsigned long snap;
+ long curr;
+ long curr_nmi;
+ long snap;
+ long snap_nmi;
- curr = (unsigned long)atomic_add_return(0, &rdp->dynticks->dynticks);
- snap = (unsigned long)rdp->dynticks_snap;
+ curr = rdp->dynticks->dynticks;
+ snap = rdp->dynticks_snap;
+ curr_nmi = rdp->dynticks->dynticks_nmi;
+ snap_nmi = rdp->dynticks_nmi_snap;
+ smp_mb(); /* force ordering with cpu entering/leaving dynticks. */
/*
* If the CPU passed through or entered a dynticks idle phase with
@@ -468,7 +477,8 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
* read-side critical section that started before the beginning
* of the current RCU grace period.
*/
- if ((curr & 0x1) == 0 || ULONG_CMP_GE(curr, snap + 2)) {
+ if ((curr != snap || (curr & 0x1) == 0) &&
+ (curr_nmi != snap_nmi || (curr_nmi & 0x1) == 0)) {
rdp->dynticks_fqs++;
return 1;
}
@@ -898,11 +908,6 @@ static void rcu_report_qs_rsp(struct rcu_state *rsp, unsigned long flags)
WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
- /*
- * Ensure that all grace-period and pre-grace-period activity
- * is seen before the assignment to rsp->completed.
- */
- smp_mb(); /* See above block comment. */
gp_duration = jiffies - rsp->gp_start;
if (gp_duration > rsp->gp_max)
rsp->gp_max = gp_duration;
@@ -1450,11 +1455,25 @@ __rcu_process_callbacks(struct rcu_state *rsp, struct rcu_data *rdp)
*/
static void rcu_process_callbacks(void)
{
+ /*
+ * Memory references from any prior RCU read-side critical sections
+ * executed by the interrupted code must be seen before any RCU
+ * grace-period manipulations below.
+ */
+ smp_mb(); /* See above block comment. */
+
__rcu_process_callbacks(&rcu_sched_state,
&__get_cpu_var(rcu_sched_data));
__rcu_process_callbacks(&rcu_bh_state, &__get_cpu_var(rcu_bh_data));
rcu_preempt_process_callbacks();
+ /*
+ * Memory references from any later RCU read-side critical sections
+ * executed by the interrupted code must be seen after any RCU
+ * grace-period manipulations above.
+ */
+ smp_mb(); /* See above block comment. */
+
/* If we are last CPU on way to dyntick-idle mode, accelerate it. */
rcu_needs_cpu_flush();
}
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index 93d4a1c..2576648 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -84,9 +84,11 @@
* Dynticks per-CPU state.
*/
struct rcu_dynticks {
- int dynticks_nesting; /* Track irq/process nesting level. */
- int dynticks_nmi_nesting; /* Track NMI nesting level. */
- atomic_t dynticks; /* Even value for dynticks-idle, else odd. */
+ int dynticks_nesting; /* Track nesting level, sort of. */
+ int dynticks; /* Even value for dynticks-idle, else odd. */
+ int dynticks_nmi; /* Even value for either dynticks-idle or */
+ /* not in nmi handler, else odd. So this */
+ /* remains even for nmi from irq handler. */
};
/* RCU's kthread states for tracing. */
@@ -282,6 +284,7 @@ struct rcu_data {
/* 3) dynticks interface. */
struct rcu_dynticks *dynticks; /* Shared per-CPU dynticks state. */
int dynticks_snap; /* Per-GP tracking for dynticks. */
+ int dynticks_nmi_snap; /* Per-GP tracking for dynticks_nmi. */
#endif /* #ifdef CONFIG_NO_HZ */
/* 4) reasons this CPU needed to be kicked by force_quiescent_state */
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index ed33970..3f6559a 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -1520,6 +1520,7 @@ int rcu_needs_cpu(int cpu)
{
int c = 0;
int snap;
+ int snap_nmi;
int thatcpu;
/* Check for being in the holdoff period. */
@@ -1530,10 +1531,10 @@ int rcu_needs_cpu(int cpu)
for_each_online_cpu(thatcpu) {
if (thatcpu == cpu)
continue;
- snap = atomic_add_return(0, &per_cpu(rcu_dynticks,
- thatcpu).dynticks);
+ snap = per_cpu(rcu_dynticks, thatcpu).dynticks;
+ snap_nmi = per_cpu(rcu_dynticks, thatcpu).dynticks_nmi;
smp_mb(); /* Order sampling of snap with end of grace period. */
- if ((snap & 0x1) != 0) {
+ if (((snap & 0x1) != 0) || ((snap_nmi & 0x1) != 0)) {
per_cpu(rcu_dyntick_drain, cpu) = 0;
per_cpu(rcu_dyntick_holdoff, cpu) = jiffies - 1;
return rcu_needs_cpu_quick_check(cpu);
diff --git a/kernel/rcutree_trace.c b/kernel/rcutree_trace.c
index 9678cc3..aa0fd72 100644
--- a/kernel/rcutree_trace.c
+++ b/kernel/rcutree_trace.c
@@ -69,10 +69,10 @@ static void print_one_rcu_data(struct seq_file *m, struct rcu_data *rdp)
rdp->passed_quiesc, rdp->passed_quiesc_completed,
rdp->qs_pending);
#ifdef CONFIG_NO_HZ
- seq_printf(m, " dt=%d/%d/%d df=%lu",
- atomic_read(&rdp->dynticks->dynticks),
+ seq_printf(m, " dt=%d/%d dn=%d df=%lu",
+ rdp->dynticks->dynticks,
rdp->dynticks->dynticks_nesting,
- rdp->dynticks->dynticks_nmi_nesting,
+ rdp->dynticks->dynticks_nmi,
rdp->dynticks_fqs);
#endif /* #ifdef CONFIG_NO_HZ */
seq_printf(m, " of=%lu ri=%lu", rdp->offline_fqs, rdp->resched_ipi);
@@ -141,9 +141,9 @@ static void print_one_rcu_data_csv(struct seq_file *m, struct rcu_data *rdp)
rdp->qs_pending);
#ifdef CONFIG_NO_HZ
seq_printf(m, ",%d,%d,%d,%lu",
- atomic_read(&rdp->dynticks->dynticks),
+ rdp->dynticks->dynticks,
rdp->dynticks->dynticks_nesting,
- rdp->dynticks->dynticks_nmi_nesting,
+ rdp->dynticks->dynticks_nmi,
rdp->dynticks_fqs);
#endif /* #ifdef CONFIG_NO_HZ */
seq_printf(m, ",%lu,%lu", rdp->offline_fqs, rdp->resched_ipi);
@@ -167,7 +167,7 @@ static int show_rcudata_csv(struct seq_file *m, void *unused)
{
seq_puts(m, "\"CPU\",\"Online?\",\"c\",\"g\",\"pq\",\"pqc\",\"pq\",");
#ifdef CONFIG_NO_HZ
- seq_puts(m, "\"dt\",\"dt nesting\",\"dt NMI nesting\",\"df\",");
+ seq_puts(m, "\"dt\",\"dt nesting\",\"dn\",\"df\",");
#endif /* #ifdef CONFIG_NO_HZ */
seq_puts(m, "\"of\",\"ri\",\"ql\",\"b\",\"ci\",\"co\",\"ca\"\n");
#ifdef CONFIG_TREE_PREEMPT_RCU
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 7:27 ` Yinghai Lu
@ 2011-05-12 7:42 ` Yinghai Lu
2011-05-12 9:20 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-12 7:42 UTC (permalink / raw)
To: Ingo Molnar; +Cc: paulmck, linux-kernel
On 05/12/2011 12:27 AM, Yinghai Lu wrote:
> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
>>
>> * Yinghai Lu <yinghai@kernel.org> wrote:
>>
>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>>> Date: Tue Sep 7 10:38:22 2010 -0700
>>>
>>> rcu: Decrease memory-barrier usage based on semi-formal proof
>>
>> Find below an (untested!) attempt at reverting it for debugging purposes: could
>> you please try it, does your system now boot up fine?
>>
>> Thanks,
>>
>> Ingo
>>
>
> yes, reverted manually that commit fix the problem.
on system with 8 sockets westmere-ex
it seems other commits after that commit contribute some delay too.
[ 32.240739] cpu_dev_init done
[ 73.587288] memory_dev_init done
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 7:42 ` Yinghai Lu
@ 2011-05-12 9:20 ` Paul E. McKenney
2011-05-12 17:31 ` Yinghai Lu
2011-05-12 21:36 ` Yinghai Lu
0 siblings, 2 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-12 9:20 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
> > On 05/11/2011 11:03 PM, Ingo Molnar wrote:
> >>
> >> * Yinghai Lu <yinghai@kernel.org> wrote:
> >>
> >>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
> >>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
> >>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> >>> Date: Tue Sep 7 10:38:22 2010 -0700
> >>>
> >>> rcu: Decrease memory-barrier usage based on semi-formal proof
> >>
> >> Find below an (untested!) attempt at reverting it for debugging purposes: could
> >> you please try it, does your system now boot up fine?
> >>
> >> Thanks,
> >>
> >> Ingo
> >>
> >
> > yes, reverted manually that commit fix the problem.
>
> on system with 8 sockets westmere-ex
>
> it seems other commits after that commit contribute some delay too.
>
> [ 32.240739] cpu_dev_init done
> [ 73.587288] memory_dev_init done
I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
will chase down the delay.
Could you please send along your .config file?
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 9:20 ` Paul E. McKenney
@ 2011-05-12 17:31 ` Yinghai Lu
2011-05-12 21:36 ` Yinghai Lu
1 sibling, 0 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-12 17:31 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 1216 bytes --]
On 05/12/2011 02:20 AM, Paul E. McKenney wrote:
> On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
>> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
>>> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
>>>>
>>>> * Yinghai Lu <yinghai@kernel.org> wrote:
>>>>
>>>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
>>>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
>>>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>>>>> Date: Tue Sep 7 10:38:22 2010 -0700
>>>>>
>>>>> rcu: Decrease memory-barrier usage based on semi-formal proof
>>>>
>>>> Find below an (untested!) attempt at reverting it for debugging purposes: could
>>>> you please try it, does your system now boot up fine?
>>>>
>>>> Thanks,
>>>>
>>>> Ingo
>>>>
>>>
>>> yes, reverted manually that commit fix the problem.
>>
>> on system with 8 sockets westmere-ex
>>
>> it seems other commits after that commit contribute some delay too.
>>
>> [ 32.240739] cpu_dev_init done
>> [ 73.587288] memory_dev_init done
>
> I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
> will chase down the delay.
>
> Could you please send along your .config file?
>
please check attached config.
Thanks
Yinghai
[-- Attachment #2: config.full.999996 --]
[-- Type: text/plain, Size: 86036 bytes --]
#
# Automatically generated make config: don't edit
# Linux/x86_64 2.6.39-rc7 Kernel Configuration
# Tue May 10 11:55:06 2011
#
CONFIG_64BIT=y
# CONFIG_X86_32 is not set
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_GPIO=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_CPUMASK_OF_CPU_MAP=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_HAVE_INTEL_TXT=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-rdi -fcall-saved-rsi -fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9 -fcall-saved-r10 -fcall-saved-r11"
# CONFIG_KTIME_SCALAR is not set
CONFIG_ARCH_CPU_PROBE_RELEASE=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_IRQ_WORK=y
CONFIG_IRQ_WORK=y
#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION="-yh"
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
CONFIG_KERNEL_XZ=y
# CONFIG_KERNEL_LZO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
# CONFIG_FHANDLE is not set
CONFIG_TASKSTATS=y
# CONFIG_TASK_DELAY_ACCT is not set
# CONFIG_TASK_XACCT is not set
# CONFIG_AUDIT is not set
CONFIG_HAVE_GENERIC_HARDIRQS=y
#
# IRQ subsystem
#
CONFIG_GENERIC_HARDIRQS=y
CONFIG_HAVE_SPARSE_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
# CONFIG_PREEMPT_RCU is not set
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_FANOUT=64
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_RCU_FAST_NO_HZ is not set
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=21
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
CONFIG_CGROUP_NS=y
# CONFIG_CGROUP_FREEZER is not set
# CONFIG_CGROUP_DEVICE is not set
CONFIG_CPUSETS=y
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_RESOURCE_COUNTERS=y
# CONFIG_CGROUP_MEM_RES_CTLR is not set
# CONFIG_CGROUP_PERF is not set
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
# CONFIG_BLK_CGROUP is not set
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
# CONFIG_SCHED_AUTOGROUP is not set
CONFIG_SCHED_TTWU_QUEUE=y
# CONFIG_SYSFS_DEPRECATED is not set
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
# CONFIG_EXPERT is not set
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y
#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
CONFIG_PERF_COUNTERS=y
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
CONFIG_SLUB_DEBUG=y
CONFIG_COMPAT_BRK=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_PROFILING is not set
CONFIG_TRACEPOINTS=y
CONFIG_HAVE_OPROFILE=y
CONFIG_KPROBES=y
# CONFIG_JUMP_LABEL is not set
CONFIG_OPTPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_KRETPROBES=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
#
# GCOV-based kernel profiling
#
# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_BLK_DEV_BSG=y
CONFIG_BLK_DEV_INTEGRITY=y
CONFIG_BLOCK_COMPAT=y
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_INLINE_SPIN_TRYLOCK is not set
# CONFIG_INLINE_SPIN_TRYLOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK is not set
# CONFIG_INLINE_SPIN_LOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK_IRQ is not set
# CONFIG_INLINE_SPIN_LOCK_IRQSAVE is not set
# CONFIG_INLINE_SPIN_UNLOCK is not set
# CONFIG_INLINE_SPIN_UNLOCK_BH is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQ is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_READ_TRYLOCK is not set
# CONFIG_INLINE_READ_LOCK is not set
# CONFIG_INLINE_READ_LOCK_BH is not set
# CONFIG_INLINE_READ_LOCK_IRQ is not set
# CONFIG_INLINE_READ_LOCK_IRQSAVE is not set
# CONFIG_INLINE_READ_UNLOCK is not set
# CONFIG_INLINE_READ_UNLOCK_BH is not set
# CONFIG_INLINE_READ_UNLOCK_IRQ is not set
# CONFIG_INLINE_READ_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_WRITE_TRYLOCK is not set
# CONFIG_INLINE_WRITE_LOCK is not set
# CONFIG_INLINE_WRITE_LOCK_BH is not set
# CONFIG_INLINE_WRITE_LOCK_IRQ is not set
# CONFIG_INLINE_WRITE_LOCK_IRQSAVE is not set
# CONFIG_INLINE_WRITE_UNLOCK is not set
# CONFIG_INLINE_WRITE_UNLOCK_BH is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQ is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE is not set
# CONFIG_MUTEX_SPIN_ON_OWNER is not set
CONFIG_FREEZER=y
#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_X86_X2APIC=y
CONFIG_X86_MPPARSE=y
# CONFIG_X86_EXTENDED_PLATFORM is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
CONFIG_NO_BOOTMEM=y
CONFIG_MEMTEST=y
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=7
CONFIG_X86_CMPXCHG=y
CONFIG_CMPXCHG_LOCAL=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
CONFIG_GART_IOMMU=y
# CONFIG_CALGARY_IOMMU is not set
CONFIG_AMD_IOMMU=y
CONFIG_AMD_IOMMU_STATS=y
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
# CONFIG_MAXSMP is not set
CONFIG_NR_CPUS=255
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_IRQ_TIME_ACCOUNTING is not set
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
CONFIG_X86_MCE_AMD=y
CONFIG_X86_MCE_THRESHOLD=y
CONFIG_X86_MCE_INJECT=y
CONFIG_X86_THERMAL_VECTOR=y
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_DIRECT_GBPAGES=y
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
CONFIG_NODES_SPAN_OTHER_NODES=y
# CONFIG_NUMA_EMU is not set
CONFIG_NODES_SHIFT=6
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ARCH_MEMORY_PROBE=y
CONFIG_ARCH_PROC_KCORE_TEXT=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_NEED_MULTIPLE_NODES=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_MEMORY_HOTPLUG_SPARSE=y
CONFIG_MEMORY_HOTREMOVE=y
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=999999
# CONFIG_COMPACTION is not set
CONFIG_MIGRATION=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_MEMORY_FAILURE is not set
# CONFIG_TRANSPARENT_HUGEPAGE is not set
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
CONFIG_X86_RESERVE_LOW=64
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=1
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_EFI=y
CONFIG_SECCOMP=y
CONFIG_CC_STACKPROTECTOR=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
CONFIG_CRASH_DUMP=y
CONFIG_PHYSICAL_START=0x200000
CONFIG_RELOCATABLE=y
CONFIG_PHYSICAL_ALIGN=0x1000000
CONFIG_HOTPLUG_CPU=y
CONFIG_COMPAT_VDSO=y
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
#
# Power management and ACPI options
#
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
# CONFIG_HIBERNATION is not set
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_RUNTIME=y
CONFIG_PM=y
CONFIG_PM_DEBUG=y
# CONFIG_PM_VERBOSE is not set
# CONFIG_PM_ADVANCED_DEBUG is not set
# CONFIG_PM_TEST_SUSPEND is not set
CONFIG_CAN_PM_TRACE=y
# CONFIG_PM_TRACE_RTC is not set
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
# CONFIG_ACPI_PROCFS is not set
# CONFIG_ACPI_PROCFS_POWER is not set
CONFIG_ACPI_POWER_METER=y
# CONFIG_ACPI_EC_DEBUGFS is not set
# CONFIG_ACPI_PROC_EVENT is not set
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR=y
# CONFIG_ACPI_IPMI is not set
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=y
CONFIG_ACPI_THERMAL=y
CONFIG_ACPI_NUMA=y
CONFIG_ACPI_CUSTOM_DSDT_FILE=""
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
CONFIG_ACPI_DEBUG=y
# CONFIG_ACPI_DEBUG_FUNC_TRACE is not set
CONFIG_ACPI_PCI_SLOT=y
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_HOTPLUG_MEMORY=y
# CONFIG_ACPI_SBS is not set
CONFIG_ACPI_HED=y
CONFIG_ACPI_APEI=y
CONFIG_ACPI_APEI_GHES=y
CONFIG_ACPI_APEI_PCIEAER=y
CONFIG_ACPI_APEI_EINJ=y
CONFIG_ACPI_APEI_ERST_DEBUG=y
# CONFIG_SFI is not set
#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
CONFIG_CPU_FREQ_DEBUG=y
CONFIG_CPU_FREQ_STAT=y
# CONFIG_CPU_FREQ_STAT_DETAILS is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
#
# CPUFreq processor drivers
#
CONFIG_X86_PCC_CPUFREQ=y
CONFIG_X86_ACPI_CPUFREQ=y
CONFIG_X86_POWERNOW_K8=y
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_P4_CLOCKMOD is not set
#
# shared options
#
# CONFIG_X86_SPEEDSTEP_LIB is not set
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
# CONFIG_INTEL_IDLE is not set
#
# Memory power savings
#
CONFIG_I7300_IDLE_IOAT_CHANNEL=y
CONFIG_I7300_IDLE=y
#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
CONFIG_DMAR=y
CONFIG_DMAR_DEFAULT_ON=y
CONFIG_DMAR_FLOPPY_WA=y
CONFIG_INTR_REMAP=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_PCIEAER=y
CONFIG_PCIE_ECRC=y
CONFIG_PCIEAER_INJECT=y
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEBUG=y
CONFIG_PCIE_PME=y
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_DEBUG=y
CONFIG_PCI_STUB=y
CONFIG_HT_IRQ=y
CONFIG_PCI_IOV=y
CONFIG_PCI_IOAPIC=y
CONFIG_PCI_LABEL=y
CONFIG_ISA_DMA_API=y
CONFIG_AMD_NB=y
CONFIG_PCCARD=y
# CONFIG_PCMCIA is not set
CONFIG_CARDBUS=y
#
# PC-card bridges
#
CONFIG_YENTA=y
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
CONFIG_HOTPLUG_PCI=y
# CONFIG_HOTPLUG_PCI_FAKE is not set
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=y
CONFIG_HOTPLUG_PCI_CPCI=y
# CONFIG_HOTPLUG_PCI_CPCI_ZT5550 is not set
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=y
CONFIG_HOTPLUG_PCI_SHPC=y
# CONFIG_RAPIDIO is not set
#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
# CONFIG_HAVE_AOUT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_AOUT is not set
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_KEYS_COMPAT=y
CONFIG_HAVE_TEXT_POKE_SMP=y
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=y
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y
CONFIG_IP_PNP_RARP=y
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE_DEMUX is not set
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
# CONFIG_INET_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_ADVANCED=y
# CONFIG_TCP_CONG_BIC is not set
CONFIG_TCP_CONG_CUBIC=y
# CONFIG_TCP_CONG_WESTWOOD is not set
# CONFIG_TCP_CONG_HTCP is not set
CONFIG_TCP_CONG_HSTCP=y
# CONFIG_TCP_CONG_HYBLA is not set
# CONFIG_TCP_CONG_VEGAS is not set
CONFIG_TCP_CONG_SCALABLE=y
# CONFIG_TCP_CONG_LP is not set
# CONFIG_TCP_CONG_VENO is not set
# CONFIG_TCP_CONG_YEAH is not set
# CONFIG_TCP_CONG_ILLINOIS is not set
CONFIG_DEFAULT_CUBIC=y
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
# CONFIG_NETFILTER is not set
CONFIG_IP_DCCP=y
CONFIG_INET_DCCP_DIAG=y
#
# DCCP CCIDs Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP_CCID2_DEBUG is not set
CONFIG_IP_DCCP_CCID3=y
# CONFIG_IP_DCCP_CCID3_DEBUG is not set
CONFIG_IP_DCCP_TFRC_LIB=y
#
# DCCP Kernel Hacking
#
# CONFIG_IP_DCCP_DEBUG is not set
# CONFIG_NET_DCCPPROBE is not set
CONFIG_IP_SCTP=y
# CONFIG_NET_SCTPPROBE is not set
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
CONFIG_RDS=y
CONFIG_RDS_RDMA=y
CONFIG_RDS_TCP=y
# CONFIG_RDS_DEBUG is not set
CONFIG_TIPC=y
# CONFIG_TIPC_ADVANCED is not set
# CONFIG_TIPC_DEBUG is not set
# CONFIG_ATM is not set
# CONFIG_L2TP is not set
# CONFIG_BRIDGE is not set
# CONFIG_NET_DSA is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_PHONET is not set
# CONFIG_IEEE802154 is not set
# CONFIG_NET_SCHED is not set
# CONFIG_DCB is not set
CONFIG_DNS_RESOLVER=y
# CONFIG_BATMAN_ADV is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
CONFIG_HAVE_BPF_JIT=y
# CONFIG_BPF_JIT is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NET_TCPPROBE is not set
# CONFIG_NET_DROP_MONITOR is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
CONFIG_WIRELESS=y
# CONFIG_CFG80211 is not set
# CONFIG_LIB80211 is not set
#
# CFG80211 needs to be enabled for MAC80211
#
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
# CONFIG_CAIF is not set
# CONFIG_CEPH_LIB is not set
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_DEVTMPFS is not set
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE="ql2100_fw.bin ql2300_fw.bin ql2400_fw.bin ql2200_fw.bin ql2322_fw.bin ql2500_fw.bin"
CONFIG_EXTRA_FIRMWARE_DIR="../firmware"
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
CONFIG_ARCH_NO_SYSDEV_OPS=y
CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
# CONFIG_MTD is not set
# CONFIG_PARPORT is not set
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y
#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_DRBD is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_OSD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=262144
# CONFIG_BLK_DEV_XIP is not set
# CONFIG_CDROM_PKTCDVD is not set
CONFIG_ATA_OVER_ETH=y
CONFIG_VIRTIO_BLK=y
# CONFIG_BLK_DEV_HD is not set
# CONFIG_BLK_DEV_RBD is not set
# CONFIG_SENSORS_LIS3LV02D is not set
CONFIG_MISC_DEVICES=y
# CONFIG_AD525X_DPOT is not set
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
# CONFIG_SGI_IOC4 is not set
CONFIG_TIFM_CORE=y
CONFIG_TIFM_7XX1=y
# CONFIG_ICS932S401 is not set
CONFIG_ENCLOSURE_SERVICES=y
# CONFIG_CS5535_MFGPT is not set
# CONFIG_HP_ILO is not set
# CONFIG_APDS9802ALS is not set
# CONFIG_ISL29003 is not set
# CONFIG_ISL29020 is not set
# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_SENSORS_BH1780 is not set
# CONFIG_SENSORS_BH1770 is not set
# CONFIG_SENSORS_APDS990X is not set
# CONFIG_HMC6352 is not set
# CONFIG_DS1682 is not set
# CONFIG_TI_DAC7512 is not set
# CONFIG_VMWARE_BALLOON is not set
# CONFIG_BMP085 is not set
# CONFIG_PCH_PHUB is not set
# CONFIG_C2PORT is not set
#
# EEPROM support
#
CONFIG_EEPROM_AT24=y
CONFIG_EEPROM_AT25=y
# CONFIG_EEPROM_LEGACY is not set
# CONFIG_EEPROM_MAX6875 is not set
# CONFIG_EEPROM_93CX6 is not set
CONFIG_CB710_CORE=y
# CONFIG_CB710_DEBUG is not set
CONFIG_CB710_DEBUG_ASSUMPTIONS=y
# CONFIG_IWMC3200TOP is not set
#
# Texas Instruments shared transport line discipline
#
# CONFIG_SENSORS_LIS3_I2C is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_TGT=y
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=y
CONFIG_CHR_DEV_SCH=y
CONFIG_SCSI_ENCLOSURE=y
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
# CONFIG_SCSI_SCAN_ASYNC is not set
CONFIG_SCSI_WAIT_SCAN=m
#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=y
CONFIG_SCSI_FC_ATTRS=y
CONFIG_SCSI_FC_TGT_ATTRS=y
CONFIG_SCSI_ISCSI_ATTRS=y
CONFIG_SCSI_SAS_ATTRS=y
CONFIG_SCSI_SAS_LIBSAS=y
CONFIG_SCSI_SAS_ATA=y
CONFIG_SCSI_SAS_HOST_SMP=y
CONFIG_SCSI_SRP_ATTRS=y
CONFIG_SCSI_SRP_TGT_ATTRS=y
CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=y
CONFIG_ISCSI_BOOT_SYSFS=y
CONFIG_SCSI_CXGB3_ISCSI=y
CONFIG_SCSI_CXGB4_ISCSI=y
CONFIG_SCSI_BNX2_ISCSI=y
CONFIG_SCSI_BNX2X_FCOE=y
CONFIG_BE2ISCSI=y
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_HPSA is not set
CONFIG_SCSI_3W_9XXX=y
# CONFIG_SCSI_3W_SAS is not set
# CONFIG_SCSI_ACARD is not set
CONFIG_SCSI_AACRAID=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_AIC7XXX_CMDS_PER_DEVICE=32
CONFIG_AIC7XXX_RESET_DELAY_MS=5000
# CONFIG_AIC7XXX_BUILD_FIRMWARE is not set
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
# CONFIG_SCSI_AIC7XXX_OLD is not set
CONFIG_SCSI_AIC79XX=y
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=5000
# CONFIG_AIC79XX_BUILD_FIRMWARE is not set
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC94XX=y
# CONFIG_AIC94XX_DEBUG is not set
CONFIG_SCSI_MVSAS=y
CONFIG_SCSI_MVSAS_DEBUG=y
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_ARCMSR is not set
CONFIG_MEGARAID_NEWGEN=y
CONFIG_MEGARAID_MM=y
CONFIG_MEGARAID_MAILBOX=y
CONFIG_MEGARAID_LEGACY=y
CONFIG_MEGARAID_SAS=y
CONFIG_SCSI_MPT2SAS=y
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS_LOGGING=y
# CONFIG_SCSI_HPTIOP is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_VMWARE_PVSCSI is not set
CONFIG_LIBFC=y
CONFIG_LIBFCOE=y
CONFIG_FCOE=y
CONFIG_FCOE_FNIC=y
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_STEX is not set
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y
# CONFIG_SCSI_IPR is not set
CONFIG_SCSI_QLOGIC_1280=y
CONFIG_SCSI_QLA_FC=y
CONFIG_SCSI_QLA_ISCSI=y
CONFIG_SCSI_LPFC=y
CONFIG_SCSI_LPFC_DEBUG_FS=y
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
CONFIG_SCSI_PMCRAID=y
# CONFIG_SCSI_PM8001 is not set
CONFIG_SCSI_SRP=y
CONFIG_SCSI_BFA_FC=y
CONFIG_SCSI_DH=y
CONFIG_SCSI_DH_RDAC=y
# CONFIG_SCSI_DH_HP_SW is not set
# CONFIG_SCSI_DH_EMC is not set
# CONFIG_SCSI_DH_ALUA is not set
CONFIG_SCSI_OSD_INITIATOR=y
CONFIG_SCSI_OSD_ULD=y
CONFIG_SCSI_OSD_DPRINT_SENSE=1
CONFIG_SCSI_OSD_DEBUG=y
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_ACPI=y
CONFIG_SATA_PMP=y
#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_SATA_ACARD_AHCI is not set
# CONFIG_SATA_SIL24 is not set
CONFIG_ATA_SFF=y
#
# SFF controllers with custom DMA interface
#
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_SX4 is not set
CONFIG_ATA_BMDMA=y
#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=y
CONFIG_SATA_MV=y
CONFIG_SATA_NV=y
# CONFIG_SATA_PROMISE is not set
CONFIG_SATA_SIL=y
# CONFIG_SATA_SIS is not set
CONFIG_SATA_SVW=y
# CONFIG_SATA_ULI is not set
CONFIG_SATA_VIA=y
# CONFIG_SATA_VITESSE is not set
#
# PATA SFF controllers with BMDMA
#
# CONFIG_PATA_ALI is not set
CONFIG_PATA_AMD=y
CONFIG_PATA_ARASAN_CF=y
# CONFIG_PATA_ARTOP is not set
CONFIG_PATA_ATIIXP=y
# CONFIG_PATA_ATP867X is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CS5536 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87415 is not set
CONFIG_PATA_OLDPIIX=y
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RDC is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SCH is not set
CONFIG_PATA_SERVERWORKS=y
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_TOSHIBA is not set
# CONFIG_PATA_TRIFLEX is not set
CONFIG_PATA_VIA=y
# CONFIG_PATA_WINBOND is not set
#
# PIO-only SFF controllers
#
# CONFIG_PATA_CMD640_PCI is not set
CONFIG_PATA_MPIIX=y
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_RZ1000 is not set
#
# Generic fallback / legacy drivers
#
# CONFIG_PATA_ACPI is not set
# CONFIG_ATA_GENERIC is not set
CONFIG_PATA_LEGACY=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=y
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=y
CONFIG_MD_RAID456=y
# CONFIG_MULTICORE_RAID456 is not set
CONFIG_MD_MULTIPATH=y
CONFIG_MD_FAULTY=y
CONFIG_BLK_DEV_DM=y
CONFIG_DM_DEBUG=y
CONFIG_DM_CRYPT=y
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
# CONFIG_DM_RAID is not set
# CONFIG_DM_LOG_USERSPACE is not set
CONFIG_DM_ZERO=y
CONFIG_DM_MULTIPATH=y
# CONFIG_DM_MULTIPATH_QL is not set
# CONFIG_DM_MULTIPATH_ST is not set
CONFIG_DM_DELAY=y
CONFIG_DM_UEVENT=y
# CONFIG_DM_FLAKEY is not set
# CONFIG_TARGET_CORE is not set
CONFIG_FUSION=y
CONFIG_FUSION_SPI=y
CONFIG_FUSION_FC=y
CONFIG_FUSION_SAS=y
CONFIG_FUSION_MAX_SGE=128
CONFIG_FUSION_CTL=y
CONFIG_FUSION_LAN=y
# CONFIG_FUSION_LOGGING is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_FIREWIRE_NOSY is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
CONFIG_MACVLAN=y
CONFIG_MACVTAP=y
# CONFIG_EQUALIZER is not set
CONFIG_TUN=y
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
CONFIG_MII=y
CONFIG_PHYLIB=y
#
# MII PHY device drivers
#
CONFIG_MARVELL_PHY=y
CONFIG_DAVICOM_PHY=y
CONFIG_QSEMI_PHY=y
CONFIG_LXT_PHY=y
CONFIG_CICADA_PHY=y
CONFIG_VITESSE_PHY=y
CONFIG_SMSC_PHY=y
CONFIG_BROADCOM_PHY=y
# CONFIG_BCM63XX_PHY is not set
CONFIG_ICPLUS_PHY=y
CONFIG_REALTEK_PHY=y
CONFIG_NATIONAL_PHY=y
CONFIG_STE10XP=y
CONFIG_LSI_ET1011C_PHY=y
# CONFIG_MICREL_PHY is not set
CONFIG_FIXED_PHY=y
CONFIG_MDIO_BITBANG=y
# CONFIG_MDIO_GPIO is not set
CONFIG_NET_ETHERNET=y
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_ENC28J60 is not set
# CONFIG_ETHOC is not set
# CONFIG_DNET is not set
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
# CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
# CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=y
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_KSZ884X_PCI is not set
# CONFIG_B44 is not set
CONFIG_FORCEDETH=y
CONFIG_E100=y
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
CONFIG_8139CP=y
# CONFIG_8139TOO is not set
# CONFIG_R6040 is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SMSC9420 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_KS8842 is not set
# CONFIG_KS8851 is not set
# CONFIG_KS8851_MLL is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_SC92031 is not set
# CONFIG_ATL2 is not set
CONFIG_NETDEV_1000=y
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
CONFIG_E1000=y
CONFIG_E1000E=y
# CONFIG_IP1000 is not set
CONFIG_IGB=y
CONFIG_IGBVF=y
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
CONFIG_SKGE=y
# CONFIG_SKGE_DEBUG is not set
CONFIG_SKY2=y
# CONFIG_SKY2_DEBUG is not set
CONFIG_VIA_VELOCITY=y
CONFIG_TIGON3=y
CONFIG_BNX2=y
CONFIG_CNIC=y
CONFIG_QLA3XXX=y
CONFIG_ATL1=y
CONFIG_ATL1E=y
CONFIG_ATL1C=y
CONFIG_JME=y
# CONFIG_STMMAC_ETH is not set
# CONFIG_PCH_GBE is not set
CONFIG_NETDEV_10000=y
CONFIG_MDIO=y
CONFIG_CHELSIO_T1=y
CONFIG_CHELSIO_T1_1G=y
CONFIG_CHELSIO_T3=y
CONFIG_CHELSIO_T4=y
CONFIG_CHELSIO_T4VF=y
CONFIG_ENIC=y
CONFIG_IXGBE=y
CONFIG_IXGBEVF=y
CONFIG_IXGB=y
CONFIG_S2IO=y
CONFIG_MYRI10GE=y
CONFIG_NIU=y
CONFIG_MLX4_EN=y
CONFIG_MLX4_CORE=y
CONFIG_MLX4_DEBUG=y
CONFIG_TEHUTI=y
CONFIG_BNX2X=y
CONFIG_QLCNIC=y
CONFIG_QLGE=y
CONFIG_BNA=y
CONFIG_SFC=y
# CONFIG_BE2NET is not set
# CONFIG_TR is not set
CONFIG_WLAN=y
# CONFIG_AIRO is not set
# CONFIG_ATMEL is not set
# CONFIG_PRISM54 is not set
# CONFIG_USB_ZD1201 is not set
# CONFIG_HOSTAP is not set
#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#
#
# USB Network Adapters
#
CONFIG_USB_CATC=y
CONFIG_USB_KAWETH=y
CONFIG_USB_PEGASUS=y
CONFIG_USB_RTL8150=y
CONFIG_USB_USBNET=y
CONFIG_USB_NET_AX8817X=y
CONFIG_USB_NET_CDCETHER=y
# CONFIG_USB_NET_CDC_EEM is not set
CONFIG_USB_NET_CDC_NCM=y
CONFIG_USB_NET_DM9601=y
# CONFIG_USB_NET_SMSC75XX is not set
CONFIG_USB_NET_SMSC95XX=y
CONFIG_USB_NET_GL620A=y
CONFIG_USB_NET_NET1080=y
CONFIG_USB_NET_PLUSB=y
CONFIG_USB_NET_MCS7830=y
CONFIG_USB_NET_RNDIS_HOST=y
CONFIG_USB_NET_CDC_SUBSET=y
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=y
# CONFIG_USB_NET_CX82310_ETH is not set
# CONFIG_USB_NET_INT51X1 is not set
CONFIG_USB_IPHETH=y
CONFIG_USB_SIERRA_NET=y
# CONFIG_USB_VL600 is not set
# CONFIG_WAN is not set
#
# CAIF transport drivers
#
CONFIG_FDDI=y
# CONFIG_DEFXX is not set
CONFIG_SKFP=y
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
CONFIG_NET_FC=y
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
CONFIG_VIRTIO_NET=y
# CONFIG_VMXNET3 is not set
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_FF_MEMLESS=y
# CONFIG_INPUT_POLLDEV is not set
# CONFIG_INPUT_SPARSEKMAP is not set
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1070 is not set
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_GPIO is not set
# CONFIG_KEYBOARD_GPIO_POLLED is not set
# CONFIG_KEYBOARD_TCA6416 is not set
# CONFIG_KEYBOARD_MATRIX is not set
# CONFIG_KEYBOARD_LM8323 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_MCS is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
# CONFIG_MOUSE_PS2_ELANTECH is not set
# CONFIG_MOUSE_PS2_SENTELIC is not set
# CONFIG_MOUSE_PS2_TOUCHKIT is not set
CONFIG_MOUSE_SERIAL=y
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_BCM5974 is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_MOUSE_GPIO is not set
# CONFIG_MOUSE_SYNAPTICS_I2C is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
# CONFIG_INPUT_AD714X is not set
CONFIG_INPUT_PCSPKR=y
# CONFIG_INPUT_APANEL is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
# CONFIG_INPUT_CM109 is not set
# CONFIG_INPUT_UINPUT is not set
# CONFIG_INPUT_PCF8574 is not set
# CONFIG_INPUT_GPIO_ROTARY_ENCODER is not set
# CONFIG_INPUT_ADXL34X is not set
# CONFIG_INPUT_CMA3000 is not set
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
CONFIG_SERIO_PCIPS2=y
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=y
# CONFIG_SERIO_ALTERA_PS2 is not set
# CONFIG_SERIO_PS2MULT is not set
# CONFIG_GAMEPORT is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NOZOMI is not set
# CONFIG_N_GSM is not set
CONFIG_DEVKMEM=y
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
# CONFIG_SERIAL_8250_MANY_PORTS is not set
# CONFIG_SERIAL_8250_SHARE_IRQ is not set
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
# CONFIG_SERIAL_8250_RSA is not set
#
# Non-8250 serial port support
#
# CONFIG_SERIAL_MAX3100 is not set
# CONFIG_SERIAL_MAX3107 is not set
# CONFIG_SERIAL_MFD_HSU is not set
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_JSM=y
# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_IFX6X60 is not set
# CONFIG_SERIAL_PCH_UART is not set
# CONFIG_VIRTIO_CONSOLE is not set
CONFIG_IPMI_HANDLER=y
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=y
CONFIG_IPMI_SI=y
# CONFIG_IPMI_WATCHDOG is not set
# CONFIG_IPMI_POWEROFF is not set
CONFIG_HW_RANDOM=y
# CONFIG_HW_RANDOM_TIMERIOMEM is not set
CONFIG_HW_RANDOM_INTEL=y
CONFIG_HW_RANDOM_AMD=y
CONFIG_HW_RANDOM_VIA=y
CONFIG_HW_RANDOM_VIRTIO=y
CONFIG_NVRAM=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_MWAVE is not set
CONFIG_RAW_DRIVER=y
CONFIG_MAX_RAW_DEVS=256
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
# CONFIG_HANGCHECK_TIMER is not set
CONFIG_TCG_TPM=y
CONFIG_TCG_TIS=y
CONFIG_TCG_NSC=y
CONFIG_TCG_ATMEL=y
CONFIG_TCG_INFINEON=y
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_RAMOOPS is not set
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
# CONFIG_I2C_MUX is not set
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_ALGOBIT=y
#
# I2C Hardware Bus support
#
#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
CONFIG_I2C_AMD756=y
CONFIG_I2C_AMD8111=y
CONFIG_I2C_I801=y
CONFIG_I2C_ISCH=y
CONFIG_I2C_PIIX4=y
CONFIG_I2C_NFORCE2=y
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
#
# ACPI drivers
#
# CONFIG_I2C_SCMI is not set
#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_GPIO is not set
# CONFIG_I2C_INTEL_MID is not set
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_PXA_PCI is not set
# CONFIG_I2C_SIMTEC is not set
# CONFIG_I2C_XILINX is not set
# CONFIG_I2C_EG20T is not set
#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_DIOLAN_U2C is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set
#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
CONFIG_SPI=y
# CONFIG_SPI_DEBUG is not set
CONFIG_SPI_MASTER=y
#
# SPI Master Controller Drivers
#
# CONFIG_SPI_ALTERA is not set
CONFIG_SPI_BITBANG=y
# CONFIG_SPI_GPIO is not set
# CONFIG_SPI_OC_TINY is not set
# CONFIG_SPI_PXA2XX_PCI is not set
# CONFIG_SPI_TOPCLIFF_PCH is not set
# CONFIG_SPI_XILINX is not set
# CONFIG_SPI_DESIGNWARE is not set
#
# SPI Protocol Masters
#
# CONFIG_SPI_SPIDEV is not set
# CONFIG_SPI_TLE62X0 is not set
#
# PPS support
#
# CONFIG_PPS is not set
#
# PPS generators support
#
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
CONFIG_GPIOLIB=y
# CONFIG_DEBUG_GPIO is not set
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_MAX730X=y
#
# Memory mapped GPIO expanders:
#
# CONFIG_GPIO_BASIC_MMIO is not set
# CONFIG_GPIO_IT8761E is not set
CONFIG_GPIO_SCH=y
# CONFIG_GPIO_VX855 is not set
#
# I2C GPIO expanders:
#
# CONFIG_GPIO_MAX7300 is not set
CONFIG_GPIO_MAX732X=y
# CONFIG_GPIO_MAX732X_IRQ is not set
CONFIG_GPIO_PCA953X=y
# CONFIG_GPIO_PCA953X_IRQ is not set
CONFIG_GPIO_PCF857X=y
# CONFIG_GPIO_SX150X is not set
# CONFIG_GPIO_ADP5588 is not set
#
# PCI GPIO expanders:
#
# CONFIG_GPIO_CS5535 is not set
CONFIG_GPIO_BT8XX=y
# CONFIG_GPIO_LANGWELL is not set
# CONFIG_GPIO_PCH is not set
# CONFIG_GPIO_ML_IOH is not set
# CONFIG_GPIO_RDC321X is not set
#
# SPI GPIO expanders:
#
CONFIG_GPIO_MAX7301=y
CONFIG_GPIO_MCP23S08=y
# CONFIG_GPIO_MC33880 is not set
# CONFIG_GPIO_74X164 is not set
#
# AC97 GPIO expanders:
#
#
# MODULbus GPIO expanders:
#
CONFIG_W1=y
CONFIG_W1_CON=y
#
# 1-wire Bus Masters
#
CONFIG_W1_MASTER_MATROX=y
CONFIG_W1_MASTER_DS2490=y
CONFIG_W1_MASTER_DS2482=y
CONFIG_W1_MASTER_GPIO=y
#
# 1-wire Slaves
#
CONFIG_W1_SLAVE_THERM=y
CONFIG_W1_SLAVE_SMEM=y
# CONFIG_W1_SLAVE_DS2423 is not set
# CONFIG_W1_SLAVE_DS2431 is not set
CONFIG_W1_SLAVE_DS2433=y
# CONFIG_W1_SLAVE_DS2433_CRC is not set
CONFIG_W1_SLAVE_DS2760=y
# CONFIG_W1_SLAVE_BQ27000 is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
# CONFIG_PDA_POWER is not set
# CONFIG_TEST_POWER is not set
# CONFIG_BATTERY_DS2760 is not set
# CONFIG_BATTERY_DS2782 is not set
# CONFIG_BATTERY_BQ20Z75 is not set
# CONFIG_BATTERY_BQ27x00 is not set
# CONFIG_BATTERY_MAX17040 is not set
# CONFIG_BATTERY_MAX17042 is not set
# CONFIG_CHARGER_GPIO is not set
CONFIG_HWMON=y
CONFIG_HWMON_VID=y
# CONFIG_HWMON_DEBUG_CHIP is not set
#
# Native drivers
#
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_ABITUGURU3 is not set
# CONFIG_SENSORS_AD7414 is not set
# CONFIG_SENSORS_AD7418 is not set
# CONFIG_SENSORS_ADCXX is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
CONFIG_SENSORS_ADM1026=y
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ADT7411 is not set
# CONFIG_SENSORS_ADT7462 is not set
# CONFIG_SENSORS_ADT7470 is not set
# CONFIG_SENSORS_ADT7475 is not set
# CONFIG_SENSORS_ASC7621 is not set
CONFIG_SENSORS_K8TEMP=y
CONFIG_SENSORS_K10TEMP=y
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS620 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_F71882FG is not set
# CONFIG_SENSORS_F75375S is not set
# CONFIG_SENSORS_FSCHMD is not set
# CONFIG_SENSORS_G760A is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_GPIO_FAN is not set
CONFIG_SENSORS_CORETEMP=y
# CONFIG_SENSORS_PKGTEMP is not set
# CONFIG_SENSORS_IBMAEM is not set
# CONFIG_SENSORS_IBMPEX is not set
CONFIG_SENSORS_IT87=y
# CONFIG_SENSORS_JC42 is not set
# CONFIG_SENSORS_LINEAGE is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM70 is not set
# CONFIG_SENSORS_LM73 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
CONFIG_SENSORS_LM83=y
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_LM93 is not set
# CONFIG_SENSORS_LTC4151 is not set
# CONFIG_SENSORS_LTC4215 is not set
# CONFIG_SENSORS_LTC4245 is not set
# CONFIG_SENSORS_LTC4261 is not set
# CONFIG_SENSORS_LM95241 is not set
# CONFIG_SENSORS_MAX1111 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_MAX6639 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
CONFIG_SENSORS_PC87427=y
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_PMBUS is not set
# CONFIG_SENSORS_SHT15 is not set
# CONFIG_SENSORS_SHT21 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMM665 is not set
CONFIG_SENSORS_DME1737=y
# CONFIG_SENSORS_EMC1403 is not set
# CONFIG_SENSORS_EMC2103 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47M192 is not set
CONFIG_SENSORS_SMSC47B397=y
# CONFIG_SENSORS_SCH5627 is not set
# CONFIG_SENSORS_ADS1015 is not set
# CONFIG_SENSORS_ADS7828 is not set
# CONFIG_SENSORS_ADS7871 is not set
# CONFIG_SENSORS_AMC6821 is not set
# CONFIG_SENSORS_THMC50 is not set
# CONFIG_SENSORS_TMP102 is not set
# CONFIG_SENSORS_TMP401 is not set
# CONFIG_SENSORS_TMP421 is not set
# CONFIG_SENSORS_VIA_CPUTEMP is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
# CONFIG_SENSORS_VT8231 is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83791D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83795 is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83L786NG is not set
CONFIG_SENSORS_W83627HF=y
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_APPLESMC is not set
#
# ACPI drivers
#
# CONFIG_SENSORS_ATK0110 is not set
CONFIG_THERMAL=y
CONFIG_THERMAL_HWMON=y
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
#
# Watchdog Device Drivers
#
# CONFIG_SOFT_WATCHDOG is not set
# CONFIG_ACQUIRE_WDT is not set
# CONFIG_ADVANTECH_WDT is not set
# CONFIG_ALIM1535_WDT is not set
# CONFIG_ALIM7101_WDT is not set
# CONFIG_F71808E_WDT is not set
CONFIG_SP5100_TCO=y
# CONFIG_SC520_WDT is not set
# CONFIG_EUROTECH_WDT is not set
# CONFIG_IB700_WDT is not set
# CONFIG_IBMASR is not set
# CONFIG_WAFER_WDT is not set
# CONFIG_I6300ESB_WDT is not set
# CONFIG_ITCO_WDT is not set
# CONFIG_IT8712F_WDT is not set
# CONFIG_IT87_WDT is not set
# CONFIG_HP_WATCHDOG is not set
# CONFIG_SC1200_WDT is not set
# CONFIG_PC87413_WDT is not set
# CONFIG_NV_TCO is not set
# CONFIG_60XX_WDT is not set
# CONFIG_SBC8360_WDT is not set
# CONFIG_CPU5_WDT is not set
# CONFIG_SMSC_SCH311X_WDT is not set
# CONFIG_SMSC37B787_WDT is not set
# CONFIG_W83627HF_WDT is not set
# CONFIG_W83697HF_WDT is not set
# CONFIG_W83697UG_WDT is not set
# CONFIG_W83877F_WDT is not set
# CONFIG_W83977F_WDT is not set
# CONFIG_MACHZ_WDT is not set
# CONFIG_SBC_EPX_C3_WATCHDOG is not set
#
# PCI-based Watchdog Cards
#
# CONFIG_PCIPCWATCHDOG is not set
# CONFIG_WDTPCI is not set
#
# USB-based Watchdog Cards
#
# CONFIG_USBPCWATCHDOG is not set
CONFIG_SSB_POSSIBLE=y
#
# Sonics Silicon Backplane
#
# CONFIG_SSB is not set
CONFIG_MFD_SUPPORT=y
CONFIG_MFD_CORE=y
# CONFIG_MFD_88PM860X is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_HTC_I2CPLD is not set
# CONFIG_UCB1400_CORE is not set
# CONFIG_TPS6105X is not set
# CONFIG_TPS65010 is not set
# CONFIG_TPS6507X is not set
# CONFIG_TWL4030_CORE is not set
# CONFIG_MFD_STMPE is not set
# CONFIG_MFD_TC3589X is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_PMIC_DA903X is not set
# CONFIG_PMIC_ADP5520 is not set
# CONFIG_MFD_MAX8925 is not set
# CONFIG_MFD_MAX8997 is not set
# CONFIG_MFD_MAX8998 is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X_I2C is not set
# CONFIG_MFD_WM831X_SPI is not set
# CONFIG_MFD_WM8350_I2C is not set
# CONFIG_MFD_WM8994 is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_MFD_MC13XXX is not set
# CONFIG_ABX500_CORE is not set
# CONFIG_EZX_PCAP is not set
# CONFIG_MFD_CS5535 is not set
# CONFIG_MFD_TIMBERDALE is not set
CONFIG_LPC_SCH=y
# CONFIG_MFD_RDC321X is not set
# CONFIG_MFD_JANZ_CMODIO is not set
# CONFIG_MFD_TPS6586X is not set
# CONFIG_MFD_VX855 is not set
# CONFIG_MFD_WL1273_CORE is not set
# CONFIG_REGULATOR is not set
CONFIG_MEDIA_SUPPORT=y
#
# Multimedia core support
#
# CONFIG_MEDIA_CONTROLLER is not set
CONFIG_VIDEO_DEV=y
CONFIG_VIDEO_V4L2_COMMON=y
CONFIG_DVB_CORE=y
CONFIG_VIDEO_MEDIA=y
#
# Multimedia drivers
#
CONFIG_RC_CORE=y
CONFIG_LIRC=y
CONFIG_RC_MAP=y
CONFIG_IR_NEC_DECODER=y
CONFIG_IR_RC5_DECODER=y
CONFIG_IR_RC6_DECODER=y
CONFIG_IR_JVC_DECODER=y
CONFIG_IR_SONY_DECODER=y
CONFIG_IR_RC5_SZ_DECODER=y
CONFIG_IR_LIRC_CODEC=y
# CONFIG_IR_ENE is not set
# CONFIG_IR_IMON is not set
# CONFIG_IR_MCEUSB is not set
# CONFIG_IR_ITE_CIR is not set
# CONFIG_IR_NUVOTON is not set
# CONFIG_IR_STREAMZAP is not set
# CONFIG_IR_WINBOND_CIR is not set
# CONFIG_RC_LOOPBACK is not set
# CONFIG_MEDIA_ATTACH is not set
CONFIG_MEDIA_TUNER=y
# CONFIG_MEDIA_TUNER_CUSTOMISE is not set
CONFIG_MEDIA_TUNER_SIMPLE=y
CONFIG_MEDIA_TUNER_TDA8290=y
CONFIG_MEDIA_TUNER_TDA827X=y
CONFIG_MEDIA_TUNER_TDA18271=y
CONFIG_MEDIA_TUNER_TDA9887=y
CONFIG_MEDIA_TUNER_TEA5761=y
CONFIG_MEDIA_TUNER_TEA5767=y
CONFIG_MEDIA_TUNER_MT20XX=y
CONFIG_MEDIA_TUNER_XC2028=y
CONFIG_MEDIA_TUNER_XC5000=y
CONFIG_MEDIA_TUNER_MC44S803=y
CONFIG_VIDEO_V4L2=y
CONFIG_VIDEOBUF_GEN=y
CONFIG_VIDEOBUF2_CORE=y
CONFIG_VIDEO_CAPTURE_DRIVERS=y
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
CONFIG_VIDEO_HELPER_CHIPS_AUTO=y
CONFIG_VIDEO_IR_I2C=y
#
# Audio decoders
#
#
# RDS decoders
#
#
# Video decoders
#
#
# Video and audio decoders
#
#
# MPEG video encoders
#
#
# Video encoders
#
#
# Video improvement chips
#
# CONFIG_VIDEO_VIVI is not set
# CONFIG_VIDEO_BT848 is not set
# CONFIG_VIDEO_CPIA2 is not set
# CONFIG_VIDEO_ZORAN is not set
# CONFIG_VIDEO_SAA7134 is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_TIMBERDALE is not set
# CONFIG_VIDEO_CX88 is not set
# CONFIG_VIDEO_CX23885 is not set
# CONFIG_VIDEO_IVTV is not set
# CONFIG_VIDEO_CX18 is not set
# CONFIG_VIDEO_SAA7164 is not set
# CONFIG_VIDEO_CAFE_CCIC is not set
# CONFIG_VIDEO_SR030PC30 is not set
# CONFIG_VIDEO_NOON010PC30 is not set
CONFIG_SOC_CAMERA=y
# CONFIG_SOC_CAMERA_IMX074 is not set
# CONFIG_SOC_CAMERA_MT9M001 is not set
# CONFIG_SOC_CAMERA_MT9M111 is not set
# CONFIG_SOC_CAMERA_MT9T031 is not set
# CONFIG_SOC_CAMERA_MT9T112 is not set
# CONFIG_SOC_CAMERA_MT9V022 is not set
# CONFIG_SOC_CAMERA_RJ54N1 is not set
# CONFIG_SOC_CAMERA_TW9910 is not set
# CONFIG_SOC_CAMERA_PLATFORM is not set
# CONFIG_SOC_CAMERA_OV2640 is not set
# CONFIG_SOC_CAMERA_OV6650 is not set
# CONFIG_SOC_CAMERA_OV772X is not set
# CONFIG_SOC_CAMERA_OV9640 is not set
# CONFIG_SOC_CAMERA_OV9740 is not set
CONFIG_V4L_USB_DRIVERS=y
# CONFIG_USB_VIDEO_CLASS is not set
CONFIG_USB_VIDEO_CLASS_INPUT_EVDEV=y
CONFIG_USB_GSPCA=y
# CONFIG_USB_M5602 is not set
# CONFIG_USB_STV06XX is not set
# CONFIG_USB_GL860 is not set
# CONFIG_USB_GSPCA_BENQ is not set
# CONFIG_USB_GSPCA_CONEX is not set
# CONFIG_USB_GSPCA_CPIA1 is not set
# CONFIG_USB_GSPCA_ETOMS is not set
# CONFIG_USB_GSPCA_FINEPIX is not set
# CONFIG_USB_GSPCA_JEILINJ is not set
# CONFIG_USB_GSPCA_KONICA is not set
# CONFIG_USB_GSPCA_MARS is not set
# CONFIG_USB_GSPCA_MR97310A is not set
# CONFIG_USB_GSPCA_NW80X is not set
# CONFIG_USB_GSPCA_OV519 is not set
# CONFIG_USB_GSPCA_OV534 is not set
# CONFIG_USB_GSPCA_OV534_9 is not set
# CONFIG_USB_GSPCA_PAC207 is not set
# CONFIG_USB_GSPCA_PAC7302 is not set
# CONFIG_USB_GSPCA_PAC7311 is not set
# CONFIG_USB_GSPCA_SN9C2028 is not set
# CONFIG_USB_GSPCA_SN9C20X is not set
# CONFIG_USB_GSPCA_SONIXB is not set
# CONFIG_USB_GSPCA_SONIXJ is not set
# CONFIG_USB_GSPCA_SPCA500 is not set
# CONFIG_USB_GSPCA_SPCA501 is not set
CONFIG_USB_GSPCA_SPCA505=y
# CONFIG_USB_GSPCA_SPCA506 is not set
# CONFIG_USB_GSPCA_SPCA508 is not set
# CONFIG_USB_GSPCA_SPCA561 is not set
# CONFIG_USB_GSPCA_SPCA1528 is not set
# CONFIG_USB_GSPCA_SQ905 is not set
# CONFIG_USB_GSPCA_SQ905C is not set
# CONFIG_USB_GSPCA_SQ930X is not set
# CONFIG_USB_GSPCA_STK014 is not set
# CONFIG_USB_GSPCA_STV0680 is not set
# CONFIG_USB_GSPCA_SUNPLUS is not set
# CONFIG_USB_GSPCA_T613 is not set
# CONFIG_USB_GSPCA_TV8532 is not set
# CONFIG_USB_GSPCA_VC032X is not set
# CONFIG_USB_GSPCA_VICAM is not set
# CONFIG_USB_GSPCA_XIRLINK_CIT is not set
# CONFIG_USB_GSPCA_ZC3XX is not set
# CONFIG_VIDEO_PVRUSB2 is not set
# CONFIG_VIDEO_HDPVR is not set
# CONFIG_VIDEO_EM28XX is not set
# CONFIG_VIDEO_TLG2300 is not set
# CONFIG_VIDEO_CX231XX is not set
# CONFIG_VIDEO_USBVISION is not set
# CONFIG_USB_ET61X251 is not set
# CONFIG_USB_SN9C102 is not set
# CONFIG_USB_PWC is not set
# CONFIG_USB_ZR364XX is not set
# CONFIG_USB_STKWEBCAM is not set
# CONFIG_USB_S2255 is not set
# CONFIG_V4L_MEM2MEM_DRIVERS is not set
CONFIG_RADIO_ADAPTERS=y
# CONFIG_RADIO_MAXIRADIO is not set
# CONFIG_RADIO_MAESTRO is not set
# CONFIG_I2C_SI4713 is not set
# CONFIG_RADIO_SI4713 is not set
# CONFIG_USB_DSBR is not set
# CONFIG_RADIO_SI470X is not set
# CONFIG_USB_MR800 is not set
# CONFIG_RADIO_TEA5764 is not set
# CONFIG_RADIO_SAA7706H is not set
# CONFIG_RADIO_TEF6862 is not set
# CONFIG_RADIO_WL1273 is not set
#
# Texas Instruments WL128x FM driver (ST based)
#
CONFIG_DVB_MAX_ADAPTERS=8
CONFIG_DVB_DYNAMIC_MINORS=y
CONFIG_DVB_CAPTURE_DRIVERS=y
#
# Supported SAA7146 based PCI Adapters
#
# CONFIG_TTPCI_EEPROM is not set
# CONFIG_DVB_AV7110 is not set
# CONFIG_DVB_BUDGET_CORE is not set
#
# Supported USB Adapters
#
# CONFIG_DVB_USB is not set
# CONFIG_DVB_TTUSB_BUDGET is not set
# CONFIG_DVB_TTUSB_DEC is not set
# CONFIG_SMS_SIANO_MDTV is not set
#
# Supported FlexCopII (B2C2) Adapters
#
# CONFIG_DVB_B2C2_FLEXCOP is not set
#
# Supported BT878 Adapters
#
#
# Supported Pluto2 Adapters
#
# CONFIG_DVB_PLUTO2 is not set
#
# Supported SDMC DM1105 Adapters
#
# CONFIG_DVB_DM1105 is not set
#
# Supported Earthsoft PT1 Adapters
#
# CONFIG_DVB_PT1 is not set
#
# Supported Mantis Adapters
#
CONFIG_MANTIS_CORE=y
CONFIG_DVB_MANTIS=y
CONFIG_DVB_HOPPER=y
#
# Supported nGene Adapters
#
# CONFIG_DVB_NGENE is not set
#
# Supported DVB Frontends
#
# CONFIG_DVB_FE_CUSTOMISE is not set
#
# Multistandard (satellite) frontends
#
CONFIG_DVB_STB0899=y
CONFIG_DVB_STB6100=y
#
# DVB-S (satellite) frontends
#
CONFIG_DVB_STV0299=y
CONFIG_DVB_MB86A16=y
#
# DVB-T (terrestrial) frontends
#
CONFIG_DVB_ZL10353=y
#
# DVB-C (cable) frontends
#
CONFIG_DVB_TDA10021=y
CONFIG_DVB_TDA10023=y
#
# ATSC (North American/Korean Terrestrial/Cable DTV) frontends
#
#
# ISDB-T (terrestrial) frontends
#
#
# Digital terrestrial only tuners/PLL
#
CONFIG_DVB_PLL=y
#
# SEC control devices for DVB-S
#
CONFIG_DVB_LNBP21=y
CONFIG_DVB_TDA665x=y
#
# Tools to develop new frontends
#
# CONFIG_DVB_DUMMY_FE is not set
#
# Graphics support
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_VIA is not set
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=16
# CONFIG_VGA_SWITCHEROO is not set
CONFIG_DRM=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_STUB_POULSBO is not set
# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
# CONFIG_FB_DDC is not set
CONFIG_FB_BOOT_VESA_SUPPORT=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_FOREIGN_ENDIAN is not set
# CONFIG_FB_SYS_FOPS is not set
# CONFIG_FB_WMT_GE_ROPS is not set
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_UVESA is not set
CONFIG_FB_VESA=y
CONFIG_FB_EFI=y
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_TMIO is not set
# CONFIG_FB_UDL is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_FB_BROADSHEET is not set
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_LCD_CLASS_DEVICE=m
# CONFIG_LCD_L4F00242T03 is not set
# CONFIG_LCD_LMS283GF05 is not set
# CONFIG_LCD_LTV350QV is not set
# CONFIG_LCD_TDO24M is not set
# CONFIG_LCD_VGG2432A4 is not set
# CONFIG_LCD_PLATFORM is not set
# CONFIG_LCD_S6E63M0 is not set
# CONFIG_LCD_LD9040 is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_BACKLIGHT_GENERIC=y
# CONFIG_BACKLIGHT_PROGEAR is not set
# CONFIG_BACKLIGHT_APPLE is not set
# CONFIG_BACKLIGHT_SAHARA is not set
# CONFIG_BACKLIGHT_ADP8860 is not set
#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_VGACON_SOFT_SCROLLBACK is not set
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
CONFIG_FONTS=y
# CONFIG_FONT_8x8 is not set
CONFIG_FONT_8x16=y
# CONFIG_FONT_6x11 is not set
# CONFIG_FONT_7x14 is not set
# CONFIG_FONT_PEARL_8x8 is not set
# CONFIG_FONT_ACORN_8x8 is not set
# CONFIG_FONT_MINI_4x6 is not set
# CONFIG_FONT_SUN8x16 is not set
# CONFIG_FONT_SUN12x22 is not set
# CONFIG_FONT_10x18 is not set
CONFIG_LOGO=y
CONFIG_LOGO_LINUX_MONO=y
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y
CONFIG_SOUND=y
# CONFIG_SOUND_OSS_CORE is not set
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_HWDEP=y
CONFIG_SND_RAWMIDI=y
CONFIG_SND_JACK=y
# CONFIG_SND_SEQUENCER is not set
# CONFIG_SND_MIXER_OSS is not set
# CONFIG_SND_PCM_OSS is not set
# CONFIG_SND_HRTIMER is not set
CONFIG_SND_DYNAMIC_MINORS=y
# CONFIG_SND_SUPPORT_OLD_API is not set
CONFIG_SND_VERBOSE_PROCFS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_VMASTER=y
CONFIG_SND_DMA_SGBUF=y
# CONFIG_SND_RAWMIDI_SEQ is not set
# CONFIG_SND_OPL3_LIB_SEQ is not set
# CONFIG_SND_OPL4_LIB_SEQ is not set
# CONFIG_SND_SBAWE_SEQ is not set
# CONFIG_SND_EMU10K1_SEQ is not set
CONFIG_SND_MPU401_UART=y
CONFIG_SND_AC97_CODEC=y
CONFIG_SND_DRIVERS=y
# CONFIG_SND_PCSP is not set
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_ALOOP is not set
CONFIG_SND_SERIAL_U16550=y
CONFIG_SND_MPU401=y
# CONFIG_SND_AC97_POWER_SAVE is not set
CONFIG_SND_PCI=y
# CONFIG_SND_AD1889 is not set
# CONFIG_SND_ALS300 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ASIHPI is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AW2 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_OXYGEN is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS5530 is not set
# CONFIG_SND_CS5535AUDIO is not set
# CONFIG_SND_CTXFI is not set
# CONFIG_SND_DARLA20 is not set
# CONFIG_SND_GINA20 is not set
# CONFIG_SND_LAYLA20 is not set
# CONFIG_SND_DARLA24 is not set
# CONFIG_SND_GINA24 is not set
# CONFIG_SND_LAYLA24 is not set
# CONFIG_SND_MONA is not set
# CONFIG_SND_MIA is not set
# CONFIG_SND_ECHO3G is not set
# CONFIG_SND_INDIGO is not set
# CONFIG_SND_INDIGOIO is not set
# CONFIG_SND_INDIGODJ is not set
# CONFIG_SND_INDIGOIOX is not set
# CONFIG_SND_INDIGODJX is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_FM801 is not set
CONFIG_SND_HDA_INTEL=y
# CONFIG_SND_HDA_HWDEP is not set
# CONFIG_SND_HDA_INPUT_BEEP is not set
# CONFIG_SND_HDA_INPUT_JACK is not set
# CONFIG_SND_HDA_PATCH_LOADER is not set
CONFIG_SND_HDA_CODEC_REALTEK=y
CONFIG_SND_HDA_CODEC_ANALOG=y
CONFIG_SND_HDA_CODEC_SIGMATEL=y
CONFIG_SND_HDA_CODEC_VIA=y
CONFIG_SND_HDA_CODEC_HDMI=y
CONFIG_SND_HDA_CODEC_CIRRUS=y
CONFIG_SND_HDA_CODEC_CONEXANT=y
CONFIG_SND_HDA_CODEC_CA0110=y
CONFIG_SND_HDA_CODEC_CMEDIA=y
CONFIG_SND_HDA_CODEC_SI3054=y
CONFIG_SND_HDA_GENERIC=y
# CONFIG_SND_HDA_POWER_SAVE is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
CONFIG_SND_INTEL8X0=y
CONFIG_SND_INTEL8X0M=y
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_LX6464ES is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_PCXHR is not set
# CONFIG_SND_RIPTIDE is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VIRTUOSO is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_YMFPCI is not set
CONFIG_SND_SPI=y
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=y
# CONFIG_SND_USB_UA101 is not set
# CONFIG_SND_USB_USX2Y is not set
CONFIG_SND_USB_CAIAQ=y
CONFIG_SND_USB_CAIAQ_INPUT=y
# CONFIG_SND_USB_US122L is not set
# CONFIG_SND_USB_6FIRE is not set
CONFIG_SND_SOC=y
# CONFIG_SND_SOC_CACHE_LZO is not set
CONFIG_SND_SOC_I2C_AND_SPI=y
# CONFIG_SND_SOC_ALL_CODECS is not set
# CONFIG_SOUND_PRIME is not set
CONFIG_AC97_BUS=y
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
# CONFIG_HIDRAW is not set
#
# USB Input Devices
#
CONFIG_USB_HID=y
# CONFIG_HID_PID is not set
CONFIG_USB_HIDDEV=y
#
# Special HID drivers
#
# CONFIG_HID_3M_PCT is not set
CONFIG_HID_A4TECH=y
# CONFIG_HID_ACRUX is not set
CONFIG_HID_APPLE=y
CONFIG_HID_BELKIN=y
# CONFIG_HID_CANDO is not set
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
# CONFIG_HID_PRODIKEYS is not set
CONFIG_HID_CYPRESS=y
CONFIG_HID_DRAGONRISE=y
# CONFIG_DRAGONRISE_FF is not set
# CONFIG_HID_EMS_FF is not set
CONFIG_HID_EZKEY=y
# CONFIG_HID_KEYTOUCH is not set
CONFIG_HID_KYE=y
# CONFIG_HID_UCLOGIC is not set
# CONFIG_HID_WALTOP is not set
CONFIG_HID_GYRATION=y
CONFIG_HID_TWINHAN=y
CONFIG_HID_KENSINGTON=y
# CONFIG_HID_LCPOWER is not set
CONFIG_HID_LOGITECH=y
# CONFIG_LOGITECH_FF is not set
# CONFIG_LOGIRUMBLEPAD2_FF is not set
# CONFIG_LOGIG940_FF is not set
# CONFIG_LOGIWII_FF is not set
CONFIG_HID_MICROSOFT=y
# CONFIG_HID_MOSART is not set
CONFIG_HID_MONTEREY=y
# CONFIG_HID_MULTITOUCH is not set
CONFIG_HID_NTRIG=y
CONFIG_HID_ORTEK=y
CONFIG_HID_PANTHERLORD=y
# CONFIG_PANTHERLORD_FF is not set
CONFIG_HID_PETALYNX=y
# CONFIG_HID_PICOLCD is not set
# CONFIG_HID_QUANTA is not set
# CONFIG_HID_ROCCAT is not set
# CONFIG_HID_ROCCAT_ARVO is not set
# CONFIG_HID_ROCCAT_KONE is not set
# CONFIG_HID_ROCCAT_KONEPLUS is not set
# CONFIG_HID_ROCCAT_KOVAPLUS is not set
# CONFIG_HID_ROCCAT_PYRA is not set
CONFIG_HID_SAMSUNG=y
CONFIG_HID_SONY=y
# CONFIG_HID_STANTUM is not set
CONFIG_HID_SUNPLUS=y
CONFIG_HID_GREENASIA=y
# CONFIG_GREENASIA_FF is not set
CONFIG_HID_SMARTJOYPLUS=y
# CONFIG_SMARTJOYPLUS_FF is not set
CONFIG_HID_TOPSEED=y
CONFIG_HID_THRUSTMASTER=y
# CONFIG_THRUSTMASTER_FF is not set
CONFIG_HID_ZEROPLUS=y
# CONFIG_ZEROPLUS_FF is not set
# CONFIG_HID_ZYDACRON is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
#
# Miscellaneous USB options
#
# CONFIG_USB_DEVICEFS is not set
# CONFIG_USB_DEVICE_CLASS is not set
CONFIG_USB_DYNAMIC_MINORS=y
CONFIG_USB_SUSPEND=y
# CONFIG_USB_OTG is not set
# CONFIG_USB_MON is not set
CONFIG_USB_WUSB=y
CONFIG_USB_WUSB_CBAF=y
# CONFIG_USB_WUSB_CBAF_DEBUG is not set
#
# USB Host Controller Drivers
#
CONFIG_USB_C67X00_HCD=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_HCD_DEBUGGING=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
CONFIG_USB_OXU210HP_HCD=y
CONFIG_USB_ISP116X_HCD=y
CONFIG_USB_ISP1760_HCD=y
CONFIG_USB_ISP1362_HCD=y
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=y
# CONFIG_USB_U132_HCD is not set
CONFIG_USB_SL811_HCD=y
CONFIG_USB_R8A66597_HCD=y
CONFIG_USB_HWA_HCD=y
#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
CONFIG_USB_PRINTER=y
CONFIG_USB_WDM=y
CONFIG_USB_TMC=y
#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#
#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_REALTEK is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_USBAT=y
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
CONFIG_USB_STORAGE_ALAUDA=y
CONFIG_USB_STORAGE_ONETOUCH=y
CONFIG_USB_STORAGE_KARMA=y
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_STORAGE_ENE_UB6250 is not set
# CONFIG_USB_UAS is not set
CONFIG_USB_LIBUSUAL=y
#
# USB Imaging devices
#
CONFIG_USB_MDC800=y
CONFIG_USB_MICROTEK=y
#
# USB port drivers
#
CONFIG_USB_SERIAL=y
CONFIG_USB_SERIAL_CONSOLE=y
CONFIG_USB_EZUSB=y
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRCABLE is not set
# CONFIG_USB_SERIAL_ARK3116 is not set
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_CH341 is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP210X is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
# CONFIG_USB_SERIAL_FUNSOFT is not set
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_IUU is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_MOS7720 is not set
# CONFIG_USB_SERIAL_MOS7840 is not set
# CONFIG_USB_SERIAL_MOTOROLA is not set
# CONFIG_USB_SERIAL_NAVMAN is not set
CONFIG_USB_SERIAL_PL2303=y
# CONFIG_USB_SERIAL_OTI6858 is not set
# CONFIG_USB_SERIAL_QCAUX is not set
# CONFIG_USB_SERIAL_QUALCOMM is not set
# CONFIG_USB_SERIAL_SPCP8X5 is not set
# CONFIG_USB_SERIAL_HP4X is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_SAMBA is not set
# CONFIG_USB_SERIAL_SIEMENS_MPI is not set
# CONFIG_USB_SERIAL_SIERRAWIRELESS is not set
# CONFIG_USB_SERIAL_SYMBOL is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OPTION is not set
# CONFIG_USB_SERIAL_OMNINET is not set
# CONFIG_USB_SERIAL_OPTICON is not set
# CONFIG_USB_SERIAL_VIVOPAY_SERIAL is not set
# CONFIG_USB_SERIAL_ZIO is not set
# CONFIG_USB_SERIAL_SSU100 is not set
CONFIG_USB_SERIAL_DEBUG=y
#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=y
CONFIG_USB_EMI26=y
CONFIG_USB_ADUTUX=y
CONFIG_USB_SEVSEG=y
CONFIG_USB_RIO500=y
CONFIG_USB_LEGOTOWER=y
CONFIG_USB_LCD=y
CONFIG_USB_LED=y
CONFIG_USB_CYPRESS_CY7C63=y
CONFIG_USB_CYTHERM=y
CONFIG_USB_IDMOUSE=y
CONFIG_USB_FTDI_ELAN=y
CONFIG_USB_APPLEDISPLAY=y
CONFIG_USB_SISUSBVGA=y
# CONFIG_USB_SISUSBVGA_CON is not set
CONFIG_USB_LD=y
# CONFIG_USB_TRANCEVIBRATOR is not set
CONFIG_USB_IOWARRIOR=y
# CONFIG_USB_TEST is not set
CONFIG_USB_ISIGHTFW=y
# CONFIG_USB_YUREX is not set
# CONFIG_USB_GADGET is not set
#
# OTG and related infrastructure
#
# CONFIG_USB_GPIO_VBUS is not set
# CONFIG_NOP_USB_XCEIV is not set
CONFIG_UWB=y
CONFIG_UWB_HWA=y
CONFIG_UWB_WHCI=y
CONFIG_UWB_I1480U=y
CONFIG_MMC=y
# CONFIG_MMC_DEBUG is not set
# CONFIG_MMC_UNSAFE_RESUME is not set
# CONFIG_MMC_CLKGATE is not set
#
# MMC/SD/SDIO Card Drivers
#
CONFIG_MMC_BLOCK=y
CONFIG_MMC_BLOCK_MINORS=8
CONFIG_MMC_BLOCK_BOUNCE=y
CONFIG_SDIO_UART=y
# CONFIG_MMC_TEST is not set
#
# MMC/SD/SDIO Host Controller Drivers
#
CONFIG_MMC_SDHCI=y
CONFIG_MMC_SDHCI_PCI=y
# CONFIG_MMC_RICOH_MMC is not set
CONFIG_MMC_SDHCI_PLTFM=y
CONFIG_MMC_WBSD=y
CONFIG_MMC_TIFM_SD=y
CONFIG_MMC_SPI=y
CONFIG_MMC_CB710=y
CONFIG_MMC_VIA_SDMMC=y
# CONFIG_MMC_USHC is not set
CONFIG_MEMSTICK=y
CONFIG_MEMSTICK_DEBUG=y
#
# MemoryStick drivers
#
# CONFIG_MEMSTICK_UNSAFE_RESUME is not set
CONFIG_MSPRO_BLOCK=y
#
# MemoryStick Host Controller Drivers
#
CONFIG_MEMSTICK_TIFM_MS=y
CONFIG_MEMSTICK_JMICRON_38X=y
# CONFIG_MEMSTICK_R592 is not set
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
#
# LED drivers
#
# CONFIG_LEDS_LM3530 is not set
# CONFIG_LEDS_ALIX2 is not set
# CONFIG_LEDS_PCA9532 is not set
# CONFIG_LEDS_GPIO is not set
# CONFIG_LEDS_LP3944 is not set
# CONFIG_LEDS_LP5521 is not set
# CONFIG_LEDS_LP5523 is not set
# CONFIG_LEDS_CLEVO_MAIL is not set
# CONFIG_LEDS_PCA955X is not set
# CONFIG_LEDS_DAC124S085 is not set
# CONFIG_LEDS_BD2802 is not set
# CONFIG_LEDS_LT3593 is not set
# CONFIG_LEDS_DELL_NETBOOKS is not set
CONFIG_LEDS_TRIGGERS=y
#
# LED Triggers
#
CONFIG_LEDS_TRIGGER_TIMER=y
CONFIG_LEDS_TRIGGER_HEARTBEAT=y
# CONFIG_LEDS_TRIGGER_BACKLIGHT is not set
# CONFIG_LEDS_TRIGGER_GPIO is not set
# CONFIG_LEDS_TRIGGER_DEFAULT_ON is not set
#
# iptables trigger is under Netfilter config (LED target)
#
# CONFIG_NFC_DEVICES is not set
# CONFIG_ACCESSIBILITY is not set
CONFIG_INFINIBAND=y
CONFIG_INFINIBAND_USER_MAD=y
CONFIG_INFINIBAND_USER_ACCESS=y
CONFIG_INFINIBAND_USER_MEM=y
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_MTHCA=y
CONFIG_INFINIBAND_MTHCA_DEBUG=y
CONFIG_INFINIBAND_IPATH=y
CONFIG_INFINIBAND_QIB=y
CONFIG_INFINIBAND_AMSO1100=y
CONFIG_INFINIBAND_AMSO1100_DEBUG=y
CONFIG_INFINIBAND_CXGB3=y
CONFIG_INFINIBAND_CXGB3_DEBUG=y
CONFIG_INFINIBAND_CXGB4=y
CONFIG_MLX4_INFINIBAND=y
CONFIG_INFINIBAND_NES=y
CONFIG_INFINIBAND_NES_DEBUG=y
CONFIG_INFINIBAND_IPOIB=y
CONFIG_INFINIBAND_IPOIB_CM=y
CONFIG_INFINIBAND_IPOIB_DEBUG=y
CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=y
CONFIG_INFINIBAND_SRP=y
CONFIG_INFINIBAND_ISER=y
CONFIG_EDAC=y
#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
# CONFIG_EDAC_DECODE_MCE is not set
# CONFIG_EDAC_MM_EDAC is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set
#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set
#
# I2C RTC drivers
#
# CONFIG_RTC_DRV_DS1307 is not set
# CONFIG_RTC_DRV_DS1374 is not set
# CONFIG_RTC_DRV_DS1672 is not set
# CONFIG_RTC_DRV_DS3232 is not set
# CONFIG_RTC_DRV_MAX6900 is not set
# CONFIG_RTC_DRV_RS5C372 is not set
# CONFIG_RTC_DRV_ISL1208 is not set
# CONFIG_RTC_DRV_ISL12022 is not set
# CONFIG_RTC_DRV_X1205 is not set
# CONFIG_RTC_DRV_PCF8563 is not set
# CONFIG_RTC_DRV_PCF8583 is not set
# CONFIG_RTC_DRV_M41T80 is not set
# CONFIG_RTC_DRV_BQ32K is not set
# CONFIG_RTC_DRV_S35390A is not set
# CONFIG_RTC_DRV_FM3130 is not set
# CONFIG_RTC_DRV_RX8581 is not set
# CONFIG_RTC_DRV_RX8025 is not set
#
# SPI RTC drivers
#
# CONFIG_RTC_DRV_M41T94 is not set
# CONFIG_RTC_DRV_DS1305 is not set
# CONFIG_RTC_DRV_DS1390 is not set
# CONFIG_RTC_DRV_MAX6902 is not set
# CONFIG_RTC_DRV_R9701 is not set
# CONFIG_RTC_DRV_RS5C348 is not set
# CONFIG_RTC_DRV_DS3234 is not set
# CONFIG_RTC_DRV_PCF2123 is not set
#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=y
# CONFIG_RTC_DRV_DS1286 is not set
# CONFIG_RTC_DRV_DS1511 is not set
# CONFIG_RTC_DRV_DS1553 is not set
# CONFIG_RTC_DRV_DS1742 is not set
# CONFIG_RTC_DRV_STK17TA8 is not set
# CONFIG_RTC_DRV_M48T86 is not set
# CONFIG_RTC_DRV_M48T35 is not set
# CONFIG_RTC_DRV_M48T59 is not set
# CONFIG_RTC_DRV_MSM6242 is not set
# CONFIG_RTC_DRV_BQ4802 is not set
# CONFIG_RTC_DRV_RP5C01 is not set
# CONFIG_RTC_DRV_V3020 is not set
#
# on-CPU RTC drivers
#
CONFIG_DMADEVICES=y
# CONFIG_DMADEVICES_DEBUG is not set
#
# DMA Devices
#
# CONFIG_INTEL_MID_DMAC is not set
# CONFIG_INTEL_IOATDMA is not set
# CONFIG_TIMB_DMA is not set
# CONFIG_PCH_DMA is not set
CONFIG_DMA_ENGINE=y
#
# DMA Clients
#
# CONFIG_NET_DMA is not set
# CONFIG_ASYNC_TX_DMA is not set
# CONFIG_DMATEST is not set
CONFIG_AUXDISPLAY=y
CONFIG_UIO=y
# CONFIG_UIO_CIF is not set
CONFIG_UIO_PDRV=y
CONFIG_UIO_PDRV_GENIRQ=y
# CONFIG_UIO_AEC is not set
# CONFIG_UIO_SERCOS3 is not set
# CONFIG_UIO_PCI_GENERIC is not set
# CONFIG_UIO_NETX is not set
# CONFIG_STAGING is not set
CONFIG_X86_PLATFORM_DEVICES=y
# CONFIG_ACER_WMI is not set
# CONFIG_ACERHDF is not set
# CONFIG_ASUS_LAPTOP is not set
# CONFIG_DELL_WMI is not set
# CONFIG_DELL_WMI_AIO is not set
# CONFIG_FUJITSU_LAPTOP is not set
# CONFIG_HP_ACCEL is not set
# CONFIG_HP_WMI is not set
# CONFIG_PANASONIC_LAPTOP is not set
# CONFIG_THINKPAD_ACPI is not set
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_INTEL_MENLOW is not set
# CONFIG_EEEPC_LAPTOP is not set
# CONFIG_ASUS_WMI is not set
CONFIG_ACPI_WMI=y
# CONFIG_MSI_WMI is not set
# CONFIG_ACPI_ASUS is not set
# CONFIG_TOPSTAR_LAPTOP is not set
# CONFIG_ACPI_TOSHIBA is not set
# CONFIG_TOSHIBA_BT_RFKILL is not set
# CONFIG_ACPI_CMPC is not set
# CONFIG_INTEL_IPS is not set
# CONFIG_IBM_RTL is not set
# CONFIG_XO15_EBOOK is not set
#
# Firmware Drivers
#
CONFIG_EDD=y
# CONFIG_EDD_OFF is not set
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_EFI_VARS=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
CONFIG_DMIID=y
CONFIG_DMI_SYSFS=y
CONFIG_ISCSI_IBFT_FIND=y
CONFIG_ISCSI_IBFT=y
# CONFIG_SIGMA is not set
#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT2_FS_XIP=y
CONFIG_EXT3_FS=y
# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_FS_XIP=y
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_JBD2=y
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
CONFIG_REISERFS_PROC_INFO=y
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
CONFIG_REISERFS_FS_SECURITY=y
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
CONFIG_GFS2_FS=y
# CONFIG_GFS2_FS_LOCKING_DLM is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
CONFIG_FS_POSIX_ACL=y
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
# CONFIG_FANOTIFY is not set
CONFIG_QUOTA=y
# CONFIG_QUOTA_NETLINK_INTERFACE is not set
CONFIG_PRINT_QUOTA_WARNING=y
# CONFIG_QUOTA_DEBUG is not set
# CONFIG_QFMT_V1 is not set
# CONFIG_QFMT_V2 is not set
CONFIG_QUOTACTL=y
CONFIG_QUOTACTL_COMPAT=y
CONFIG_AUTOFS4_FS=y
# CONFIG_FUSE_FS is not set
#
# Caches
#
CONFIG_FSCACHE=y
CONFIG_FSCACHE_STATS=y
CONFIG_FSCACHE_HISTOGRAM=y
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=y
# CONFIG_CACHEFILES_DEBUG is not set
CONFIG_CACHEFILES_HISTOGRAM=y
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=y
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_VMCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_CONFIGFS_FS=y
CONFIG_MISC_FILESYSTEMS=y
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_ECRYPT_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_LOGFS is not set
CONFIG_CRAMFS=y
CONFIG_SQUASHFS=y
# CONFIG_SQUASHFS_XATTR is not set
# CONFIG_SQUASHFS_LZO is not set
# CONFIG_SQUASHFS_XZ is not set
CONFIG_SQUASHFS_EMBEDDED=y
CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3
CONFIG_VXFS_FS=y
CONFIG_MINIX_FS=y
CONFIG_OMFS_FS=y
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
CONFIG_ROMFS_FS=y
CONFIG_ROMFS_BACKED_BY_BLOCK=y
CONFIG_ROMFS_ON_BLOCK=y
CONFIG_PSTORE=y
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
# CONFIG_EXOFS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V3_ACL is not set
CONFIG_NFS_V4=y
# CONFIG_NFS_V4_1 is not set
CONFIG_ROOT_NFS=y
CONFIG_NFS_FSCACHE=y
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
# CONFIG_NFS_USE_NEW_IDMAPPER is not set
# CONFIG_NFSD is not set
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_SUNRPC_XPRT_RDMA=y
# CONFIG_CEPH_FS is not set
CONFIG_CIFS=y
CONFIG_CIFS_STATS=y
CONFIG_CIFS_STATS2=y
CONFIG_CIFS_WEAK_PW_HASH=y
# CONFIG_CIFS_UPCALL is not set
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DFS_UPCALL is not set
# CONFIG_CIFS_FSCACHE is not set
CONFIG_CIFS_EXPERIMENTAL=y
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
# CONFIG_MINIX_SUBPARTITION is not set
CONFIG_SOLARIS_X86_PARTITION=y
# CONFIG_UNIXWARE_DISKLABEL is not set
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_BASE=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=y
CONFIG_DLM=y
# CONFIG_DLM_DEBUG is not set
#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_PRINTK_TIME=y
CONFIG_DEFAULT_MESSAGE_LOGLEVEL=4
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=2048
CONFIG_MAGIC_SYSRQ=y
CONFIG_STRIP_ASM_SYMS=y
# CONFIG_UNUSED_SYMBOLS is not set
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
CONFIG_LOCKUP_DETECTOR=y
CONFIG_HARDLOCKUP_DETECTOR=y
# CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=0
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_SLUB_STATS is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
# CONFIG_PROVE_RCU is not set
# CONFIG_SPARSE_RCU_POINTER is not set
CONFIG_LOCKDEP=y
CONFIG_LOCK_STAT=y
CONFIG_DEBUG_LOCKDEP=y
CONFIG_TRACE_IRQFLAGS=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_REDUCED=y
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_WRITECOUNT is not set
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_DEBUG_LIST=y
# CONFIG_TEST_LIST_SORT is not set
CONFIG_DEBUG_SG=y
CONFIG_DEBUG_NOTIFIERS=y
# CONFIG_DEBUG_CREDENTIALS is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_CPU_STALL_DETECTOR is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
CONFIG_LKDTM=m
# CONFIG_CPU_NOTIFIER_ERROR_INJECT is not set
CONFIG_FAULT_INJECTION=y
CONFIG_FAILSLAB=y
CONFIG_FAIL_PAGE_ALLOC=y
CONFIG_FAIL_MAKE_REQUEST=y
# CONFIG_FAIL_IO_TIMEOUT is not set
CONFIG_FAULT_INJECTION_DEBUG_FS=y
CONFIG_LATENCYTOP=y
CONFIG_SYSCTL_SYSCALL_CHECK=y
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FTRACE_NMI_ENTER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_RING_BUFFER=y
CONFIG_FTRACE_NMI_ENTER=y
CONFIG_EVENT_TRACING=y
CONFIG_EVENT_POWER_TRACING_DEPRECATED=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_RING_BUFFER_ALLOW_SWAP=y
CONFIG_TRACING=y
CONFIG_GENERIC_TRACER=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
CONFIG_IRQSOFF_TRACER=y
CONFIG_SCHED_TRACER=y
CONFIG_FTRACE_SYSCALLS=y
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
# CONFIG_PROFILE_ALL_BRANCHES is not set
CONFIG_STACK_TRACER=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_KPROBE_EVENT=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_FUNCTION_PROFILER=y
CONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_STARTUP_TEST is not set
CONFIG_MMIOTRACE=y
# CONFIG_MMIOTRACE_TEST is not set
# CONFIG_RING_BUFFER_BENCHMARK is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DMA_API_DEBUG is not set
CONFIG_ATOMIC64_SELFTEST=y
# CONFIG_ASYNC_RAID6_TEST is not set
CONFIG_SAMPLES=y
# CONFIG_SAMPLE_TRACEPOINTS is not set
CONFIG_SAMPLE_TRACE_EVENTS=m
CONFIG_SAMPLE_KOBJECT=m
# CONFIG_SAMPLE_KPROBES is not set
# CONFIG_SAMPLE_HW_BREAKPOINT is not set
# CONFIG_SAMPLE_KFIFO is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
# CONFIG_TEST_KSTRTOX is not set
# CONFIG_STRICT_DEVMEM is not set
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_PRINTK_DBGP=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
CONFIG_X86_PTDUMP=y
# CONFIG_DEBUG_RODATA is not set
# CONFIG_DEBUG_SET_MODULE_RONX is not set
# CONFIG_DEBUG_NX_TEST is not set
# CONFIG_IOMMU_DEBUG is not set
# CONFIG_IOMMU_STRESS is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
# CONFIG_X86_DECODER_SELFTEST is not set
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
CONFIG_DEBUG_BOOT_PARAMS=y
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set
#
# Security options
#
CONFIG_KEYS=y
# CONFIG_TRUSTED_KEYS is not set
# CONFIG_KEYS_DEBUG_PROC_KEYS is not set
# CONFIG_SECURITY_DMESG_RESTRICT is not set
# CONFIG_SECURITY is not set
CONFIG_SECURITYFS=y
# CONFIG_INTEL_TXT is not set
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
CONFIG_XOR_BLOCKS=y
CONFIG_ASYNC_CORE=y
CONFIG_ASYNC_MEMCPY=y
CONFIG_ASYNC_XOR=y
CONFIG_ASYNC_PQ=y
CONFIG_ASYNC_RAID6_RECOV=y
CONFIG_CRYPTO=y
#
# Crypto core or helper
#
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_PCOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
CONFIG_CRYPTO_MANAGER_DISABLE_TESTS=y
# CONFIG_CRYPTO_GF128MUL is not set
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_PCRYPT is not set
CONFIG_CRYPTO_WORKQUEUE=y
# CONFIG_CRYPTO_CRYPTD is not set
# CONFIG_CRYPTO_AUTHENC is not set
# CONFIG_CRYPTO_TEST is not set
#
# Authenticated Encryption with Associated Data
#
# CONFIG_CRYPTO_CCM is not set
# CONFIG_CRYPTO_GCM is not set
# CONFIG_CRYPTO_SEQIV is not set
#
# Block modes
#
CONFIG_CRYPTO_CBC=y
# CONFIG_CRYPTO_CTR is not set
# CONFIG_CRYPTO_CTS is not set
# CONFIG_CRYPTO_ECB is not set
# CONFIG_CRYPTO_LRW is not set
# CONFIG_CRYPTO_PCBC is not set
# CONFIG_CRYPTO_XTS is not set
#
# Hash modes
#
CONFIG_CRYPTO_HMAC=y
# CONFIG_CRYPTO_XCBC is not set
# CONFIG_CRYPTO_VMAC is not set
#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
# CONFIG_CRYPTO_CRC32C_INTEL is not set
# CONFIG_CRYPTO_GHASH is not set
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=y
# CONFIG_CRYPTO_RMD128 is not set
# CONFIG_CRYPTO_RMD160 is not set
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not set
CONFIG_CRYPTO_SHA1=y
# CONFIG_CRYPTO_SHA256 is not set
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_TGR192 is not set
# CONFIG_CRYPTO_WP512 is not set
# CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL is not set
#
# Ciphers
#
CONFIG_CRYPTO_AES=y
# CONFIG_CRYPTO_AES_X86_64 is not set
# CONFIG_CRYPTO_AES_NI_INTEL is not set
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_ARC4=y
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_CAMELLIA is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
CONFIG_CRYPTO_DES=y
# CONFIG_CRYPTO_FCRYPT is not set
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_SALSA20 is not set
# CONFIG_CRYPTO_SALSA20_X86_64 is not set
# CONFIG_CRYPTO_SEED is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_TWOFISH_X86_64 is not set
#
# Compression
#
# CONFIG_CRYPTO_DEFLATE is not set
# CONFIG_CRYPTO_ZLIB is not set
# CONFIG_CRYPTO_LZO is not set
#
# Random Number Generation
#
# CONFIG_CRYPTO_ANSI_CPRNG is not set
# CONFIG_CRYPTO_USER_API_HASH is not set
# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
CONFIG_CRYPTO_HW=y
# CONFIG_CRYPTO_DEV_PADLOCK is not set
# CONFIG_CRYPTO_DEV_HIFN_795X is not set
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_APIC_ARCHITECTURE=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_KVM_INTEL=y
CONFIG_KVM_AMD=y
# CONFIG_KVM_MMU_AUDIT is not set
# CONFIG_VHOST_NET is not set
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_BINARY_PRINTF=y
#
# Library routines
#
CONFIG_RAID6_PQ=y
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=y
CONFIG_CRC32=y
CONFIG_CRC7=y
CONFIG_LIBCRC32C=y
CONFIG_ZLIB_INFLATE=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_XZ_DEC=y
CONFIG_XZ_DEC_X86=y
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
CONFIG_XZ_DEC_ARM=y
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
# CONFIG_XZ_DEC_TEST is not set
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_DECOMPRESS_LZO=y
CONFIG_GENERIC_ALLOCATOR=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_CPU_RMAP=y
CONFIG_NLATTR=y
# CONFIG_AVERAGE is not set
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 9:20 ` Paul E. McKenney
2011-05-12 17:31 ` Yinghai Lu
@ 2011-05-12 21:36 ` Yinghai Lu
2011-05-13 1:28 ` Yinghai Lu
2011-05-13 21:08 ` Yinghai Lu
1 sibling, 2 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-12 21:36 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/12/2011 02:20 AM, Paul E. McKenney wrote:
> On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
>> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
>>> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
>>>>
>>>> * Yinghai Lu <yinghai@kernel.org> wrote:
>>>>
>>>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
>>>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
>>>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>>>>> Date: Tue Sep 7 10:38:22 2010 -0700
>>>>>
>>>>> rcu: Decrease memory-barrier usage based on semi-formal proof
>>>>
>>>> Find below an (untested!) attempt at reverting it for debugging purposes: could
>>>> you please try it, does your system now boot up fine?
>>>>
>>>> Thanks,
>>>>
>>>> Ingo
>>>>
>>>
>>> yes, reverted manually that commit fix the problem.
>>
>> on system with 8 sockets westmere-ex
>>
>> it seems other commits after that commit contribute some delay too.
>>
>> [ 32.240739] cpu_dev_init done
>> [ 73.587288] memory_dev_init done
>
> I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
> will chase down the delay.
>
it seems still need to revert following one in addition e59fb3120becfb36b22ddb8bd27d065d3cdca499.
[root@mpk14-2404-239-158 linux-2.6]# git bisect good
a26ac2455ffcf3be5c6ef92bc6df7182700f2114 is the first bad commit
commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114
Author: Paul E. McKenney <paul.mckenney@linaro.org>
Date: Wed Jan 12 14:10:23 2011 -0800
rcu: move TREE_RCU from softirq to kthread
If RCU priority boosting is to be meaningful, callback invocation must
be boosted in addition to preempted RCU readers. Otherwise, in presence
of CPU real-time threads, the grace period ends, but the callbacks don't
get invoked. If the callbacks don't get invoked, the associated memory
doesn't get freed, so the system is still subject to OOM.
But it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit
moves the callback invocations to a kthread, which can be boosted easily.
Also add comments and properly synchronized all accesses to
rcu_cpu_kthread_task, as suggested by Lai Jiangshan.
Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
:040000 040000 e40306ac6405952c1d387325a98588442209abe8 efe9ea2f408c62daaccf49e6d1339dff3a74f049 M Documentation
:040000 040000 8f9e7a8fa3a728d4ae58e2efb8ada7cf08aed00e 9b44deba45ba905c5d9b3cc314812f0ba3f7e639 M include
:040000 040000 4b10b719a2d56ed4bc796a9f43775732bb5ff144 4db269277ccf607e1a6a7d7f4c2a7cf8d592d46a M kernel
:040000 040000 881f102e6831381beed016ed240d690f6a2ccd5e 57d2fc6f84e47394c116bc617a9a0ef9b8b6dbd4 M tools
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 21:36 ` Yinghai Lu
@ 2011-05-13 1:28 ` Yinghai Lu
2011-05-13 8:42 ` Ingo Molnar
2011-05-13 21:08 ` Yinghai Lu
1 sibling, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-13 1:28 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/12/2011 02:36 PM, Yinghai Lu wrote:
> On 05/12/2011 02:20 AM, Paul E. McKenney wrote:
>> On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
>>> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
>>>> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
>>>>>
>>>>> * Yinghai Lu <yinghai@kernel.org> wrote:
>>>>>
>>>>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
>>>>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
>>>>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>>>>>> Date: Tue Sep 7 10:38:22 2010 -0700
>>>>>>
>>>>>> rcu: Decrease memory-barrier usage based on semi-formal proof
>>>>>
>>>>> Find below an (untested!) attempt at reverting it for debugging purposes: could
>>>>> you please try it, does your system now boot up fine?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ingo
>>>>>
>>>>
>>>> yes, reverted manually that commit fix the problem.
>>>
>>> on system with 8 sockets westmere-ex
>>>
>>> it seems other commits after that commit contribute some delay too.
>>>
>>> [ 32.240739] cpu_dev_init done
>>> [ 73.587288] memory_dev_init done
>>
>> I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
>> will chase down the delay.
>>
>
> it seems still need to revert following one in addition e59fb3120becfb36b22ddb8bd27d065d3cdca499.
>
> [root@mpk14-2404-239-158 linux-2.6]# git bisect good
> a26ac2455ffcf3be5c6ef92bc6df7182700f2114 is the first bad commit
> commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> Author: Paul E. McKenney <paul.mckenney@linaro.org>
> Date: Wed Jan 12 14:10:23 2011 -0800
>
> rcu: move TREE_RCU from softirq to kthread
>
> If RCU priority boosting is to be meaningful, callback invocation must
> be boosted in addition to preempted RCU readers. Otherwise, in presence
> of CPU real-time threads, the grace period ends, but the callbacks don't
> get invoked. If the callbacks don't get invoked, the associated memory
> doesn't get freed, so the system is still subject to OOM.
>
> But it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit
> moves the callback invocations to a kthread, which can be boosted easily.
>
> Also add comments and properly synchronized all accesses to
> rcu_cpu_kthread_task, as suggested by Lai Jiangshan.
>
> Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Reviewed-by: Josh Triplett <josh@joshtriplett.org>
>
> :040000 040000 e40306ac6405952c1d387325a98588442209abe8 efe9ea2f408c62daaccf49e6d1339dff3a74f049 M Documentation
> :040000 040000 8f9e7a8fa3a728d4ae58e2efb8ada7cf08aed00e 9b44deba45ba905c5d9b3cc314812f0ba3f7e639 M include
> :040000 040000 4b10b719a2d56ed4bc796a9f43775732bb5ff144 4db269277ccf607e1a6a7d7f4c2a7cf8d592d46a M kernel
> :040000 040000 881f102e6831381beed016ed240d690f6a2ccd5e 57d2fc6f84e47394c116bc617a9a0ef9b8b6dbd4 M tools
more info:
if only reverting: e59fb3120becfb36b22ddb8bd27d065d3cdca499
when compiled with gcc from opensuse 11.3
[ 32.579585] cpu_dev_init done
[ 81.203193] memory_dev_init done
but when compile that with gcc from fedora 14, then will less delay.
[ 32.404140] cpu_dev_init done
[ 37.602986] memory_dev_init done
gcc version on opensuse 11.3
gcc (SUSE Linux) 4.5.0 20100604 [gcc-4_5-branch revision 160292]
gcc version on fedora 14:
gcc (GCC) 4.5.1 20100924 (Red Hat 4.5.1-4)
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 1:28 ` Yinghai Lu
@ 2011-05-13 8:42 ` Ingo Molnar
2011-05-13 12:19 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-13 8:42 UTC (permalink / raw)
To: Yinghai Lu; +Cc: paulmck, linux-kernel
* Yinghai Lu <yinghai@kernel.org> wrote:
> more info:
> if only reverting: e59fb3120becfb36b22ddb8bd27d065d3cdca499
>
> when compiled with gcc from opensuse 11.3
> [ 32.579585] cpu_dev_init done
> [ 81.203193] memory_dev_init done
>
> but when compile that with gcc from fedora 14, then will less delay.
> [ 32.404140] cpu_dev_init done
> [ 37.602986] memory_dev_init done
>
> gcc version on opensuse 11.3
> gcc (SUSE Linux) 4.5.0 20100604 [gcc-4_5-branch revision 160292]
>
> gcc version on fedora 14:
> gcc (GCC) 4.5.1 20100924 (Red Hat 4.5.1-4)
Would it be possible to narrow down it a bit which commit causes the boot
slowdown?
known bad : 11c476f31a0f: net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
known good: 0ee5623f9a6e: Linux 2.6.39-rc6
There's 76 commits inbetween so an about 6-step bisection ought to be able to
narrow the source of regression down quite a bit ...
The complication with that bisection will be the need to revert the other
broken commit you found:
e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
Which is unfortunately very early in the series, so you'd have to revert this
at almost every step of the bisection ...
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 8:42 ` Ingo Molnar
@ 2011-05-13 12:19 ` Ingo Molnar
2011-05-13 13:04 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-13 12:19 UTC (permalink / raw)
To: Yinghai Lu; +Cc: paulmck, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 725 bytes --]
btw., i'm seeing a boot hang too, with the attached config:
PM: Adding info for platform:serial8250
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
device: 'ttyS0': device_add
PM: Adding info for No Bus:ttyS0
INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=6002 jiffies)
INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=24034 jiffies)
INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=42066 jiffies)
INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=6002 jiffies)
INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=24034 jiffies)
(and the stall warnings go on forever)
Thanks,
Ingo
[-- Attachment #2: config --]
[-- Type: text/plain, Size: 67118 bytes --]
#
# Automatically generated make config: don't edit
# Linux/i386 2.6.39-rc7 Kernel Configuration
# Fri May 13 15:37:17 2011
#
# CONFIG_64BIT is not set
CONFIG_X86_32=y
# CONFIG_X86_64 is not set
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf32-i386"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_GPIO=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
# CONFIG_GENERIC_TIME_VSYSCALL is not set
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
# CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
# CONFIG_ZONE_DMA32 is not set
CONFIG_ARCH_POPULATES_NODE_MAP=y
# CONFIG_AUDIT_ARCH is not set
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_X86_32_SMP=y
CONFIG_X86_HT=y
CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-ecx -fcall-saved-edx"
CONFIG_KTIME_SCALAR=y
CONFIG_ARCH_CPU_PROBE_RELEASE=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_IRQ_WORK=y
CONFIG_IRQ_WORK=y
#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_SWAP is not set
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_FHANDLE is not set
CONFIG_TASKSTATS=y
# CONFIG_TASK_DELAY_ACCT is not set
CONFIG_TASK_XACCT=y
# CONFIG_TASK_IO_ACCOUNTING is not set
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_AUDIT_WATCH=y
CONFIG_AUDIT_TREE=y
CONFIG_HAVE_GENERIC_HARDIRQS=y
#
# IRQ subsystem
#
CONFIG_GENERIC_HARDIRQS=y
CONFIG_HAVE_SPARSE_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
# CONFIG_PREEMPT_RCU is not set
CONFIG_RCU_TRACE=y
CONFIG_RCU_FANOUT=32
CONFIG_RCU_FANOUT_EXACT=y
# CONFIG_RCU_FAST_NO_HZ is not set
CONFIG_TREE_RCU_TRACE=y
# CONFIG_IKCONFIG is not set
CONFIG_LOG_BUF_SHIFT=20
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
# CONFIG_CPUSETS is not set
# CONFIG_CGROUP_CPUACCT is not set
CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEM_RES_CTLR=y
# CONFIG_CGROUP_PERF is not set
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
# CONFIG_BLK_CGROUP is not set
CONFIG_NAMESPACES=y
# CONFIG_UTS_NS is not set
CONFIG_IPC_NS=y
# CONFIG_USER_NS is not set
# CONFIG_PID_NS is not set
CONFIG_NET_NS=y
# CONFIG_SCHED_AUTOGROUP is not set
CONFIG_SCHED_TTWU_QUEUE=y
CONFIG_MM_OWNER=y
CONFIG_SYSFS_DEPRECATED=y
# CONFIG_SYSFS_DEPRECATED_V2 is not set
# CONFIG_RELAY is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
# CONFIG_EXPERT is not set
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y
CONFIG_PERF_USE_VMALLOC=y
#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
CONFIG_PERF_COUNTERS=y
CONFIG_DEBUG_PERF_USE_VMALLOC=y
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
CONFIG_SLUB_DEBUG=y
CONFIG_COMPAT_BRK=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_PROFILING is not set
CONFIG_HAVE_OPROFILE=y
# CONFIG_KPROBES is not set
# CONFIG_JUMP_LABEL is not set
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
#
# GCOV-based kernel profiling
#
CONFIG_HAVE_GENERIC_DMA_COHERENT=y
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
# CONFIG_MODULE_UNLOAD is not set
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
# CONFIG_LBDAF is not set
CONFIG_BLK_DEV_BSG=y
CONFIG_BLK_DEV_INTEGRITY=y
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=m
# CONFIG_IOSCHED_CFQ is not set
CONFIG_DEFAULT_NOOP=y
CONFIG_DEFAULT_IOSCHED="noop"
# CONFIG_INLINE_SPIN_TRYLOCK is not set
# CONFIG_INLINE_SPIN_TRYLOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK is not set
# CONFIG_INLINE_SPIN_LOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK_IRQ is not set
# CONFIG_INLINE_SPIN_LOCK_IRQSAVE is not set
# CONFIG_INLINE_SPIN_UNLOCK is not set
# CONFIG_INLINE_SPIN_UNLOCK_BH is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQ is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_READ_TRYLOCK is not set
# CONFIG_INLINE_READ_LOCK is not set
# CONFIG_INLINE_READ_LOCK_BH is not set
# CONFIG_INLINE_READ_LOCK_IRQ is not set
# CONFIG_INLINE_READ_LOCK_IRQSAVE is not set
# CONFIG_INLINE_READ_UNLOCK is not set
# CONFIG_INLINE_READ_UNLOCK_BH is not set
# CONFIG_INLINE_READ_UNLOCK_IRQ is not set
# CONFIG_INLINE_READ_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_WRITE_TRYLOCK is not set
# CONFIG_INLINE_WRITE_LOCK is not set
# CONFIG_INLINE_WRITE_LOCK_BH is not set
# CONFIG_INLINE_WRITE_LOCK_IRQ is not set
# CONFIG_INLINE_WRITE_LOCK_IRQSAVE is not set
# CONFIG_INLINE_WRITE_UNLOCK is not set
# CONFIG_INLINE_WRITE_UNLOCK_BH is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQ is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE is not set
# CONFIG_MUTEX_SPIN_ON_OWNER is not set
CONFIG_FREEZER=y
#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_X86_MPPARSE=y
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_EXTENDED_PLATFORM is not set
CONFIG_X86_32_IRIS=m
CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
CONFIG_NO_BOOTMEM=y
# CONFIG_MEMTEST is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
CONFIG_M686=y
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MELAN is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_INTERNODE_CACHE_SHIFT=5
CONFIG_X86_CMPXCHG=y
CONFIG_CMPXCHG_LOCAL=y
CONFIG_X86_L1_CACHE_SHIFT=5
CONFIG_X86_XADD=y
# CONFIG_X86_PPRO_FENCE is not set
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=5
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_CYRIX_32=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_CPU_SUP_TRANSMETA_32=y
CONFIG_CPU_SUP_UMC_32=y
CONFIG_HPET_TIMER=y
CONFIG_DMI=y
# CONFIG_IOMMU_HELPER is not set
# CONFIG_IOMMU_API is not set
CONFIG_NR_CPUS=8
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
# CONFIG_X86_MCE is not set
CONFIG_VM86=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_X86_REBOOTFIXUPS is not set
CONFIG_MICROCODE=m
# CONFIG_MICROCODE_INTEL is not set
# CONFIG_MICROCODE_AMD is not set
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=m
# CONFIG_X86_CPUID is not set
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_HIGHMEM=y
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
# CONFIG_ARCH_DMA_ADDR_T_64BIT is not set
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=999999
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
# CONFIG_PHYS_ADDR_T_64BIT is not set
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
CONFIG_HIGHPTE=y
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
# CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK is not set
CONFIG_X86_RESERVE_LOW=64
CONFIG_MATH_EMULATION=y
CONFIG_MTRR=y
# CONFIG_MTRR_SANITIZER is not set
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
# CONFIG_EFI is not set
# CONFIG_SECCOMP is not set
CONFIG_CC_STACKPROTECTOR=y
CONFIG_HZ_100=y
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=100
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x1000000
CONFIG_RELOCATABLE=y
CONFIG_X86_NEED_RELOCS=y
CONFIG_PHYSICAL_ALIGN=0x1000000
CONFIG_HOTPLUG_CPU=y
# CONFIG_COMPAT_VDSO is not set
CONFIG_CMDLINE_BOOL=y
CONFIG_CMDLINE=""
# CONFIG_CMDLINE_OVERRIDE is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
#
# Power management and ACPI options
#
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_RUNTIME=y
CONFIG_PM=y
CONFIG_PM_DEBUG=y
# CONFIG_PM_VERBOSE is not set
# CONFIG_PM_ADVANCED_DEBUG is not set
# CONFIG_PM_TEST_SUSPEND is not set
CONFIG_CAN_PM_TRACE=y
CONFIG_PM_TRACE=y
CONFIG_PM_TRACE_RTC=y
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_PROCFS_POWER=y
# CONFIG_ACPI_POWER_METER is not set
# CONFIG_ACPI_EC_DEBUGFS is not set
# CONFIG_ACPI_PROC_EVENT is not set
CONFIG_ACPI_AC=m
# CONFIG_ACPI_BATTERY is not set
# CONFIG_ACPI_BUTTON is not set
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=m
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_IPMI=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=m
# CONFIG_ACPI_THERMAL is not set
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=m
CONFIG_ACPI_SBS=m
# CONFIG_ACPI_HED is not set
# CONFIG_ACPI_APEI is not set
# CONFIG_SFI is not set
# CONFIG_APM is not set
#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=m
# CONFIG_CPU_FREQ_DEBUG is not set
CONFIG_CPU_FREQ_STAT=m
# CONFIG_CPU_FREQ_STAT_DETAILS is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
CONFIG_CPU_FREQ_GOV_ONDEMAND=m
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
#
# CPUFreq processor drivers
#
# CONFIG_X86_PCC_CPUFREQ is not set
# CONFIG_X86_ACPI_CPUFREQ is not set
CONFIG_X86_POWERNOW_K6=m
CONFIG_X86_POWERNOW_K7=m
CONFIG_X86_POWERNOW_K7_ACPI=y
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_GX_SUSPMOD is not set
CONFIG_X86_SPEEDSTEP_CENTRINO=m
CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
# CONFIG_X86_SPEEDSTEP_ICH is not set
CONFIG_X86_SPEEDSTEP_SMI=m
# CONFIG_X86_P4_CLOCKMOD is not set
CONFIG_X86_CPUFREQ_NFORCE2=m
# CONFIG_X86_LONGRUN is not set
# CONFIG_X86_LONGHAUL is not set
CONFIG_X86_E_POWERSAVER=m
#
# shared options
#
CONFIG_X86_SPEEDSTEP_LIB=m
# CONFIG_X86_SPEEDSTEP_RELAXED_CAP_CHECK is not set
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
CONFIG_INTEL_IDLE=y
#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
# CONFIG_PCIEPORTBUS is not set
CONFIG_ARCH_SUPPORTS_MSI=y
# CONFIG_PCI_MSI is not set
CONFIG_PCI_DEBUG=y
# CONFIG_PCI_STUB is not set
CONFIG_HT_IRQ=y
CONFIG_PCI_IOV=y
CONFIG_PCI_IOAPIC=y
CONFIG_PCI_LABEL=y
CONFIG_ISA_DMA_API=y
CONFIG_ISA=y
# CONFIG_EISA is not set
CONFIG_MCA=y
CONFIG_MCA_LEGACY=y
CONFIG_MCA_PROC_FS=y
# CONFIG_SCx200 is not set
# CONFIG_OLPC is not set
CONFIG_AMD_NB=y
CONFIG_PCCARD=m
CONFIG_PCMCIA=m
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_CARDBUS=y
#
# PC-card bridges
#
# CONFIG_YENTA is not set
# CONFIG_PD6729 is not set
CONFIG_I82092=m
CONFIG_I82365=m
CONFIG_TCIC=m
CONFIG_PCMCIA_PROBE=y
CONFIG_PCCARD_NONSTATIC=y
# CONFIG_HOTPLUG_PCI is not set
CONFIG_RAPIDIO=y
CONFIG_RAPIDIO_DISC_TIMEOUT=30
# CONFIG_RAPIDIO_ENABLE_RX_TX_PORTS is not set
# CONFIG_RAPIDIO_TSI57X is not set
# CONFIG_RAPIDIO_CPS_XX is not set
CONFIG_RAPIDIO_TSI568=y
CONFIG_RAPIDIO_CPS_GEN2=y
CONFIG_RAPIDIO_TSI500=y
# CONFIG_RAPIDIO_DEBUG is not set
#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
CONFIG_HAVE_AOUT=y
# CONFIG_BINFMT_AOUT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_HAVE_ATOMIC_IOMAP=y
CONFIG_HAVE_TEXT_POKE_SMP=y
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_XFRM=y
# CONFIG_XFRM_USER is not set
CONFIG_XFRM_SUB_POLICY=y
CONFIG_XFRM_MIGRATE=y
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_FIB_TRIE_STATS=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
# CONFIG_IP_ROUTE_VERBOSE is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE_DEMUX is not set
CONFIG_ARPD=y
CONFIG_SYN_COOKIES=y
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=m
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET_LRO=m
# CONFIG_INET_DIAG is not set
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
CONFIG_NETLABEL=y
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETFILTER=y
CONFIG_NETFILTER_DEBUG=y
CONFIG_NETFILTER_ADVANCED=y
#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
# CONFIG_NF_CONNTRACK is not set
# CONFIG_NETFILTER_XTABLES is not set
CONFIG_IP_SET=m
CONFIG_IP_SET_MAX=256
# CONFIG_IP_SET_BITMAP_IP is not set
# CONFIG_IP_SET_BITMAP_IPMAC is not set
# CONFIG_IP_SET_BITMAP_PORT is not set
# CONFIG_IP_SET_HASH_IP is not set
# CONFIG_IP_SET_HASH_IPPORT is not set
CONFIG_IP_SET_HASH_IPPORTIP=m
# CONFIG_IP_SET_HASH_IPPORTNET is not set
# CONFIG_IP_SET_HASH_NET is not set
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_IP_VS=m
CONFIG_IP_VS_DEBUG=y
CONFIG_IP_VS_TAB_BITS=12
#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
# CONFIG_IP_VS_PROTO_ESP is not set
CONFIG_IP_VS_PROTO_AH=y
# CONFIG_IP_VS_PROTO_SCTP is not set
#
# IPVS scheduler
#
# CONFIG_IP_VS_RR is not set
# CONFIG_IP_VS_WRR is not set
# CONFIG_IP_VS_LC is not set
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
# CONFIG_IP_VS_LBLCR is not set
# CONFIG_IP_VS_DH is not set
# CONFIG_IP_VS_SH is not set
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m
#
# IPVS application helper
#
#
# IP: Netfilter Configuration
#
# CONFIG_NF_DEFRAG_IPV4 is not set
CONFIG_IP_NF_QUEUE=m
# CONFIG_IP_NF_IPTABLES is not set
# CONFIG_IP_NF_ARPTABLES is not set
#
# DECnet: Netfilter Configuration
#
# CONFIG_DECNET_NF_GRABULATOR is not set
CONFIG_IP_DCCP=m
#
# DCCP CCIDs Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP_CCID2_DEBUG is not set
# CONFIG_IP_DCCP_CCID3 is not set
#
# DCCP Kernel Hacking
#
CONFIG_IP_DCCP_DEBUG=y
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
CONFIG_SCTP_HMAC_NONE=y
# CONFIG_SCTP_HMAC_SHA1 is not set
# CONFIG_SCTP_HMAC_MD5 is not set
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_L2TP is not set
# CONFIG_BRIDGE is not set
# CONFIG_NET_DSA is not set
CONFIG_VLAN_8021Q=m
# CONFIG_VLAN_8021Q_GVRP is not set
CONFIG_DECNET=m
# CONFIG_DECNET_ROUTER is not set
CONFIG_LLC=m
# CONFIG_LLC2 is not set
CONFIG_IPX=m
# CONFIG_IPX_INTERN is not set
CONFIG_ATALK=m
# CONFIG_DEV_APPLETALK is not set
# CONFIG_X25 is not set
CONFIG_LAPB=m
CONFIG_ECONET=m
# CONFIG_ECONET_AUNUDP is not set
# CONFIG_ECONET_NATIVE is not set
CONFIG_WAN_ROUTER=m
CONFIG_PHONET=m
CONFIG_IEEE802154=m
# CONFIG_NET_SCHED is not set
CONFIG_DCB=y
# CONFIG_DNS_RESOLVER is not set
# CONFIG_BATMAN_ADV is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
CONFIG_BT=m
CONFIG_BT_L2CAP=y
CONFIG_BT_SCO=y
# CONFIG_BT_RFCOMM is not set
CONFIG_BT_BNEP=m
# CONFIG_BT_BNEP_MC_FILTER is not set
CONFIG_BT_BNEP_PROTO_FILTER=y
#
# Bluetooth device drivers
#
# CONFIG_BT_HCIBTUSB is not set
# CONFIG_BT_HCIUART is not set
# CONFIG_BT_HCIBCM203X is not set
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIDTL1=m
# CONFIG_BT_HCIBT3C is not set
CONFIG_BT_HCIBLUECARD=m
CONFIG_BT_HCIBTUART=m
CONFIG_BT_HCIVHCI=m
CONFIG_BT_MRVL=m
# CONFIG_AF_RXRPC is not set
CONFIG_FIB_RULES=y
CONFIG_WIRELESS=y
CONFIG_WIRELESS_EXT=y
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_WEXT_SPY=y
CONFIG_WEXT_PRIV=y
# CONFIG_CFG80211 is not set
# CONFIG_WIRELESS_EXT_SYSFS is not set
CONFIG_LIB80211=m
CONFIG_LIB80211_CRYPT_WEP=m
CONFIG_LIB80211_CRYPT_CCMP=m
CONFIG_LIB80211_CRYPT_TKIP=m
CONFIG_LIB80211_DEBUG=y
#
# CFG80211 needs to be enabled for MAC80211
#
# CONFIG_WIMAX is not set
CONFIG_RFKILL=m
CONFIG_RFKILL_INPUT=y
CONFIG_CAIF=m
# CONFIG_CAIF_DEBUG is not set
CONFIG_CAIF_NETDEV=m
CONFIG_CEPH_LIB=m
CONFIG_CEPH_LIB_PRETTYDEBUG=y
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH=""
# CONFIG_DEVTMPFS is not set
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""
CONFIG_DEBUG_DRIVER=y
CONFIG_DEBUG_DEVRES=y
# CONFIG_SYS_HYPERVISOR is not set
CONFIG_ARCH_NO_SYSDEV_OPS=y
CONFIG_CONNECTOR=m
# CONFIG_MTD is not set
CONFIG_PARPORT=m
# CONFIG_PARPORT_PC is not set
# CONFIG_PARPORT_GSC is not set
CONFIG_PARPORT_AX88796=m
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y
#
# Protocols
#
CONFIG_ISAPNP=y
CONFIG_PNPBIOS=y
CONFIG_PNPBIOS_PROC_FS=y
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_FD is not set
CONFIG_BLK_DEV_XD=m
CONFIG_BLK_CPQ_DA=y
CONFIG_BLK_CPQ_CISS_DA=m
# CONFIG_CISS_SCSI_TAPE is not set
CONFIG_BLK_DEV_DAC960=m
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_DRBD=m
# CONFIG_DRBD_FAULT_INJECTION is not set
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_SX8=m
# CONFIG_BLK_DEV_UB is not set
# CONFIG_BLK_DEV_RAM is not set
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
CONFIG_CDROM_PKTCDVD_WCACHE=y
# CONFIG_ATA_OVER_ETH is not set
CONFIG_BLK_DEV_HD=y
CONFIG_BLK_DEV_RBD=m
# CONFIG_SENSORS_LIS3LV02D is not set
# CONFIG_MISC_DEVICES is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_TGT=m
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
# CONFIG_BLK_DEV_SR_VENDOR is not set
# CONFIG_CHR_DEV_SG is not set
CONFIG_CHR_DEV_SCH=m
CONFIG_SCSI_MULTI_LUN=y
# CONFIG_SCSI_CONSTANTS is not set
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_WAIT_SCAN=m
#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=y
CONFIG_SCSI_FC_ATTRS=m
CONFIG_SCSI_FC_TGT_ATTRS=y
CONFIG_SCSI_ISCSI_ATTRS=m
CONFIG_SCSI_SAS_ATTRS=m
CONFIG_SCSI_SAS_LIBSAS=m
CONFIG_SCSI_SAS_ATA=y
# CONFIG_SCSI_SAS_HOST_SMP is not set
CONFIG_SCSI_SRP_ATTRS=m
CONFIG_SCSI_SRP_TGT_ATTRS=y
# CONFIG_SCSI_LOWLEVEL is not set
CONFIG_SCSI_LOWLEVEL_PCMCIA=y
# CONFIG_PCMCIA_AHA152X is not set
CONFIG_PCMCIA_FDOMAIN=m
CONFIG_PCMCIA_NINJA_SCSI=m
CONFIG_PCMCIA_QLOGIC=m
CONFIG_PCMCIA_SYM53C500=m
CONFIG_SCSI_DH=m
# CONFIG_SCSI_DH_RDAC is not set
# CONFIG_SCSI_DH_HP_SW is not set
CONFIG_SCSI_DH_EMC=m
CONFIG_SCSI_DH_ALUA=m
CONFIG_SCSI_OSD_INITIATOR=m
# CONFIG_SCSI_OSD_ULD is not set
CONFIG_SCSI_OSD_DPRINT_SENSE=1
# CONFIG_SCSI_OSD_DEBUG is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
# CONFIG_ATA_ACPI is not set
# CONFIG_SATA_PMP is not set
#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_SATA_ACARD_AHCI is not set
CONFIG_SATA_SIL24=m
CONFIG_ATA_SFF=y
#
# SFF controllers with custom DMA interface
#
CONFIG_PDC_ADMA=m
# CONFIG_SATA_QSTOR is not set
CONFIG_SATA_SX4=m
CONFIG_ATA_BMDMA=y
#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=y
# CONFIG_SATA_MV is not set
CONFIG_SATA_NV=y
CONFIG_SATA_PROMISE=m
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIS is not set
CONFIG_SATA_SVW=m
# CONFIG_SATA_ULI is not set
CONFIG_SATA_VIA=m
CONFIG_SATA_VITESSE=m
#
# PATA SFF controllers with BMDMA
#
CONFIG_PATA_ALI=m
CONFIG_PATA_AMD=y
# CONFIG_PATA_ARASAN_CF is not set
CONFIG_PATA_ARTOP=m
CONFIG_PATA_ATIIXP=m
# CONFIG_PATA_ATP867X is not set
CONFIG_PATA_CMD64X=m
# CONFIG_PATA_CS5520 is not set
CONFIG_PATA_CS5530=m
CONFIG_PATA_CS5535=m
CONFIG_PATA_CS5536=m
CONFIG_PATA_CYPRESS=m
CONFIG_PATA_EFAR=m
CONFIG_PATA_HPT366=m
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
CONFIG_PATA_HPT3X3=m
# CONFIG_PATA_HPT3X3_DMA is not set
# CONFIG_PATA_IT8213 is not set
CONFIG_PATA_IT821X=m
CONFIG_PATA_JMICRON=m
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87415 is not set
CONFIG_PATA_OLDPIIX=y
# CONFIG_PATA_OPTIDMA is not set
CONFIG_PATA_PDC2027X=m
CONFIG_PATA_PDC_OLD=m
# CONFIG_PATA_RADISYS is not set
CONFIG_PATA_RDC=m
CONFIG_PATA_SC1200=m
# CONFIG_PATA_SCH is not set
CONFIG_PATA_SERVERWORKS=m
CONFIG_PATA_SIL680=m
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_TOSHIBA is not set
CONFIG_PATA_TRIFLEX=m
CONFIG_PATA_VIA=y
# CONFIG_PATA_WINBOND is not set
#
# PIO-only SFF controllers
#
# CONFIG_PATA_CMD640_PCI is not set
CONFIG_PATA_ISAPNP=m
# CONFIG_PATA_MPIIX is not set
CONFIG_PATA_NS87410=m
CONFIG_PATA_OPTI=m
CONFIG_PATA_PCMCIA=m
CONFIG_PATA_QDI=m
CONFIG_PATA_RZ1000=m
CONFIG_PATA_WINBOND_VLB=m
#
# Generic fallback / legacy drivers
#
CONFIG_ATA_GENERIC=m
CONFIG_PATA_LEGACY=m
# CONFIG_MD is not set
# CONFIG_TARGET_CORE is not set
# CONFIG_FUSION is not set
#
# IEEE 1394 (FireWire) support
#
CONFIG_FIREWIRE=m
# CONFIG_FIREWIRE_OHCI is not set
CONFIG_FIREWIRE_SBP2=m
# CONFIG_FIREWIRE_NET is not set
# CONFIG_FIREWIRE_NOSY is not set
# CONFIG_I2O is not set
CONFIG_MACINTOSH_DRIVERS=y
# CONFIG_MAC_EMUMOUSEBTN is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
CONFIG_ARCNET=m
# CONFIG_ARCNET_1201 is not set
CONFIG_ARCNET_1051=m
# CONFIG_ARCNET_RAW is not set
# CONFIG_ARCNET_CAP is not set
CONFIG_ARCNET_COM90xx=m
# CONFIG_ARCNET_COM90xxIO is not set
# CONFIG_ARCNET_RIM_I is not set
# CONFIG_ARCNET_COM20020 is not set
CONFIG_MII=y
CONFIG_PHYLIB=y
#
# MII PHY device drivers
#
# CONFIG_MARVELL_PHY is not set
# CONFIG_DAVICOM_PHY is not set
CONFIG_QSEMI_PHY=m
CONFIG_LXT_PHY=m
CONFIG_CICADA_PHY=m
# CONFIG_VITESSE_PHY is not set
CONFIG_SMSC_PHY=m
CONFIG_BROADCOM_PHY=m
# CONFIG_BCM63XX_PHY is not set
CONFIG_ICPLUS_PHY=m
# CONFIG_REALTEK_PHY is not set
# CONFIG_NATIONAL_PHY is not set
CONFIG_STE10XP=m
CONFIG_LSI_ET1011C_PHY=m
# CONFIG_MICREL_PHY is not set
CONFIG_FIXED_PHY=y
# CONFIG_MDIO_BITBANG is not set
CONFIG_NET_ETHERNET=y
# CONFIG_HAPPYMEAL is not set
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
# CONFIG_NET_VENDOR_3COM is not set
CONFIG_LANCE=m
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_ENC28J60 is not set
CONFIG_ETHOC=m
# CONFIG_NET_VENDOR_RACAL is not set
CONFIG_DNET=m
# CONFIG_NET_TULIP is not set
CONFIG_AT1700=m
CONFIG_DEPCA=m
CONFIG_HP100=m
# CONFIG_NET_ISA is not set
# CONFIG_NE2_MCA is not set
CONFIG_IBMLANA=m
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
# CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
# CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
CONFIG_ADAPTEC_STARFIRE=m
# CONFIG_AC3200 is not set
# CONFIG_KSZ884X_PCI is not set
CONFIG_APRICOT=m
# CONFIG_B44 is not set
CONFIG_FORCEDETH=y
CONFIG_CS89x0=m
CONFIG_E100=y
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=y
# CONFIG_8139TOO_PIO is not set
CONFIG_8139TOO_TUNE_TWISTER=y
# CONFIG_8139TOO_8129 is not set
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_R6040 is not set
# CONFIG_SIS900 is not set
CONFIG_EPIC100=m
# CONFIG_SMSC9420 is not set
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
# CONFIG_TLAN is not set
# CONFIG_KS8851 is not set
CONFIG_KS8851_MLL=m
CONFIG_VIA_RHINE=m
CONFIG_VIA_RHINE_MMIO=y
# CONFIG_SC92031 is not set
CONFIG_NET_POCKET=y
CONFIG_ATP=m
# CONFIG_DE600 is not set
# CONFIG_DE620 is not set
CONFIG_ATL2=m
CONFIG_NETDEV_1000=y
CONFIG_ACENIC=m
CONFIG_ACENIC_OMIT_TIGON_I=y
# CONFIG_DL2K is not set
CONFIG_E1000=m
CONFIG_E1000E=y
CONFIG_IP1000=m
# CONFIG_IGB is not set
# CONFIG_IGBVF is not set
CONFIG_NS83820=m
CONFIG_HAMACHI=m
# CONFIG_YELLOWFIN is not set
CONFIG_R8169=m
# CONFIG_SIS190 is not set
CONFIG_SKGE=y
# CONFIG_SKGE_DEBUG is not set
# CONFIG_SKY2 is not set
CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=y
CONFIG_BNX2=m
CONFIG_CNIC=m
CONFIG_QLA3XXX=m
# CONFIG_ATL1 is not set
CONFIG_ATL1E=m
# CONFIG_ATL1C is not set
CONFIG_JME=m
# CONFIG_STMMAC_ETH is not set
# CONFIG_PCH_GBE is not set
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set
CONFIG_WLAN=y
CONFIG_PCMCIA_RAYCS=m
# CONFIG_AIRO is not set
# CONFIG_ATMEL is not set
# CONFIG_AIRO_CS is not set
CONFIG_PCMCIA_WL3501=m
# CONFIG_PRISM54 is not set
# CONFIG_USB_ZD1201 is not set
CONFIG_HOSTAP=m
# CONFIG_HOSTAP_FIRMWARE is not set
# CONFIG_HOSTAP_PLX is not set
# CONFIG_HOSTAP_PCI is not set
# CONFIG_HOSTAP_CS is not set
#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#
#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
CONFIG_USB_NET_CDC_EEM=m
CONFIG_USB_NET_CDC_NCM=m
CONFIG_USB_NET_DM9601=m
# CONFIG_USB_NET_SMSC75XX is not set
CONFIG_USB_NET_SMSC95XX=m
# CONFIG_USB_NET_GL620A is not set
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
CONFIG_USB_NET_MCS7830=m
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
# CONFIG_USB_EPSON2888 is not set
# CONFIG_USB_KC2190 is not set
CONFIG_USB_NET_ZAURUS=m
# CONFIG_USB_NET_CX82310_ETH is not set
CONFIG_USB_HSO=m
# CONFIG_USB_NET_INT51X1 is not set
CONFIG_USB_CDC_PHONET=m
CONFIG_USB_IPHETH=m
# CONFIG_USB_SIERRA_NET is not set
CONFIG_USB_VL600=m
# CONFIG_NET_PCMCIA is not set
CONFIG_WAN=y
CONFIG_HOSTESS_SV11=m
# CONFIG_COSA is not set
# CONFIG_LANMEDIA is not set
# CONFIG_SEALEVEL_4021 is not set
CONFIG_HDLC=m
CONFIG_HDLC_RAW=m
# CONFIG_HDLC_RAW_ETH is not set
# CONFIG_HDLC_CISCO is not set
# CONFIG_HDLC_FR is not set
# CONFIG_HDLC_PPP is not set
CONFIG_HDLC_X25=m
# CONFIG_PCI200SYN is not set
CONFIG_WANXL=m
# CONFIG_PC300TOO is not set
# CONFIG_N2 is not set
# CONFIG_C101 is not set
CONFIG_FARSYNC=m
CONFIG_DSCC4=m
# CONFIG_DSCC4_PCISYNC is not set
# CONFIG_DSCC4_PCI_RST is not set
CONFIG_DLCI=m
CONFIG_DLCI_MAX=8
# CONFIG_SDLA is not set
# CONFIG_WAN_ROUTER_DRIVERS is not set
CONFIG_SBNI=m
CONFIG_SBNI_MULTILINE=y
CONFIG_IEEE802154_DRIVERS=m
CONFIG_IEEE802154_FAKEHARD=m
#
# CAIF transport drivers
#
# CONFIG_CAIF_TTY is not set
CONFIG_CAIF_SPI_SLAVE=m
CONFIG_CAIF_SPI_SYNC=y
CONFIG_RIONET=m
CONFIG_RIONET_TX_SIZE=128
CONFIG_RIONET_RX_SIZE=128
CONFIG_FDDI=m
CONFIG_DEFXX=m
# CONFIG_DEFXX_MMIO is not set
CONFIG_SKFP=m
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
# CONFIG_PPP_FILTER is not set
# CONFIG_PPP_ASYNC is not set
# CONFIG_PPP_SYNC_TTY is not set
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
# CONFIG_PPP_MPPE is not set
# CONFIG_PPPOE is not set
CONFIG_SLIP=m
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLHC=m
CONFIG_SLIP_SMART=y
CONFIG_SLIP_MODE_SLIP6=y
# CONFIG_NET_FC is not set
CONFIG_NETCONSOLE=y
# CONFIG_NETCONSOLE_DYNAMIC is not set
CONFIG_NETPOLL=y
CONFIG_NETPOLL_TRAP=y
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_VMXNET3 is not set
CONFIG_ISDN=y
CONFIG_ISDN_I4L=m
# CONFIG_ISDN_PPP is not set
# CONFIG_ISDN_AUDIO is not set
#
# ISDN feature submodules
#
# CONFIG_ISDN_DIVERSION is not set
#
# ISDN4Linux hardware drivers
#
#
# Passive cards
#
CONFIG_ISDN_DRV_HISAX=m
#
# D-channel protocol features
#
CONFIG_HISAX_EURO=y
# CONFIG_DE_AOC is not set
# CONFIG_HISAX_NO_SENDCOMPLETE is not set
CONFIG_HISAX_NO_LLC=y
CONFIG_HISAX_NO_KEYPAD=y
# CONFIG_HISAX_1TR6 is not set
# CONFIG_HISAX_NI1 is not set
CONFIG_HISAX_MAX_CARDS=8
#
# HiSax supported cards
#
CONFIG_HISAX_16_0=y
CONFIG_HISAX_16_3=y
# CONFIG_HISAX_TELESPCI is not set
# CONFIG_HISAX_S0BOX is not set
# CONFIG_HISAX_AVM_A1 is not set
# CONFIG_HISAX_FRITZPCI is not set
CONFIG_HISAX_AVM_A1_PCMCIA=y
# CONFIG_HISAX_ELSA is not set
CONFIG_HISAX_IX1MICROR2=y
CONFIG_HISAX_DIEHLDIVA=y
# CONFIG_HISAX_ASUSCOM is not set
# CONFIG_HISAX_TELEINT is not set
# CONFIG_HISAX_HFCS is not set
CONFIG_HISAX_SEDLBAUER=y
# CONFIG_HISAX_SPORTSTER is not set
CONFIG_HISAX_MIC=y
CONFIG_HISAX_NETJET=y
CONFIG_HISAX_NETJET_U=y
# CONFIG_HISAX_NICCY is not set
# CONFIG_HISAX_ISURF is not set
# CONFIG_HISAX_HSTSAPHIR is not set
# CONFIG_HISAX_BKM_A4T is not set
# CONFIG_HISAX_SCT_QUADRO is not set
# CONFIG_HISAX_GAZEL is not set
# CONFIG_HISAX_HFC_PCI is not set
# CONFIG_HISAX_W6692 is not set
CONFIG_HISAX_HFC_SX=y
CONFIG_HISAX_ENTERNOW_PCI=y
CONFIG_HISAX_DEBUG=y
#
# HiSax PCMCIA card service modules
#
CONFIG_HISAX_SEDLBAUER_CS=m
# CONFIG_HISAX_AVM_A1_CS is not set
# CONFIG_HISAX_TELES_CS is not set
#
# HiSax sub driver modules
#
# CONFIG_HISAX_ST5481 is not set
# CONFIG_HISAX_HFCUSB is not set
CONFIG_HISAX_HFC4S8S=m
# CONFIG_HISAX_FRITZ_PCIPNP is not set
#
# Active cards
#
# CONFIG_ISDN_DRV_ICN is not set
# CONFIG_ISDN_DRV_PCBIT is not set
CONFIG_ISDN_DRV_SC=m
CONFIG_ISDN_DRV_ACT2000=m
# CONFIG_ISDN_CAPI is not set
CONFIG_ISDN_DRV_GIGASET=m
CONFIG_GIGASET_I4L=y
# CONFIG_GIGASET_DUMMYLL is not set
# CONFIG_GIGASET_BASE is not set
CONFIG_GIGASET_M105=m
# CONFIG_GIGASET_M101 is not set
# CONFIG_GIGASET_DEBUG is not set
CONFIG_HYSDN=m
CONFIG_PHONE=m
CONFIG_PHONE_IXJ=m
# CONFIG_PHONE_IXJ_PCMCIA is not set
#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
CONFIG_INPUT_POLLDEV=y
# CONFIG_INPUT_SPARSEKMAP is not set
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=m
CONFIG_INPUT_EVBUG=m
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1070 is not set
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_LKKBD is not set
CONFIG_KEYBOARD_GPIO=m
# CONFIG_KEYBOARD_GPIO_POLLED is not set
CONFIG_KEYBOARD_TCA6416=m
CONFIG_KEYBOARD_MATRIX=m
CONFIG_KEYBOARD_MAX7359=m
CONFIG_KEYBOARD_MCS=m
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
CONFIG_KEYBOARD_STOWAWAY=m
CONFIG_KEYBOARD_SUNKBD=m
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_AD714X=m
# CONFIG_INPUT_AD714X_I2C is not set
# CONFIG_INPUT_AD714X_SPI is not set
# CONFIG_INPUT_PCSPKR is not set
# CONFIG_INPUT_WISTRON_BTNS is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
CONFIG_INPUT_ATI_REMOTE2=m
CONFIG_INPUT_KEYSPAN_REMOTE=m
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
CONFIG_INPUT_CM109=m
CONFIG_INPUT_UINPUT=m
CONFIG_INPUT_PCF50633_PMU=m
# CONFIG_INPUT_PCF8574 is not set
CONFIG_INPUT_GPIO_ROTARY_ENCODER=m
# CONFIG_INPUT_PCAP is not set
CONFIG_INPUT_ADXL34X=m
CONFIG_INPUT_ADXL34X_I2C=m
# CONFIG_INPUT_ADXL34X_SPI is not set
CONFIG_INPUT_CMA3000=m
# CONFIG_INPUT_CMA3000_I2C is not set
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_SERIO_ALTERA_PS2 is not set
CONFIG_SERIO_PS2MULT=m
# CONFIG_GAMEPORT is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NOZOMI is not set
# CONFIG_N_GSM is not set
# CONFIG_DEVKMEM is not set
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_CS=m
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
# CONFIG_SERIAL_8250_FOURPORT is not set
CONFIG_SERIAL_8250_ACCENT=m
CONFIG_SERIAL_8250_BOCA=m
# CONFIG_SERIAL_8250_EXAR_ST16C554 is not set
# CONFIG_SERIAL_8250_HUB6 is not set
# CONFIG_SERIAL_8250_SHARE_IRQ is not set
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_RSA=y
CONFIG_SERIAL_8250_MCA=m
#
# Non-8250 serial port support
#
# CONFIG_SERIAL_MAX3100 is not set
CONFIG_SERIAL_MAX3107=m
# CONFIG_SERIAL_MFD_HSU is not set
# CONFIG_SERIAL_UARTLITE is not set
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
CONFIG_SERIAL_TIMBERDALE=m
CONFIG_SERIAL_ALTERA_JTAGUART=m
CONFIG_SERIAL_ALTERA_UART=m
CONFIG_SERIAL_ALTERA_UART_MAXPORTS=4
CONFIG_SERIAL_ALTERA_UART_BAUDRATE=115200
# CONFIG_SERIAL_IFX6X60 is not set
# CONFIG_SERIAL_PCH_UART is not set
# CONFIG_PRINTER is not set
# CONFIG_PPDEV is not set
CONFIG_IPMI_HANDLER=m
CONFIG_IPMI_PANIC_EVENT=y
# CONFIG_IPMI_PANIC_STRING is not set
# CONFIG_IPMI_DEVICE_INTERFACE is not set
CONFIG_IPMI_SI=m
# CONFIG_IPMI_WATCHDOG is not set
# CONFIG_IPMI_POWEROFF is not set
# CONFIG_HW_RANDOM is not set
CONFIG_NVRAM=m
# CONFIG_DTLK is not set
CONFIG_R3964=m
# CONFIG_APPLICOM is not set
CONFIG_SONYPI=m
#
# PCMCIA character devices
#
CONFIG_SYNCLINK_CS=m
CONFIG_CARDMAN_4000=m
# CONFIG_CARDMAN_4040 is not set
# CONFIG_IPWIRELESS is not set
CONFIG_MWAVE=m
CONFIG_PC8736x_GPIO=m
CONFIG_NSC_GPIO=m
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
# CONFIG_HPET_MMAP is not set
CONFIG_HANGCHECK_TIMER=m
CONFIG_TCG_TPM=y
CONFIG_TCG_TIS=y
# CONFIG_TCG_NSC is not set
CONFIG_TCG_ATMEL=m
# CONFIG_TCG_INFINEON is not set
CONFIG_TELCLOCK=m
CONFIG_DEVPORT=y
# CONFIG_RAMOOPS is not set
CONFIG_I2C=m
CONFIG_I2C_BOARDINFO=y
# CONFIG_I2C_COMPAT is not set
# CONFIG_I2C_CHARDEV is not set
# CONFIG_I2C_MUX is not set
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCA=m
#
# I2C Hardware Bus support
#
#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_SIS5595=m
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
#
# ACPI drivers
#
# CONFIG_I2C_SCMI is not set
#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
CONFIG_I2C_GPIO=m
CONFIG_I2C_INTEL_MID=m
# CONFIG_I2C_OCORES is not set
CONFIG_I2C_PCA_PLATFORM=m
# CONFIG_I2C_PXA_PCI is not set
CONFIG_I2C_SIMTEC=m
CONFIG_I2C_XILINX=m
CONFIG_I2C_EG20T=m
#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_DIOLAN_U2C=m
CONFIG_I2C_PARPORT=m
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set
#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_PCA_ISA is not set
CONFIG_I2C_STUB=m
CONFIG_SCx200_ACB=m
# CONFIG_I2C_DEBUG_CORE is not set
CONFIG_I2C_DEBUG_ALGO=y
CONFIG_I2C_DEBUG_BUS=y
CONFIG_SPI=y
CONFIG_SPI_DEBUG=y
CONFIG_SPI_MASTER=y
#
# SPI Master Controller Drivers
#
# CONFIG_SPI_ALTERA is not set
# CONFIG_SPI_BITBANG is not set
# CONFIG_SPI_BUTTERFLY is not set
# CONFIG_SPI_GPIO is not set
# CONFIG_SPI_LM70_LLP is not set
# CONFIG_SPI_OC_TINY is not set
CONFIG_SPI_PXA2XX=m
CONFIG_SPI_PXA2XX_PCI=y
CONFIG_SPI_TOPCLIFF_PCH=m
# CONFIG_SPI_XILINX is not set
# CONFIG_SPI_DESIGNWARE is not set
#
# SPI Protocol Masters
#
CONFIG_SPI_SPIDEV=m
# CONFIG_SPI_TLE62X0 is not set
#
# PPS support
#
# CONFIG_PPS is not set
#
# PPS generators support
#
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
CONFIG_GPIOLIB=y
# CONFIG_DEBUG_GPIO is not set
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_MAX730X=m
#
# Memory mapped GPIO expanders:
#
CONFIG_GPIO_BASIC_MMIO=m
# CONFIG_GPIO_IT8761E is not set
CONFIG_GPIO_SCH=m
CONFIG_GPIO_VX855=m
#
# I2C GPIO expanders:
#
# CONFIG_GPIO_MAX7300 is not set
CONFIG_GPIO_MAX732X=m
CONFIG_GPIO_PCA953X=m
CONFIG_GPIO_PCF857X=m
# CONFIG_GPIO_ADP5588 is not set
#
# PCI GPIO expanders:
#
# CONFIG_GPIO_CS5535 is not set
CONFIG_GPIO_BT8XX=m
CONFIG_GPIO_LANGWELL=y
# CONFIG_GPIO_PCH is not set
CONFIG_GPIO_ML_IOH=m
# CONFIG_GPIO_TIMBERDALE is not set
CONFIG_GPIO_RDC321X=m
#
# SPI GPIO expanders:
#
CONFIG_GPIO_MAX7301=m
CONFIG_GPIO_MCP23S08=m
CONFIG_GPIO_MC33880=m
CONFIG_GPIO_74X164=m
#
# AC97 GPIO expanders:
#
#
# MODULbus GPIO expanders:
#
# CONFIG_GPIO_JANZ_TTL is not set
CONFIG_W1=m
CONFIG_W1_CON=y
#
# 1-wire Bus Masters
#
# CONFIG_W1_MASTER_MATROX is not set
CONFIG_W1_MASTER_DS2490=m
# CONFIG_W1_MASTER_DS2482 is not set
CONFIG_W1_MASTER_GPIO=m
#
# 1-wire Slaves
#
CONFIG_W1_SLAVE_THERM=m
CONFIG_W1_SLAVE_SMEM=m
# CONFIG_W1_SLAVE_DS2423 is not set
# CONFIG_W1_SLAVE_DS2431 is not set
CONFIG_W1_SLAVE_DS2433=m
CONFIG_W1_SLAVE_DS2433_CRC=y
CONFIG_W1_SLAVE_DS2760=m
# CONFIG_W1_SLAVE_BQ27000 is not set
CONFIG_POWER_SUPPLY=m
CONFIG_POWER_SUPPLY_DEBUG=y
# CONFIG_PDA_POWER is not set
CONFIG_TEST_POWER=m
CONFIG_BATTERY_DS2760=m
CONFIG_BATTERY_DS2782=m
CONFIG_BATTERY_BQ20Z75=m
CONFIG_BATTERY_BQ27x00=m
# CONFIG_BATTERY_BQ27X00_I2C is not set
# CONFIG_BATTERY_BQ27X00_PLATFORM is not set
CONFIG_BATTERY_MAX17040=m
CONFIG_BATTERY_MAX17042=m
# CONFIG_CHARGER_PCF50633 is not set
# CONFIG_CHARGER_ISP1704 is not set
CONFIG_CHARGER_GPIO=m
CONFIG_HWMON=m
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set
#
# Native drivers
#
# CONFIG_SENSORS_ABITUGURU is not set
CONFIG_SENSORS_ABITUGURU3=m
# CONFIG_SENSORS_AD7414 is not set
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADCXX=m
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7411=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
# CONFIG_SENSORS_ADT7475 is not set
CONFIG_SENSORS_ASC7621=m
# CONFIG_SENSORS_K8TEMP is not set
CONFIG_SENSORS_K10TEMP=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_ATXP1=m
# CONFIG_SENSORS_DS620 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_FSCHMD=m
CONFIG_SENSORS_G760A=m
CONFIG_SENSORS_GL518SM=m
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_GPIO_FAN is not set
# CONFIG_SENSORS_CORETEMP is not set
CONFIG_SENSORS_PKGTEMP=m
CONFIG_SENSORS_IBMAEM=m
CONFIG_SENSORS_IBMPEX=m
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_JC42 is not set
# CONFIG_SENSORS_LINEAGE is not set
CONFIG_SENSORS_LM63=m
# CONFIG_SENSORS_LM70 is not set
# CONFIG_SENSORS_LM73 is not set
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
# CONFIG_SENSORS_LM78 is not set
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_LM93 is not set
CONFIG_SENSORS_LTC4151=m
# CONFIG_SENSORS_LTC4215 is not set
CONFIG_SENSORS_LTC4245=m
CONFIG_SENSORS_LTC4261=m
CONFIG_SENSORS_LM95241=m
# CONFIG_SENSORS_MAX1111 is not set
CONFIG_SENSORS_MAX1619=m
# CONFIG_SENSORS_MAX6639 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_PCF8591=m
# CONFIG_PMBUS is not set
CONFIG_SENSORS_SHT15=m
CONFIG_SENSORS_SHT21=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_SMM665=m
# CONFIG_SENSORS_DME1737 is not set
CONFIG_SENSORS_EMC1403=m
CONFIG_SENSORS_EMC2103=m
# CONFIG_SENSORS_SMSC47M1 is not set
CONFIG_SENSORS_SMSC47M192=m
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_SCH5627 is not set
# CONFIG_SENSORS_ADS1015 is not set
CONFIG_SENSORS_ADS7828=m
# CONFIG_SENSORS_ADS7871 is not set
CONFIG_SENSORS_AMC6821=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP102=m
# CONFIG_SENSORS_TMP401 is not set
CONFIG_SENSORS_TMP421=m
# CONFIG_SENSORS_VIA_CPUTEMP is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
CONFIG_SENSORS_VT8231=m
CONFIG_SENSORS_W83781D=m
# CONFIG_SENSORS_W83791D is not set
CONFIG_SENSORS_W83792D=m
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83795 is not set
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_APPLESMC is not set
# CONFIG_SENSORS_MC13783_ADC is not set
#
# ACPI drivers
#
# CONFIG_SENSORS_ATK0110 is not set
CONFIG_THERMAL=m
CONFIG_THERMAL_HWMON=y
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
#
# Watchdog Device Drivers
#
# CONFIG_SOFT_WATCHDOG is not set
CONFIG_ACQUIRE_WDT=m
# CONFIG_ADVANTECH_WDT is not set
# CONFIG_ALIM1535_WDT is not set
CONFIG_ALIM7101_WDT=m
CONFIG_F71808E_WDT=m
# CONFIG_SP5100_TCO is not set
# CONFIG_SC520_WDT is not set
# CONFIG_EUROTECH_WDT is not set
CONFIG_IB700_WDT=m
CONFIG_IBMASR=m
CONFIG_WAFER_WDT=m
# CONFIG_I6300ESB_WDT is not set
CONFIG_ITCO_WDT=m
CONFIG_ITCO_VENDOR_SUPPORT=y
CONFIG_IT8712F_WDT=m
CONFIG_IT87_WDT=m
CONFIG_HP_WATCHDOG=m
CONFIG_HPWDT_NMI_DECODING=y
# CONFIG_SC1200_WDT is not set
# CONFIG_PC87413_WDT is not set
# CONFIG_NV_TCO is not set
# CONFIG_60XX_WDT is not set
CONFIG_SBC8360_WDT=m
CONFIG_SBC7240_WDT=m
CONFIG_CPU5_WDT=m
CONFIG_SMSC_SCH311X_WDT=m
CONFIG_SMSC37B787_WDT=m
CONFIG_W83627HF_WDT=m
# CONFIG_W83697HF_WDT is not set
# CONFIG_W83697UG_WDT is not set
CONFIG_W83877F_WDT=m
# CONFIG_W83977F_WDT is not set
# CONFIG_MACHZ_WDT is not set
CONFIG_SBC_EPX_C3_WATCHDOG=m
#
# ISA-based Watchdog Cards
#
# CONFIG_PCWATCHDOG is not set
CONFIG_MIXCOMWD=m
# CONFIG_WDT is not set
#
# PCI-based Watchdog Cards
#
# CONFIG_PCIPCWATCHDOG is not set
CONFIG_WDTPCI=m
#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_SSB_POSSIBLE=y
#
# Sonics Silicon Backplane
#
# CONFIG_SSB is not set
CONFIG_MFD_SUPPORT=y
CONFIG_MFD_CORE=m
CONFIG_MFD_SM501=m
CONFIG_MFD_SM501_GPIO=y
# CONFIG_HTC_PASIC3 is not set
CONFIG_TPS6105X=m
# CONFIG_TPS65010 is not set
# CONFIG_TPS6507X is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X_SPI is not set
CONFIG_MFD_PCF50633=m
CONFIG_MFD_MC13783=m
CONFIG_MFD_MC13XXX=m
CONFIG_PCF50633_ADC=m
CONFIG_PCF50633_GPIO=m
CONFIG_ABX500_CORE=y
CONFIG_EZX_PCAP=y
# CONFIG_AB8500_CORE is not set
CONFIG_MFD_CS5535=m
CONFIG_MFD_TIMBERDALE=m
CONFIG_LPC_SCH=m
CONFIG_MFD_RDC321X=m
CONFIG_MFD_JANZ_CMODIO=m
CONFIG_MFD_VX855=m
# CONFIG_MFD_WL1273_CORE is not set
CONFIG_REGULATOR=y
# CONFIG_REGULATOR_DEBUG is not set
CONFIG_REGULATOR_DUMMY=y
CONFIG_REGULATOR_FIXED_VOLTAGE=m
# CONFIG_REGULATOR_VIRTUAL_CONSUMER is not set
# CONFIG_REGULATOR_USERSPACE_CONSUMER is not set
# CONFIG_REGULATOR_BQ24022 is not set
# CONFIG_REGULATOR_MAX1586 is not set
# CONFIG_REGULATOR_MAX8649 is not set
# CONFIG_REGULATOR_MAX8660 is not set
# CONFIG_REGULATOR_MAX8952 is not set
CONFIG_REGULATOR_PCF50633=m
CONFIG_REGULATOR_LP3971=m
# CONFIG_REGULATOR_LP3972 is not set
# CONFIG_REGULATOR_PCAP is not set
# CONFIG_REGULATOR_MC13783 is not set
# CONFIG_REGULATOR_MC13892 is not set
# CONFIG_REGULATOR_TPS6105X is not set
CONFIG_REGULATOR_TPS65023=m
CONFIG_REGULATOR_TPS6507X=m
CONFIG_REGULATOR_ISL6271A=m
# CONFIG_REGULATOR_AD5398 is not set
# CONFIG_REGULATOR_TPS6524X is not set
# CONFIG_MEDIA_SUPPORT is not set
#
# Graphics support
#
# CONFIG_AGP is not set
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=16
CONFIG_VGA_SWITCHEROO=y
CONFIG_DRM=m
CONFIG_DRM_KMS_HELPER=m
CONFIG_DRM_TTM=m
CONFIG_DRM_TDFX=m
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=m
CONFIG_DRM_MGA=m
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
CONFIG_STUB_POULSBO=m
CONFIG_VGASTATE=m
CONFIG_VIDEO_OUTPUT_CONTROL=m
CONFIG_FB=m
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_DDC=m
# CONFIG_FB_BOOT_VESA_SUPPORT is not set
CONFIG_FB_CFB_FILLRECT=m
CONFIG_FB_CFB_COPYAREA=m
CONFIG_FB_CFB_IMAGEBLIT=m
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
CONFIG_FB_SYS_FILLRECT=m
CONFIG_FB_SYS_COPYAREA=m
CONFIG_FB_SYS_IMAGEBLIT=m
CONFIG_FB_FOREIGN_ENDIAN=y
# CONFIG_FB_BOTH_ENDIAN is not set
# CONFIG_FB_BIG_ENDIAN is not set
CONFIG_FB_LITTLE_ENDIAN=y
CONFIG_FB_SYS_FOPS=m
# CONFIG_FB_WMT_GE_ROPS is not set
CONFIG_FB_DEFERRED_IO=y
CONFIG_FB_SVGALIB=m
# CONFIG_FB_MACMODES is not set
CONFIG_FB_BACKLIGHT=y
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y
#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
CONFIG_FB_PM2=m
# CONFIG_FB_PM2_FIFO_DISCONNECT is not set
CONFIG_FB_CYBER2000=m
CONFIG_FB_CYBER2000_DDC=y
# CONFIG_FB_ARC is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_UVESA is not set
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
CONFIG_FB_S1D13XXX=m
# CONFIG_FB_NVIDIA is not set
CONFIG_FB_RIVA=m
# CONFIG_FB_RIVA_I2C is not set
CONFIG_FB_RIVA_DEBUG=y
CONFIG_FB_RIVA_BACKLIGHT=y
# CONFIG_FB_LE80578 is not set
CONFIG_FB_MATROX=m
# CONFIG_FB_MATROX_MILLENIUM is not set
CONFIG_FB_MATROX_MYSTIQUE=y
CONFIG_FB_MATROX_G=y
# CONFIG_FB_MATROX_I2C is not set
# CONFIG_FB_RADEON is not set
CONFIG_FB_ATY128=m
CONFIG_FB_ATY128_BACKLIGHT=y
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
CONFIG_FB_NEOMAGIC=m
CONFIG_FB_KYRO=m
# CONFIG_FB_3DFX is not set
CONFIG_FB_VOODOO1=m
CONFIG_FB_VT8623=m
# CONFIG_FB_TRIDENT is not set
CONFIG_FB_ARK=m
CONFIG_FB_PM3=m
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_GEODE is not set
CONFIG_FB_TMIO=m
# CONFIG_FB_TMIO_ACCELL is not set
# CONFIG_FB_SM501 is not set
# CONFIG_FB_UDL is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
CONFIG_FB_MB862XX=m
# CONFIG_FB_MB862XX_PCI_GDC is not set
CONFIG_FB_BROADSHEET=m
CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=m
CONFIG_BACKLIGHT_GENERIC=m
CONFIG_BACKLIGHT_PROGEAR=m
CONFIG_BACKLIGHT_APPLE=m
# CONFIG_BACKLIGHT_SAHARA is not set
# CONFIG_BACKLIGHT_ADP8860 is not set
# CONFIG_BACKLIGHT_PCF50633 is not set
#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=m
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
CONFIG_FONTS=y
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_FONT_6x11=y
CONFIG_FONT_7x14=y
# CONFIG_FONT_PEARL_8x8 is not set
CONFIG_FONT_ACORN_8x8=y
# CONFIG_FONT_MINI_4x6 is not set
CONFIG_FONT_SUN8x16=y
CONFIG_FONT_SUN12x22=y
CONFIG_FONT_10x18=y
# CONFIG_LOGO is not set
# CONFIG_SOUND is not set
# CONFIG_HID_SUPPORT is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DEVICE_CLASS=y
CONFIG_USB_DYNAMIC_MINORS=y
CONFIG_USB_SUSPEND=y
# CONFIG_USB_OTG is not set
# CONFIG_USB_MON is not set
# CONFIG_USB_WUSB is not set
CONFIG_USB_WUSB_CBAF=m
# CONFIG_USB_WUSB_CBAF_DEBUG is not set
#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
# CONFIG_USB_XHCI_HCD is not set
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OXU210HP_HCD=m
CONFIG_USB_ISP116X_HCD=m
# CONFIG_USB_ISP1760_HCD is not set
# CONFIG_USB_ISP1362_HCD is not set
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_SL811_HCD=m
# CONFIG_USB_SL811_CS is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_HWA_HCD is not set
#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
# CONFIG_USB_PRINTER is not set
CONFIG_USB_WDM=m
# CONFIG_USB_TMC is not set
#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#
#
# also be needed; see USB_STORAGE Help for more info
#
# CONFIG_USB_STORAGE is not set
CONFIG_USB_UAS=m
# CONFIG_USB_LIBUSUAL is not set
#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
#
# USB port drivers
#
# CONFIG_USB_USS720 is not set
# CONFIG_USB_SERIAL is not set
#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
# CONFIG_USB_ADUTUX is not set
CONFIG_USB_SEVSEG=m
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
CONFIG_USB_CYTHERM=m
CONFIG_USB_IDMOUSE=m
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
CONFIG_USB_SISUSBVGA=m
# CONFIG_USB_SISUSBVGA_CON is not set
CONFIG_USB_LD=m
# CONFIG_USB_TRANCEVIBRATOR is not set
CONFIG_USB_IOWARRIOR=m
# CONFIG_USB_TEST is not set
CONFIG_USB_ISIGHTFW=m
CONFIG_USB_YUREX=m
CONFIG_USB_GADGET=m
CONFIG_USB_GADGET_DEBUG=y
# CONFIG_USB_GADGET_DEBUG_FILES is not set
CONFIG_USB_GADGET_DEBUG_FS=y
CONFIG_USB_GADGET_VBUS_DRAW=2
CONFIG_USB_GADGET_SELECTED=y
CONFIG_USB_GADGET_FUSB300=y
CONFIG_USB_FUSB300=m
# CONFIG_USB_GADGET_R8A66597 is not set
# CONFIG_USB_GADGET_PXA_U2O is not set
# CONFIG_USB_GADGET_M66592 is not set
# CONFIG_USB_GADGET_AMD5536UDC is not set
# CONFIG_USB_GADGET_CI13XXX_PCI is not set
# CONFIG_USB_GADGET_NET2280 is not set
# CONFIG_USB_GADGET_GOKU is not set
# CONFIG_USB_GADGET_LANGWELL is not set
# CONFIG_USB_GADGET_EG20T is not set
# CONFIG_USB_GADGET_DUMMY_HCD is not set
CONFIG_USB_GADGET_DUALSPEED=y
CONFIG_USB_ZERO=m
# CONFIG_USB_ETH is not set
CONFIG_USB_G_NCM=m
# CONFIG_USB_GADGETFS is not set
# CONFIG_USB_FUNCTIONFS is not set
CONFIG_USB_FILE_STORAGE=m
# CONFIG_USB_FILE_STORAGE_TEST is not set
# CONFIG_USB_MASS_STORAGE is not set
CONFIG_USB_G_SERIAL=m
CONFIG_USB_G_PRINTER=m
# CONFIG_USB_CDC_COMPOSITE is not set
# CONFIG_USB_G_NOKIA is not set
# CONFIG_USB_G_MULTI is not set
# CONFIG_USB_G_HID is not set
# CONFIG_USB_G_DBGP is not set
#
# OTG and related infrastructure
#
CONFIG_USB_OTG_UTILS=y
CONFIG_USB_GPIO_VBUS=m
CONFIG_NOP_USB_XCEIV=m
# CONFIG_UWB is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
CONFIG_NFC_DEVICES=y
# CONFIG_PN544_NFC is not set
CONFIG_ACCESSIBILITY=y
# CONFIG_A11Y_BRAILLE_CONSOLE is not set
CONFIG_INFINIBAND=m
# CONFIG_INFINIBAND_USER_MAD is not set
# CONFIG_INFINIBAND_USER_ACCESS is not set
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_MTHCA=m
CONFIG_INFINIBAND_MTHCA_DEBUG=y
# CONFIG_INFINIBAND_AMSO1100 is not set
CONFIG_INFINIBAND_NES=m
# CONFIG_INFINIBAND_NES_DEBUG is not set
# CONFIG_INFINIBAND_IPOIB is not set
CONFIG_INFINIBAND_SRP=m
CONFIG_INFINIBAND_ISER=m
# CONFIG_EDAC is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
CONFIG_RTC_DEBUG=y
#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
# CONFIG_RTC_INTF_PROC is not set
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set
#
# I2C RTC drivers
#
CONFIG_RTC_DRV_DS1307=m
# CONFIG_RTC_DRV_DS1374 is not set
# CONFIG_RTC_DRV_DS1672 is not set
# CONFIG_RTC_DRV_DS3232 is not set
# CONFIG_RTC_DRV_MAX6900 is not set
# CONFIG_RTC_DRV_RS5C372 is not set
CONFIG_RTC_DRV_ISL1208=m
CONFIG_RTC_DRV_ISL12022=m
CONFIG_RTC_DRV_X1205=m
# CONFIG_RTC_DRV_PCF8563 is not set
CONFIG_RTC_DRV_PCF8583=m
CONFIG_RTC_DRV_M41T80=m
CONFIG_RTC_DRV_M41T80_WDT=y
# CONFIG_RTC_DRV_BQ32K is not set
# CONFIG_RTC_DRV_S35390A is not set
# CONFIG_RTC_DRV_FM3130 is not set
# CONFIG_RTC_DRV_RX8581 is not set
CONFIG_RTC_DRV_RX8025=m
#
# SPI RTC drivers
#
CONFIG_RTC_DRV_M41T94=m
# CONFIG_RTC_DRV_DS1305 is not set
# CONFIG_RTC_DRV_DS1390 is not set
# CONFIG_RTC_DRV_MAX6902 is not set
CONFIG_RTC_DRV_R9701=m
CONFIG_RTC_DRV_RS5C348=m
CONFIG_RTC_DRV_DS3234=m
CONFIG_RTC_DRV_PCF2123=m
#
# Platform RTC drivers
#
# CONFIG_RTC_DRV_CMOS is not set
CONFIG_RTC_DRV_DS1286=m
CONFIG_RTC_DRV_DS1511=m
# CONFIG_RTC_DRV_DS1553 is not set
CONFIG_RTC_DRV_DS1742=m
CONFIG_RTC_DRV_STK17TA8=m
CONFIG_RTC_DRV_M48T86=m
# CONFIG_RTC_DRV_M48T35 is not set
CONFIG_RTC_DRV_M48T59=m
# CONFIG_RTC_DRV_MSM6242 is not set
# CONFIG_RTC_DRV_BQ4802 is not set
CONFIG_RTC_DRV_RP5C01=m
CONFIG_RTC_DRV_V3020=m
CONFIG_RTC_DRV_PCF50633=m
#
# on-CPU RTC drivers
#
# CONFIG_RTC_DRV_PCAP is not set
CONFIG_RTC_DRV_MC13XXX=m
# CONFIG_DMADEVICES is not set
CONFIG_AUXDISPLAY=y
CONFIG_UIO=m
# CONFIG_UIO_CIF is not set
# CONFIG_UIO_PDRV is not set
CONFIG_UIO_PDRV_GENIRQ=m
CONFIG_UIO_AEC=m
# CONFIG_UIO_SERCOS3 is not set
CONFIG_UIO_PCI_GENERIC=m
CONFIG_UIO_NETX=m
# CONFIG_STAGING is not set
# CONFIG_X86_PLATFORM_DEVICES is not set
#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
# CONFIG_DMIID is not set
CONFIG_DMI_SYSFS=m
# CONFIG_ISCSI_IBFT_FIND is not set
CONFIG_SIGMA=m
#
# File systems
#
# CONFIG_EXT2_FS is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_EXT4_FS=m
# CONFIG_EXT4_USE_FOR_EXT23 is not set
# CONFIG_EXT4_FS_XATTR is not set
CONFIG_EXT4_DEBUG=y
CONFIG_JBD=y
CONFIG_JBD_DEBUG=y
CONFIG_JBD2=m
CONFIG_JBD2_DEBUG=y
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
CONFIG_JFS_FS=m
# CONFIG_JFS_POSIX_ACL is not set
# CONFIG_JFS_SECURITY is not set
CONFIG_JFS_DEBUG=y
CONFIG_JFS_STATISTICS=y
# CONFIG_XFS_FS is not set
# CONFIG_OCFS2_FS is not set
CONFIG_BTRFS_FS=m
CONFIG_BTRFS_FS_POSIX_ACL=y
# CONFIG_NILFS2_FS is not set
CONFIG_FS_POSIX_ACL=y
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_FANOTIFY=y
CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y
# CONFIG_QUOTA is not set
# CONFIG_QUOTACTL is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set
#
# Caches
#
# CONFIG_FSCACHE is not set
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
# CONFIG_JOLIET is not set
CONFIG_ZISOFS=y
# CONFIG_UDF_FS is not set
#
# DOS/FAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
# CONFIG_TMPFS is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_CONFIGFS_FS=m
# CONFIG_MISC_FILESYSTEMS is not set
# CONFIG_NETWORK_FILESYSTEMS is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
CONFIG_ACORN_PARTITION=y
# CONFIG_ACORN_PARTITION_CUMANA is not set
# CONFIG_ACORN_PARTITION_EESOX is not set
# CONFIG_ACORN_PARTITION_ICS is not set
# CONFIG_ACORN_PARTITION_ADFS is not set
# CONFIG_ACORN_PARTITION_POWERTEC is not set
CONFIG_ACORN_PARTITION_RISCIX=y
# CONFIG_OSF_PARTITION is not set
CONFIG_AMIGA_PARTITION=y
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
CONFIG_LDM_PARTITION=y
CONFIG_LDM_DEBUG=y
# CONFIG_SGI_PARTITION is not set
CONFIG_ULTRIX_PARTITION=y
# CONFIG_SUN_PARTITION is not set
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_BASE=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
# CONFIG_NLS_CODEPAGE_737 is not set
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
# CONFIG_NLS_CODEPAGE_855 is not set
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
# CONFIG_NLS_CODEPAGE_863 is not set
CONFIG_NLS_CODEPAGE_864=m
# CONFIG_NLS_CODEPAGE_865 is not set
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
# CONFIG_NLS_CODEPAGE_936 is not set
CONFIG_NLS_CODEPAGE_950=m
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
CONFIG_NLS_ISO8859_8=m
# CONFIG_NLS_CODEPAGE_1250 is not set
CONFIG_NLS_CODEPAGE_1251=m
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=m
# CONFIG_NLS_ISO8859_2 is not set
CONFIG_NLS_ISO8859_3=m
# CONFIG_NLS_ISO8859_4 is not set
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
# CONFIG_NLS_KOI8_R is not set
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m
CONFIG_DLM=m
# CONFIG_DLM_DEBUG is not set
#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
CONFIG_DEFAULT_MESSAGE_LOGLEVEL=4
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=1024
CONFIG_MAGIC_SYSRQ=y
CONFIG_STRIP_ASM_SYMS=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_DEBUG_FS=y
CONFIG_HEADERS_CHECK=y
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
# CONFIG_LOCKUP_DETECTOR is not set
# CONFIG_HARDLOCKUP_DETECTOR is not set
# CONFIG_DETECT_HUNG_TASK is not set
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_DEBUG_ON is not set
CONFIG_SLUB_STATS=y
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
# CONFIG_PROVE_LOCKING is not set
# CONFIG_SPARSE_RCU_POINTER is not set
CONFIG_LOCKDEP=y
CONFIG_LOCK_STAT=y
CONFIG_DEBUG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_HIGHMEM=y
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
CONFIG_DEBUG_VM=y
CONFIG_DEBUG_VIRTUAL=y
CONFIG_DEBUG_WRITECOUNT=y
CONFIG_DEBUG_MEMORY_INIT=y
# CONFIG_DEBUG_LIST is not set
CONFIG_TEST_LIST_SORT=y
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_DEBUG_CREDENTIALS is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
CONFIG_BOOT_PRINTK_DELAY=y
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=60
CONFIG_BACKTRACE_SELF_TEST=m
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
CONFIG_DEBUG_FORCE_WEAK_PER_CPU=y
CONFIG_LKDTM=m
# CONFIG_CPU_NOTIFIER_ERROR_INJECT is not set
CONFIG_FAULT_INJECTION=y
CONFIG_FAILSLAB=y
# CONFIG_FAIL_PAGE_ALLOC is not set
# CONFIG_FAIL_MAKE_REQUEST is not set
# CONFIG_FAIL_IO_TIMEOUT is not set
CONFIG_FAULT_INJECTION_DEBUG_FS=y
# CONFIG_FAULT_INJECTION_STACKTRACE_FILTER is not set
CONFIG_LATENCYTOP=y
CONFIG_SYSCTL_SYSCALL_CHECK=y
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_BUILD_DOCSRC is not set
CONFIG_DYNAMIC_DEBUG=y
CONFIG_DMA_API_DEBUG=y
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
# CONFIG_KMEMCHECK is not set
# CONFIG_TEST_KSTRTOX is not set
CONFIG_STRICT_DEVMEM=y
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_EARLY_PRINTK=y
# CONFIG_EARLY_PRINTK_DBGP is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
CONFIG_DEBUG_STACK_USAGE=y
CONFIG_DEBUG_PER_CPU_MAPS=y
# CONFIG_X86_PTDUMP is not set
CONFIG_DEBUG_RODATA=y
# CONFIG_DEBUG_RODATA_TEST is not set
CONFIG_DEBUG_SET_MODULE_RONX=y
CONFIG_DEBUG_NX_TEST=m
CONFIG_DOUBLEFAULT=y
CONFIG_IOMMU_STRESS=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
# CONFIG_IO_DELAY_0X80 is not set
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
CONFIG_IO_DELAY_NONE=y
CONFIG_DEFAULT_IO_DELAY_TYPE=3
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
CONFIG_OPTIMIZE_INLINING=y
#
# Security options
#
CONFIG_KEYS=y
# CONFIG_TRUSTED_KEYS is not set
# CONFIG_KEYS_DEBUG_PROC_KEYS is not set
CONFIG_SECURITY_DMESG_RESTRICT=y
CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
# CONFIG_SECURITY_NETWORK_XFRM is not set
CONFIG_SECURITY_PATH=y
# CONFIG_SECURITY_SELINUX is not set
# CONFIG_SECURITY_SMACK is not set
CONFIG_SECURITY_TOMOYO=y
# CONFIG_SECURITY_APPARMOR is not set
CONFIG_IMA=y
CONFIG_IMA_MEASURE_PCR_IDX=10
CONFIG_IMA_AUDIT=y
CONFIG_DEFAULT_SECURITY_TOMOYO=y
# CONFIG_DEFAULT_SECURITY_DAC is not set
CONFIG_DEFAULT_SECURITY="tomoyo"
CONFIG_CRYPTO=y
#
# Crypto core or helper
#
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=m
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=m
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=m
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_PCOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
CONFIG_CRYPTO_GF128MUL=m
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_PCRYPT is not set
CONFIG_CRYPTO_WORKQUEUE=y
CONFIG_CRYPTO_CRYPTD=m
# CONFIG_CRYPTO_AUTHENC is not set
CONFIG_CRYPTO_TEST=m
#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
CONFIG_CRYPTO_SEQIV=m
#
# Block modes
#
CONFIG_CRYPTO_CBC=m
CONFIG_CRYPTO_CTR=m
# CONFIG_CRYPTO_CTS is not set
CONFIG_CRYPTO_ECB=m
CONFIG_CRYPTO_LRW=m
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_XTS=m
#
# Hash modes
#
CONFIG_CRYPTO_HMAC=y
# CONFIG_CRYPTO_XCBC is not set
# CONFIG_CRYPTO_VMAC is not set
#
# Digest
#
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_CRC32C_INTEL=m
CONFIG_CRYPTO_GHASH=m
# CONFIG_CRYPTO_MD4 is not set
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=m
# CONFIG_CRYPTO_RMD128 is not set
CONFIG_CRYPTO_RMD160=m
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not set
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=m
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_TGR192 is not set
# CONFIG_CRYPTO_WP512 is not set
#
# Ciphers
#
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_586=m
# CONFIG_CRYPTO_AES_NI_INTEL is not set
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_ARC4=m
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_CAMELLIA is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_KHAZAD=m
# CONFIG_CRYPTO_SALSA20 is not set
CONFIG_CRYPTO_SALSA20_586=m
# CONFIG_CRYPTO_SEED is not set
# CONFIG_CRYPTO_SERPENT is not set
CONFIG_CRYPTO_TEA=m
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_TWOFISH_586 is not set
#
# Compression
#
CONFIG_CRYPTO_DEFLATE=m
# CONFIG_CRYPTO_ZLIB is not set
CONFIG_CRYPTO_LZO=m
#
# Random Number Generation
#
# CONFIG_CRYPTO_ANSI_CPRNG is not set
# CONFIG_CRYPTO_USER_API_HASH is not set
# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
CONFIG_CRYPTO_HW=y
# CONFIG_CRYPTO_DEV_PADLOCK is not set
CONFIG_CRYPTO_DEV_GEODE=m
# CONFIG_CRYPTO_DEV_HIFN_795X is not set
CONFIG_HAVE_KVM=y
# CONFIG_VIRTUALIZATION is not set
# CONFIG_BINARY_PRINTF is not set
#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC_CCITT=m
CONFIG_CRC16=m
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=y
# CONFIG_CRC7 is not set
CONFIG_LIBCRC32C=m
CONFIG_AUDIT_GENERIC=y
CONFIG_ZLIB_INFLATE=m
CONFIG_ZLIB_DEFLATE=m
CONFIG_LZO_COMPRESS=m
CONFIG_LZO_DECOMPRESS=m
# CONFIG_XZ_DEC is not set
# CONFIG_XZ_DEC_BCJ is not set
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_CHECK_SIGNATURE=y
# CONFIG_CPUMASK_OFFSTACK is not set
CONFIG_CPU_RMAP=y
CONFIG_NLATTR=y
CONFIG_LRU_CACHE=m
CONFIG_AVERAGE=y
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 12:19 ` Ingo Molnar
@ 2011-05-13 13:04 ` Ingo Molnar
2011-05-13 13:12 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-13 13:04 UTC (permalink / raw)
To: Yinghai Lu; +Cc: paulmck, linux-kernel
* Ingo Molnar <mingo@elte.hu> wrote:
> btw., i'm seeing a boot hang too, with the attached config:
>
> PM: Adding info for platform:serial8250
> serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> device: 'ttyS0': device_add
> PM: Adding info for No Bus:ttyS0
> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=6002 jiffies)
> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=24034 jiffies)
> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=42066 jiffies)
> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=6002 jiffies)
> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=24034 jiffies)
>
> (and the stall warnings go on forever)
I started bisecting this, and the two relevant endpoints:
bad: 11c476f: net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
good: 0ee5623f: Linux 2.6.39-rc6
very clearly indicate that this is an RCU regression.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 13:04 ` Ingo Molnar
@ 2011-05-13 13:12 ` Ingo Molnar
2011-05-13 14:14 ` Paul E. McKenney
2011-05-13 14:40 ` Ingo Molnar
0 siblings, 2 replies; 76+ messages in thread
From: Ingo Molnar @ 2011-05-13 13:12 UTC (permalink / raw)
To: Yinghai Lu; +Cc: paulmck, linux-kernel
* Ingo Molnar <mingo@elte.hu> wrote:
> I started bisecting this, and the two relevant endpoints:
>
> bad: 11c476f: net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
> good: 0ee5623f: Linux 2.6.39-rc6
>
> very clearly indicate that this is an RCU regression.
This might be the same one Yinghai found:
e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
So with the config i sent it's definitely reproducible.
At first sight couldnt this be related not to barriers, but to not setting
need_resched() like we did before?
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 13:12 ` Ingo Molnar
@ 2011-05-13 14:14 ` Paul E. McKenney
2011-05-13 15:07 ` Ingo Molnar
2011-05-13 14:40 ` Ingo Molnar
1 sibling, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-13 14:14 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Fri, May 13, 2011 at 03:12:18PM +0200, Ingo Molnar wrote:
>
> * Ingo Molnar <mingo@elte.hu> wrote:
>
> > I started bisecting this, and the two relevant endpoints:
> >
> > bad: 11c476f: net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
> > good: 0ee5623f: Linux 2.6.39-rc6
> >
> > very clearly indicate that this is an RCU regression.
>
> This might be the same one Yinghai found:
>
> e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
>
> So with the config i sent it's definitely reproducible.
>
> At first sight couldnt this be related not to barriers, but to not setting
> need_resched() like we did before?
Thank you both!!! I had inspected the commit, but missed the fact that
the new version refuses to call set_need_resched() if irqs are enabled. :-(
The following (untested) patch restores the set_need_resched() operation.
Does this help?
Thanx, Paul
------------------------------------------------------------------------
rcu: Reschedule from dyntick even if irqs are enabled.
Commit e59fb3120bec (rcu: Decrease memory-barrier usage based on
semi-formal proof) attempted to optimize by invoking set_need_resched()
only when irqs are disabled. This patch restores the prior behavior
of invoking set_need_resched() even if irqs are enabled, but leaves
the memory-barrier-usage changes untouched.
Reported-by: Yinghai Lu <yinghai@kernel.org>
Reported-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 5b0b482..8c490ef 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -335,10 +335,9 @@ void rcu_enter_nohz(void)
local_irq_restore(flags);
/* If the interrupt queued a callback, get out of dyntick mode. */
- if (in_irq() &&
- (__get_cpu_var(rcu_sched_data).nxtlist ||
- __get_cpu_var(rcu_bh_data).nxtlist ||
- rcu_preempt_needs_cpu(smp_processor_id())))
+ if (__get_cpu_var(rcu_sched_data).nxtlist ||
+ __get_cpu_var(rcu_bh_data).nxtlist ||
+ rcu_preempt_needs_cpu(smp_processor_id()))
set_need_resched();
}
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 13:12 ` Ingo Molnar
2011-05-13 14:14 ` Paul E. McKenney
@ 2011-05-13 14:40 ` Ingo Molnar
2011-05-13 16:38 ` Paul E. McKenney
1 sibling, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-13 14:40 UTC (permalink / raw)
To: Yinghai Lu; +Cc: paulmck, linux-kernel
* Ingo Molnar <mingo@elte.hu> wrote:
> > very clearly indicate that this is an RCU regression.
>
> This might be the same one Yinghai found:
>
> e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
>
> So with the config i sent it's definitely reproducible.
yes. And note that the system does not lock up, but boots up after a really
long time - perhaps as a side-effect of very rare timer irqs driving forward
the RCU state machine?
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 14:14 ` Paul E. McKenney
@ 2011-05-13 15:07 ` Ingo Molnar
2011-05-13 16:26 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-13 15:07 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> On Fri, May 13, 2011 at 03:12:18PM +0200, Ingo Molnar wrote:
> >
> > * Ingo Molnar <mingo@elte.hu> wrote:
> >
> > > I started bisecting this, and the two relevant endpoints:
> > >
> > > bad: 11c476f: net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
> > > good: 0ee5623f: Linux 2.6.39-rc6
> > >
> > > very clearly indicate that this is an RCU regression.
> >
> > This might be the same one Yinghai found:
> >
> > e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
> >
> > So with the config i sent it's definitely reproducible.
> >
> > At first sight couldnt this be related not to barriers, but to not setting
> > need_resched() like we did before?
>
> Thank you both!!! I had inspected the commit, but missed the fact that
> the new version refuses to call set_need_resched() if irqs are enabled. :-(
> The following (untested) patch restores the set_need_resched() operation.
Btw., in hindsight, e59fb3120bec was a tad big, which made analysis harder.
Would it have been possible to split it in two, one for the movement of the
notifiers, the other for the barrier changes?
That way the bisection would have fingered the movement commit. Or so.
> Does this help?
No, unfortunately not, the long delay is still there:
device: 'ttyS0': device_add
PM: Adding info for No Bus:ttyS0
INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=6002 jiffies)
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 15:07 ` Ingo Molnar
@ 2011-05-13 16:26 ` Paul E. McKenney
2011-05-16 7:08 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-13 16:26 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Fri, May 13, 2011 at 05:07:44PM +0200, Ingo Molnar wrote:
>
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>
> > On Fri, May 13, 2011 at 03:12:18PM +0200, Ingo Molnar wrote:
> > >
> > > * Ingo Molnar <mingo@elte.hu> wrote:
> > >
> > > > I started bisecting this, and the two relevant endpoints:
> > > >
> > > > bad: 11c476f: net,rcu: convert call_rcu(prl_entry_destroy_rcu) to kfree
> > > > good: 0ee5623f: Linux 2.6.39-rc6
> > > >
> > > > very clearly indicate that this is an RCU regression.
> > >
> > > This might be the same one Yinghai found:
> > >
> > > e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
> > >
> > > So with the config i sent it's definitely reproducible.
> > >
> > > At first sight couldnt this be related not to barriers, but to not setting
> > > need_resched() like we did before?
> >
> > Thank you both!!! I had inspected the commit, but missed the fact that
> > the new version refuses to call set_need_resched() if irqs are enabled. :-(
> > The following (untested) patch restores the set_need_resched() operation.
>
> Btw., in hindsight, e59fb3120bec was a tad big, which made analysis harder.
>
> Would it have been possible to split it in two, one for the movement of the
> notifiers, the other for the barrier changes?
>
> That way the bisection would have fingered the movement commit. Or so.
In hindsight, that certainly would have been better.
> > Does this help?
>
> No, unfortunately not, the long delay is still there:
>
> device: 'ttyS0': device_add
> PM: Adding info for No Bus:ttyS0
> INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 1, t=6002 jiffies)
I was afraid of that...
On the off-chance that moving the memory barriers was at fault,
the following patch restores all of them that don't have in situ
replacements. Grasping at straws, admittedly.
Thanx, Paul
------------------------------------------------------------------------
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 8c490ef..a4a2ef0 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1449,10 +1449,12 @@ __rcu_process_callbacks(struct rcu_state *rsp, struct rcu_data *rdp)
*/
static void rcu_process_callbacks(void)
{
+ smp_mb();
__rcu_process_callbacks(&rcu_sched_state,
&__get_cpu_var(rcu_sched_data));
__rcu_process_callbacks(&rcu_bh_state, &__get_cpu_var(rcu_bh_data));
rcu_preempt_process_callbacks();
+ smp_mb();
/* If we are last CPU on way to dyntick-idle mode, accelerate it. */
rcu_needs_cpu_flush();
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 14:40 ` Ingo Molnar
@ 2011-05-13 16:38 ` Paul E. McKenney
2011-05-16 7:10 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-13 16:38 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Fri, May 13, 2011 at 04:40:59PM +0200, Ingo Molnar wrote:
>
> * Ingo Molnar <mingo@elte.hu> wrote:
>
> > > very clearly indicate that this is an RCU regression.
> >
> > This might be the same one Yinghai found:
> >
> > e59fb3120bec: rcu: Decrease memory-barrier usage based on semi-formal proof
> >
> > So with the config i sent it's definitely reproducible.
>
> yes. And note that the system does not lock up, but boots up after a really
> long time - perhaps as a side-effect of very rare timer irqs driving forward
> the RCU state machine?
In the interest of focusing on the code paths that are actually being
executed...
Are you guys seeing NMIs at about the time the slowdown occurs?
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-12 21:36 ` Yinghai Lu
2011-05-13 1:28 ` Yinghai Lu
@ 2011-05-13 21:08 ` Yinghai Lu
2011-05-14 14:26 ` Paul E. McKenney
1 sibling, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-13 21:08 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On Thu, May 12, 2011 at 2:36 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On 05/12/2011 02:20 AM, Paul E. McKenney wrote:
>> On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
>>> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
>>>> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
>>>>>
>>>>> * Yinghai Lu <yinghai@kernel.org> wrote:
>>>>>
>>>>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
>>>>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
>>>>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>>>>>> Date: Tue Sep 7 10:38:22 2010 -0700
>>>>>>
>>>>>> rcu: Decrease memory-barrier usage based on semi-formal proof
>>>>>
>>>>> Find below an (untested!) attempt at reverting it for debugging purposes: could
>>>>> you please try it, does your system now boot up fine?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ingo
>>>>>
>>>>
>>>> yes, reverted manually that commit fix the problem.
>>>
>>> on system with 8 sockets westmere-ex
>>>
>>> it seems other commits after that commit contribute some delay too.
>>>
>>> [ 32.240739] cpu_dev_init done
>>> [ 73.587288] memory_dev_init done
>>
>> I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
>> will chase down the delay.
>>
>
> it seems still need to revert following one in addition e59fb3120becfb36b22ddb8bd27d065d3cdca499.
>
> [root@mpk14-2404-239-158 linux-2.6]# git bisect good
> a26ac2455ffcf3be5c6ef92bc6df7182700f2114 is the first bad commit
> commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> Author: Paul E. McKenney <paul.mckenney@linaro.org>
> Date: Wed Jan 12 14:10:23 2011 -0800
>
> rcu: move TREE_RCU from softirq to kthread
>
> If RCU priority boosting is to be meaningful, callback invocation must
> be boosted in addition to preempted RCU readers. Otherwise, in presence
> of CPU real-time threads, the grace period ends, but the callbacks don't
> get invoked. If the callbacks don't get invoked, the associated memory
> doesn't get freed, so the system is still subject to OOM.
>
> But it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit
> moves the callback invocations to a kthread, which can be boosted easily.
>
> Also add comments and properly synchronized all accesses to
> rcu_cpu_kthread_task, as suggested by Lai Jiangshan.
>
> Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Reviewed-by: Josh Triplett <josh@joshtriplett.org>
>
> :040000 040000 e40306ac6405952c1d387325a98588442209abe8 efe9ea2f408c62daaccf49e6d1339dff3a74f049 M Documentation
> :040000 040000 8f9e7a8fa3a728d4ae58e2efb8ada7cf08aed00e 9b44deba45ba905c5d9b3cc314812f0ba3f7e639 M include
> :040000 040000 4b10b719a2d56ed4bc796a9f43775732bb5ff144 4db269277ccf607e1a6a7d7f4c2a7cf8d592d46a M kernel
> :040000 040000 881f102e6831381beed016ed240d690f6a2ccd5e 57d2fc6f84e47394c116bc617a9a0ef9b8b6dbd4 M tools
so only revert e59fb3120becfb36b22ddb8bd27d065d3cdca499 is not enough.
[ 315.248277] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 315.285642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 427.405283] INFO: rcu_sched_state detected stalls on CPUs/tasks: {
0} (detected by 50, t=15002 jiffies)
[ 427.408267] sending NMI to all CPUs:
[ 427.419298] NMI backtrace for cpu 1
[ 427.420616] CPU 1
Paul, can you make one clean revert for
| a26ac2455ffcf3be5c6ef92bc6df7182700f2114
| rcu: move TREE_RCU from softirq to kthread
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 21:08 ` Yinghai Lu
@ 2011-05-14 14:26 ` Paul E. McKenney
2011-05-14 15:31 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-14 14:26 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Fri, May 13, 2011 at 02:08:21PM -0700, Yinghai Lu wrote:
> On Thu, May 12, 2011 at 2:36 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> > On 05/12/2011 02:20 AM, Paul E. McKenney wrote:
> >> On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
> >>> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
> >>>> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
> >>>>>
> >>>>> * Yinghai Lu <yinghai@kernel.org> wrote:
> >>>>>
> >>>>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
> >>>>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
> >>>>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> >>>>>> Date: Tue Sep 7 10:38:22 2010 -0700
> >>>>>>
> >>>>>> rcu: Decrease memory-barrier usage based on semi-formal proof
> >>>>>
> >>>>> Find below an (untested!) attempt at reverting it for debugging purposes: could
> >>>>> you please try it, does your system now boot up fine?
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Ingo
> >>>>>
> >>>>
> >>>> yes, reverted manually that commit fix the problem.
> >>>
> >>> on system with 8 sockets westmere-ex
> >>>
> >>> it seems other commits after that commit contribute some delay too.
> >>>
> >>> [ 32.240739] cpu_dev_init done
> >>> [ 73.587288] memory_dev_init done
> >>
> >> I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
> >> will chase down the delay.
> >>
> >
> > it seems still need to revert following one in addition e59fb3120becfb36b22ddb8bd27d065d3cdca499.
> >
> > [root@mpk14-2404-239-158 linux-2.6]# git bisect good
> > a26ac2455ffcf3be5c6ef92bc6df7182700f2114 is the first bad commit
> > commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> > Author: Paul E. McKenney <paul.mckenney@linaro.org>
> > Date: Wed Jan 12 14:10:23 2011 -0800
> >
> > rcu: move TREE_RCU from softirq to kthread
> >
> > If RCU priority boosting is to be meaningful, callback invocation must
> > be boosted in addition to preempted RCU readers. Otherwise, in presence
> > of CPU real-time threads, the grace period ends, but the callbacks don't
> > get invoked. If the callbacks don't get invoked, the associated memory
> > doesn't get freed, so the system is still subject to OOM.
> >
> > But it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit
> > moves the callback invocations to a kthread, which can be boosted easily.
> >
> > Also add comments and properly synchronized all accesses to
> > rcu_cpu_kthread_task, as suggested by Lai Jiangshan.
> >
> > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > Reviewed-by: Josh Triplett <josh@joshtriplett.org>
> >
> > :040000 040000 e40306ac6405952c1d387325a98588442209abe8 efe9ea2f408c62daaccf49e6d1339dff3a74f049 M Documentation
> > :040000 040000 8f9e7a8fa3a728d4ae58e2efb8ada7cf08aed00e 9b44deba45ba905c5d9b3cc314812f0ba3f7e639 M include
> > :040000 040000 4b10b719a2d56ed4bc796a9f43775732bb5ff144 4db269277ccf607e1a6a7d7f4c2a7cf8d592d46a M kernel
> > :040000 040000 881f102e6831381beed016ed240d690f6a2ccd5e 57d2fc6f84e47394c116bc617a9a0ef9b8b6dbd4 M tools
>
> so only revert e59fb3120becfb36b22ddb8bd27d065d3cdca499 is not enough.
>
> [ 315.248277] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> [ 315.285642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> [ 427.405283] INFO: rcu_sched_state detected stalls on CPUs/tasks: {
> 0} (detected by 50, t=15002 jiffies)
> [ 427.408267] sending NMI to all CPUs:
> [ 427.419298] NMI backtrace for cpu 1
> [ 427.420616] CPU 1
>
> Paul, can you make one clean revert for
> | a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> | rcu: move TREE_RCU from softirq to kthread
I will be continuing to look into a few things over the weekend, but
if I cannot find the cause, then changing back to softirq might be the
thing to do. It won't be so much a revert in the "git revert" sense
due to later dependencies, but it could be shifted back from kthread
to softirq. This would certainly decrease dependence on the scheduler,
at least in the common case where ksoftirqd does not run.
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-14 14:26 ` Paul E. McKenney
@ 2011-05-14 15:31 ` Paul E. McKenney
2011-05-14 18:34 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-14 15:31 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 07:26:21AM -0700, Paul E. McKenney wrote:
> On Fri, May 13, 2011 at 02:08:21PM -0700, Yinghai Lu wrote:
> > On Thu, May 12, 2011 at 2:36 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> > > On 05/12/2011 02:20 AM, Paul E. McKenney wrote:
> > >> On Thu, May 12, 2011 at 12:42:50AM -0700, Yinghai Lu wrote:
> > >>> On 05/12/2011 12:27 AM, Yinghai Lu wrote:
> > >>>> On 05/11/2011 11:03 PM, Ingo Molnar wrote:
> > >>>>>
> > >>>>> * Yinghai Lu <yinghai@kernel.org> wrote:
> > >>>>>
> > >>>>>> e59fb3120becfb36b22ddb8bd27d065d3cdca499 is the first bad commit
> > >>>>>> commit e59fb3120becfb36b22ddb8bd27d065d3cdca499
> > >>>>>> Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > >>>>>> Date: Tue Sep 7 10:38:22 2010 -0700
> > >>>>>>
> > >>>>>> rcu: Decrease memory-barrier usage based on semi-formal proof
> > >>>>>
> > >>>>> Find below an (untested!) attempt at reverting it for debugging purposes: could
> > >>>>> you please try it, does your system now boot up fine?
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>> Ingo
> > >>>>>
> > >>>>
> > >>>> yes, reverted manually that commit fix the problem.
> > >>>
> > >>> on system with 8 sockets westmere-ex
> > >>>
> > >>> it seems other commits after that commit contribute some delay too.
> > >>>
> > >>> [ 32.240739] cpu_dev_init done
> > >>> [ 73.587288] memory_dev_init done
> > >>
> > >> I am testing a revert of e59fb3120becfb36b22ddb8bd27d065d3cdca499 and
> > >> will chase down the delay.
> > >>
> > >
> > > it seems still need to revert following one in addition e59fb3120becfb36b22ddb8bd27d065d3cdca499.
> > >
> > > [root@mpk14-2404-239-158 linux-2.6]# git bisect good
> > > a26ac2455ffcf3be5c6ef92bc6df7182700f2114 is the first bad commit
> > > commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> > > Author: Paul E. McKenney <paul.mckenney@linaro.org>
> > > Date: Wed Jan 12 14:10:23 2011 -0800
> > >
> > > rcu: move TREE_RCU from softirq to kthread
> > >
> > > If RCU priority boosting is to be meaningful, callback invocation must
> > > be boosted in addition to preempted RCU readers. Otherwise, in presence
> > > of CPU real-time threads, the grace period ends, but the callbacks don't
> > > get invoked. If the callbacks don't get invoked, the associated memory
> > > doesn't get freed, so the system is still subject to OOM.
> > >
> > > But it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit
> > > moves the callback invocations to a kthread, which can be boosted easily.
> > >
> > > Also add comments and properly synchronized all accesses to
> > > rcu_cpu_kthread_task, as suggested by Lai Jiangshan.
> > >
> > > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > > Reviewed-by: Josh Triplett <josh@joshtriplett.org>
> > >
> > > :040000 040000 e40306ac6405952c1d387325a98588442209abe8 efe9ea2f408c62daaccf49e6d1339dff3a74f049 M Documentation
> > > :040000 040000 8f9e7a8fa3a728d4ae58e2efb8ada7cf08aed00e 9b44deba45ba905c5d9b3cc314812f0ba3f7e639 M include
> > > :040000 040000 4b10b719a2d56ed4bc796a9f43775732bb5ff144 4db269277ccf607e1a6a7d7f4c2a7cf8d592d46a M kernel
> > > :040000 040000 881f102e6831381beed016ed240d690f6a2ccd5e 57d2fc6f84e47394c116bc617a9a0ef9b8b6dbd4 M tools
> >
> > so only revert e59fb3120becfb36b22ddb8bd27d065d3cdca499 is not enough.
> >
> > [ 315.248277] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> > [ 315.285642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> > [ 427.405283] INFO: rcu_sched_state detected stalls on CPUs/tasks: {
> > 0} (detected by 50, t=15002 jiffies)
> > [ 427.408267] sending NMI to all CPUs:
> > [ 427.419298] NMI backtrace for cpu 1
> > [ 427.420616] CPU 1
> >
> > Paul, can you make one clean revert for
> > | a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> > | rcu: move TREE_RCU from softirq to kthread
>
> I will be continuing to look into a few things over the weekend, but
> if I cannot find the cause, then changing back to softirq might be the
> thing to do. It won't be so much a revert in the "git revert" sense
> due to later dependencies, but it could be shifted back from kthread
> to softirq. This would certainly decrease dependence on the scheduler,
> at least in the common case where ksoftirqd does not run.
So, upon reviewing Yinghai's RCU debugfs output after getting a good
night's sleep, I see that the dyntick nesting level is getting messed up.
This is shown by the "dt=7237/73" near the end of the debugfs info of
Yinghai's message from Tue, 10 May 2011 23:42:24 -0700. This says that
RCU believes that the CPU is not in dyntick-idle mode (7237 is an odd
number) and that that there are 73 levels of not being in dyntick-idle
mode, which means at least 72 interrupt levels. Unless x86 interrupts
normally nest 72 levels deep...
This situation will cause RCU to think that a given CPU is not in
dyntick-idle mode when it really is. This results in RCU waiting on
it to respond, and eventually waking it up. Which would cause needless
grace-period delays.
Before commit e59fb31 (Decrease memory-barrier usage based on semi-formal
proof), rcu_enter_nohz() would have unconditionally caused RCU to believe
that the CPU was in dyntick-idle mode. After this commit, RCU pays attention
to the (broken) nesting count. Though the broken nesting level probably
caused some trouble even before this commit.
So I am restoring the old semantics where rcu_enter_nohz() unconditionally
tells RCU that the CPU really is in nohz mode. I am also adding
some WARN_ON_ONCE() statements that will hopefully help find where the
misnesting is occurring. I will also see if I can find the mis-nesting,
but I am not as familiar with the interrupt entry/exit code as I should
be. So I will create and sanity-test the patch and post it first,
and do the inspection afterwards.
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-14 15:31 ` Paul E. McKenney
@ 2011-05-14 18:34 ` Paul E. McKenney
2011-05-15 3:59 ` Yinghai Lu
` (2 more replies)
0 siblings, 3 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-14 18:34 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 08:31:18AM -0700, Paul E. McKenney wrote:
> On Sat, May 14, 2011 at 07:26:21AM -0700, Paul E. McKenney wrote:
> > On Fri, May 13, 2011 at 02:08:21PM -0700, Yinghai Lu wrote:
> > > On Thu, May 12, 2011 at 2:36 PM, Yinghai Lu <yinghai@kernel.org> wrote:
[ . . . ]
> > > so only revert e59fb3120becfb36b22ddb8bd27d065d3cdca499 is not enough.
> > >
> > > [ 315.248277] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> > > [ 315.285642] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> > > [ 427.405283] INFO: rcu_sched_state detected stalls on CPUs/tasks: {
> > > 0} (detected by 50, t=15002 jiffies)
> > > [ 427.408267] sending NMI to all CPUs:
> > > [ 427.419298] NMI backtrace for cpu 1
> > > [ 427.420616] CPU 1
> > >
> > > Paul, can you make one clean revert for
> > > | a26ac2455ffcf3be5c6ef92bc6df7182700f2114
> > > | rcu: move TREE_RCU from softirq to kthread
> >
> > I will be continuing to look into a few things over the weekend, but
> > if I cannot find the cause, then changing back to softirq might be the
> > thing to do. It won't be so much a revert in the "git revert" sense
> > due to later dependencies, but it could be shifted back from kthread
> > to softirq. This would certainly decrease dependence on the scheduler,
> > at least in the common case where ksoftirqd does not run.
>
> So, upon reviewing Yinghai's RCU debugfs output after getting a good
> night's sleep, I see that the dyntick nesting level is getting messed up.
> This is shown by the "dt=7237/73" near the end of the debugfs info of
> Yinghai's message from Tue, 10 May 2011 23:42:24 -0700. This says that
> RCU believes that the CPU is not in dyntick-idle mode (7237 is an odd
> number) and that that there are 73 levels of not being in dyntick-idle
> mode, which means at least 72 interrupt levels. Unless x86 interrupts
> normally nest 72 levels deep...
>
> This situation will cause RCU to think that a given CPU is not in
> dyntick-idle mode when it really is. This results in RCU waiting on
> it to respond, and eventually waking it up. Which would cause needless
> grace-period delays.
>
> Before commit e59fb31 (Decrease memory-barrier usage based on semi-formal
> proof), rcu_enter_nohz() would have unconditionally caused RCU to believe
> that the CPU was in dyntick-idle mode. After this commit, RCU pays attention
> to the (broken) nesting count. Though the broken nesting level probably
> caused some trouble even before this commit.
>
> So I am restoring the old semantics where rcu_enter_nohz() unconditionally
> tells RCU that the CPU really is in nohz mode. I am also adding
> some WARN_ON_ONCE() statements that will hopefully help find where the
> misnesting is occurring. I will also see if I can find the mis-nesting,
> but I am not as familiar with the interrupt entry/exit code as I should
> be. So I will create and sanity-test the patch and post it first,
> and do the inspection afterwards.
And here is a lightly-tested patch, which applies on tip/core/rcu.
This problem could account for both the long delays seen with e59fb312
(Decrease memory-barrier usage based on semi-formal proof) and the
shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
Thanx, Paul
------------------------------------------------------------------------
rcutree.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 75 insertions(+), 16 deletions(-)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index a4a2ef0..0b541bd 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -308,23 +308,25 @@ static int rcu_implicit_offline_qs(struct rcu_data *rdp)
#ifdef CONFIG_NO_HZ
-/**
- * rcu_enter_nohz - inform RCU that current CPU is entering nohz
+/*
+ * __rcu_enter_nohz - inform RCU that current CPU is entering nohz
*
* Enter nohz mode, in other words, -leave- the mode in which RCU
* read-side critical sections can occur. (Though RCU read-side
* critical sections can occur in irq handlers in nohz mode, a possibility
* handled by rcu_irq_enter() and rcu_irq_exit()).
+ *
+ * This variant should only be called from rcu_irq_exit() because it
+ * assumes perfect nesting. The caller must have irqs disabled.
*/
-void rcu_enter_nohz(void)
+void __rcu_enter_nohz(void)
{
- unsigned long flags;
struct rcu_dynticks *rdtp;
- local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
+ WARN_ON_ONCE(rdtp->dynticks_nesting <= 0);
if (--rdtp->dynticks_nesting) {
- local_irq_restore(flags);
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
return;
}
/* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
@@ -332,7 +334,6 @@ void rcu_enter_nohz(void)
atomic_inc(&rdtp->dynticks);
smp_mb__after_atomic_inc(); /* Force ordering with next sojourn. */
WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
- local_irq_restore(flags);
/* If the interrupt queued a callback, get out of dyntick mode. */
if (__get_cpu_var(rcu_sched_data).nxtlist ||
@@ -341,21 +342,46 @@ void rcu_enter_nohz(void)
set_need_resched();
}
+/**
+ * rcu_enter_nohz - inform RCU that current CPU is entering nohz
+ *
+ * Enter nohz mode, in other words, -leave- the mode in which RCU
+ * read-side critical sections can occur. (Though RCU read-side
+ * critical sections can occur in irq handlers in nohz mode, a possibility
+ * handled by rcu_irq_enter() and rcu_irq_exit()).
+ *
+ * Don't assume perfect nesting. There can be retries in case
+ * of interrupts arriving during the nohz-entry process.
+ */
+void rcu_enter_nohz(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ local_irq_save(flags);
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ WARN_ON_ONCE(rdtp->dynticks_nesting != 1);
+ rdtp->dynticks_nesting = 1;
+ __rcu_enter_nohz();
+ local_irq_restore(flags);
+}
+
/*
- * rcu_exit_nohz - inform RCU that current CPU is leaving nohz
+ * __rcu_exit_nohz - inform RCU that current CPU is leaving nohz
*
* Exit nohz mode, in other words, -enter- the mode in which RCU
* read-side critical sections normally occur.
+ *
+ * This should only be called from rcu_irq_enter() and rcu_exit_nohz()
+ * because it assumes perfect nesting. Caller must disable irqs.
*/
-void rcu_exit_nohz(void)
+static void __rcu_exit_nohz(void)
{
- unsigned long flags;
struct rcu_dynticks *rdtp;
- local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
if (rdtp->dynticks_nesting++) {
- local_irq_restore(flags);
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
return;
}
smp_mb__before_atomic_inc(); /* Force ordering w/previous sojourn. */
@@ -363,6 +389,27 @@ void rcu_exit_nohz(void)
/* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
smp_mb__after_atomic_inc(); /* See above. */
WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+}
+
+/**
+ * rcu_exit_nohz - inform RCU that current CPU is leaving nohz
+ *
+ * Exit nohz mode, in other words, -enter- the mode in which RCU
+ * read-side critical sections normally occur.
+ *
+ * Don't assume perfect nesting. There can be retries in case
+ * of interrupts arriving during the nohz-entry process.
+ */
+void rcu_exit_nohz(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ local_irq_save(flags);
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ WARN_ON_ONCE(rdtp->dynticks_nesting != 0);
+ rdtp->dynticks_nesting = 0;
+ __rcu_exit_nohz();
local_irq_restore(flags);
}
@@ -378,8 +425,10 @@ void rcu_nmi_enter(void)
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
if (rdtp->dynticks_nmi_nesting == 0 &&
- (atomic_read(&rdtp->dynticks) & 0x1))
+ (atomic_read(&rdtp->dynticks) & 0x1)) {
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
return;
+ }
rdtp->dynticks_nmi_nesting++;
smp_mb__before_atomic_inc(); /* Force delay from prior write. */
atomic_inc(&rdtp->dynticks);
@@ -400,8 +449,10 @@ void rcu_nmi_exit(void)
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
if (rdtp->dynticks_nmi_nesting == 0 ||
- --rdtp->dynticks_nmi_nesting != 0)
+ --rdtp->dynticks_nmi_nesting != 0) {
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
return;
+ }
/* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
smp_mb__before_atomic_inc(); /* See above. */
atomic_inc(&rdtp->dynticks);
@@ -417,7 +468,11 @@ void rcu_nmi_exit(void)
*/
void rcu_irq_enter(void)
{
- rcu_exit_nohz();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ __rcu_exit_nohz();
+ local_irq_restore(flags);
}
/**
@@ -429,7 +484,11 @@ void rcu_irq_enter(void)
*/
void rcu_irq_exit(void)
{
- rcu_enter_nohz();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ __rcu_enter_nohz();
+ local_irq_restore(flags);
}
#ifdef CONFIG_SMP
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-14 18:34 ` Paul E. McKenney
@ 2011-05-15 3:59 ` Yinghai Lu
2011-05-15 4:14 ` Yinghai Lu
2011-05-15 22:01 ` Frederic Weisbecker
2 siblings, 0 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-15 3:59 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
>> and do the inspection afterwards.
>
> And here is a lightly-tested patch, which applies on tip/core/rcu.
>
> This problem could account for both the long delays seen with e59fb312
> (Decrease memory-barrier usage based on semi-formal proof) and the
> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
yes. it fixes the problem.
for 1024g system when hotadd mem enabled in kernel config
[ 31.814803] cpu_dev_init done
[ 35.437163] memory_dev_init done
even it is with gcc from opensuse 11.3
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-14 18:34 ` Paul E. McKenney
2011-05-15 3:59 ` Yinghai Lu
@ 2011-05-15 4:14 ` Yinghai Lu
2011-05-15 5:41 ` Yinghai Lu
2011-05-15 22:01 ` Frederic Weisbecker
2 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-15 4:14 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
>> and do the inspection afterwards.
>
> And here is a lightly-tested patch, which applies on tip/core/rcu.
>
> This problem could account for both the long delays seen with e59fb312
> (Decrease memory-barrier usage based on semi-formal proof) and the
> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
yes. it fixes the problem.
for 1024g system when hotadd mem enabled in kernel config
[ 31.814803] cpu_dev_init done
[ 35.437163] memory_dev_init done
even it is with gcc from opensuse 11.3
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 4:14 ` Yinghai Lu
@ 2011-05-15 5:41 ` Yinghai Lu
2011-05-15 5:49 ` Yinghai Lu
2011-05-15 6:01 ` Paul E. McKenney
0 siblings, 2 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-15 5:41 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/14/2011 09:14 PM, Yinghai Lu wrote:
> On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
>>> and do the inspection afterwards.
>>
>> And here is a lightly-tested patch, which applies on tip/core/rcu.
>>
>> This problem could account for both the long delays seen with e59fb312
>> (Decrease memory-barrier usage based on semi-formal proof) and the
>> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
>
> yes. it fixes the problem.
>
> for 1024g system when hotadd mem enabled in kernel config
>
> [ 31.814803] cpu_dev_init done
> [ 35.437163] memory_dev_init done
>
> even it is with gcc from opensuse 11.3
got:
[ 86.931217] Switched to NOHz mode on CPU #0
[ 86.931272] Switched to NOHz mode on CPU #25
[ 86.931278] ------------[ cut here ]------------
[ 86.931290] WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x44/0x76()
[ 86.931294] Hardware name: Sun Fire X4800 M2
[ 86.931297] Modules linked in:
[ 86.931303] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-04836-g5e42dc2-dirty #3
[ 86.931307] Call Trace:
[ 86.931333] [<ffffffff81080280>] warn_slowpath_common+0x85/0x9d
[ 86.931338] Switched to NOHz mode on CPU #74
[ 86.931346] [<ffffffff810802b2>] warn_slowpath_null+0x1a/0x1c
[ 86.931356] [<ffffffff810d3615>] rcu_enter_nohz+0x44/0x76
[ 86.931370] [<ffffffff810ab3cb>] tick_nohz_stop_sched_tick+0x27d/0x366
[ 86.931381] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
[ 86.931397] [<ffffffff81bd1aa3>] rest_init+0xb7/0xbe
[ 86.931408] [<ffffffff81bd19ec>] ? csum_partial_copy_generic+0x16c/0x16c
[ 86.931423] [<ffffffff82738e39>] start_kernel+0x3b2/0x3bd
[ 86.931428] Switched to NOHz mode on CPU #94
[ 86.931436] [<ffffffff827382cc>] x86_64_start_reservations+0x9c/0xa0
[ 86.931446] [<ffffffff827384a8>] x86_64_start_kernel+0x1d8/0x1e3
[ 86.931463] ---[ end trace 2cfc591bf7de931f ]---
[ 86.931598] Switched to NOHz mode on CPU #151
[ 86.931613] Switched to NOHz mode on CPU #152
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 5:41 ` Yinghai Lu
@ 2011-05-15 5:49 ` Yinghai Lu
2011-05-15 6:04 ` Paul E. McKenney
2011-05-15 6:01 ` Paul E. McKenney
1 sibling, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-15 5:49 UTC (permalink / raw)
To: paulmck; +Cc: Ingo Molnar, linux-kernel
On 05/14/2011 10:41 PM, Yinghai Lu wrote:
> On 05/14/2011 09:14 PM, Yinghai Lu wrote:
>> On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
>>>> and do the inspection afterwards.
>>>
>>> And here is a lightly-tested patch, which applies on tip/core/rcu.
>>>
>>> This problem could account for both the long delays seen with e59fb312
>>> (Decrease memory-barrier usage based on semi-formal proof) and the
>>> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
>>
>> yes. it fixes the problem.
>>
>> for 1024g system when hotadd mem enabled in kernel config
>>
>> [ 31.814803] cpu_dev_init done
>> [ 35.437163] memory_dev_init done
>>
>> even it is with gcc from opensuse 11.3
>
> got:
>
> [ 86.931217] Switched to NOHz mode on CPU #0
> [ 86.931272] Switched to NOHz mode on CPU #25
> [ 86.931278] ------------[ cut here ]------------
> [ 86.931290] WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x44/0x76()
> [ 86.931294] Hardware name: Sun Fire X4800 M2
> [ 86.931297] Modules linked in:
> [ 86.931303] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-04836-g5e42dc2-dirty #3
> [ 86.931307] Call Trace:
> [ 86.931333] [<ffffffff81080280>] warn_slowpath_common+0x85/0x9d
> [ 86.931338] Switched to NOHz mode on CPU #74
> [ 86.931346] [<ffffffff810802b2>] warn_slowpath_null+0x1a/0x1c
> [ 86.931356] [<ffffffff810d3615>] rcu_enter_nohz+0x44/0x76
> [ 86.931370] [<ffffffff810ab3cb>] tick_nohz_stop_sched_tick+0x27d/0x366
> [ 86.931381] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> [ 86.931397] [<ffffffff81bd1aa3>] rest_init+0xb7/0xbe
> [ 86.931408] [<ffffffff81bd19ec>] ? csum_partial_copy_generic+0x16c/0x16c
> [ 86.931423] [<ffffffff82738e39>] start_kernel+0x3b2/0x3bd
> [ 86.931428] Switched to NOHz mode on CPU #94
> [ 86.931436] [<ffffffff827382cc>] x86_64_start_reservations+0x9c/0xa0
> [ 86.931446] [<ffffffff827384a8>] x86_64_start_kernel+0x1d8/0x1e3
> [ 86.931463] ---[ end trace 2cfc591bf7de931f ]---
> [ 86.931598] Switched to NOHz mode on CPU #151
> [ 86.931613] Switched to NOHz mode on CPU #152
it seems gcc from Fedora 14 is not happy with this patch.
[ 35.113696] cpu_dev_init done
[ 155.963662] memory_dev_init done
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 5:41 ` Yinghai Lu
2011-05-15 5:49 ` Yinghai Lu
@ 2011-05-15 6:01 ` Paul E. McKenney
1 sibling, 0 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-15 6:01 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 10:41:29PM -0700, Yinghai Lu wrote:
> On 05/14/2011 09:14 PM, Yinghai Lu wrote:
> > On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
> >>> and do the inspection afterwards.
> >>
> >> And here is a lightly-tested patch, which applies on tip/core/rcu.
> >>
> >> This problem could account for both the long delays seen with e59fb312
> >> (Decrease memory-barrier usage based on semi-formal proof) and the
> >> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
> >
> > yes. it fixes the problem.
> >
> > for 1024g system when hotadd mem enabled in kernel config
> >
> > [ 31.814803] cpu_dev_init done
> > [ 35.437163] memory_dev_init done
> >
> > even it is with gcc from opensuse 11.3
>
> got:
>
> [ 86.931217] Switched to NOHz mode on CPU #0
> [ 86.931272] Switched to NOHz mode on CPU #25
> [ 86.931278] ------------[ cut here ]------------
> [ 86.931290] WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x44/0x76()
> [ 86.931294] Hardware name: Sun Fire X4800 M2
> [ 86.931297] Modules linked in:
> [ 86.931303] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-04836-g5e42dc2-dirty #3
> [ 86.931307] Call Trace:
> [ 86.931333] [<ffffffff81080280>] warn_slowpath_common+0x85/0x9d
> [ 86.931338] Switched to NOHz mode on CPU #74
> [ 86.931346] [<ffffffff810802b2>] warn_slowpath_null+0x1a/0x1c
> [ 86.931356] [<ffffffff810d3615>] rcu_enter_nohz+0x44/0x76
> [ 86.931370] [<ffffffff810ab3cb>] tick_nohz_stop_sched_tick+0x27d/0x366
> [ 86.931381] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> [ 86.931397] [<ffffffff81bd1aa3>] rest_init+0xb7/0xbe
> [ 86.931408] [<ffffffff81bd19ec>] ? csum_partial_copy_generic+0x16c/0x16c
> [ 86.931423] [<ffffffff82738e39>] start_kernel+0x3b2/0x3bd
> [ 86.931428] Switched to NOHz mode on CPU #94
> [ 86.931436] [<ffffffff827382cc>] x86_64_start_reservations+0x9c/0xa0
> [ 86.931446] [<ffffffff827384a8>] x86_64_start_kernel+0x1d8/0x1e3
> [ 86.931463] ---[ end trace 2cfc591bf7de931f ]---
> [ 86.931598] Switched to NOHz mode on CPU #151
> [ 86.931613] Switched to NOHz mode on CPU #152
As I expected!
There is a dyntick entry/exit mismatch somewhere. I haven't yet been able
to find it by inspection, and I cannot reproduce on the systems that I
have access to.
One way that this could happen is if the interrupt-exit code on your
architecture sometimes failed to call irq_exit().
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 5:49 ` Yinghai Lu
@ 2011-05-15 6:04 ` Paul E. McKenney
2011-05-15 6:59 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-15 6:04 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 10:49:31PM -0700, Yinghai Lu wrote:
> On 05/14/2011 10:41 PM, Yinghai Lu wrote:
> > On 05/14/2011 09:14 PM, Yinghai Lu wrote:
> >> On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
> >>>> and do the inspection afterwards.
> >>>
> >>> And here is a lightly-tested patch, which applies on tip/core/rcu.
> >>>
> >>> This problem could account for both the long delays seen with e59fb312
> >>> (Decrease memory-barrier usage based on semi-formal proof) and the
> >>> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
> >>
> >> yes. it fixes the problem.
> >>
> >> for 1024g system when hotadd mem enabled in kernel config
> >>
> >> [ 31.814803] cpu_dev_init done
> >> [ 35.437163] memory_dev_init done
> >>
> >> even it is with gcc from opensuse 11.3
> >
> > got:
> >
> > [ 86.931217] Switched to NOHz mode on CPU #0
> > [ 86.931272] Switched to NOHz mode on CPU #25
> > [ 86.931278] ------------[ cut here ]------------
> > [ 86.931290] WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x44/0x76()
> > [ 86.931294] Hardware name: Sun Fire X4800 M2
> > [ 86.931297] Modules linked in:
> > [ 86.931303] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-04836-g5e42dc2-dirty #3
> > [ 86.931307] Call Trace:
> > [ 86.931333] [<ffffffff81080280>] warn_slowpath_common+0x85/0x9d
> > [ 86.931338] Switched to NOHz mode on CPU #74
> > [ 86.931346] [<ffffffff810802b2>] warn_slowpath_null+0x1a/0x1c
> > [ 86.931356] [<ffffffff810d3615>] rcu_enter_nohz+0x44/0x76
> > [ 86.931370] [<ffffffff810ab3cb>] tick_nohz_stop_sched_tick+0x27d/0x366
> > [ 86.931381] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> > [ 86.931397] [<ffffffff81bd1aa3>] rest_init+0xb7/0xbe
> > [ 86.931408] [<ffffffff81bd19ec>] ? csum_partial_copy_generic+0x16c/0x16c
> > [ 86.931423] [<ffffffff82738e39>] start_kernel+0x3b2/0x3bd
> > [ 86.931428] Switched to NOHz mode on CPU #94
> > [ 86.931436] [<ffffffff827382cc>] x86_64_start_reservations+0x9c/0xa0
> > [ 86.931446] [<ffffffff827384a8>] x86_64_start_kernel+0x1d8/0x1e3
> > [ 86.931463] ---[ end trace 2cfc591bf7de931f ]---
> > [ 86.931598] Switched to NOHz mode on CPU #151
> > [ 86.931613] Switched to NOHz mode on CPU #152
>
> it seems gcc from Fedora 14 is not happy with this patch.
>
> [ 35.113696] cpu_dev_init done
> [ 155.963662] memory_dev_init done
Hmmm... It looks like my attempts to make RCU recover from misnesting are
not completely foolproof. I will be especially happy to look into this
if you could look for the source of the irq_enter()/irq_exit() misnesting.
(And yes, it still might be a bug in my code -- I will be looking at that
yet again as well.)
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 6:04 ` Paul E. McKenney
@ 2011-05-15 6:59 ` Paul E. McKenney
2011-05-16 7:08 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-15 6:59 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 11:04:15PM -0700, Paul E. McKenney wrote:
> On Sat, May 14, 2011 at 10:49:31PM -0700, Yinghai Lu wrote:
> > On 05/14/2011 10:41 PM, Yinghai Lu wrote:
> > > On 05/14/2011 09:14 PM, Yinghai Lu wrote:
> > >> On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
> > >>>> and do the inspection afterwards.
> > >>>
> > >>> And here is a lightly-tested patch, which applies on tip/core/rcu.
> > >>>
> > >>> This problem could account for both the long delays seen with e59fb312
> > >>> (Decrease memory-barrier usage based on semi-formal proof) and the
> > >>> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
> > >>
> > >> yes. it fixes the problem.
> > >>
> > >> for 1024g system when hotadd mem enabled in kernel config
> > >>
> > >> [ 31.814803] cpu_dev_init done
> > >> [ 35.437163] memory_dev_init done
> > >>
> > >> even it is with gcc from opensuse 11.3
> > >
> > > got:
> > >
> > > [ 86.931217] Switched to NOHz mode on CPU #0
> > > [ 86.931272] Switched to NOHz mode on CPU #25
> > > [ 86.931278] ------------[ cut here ]------------
> > > [ 86.931290] WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x44/0x76()
> > > [ 86.931294] Hardware name: Sun Fire X4800 M2
> > > [ 86.931297] Modules linked in:
> > > [ 86.931303] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-04836-g5e42dc2-dirty #3
> > > [ 86.931307] Call Trace:
> > > [ 86.931333] [<ffffffff81080280>] warn_slowpath_common+0x85/0x9d
> > > [ 86.931338] Switched to NOHz mode on CPU #74
> > > [ 86.931346] [<ffffffff810802b2>] warn_slowpath_null+0x1a/0x1c
> > > [ 86.931356] [<ffffffff810d3615>] rcu_enter_nohz+0x44/0x76
> > > [ 86.931370] [<ffffffff810ab3cb>] tick_nohz_stop_sched_tick+0x27d/0x366
> > > [ 86.931381] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> > > [ 86.931397] [<ffffffff81bd1aa3>] rest_init+0xb7/0xbe
> > > [ 86.931408] [<ffffffff81bd19ec>] ? csum_partial_copy_generic+0x16c/0x16c
> > > [ 86.931423] [<ffffffff82738e39>] start_kernel+0x3b2/0x3bd
> > > [ 86.931428] Switched to NOHz mode on CPU #94
> > > [ 86.931436] [<ffffffff827382cc>] x86_64_start_reservations+0x9c/0xa0
> > > [ 86.931446] [<ffffffff827384a8>] x86_64_start_kernel+0x1d8/0x1e3
> > > [ 86.931463] ---[ end trace 2cfc591bf7de931f ]---
> > > [ 86.931598] Switched to NOHz mode on CPU #151
> > > [ 86.931613] Switched to NOHz mode on CPU #152
> >
> > it seems gcc from Fedora 14 is not happy with this patch.
> >
> > [ 35.113696] cpu_dev_init done
> > [ 155.963662] memory_dev_init done
>
> Hmmm... It looks like my attempts to make RCU recover from misnesting are
> not completely foolproof. I will be especially happy to look into this
> if you could look for the source of the irq_enter()/irq_exit() misnesting.
>
> (And yes, it still might be a bug in my code -- I will be looking at that
> yet again as well.)
And the way you can prove that it is my code rather than the arch
code is to show that the warning happens on your system when the
irq_enter()/irq_exit() calls are perfectly nested.
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-14 18:34 ` Paul E. McKenney
2011-05-15 3:59 ` Yinghai Lu
2011-05-15 4:14 ` Yinghai Lu
@ 2011-05-15 22:01 ` Frederic Weisbecker
2011-05-16 5:56 ` Paul E. McKenney
2 siblings, 1 reply; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-15 22:01 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 11:34:53AM -0700, Paul E. McKenney wrote:
> rcutree.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++-----------
> 1 file changed, 75 insertions(+), 16 deletions(-)
>
> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index a4a2ef0..0b541bd 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -308,23 +308,25 @@ static int rcu_implicit_offline_qs(struct rcu_data *rdp)
>
> #ifdef CONFIG_NO_HZ
>
> -/**
> - * rcu_enter_nohz - inform RCU that current CPU is entering nohz
> +/*
> + * __rcu_enter_nohz - inform RCU that current CPU is entering nohz
> *
> * Enter nohz mode, in other words, -leave- the mode in which RCU
> * read-side critical sections can occur. (Though RCU read-side
> * critical sections can occur in irq handlers in nohz mode, a possibility
> * handled by rcu_irq_enter() and rcu_irq_exit()).
> + *
> + * This variant should only be called from rcu_irq_exit() because it
> + * assumes perfect nesting. The caller must have irqs disabled.
> */
> -void rcu_enter_nohz(void)
> +void __rcu_enter_nohz(void)
> {
> - unsigned long flags;
> struct rcu_dynticks *rdtp;
>
> - local_irq_save(flags);
> rdtp = &__get_cpu_var(rcu_dynticks);
> + WARN_ON_ONCE(rdtp->dynticks_nesting <= 0);
But this warning will always trigger if it's called from rcu_irq_enter()
after rcu_enter_nohz(). Right?
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 22:01 ` Frederic Weisbecker
@ 2011-05-16 5:56 ` Paul E. McKenney
2011-05-16 22:40 ` Frederic Weisbecker
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-16 5:56 UTC (permalink / raw)
To: Frederic Weisbecker; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel
On Mon, May 16, 2011 at 12:01:17AM +0200, Frederic Weisbecker wrote:
> On Sat, May 14, 2011 at 11:34:53AM -0700, Paul E. McKenney wrote:
> > rcutree.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++-----------
> > 1 file changed, 75 insertions(+), 16 deletions(-)
> >
> > diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> > index a4a2ef0..0b541bd 100644
> > --- a/kernel/rcutree.c
> > +++ b/kernel/rcutree.c
> > @@ -308,23 +308,25 @@ static int rcu_implicit_offline_qs(struct rcu_data *rdp)
> >
> > #ifdef CONFIG_NO_HZ
> >
> > -/**
> > - * rcu_enter_nohz - inform RCU that current CPU is entering nohz
> > +/*
> > + * __rcu_enter_nohz - inform RCU that current CPU is entering nohz
> > *
> > * Enter nohz mode, in other words, -leave- the mode in which RCU
> > * read-side critical sections can occur. (Though RCU read-side
> > * critical sections can occur in irq handlers in nohz mode, a possibility
> > * handled by rcu_irq_enter() and rcu_irq_exit()).
> > + *
> > + * This variant should only be called from rcu_irq_exit() because it
> > + * assumes perfect nesting. The caller must have irqs disabled.
> > */
> > -void rcu_enter_nohz(void)
> > +void __rcu_enter_nohz(void)
> > {
> > - unsigned long flags;
> > struct rcu_dynticks *rdtp;
> >
> > - local_irq_save(flags);
> > rdtp = &__get_cpu_var(rcu_dynticks);
> > + WARN_ON_ONCE(rdtp->dynticks_nesting <= 0);
>
> But this warning will always trigger if it's called from rcu_irq_enter()
> after rcu_enter_nohz(). Right?
Hello, Frederic,
Thank you for looking this over!
But you lost me on this one. Isn't __rcu_enter_nohz() called from
rcu_irq_exit() rather than from rcu_irq_enter()?
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 16:26 ` Paul E. McKenney
@ 2011-05-16 7:08 ` Ingo Molnar
2011-05-16 7:48 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-16 7:08 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> > Would it have been possible to split it in two, one for the movement of the
> > notifiers, the other for the barrier changes?
> >
> > That way the bisection would have fingered the movement commit. Or so.
>
> In hindsight, that certainly would have been better.
This is the Linux kernel and we *can* turn back the clock!
> I was afraid of that...
>
> On the off-chance that moving the memory barriers was at fault, the following
> patch restores all of them that don't have in situ replacements. Grasping at
> straws, admittedly.
Well, the nice thing is that we really do not have to grasp at straws, and even
while we have no good ideas we can debug this *much* better.
Could you please do a simple test-tree that does has 3 commits:
first one reverts the offending commit
second one applies the barrier part of it
this one applies the need_resched part of it
( You can do even more finegrained steps, if you find harmless-looking bits of
it that can be applied separately! )
Note, the important thing is that the tree should be a 'null pull' - i.e. the
revert plus the patches applied will not change anything in core/rcu.
Obviously it would be nice if each step built fine - no need to boot test each
step as long as you are reasonably sure it will boot fine.
Then i could take my reproducer and come up with a very precise bisection
result for you, with just a couple of minutes time spent on testing. One of the
commits after the revert will trigger the hang/slowdown.
My prediction is that we will be much wiser after that! :-)
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-15 6:59 ` Paul E. McKenney
@ 2011-05-16 7:08 ` Paul E. McKenney
2011-05-16 7:39 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-16 7:08 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Sat, May 14, 2011 at 11:59:49PM -0700, Paul E. McKenney wrote:
> On Sat, May 14, 2011 at 11:04:15PM -0700, Paul E. McKenney wrote:
> > On Sat, May 14, 2011 at 10:49:31PM -0700, Yinghai Lu wrote:
> > > On 05/14/2011 10:41 PM, Yinghai Lu wrote:
> > > > On 05/14/2011 09:14 PM, Yinghai Lu wrote:
> > > >> On 05/14/2011 11:34 AM, Paul E. McKenney wrote:
> > > >>>> and do the inspection afterwards.
> > > >>>
> > > >>> And here is a lightly-tested patch, which applies on tip/core/rcu.
> > > >>>
> > > >>> This problem could account for both the long delays seen with e59fb312
> > > >>> (Decrease memory-barrier usage based on semi-formal proof) and the
> > > >>> shorter delays seen with a26ac245 (move TREE_RCU from softirq to kthread).
> > > >>
> > > >> yes. it fixes the problem.
> > > >>
> > > >> for 1024g system when hotadd mem enabled in kernel config
> > > >>
> > > >> [ 31.814803] cpu_dev_init done
> > > >> [ 35.437163] memory_dev_init done
> > > >>
> > > >> even it is with gcc from opensuse 11.3
> > > >
> > > > got:
> > > >
> > > > [ 86.931217] Switched to NOHz mode on CPU #0
> > > > [ 86.931272] Switched to NOHz mode on CPU #25
> > > > [ 86.931278] ------------[ cut here ]------------
> > > > [ 86.931290] WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x44/0x76()
> > > > [ 86.931294] Hardware name: Sun Fire X4800 M2
> > > > [ 86.931297] Modules linked in:
> > > > [ 86.931303] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-04836-g5e42dc2-dirty #3
> > > > [ 86.931307] Call Trace:
> > > > [ 86.931333] [<ffffffff81080280>] warn_slowpath_common+0x85/0x9d
> > > > [ 86.931338] Switched to NOHz mode on CPU #74
> > > > [ 86.931346] [<ffffffff810802b2>] warn_slowpath_null+0x1a/0x1c
> > > > [ 86.931356] [<ffffffff810d3615>] rcu_enter_nohz+0x44/0x76
> > > > [ 86.931370] [<ffffffff810ab3cb>] tick_nohz_stop_sched_tick+0x27d/0x366
> > > > [ 86.931381] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> > > > [ 86.931397] [<ffffffff81bd1aa3>] rest_init+0xb7/0xbe
> > > > [ 86.931408] [<ffffffff81bd19ec>] ? csum_partial_copy_generic+0x16c/0x16c
> > > > [ 86.931423] [<ffffffff82738e39>] start_kernel+0x3b2/0x3bd
> > > > [ 86.931428] Switched to NOHz mode on CPU #94
> > > > [ 86.931436] [<ffffffff827382cc>] x86_64_start_reservations+0x9c/0xa0
> > > > [ 86.931446] [<ffffffff827384a8>] x86_64_start_kernel+0x1d8/0x1e3
> > > > [ 86.931463] ---[ end trace 2cfc591bf7de931f ]---
> > > > [ 86.931598] Switched to NOHz mode on CPU #151
> > > > [ 86.931613] Switched to NOHz mode on CPU #152
> > >
> > > it seems gcc from Fedora 14 is not happy with this patch.
> > >
> > > [ 35.113696] cpu_dev_init done
> > > [ 155.963662] memory_dev_init done
> >
> > Hmmm... It looks like my attempts to make RCU recover from misnesting are
> > not completely foolproof. I will be especially happy to look into this
> > if you could look for the source of the irq_enter()/irq_exit() misnesting.
> >
> > (And yes, it still might be a bug in my code -- I will be looking at that
> > yet again as well.)
>
> And the way you can prove that it is my code rather than the arch
> code is to show that the warning happens on your system when the
> irq_enter()/irq_exit() calls are perfectly nested.
So I took another look at the RCU debugfs stats you provided earlier,
and realized that your system gets a lot more NMIs than do the ones
that I have access to. So as a diagnostic patch, I ifdefed out the
body of rcu_nmi_enter() and rcu_nmi_exit().
If everything works perfectly with this patch applied, that would point
to a race in those two functions. Please feel free to apply on top
of my earlier diagnostic patches or directly on tip/core/rcu -- either
would provide good information.
Thanx, Paul
------------------------------------------------------------------------
rcutree.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 4a9e4aa..0d4a5b5 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -430,6 +430,7 @@ void rcu_exit_nohz(void)
*/
void rcu_nmi_enter(void)
{
+#if 0
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
if (rdtp->dynticks_nmi_nesting == 0 &&
@@ -443,6 +444,7 @@ void rcu_nmi_enter(void)
/* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
smp_mb__after_atomic_inc(); /* See above. */
WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+#endif
}
/**
@@ -454,6 +456,7 @@ void rcu_nmi_enter(void)
*/
void rcu_nmi_exit(void)
{
+#if 0
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
if (rdtp->dynticks_nmi_nesting == 0 ||
@@ -466,6 +469,7 @@ void rcu_nmi_exit(void)
atomic_inc(&rdtp->dynticks);
smp_mb__after_atomic_inc(); /* Force delay to next write. */
WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
+#endif
}
/**
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-13 16:38 ` Paul E. McKenney
@ 2011-05-16 7:10 ` Ingo Molnar
0 siblings, 0 replies; 76+ messages in thread
From: Ingo Molnar @ 2011-05-16 7:10 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> Are you guys seeing NMIs at about the time the slowdown occurs?
None that produces a kernel message.
Also, the kernel config has:
# CONFIG_LOCKUP_DETECTOR is not set
# CONFIG_HARDLOCKUP_DETECTOR is not set
# CONFIG_DETECT_HUNG_TASK is not set
So the NMI watchdog ought to be off. This system isnt producing spurious NMIs.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 7:08 ` Paul E. McKenney
@ 2011-05-16 7:39 ` Ingo Molnar
0 siblings, 0 replies; 76+ messages in thread
From: Ingo Molnar @ 2011-05-16 7:39 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> > And the way you can prove that it is my code rather than the arch
> > code is to show that the warning happens on your system when the
> > irq_enter()/irq_exit() calls are perfectly nested.
>
> So I took another look at the RCU debugfs stats you provided earlier,
> and realized that your system gets a lot more NMIs than do the ones
> that I have access to. So as a diagnostic patch, I ifdefed out the
> body of rcu_nmi_enter() and rcu_nmi_exit().
Well, but the delays are occuring all the time (and it's bisectable) and NMIs
are generally not deterministic.
I'd really suggest the creation of a revert + finegrained series on top of
core/rcu which would IMHO help us narrow this down a lot more directly than
jumping between the 'need_resched bug', 'nmi bug' and 'barrier bug' hypoteses.
( Btw., the bug still has the feeling of a need_resched/scheduling/timing
artifact to me, not barriers or NMI. )
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 7:08 ` Ingo Molnar
@ 2011-05-16 7:48 ` Paul E. McKenney
2011-05-16 11:51 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-16 7:48 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Mon, May 16, 2011 at 09:08:08AM +0200, Ingo Molnar wrote:
>
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
>
> > > Would it have been possible to split it in two, one for the movement of the
> > > notifiers, the other for the barrier changes?
> > >
> > > That way the bisection would have fingered the movement commit. Or so.
> >
> > In hindsight, that certainly would have been better.
>
> This is the Linux kernel and we *can* turn back the clock!
Yay for source-code control systems in general and git in particular! ;-)
> > I was afraid of that...
> >
> > On the off-chance that moving the memory barriers was at fault, the following
> > patch restores all of them that don't have in situ replacements. Grasping at
> > straws, admittedly.
>
> Well, the nice thing is that we really do not have to grasp at straws, and even
> while we have no good ideas we can debug this *much* better.
>
> Could you please do a simple test-tree that does has 3 commits:
>
> first one reverts the offending commit
> second one applies the barrier part of it
> this one applies the need_resched part of it
>
> ( You can do even more finegrained steps, if you find harmless-looking bits of
> it that can be applied separately! )
>
> Note, the important thing is that the tree should be a 'null pull' - i.e. the
> revert plus the patches applied will not change anything in core/rcu.
>
> Obviously it would be nice if each step built fine - no need to boot test each
> step as long as you are reasonably sure it will boot fine.
>
> Then i could take my reproducer and come up with a very precise bisection
> result for you, with just a couple of minutes time spent on testing. One of the
> commits after the revert will trigger the hang/slowdown.
>
> My prediction is that we will be much wiser after that! :-)
I will put this together!
In the meantime, would you be willing to try out the patch at
https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai
in several configurations.
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 7:48 ` Paul E. McKenney
@ 2011-05-16 11:51 ` Ingo Molnar
2011-05-16 12:23 ` Ingo Molnar
0 siblings, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-16 11:51 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> On Mon, May 16, 2011 at 09:08:08AM +0200, Ingo Molnar wrote:
> >
> > * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> >
> > > > Would it have been possible to split it in two, one for the movement of the
> > > > notifiers, the other for the barrier changes?
> > > >
> > > > That way the bisection would have fingered the movement commit. Or so.
> > >
> > > In hindsight, that certainly would have been better.
> >
> > This is the Linux kernel and we *can* turn back the clock!
>
> Yay for source-code control systems in general and git in particular! ;-)
>
> > > I was afraid of that...
> > >
> > > On the off-chance that moving the memory barriers was at fault, the following
> > > patch restores all of them that don't have in situ replacements. Grasping at
> > > straws, admittedly.
> >
> > Well, the nice thing is that we really do not have to grasp at straws, and even
> > while we have no good ideas we can debug this *much* better.
> >
> > Could you please do a simple test-tree that does has 3 commits:
> >
> > first one reverts the offending commit
> > second one applies the barrier part of it
> > this one applies the need_resched part of it
> >
> > ( You can do even more finegrained steps, if you find harmless-looking bits of
> > it that can be applied separately! )
> >
> > Note, the important thing is that the tree should be a 'null pull' - i.e. the
> > revert plus the patches applied will not change anything in core/rcu.
> >
> > Obviously it would be nice if each step built fine - no need to boot test each
> > step as long as you are reasonably sure it will boot fine.
> >
> > Then i could take my reproducer and come up with a very precise bisection
> > result for you, with just a couple of minutes time spent on testing. One of the
> > commits after the revert will trigger the hang/slowdown.
> >
> > My prediction is that we will be much wiser after that! :-)
>
> I will put this together!
>
> In the meantime, would you be willing to try out the patch at
> https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai
> in several configurations.
Wasn't this the one i tested - or is it a new iteration?
I'll try it in any case.
If the bug is fixed for good then we can learn no more from it and then i'd
suggest for you to not waste much time with a more finegrained queue :-)
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 11:51 ` Ingo Molnar
@ 2011-05-16 12:23 ` Ingo Molnar
2011-05-16 14:30 ` Ingo Molnar
2011-05-16 21:24 ` Paul E. McKenney
0 siblings, 2 replies; 76+ messages in thread
From: Ingo Molnar @ 2011-05-16 12:23 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Ingo Molnar <mingo@elte.hu> wrote:
> > In the meantime, would you be willing to try out the patch at
> > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > several configurations.
>
> Wasn't this the one i tested - or is it a new iteration?
>
> I'll try it in any case.
oh, this was a new iteration, mea culpa!
And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 12:23 ` Ingo Molnar
@ 2011-05-16 14:30 ` Ingo Molnar
2011-05-16 21:33 ` Paul E. McKenney
2011-05-16 21:24 ` Paul E. McKenney
1 sibling, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-16 14:30 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, linux-kernel
* Ingo Molnar <mingo@elte.hu> wrote:
> > I'll try it in any case.
>
> oh, this was a new iteration, mea culpa!
>
> And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
FYI, i am also getting the warning below with a defconfig.
Ingo
initcall init_per_zone_wmark_min+0x0/0x5b returned 0 after 41 usecs
calling kswapd_init+0x0/0x1d @ 1
------------[ cut here ]------------
WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x4f/0x60()
Hardware name: System Product Name
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-03260-gb177656-dirty #126707
Call Trace:
[<c13974b6>] ? printk+0x18/0x1a
[<c1038f0d>] warn_slowpath_common+0x6d/0xa0
[<c1089bdf>] ? rcu_enter_nohz+0x4f/0x60
[<c1089bdf>] ? rcu_enter_nohz+0x4f/0x60
[<c1038f5d>] warn_slowpath_null+0x1d/0x20
[<c1089bdf>] rcu_enter_nohz+0x4f/0x60
[<c10639ca>] tick_nohz_stop_sched_tick+0x22a/0x470
[<c1001685>] cpu_idle+0x65/0xe0
[<c137d789>] rest_init+0xa1/0xa8
[<c137d6e8>] ? reciprocal_value+0x48/0x48
[<c156a6ef>] start_kernel+0x303/0x30b
[<c156a1fd>] ? obsolete_checksetup+0x95/0x95
[<c156a067>] i386_start_kernel+0x67/0x6d
---[ end trace fe4ebffb2b8ff187 ]---
initcall kswapd_init+0x0/0x1d returned 0 after 70614 usecs
calling extfrag_debug_init+0x0/0x78 @ 1
initcall extfrag_debug_init+0x0/0x78 returned 0 after 79 usecs
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 12:23 ` Ingo Molnar
2011-05-16 14:30 ` Ingo Molnar
@ 2011-05-16 21:24 ` Paul E. McKenney
2011-05-16 23:52 ` Frederic Weisbecker
2011-05-17 2:40 ` Frederic Weisbecker
1 sibling, 2 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-16 21:24 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
>
> * Ingo Molnar <mingo@elte.hu> wrote:
>
> > > In the meantime, would you be willing to try out the patch at
> > > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > > several configurations.
> >
> > Wasn't this the one i tested - or is it a new iteration?
> >
> > I'll try it in any case.
>
> oh, this was a new iteration, mea culpa!
>
> And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
;-)
Unfortunately, the only reason I can see that it works is (1) there
is some obscure bug in my code or (2) someone somewhere is failing to
call irq_exit() on some interrupt-exit path. Much as I might be tempted
to paper this one over, I believe that we do need to find whatever the
underlying bug is.
Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
But I couldn't see any evidence of this in my admittedly cursory scan
of the x86 interrupt-handling code.
So until I learn differently, I am assuming that each and every
irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
is called after the final irq_exit() of a given burst of interrupts.
If my assumptions are mistaken, please do let me know!
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 14:30 ` Ingo Molnar
@ 2011-05-16 21:33 ` Paul E. McKenney
2011-05-16 22:07 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-16 21:33 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Mon, May 16, 2011 at 04:30:05PM +0200, Ingo Molnar wrote:
>
> * Ingo Molnar <mingo@elte.hu> wrote:
>
> > > I'll try it in any case.
> >
> > oh, this was a new iteration, mea culpa!
> >
> > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
>
> FYI, i am also getting the warning below with a defconfig.
Yep, this is me whining that rcu_enter_nohz() was called before the
irq nesting count (->dynticks_nesting) had dropped to zero. In other
words, either I am mis-counting interrupts or we entered some irq
handler and then somehow never left it, but still managed to get
to process-level execution. Most likely that I am miscounting
somehow.
I suppose that I could remove the WARN_ON_ONCE() and pretend that all
was well, but... ;-)
Thanx, Paul
> Ingo
>
> initcall init_per_zone_wmark_min+0x0/0x5b returned 0 after 41 usecs
> calling kswapd_init+0x0/0x1d @ 1
> ------------[ cut here ]------------
> WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x4f/0x60()
> Hardware name: System Product Name
> Modules linked in:
> Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-03260-gb177656-dirty #126707
> Call Trace:
> [<c13974b6>] ? printk+0x18/0x1a
> [<c1038f0d>] warn_slowpath_common+0x6d/0xa0
> [<c1089bdf>] ? rcu_enter_nohz+0x4f/0x60
> [<c1089bdf>] ? rcu_enter_nohz+0x4f/0x60
> [<c1038f5d>] warn_slowpath_null+0x1d/0x20
> [<c1089bdf>] rcu_enter_nohz+0x4f/0x60
> [<c10639ca>] tick_nohz_stop_sched_tick+0x22a/0x470
> [<c1001685>] cpu_idle+0x65/0xe0
> [<c137d789>] rest_init+0xa1/0xa8
> [<c137d6e8>] ? reciprocal_value+0x48/0x48
> [<c156a6ef>] start_kernel+0x303/0x30b
> [<c156a1fd>] ? obsolete_checksetup+0x95/0x95
> [<c156a067>] i386_start_kernel+0x67/0x6d
> ---[ end trace fe4ebffb2b8ff187 ]---
> initcall kswapd_init+0x0/0x1d returned 0 after 70614 usecs
> calling extfrag_debug_init+0x0/0x78 @ 1
> initcall extfrag_debug_init+0x0/0x78 returned 0 after 79 usecs
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 21:33 ` Paul E. McKenney
@ 2011-05-16 22:07 ` Paul E. McKenney
0 siblings, 0 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-16 22:07 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Yinghai Lu, linux-kernel
On Mon, May 16, 2011 at 02:33:31PM -0700, Paul E. McKenney wrote:
> On Mon, May 16, 2011 at 04:30:05PM +0200, Ingo Molnar wrote:
> >
> > * Ingo Molnar <mingo@elte.hu> wrote:
> >
> > > > I'll try it in any case.
> > >
> > > oh, this was a new iteration, mea culpa!
> > >
> > > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
> >
> > FYI, i am also getting the warning below with a defconfig.
>
> Yep, this is me whining that rcu_enter_nohz() was called before the
> irq nesting count (->dynticks_nesting) had dropped to zero. In other
> words, either I am mis-counting interrupts or we entered some irq
> handler and then somehow never left it, but still managed to get
> to process-level execution. Most likely that I am miscounting
> somehow.
>
> I suppose that I could remove the WARN_ON_ONCE() and pretend that all
> was well, but... ;-)
Actually, your test did prove something very interesting. I have
been suspecting NMIs, but since the patch you just tested commented
out rcu_nmi_enter() and rcu_nmi_exit() completely, we know that there
is some bug outside of these two functions. So if I am miscounting
nesting levels, I must be doing soe in rcu_enter_nohz(), rcu_exit_nohz(),
rcu_irq_enter(), or rcu_irq_exit(). Or some combination of those four.
Thanx, Paul
> > Ingo
> >
> > initcall init_per_zone_wmark_min+0x0/0x5b returned 0 after 41 usecs
> > calling kswapd_init+0x0/0x1d @ 1
> > ------------[ cut here ]------------
> > WARNING: at kernel/rcutree.c:364 rcu_enter_nohz+0x4f/0x60()
> > Hardware name: System Product Name
> > Modules linked in:
> > Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-03260-gb177656-dirty #126707
> > Call Trace:
> > [<c13974b6>] ? printk+0x18/0x1a
> > [<c1038f0d>] warn_slowpath_common+0x6d/0xa0
> > [<c1089bdf>] ? rcu_enter_nohz+0x4f/0x60
> > [<c1089bdf>] ? rcu_enter_nohz+0x4f/0x60
> > [<c1038f5d>] warn_slowpath_null+0x1d/0x20
> > [<c1089bdf>] rcu_enter_nohz+0x4f/0x60
> > [<c10639ca>] tick_nohz_stop_sched_tick+0x22a/0x470
> > [<c1001685>] cpu_idle+0x65/0xe0
> > [<c137d789>] rest_init+0xa1/0xa8
> > [<c137d6e8>] ? reciprocal_value+0x48/0x48
> > [<c156a6ef>] start_kernel+0x303/0x30b
> > [<c156a1fd>] ? obsolete_checksetup+0x95/0x95
> > [<c156a067>] i386_start_kernel+0x67/0x6d
> > ---[ end trace fe4ebffb2b8ff187 ]---
> > initcall kswapd_init+0x0/0x1d returned 0 after 70614 usecs
> > calling extfrag_debug_init+0x0/0x78 @ 1
> > initcall extfrag_debug_init+0x0/0x78 returned 0 after 79 usecs
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 5:56 ` Paul E. McKenney
@ 2011-05-16 22:40 ` Frederic Weisbecker
0 siblings, 0 replies; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-16 22:40 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel
On Sun, May 15, 2011 at 10:56:15PM -0700, Paul E. McKenney wrote:
> On Mon, May 16, 2011 at 12:01:17AM +0200, Frederic Weisbecker wrote:
> > On Sat, May 14, 2011 at 11:34:53AM -0700, Paul E. McKenney wrote:
> > > rcutree.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++-----------
> > > 1 file changed, 75 insertions(+), 16 deletions(-)
> > >
> > > diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> > > index a4a2ef0..0b541bd 100644
> > > --- a/kernel/rcutree.c
> > > +++ b/kernel/rcutree.c
> > > @@ -308,23 +308,25 @@ static int rcu_implicit_offline_qs(struct rcu_data *rdp)
> > >
> > > #ifdef CONFIG_NO_HZ
> > >
> > > -/**
> > > - * rcu_enter_nohz - inform RCU that current CPU is entering nohz
> > > +/*
> > > + * __rcu_enter_nohz - inform RCU that current CPU is entering nohz
> > > *
> > > * Enter nohz mode, in other words, -leave- the mode in which RCU
> > > * read-side critical sections can occur. (Though RCU read-side
> > > * critical sections can occur in irq handlers in nohz mode, a possibility
> > > * handled by rcu_irq_enter() and rcu_irq_exit()).
> > > + *
> > > + * This variant should only be called from rcu_irq_exit() because it
> > > + * assumes perfect nesting. The caller must have irqs disabled.
> > > */
> > > -void rcu_enter_nohz(void)
> > > +void __rcu_enter_nohz(void)
> > > {
> > > - unsigned long flags;
> > > struct rcu_dynticks *rdtp;
> > >
> > > - local_irq_save(flags);
> > > rdtp = &__get_cpu_var(rcu_dynticks);
> > > + WARN_ON_ONCE(rdtp->dynticks_nesting <= 0);
> >
> > But this warning will always trigger if it's called from rcu_irq_enter()
> > after rcu_enter_nohz(). Right?
>
> Hello, Frederic,
>
> Thank you for looking this over!
>
> But you lost me on this one. Isn't __rcu_enter_nohz() called from
> rcu_irq_exit() rather than from rcu_irq_enter()?
Oops, right I read it too quickly.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 21:24 ` Paul E. McKenney
@ 2011-05-16 23:52 ` Frederic Weisbecker
2011-05-17 2:40 ` Frederic Weisbecker
1 sibling, 0 replies; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-16 23:52 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Ingo Molnar, Yinghai Lu, linux-kernel
On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
> On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
> >
> > * Ingo Molnar <mingo@elte.hu> wrote:
> >
> > > > In the meantime, would you be willing to try out the patch at
> > > > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > > > several configurations.
> > >
> > > Wasn't this the one i tested - or is it a new iteration?
> > >
> > > I'll try it in any case.
> >
> > oh, this was a new iteration, mea culpa!
> >
> > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
>
> ;-)
>
> Unfortunately, the only reason I can see that it works is (1) there
> is some obscure bug in my code or (2) someone somewhere is failing to
> call irq_exit() on some interrupt-exit path. Much as I might be tempted
> to paper this one over, I believe that we do need to find whatever the
> underlying bug is.
>
> Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
> a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
> But I couldn't see any evidence of this in my admittedly cursory scan
> of the x86 interrupt-handling code.
>
> So until I learn differently, I am assuming that each and every
> irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
> is called after the final irq_exit() of a given burst of interrupts.
>
> If my assumptions are mistaken, please do let me know!
About 2), I believe that such an unpairing would have been detected before
your whole patchset was merged.
For example if an interrupt failed to call rcu_irq_exit(), we would have
found cases where we have:
rcu_enter_nohz()
<irq>
rcu_irq_enter()
</irq>
rcu_exit_nohz()
And then that last call would trigger "WARN_ON_ONCE(!(rdtp->dynticks & 0x1))".
But may be there was a patch in your set that touched one of these rcu_irq_...
callsites.
About 3), it shouldn't happen because preempt_schedule_irq() is called in the
exit path of the low level interrupt handler. rcu_exit_irq() is called from
the higher level, before resuming to the low level.
That said there might be something nasty that the old checks in the QS APIs
were missing.
I think it would be nice to add some checks in rcu-lockdep inside
rcu_read_lock()/rcu_dereference() to ensure rdp->dynticks is not even, ie
that we are not in an extended qs. That's something I planned to add for
my next nohz tasks patchset version, because I bring more dance with the
extended quiescent state, but given the problems we are facing today, it
may be better sooner.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-16 21:24 ` Paul E. McKenney
2011-05-16 23:52 ` Frederic Weisbecker
@ 2011-05-17 2:40 ` Frederic Weisbecker
2011-05-17 7:53 ` Paul E. McKenney
2011-05-18 21:10 ` Yinghai Lu
1 sibling, 2 replies; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-17 2:40 UTC (permalink / raw)
To: Paul E. McKenney, Yinghai Lu; +Cc: Ingo Molnar, linux-kernel
On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
> On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
> >
> > * Ingo Molnar <mingo@elte.hu> wrote:
> >
> > > > In the meantime, would you be willing to try out the patch at
> > > > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > > > several configurations.
> > >
> > > Wasn't this the one i tested - or is it a new iteration?
> > >
> > > I'll try it in any case.
> >
> > oh, this was a new iteration, mea culpa!
> >
> > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
>
> ;-)
>
> Unfortunately, the only reason I can see that it works is (1) there
> is some obscure bug in my code or (2) someone somewhere is failing to
> call irq_exit() on some interrupt-exit path. Much as I might be tempted
> to paper this one over, I believe that we do need to find whatever the
> underlying bug is.
>
> Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
> a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
> But I couldn't see any evidence of this in my admittedly cursory scan
> of the x86 interrupt-handling code.
>
> So until I learn differently, I am assuming that each and every
> irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
> is called after the final irq_exit() of a given burst of interrupts.
>
> If my assumptions are mistaken, please do let me know!
So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
before the unpairing happened.
I have tried to reproduce it but couldn't trigger anything.
So it would be nice if Yinghai can test the patch below, since he was able
to trigger the warning.
This is essentially Paul's patch but with stacktrace of the calls recorded.
Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
sanity check is positive. Beware as the trace will be dumped everytime
WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
rest.
This requires CONFIG_TRACING. May be a good thing to boot with
"ftrace=nop" parameter, so that ftrace will set up a long enough buffer
to have an interesting trace.
PS: the first check in rcu_nmi_enter() doesn't seem to make sense.
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 5616b17..a4e6d1b 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -316,24 +316,23 @@ static int rcu_implicit_offline_qs(struct rcu_data *rdp)
* critical sections can occur in irq handlers in nohz mode, a possibility
* handled by rcu_irq_enter() and rcu_irq_exit()).
*/
-void rcu_enter_nohz(void)
+void __rcu_enter_nohz(void)
{
- unsigned long flags;
struct rcu_dynticks *rdtp;
- local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
+ WARN_ON_ONCE(rdtp->dynticks_nesting <= 0);
if (--rdtp->dynticks_nesting) {
- local_irq_restore(flags);
+ if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
+ ftrace_dump(DUMP_ORIG);
return;
}
/* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
smp_mb__before_atomic_inc(); /* See above. */
atomic_inc(&rdtp->dynticks);
smp_mb__after_atomic_inc(); /* Force ordering with next sojourn. */
- WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
- local_irq_restore(flags);
-
+ if (WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1))
+ ftrace_dump(DUMP_ORIG);
/* If the interrupt queued a callback, get out of dyntick mode. */
if (in_irq() &&
(__get_cpu_var(rcu_sched_data).nxtlist ||
@@ -342,28 +341,57 @@ void rcu_enter_nohz(void)
set_need_resched();
}
+void rcu_enter_nohz(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ local_irq_save(flags);
+ trace_dump_stack();
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ if (WARN_ON_ONCE(rdtp->dynticks_nesting != 1))
+ ftrace_dump(DUMP_ORIG);
+ rdtp->dynticks_nesting = 1;
+ __rcu_enter_nohz();
+ local_irq_restore(flags);
+}
+
/*
* rcu_exit_nohz - inform RCU that current CPU is leaving nohz
*
* Exit nohz mode, in other words, -enter- the mode in which RCU
* read-side critical sections normally occur.
*/
-void rcu_exit_nohz(void)
+void __rcu_exit_nohz(void)
{
- unsigned long flags;
struct rcu_dynticks *rdtp;
- local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
if (rdtp->dynticks_nesting++) {
- local_irq_restore(flags);
+ if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
+ ftrace_dump(DUMP_ORIG);
return;
}
smp_mb__before_atomic_inc(); /* Force ordering w/previous sojourn. */
atomic_inc(&rdtp->dynticks);
/* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
smp_mb__after_atomic_inc(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
+ ftrace_dump(DUMP_ORIG);
+}
+
+void rcu_exit_nohz(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ local_irq_save(flags);
+ trace_dump_stack();
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ if (WARN_ON_ONCE(rdtp->dynticks_nesting != 0))
+ ftrace_dump(DUMP_ORIG);
+ rdtp->dynticks_nesting = 0;
+ __rcu_exit_nohz();
local_irq_restore(flags);
}
@@ -378,15 +406,21 @@ void rcu_nmi_enter(void)
{
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
+ trace_dump_stack();
+
if (rdtp->dynticks_nmi_nesting == 0 &&
- (atomic_read(&rdtp->dynticks) & 0x1))
+ (atomic_read(&rdtp->dynticks) & 0x1)) {
+ if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
+ ftrace_dump(DUMP_ORIG);
return;
+ }
rdtp->dynticks_nmi_nesting++;
smp_mb__before_atomic_inc(); /* Force delay from prior write. */
atomic_inc(&rdtp->dynticks);
/* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
smp_mb__after_atomic_inc(); /* See above. */
- WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
+ if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
+ ftrace_dump(DUMP_ORIG);
}
/**
@@ -400,14 +434,20 @@ void rcu_nmi_exit(void)
{
struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
+ trace_dump_stack();
+
if (rdtp->dynticks_nmi_nesting == 0 ||
- --rdtp->dynticks_nmi_nesting != 0)
+ --rdtp->dynticks_nmi_nesting != 0) {
+ if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
+ ftrace_dump(DUMP_ORIG);
return;
+ }
/* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
smp_mb__before_atomic_inc(); /* See above. */
atomic_inc(&rdtp->dynticks);
smp_mb__after_atomic_inc(); /* Force delay to next write. */
- WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
+ if (WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1))
+ ftrace_dump(DUMP_ORIG);
}
/**
@@ -418,7 +458,12 @@ void rcu_nmi_exit(void)
*/
void rcu_irq_enter(void)
{
- rcu_exit_nohz();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ trace_dump_stack();
+ __rcu_exit_nohz();
+ local_irq_restore(flags);
}
/**
@@ -430,7 +475,12 @@ void rcu_irq_enter(void)
*/
void rcu_irq_exit(void)
{
- rcu_enter_nohz();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ trace_dump_stack();
+ __rcu_enter_nohz();
+ local_irq_restore(flags);
}
#ifdef CONFIG_SMP
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-17 2:40 ` Frederic Weisbecker
@ 2011-05-17 7:53 ` Paul E. McKenney
2011-05-17 12:43 ` Frederic Weisbecker
2011-05-18 21:10 ` Yinghai Lu
1 sibling, 1 reply; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-17 7:53 UTC (permalink / raw)
To: Frederic Weisbecker; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel
On Tue, May 17, 2011 at 04:40:03AM +0200, Frederic Weisbecker wrote:
> On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
> > On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
> > >
> > > * Ingo Molnar <mingo@elte.hu> wrote:
> > >
> > > > > In the meantime, would you be willing to try out the patch at
> > > > > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > > > > several configurations.
> > > >
> > > > Wasn't this the one i tested - or is it a new iteration?
> > > >
> > > > I'll try it in any case.
> > >
> > > oh, this was a new iteration, mea culpa!
> > >
> > > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
> >
> > ;-)
> >
> > Unfortunately, the only reason I can see that it works is (1) there
> > is some obscure bug in my code or (2) someone somewhere is failing to
> > call irq_exit() on some interrupt-exit path. Much as I might be tempted
> > to paper this one over, I believe that we do need to find whatever the
> > underlying bug is.
> >
> > Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
> > a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
> > But I couldn't see any evidence of this in my admittedly cursory scan
> > of the x86 interrupt-handling code.
> >
> > So until I learn differently, I am assuming that each and every
> > irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
> > is called after the final irq_exit() of a given burst of interrupts.
> >
> > If my assumptions are mistaken, please do let me know!
>
> So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
> before the unpairing happened.
>
> I have tried to reproduce it but couldn't trigger anything.
>
> So it would be nice if Yinghai can test the patch below, since he was able
> to trigger the warning.
>
> This is essentially Paul's patch but with stacktrace of the calls recorded.
> Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
> sanity check is positive. Beware as the trace will be dumped everytime
> WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
> rest.
>
> This requires CONFIG_TRACING. May be a good thing to boot with
> "ftrace=nop" parameter, so that ftrace will set up a long enough buffer
> to have an interesting trace.
Very cool, thank you!!! I was going to do something like this next,
but given my lack of familiarity with tracing, your patch looks much
nicer than mine would have been.
It applies fine on top of tip/core/rcu and builds OK. I cannot reproduce
the problem, either, so I am hoping that either Yinghai or Ingo can
run this, and hopefully doing so will provide some enlightenment.
I have pushed this as:
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git diag.2011.05.16b
I also #ifdefed out the bodies of rcu_nmi_enter() and rcu_nmi_exit()
to match the earlier patches.
> PS: the first check in rcu_nmi_enter() doesn't seem to make sense.
Here is what it is doing:
o rdtp->dynticks_nmi_nesting == 0:
Is this is the first-level NMI? In theory this should always
be true, but I don't trust NMIs to mask each other. I have seen
many systems where NMIs could interrupt other NMIs.
The idea is that if we already recorded one level of NMI, we
had better record them all so we can figure out when we exit
the last level of NMI handler.
o atomic_read(&rdtp->dynticks) & 0x1):
Did the NMI interrupt a non-dyntick code segment? If we did,
then there is no need to tell RCU anything -- RCU is already
paying attention to this CPU anyway due to the fact that the
interrupted code segment was not in dyntick mode.
As for the check on rcu_nmi_exit():
o rdtp->dynticks_nmi_nesting == 0:
We didn't record the corresponding entry to the NMI handler,
so we had better not record the exit.
o --rdtp->dynticks_nmi_nesting != 0:
We are still nested in a NMI handler, so decrement the nesting
count, but don't mess with rdtp->dynticks.
And yes, I do need some comments. Or maybe to rewrite these in a
less-confusing form. LIke you, I decided that these were incorrect
several times over the past few days. So even if my analysis above turns
out to be correct, some change is needed, a comment at the very least.
But I also know that there is at least one problem outside of
rcu_nmi_enter() and rcu_nmi_exit() because Ingo verified that he hits
the problem even with the bodies of rcu_nmi_enter() and rcu_nmi_exit()
ifdefed out.
Again, thank you for adding the tracing!
> diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> index 5616b17..a4e6d1b 100644
> --- a/kernel/rcutree.c
> +++ b/kernel/rcutree.c
> @@ -316,24 +316,23 @@ static int rcu_implicit_offline_qs(struct rcu_data *rdp)
> * critical sections can occur in irq handlers in nohz mode, a possibility
> * handled by rcu_irq_enter() and rcu_irq_exit()).
> */
> -void rcu_enter_nohz(void)
> +void __rcu_enter_nohz(void)
> {
> - unsigned long flags;
> struct rcu_dynticks *rdtp;
>
> - local_irq_save(flags);
> rdtp = &__get_cpu_var(rcu_dynticks);
> + WARN_ON_ONCE(rdtp->dynticks_nesting <= 0);
> if (--rdtp->dynticks_nesting) {
> - local_irq_restore(flags);
> + if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
> + ftrace_dump(DUMP_ORIG);
> return;
> }
> /* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
> smp_mb__before_atomic_inc(); /* See above. */
> atomic_inc(&rdtp->dynticks);
> smp_mb__after_atomic_inc(); /* Force ordering with next sojourn. */
> - WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
> - local_irq_restore(flags);
> -
> + if (WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1))
> + ftrace_dump(DUMP_ORIG);
> /* If the interrupt queued a callback, get out of dyntick mode. */
> if (in_irq() &&
> (__get_cpu_var(rcu_sched_data).nxtlist ||
> @@ -342,28 +341,57 @@ void rcu_enter_nohz(void)
> set_need_resched();
> }
>
> +void rcu_enter_nohz(void)
> +{
> + unsigned long flags;
> + struct rcu_dynticks *rdtp;
> +
> + local_irq_save(flags);
> + trace_dump_stack();
> + rdtp = &__get_cpu_var(rcu_dynticks);
> + if (WARN_ON_ONCE(rdtp->dynticks_nesting != 1))
> + ftrace_dump(DUMP_ORIG);
> + rdtp->dynticks_nesting = 1;
> + __rcu_enter_nohz();
> + local_irq_restore(flags);
> +}
> +
> /*
> * rcu_exit_nohz - inform RCU that current CPU is leaving nohz
> *
> * Exit nohz mode, in other words, -enter- the mode in which RCU
> * read-side critical sections normally occur.
> */
> -void rcu_exit_nohz(void)
> +void __rcu_exit_nohz(void)
> {
> - unsigned long flags;
> struct rcu_dynticks *rdtp;
>
> - local_irq_save(flags);
> rdtp = &__get_cpu_var(rcu_dynticks);
> if (rdtp->dynticks_nesting++) {
> - local_irq_restore(flags);
> + if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
> + ftrace_dump(DUMP_ORIG);
> return;
> }
> smp_mb__before_atomic_inc(); /* Force ordering w/previous sojourn. */
> atomic_inc(&rdtp->dynticks);
> /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
> smp_mb__after_atomic_inc(); /* See above. */
> - WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
> + if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
> + ftrace_dump(DUMP_ORIG);
> +}
> +
> +void rcu_exit_nohz(void)
> +{
> + unsigned long flags;
> + struct rcu_dynticks *rdtp;
> +
> + local_irq_save(flags);
> + trace_dump_stack();
> + rdtp = &__get_cpu_var(rcu_dynticks);
> + if (WARN_ON_ONCE(rdtp->dynticks_nesting != 0))
> + ftrace_dump(DUMP_ORIG);
> + rdtp->dynticks_nesting = 0;
> + __rcu_exit_nohz();
> local_irq_restore(flags);
> }
>
> @@ -378,15 +406,21 @@ void rcu_nmi_enter(void)
> {
> struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
>
> + trace_dump_stack();
> +
> if (rdtp->dynticks_nmi_nesting == 0 &&
> - (atomic_read(&rdtp->dynticks) & 0x1))
> + (atomic_read(&rdtp->dynticks) & 0x1)) {
> + if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
> + ftrace_dump(DUMP_ORIG);
> return;
> + }
> rdtp->dynticks_nmi_nesting++;
> smp_mb__before_atomic_inc(); /* Force delay from prior write. */
> atomic_inc(&rdtp->dynticks);
> /* CPUs seeing atomic_inc() must see later RCU read-side crit sects */
> smp_mb__after_atomic_inc(); /* See above. */
> - WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
> + if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
> + ftrace_dump(DUMP_ORIG);
> }
>
> /**
> @@ -400,14 +434,20 @@ void rcu_nmi_exit(void)
> {
> struct rcu_dynticks *rdtp = &__get_cpu_var(rcu_dynticks);
>
> + trace_dump_stack();
> +
> if (rdtp->dynticks_nmi_nesting == 0 ||
> - --rdtp->dynticks_nmi_nesting != 0)
> + --rdtp->dynticks_nmi_nesting != 0) {
> + if (WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1)))
> + ftrace_dump(DUMP_ORIG);
> return;
> + }
> /* CPUs seeing atomic_inc() must see prior RCU read-side crit sects */
> smp_mb__before_atomic_inc(); /* See above. */
> atomic_inc(&rdtp->dynticks);
> smp_mb__after_atomic_inc(); /* Force delay to next write. */
> - WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
> + if (WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1))
> + ftrace_dump(DUMP_ORIG);
> }
>
> /**
> @@ -418,7 +458,12 @@ void rcu_nmi_exit(void)
> */
> void rcu_irq_enter(void)
> {
> - rcu_exit_nohz();
> + unsigned long flags;
> +
> + local_irq_save(flags);
> + trace_dump_stack();
> + __rcu_exit_nohz();
> + local_irq_restore(flags);
> }
>
> /**
> @@ -430,7 +475,12 @@ void rcu_irq_enter(void)
> */
> void rcu_irq_exit(void)
> {
> - rcu_enter_nohz();
> + unsigned long flags;
> +
> + local_irq_save(flags);
> + trace_dump_stack();
> + __rcu_enter_nohz();
> + local_irq_restore(flags);
> }
>
> #ifdef CONFIG_SMP
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-17 7:53 ` Paul E. McKenney
@ 2011-05-17 12:43 ` Frederic Weisbecker
2011-05-17 22:21 ` Paul E. McKenney
0 siblings, 1 reply; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-17 12:43 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel
On Tue, May 17, 2011 at 12:53:49AM -0700, Paul E. McKenney wrote:
> On Tue, May 17, 2011 at 04:40:03AM +0200, Frederic Weisbecker wrote:
> > On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
> > > On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
> > > >
> > > > * Ingo Molnar <mingo@elte.hu> wrote:
> > > >
> > > > > > In the meantime, would you be willing to try out the patch at
> > > > > > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > > > > > several configurations.
> > > > >
> > > > > Wasn't this the one i tested - or is it a new iteration?
> > > > >
> > > > > I'll try it in any case.
> > > >
> > > > oh, this was a new iteration, mea culpa!
> > > >
> > > > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
> > >
> > > ;-)
> > >
> > > Unfortunately, the only reason I can see that it works is (1) there
> > > is some obscure bug in my code or (2) someone somewhere is failing to
> > > call irq_exit() on some interrupt-exit path. Much as I might be tempted
> > > to paper this one over, I believe that we do need to find whatever the
> > > underlying bug is.
> > >
> > > Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
> > > a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
> > > But I couldn't see any evidence of this in my admittedly cursory scan
> > > of the x86 interrupt-handling code.
> > >
> > > So until I learn differently, I am assuming that each and every
> > > irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
> > > is called after the final irq_exit() of a given burst of interrupts.
> > >
> > > If my assumptions are mistaken, please do let me know!
> >
> > So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
> > before the unpairing happened.
> >
> > I have tried to reproduce it but couldn't trigger anything.
> >
> > So it would be nice if Yinghai can test the patch below, since he was able
> > to trigger the warning.
> >
> > This is essentially Paul's patch but with stacktrace of the calls recorded.
> > Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
> > sanity check is positive. Beware as the trace will be dumped everytime
> > WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
> > rest.
> >
> > This requires CONFIG_TRACING. May be a good thing to boot with
> > "ftrace=nop" parameter, so that ftrace will set up a long enough buffer
> > to have an interesting trace.
>
> Very cool, thank you!!! I was going to do something like this next,
> but given my lack of familiarity with tracing, your patch looks much
> nicer than mine would have been.
>
> It applies fine on top of tip/core/rcu and builds OK. I cannot reproduce
> the problem, either, so I am hoping that either Yinghai or Ingo can
> run this, and hopefully doing so will provide some enlightenment.
>
> I have pushed this as:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git diag.2011.05.16b
>
> I also #ifdefed out the bodies of rcu_nmi_enter() and rcu_nmi_exit()
> to match the earlier patches.
>
> > PS: the first check in rcu_nmi_enter() doesn't seem to make sense.
>
> Here is what it is doing:
>
> o rdtp->dynticks_nmi_nesting == 0:
>
> Is this is the first-level NMI? In theory this should always
> be true, but I don't trust NMIs to mask each other. I have seen
> many systems where NMIs could interrupt other NMIs.
>
> The idea is that if we already recorded one level of NMI, we
> had better record them all so we can figure out when we exit
> the last level of NMI handler.
>
> o atomic_read(&rdtp->dynticks) & 0x1):
>
> Did the NMI interrupt a non-dyntick code segment? If we did,
> then there is no need to tell RCU anything -- RCU is already
> paying attention to this CPU anyway due to the fact that the
> interrupted code segment was not in dyntick mode.
In fact I was rather referring to your last added check:
if (rdtp->dynticks_nmi_nesting == 0 &&
- (atomic_read(&rdtp->dynticks) & 0x1))
+ (atomic_read(&rdtp->dynticks) & 0x1)) {
+ WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
return;
+ }
> Again, thank you for adding the tracing!
No problem, I hope it will work as I couldn't test it myself.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-17 12:43 ` Frederic Weisbecker
@ 2011-05-17 22:21 ` Paul E. McKenney
0 siblings, 0 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-17 22:21 UTC (permalink / raw)
To: Frederic Weisbecker; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel
On Tue, May 17, 2011 at 02:43:48PM +0200, Frederic Weisbecker wrote:
> On Tue, May 17, 2011 at 12:53:49AM -0700, Paul E. McKenney wrote:
> > On Tue, May 17, 2011 at 04:40:03AM +0200, Frederic Weisbecker wrote:
> > > On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
> > > > On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
> > > > >
> > > > > * Ingo Molnar <mingo@elte.hu> wrote:
> > > > >
> > > > > > > In the meantime, would you be willing to try out the patch at
> > > > > > > https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> > > > > > > several configurations.
> > > > > >
> > > > > > Wasn't this the one i tested - or is it a new iteration?
> > > > > >
> > > > > > I'll try it in any case.
> > > > >
> > > > > oh, this was a new iteration, mea culpa!
> > > > >
> > > > > And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
> > > >
> > > > ;-)
> > > >
> > > > Unfortunately, the only reason I can see that it works is (1) there
> > > > is some obscure bug in my code or (2) someone somewhere is failing to
> > > > call irq_exit() on some interrupt-exit path. Much as I might be tempted
> > > > to paper this one over, I believe that we do need to find whatever the
> > > > underlying bug is.
> > > >
> > > > Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
> > > > a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
> > > > But I couldn't see any evidence of this in my admittedly cursory scan
> > > > of the x86 interrupt-handling code.
> > > >
> > > > So until I learn differently, I am assuming that each and every
> > > > irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
> > > > is called after the final irq_exit() of a given burst of interrupts.
> > > >
> > > > If my assumptions are mistaken, please do let me know!
> > >
> > > So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
> > > before the unpairing happened.
> > >
> > > I have tried to reproduce it but couldn't trigger anything.
> > >
> > > So it would be nice if Yinghai can test the patch below, since he was able
> > > to trigger the warning.
> > >
> > > This is essentially Paul's patch but with stacktrace of the calls recorded.
> > > Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
> > > sanity check is positive. Beware as the trace will be dumped everytime
> > > WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
> > > rest.
> > >
> > > This requires CONFIG_TRACING. May be a good thing to boot with
> > > "ftrace=nop" parameter, so that ftrace will set up a long enough buffer
> > > to have an interesting trace.
> >
> > Very cool, thank you!!! I was going to do something like this next,
> > but given my lack of familiarity with tracing, your patch looks much
> > nicer than mine would have been.
> >
> > It applies fine on top of tip/core/rcu and builds OK. I cannot reproduce
> > the problem, either, so I am hoping that either Yinghai or Ingo can
> > run this, and hopefully doing so will provide some enlightenment.
> >
> > I have pushed this as:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git diag.2011.05.16b
> >
> > I also #ifdefed out the bodies of rcu_nmi_enter() and rcu_nmi_exit()
> > to match the earlier patches.
> >
> > > PS: the first check in rcu_nmi_enter() doesn't seem to make sense.
> >
> > Here is what it is doing:
> >
> > o rdtp->dynticks_nmi_nesting == 0:
> >
> > Is this is the first-level NMI? In theory this should always
> > be true, but I don't trust NMIs to mask each other. I have seen
> > many systems where NMIs could interrupt other NMIs.
> >
> > The idea is that if we already recorded one level of NMI, we
> > had better record them all so we can figure out when we exit
> > the last level of NMI handler.
> >
> > o atomic_read(&rdtp->dynticks) & 0x1):
> >
> > Did the NMI interrupt a non-dyntick code segment? If we did,
> > then there is no need to tell RCU anything -- RCU is already
> > paying attention to this CPU anyway due to the fact that the
> > interrupted code segment was not in dyntick mode.
>
> In fact I was rather referring to your last added check:
>
> if (rdtp->dynticks_nmi_nesting == 0 &&
> - (atomic_read(&rdtp->dynticks) & 0x1))
> + (atomic_read(&rdtp->dynticks) & 0x1)) {
> + WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
> return;
> + }
Yes, a bit redundant, but if some other CPU is messing with
rdtp->dynticks, this WARN_ON_ONCE() might catch it.
> > Again, thank you for adding the tracing!
>
> No problem, I hope it will work as I couldn't test it myself.
Me too!
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-17 2:40 ` Frederic Weisbecker
2011-05-17 7:53 ` Paul E. McKenney
@ 2011-05-18 21:10 ` Yinghai Lu
2011-05-18 23:13 ` Frederic Weisbecker
1 sibling, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-18 21:10 UTC (permalink / raw)
To: Frederic Weisbecker; +Cc: Paul E. McKenney, Ingo Molnar, linux-kernel
On 05/16/2011 07:40 PM, Frederic Weisbecker wrote:
> On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
>> On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
>>>
>>> * Ingo Molnar <mingo@elte.hu> wrote:
>>>
>>>>> In the meantime, would you be willing to try out the patch at
>>>>> https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
>>>>> several configurations.
>>>>
>>>> Wasn't this the one i tested - or is it a new iteration?
>>>>
>>>> I'll try it in any case.
>>>
>>> oh, this was a new iteration, mea culpa!
>>>
>>> And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
>>
>> ;-)
>>
>> Unfortunately, the only reason I can see that it works is (1) there
>> is some obscure bug in my code or (2) someone somewhere is failing to
>> call irq_exit() on some interrupt-exit path. Much as I might be tempted
>> to paper this one over, I believe that we do need to find whatever the
>> underlying bug is.
>>
>> Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
>> a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
>> But I couldn't see any evidence of this in my admittedly cursory scan
>> of the x86 interrupt-handling code.
>>
>> So until I learn differently, I am assuming that each and every
>> irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
>> is called after the final irq_exit() of a given burst of interrupts.
>>
>> If my assumptions are mistaken, please do let me know!
>
> So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
> before the unpairing happened.
>
> I have tried to reproduce it but couldn't trigger anything.
>
> So it would be nice if Yinghai can test the patch below, since he was able
> to trigger the warning.
>
> This is essentially Paul's patch but with stacktrace of the calls recorded.
> Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
> sanity check is positive. Beware as the trace will be dumped everytime
> WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
> rest.
>
> This requires CONFIG_TRACING. May be a good thing to boot with
> "ftrace=nop" parameter, so that ftrace will set up a long enough buffer
> to have an interesting trace.
with this patches if the kernel is compiled from opensuse 11.3 no delay anymore, but have one warning:
[ 82.895182] ------------[ cut here ]------------
[ 82.895189] WARNING: at kernel/rcutree.c:352 rcu_enter_nohz+0x49/0x8b()
[ 82.895193] Switched to NOHz mode on CPU #90
[ 82.895199] Switched to NOHz mode on CPU #8
[ 82.895202] Modules linked in:
[ 82.895206] Switched to NOHz mode on CPU #28
[ 82.895211] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-05234-g3a108a0-dirty #1016
[ 82.895213] Call Trace:
[ 82.895233] [<ffffffff81080144>] warn_slowpath_common+0x85/0x9d
[ 82.895238] [<ffffffff81080176>] warn_slowpath_null+0x1a/0x1c
[ 82.895242] [<ffffffff810d32cc>] rcu_enter_nohz+0x49/0x8b
[ 82.895250] [<ffffffff810ab121>] tick_nohz_stop_sched_tick+0x27d/0x366
[ 82.895255] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
[ 82.895261] [<ffffffff81bda6e3>] rest_init+0xb7/0xbe
[ 82.895266] [<ffffffff81bda62c>] ? csum_partial_copy_generic+0x16c/0x16c
[ 82.895272] [<ffffffff82742e39>] start_kernel+0x3b2/0x3bd
[ 82.895276] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
[ 82.895281] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
[ 82.895290] ---[ end trace 2cfc591bf7de931f ]---
[ 82.895310] Switched to NOHz mode on CPU #72
[ 82.895315] Dumping ftrace buffer:
[ 82.895328] ---------------------------------
[ 82.895340] CPU:0 [LOST 35328 EVENTS]
[ 82.895341] <idle>-0 0d... 82735399us : Unknown type 4
[ 82.895347] <idle>-0 0dN.. 82735431us : Unknown type 4
[ 82.895353] <idle>-0 0d... 82739390us : Unknown type 4
[ 82.895358] <idle>-0 0dN.. 82739415us : Unknown type 4
[ 82.895364] <idle>-0 0d... 82743384us : Unknown type 4
[ 82.895369] <idle>-0 0dN.. 82743408us : Unknown type 4
[ 82.895375] <idle>-0 0d... 82747376us : Unknown type 4
[ 82.895379] Switched to NOHz mode on CPU #53
[ 82.895385] <idle>-0 0dN.. 82747403us : Unknown type 4
[ 82.895390] <idle>-0 0d... 82751370us : Unknown type 4
[ 82.895395] <idle>-0 0dN.. 82751391us : Unknown type 4
[ 82.895400] Switched to NOHz mode on CPU #60
[ 82.895405] <idle>-0 0d... 82755364us : Unknown type 4
[ 82.895411] <idle>-0 0dN.. 82755386us : Unknown type 4
[ 82.895416] <idle>-0 0d... 82759355us : Unknown type 4
[ 82.895421] <idle>-0 0dN.. 82759378us : Unknown type 4
[ 82.895428] Switched to NOHz mode on CPU #102
[ 82.895431] <idle>-0 0d... 82763350us : Unknown type 4
[ 82.895436] <idle>-0 0dN.. 82763372us : Unknown type 4
[ 82.895441] <idle>-0 0d... 82767341us : Unknown type 4
[ 82.895448] Switched to NOHz mode on CPU #155
[ 82.895453] <idle>-0 0dN.. 82767364us : Unknown type 4
[ 82.895459] <idle>-0 0d... 82771334us : Unknown type 4
[ 82.895464] <idle>-0 0dN.. 82771357us : Unknown type 4
[ 82.895469] <idle>-0 0d... 82775328us : Unknown type 4
[ 82.895474] <idle>-0 0dN.. 82775355us : Unknown type 4
[ 82.895480] <idle>-0 0d... 82779321us : Unknown type 4
[ 82.895485] <idle>-0 0dN.. 82779345us : Unknown type 4
[ 82.895490] <idle>-0 0d... 82783313us : Unknown type 4
[ 82.895495] <idle>-0 0dN.. 82783340us : Unknown type 4
[ 82.895501] <idle>-0 0d... 82787308us : Unknown type 4
[ 82.895506] <idle>-0 0dN.. 82787331us : Unknown type 4
[ 82.895511] <idle>-0 0d... 82791300us : Unknown type 4
[ 82.895516] <idle>-0 0dN.. 82791322us : Unknown type 4
[ 82.895522] <idle>-0 0d... 82795293us : Unknown type 4
[ 82.895527] <idle>-0 0dN.. 82795320us : Unknown type 4
[ 82.895532] <idle>-0 0d... 82799287us : Unknown type 4
[ 82.895537] <idle>-0 0dN.. 82799310us : Unknown type 4
[ 82.895542] <idle>-0 0d... 82803279us : Unknown type 4
[ 82.895547] <idle>-0 0dN.. 82803302us : Unknown type 4
[ 82.895552] <idle>-0 0d... 82807272us : Unknown type 4
[ 82.895558] <idle>-0 0dN.. 82807294us : Unknown type 4
[ 82.895563] <idle>-0 0d... 82811264us : Unknown type 4
[ 82.895568] <idle>-0 0dN.. 82811288us : Unknown type 4
[ 82.895573] <idle>-0 0d... 82815258us : Unknown type 4
[ 82.895578] <idle>-0 0dN.. 82815281us : Unknown type 4
[ 82.895583] <idle>-0 0d... 82819250us : Unknown type 4
[ 82.895588] <idle>-0 0dN.. 82819277us : Unknown type 4
[ 82.895594] <idle>-0 0d... 82823244us : Unknown type 4
[ 82.895599] <idle>-0 0dN.. 82823266us : Unknown type 4
[ 82.895604] <idle>-0 0d... 82827237us : Unknown type 4
[ 82.895609] <idle>-0 0dN.. 82827261us : Unknown type 4
[ 82.895615] <idle>-0 0d... 82831230us : Unknown type 4
[ 82.895620] <idle>-0 0dN.. 82831250us : Unknown type 4
[ 82.895625] <idle>-0 0d... 82835223us : Unknown type 4
[ 82.895631] <idle>-0 0dN.. 82835249us : Unknown type 4
[ 82.895635] Switched to NOHz mode on CPU #54
[ 82.895640] <idle>-0 0d... 82839217us : Unknown type 4
[ 82.895645] <idle>-0 0dN.. 82839243us : Unknown type 4
[ 82.895651] <idle>-0 0d... 82843208us : Unknown type 4
[ 82.895656] <idle>-0 0dN.. 82843231us : Unknown type 4
[ 82.895661] <idle>-0 0d... 82847201us : Unknown type 4
[ 82.895666] <idle>-0 0dN.. 82847224us : Unknown type 4
[ 82.895671] <idle>-0 0d... 82851195us : Unknown type 4
[ 82.895677] <idle>-0 0dN.. 82851219us : Unknown type 4
[ 82.895682] <idle>-0 0d... 82855188us : Unknown type 4
[ 82.895686] Switched to NOHz mode on CPU #42
[ 82.895693] Switched to NOHz mode on CPU #46
[ 82.895699] Switched to NOHz mode on CPU #49
[ 82.895705] <idle>-0 0dN.. 82855213us : Unknown type 4
[ 82.895709] Switched to NOHz mode on CPU #109
[ 82.895715] Switched to NOHz mode on CPU #111
[ 82.895720] <idle>-0 0d... 82859183us : Unknown type 4
[ 82.895724] Switched to NOHz mode on CPU #101
[ 82.895729] <idle>-0 0dN.. 82859211us : Unknown type 4
[ 82.895733] Switched to NOHz mode on CPU #98
[ 82.895739] Switched to NOHz mode on CPU #96
[ 82.895744] <idle>-0 0d... 82863174us : Unknown type 4
[ 82.895749] <idle>-0 0dN.. 82863198us : Unknown type 4
[ 82.895754] <idle>-0 0d... 82867167us : Unknown type 4
[ 82.895759] <idle>-0 0dN.. 82867191us : Unknown type 4
[ 82.895765] <idle>-0 0d... 82871161us : Unknown type 4
[ 82.895770] <idle>-0 0dN.. 82871185us : Unknown type 4
[ 82.895775] <idle>-0 0d... 82875153us : Unknown type 4
[ 82.895780] Switched to NOHz mode on CPU #17
[ 82.895784] <idle>-0 0dN.. 82875174us : Unknown type 4
[ 82.895790] Switched to NOHz mode on CPU #14
[ 82.895793] <idle>-0 0d... 82879147us : Unknown type 4
[ 82.895799] Switched to NOHz mode on CPU #3
[ 82.895801] <idle>-0 0dN.. 82879171us : Unknown type 4
[ 82.895806] <idle>-0 0d... 82883139us : Unknown type 4
[ 82.895811] <idle>-0 0dN.. 82883164us : Unknown type 4
[ 82.895814] Switched to NOHz mode on CPU #4
[ 82.895817] <idle>-0 0d... 82887132us : Unknown type 4
[ 82.895822] Switched to NOHz mode on CPU #153
[ 82.895827] <idle>-0 0dN.. 82887154us : Unknown type 4
[ 82.895833] <idle>-0 0d... 82891124us : Unknown type 4
[ 82.895838] <idle>-0 0dN.. 82891148us : Unknown type 4
[ 82.895843] <idle>-0 0d... 82893612us : Unknown type 4
[ 82.895848] <idle>-0 0d... 82893643us : Unknown type 4
[ 82.895853] <idle>-0 0d... 82895118us : Unknown type 4
[ 82.895859] <idle>-0 0dN.. 82895147us : Unknown type 4
[ 82.895864] <idle>-0 0d... 82895177us : Unknown type 4
[ 82.895865] ---------------------------------
but if compile from fedora 14 gcc, will still have some delay.
[ 33.464561] cpu_dev_init done
[ 51.953005] memory_dev_init done
and it also have that warning...
Thanks
Yinghai
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-18 21:10 ` Yinghai Lu
@ 2011-05-18 23:13 ` Frederic Weisbecker
2011-05-19 4:33 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-18 23:13 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On Wed, May 18, 2011 at 02:10:26PM -0700, Yinghai Lu wrote:
> On 05/16/2011 07:40 PM, Frederic Weisbecker wrote:
> > On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
> >> On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
> >>>
> >>> * Ingo Molnar <mingo@elte.hu> wrote:
> >>>
> >>>>> In the meantime, would you be willing to try out the patch at
> >>>>> https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
> >>>>> several configurations.
> >>>>
> >>>> Wasn't this the one i tested - or is it a new iteration?
> >>>>
> >>>> I'll try it in any case.
> >>>
> >>> oh, this was a new iteration, mea culpa!
> >>>
> >>> And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
> >>
> >> ;-)
> >>
> >> Unfortunately, the only reason I can see that it works is (1) there
> >> is some obscure bug in my code or (2) someone somewhere is failing to
> >> call irq_exit() on some interrupt-exit path. Much as I might be tempted
> >> to paper this one over, I believe that we do need to find whatever the
> >> underlying bug is.
> >>
> >> Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
> >> a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
> >> But I couldn't see any evidence of this in my admittedly cursory scan
> >> of the x86 interrupt-handling code.
> >>
> >> So until I learn differently, I am assuming that each and every
> >> irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
> >> is called after the final irq_exit() of a given burst of interrupts.
> >>
> >> If my assumptions are mistaken, please do let me know!
> >
> > So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
> > before the unpairing happened.
> >
> > I have tried to reproduce it but couldn't trigger anything.
> >
> > So it would be nice if Yinghai can test the patch below, since he was able
> > to trigger the warning.
> >
> > This is essentially Paul's patch but with stacktrace of the calls recorded.
> > Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
> > sanity check is positive. Beware as the trace will be dumped everytime
> > WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
> > rest.
> >
> > This requires CONFIG_TRACING. May be a good thing to boot with
> > "ftrace=nop" parameter, so that ftrace will set up a long enough buffer
> > to have an interesting trace.
>
> with this patches if the kernel is compiled from opensuse 11.3 no delay anymore, but have one warning:
>
> [ 82.895182] ------------[ cut here ]------------
> [ 82.895189] WARNING: at kernel/rcutree.c:352 rcu_enter_nohz+0x49/0x8b()
> [ 82.895193] Switched to NOHz mode on CPU #90
> [ 82.895199] Switched to NOHz mode on CPU #8
> [ 82.895202] Modules linked in:
> [ 82.895206] Switched to NOHz mode on CPU #28
> [ 82.895211] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-05234-g3a108a0-dirty #1016
> [ 82.895213] Call Trace:
> [ 82.895233] [<ffffffff81080144>] warn_slowpath_common+0x85/0x9d
> [ 82.895238] [<ffffffff81080176>] warn_slowpath_null+0x1a/0x1c
> [ 82.895242] [<ffffffff810d32cc>] rcu_enter_nohz+0x49/0x8b
> [ 82.895250] [<ffffffff810ab121>] tick_nohz_stop_sched_tick+0x27d/0x366
> [ 82.895255] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> [ 82.895261] [<ffffffff81bda6e3>] rest_init+0xb7/0xbe
> [ 82.895266] [<ffffffff81bda62c>] ? csum_partial_copy_generic+0x16c/0x16c
> [ 82.895272] [<ffffffff82742e39>] start_kernel+0x3b2/0x3bd
> [ 82.895276] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
> [ 82.895281] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
> [ 82.895290] ---[ end trace 2cfc591bf7de931f ]---
> [ 82.895310] Switched to NOHz mode on CPU #72
> [ 82.895315] Dumping ftrace buffer:
> [ 82.895328] ---------------------------------
> [ 82.895340] CPU:0 [LOST 35328 EVENTS]
> [ 82.895341] <idle>-0 0d... 82735399us : Unknown type 4
Doh, ftrace couldn't recognize that trace is of stacktrace type. We
need to initialize the events output earlier.
Sorry, can you retry after applying the following patch?
diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c
index cf535cc..ad92d9c 100644
--- a/kernel/trace/trace_output.c
+++ b/kernel/trace/trace_output.c
@@ -1278,4 +1278,4 @@ __init static int init_events(void)
return 0;
}
-device_initcall(init_events);
+early_initcall(init_events);
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-18 23:13 ` Frederic Weisbecker
@ 2011-05-19 4:33 ` Yinghai Lu
2011-05-19 14:47 ` Frederic Weisbecker
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-19 4:33 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On 05/18/2011 04:13 PM, Frederic Weisbecker wrote:
> On Wed, May 18, 2011 at 02:10:26PM -0700, Yinghai Lu wrote:
>> On 05/16/2011 07:40 PM, Frederic Weisbecker wrote:
>>> On Mon, May 16, 2011 at 02:24:49PM -0700, Paul E. McKenney wrote:
>>>> On Mon, May 16, 2011 at 02:23:29PM +0200, Ingo Molnar wrote:
>>>>>
>>>>> * Ingo Molnar <mingo@elte.hu> wrote:
>>>>>
>>>>>>> In the meantime, would you be willing to try out the patch at
>>>>>>> https://lkml.org/lkml/2011/5/14/89? This patch helped out Yinghai in
>>>>>>> several configurations.
>>>>>>
>>>>>> Wasn't this the one i tested - or is it a new iteration?
>>>>>>
>>>>>> I'll try it in any case.
>>>>>
>>>>> oh, this was a new iteration, mea culpa!
>>>>>
>>>>> And yes, it solves all problems for me as well. Mind pushing it as a fix? :-)
>>>>
>>>> ;-)
>>>>
>>>> Unfortunately, the only reason I can see that it works is (1) there
>>>> is some obscure bug in my code or (2) someone somewhere is failing to
>>>> call irq_exit() on some interrupt-exit path. Much as I might be tempted
>>>> to paper this one over, I believe that we do need to find whatever the
>>>> underlying bug is.
>>>>
>>>> Oh, yes, there is option (3) as well: maybe if an interrupt deschedules
>>>> a process, the final irq_exit() is omitted in favor of rcu_enter_nohz()?
>>>> But I couldn't see any evidence of this in my admittedly cursory scan
>>>> of the x86 interrupt-handling code.
>>>>
>>>> So until I learn differently, I am assuming that each and every
>>>> irq_enter() has a matching call to irq_exit(), and that rcu_enter_nohz()
>>>> is called after the final irq_exit() of a given burst of interrupts.
>>>>
>>>> If my assumptions are mistaken, please do let me know!
>>>
>>> So it would be nice to have a trace of the calls to rcu_irq_*() / rcu_*_nohz()
>>> before the unpairing happened.
>>>
>>> I have tried to reproduce it but couldn't trigger anything.
>>>
>>> So it would be nice if Yinghai can test the patch below, since he was able
>>> to trigger the warning.
>>>
>>> This is essentially Paul's patch but with stacktrace of the calls recorded.
>>> Then the whole trace is dumped on the console when one of the WARN_ON_ONCE
>>> sanity check is positive. Beware as the trace will be dumped everytime
>>> WARN_ON_ONCE() is positive. So the first dump is enough, you can ignore the
>>> rest.
>>>
>>> This requires CONFIG_TRACING. May be a good thing to boot with
>>> "ftrace=nop" parameter, so that ftrace will set up a long enough buffer
>>> to have an interesting trace.
>>
>> with this patches if the kernel is compiled from opensuse 11.3 no delay anymore, but have one warning:
>>
>> [ 82.895182] ------------[ cut here ]------------
>> [ 82.895189] WARNING: at kernel/rcutree.c:352 rcu_enter_nohz+0x49/0x8b()
>> [ 82.895193] Switched to NOHz mode on CPU #90
>> [ 82.895199] Switched to NOHz mode on CPU #8
>> [ 82.895202] Modules linked in:
>> [ 82.895206] Switched to NOHz mode on CPU #28
>> [ 82.895211] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-05234-g3a108a0-dirty #1016
>> [ 82.895213] Call Trace:
>> [ 82.895233] [<ffffffff81080144>] warn_slowpath_common+0x85/0x9d
>> [ 82.895238] [<ffffffff81080176>] warn_slowpath_null+0x1a/0x1c
>> [ 82.895242] [<ffffffff810d32cc>] rcu_enter_nohz+0x49/0x8b
>> [ 82.895250] [<ffffffff810ab121>] tick_nohz_stop_sched_tick+0x27d/0x366
>> [ 82.895255] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
>> [ 82.895261] [<ffffffff81bda6e3>] rest_init+0xb7/0xbe
>> [ 82.895266] [<ffffffff81bda62c>] ? csum_partial_copy_generic+0x16c/0x16c
>> [ 82.895272] [<ffffffff82742e39>] start_kernel+0x3b2/0x3bd
>> [ 82.895276] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
>> [ 82.895281] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
>> [ 82.895290] ---[ end trace 2cfc591bf7de931f ]---
>> [ 82.895310] Switched to NOHz mode on CPU #72
>> [ 82.895315] Dumping ftrace buffer:
>> [ 82.895328] ---------------------------------
>> [ 82.895340] CPU:0 [LOST 35328 EVENTS]
>> [ 82.895341] <idle>-0 0d... 82735399us : Unknown type 4
>
> Doh, ftrace couldn't recognize that trace is of stacktrace type. We
> need to initialize the events output earlier.
>
> Sorry, can you retry after applying the following patch?
>
> diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c
> index cf535cc..ad92d9c 100644
> --- a/kernel/trace/trace_output.c
> +++ b/kernel/trace/trace_output.c
> @@ -1278,4 +1278,4 @@ __init static int init_events(void)
>
> return 0;
> }
> -device_initcall(init_events);
> +early_initcall(init_events);
...
[ 89.913205] ------------[ cut here ]------------
[ 89.913216] WARNING: at kernel/rcutree.c:352 rcu_enter_nohz+0x49/0x8b()
[ 89.913220] Hardware name: Sun Fire X4800 M2
[ 89.913224] Modules linked in:
[ 89.913230] Switched to NOHz mode on CPU #34
[ 89.913237] Pid: 0, comm: swapper Tainted: G W 2.6.39-rc7-tip-yh-05281-g8e9254b-dirty #1019
[ 89.913242] Call Trace:
[ 89.913269] [<ffffffff81080144>] warn_slowpath_common+0x85/0x9d
[ 89.913280] [<ffffffff81080176>] warn_slowpath_null+0x1a/0x1c
[ 89.913290] [<ffffffff810d32cc>] rcu_enter_nohz+0x49/0x8b
[ 89.913305] [<ffffffff810ab121>] tick_nohz_stop_sched_tick+0x27d/0x366
[ 89.913317] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
[ 89.913329] [<ffffffff81bda6e3>] rest_init+0xb7/0xbe
[ 89.913340] [<ffffffff81bda62c>] ? csum_partial_copy_generic+0x16c/0x16c
[ 89.913352] [<ffffffff82742e39>] start_kernel+0x3b2/0x3bd
[ 89.913363] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
[ 89.913374] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
[ 89.913379] ---[ end trace a7919e7f17c0a726 ]---
[ 89.913401] Dumping ftrace buffer:
[ 89.913417] Switched to NOHz mode on CPU #4
[ 89.913428] ---------------------------------
[ 89.913444] Switched to NOHz mode on CPU #54
[ 89.913467] CPU:0 [LOST 39120 EVENTS]
[ 89.913469] <idle>-0 0d... 89769327us : <stack trace>
[ 89.913472] => rcu_irq_enter
[ 89.913474] => irq_enter
[ 89.913475] => smp_apic_timer_interrupt
[ 89.913478] => apic_timer_interrupt
[ 89.913479] => cpu_idle
[ 89.913481] => rest_init
[ 89.913482] => start_kernel
[ 89.913484] => x86_64_start_reservations
[ 89.913490] Switched to NOHz mode on CPU #48
[ 89.913496] Switched to NOHz mode on CPU #42
[ 89.913509] <idle>-0 0dN.. 89769365us : <stack trace>
[ 89.913512] => rcu_irq_exit
[ 89.913513] => irq_exit
[ 89.913515] => smp_apic_timer_interrupt
[ 89.913517] => apic_timer_interrupt
[ 89.913518] => cpu_idle
[ 89.913520] => rest_init
[ 89.913521] => start_kernel
[ 89.913523] => x86_64_start_reservations
[ 89.913546] <idle>-0 0d... 89773320us : <stack trace>
[ 89.913549] => rcu_irq_enter
[ 89.913550] => irq_enter
[ 89.913552] => smp_apic_timer_interrupt
[ 89.913554] => apic_timer_interrupt
[ 89.913556] => cpu_idle
[ 89.913557] => rest_init
[ 89.913558] => start_kernel
[ 89.913560] => x86_64_start_reservations
[ 89.913583] <idle>-0 0dN.. 89773360us : <stack trace>
[ 89.913588] => rcu_irq_exit
[ 89.913589] => irq_exit
[ 89.913592] => smp_apic_timer_interrupt
[ 89.913594] => apic_timer_interrupt
[ 89.913595] => cpu_idle
[ 89.913597] => rest_init
[ 89.913598] => start_kernel
[ 89.913600] => x86_64_start_reservations
[ 89.913606] Switched to NOHz mode on CPU #33
[ 89.913613] Switched to NOHz mode on CPU #61
[ 89.913623] Switched to NOHz mode on CPU #102
[ 89.913629] Switched to NOHz mode on CPU #50
[ 89.913634] <idle>-0 0d... 89777313us : <stack trace>
[ 89.913637] => rcu_irq_enter
[ 89.913638] => irq_enter
[ 89.913640] => smp_apic_timer_interrupt
[ 89.913641] => apic_timer_interrupt
[ 89.913643] => cpu_idle
[ 89.913645] => rest_init
[ 89.913646] => start_kernel
[ 89.913648] => x86_64_start_reservations
[ 89.913662] Switched to NOHz mode on CPU #41
[ 89.913668] Switched to NOHz mode on CPU #45
[ 89.913674] <idle>-0 0dN.. 89777352us : <stack trace>
[ 89.913677] => rcu_irq_exit
[ 89.913678] => irq_exit
[ 89.913680] => smp_apic_timer_interrupt
[ 89.913682] => apic_timer_interrupt
[ 89.913683] => cpu_idle
[ 89.913685] => rest_init
[ 89.913686] => start_kernel
[ 89.913688] => x86_64_start_reservations
[ 89.913711] <idle>-0 0d... 89781306us : <stack trace>
[ 89.913713] => rcu_irq_enter
[ 89.913715] => irq_enter
[ 89.913717] => smp_apic_timer_interrupt
[ 89.913719] => apic_timer_interrupt
[ 89.913720] => cpu_idle
[ 89.913722] => rest_init
[ 89.913723] => start_kernel
[ 89.913725] => x86_64_start_reservations
[ 89.913747] <idle>-0 0dN.. 89781340us : <stack trace>
[ 89.913751] => rcu_irq_exit
[ 89.913753] => irq_exit
[ 89.913754] => smp_apic_timer_interrupt
[ 89.913756] => apic_timer_interrupt
[ 89.913758] => cpu_idle
[ 89.913759] => rest_init
[ 89.913761] => start_kernel
[ 89.913762] => x86_64_start_reservations
[ 89.913768] Switched to NOHz mode on CPU #35
[ 89.913786] <idle>-0 0d... 89785299us : <stack trace>
[ 89.913788] => rcu_irq_enter
[ 89.913790] => irq_enter
[ 89.913791] => smp_apic_timer_interrupt
[ 89.913793] => apic_timer_interrupt
[ 89.913795] => cpu_idle
[ 89.913796] => rest_init
[ 89.913798] => start_kernel
[ 89.913799] => x86_64_start_reservations
[ 89.913822] <idle>-0 0dN.. 89785334us : <stack trace>
[ 89.913825] => rcu_irq_exit
[ 89.913826] => irq_exit
[ 89.913828] => smp_apic_timer_interrupt
[ 89.913830] => apic_timer_interrupt
[ 89.913831] => cpu_idle
[ 89.913833] => rest_init
[ 89.913834] => start_kernel
[ 89.913836] => x86_64_start_reservations
[ 89.913859] <idle>-0 0d... 89789292us : <stack trace>
[ 89.913861] => rcu_irq_enter
[ 89.913863] => irq_enter
[ 89.913865] => smp_apic_timer_interrupt
[ 89.913866] => apic_timer_interrupt
[ 89.913868] => cpu_idle
[ 89.913870] => rest_init
[ 89.913871] => start_kernel
[ 89.913873] => x86_64_start_reservations
[ 89.913895] <idle>-0 0dN.. 89789326us : <stack trace>
[ 89.913898] => rcu_irq_exit
[ 89.913900] => irq_exit
[ 89.913901] => smp_apic_timer_interrupt
[ 89.913903] => apic_timer_interrupt
[ 89.913905] => cpu_idle
[ 89.913906] => rest_init
[ 89.913908] => start_kernel
[ 89.913909] => x86_64_start_reservations
[ 89.913932] <idle>-0 0d... 89793285us : <stack trace>
[ 89.913935] => rcu_irq_enter
[ 89.913936] => irq_enter
[ 89.913938] => smp_apic_timer_interrupt
[ 89.913939] => apic_timer_interrupt
[ 89.913941] => cpu_idle
[ 89.913943] => rest_init
[ 89.913944] => start_kernel
[ 89.913946] => x86_64_start_reservations
[ 89.913968] <idle>-0 0dN.. 89793320us : <stack trace>
[ 89.913971] => rcu_irq_exit
[ 89.913973] => irq_exit
[ 89.913974] => smp_apic_timer_interrupt
[ 89.913976] => apic_timer_interrupt
[ 89.913978] => cpu_idle
[ 89.913979] => rest_init
[ 89.913981] => start_kernel
[ 89.913982] => x86_64_start_reservations
[ 89.914005] <idle>-0 0d... 89797278us : <stack trace>
[ 89.914008] => rcu_irq_enter
[ 89.914009] => irq_enter
[ 89.914011] => smp_apic_timer_interrupt
[ 89.914013] => apic_timer_interrupt
[ 89.914014] => cpu_idle
[ 89.914016] => rest_init
[ 89.914017] => start_kernel
[ 89.914019] => x86_64_start_reservations
[ 89.914042] <idle>-0 0dN.. 89797319us : <stack trace>
[ 89.914044] => rcu_irq_exit
[ 89.914046] => irq_exit
[ 89.914047] => smp_apic_timer_interrupt
[ 89.914049] => apic_timer_interrupt
[ 89.914051] => cpu_idle
[ 89.914052] => rest_init
[ 89.914054] => start_kernel
[ 89.914055] => x86_64_start_reservations
[ 89.914078] <idle>-0 0d... 89801271us : <stack trace>
[ 89.914081] => rcu_irq_enter
[ 89.914082] => irq_enter
[ 89.914084] => smp_apic_timer_interrupt
[ 89.914086] => apic_timer_interrupt
[ 89.914088] => cpu_idle
[ 89.914089] => rest_init
[ 89.914090] => start_kernel
[ 89.914092] => x86_64_start_reservations
[ 89.914115] <idle>-0 0dN.. 89801311us : <stack trace>
[ 89.914118] => rcu_irq_exit
[ 89.914119] => irq_exit
[ 89.914121] => smp_apic_timer_interrupt
[ 89.914123] => apic_timer_interrupt
[ 89.914124] => cpu_idle
[ 89.914126] => rest_init
[ 89.914127] => start_kernel
[ 89.914129] => x86_64_start_reservations
[ 89.914152] <idle>-0 0d... 89805264us : <stack trace>
[ 89.914154] => rcu_irq_enter
[ 89.914156] => irq_enter
[ 89.914158] => smp_apic_timer_interrupt
[ 89.914159] => apic_timer_interrupt
[ 89.914161] => cpu_idle
[ 89.914163] => rest_init
[ 89.914164] => start_kernel
[ 89.914166] => x86_64_start_reservations
[ 89.914188] <idle>-0 0dN.. 89805297us : <stack trace>
[ 89.914191] => rcu_irq_exit
[ 89.914193] => irq_exit
[ 89.914194] => smp_apic_timer_interrupt
[ 89.914196] => apic_timer_interrupt
[ 89.914198] => cpu_idle
[ 89.914199] => rest_init
[ 89.914200] => start_kernel
[ 89.914202] => x86_64_start_reservations
[ 89.914225] <idle>-0 0d... 89809257us : <stack trace>
[ 89.914227] => rcu_irq_enter
[ 89.914229] => irq_enter
[ 89.914231] => smp_apic_timer_interrupt
[ 89.914232] => apic_timer_interrupt
[ 89.914234] => cpu_idle
[ 89.914236] => rest_init
[ 89.914237] => start_kernel
[ 89.914239] => x86_64_start_reservations
[ 89.914261] <idle>-0 0dN.. 89809292us : <stack trace>
[ 89.914264] => rcu_irq_exit
[ 89.914266] => irq_exit
[ 89.914267] => smp_apic_timer_interrupt
[ 89.914269] => apic_timer_interrupt
[ 89.914271] => cpu_idle
[ 89.914272] => rest_init
[ 89.914274] => start_kernel
[ 89.914275] => x86_64_start_reservations
[ 89.914298] <idle>-0 0d... 89813250us : <stack trace>
[ 89.914301] => rcu_irq_enter
[ 89.914302] => irq_enter
[ 89.914304] => smp_apic_timer_interrupt
[ 89.914306] => apic_timer_interrupt
[ 89.914307] => cpu_idle
[ 89.914309] => rest_init
[ 89.914310] => start_kernel
[ 89.914312] => x86_64_start_reservations
[ 89.914329] Switched to NOHz mode on CPU #38
[ 89.914335] <idle>-0 0dN.. 89813283us : <stack trace>
[ 89.914338] => rcu_irq_exit
[ 89.914339] => irq_exit
[ 89.914341] => smp_apic_timer_interrupt
[ 89.914343] => apic_timer_interrupt
[ 89.914344] => cpu_idle
[ 89.914346] => rest_init
[ 89.914347] => start_kernel
[ 89.914349] => x86_64_start_reservations
[ 89.914359] Switched to NOHz mode on CPU #154
[ 89.914365] Switched to NOHz mode on CPU #159
[ 89.914373] <idle>-0 0d... 89817243us : <stack trace>
[ 89.914376] => rcu_irq_enter
[ 89.914378] => irq_enter
[ 89.914381] => smp_apic_timer_interrupt
[ 89.914384] => apic_timer_interrupt
[ 89.914385] => cpu_idle
[ 89.914387] => rest_init
[ 89.914388] => start_kernel
[ 89.914390] => x86_64_start_reservations
[ 89.914400] Switched to NOHz mode on CPU #142
[ 89.914407] Switched to NOHz mode on CPU #148
[ 89.914415] Switched to NOHz mode on CPU #147
[ 89.914423] Switched to NOHz mode on CPU #153
[ 89.914430] Switched to NOHz mode on CPU #157
[ 89.914438] <idle>-0 0dN.. 89817279us : <stack trace>
[ 89.914441] => rcu_irq_exit
[ 89.914443] => irq_exit
[ 89.914444] => smp_apic_timer_interrupt
[ 89.914446] => apic_timer_interrupt
[ 89.914448] => cpu_idle
[ 89.914449] => rest_init
[ 89.914451] => start_kernel
[ 89.914452] => x86_64_start_reservations
[ 89.914458] Switched to NOHz mode on CPU #146
[ 89.914462] Switched to NOHz mode on CPU #144
[ 89.914467] Switched to NOHz mode on CPU #136
[ 89.914472] Switched to NOHz mode on CPU #133
[ 89.914479] <idle>-0 0d... 89821236us : <stack trace>
[ 89.914482] => rcu_irq_enter
[ 89.914484] => irq_enter
[ 89.914485] => smp_apic_timer_interrupt
[ 89.914487] => apic_timer_interrupt
[ 89.914489] => cpu_idle
[ 89.914490] => rest_init
[ 89.914492] => start_kernel
[ 89.914494] => x86_64_start_reservations
[ 89.914500] Switched to NOHz mode on CPU #123
[ 89.914505] Switched to NOHz mode on CPU #128
[ 89.914510] Switched to NOHz mode on CPU #119
[ 89.914515] Switched to NOHz mode on CPU #117
[ 89.914524] Switched to NOHz mode on CPU #116
[ 89.914533] Switched to NOHz mode on CPU #115
[ 89.914541] Switched to NOHz mode on CPU #131
[ 89.914547] Switched to NOHz mode on CPU #150
[ 89.914554] <idle>-0 0dN.. 89821271us : <stack trace>
[ 89.914557] => rcu_irq_exit
[ 89.914559] => irq_exit
[ 89.914560] => smp_apic_timer_interrupt
[ 89.914562] => apic_timer_interrupt
[ 89.914564] => cpu_idle
[ 89.914566] => rest_init
[ 89.914567] => start_kernel
[ 89.914569] => x86_64_start_reservations
[ 89.914576] Switched to NOHz mode on CPU #109
[ 89.914585] Switched to NOHz mode on CPU #106
[ 89.914593] Switched to NOHz mode on CPU #100
[ 89.914601] Switched to NOHz mode on CPU #101
[ 89.914611] Switched to NOHz mode on CPU #111
[ 89.914619] Switched to NOHz mode on CPU #124
[ 89.914628] Switched to NOHz mode on CPU #83
[ 89.914636] <idle>-0 0d... 89825229us : <stack trace>
[ 89.914642] => rcu_irq_enter
[ 89.914646] => irq_enter
[ 89.914648] => smp_apic_timer_interrupt
[ 89.914651] => apic_timer_interrupt
[ 89.914653] => cpu_idle
[ 89.914655] => rest_init
[ 89.914659] => start_kernel
[ 89.914660] => x86_64_start_reservations
[ 89.914673] Switched to NOHz mode on CPU #7
[ 89.914682] Switched to NOHz mode on CPU #87
[ 89.914697] Switched to NOHz mode on CPU #9
[ 89.914711] Switched to NOHz mode on CPU #6
[ 89.914736] <idle>-0 0dN.. 89825262us : <stack trace>
[ 89.914740] => rcu_irq_exit
[ 89.914743] => irq_exit
[ 89.914745] => smp_apic_timer_interrupt
[ 89.914748] => apic_timer_interrupt
[ 89.914759] => cpu_idle
[ 89.914761] => rest_init
[ 89.914763] => start_kernel
[ 89.914766] => x86_64_start_reservations
[ 89.914773] Switched to NOHz mode on CPU #80
[ 89.914794] Switched to NOHz mode on CPU #2
[ 89.914804] Switched to NOHz mode on CPU #82
[ 89.914824] Switched to NOHz mode on CPU #85
[ 89.914847] Switched to NOHz mode on CPU #88
[ 89.914881] <idle>-0 0d... 89829222us : <stack trace>
[ 89.914891] => rcu_irq_enter
[ 89.914893] => irq_enter
[ 89.914894] => smp_apic_timer_interrupt
[ 89.914896] => apic_timer_interrupt
[ 89.914898] => cpu_idle
[ 89.914899] => rest_init
[ 89.914901] => start_kernel
[ 89.914902] => x86_64_start_reservations
[ 89.914926] Switched to NOHz mode on CPU #97
[ 89.914954] Switched to NOHz mode on CPU #95
[ 89.915011] Switched to NOHz mode on CPU #99
[ 89.915049] Switched to NOHz mode on CPU #10
[ 89.915080] Switched to NOHz mode on CPU #90
[ 89.915142] Switched to NOHz mode on CPU #13
[ 89.915166] Switched to NOHz mode on CPU #93
[ 89.915209] Switched to NOHz mode on CPU #15
[ 89.915274] Switched to NOHz mode on CPU #98
[ 89.915294] Switched to NOHz mode on CPU #18
[ 89.915333] Switched to NOHz mode on CPU #92
[ 89.915381] Switched to NOHz mode on CPU #12
[ 89.915450] Switched to NOHz mode on CPU #14
[ 89.915512] Switched to NOHz mode on CPU #94
[ 89.915551] Switched to NOHz mode on CPU #91
[ 89.915600] Switched to NOHz mode on CPU #96
[ 89.915639] Switched to NOHz mode on CPU #21
[ 89.915677] Switched to NOHz mode on CPU #22
[ 89.915722] Switched to NOHz mode on CPU #29
[ 89.915761] Switched to NOHz mode on CPU #107
[ 89.915789] Switched to NOHz mode on CPU #26
[ 89.915857] Switched to NOHz mode on CPU #105
[ 89.915876] Switched to NOHz mode on CPU #25
[ 89.915902] Switched to NOHz mode on CPU #108
[ 89.915911] Switched to NOHz mode on CPU #28
[ 89.915942] Switched to NOHz mode on CPU #23
[ 89.915964] Switched to NOHz mode on CPU #103
[ 89.915994] Switched to NOHz mode on CPU #20
[ 89.916024] Switched to NOHz mode on CPU #27
[ 89.916043] Switched to NOHz mode on CPU #24
[ 89.916059] Switched to NOHz mode on CPU #104
[ 89.916104] Switched to NOHz mode on CPU #31
[ 89.916140] Switched to NOHz mode on CPU #32
[ 89.916153] Switched to NOHz mode on CPU #112
[ 89.916179] Switched to NOHz mode on CPU #114
[ 89.916216] Switched to NOHz mode on CPU #30
[ 89.916233] Switched to NOHz mode on CPU #110
[ 89.916253] Switched to NOHz mode on CPU #113
[ 89.916276] Switched to NOHz mode on CPU #118
[ 89.916305] Switched to NOHz mode on CPU #129
[ 89.916323] Switched to NOHz mode on CPU #121
[ 89.916340] Switched to NOHz mode on CPU #125
[ 89.916367] Switched to NOHz mode on CPU #122
[ 89.916387] Switched to NOHz mode on CPU #127
[ 89.916415] Switched to NOHz mode on CPU #120
[ 89.916437] Switched to NOHz mode on CPU #126
[ 89.916454] Switched to NOHz mode on CPU #139
[ 89.916479] Switched to NOHz mode on CPU #135
[ 89.916492] Switched to NOHz mode on CPU #137
[ 89.916513] Switched to NOHz mode on CPU #134
[ 89.916530] Switched to NOHz mode on CPU #138
[ 89.916539] Switched to NOHz mode on CPU #132
[ 89.916560] Switched to NOHz mode on CPU #130
[ 89.916575] Switched to NOHz mode on CPU #68
[ 89.916581] Switched to NOHz mode on CPU #64
[ 89.916597] Switched to NOHz mode on CPU #65
[ 89.916608] Switched to NOHz mode on CPU #145
[ 89.916620] Switched to NOHz mode on CPU #141
[ 89.916633] Switched to NOHz mode on CPU #63
[ 89.916642] Switched to NOHz mode on CPU #67
[ 89.916650] Switched to NOHz mode on CPU #152
[ 89.916659] Switched to NOHz mode on CPU #72
[ 89.916673] Switched to NOHz mode on CPU #77
[ 89.916688] Switched to NOHz mode on CPU #158
[ 89.916699] Switched to NOHz mode on CPU #78
[ 89.916713] Switched to NOHz mode on CPU #70
[ 89.916725] Switched to NOHz mode on CPU #76
[ 89.916736] Switched to NOHz mode on CPU #156
[ 89.916748] Switched to NOHz mode on CPU #151
[ 89.916756] Switched to NOHz mode on CPU #71
[ 89.916768] Switched to NOHz mode on CPU #75
[ 89.916775] Switched to NOHz mode on CPU #155
[ 89.916786] Switched to NOHz mode on CPU #73
[ 89.916795] Switched to NOHz mode on CPU #74
[ 89.916806] <idle>-0 0dN.. 89829255us : <stack trace>
[ 89.916812] => rcu_irq_exit
[ 89.916814] => irq_exit
[ 89.916816] => smp_apic_timer_interrupt
[ 89.916818] => apic_timer_interrupt
[ 89.916819] => cpu_idle
[ 89.916821] => rest_init
[ 89.916823] => start_kernel
[ 89.916824] => x86_64_start_reservations
[ 89.916832] Switched to NOHz mode on CPU #89
[ 89.916841] Switched to NOHz mode on CPU #84
[ 89.916853] Switched to NOHz mode on CPU #81
[ 89.916863] Switched to NOHz mode on CPU #1
[ 89.916872] Switched to NOHz mode on CPU #5
[ 89.916882] Switched to NOHz mode on CPU #8
[ 89.916889] Switched to NOHz mode on CPU #3
[ 89.916898] Switched to NOHz mode on CPU #86
[ 89.916904] <idle>-0 0d... 89833215us : <stack trace>
[ 89.916907] => rcu_irq_enter
[ 89.916909] => irq_enter
[ 89.916911] => smp_apic_timer_interrupt
[ 89.916913] => apic_timer_interrupt
[ 89.916915] => cpu_idle
[ 89.916916] => rest_init
[ 89.916918] => start_kernel
[ 89.916919] => x86_64_start_reservations
[ 89.916926] Switched to NOHz mode on CPU #16
[ 89.916932] Switched to NOHz mode on CPU #17
[ 89.916939] Switched to NOHz mode on CPU #11
[ 89.916946] Switched to NOHz mode on CPU #19
[ 89.916952] Switched to NOHz mode on CPU #66
[ 89.916956] Switched to NOHz mode on CPU #69
[ 89.916960] Switched to NOHz mode on CPU #149
[ 89.916964] Switched to NOHz mode on CPU #140
[ 89.916969] <idle>-0 0dN.. 89833252us : <stack trace>
[ 89.916971] => rcu_irq_exit
[ 89.916973] => irq_exit
[ 89.916974] => smp_apic_timer_interrupt
[ 89.916976] => apic_timer_interrupt
[ 89.916978] => cpu_idle
[ 89.916980] => rest_init
[ 89.916981] => start_kernel
[ 89.916983] => x86_64_start_reservations
[ 89.917006] <idle>-0 0d... 89837208us : <stack trace>
[ 89.917009] => rcu_irq_enter
[ 89.917010] => irq_enter
[ 89.917012] => smp_apic_timer_interrupt
[ 89.917014] => apic_timer_interrupt
[ 89.917015] => cpu_idle
[ 89.917017] => rest_init
[ 89.917018] => start_kernel
[ 89.917020] => x86_64_start_reservations
[ 89.917043] <idle>-0 0dN.. 89837248us : <stack trace>
[ 89.917046] => rcu_irq_exit
[ 89.917047] => irq_exit
[ 89.917049] => smp_apic_timer_interrupt
[ 89.917051] => apic_timer_interrupt
[ 89.917053] => cpu_idle
[ 89.917054] => rest_init
[ 89.917055] => start_kernel
[ 89.917057] => x86_64_start_reservations
[ 89.917080] <idle>-0 0d... 89841201us : <stack trace>
[ 89.917083] => rcu_irq_enter
[ 89.917084] => irq_enter
[ 89.917086] => smp_apic_timer_interrupt
[ 89.917088] => apic_timer_interrupt
[ 89.917089] => cpu_idle
[ 89.917091] => rest_init
[ 89.917092] => start_kernel
[ 89.917094] => x86_64_start_reservations
[ 89.917117] <idle>-0 0dN.. 89841242us : <stack trace>
[ 89.917119] => rcu_irq_exit
[ 89.917121] => irq_exit
[ 89.917123] => smp_apic_timer_interrupt
[ 89.917125] => apic_timer_interrupt
[ 89.917126] => cpu_idle
[ 89.917128] => rest_init
[ 89.917129] => start_kernel
[ 89.917131] => x86_64_start_reservations
[ 89.917154] <idle>-0 0d... 89845194us : <stack trace>
[ 89.917156] => rcu_irq_enter
[ 89.917158] => irq_enter
[ 89.917160] => smp_apic_timer_interrupt
[ 89.917161] => apic_timer_interrupt
[ 89.917163] => cpu_idle
[ 89.917165] => rest_init
[ 89.917166] => start_kernel
[ 89.917168] => x86_64_start_reservations
[ 89.917190] <idle>-0 0dN.. 89845234us : <stack trace>
[ 89.917193] => rcu_irq_exit
[ 89.917195] => irq_exit
[ 89.917196] => smp_apic_timer_interrupt
[ 89.917198] => apic_timer_interrupt
[ 89.917200] => cpu_idle
[ 89.917201] => rest_init
[ 89.917203] => start_kernel
[ 89.917204] => x86_64_start_reservations
[ 89.917227] <idle>-0 0d... 89849187us : <stack trace>
[ 89.917230] => rcu_irq_enter
[ 89.917231] => irq_enter
[ 89.917233] => smp_apic_timer_interrupt
[ 89.917235] => apic_timer_interrupt
[ 89.917237] => cpu_idle
[ 89.917238] => rest_init
[ 89.917240] => start_kernel
[ 89.917241] => x86_64_start_reservations
[ 89.917264] <idle>-0 0dN.. 89849228us : <stack trace>
[ 89.917267] => rcu_irq_exit
[ 89.917269] => irq_exit
[ 89.917270] => smp_apic_timer_interrupt
[ 89.917272] => apic_timer_interrupt
[ 89.917274] => cpu_idle
[ 89.917275] => rest_init
[ 89.917276] => start_kernel
[ 89.917278] => x86_64_start_reservations
[ 89.917301] <idle>-0 0d... 89853180us : <stack trace>
[ 89.917304] => rcu_irq_enter
[ 89.917305] => irq_enter
[ 89.917307] => smp_apic_timer_interrupt
[ 89.917309] => apic_timer_interrupt
[ 89.917310] => cpu_idle
[ 89.917312] => rest_init
[ 89.917313] => start_kernel
[ 89.917315] => x86_64_start_reservations
[ 89.917338] <idle>-0 0dN.. 89853222us : <stack trace>
[ 89.917340] => rcu_irq_exit
[ 89.917342] => irq_exit
[ 89.917343] => smp_apic_timer_interrupt
[ 89.917345] => apic_timer_interrupt
[ 89.917347] => cpu_idle
[ 89.917349] => rest_init
[ 89.917350] => start_kernel
[ 89.917352] => x86_64_start_reservations
[ 89.917375] <idle>-0 0d... 89857173us : <stack trace>
[ 89.917378] => rcu_irq_enter
[ 89.917379] => irq_enter
[ 89.917381] => smp_apic_timer_interrupt
[ 89.917383] => apic_timer_interrupt
[ 89.917384] => cpu_idle
[ 89.917386] => rest_init
[ 89.917387] => start_kernel
[ 89.917389] => x86_64_start_reservations
[ 89.917412] <idle>-0 0dN.. 89857211us : <stack trace>
[ 89.917414] => rcu_irq_exit
[ 89.917416] => irq_exit
[ 89.917418] => smp_apic_timer_interrupt
[ 89.917419] => apic_timer_interrupt
[ 89.917421] => cpu_idle
[ 89.917423] => rest_init
[ 89.917424] => start_kernel
[ 89.917426] => x86_64_start_reservations
[ 89.917449] <idle>-0 0d... 89861166us : <stack trace>
[ 89.917451] => rcu_irq_enter
[ 89.917453] => irq_enter
[ 89.917454] => smp_apic_timer_interrupt
[ 89.917456] => apic_timer_interrupt
[ 89.917458] => cpu_idle
[ 89.917459] => rest_init
[ 89.917461] => start_kernel
[ 89.917462] => x86_64_start_reservations
[ 89.917485] <idle>-0 0dN.. 89861199us : <stack trace>
[ 89.917488] => rcu_irq_exit
[ 89.917490] => irq_exit
[ 89.917491] => smp_apic_timer_interrupt
[ 89.917493] => apic_timer_interrupt
[ 89.917495] => cpu_idle
[ 89.917496] => rest_init
[ 89.917498] => start_kernel
[ 89.917499] => x86_64_start_reservations
[ 89.917524] <idle>-0 0d... 89865159us : <stack trace>
[ 89.917526] => rcu_irq_enter
[ 89.917528] => irq_enter
[ 89.917530] => smp_apic_timer_interrupt
[ 89.917531] => apic_timer_interrupt
[ 89.917533] => cpu_idle
[ 89.917535] => rest_init
[ 89.917536] => start_kernel
[ 89.917538] => x86_64_start_reservations
[ 89.917560] <idle>-0 0dN.. 89865193us : <stack trace>
[ 89.917563] => rcu_irq_exit
[ 89.917565] => irq_exit
[ 89.917566] => smp_apic_timer_interrupt
[ 89.917568] => apic_timer_interrupt
[ 89.917570] => cpu_idle
[ 89.917571] => rest_init
[ 89.917573] => start_kernel
[ 89.917574] => x86_64_start_reservations
[ 89.917597] <idle>-0 0d... 89869152us : <stack trace>
[ 89.917600] => rcu_irq_enter
[ 89.917602] => irq_enter
[ 89.917603] => smp_apic_timer_interrupt
[ 89.917605] => apic_timer_interrupt
[ 89.917607] => cpu_idle
[ 89.917608] => rest_init
[ 89.917610] => start_kernel
[ 89.917611] => x86_64_start_reservations
[ 89.917634] <idle>-0 0dN.. 89869185us : <stack trace>
[ 89.917636] => rcu_irq_exit
[ 89.917638] => irq_exit
[ 89.917640] => smp_apic_timer_interrupt
[ 89.917641] => apic_timer_interrupt
[ 89.917643] => cpu_idle
[ 89.917645] => rest_init
[ 89.917646] => start_kernel
[ 89.917648] => x86_64_start_reservations
[ 89.917671] <idle>-0 0d... 89873145us : <stack trace>
[ 89.917673] => rcu_irq_enter
[ 89.917675] => irq_enter
[ 89.917677] => smp_apic_timer_interrupt
[ 89.917678] => apic_timer_interrupt
[ 89.917680] => cpu_idle
[ 89.917682] => rest_init
[ 89.917683] => start_kernel
[ 89.917685] => x86_64_start_reservations
[ 89.917708] <idle>-0 0dN.. 89873178us : <stack trace>
[ 89.917710] => rcu_irq_exit
[ 89.917712] => irq_exit
[ 89.917713] => smp_apic_timer_interrupt
[ 89.917715] => apic_timer_interrupt
[ 89.917717] => cpu_idle
[ 89.917718] => rest_init
[ 89.917720] => start_kernel
[ 89.917721] => x86_64_start_reservations
[ 89.917744] <idle>-0 0d... 89877138us : <stack trace>
[ 89.917747] => rcu_irq_enter
[ 89.917749] => irq_enter
[ 89.917750] => smp_apic_timer_interrupt
[ 89.917752] => apic_timer_interrupt
[ 89.917754] => cpu_idle
[ 89.917755] => rest_init
[ 89.917757] => start_kernel
[ 89.917758] => x86_64_start_reservations
[ 89.917781] <idle>-0 0dN.. 89877174us : <stack trace>
[ 89.917784] => rcu_irq_exit
[ 89.917786] => irq_exit
[ 89.917787] => smp_apic_timer_interrupt
[ 89.917789] => apic_timer_interrupt
[ 89.917791] => cpu_idle
[ 89.917792] => rest_init
[ 89.917794] => start_kernel
[ 89.917795] => x86_64_start_reservations
[ 89.917818] <idle>-0 0d... 89881131us : <stack trace>
[ 89.917821] => rcu_irq_enter
[ 89.917823] => irq_enter
[ 89.917824] => smp_apic_timer_interrupt
[ 89.917826] => apic_timer_interrupt
[ 89.917828] => cpu_idle
[ 89.917829] => rest_init
[ 89.917831] => start_kernel
[ 89.917832] => x86_64_start_reservations
[ 89.917855] <idle>-0 0dN.. 89881165us : <stack trace>
[ 89.917858] => rcu_irq_exit
[ 89.917859] => irq_exit
[ 89.917861] => smp_apic_timer_interrupt
[ 89.917863] => apic_timer_interrupt
[ 89.917865] => cpu_idle
[ 89.917866] => rest_init
[ 89.917867] => start_kernel
[ 89.917869] => x86_64_start_reservations
[ 89.917892] <idle>-0 0d... 89885124us : <stack trace>
[ 89.917894] => rcu_irq_enter
[ 89.917896] => irq_enter
[ 89.917898] => smp_apic_timer_interrupt
[ 89.917899] => apic_timer_interrupt
[ 89.917901] => cpu_idle
[ 89.917903] => rest_init
[ 89.917904] => start_kernel
[ 89.917906] => x86_64_start_reservations
[ 89.917929] <idle>-0 0dN.. 89885161us : <stack trace>
[ 89.917931] => rcu_irq_exit
[ 89.917933] => irq_exit
[ 89.917934] => smp_apic_timer_interrupt
[ 89.917936] => apic_timer_interrupt
[ 89.917938] => cpu_idle
[ 89.917940] => rest_init
[ 89.917941] => start_kernel
[ 89.917943] => x86_64_start_reservations
[ 89.917965] <idle>-0 0d... 89889117us : <stack trace>
[ 89.917968] => rcu_irq_enter
[ 89.917970] => irq_enter
[ 89.917971] => smp_apic_timer_interrupt
[ 89.917973] => apic_timer_interrupt
[ 89.917975] => cpu_idle
[ 89.917976] => rest_init
[ 89.917978] => start_kernel
[ 89.917979] => x86_64_start_reservations
[ 89.918002] <idle>-0 0dN.. 89889150us : <stack trace>
[ 89.918005] => rcu_irq_exit
[ 89.918006] => irq_exit
[ 89.918008] => smp_apic_timer_interrupt
[ 89.918010] => apic_timer_interrupt
[ 89.918011] => cpu_idle
[ 89.918013] => rest_init
[ 89.918014] => start_kernel
[ 89.918016] => x86_64_start_reservations
[ 89.918039] <idle>-0 0d... 89893110us : <stack trace>
[ 89.918041] => rcu_irq_enter
[ 89.918043] => irq_enter
[ 89.918045] => smp_apic_timer_interrupt
[ 89.918046] => apic_timer_interrupt
[ 89.918048] => cpu_idle
[ 89.918050] => rest_init
[ 89.918051] => start_kernel
[ 89.918053] => x86_64_start_reservations
[ 89.918075] <idle>-0 0dN.. 89893147us : <stack trace>
[ 89.918078] => rcu_irq_exit
[ 89.918080] => irq_exit
[ 89.918081] => smp_apic_timer_interrupt
[ 89.918083] => apic_timer_interrupt
[ 89.918085] => cpu_idle
[ 89.918086] => rest_init
[ 89.918088] => start_kernel
[ 89.918089] => x86_64_start_reservations
[ 89.918112] <idle>-0 0d... 89897103us : <stack trace>
[ 89.918115] => rcu_irq_enter
[ 89.918116] => irq_enter
[ 89.918118] => smp_apic_timer_interrupt
[ 89.918120] => apic_timer_interrupt
[ 89.918121] => cpu_idle
[ 89.918123] => rest_init
[ 89.918124] => start_kernel
[ 89.918126] => x86_64_start_reservations
[ 89.918149] <idle>-0 0dN.. 89897136us : <stack trace>
[ 89.918151] => rcu_irq_exit
[ 89.918153] => irq_exit
[ 89.918155] => smp_apic_timer_interrupt
[ 89.918157] => apic_timer_interrupt
[ 89.918158] => cpu_idle
[ 89.918160] => rest_init
[ 89.918161] => start_kernel
[ 89.918163] => x86_64_start_reservations
[ 89.918186] <idle>-0 0d... 89901096us : <stack trace>
[ 89.918188] => rcu_irq_enter
[ 89.918190] => irq_enter
[ 89.918192] => smp_apic_timer_interrupt
[ 89.918193] => apic_timer_interrupt
[ 89.918195] => cpu_idle
[ 89.918197] => rest_init
[ 89.918198] => start_kernel
[ 89.918200] => x86_64_start_reservations
[ 89.918223] <idle>-0 0dN.. 89901133us : <stack trace>
[ 89.918225] => rcu_irq_exit
[ 89.918227] => irq_exit
[ 89.918228] => smp_apic_timer_interrupt
[ 89.918230] => apic_timer_interrupt
[ 89.918232] => cpu_idle
[ 89.918234] => rest_init
[ 89.918235] => start_kernel
[ 89.918237] => x86_64_start_reservations
[ 89.918260] <idle>-0 0d... 89905089us : <stack trace>
[ 89.918262] => rcu_irq_enter
[ 89.918264] => irq_enter
[ 89.918265] => smp_apic_timer_interrupt
[ 89.918267] => apic_timer_interrupt
[ 89.918269] => cpu_idle
[ 89.918270] => rest_init
[ 89.918272] => start_kernel
[ 89.918273] => x86_64_start_reservations
[ 89.918296] <idle>-0 0dN.. 89905128us : <stack trace>
[ 89.918299] => rcu_irq_exit
[ 89.918300] => irq_exit
[ 89.918302] => smp_apic_timer_interrupt
[ 89.918304] => apic_timer_interrupt
[ 89.918305] => cpu_idle
[ 89.918307] => rest_init
[ 89.918308] => start_kernel
[ 89.918310] => x86_64_start_reservations
[ 89.918333] <idle>-0 0d... 89909082us : <stack trace>
[ 89.918336] => rcu_irq_enter
[ 89.918337] => irq_enter
[ 89.918339] => smp_apic_timer_interrupt
[ 89.918341] => apic_timer_interrupt
[ 89.918342] => cpu_idle
[ 89.918344] => rest_init
[ 89.918345] => start_kernel
[ 89.918347] => x86_64_start_reservations
[ 89.918370] <idle>-0 0dN.. 89909122us : <stack trace>
[ 89.918372] => rcu_irq_exit
[ 89.918374] => irq_exit
[ 89.918376] => smp_apic_timer_interrupt
[ 89.918377] => apic_timer_interrupt
[ 89.918379] => cpu_idle
[ 89.918381] => rest_init
[ 89.918382] => start_kernel
[ 89.918384] => x86_64_start_reservations
[ 89.918408] <idle>-0 0d... 89912671us : <stack trace>
[ 89.918410] => rcu_irq_enter
[ 89.918412] => irq_enter
[ 89.918413] => do_IRQ
[ 89.918415] => ret_from_intr
[ 89.918416] => cpu_idle
[ 89.918418] => rest_init
[ 89.918419] => start_kernel
[ 89.918421] => x86_64_start_reservations
[ 89.918443] <idle>-0 0d... 89912711us : <stack trace>
[ 89.918446] => rcu_irq_exit
[ 89.918448] => irq_exit
[ 89.918449] => do_IRQ
[ 89.918451] => ret_from_intr
[ 89.918452] => cpu_idle
[ 89.918454] => rest_init
[ 89.918455] => start_kernel
[ 89.918457] => x86_64_start_reservations
[ 89.918480] <idle>-0 0d... 89913074us : <stack trace>
[ 89.918482] => rcu_irq_enter
[ 89.918484] => irq_enter
[ 89.918486] => smp_apic_timer_interrupt
[ 89.918488] => apic_timer_interrupt
[ 89.918489] => cpu_idle
[ 89.918491] => rest_init
[ 89.918492] => start_kernel
[ 89.918494] => x86_64_start_reservations
[ 89.918517] <idle>-0 0dN.. 89913130us : <stack trace>
[ 89.918519] => rcu_irq_exit
[ 89.918521] => irq_exit
[ 89.918522] => smp_apic_timer_interrupt
[ 89.918524] => apic_timer_interrupt
[ 89.918526] => cpu_idle
[ 89.918527] => rest_init
[ 89.918529] => start_kernel
[ 89.918530] => x86_64_start_reservations
[ 89.918552] <idle>-0 0d... 89913196us : <stack trace>
[ 89.918555] => rcu_enter_nohz
[ 89.918556] => tick_nohz_stop_sched_tick
[ 89.918558] => cpu_idle
[ 89.918560] => rest_init
[ 89.918561] => start_kernel
[ 89.918563] => x86_64_start_reservations
[ 89.918565] => x86_64_start_kernel
[ 89.918569] ---------------------------------
also have
[ 1.200814] ftrace: allocating 51307 entries in 202 pages
[ 1.226119] ------------[ cut here ]------------
[ 1.227489] WARNING: at kernel/trace/ftrace.c:984 ftrace_bug+0x79/0x189()
[ 1.237463] Hardware name: Sun Fire X4800 M2
[ 1.238845] Modules linked in:
[ 1.240113] Pid: 0, comm: swapper Not tainted 2.6.39-rc7-tip-yh-05281-g8e9254b-dirty #1019
[ 1.260717] Call Trace:
[ 1.261389] [<ffffffff81080144>] warn_slowpath_common+0x85/0x9d
[ 1.278249] [<ffffffff814d5d97>] ? sci_base_state_machine_get_state+0x5/0xe
[ 1.280207] [<ffffffff814d5d97>] ? sci_base_state_machine_get_state+0x5/0xe
[ 1.300928] [<ffffffff81080176>] warn_slowpath_null+0x1a/0x1c
[ 1.317318] [<ffffffff810d9457>] ftrace_bug+0x79/0x189
[ 1.318672] [<ffffffff814d5d97>] ? sci_base_state_machine_get_state+0x5/0xe
[ 1.339872] [<ffffffff810d9699>] ftrace_update_code+0x75/0x118
[ 1.341795] [<ffffffff81bda3aa>] ? csum_partial_copy_from_user+0x11/0xe7
[ 1.357948] [<ffffffff810d9873>] ftrace_process_locs+0x69/0x80
[ 1.359587] [<ffffffff8275fd74>] ftrace_init+0x191/0x1b5
[ 1.380143] [<ffffffff82742e34>] start_kernel+0x3ad/0x3bd
[ 1.381479] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
[ 1.397886] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
[ 1.399577] ---[ end trace a7919e7f17c0a725 ]---
[ 1.420373] ftrace failed to modify [<ffffffff814d5d97>] sci_base_state_machine_get_state+0x5/0xe
[ 1.437443] actual: a5:f3:75:00:8b
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-19 4:33 ` Yinghai Lu
@ 2011-05-19 14:47 ` Frederic Weisbecker
2011-05-19 19:51 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-19 14:47 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On Wed, May 18, 2011 at 09:33:51PM -0700, Yinghai Lu wrote:
> ...
> [ 89.913205] ------------[ cut here ]------------
> [ 89.913216] WARNING: at kernel/rcutree.c:352 rcu_enter_nohz+0x49/0x8b()
> [ 89.913220] Hardware name: Sun Fire X4800 M2
> [ 89.913224] Modules linked in:
> [ 89.913230] Switched to NOHz mode on CPU #34
> [ 89.913237] Pid: 0, comm: swapper Tainted: G W 2.6.39-rc7-tip-yh-05281-g8e9254b-dirty #1019
> [ 89.913242] Call Trace:
> [ 89.913269] [<ffffffff81080144>] warn_slowpath_common+0x85/0x9d
> [ 89.913280] [<ffffffff81080176>] warn_slowpath_null+0x1a/0x1c
> [ 89.913290] [<ffffffff810d32cc>] rcu_enter_nohz+0x49/0x8b
> [ 89.913305] [<ffffffff810ab121>] tick_nohz_stop_sched_tick+0x27d/0x366
> [ 89.913317] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
> [ 89.913329] [<ffffffff81bda6e3>] rest_init+0xb7/0xbe
> [ 89.913340] [<ffffffff81bda62c>] ? csum_partial_copy_generic+0x16c/0x16c
> [ 89.913352] [<ffffffff82742e39>] start_kernel+0x3b2/0x3bd
> [ 89.913363] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
> [ 89.913374] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
> [ 89.913379] ---[ end trace a7919e7f17c0a726 ]---
> [ 89.913401] Dumping ftrace buffer:
> [ 89.913417] Switched to NOHz mode on CPU #4
> [ 89.913428] ---------------------------------
> [ 89.913444] Switched to NOHz mode on CPU #54
> [ 89.913467] CPU:0 [LOST 39120 EVENTS]
<snip>
> [ 89.918480] <idle>-0 0d... 89913074us : <stack trace>
> [ 89.918482] => rcu_irq_enter
> [ 89.918484] => irq_enter
> [ 89.918486] => smp_apic_timer_interrupt
> [ 89.918488] => apic_timer_interrupt
> [ 89.918489] => cpu_idle
> [ 89.918491] => rest_init
> [ 89.918492] => start_kernel
> [ 89.918494] => x86_64_start_reservations
> [ 89.918517] <idle>-0 0dN.. 89913130us : <stack trace>
> [ 89.918519] => rcu_irq_exit
> [ 89.918521] => irq_exit
> [ 89.918522] => smp_apic_timer_interrupt
> [ 89.918524] => apic_timer_interrupt
> [ 89.918526] => cpu_idle
> [ 89.918527] => rest_init
> [ 89.918529] => start_kernel
> [ 89.918530] => x86_64_start_reservations
> [ 89.918552] <idle>-0 0d... 89913196us : <stack trace>
> [ 89.918555] => rcu_enter_nohz
> [ 89.918556] => tick_nohz_stop_sched_tick
> [ 89.918558] => cpu_idle
> [ 89.918560] => rest_init
> [ 89.918561] => start_kernel
> [ 89.918563] => x86_64_start_reservations
> [ 89.918565] => x86_64_start_kernel
> [ 89.918569] ---------------------------------
So, the warning triggered there. But the pairing looks actually good,
given we had an interrupt right before that and it has called
irq_enter and irq_exit.
I have no clue about what happened there.
It may be easier if we have the value of dynticks and dynticks_nesting.
Can you please test the following branch?
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
rcu/debug
It's Paul's branch + the early_initcall fix + the following patch:
---
commit 14667f347dea62d9e5a08e0f614840e50f73002c
Author: Frederic Weisbecker <fweisbec@gmail.com>
Date: Thu May 19 16:25:19 2011 +0200
rcu: Trace dynticks internal values
To debug some nasty count bug.
Not-signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index f30aea3..49d6348 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -349,6 +349,7 @@ void rcu_enter_nohz(void)
local_irq_save(flags);
trace_dump_stack();
rdtp = &__get_cpu_var(rcu_dynticks);
+ trace_printk("%x %x\n", atomic_read(&rdtp->dynticks), rdtp->dynticks_nesting);
if (WARN_ON_ONCE(rdtp->dynticks_nesting != 1))
ftrace_dump(DUMP_ORIG);
rdtp->dynticks_nesting = 1;
@@ -388,6 +389,7 @@ void rcu_exit_nohz(void)
local_irq_save(flags);
trace_dump_stack();
rdtp = &__get_cpu_var(rcu_dynticks);
+ trace_printk("%x %x\n", atomic_read(&rdtp->dynticks), rdtp->dynticks_nesting);
if (WARN_ON_ONCE(rdtp->dynticks_nesting != 0))
ftrace_dump(DUMP_ORIG);
rdtp->dynticks_nesting = 0;
@@ -463,9 +465,12 @@ void rcu_nmi_exit(void)
void rcu_irq_enter(void)
{
unsigned long flags;
+ struct rcu_dynticks *rdtp;
local_irq_save(flags);
trace_dump_stack();
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ trace_printk("%x %x\n", atomic_read(&rdtp->dynticks), rdtp->dynticks_nesting);
__rcu_exit_nohz();
local_irq_restore(flags);
}
@@ -480,9 +485,12 @@ void rcu_irq_enter(void)
void rcu_irq_exit(void)
{
unsigned long flags;
+ struct rcu_dynticks *rdtp;
local_irq_save(flags);
trace_dump_stack();
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ trace_printk("%x %x\n", atomic_read(&rdtp->dynticks), rdtp->dynticks_nesting);
__rcu_enter_nohz();
local_irq_restore(flags);
}
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-19 14:47 ` Frederic Weisbecker
@ 2011-05-19 19:51 ` Yinghai Lu
2011-05-19 21:15 ` Frederic Weisbecker
0 siblings, 1 reply; 76+ messages in thread
From: Yinghai Lu @ 2011-05-19 19:51 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On 05/19/2011 07:47 AM, Frederic Weisbecker wrote:
> Can you please test the following branch?
>
> git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
> rcu/debug
>
> It's Paul's branch + the early_initcall fix + the following patch:
>
> ---
> commit 14667f347dea62d9e5a08e0f614840e50f73002c
> Author: Frederic Weisbecker <fweisbec@gmail.com>
> Date: Thu May 19 16:25:19 2011 +0200
>
> rcu: Trace dynticks internal values
>
> To debug some nasty count bug.
>
> Not-signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
[ 85.194251] ------------[ cut here ]------------
[ 85.194260] WARNING: at kernel/rcutree.c:353 rcu_enter_nohz+0x63/0xa5()
[ 85.194262] Hardware name: Sun Fire X4800 M2
[ 85.194264] Modules linked in:
[ 85.194268] Pid: 0, comm: swapper Tainted: G W 2.6.39-tip-yh-05046-g8a9149b-dirty #1021
[ 85.194270] Call Trace:
[ 85.194291] [<ffffffff810800f4>] warn_slowpath_common+0x85/0x9d
[ 85.194296] [<ffffffff81080126>] warn_slowpath_null+0x1a/0x1c
[ 85.194301] [<ffffffff810d3326>] rcu_enter_nohz+0x63/0xa5
[ 85.194311] [<ffffffff810ab161>] tick_nohz_stop_sched_tick+0x27d/0x366
[ 85.194317] [<ffffffff810391bc>] cpu_idle+0x7a/0xcc
[ 85.194326] [<ffffffff81bda823>] rest_init+0xb7/0xbe
[ 85.194331] [<ffffffff81bda76c>] ? csum_partial_copy_generic+0x16c/0x16c
[ 85.194338] [<ffffffff82742e39>] start_kernel+0x3b2/0x3bd
[ 85.194344] [<ffffffff827422cc>] x86_64_start_reservations+0x9c/0xa0
[ 85.194348] [<ffffffff827424a8>] x86_64_start_kernel+0x1d8/0x1e3
[ 85.194351] ---[ end trace a7919e7f17c0a726 ]---
[ 85.194366] Dumping ftrace buffer:
[ 85.194391] ---------------------------------
[ 85.194415] CPU:0 [LOST 74512 EVENTS]
[ 85.194416] <idle>-0 0d... 85121004us : <stack trace>
[ 85.194418] => rcu_irq_enter
[ 85.194418] => irq_enter
[ 85.194419] => smp_apic_timer_interrupt
[ 85.194420] => apic_timer_interrupt
[ 85.194421] => cpu_idle
[ 85.194421] => rest_init
[ 85.194422] => start_kernel
[ 85.194423] => x86_64_start_reservations
[ 85.194430] <idle>-0 0d... 85121009us : rcu_irq_enter: 1 49
[ 85.194441] <idle>-0 0dN.. 85121429us : <stack trace>
[ 85.194443] => rcu_irq_exit
[ 85.194443] => irq_exit
[ 85.194444] => smp_apic_timer_interrupt
[ 85.194445] => apic_timer_interrupt
[ 85.194446] => cpu_idle
[ 85.194447] => rest_init
[ 85.194447] => start_kernel
[ 85.194448] => x86_64_start_reservations
[ 85.194454] <idle>-0 0dN.. 85121433us : rcu_irq_exit: 1 4a
[ 85.194465] <idle>-0 0d... 85124995us : <stack trace>
[ 85.194466] => rcu_irq_enter
[ 85.194467] => irq_enter
[ 85.194468] => smp_apic_timer_interrupt
[ 85.194469] => apic_timer_interrupt
[ 85.194470] => cpu_idle
[ 85.194470] => rest_init
[ 85.194471] => start_kernel
[ 85.194472] => x86_64_start_reservations
[ 85.194477] <idle>-0 0d... 85125007us : rcu_irq_enter: 1 49
[ 85.194488] <idle>-0 0dN.. 85125366us : <stack trace>
[ 85.194489] => rcu_irq_exit
[ 85.194490] => irq_exit
[ 85.194491] => smp_apic_timer_interrupt
[ 85.194492] => apic_timer_interrupt
[ 85.194492] => cpu_idle
[ 85.194493] => rest_init
[ 85.194494] => start_kernel
[ 85.194494] => x86_64_start_reservations
[ 85.194500] <idle>-0 0dN.. 85125371us : rcu_irq_exit: 1 4a
[ 85.194511] <idle>-0 0d... 85128990us : <stack trace>
[ 85.194512] => rcu_irq_enter
[ 85.194513] => irq_enter
[ 85.194514] => smp_apic_timer_interrupt
[ 85.194514] => apic_timer_interrupt
[ 85.194515] => cpu_idle
[ 85.194516] => rest_init
[ 85.194517] => start_kernel
[ 85.194517] => x86_64_start_reservations
[ 85.194523] <idle>-0 0d... 85129001us : rcu_irq_enter: 1 49
[ 85.194534] <idle>-0 0dN.. 85129033us : <stack trace>
[ 85.194535] => rcu_irq_exit
[ 85.194536] => irq_exit
[ 85.194536] => smp_apic_timer_interrupt
[ 85.194537] => apic_timer_interrupt
[ 85.194538] => cpu_idle
[ 85.194539] => rest_init
[ 85.194539] => start_kernel
[ 85.194540] => x86_64_start_reservations
[ 85.194546] <idle>-0 0dN.. 85129043us : rcu_irq_exit: 1 4a
[ 85.194556] <idle>-0 0d... 85132981us : <stack trace>
[ 85.194558] => rcu_irq_enter
[ 85.194558] => irq_enter
[ 85.194559] => smp_apic_timer_interrupt
[ 85.194560] => apic_timer_interrupt
[ 85.194561] => cpu_idle
[ 85.194561] => rest_init
[ 85.194562] => start_kernel
[ 85.194563] => x86_64_start_reservations
[ 85.194568] <idle>-0 0d... 85132986us : rcu_irq_enter: 1 49
[ 85.194579] <idle>-0 0dN.. 85133066us : <stack trace>
[ 85.194580] => rcu_irq_exit
[ 85.194581] => irq_exit
[ 85.194582] => smp_apic_timer_interrupt
[ 85.194583] => apic_timer_interrupt
[ 85.194583] => cpu_idle
[ 85.194584] => rest_init
[ 85.194585] => start_kernel
[ 85.194586] => x86_64_start_reservations
[ 85.194592] <idle>-0 0dN.. 85133077us : rcu_irq_exit: 1 4a
[ 85.194602] <idle>-0 0d... 85136976us : <stack trace>
[ 85.194604] => rcu_irq_enter
[ 85.194604] => irq_enter
[ 85.194605] => smp_apic_timer_interrupt
[ 85.194606] => apic_timer_interrupt
[ 85.194607] => cpu_idle
[ 85.194608] => rest_init
[ 85.194608] => start_kernel
[ 85.194609] => x86_64_start_reservations
[ 85.194615] <idle>-0 0d... 85136987us : rcu_irq_enter: 1 49
[ 85.194626] <idle>-0 0dN.. 85137062us : <stack trace>
[ 85.194627] => rcu_irq_exit
[ 85.194628] => irq_exit
[ 85.194628] => smp_apic_timer_interrupt
[ 85.194629] => apic_timer_interrupt
[ 85.194630] => cpu_idle
[ 85.194631] => rest_init
[ 85.194631] => start_kernel
[ 85.194632] => x86_64_start_reservations
[ 85.194638] <idle>-0 0dN.. 85137072us : rcu_irq_exit: 1 4a
[ 85.194649] <idle>-0 0d... 85140967us : <stack trace>
[ 85.194650] => rcu_irq_enter
[ 85.194651] => irq_enter
[ 85.194651] => smp_apic_timer_interrupt
[ 85.194652] => apic_timer_interrupt
[ 85.194653] => cpu_idle
[ 85.194653] => rest_init
[ 85.194654] => start_kernel
[ 85.194655] => x86_64_start_reservations
[ 85.194661] <idle>-0 0d... 85140974us : rcu_irq_enter: 1 49
[ 85.194671] <idle>-0 0dN.. 85141463us : <stack trace>
[ 85.194673] => rcu_irq_exit
[ 85.194673] => irq_exit
[ 85.194674] => smp_apic_timer_interrupt
[ 85.194675] => apic_timer_interrupt
[ 85.194676] => cpu_idle
[ 85.194676] => rest_init
[ 85.194677] => start_kernel
[ 85.194678] => x86_64_start_reservations
[ 85.194683] <idle>-0 0dN.. 85141467us : rcu_irq_exit: 1 4a
[ 85.194694] <idle>-0 0d... 85144960us : <stack trace>
[ 85.194695] => rcu_irq_enter
[ 85.194696] => irq_enter
[ 85.194697] => smp_apic_timer_interrupt
[ 85.194698] => apic_timer_interrupt
[ 85.194698] => cpu_idle
[ 85.194699] => rest_init
[ 85.194700] => start_kernel
[ 85.194700] => x86_64_start_reservations
[ 85.194706] <idle>-0 0d... 85144967us : rcu_irq_enter: 1 49
[ 85.194717] <idle>-0 0dN.. 85145234us : <stack trace>
[ 85.194718] => rcu_irq_exit
[ 85.194719] => irq_exit
[ 85.194720] => smp_apic_timer_interrupt
[ 85.194720] => apic_timer_interrupt
[ 85.194721] => cpu_idle
[ 85.194722] => rest_init
[ 85.194722] => start_kernel
[ 85.194723] => x86_64_start_reservations
[ 85.194729] <idle>-0 0dN.. 85145238us : rcu_irq_exit: 1 4a
[ 85.194740] <idle>-0 0d... 85148953us : <stack trace>
[ 85.194741] => rcu_irq_enter
[ 85.194742] => irq_enter
[ 85.194742] => smp_apic_timer_interrupt
[ 85.194743] => apic_timer_interrupt
[ 85.194744] => cpu_idle
[ 85.194744] => rest_init
[ 85.194745] => start_kernel
[ 85.194746] => x86_64_start_reservations
[ 85.194752] <idle>-0 0d... 85148958us : rcu_irq_enter: 1 49
[ 85.194762] <idle>-0 0dN.. 85149096us : <stack trace>
[ 85.194764] => rcu_irq_exit
[ 85.194764] => irq_exit
[ 85.194765] => smp_apic_timer_interrupt
[ 85.194766] => apic_timer_interrupt
[ 85.194767] => cpu_idle
[ 85.194767] => rest_init
[ 85.194768] => start_kernel
[ 85.194769] => x86_64_start_reservations
[ 85.194774] <idle>-0 0dN.. 85149101us : rcu_irq_exit: 1 4a
[ 85.194785] <idle>-0 0d... 85152946us : <stack trace>
[ 85.194786] => rcu_irq_enter
[ 85.194787] => irq_enter
[ 85.194788] => smp_apic_timer_interrupt
[ 85.194789] => apic_timer_interrupt
[ 85.194789] => cpu_idle
[ 85.194790] => rest_init
[ 85.194791] => start_kernel
[ 85.194791] => x86_64_start_reservations
[ 85.194797] <idle>-0 0d... 85152953us : rcu_irq_enter: 1 49
[ 85.194808] <idle>-0 0dN.. 85153081us : <stack trace>
[ 85.194809] => rcu_irq_exit
[ 85.194810] => irq_exit
[ 85.194811] => smp_apic_timer_interrupt
[ 85.194812] => apic_timer_interrupt
[ 85.194813] => cpu_idle
[ 85.194813] => rest_init
[ 85.194814] => start_kernel
[ 85.194815] => x86_64_start_reservations
[ 85.194821] <idle>-0 0dN.. 85153085us : rcu_irq_exit: 1 4a
[ 85.194832] <idle>-0 0d... 85156939us : <stack trace>
[ 85.194833] => rcu_irq_enter
[ 85.194834] => irq_enter
[ 85.194834] => smp_apic_timer_interrupt
[ 85.194835] => apic_timer_interrupt
[ 85.194836] => cpu_idle
[ 85.194837] => rest_init
[ 85.194837] => start_kernel
[ 85.194838] => x86_64_start_reservations
[ 85.194844] <idle>-0 0d... 85156948us : rcu_irq_enter: 1 49
[ 85.194854] <idle>-0 0dN.. 85157535us : <stack trace>
[ 85.194856] => rcu_irq_exit
[ 85.194856] => irq_exit
[ 85.194857] => smp_apic_timer_interrupt
[ 85.194858] => apic_timer_interrupt
[ 85.194859] => cpu_idle
[ 85.194859] => rest_init
[ 85.194860] => start_kernel
[ 85.194861] => x86_64_start_reservations
[ 85.194866] <idle>-0 0dN.. 85157540us : rcu_irq_exit: 1 4a
[ 85.194877] <idle>-0 0d... 85160932us : <stack trace>
[ 85.194878] => rcu_irq_enter
[ 85.194879] => irq_enter
[ 85.194880] => smp_apic_timer_interrupt
[ 85.194881] => apic_timer_interrupt
[ 85.194881] => cpu_idle
[ 85.194882] => rest_init
[ 85.194883] => start_kernel
[ 85.194883] => x86_64_start_reservations
[ 85.194889] <idle>-0 0d... 85160941us : rcu_irq_enter: 1 49
[ 85.194900] <idle>-0 0dN.. 85161127us : <stack trace>
[ 85.194901] => rcu_irq_exit
[ 85.194902] => irq_exit
[ 85.194903] => smp_apic_timer_interrupt
[ 85.194903] => apic_timer_interrupt
[ 85.194904] => cpu_idle
[ 85.194905] => rest_init
[ 85.194905] => start_kernel
[ 85.194906] => x86_64_start_reservations
[ 85.194912] <idle>-0 0dN.. 85161132us : rcu_irq_exit: 1 4a
[ 85.194923] <idle>-0 0d... 85164925us : <stack trace>
[ 85.194924] => rcu_irq_enter
[ 85.194925] => irq_enter
[ 85.194925] => smp_apic_timer_interrupt
[ 85.194926] => apic_timer_interrupt
[ 85.194927] => cpu_idle
[ 85.194928] => rest_init
[ 85.194928] => start_kernel
[ 85.194929] => x86_64_start_reservations
[ 85.194935] <idle>-0 0d... 85164930us : rcu_irq_enter: 1 49
[ 85.194945] <idle>-0 0dN.. 85165009us : <stack trace>
[ 85.194947] => rcu_irq_exit
[ 85.194947] => irq_exit
[ 85.194948] => smp_apic_timer_interrupt
[ 85.194949] => apic_timer_interrupt
[ 85.194950] => cpu_idle
[ 85.194950] => rest_init
[ 85.194951] => start_kernel
[ 85.194952] => x86_64_start_reservations
[ 85.194957] <idle>-0 0dN.. 85165013us : rcu_irq_exit: 1 4a
[ 85.194968] <idle>-0 0d... 85168920us : <stack trace>
[ 85.194969] => rcu_irq_enter
[ 85.194970] => irq_enter
[ 85.194971] => smp_apic_timer_interrupt
[ 85.194972] => apic_timer_interrupt
[ 85.194972] => cpu_idle
[ 85.194973] => rest_init
[ 85.194974] => start_kernel
[ 85.194974] => x86_64_start_reservations
[ 85.194980] <idle>-0 0d... 85168931us : rcu_irq_enter: 1 49
[ 85.194991] <idle>-0 0dN.. 85168970us : <stack trace>
[ 85.194992] => rcu_irq_exit
[ 85.194993] => irq_exit
[ 85.194993] => smp_apic_timer_interrupt
[ 85.194994] => apic_timer_interrupt
[ 85.194995] => cpu_idle
[ 85.194996] => rest_init
[ 85.194996] => start_kernel
[ 85.194997] => x86_64_start_reservations
[ 85.195003] <idle>-0 0dN.. 85168980us : rcu_irq_exit: 1 4a
[ 85.195014] <idle>-0 0d... 85172911us : <stack trace>
[ 85.195015] => rcu_irq_enter
[ 85.195016] => irq_enter
[ 85.195016] => smp_apic_timer_interrupt
[ 85.195017] => apic_timer_interrupt
[ 85.195018] => cpu_idle
[ 85.195019] => rest_init
[ 85.195020] => start_kernel
[ 85.195020] => x86_64_start_reservations
[ 85.195026] <idle>-0 0d... 85172916us : rcu_irq_enter: 1 49
[ 85.195037] <idle>-0 0dN.. 85173155us : <stack trace>
[ 85.195038] => rcu_irq_exit
[ 85.195039] => irq_exit
[ 85.195040] => smp_apic_timer_interrupt
[ 85.195041] => apic_timer_interrupt
[ 85.195042] => cpu_idle
[ 85.195042] => rest_init
[ 85.195043] => start_kernel
[ 85.195044] => x86_64_start_reservations
[ 85.195050] <idle>-0 0dN.. 85173160us : rcu_irq_exit: 1 4a
[ 85.195060] <idle>-0 0d... 85176904us : <stack trace>
[ 85.195061] => rcu_irq_enter
[ 85.195062] => irq_enter
[ 85.195063] => smp_apic_timer_interrupt
[ 85.195064] => apic_timer_interrupt
[ 85.195064] => cpu_idle
[ 85.195065] => rest_init
[ 85.195066] => start_kernel
[ 85.195066] => x86_64_start_reservations
[ 85.195072] <idle>-0 0d... 85176913us : rcu_irq_enter: 1 49
[ 85.195083] <idle>-0 0dN.. 85177864us : <stack trace>
[ 85.195084] => rcu_irq_exit
[ 85.195085] => irq_exit
[ 85.195086] => smp_apic_timer_interrupt
[ 85.195086] => apic_timer_interrupt
[ 85.195087] => cpu_idle
[ 85.195088] => rest_init
[ 85.195088] => start_kernel
[ 85.195089] => x86_64_start_reservations
[ 85.195095] <idle>-0 0dN.. 85177869us : rcu_irq_exit: 1 4a
[ 85.195106] <idle>-0 0d... 85180897us : <stack trace>
[ 85.195107] => rcu_irq_enter
[ 85.195108] => irq_enter
[ 85.195108] => smp_apic_timer_interrupt
[ 85.195109] => apic_timer_interrupt
[ 85.195110] => cpu_idle
[ 85.195111] => rest_init
[ 85.195111] => start_kernel
[ 85.195112] => x86_64_start_reservations
[ 85.195118] <idle>-0 0d... 85180907us : rcu_irq_enter: 1 49
[ 85.195129] <idle>-0 0dN.. 85181665us : <stack trace>
[ 85.195130] => rcu_irq_exit
[ 85.195130] => irq_exit
[ 85.195131] => smp_apic_timer_interrupt
[ 85.195132] => apic_timer_interrupt
[ 85.195133] => cpu_idle
[ 85.195133] => rest_init
[ 85.195134] => start_kernel
[ 85.195135] => x86_64_start_reservations
[ 85.195141] <idle>-0 0dN.. 85181669us : rcu_irq_exit: 1 4a
[ 85.195152] <idle>-0 0d... 85184890us : <stack trace>
[ 85.195153] => rcu_irq_enter
[ 85.195154] => irq_enter
[ 85.195154] => smp_apic_timer_interrupt
[ 85.195155] => apic_timer_interrupt
[ 85.195156] => cpu_idle
[ 85.195157] => rest_init
[ 85.195157] => start_kernel
[ 85.195158] => x86_64_start_reservations
[ 85.195164] <idle>-0 0d... 85184895us : rcu_irq_enter: 1 49
[ 85.195175] <idle>-0 0dN.. 85185146us : <stack trace>
[ 85.195176] => rcu_irq_exit
[ 85.195177] => irq_exit
[ 85.195177] => smp_apic_timer_interrupt
[ 85.195178] => apic_timer_interrupt
[ 85.195179] => cpu_idle
[ 85.195179] => rest_init
[ 85.195180] => start_kernel
[ 85.195181] => x86_64_start_reservations
[ 85.195187] <idle>-0 0dN.. 85185151us : rcu_irq_exit: 1 4a
[ 85.195203] <...>-6 0d... 85188881us : <stack trace>
[ 85.195204] => rcu_irq_enter
[ 85.195205] => irq_enter
[ 85.195206] => smp_apic_timer_interrupt
[ 85.195206] => apic_timer_interrupt
[ 85.195207] => cpu_stopper_thread
[ 85.195208] => kthread
[ 85.195209] => kernel_thread_helper
[ 85.195215] <...>-6 0d... 85188894us : rcu_irq_enter: 1 49
[ 85.195225] <...>-6 0d... 85188982us : <stack trace>
[ 85.195226] => rcu_irq_exit
[ 85.195227] => irq_exit
[ 85.195228] => smp_apic_timer_interrupt
[ 85.195229] => apic_timer_interrupt
[ 85.195230] => cpu_stopper_thread
[ 85.195231] => kthread
[ 85.195231] => kernel_thread_helper
[ 85.195237] <...>-6 0d... 85188989us : rcu_irq_exit: 1 4a
[ 85.195249] <idle>-0 0d... 85190288us : <stack trace>
[ 85.195250] => rcu_irq_enter
[ 85.195251] => irq_enter
[ 85.195251] => do_IRQ
[ 85.195252] => ret_from_intr
[ 85.195253] => cpu_idle
[ 85.195253] => rest_init
[ 85.195254] => start_kernel
[ 85.195255] => x86_64_start_reservations
[ 85.195261] <idle>-0 0d... 85190299us : rcu_irq_enter: 1 49
[ 85.195271] <idle>-0 0d... 85190336us : <stack trace>
[ 85.195272] => rcu_irq_exit
[ 85.195273] => irq_exit
[ 85.195274] => do_IRQ
[ 85.195274] => ret_from_intr
[ 85.195275] => cpu_idle
[ 85.195276] => rest_init
[ 85.195276] => start_kernel
[ 85.195277] => x86_64_start_reservations
[ 85.195283] <idle>-0 0d... 85190346us : rcu_irq_exit: 1 4a
[ 85.195293] <idle>-0 0d... 85192902us : <stack trace>
[ 85.195295] => rcu_irq_enter
[ 85.195295] => irq_enter
[ 85.195296] => smp_apic_timer_interrupt
[ 85.195297] => apic_timer_interrupt
[ 85.195298] => cpu_idle
[ 85.195298] => rest_init
[ 85.195299] => start_kernel
[ 85.195300] => x86_64_start_reservations
[ 85.195305] <idle>-0 0d... 85193714us : rcu_irq_enter: 1 49
[ 85.195316] <idle>-0 0dN.. 85193856us : <stack trace>
[ 85.195317] => rcu_irq_exit
[ 85.195318] => irq_exit
[ 85.195319] => smp_apic_timer_interrupt
[ 85.195320] => apic_timer_interrupt
[ 85.195320] => cpu_idle
[ 85.195321] => rest_init
[ 85.195322] => start_kernel
[ 85.195322] => x86_64_start_reservations
[ 85.195328] <idle>-0 0dN.. 85194210us : rcu_irq_exit: 1 4a
[ 85.195338] <idle>-0 0d... 85194247us : <stack trace>
[ 85.195340] => rcu_enter_nohz
[ 85.195340] => tick_nohz_stop_sched_tick
[ 85.195341] => cpu_idle
[ 85.195342] => rest_init
[ 85.195342] => start_kernel
[ 85.195343] => x86_64_start_reservations
[ 85.195344] => x86_64_start_kernel
[ 85.195350] <idle>-0 0d... 85194250us : rcu_enter_nohz: 1 49
[ 85.195352] ---------------------------------
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-19 19:51 ` Yinghai Lu
@ 2011-05-19 21:15 ` Frederic Weisbecker
2011-05-19 21:45 ` Yinghai Lu
0 siblings, 1 reply; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-19 21:15 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On Thu, May 19, 2011 at 12:51:26PM -0700, Yinghai Lu wrote:
> [ 85.194797] <idle>-0 0d... 85152953us : rcu_irq_enter: 1 49
> [ 85.194808] <idle>-0 0dN.. 85153081us : <stack trace>
> [ 85.194809] => rcu_irq_exit
> [ 85.194810] => irq_exit
> [ 85.194811] => smp_apic_timer_interrupt
> [ 85.194812] => apic_timer_interrupt
> [ 85.194813] => cpu_idle
> [ 85.194813] => rest_init
> [ 85.194814] => start_kernel
> [ 85.194815] => x86_64_start_reservations
So it's the dynticks_nesting going crazy. As if we had rcu_irq_enter()
without rcu_irq_exit().
I see you have CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
and the locking api selftest is doing a strange thing: if we look
at HARDIRQ_ENTER(), it calls irq_enter(), but HARDIRQ_EXIT() only
calls __irq_exit(). Which means it lacks the rcu_irq_exit().
So, if I understood correctly the thing there, some selftests simulating
the hardirq context are unbalancing the rcu state.
Does that help if you unset CONFIG_DEBUG_LOCKING_API_SELFTESTS ?
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-19 21:15 ` Frederic Weisbecker
@ 2011-05-19 21:45 ` Yinghai Lu
2011-05-20 0:09 ` [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests Frederic Weisbecker
2011-05-20 0:14 ` [GIT PULL rcu/next] rcu commits for 2.6.40 Frederic Weisbecker
0 siblings, 2 replies; 76+ messages in thread
From: Yinghai Lu @ 2011-05-19 21:45 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On 05/19/2011 02:15 PM, Frederic Weisbecker wrote:
> On Thu, May 19, 2011 at 12:51:26PM -0700, Yinghai Lu wrote:
>> [ 85.194797] <idle>-0 0d... 85152953us : rcu_irq_enter: 1 49
>> [ 85.194808] <idle>-0 0dN.. 85153081us : <stack trace>
>> [ 85.194809] => rcu_irq_exit
>> [ 85.194810] => irq_exit
>> [ 85.194811] => smp_apic_timer_interrupt
>> [ 85.194812] => apic_timer_interrupt
>> [ 85.194813] => cpu_idle
>> [ 85.194813] => rest_init
>> [ 85.194814] => start_kernel
>> [ 85.194815] => x86_64_start_reservations
>
> So it's the dynticks_nesting going crazy. As if we had rcu_irq_enter()
> without rcu_irq_exit().
>
> I see you have CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
> and the locking api selftest is doing a strange thing: if we look
> at HARDIRQ_ENTER(), it calls irq_enter(), but HARDIRQ_EXIT() only
> calls __irq_exit(). Which means it lacks the rcu_irq_exit().
>
> So, if I understood correctly the thing there, some selftests simulating
> the hardirq context are unbalancing the rcu state.
>
> Does that help if you unset CONFIG_DEBUG_LOCKING_API_SELFTESTS ?
yes, after unset that, no warning and delay...
Thanks
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests
2011-05-19 21:45 ` Yinghai Lu
@ 2011-05-20 0:09 ` Frederic Weisbecker
2011-05-20 8:36 ` Ingo Molnar
2011-05-20 15:11 ` Paul E. McKenney
2011-05-20 0:14 ` [GIT PULL rcu/next] rcu commits for 2.6.40 Frederic Weisbecker
1 sibling, 2 replies; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-20 0:09 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Paul E. McKenney, Ingo Molnar,
Peter Zijlstra, Stable
HARDIRQ_ENTER() maps to irq_enter() which calls rcu_irq_enter().
But HARDIRQ_EXIT() maps to __irq_exit() which doesn't call
rcu_irq_exit().
So for every locking selftest that simulates hardirq disabled,
we create an imbalance in the rcu extended quiescent state
internal state.
As a result, after the first missing rcu_irq_exit(), subsequent
irqs won't exit any extended quiescent state the time of the
interrupt servicing. Rcu read side critical section inside irqs
on that CPU won't be guaranteed anymore.
To fix this, just use __irq_enter() to simulate the hardirq
context. This is sufficient for the locking selftests as we
don't need to exit any extended quiescent state or perform
any check that irqs normally do when they wake up from idle.
As a side effect, this patch should make it possible to
restore
"rcu: Decrease memory-barrier usage based on semi-formal proof",
which eventually helped finding this bug.
Reported-and-tested-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stable <stable@kernel.org>
---
lib/locking-selftest.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c
index 619313e..507a22f 100644
--- a/lib/locking-selftest.c
+++ b/lib/locking-selftest.c
@@ -144,7 +144,7 @@ static void init_shared_classes(void)
#define HARDIRQ_ENTER() \
local_irq_disable(); \
- irq_enter(); \
+ __irq_enter(); \
WARN_ON(!in_irq());
#define HARDIRQ_EXIT() \
--
1.7.3.2
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [GIT PULL rcu/next] rcu commits for 2.6.40
2011-05-19 21:45 ` Yinghai Lu
2011-05-20 0:09 ` [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests Frederic Weisbecker
@ 2011-05-20 0:14 ` Frederic Weisbecker
1 sibling, 0 replies; 76+ messages in thread
From: Frederic Weisbecker @ 2011-05-20 0:14 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Paul E. McKenney, Ingo Molnar, linux-kernel, Steven Rostedt
On Thu, May 19, 2011 at 02:45:52PM -0700, Yinghai Lu wrote:
> On 05/19/2011 02:15 PM, Frederic Weisbecker wrote:
> > On Thu, May 19, 2011 at 12:51:26PM -0700, Yinghai Lu wrote:
> >> [ 85.194797] <idle>-0 0d... 85152953us : rcu_irq_enter: 1 49
> >> [ 85.194808] <idle>-0 0dN.. 85153081us : <stack trace>
> >> [ 85.194809] => rcu_irq_exit
> >> [ 85.194810] => irq_exit
> >> [ 85.194811] => smp_apic_timer_interrupt
> >> [ 85.194812] => apic_timer_interrupt
> >> [ 85.194813] => cpu_idle
> >> [ 85.194813] => rest_init
> >> [ 85.194814] => start_kernel
> >> [ 85.194815] => x86_64_start_reservations
> >
> > So it's the dynticks_nesting going crazy. As if we had rcu_irq_enter()
> > without rcu_irq_exit().
> >
> > I see you have CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
> > and the locking api selftest is doing a strange thing: if we look
> > at HARDIRQ_ENTER(), it calls irq_enter(), but HARDIRQ_EXIT() only
> > calls __irq_exit(). Which means it lacks the rcu_irq_exit().
> >
> > So, if I understood correctly the thing there, some selftests simulating
> > the hardirq context are unbalancing the rcu state.
> >
> > Does that help if you unset CONFIG_DEBUG_LOCKING_API_SELFTESTS ?
>
>
> yes, after unset that, no warning and delay...
Thanks a lot for all your testing! I've just proposed a fix, let's see
how that goes.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests
2011-05-20 0:09 ` [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests Frederic Weisbecker
@ 2011-05-20 8:36 ` Ingo Molnar
2011-05-20 15:12 ` Paul E. McKenney
2011-05-20 15:11 ` Paul E. McKenney
1 sibling, 1 reply; 76+ messages in thread
From: Ingo Molnar @ 2011-05-20 8:36 UTC (permalink / raw)
To: Frederic Weisbecker; +Cc: LKML, Paul E. McKenney, Peter Zijlstra, Stable
* Frederic Weisbecker <fweisbec@gmail.com> wrote:
> HARDIRQ_ENTER() maps to irq_enter() which calls rcu_irq_enter().
> But HARDIRQ_EXIT() maps to __irq_exit() which doesn't call
> rcu_irq_exit().
>
> So for every locking selftest that simulates hardirq disabled,
> we create an imbalance in the rcu extended quiescent state
> internal state.
>
> As a result, after the first missing rcu_irq_exit(), subsequent
> irqs won't exit any extended quiescent state the time of the
> interrupt servicing. Rcu read side critical section inside irqs
> on that CPU won't be guaranteed anymore.
>
> To fix this, just use __irq_enter() to simulate the hardirq
> context. This is sufficient for the locking selftests as we
> don't need to exit any extended quiescent state or perform
> any check that irqs normally do when they wake up from idle.
>
> As a side effect, this patch should make it possible to
> restore
> "rcu: Decrease memory-barrier usage based on semi-formal proof",
> which eventually helped finding this bug.
>
> Reported-and-tested-by: Yinghai Lu <yinghai@kernel.org>
> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Stable <stable@kernel.org>
> ---
> lib/locking-selftest.c | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c
> index 619313e..507a22f 100644
> --- a/lib/locking-selftest.c
> +++ b/lib/locking-selftest.c
> @@ -144,7 +144,7 @@ static void init_shared_classes(void)
>
> #define HARDIRQ_ENTER() \
> local_irq_disable(); \
> - irq_enter(); \
> + __irq_enter(); \
> WARN_ON(!in_irq());
>
> #define HARDIRQ_EXIT() \
Very nice!
Note that a cherry-picked version of the manual revert from Paul is upstream
now with the main body of RCU changes, so we can do the finegrained split-up
queue approach for this one problematic commit (that is no longer problematic).
I think it can still go in for v2.6.40. Paul, what do you think?
Thanks,
Ingo
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests
2011-05-20 0:09 ` [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests Frederic Weisbecker
2011-05-20 8:36 ` Ingo Molnar
@ 2011-05-20 15:11 ` Paul E. McKenney
1 sibling, 0 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-20 15:11 UTC (permalink / raw)
To: Frederic Weisbecker; +Cc: Ingo Molnar, LKML, Peter Zijlstra, Stable
On Fri, May 20, 2011 at 02:09:54AM +0200, Frederic Weisbecker wrote:
> HARDIRQ_ENTER() maps to irq_enter() which calls rcu_irq_enter().
> But HARDIRQ_EXIT() maps to __irq_exit() which doesn't call
> rcu_irq_exit().
>
> So for every locking selftest that simulates hardirq disabled,
> we create an imbalance in the rcu extended quiescent state
> internal state.
>
> As a result, after the first missing rcu_irq_exit(), subsequent
> irqs won't exit any extended quiescent state the time of the
> interrupt servicing. Rcu read side critical section inside irqs
> on that CPU won't be guaranteed anymore.
>
> To fix this, just use __irq_enter() to simulate the hardirq
> context. This is sufficient for the locking selftests as we
> don't need to exit any extended quiescent state or perform
> any check that irqs normally do when they wake up from idle.
>
> As a side effect, this patch should make it possible to
> restore
> "rcu: Decrease memory-barrier usage based on semi-formal proof",
> which eventually helped finding this bug.
>
> Reported-and-tested-by: Yinghai Lu <yinghai@kernel.org>
> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
You know, I looked at this code, but failed to see __irq_exit() vs
irq_enter()...
Good catch, Frederic, and thanks to Yinghai and Ingo for all the
testing!
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Stable <stable@kernel.org>
> ---
> lib/locking-selftest.c | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c
> index 619313e..507a22f 100644
> --- a/lib/locking-selftest.c
> +++ b/lib/locking-selftest.c
> @@ -144,7 +144,7 @@ static void init_shared_classes(void)
>
> #define HARDIRQ_ENTER() \
> local_irq_disable(); \
> - irq_enter(); \
> + __irq_enter(); \
> WARN_ON(!in_irq());
>
> #define HARDIRQ_EXIT() \
> --
> 1.7.3.2
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests
2011-05-20 8:36 ` Ingo Molnar
@ 2011-05-20 15:12 ` Paul E. McKenney
0 siblings, 0 replies; 76+ messages in thread
From: Paul E. McKenney @ 2011-05-20 15:12 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Frederic Weisbecker, LKML, Peter Zijlstra, Stable
On Fri, May 20, 2011 at 10:36:48AM +0200, Ingo Molnar wrote:
>
> * Frederic Weisbecker <fweisbec@gmail.com> wrote:
>
> > HARDIRQ_ENTER() maps to irq_enter() which calls rcu_irq_enter().
> > But HARDIRQ_EXIT() maps to __irq_exit() which doesn't call
> > rcu_irq_exit().
> >
> > So for every locking selftest that simulates hardirq disabled,
> > we create an imbalance in the rcu extended quiescent state
> > internal state.
> >
> > As a result, after the first missing rcu_irq_exit(), subsequent
> > irqs won't exit any extended quiescent state the time of the
> > interrupt servicing. Rcu read side critical section inside irqs
> > on that CPU won't be guaranteed anymore.
> >
> > To fix this, just use __irq_enter() to simulate the hardirq
> > context. This is sufficient for the locking selftests as we
> > don't need to exit any extended quiescent state or perform
> > any check that irqs normally do when they wake up from idle.
> >
> > As a side effect, this patch should make it possible to
> > restore
> > "rcu: Decrease memory-barrier usage based on semi-formal proof",
> > which eventually helped finding this bug.
> >
> > Reported-and-tested-by: Yinghai Lu <yinghai@kernel.org>
> > Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> > Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > Cc: Ingo Molnar <mingo@elte.hu>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Cc: Stable <stable@kernel.org>
> > ---
> > lib/locking-selftest.c | 2 +-
> > 1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c
> > index 619313e..507a22f 100644
> > --- a/lib/locking-selftest.c
> > +++ b/lib/locking-selftest.c
> > @@ -144,7 +144,7 @@ static void init_shared_classes(void)
> >
> > #define HARDIRQ_ENTER() \
> > local_irq_disable(); \
> > - irq_enter(); \
> > + __irq_enter(); \
> > WARN_ON(!in_irq());
> >
> > #define HARDIRQ_EXIT() \
>
> Very nice!
>
> Note that a cherry-picked version of the manual revert from Paul is upstream
> now with the main body of RCU changes, so we can do the finegrained split-up
> queue approach for this one problematic commit (that is no longer problematic).
>
> I think it can still go in for v2.6.40. Paul, what do you think?
I agree. I will rebase my split-up version, rebase on that, retest
and push.
Thanx, Paul
^ permalink raw reply [flat|nested] 76+ messages in thread
end of thread, other threads:[~2011-05-20 15:12 UTC | newest]
Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-05-08 15:18 [GIT PULL rcu/next] rcu commits for 2.6.40 Paul E. McKenney
2011-05-09 7:36 ` Ingo Molnar
2011-05-09 21:09 ` Yinghai Lu
2011-05-10 8:56 ` Paul E. McKenney
2011-05-10 9:37 ` Ingo Molnar
2011-05-10 18:04 ` Yinghai Lu
2011-05-10 19:32 ` Paul E. McKenney
2011-05-10 20:52 ` Yinghai Lu
2011-05-11 4:54 ` Paul E. McKenney
2011-05-11 6:03 ` Yinghai Lu
2011-05-11 6:42 ` Yinghai Lu
2011-05-11 20:13 ` Paul E. McKenney
2011-05-11 16:54 ` Yinghai Lu
2011-05-11 16:56 ` Yinghai Lu
2011-05-11 20:18 ` Paul E. McKenney
2011-05-11 20:59 ` Yinghai Lu
2011-05-11 21:30 ` Yinghai Lu
2011-05-11 23:02 ` Yinghai Lu
2011-05-12 6:03 ` Ingo Molnar
2011-05-12 7:27 ` Yinghai Lu
2011-05-12 7:42 ` Yinghai Lu
2011-05-12 9:20 ` Paul E. McKenney
2011-05-12 17:31 ` Yinghai Lu
2011-05-12 21:36 ` Yinghai Lu
2011-05-13 1:28 ` Yinghai Lu
2011-05-13 8:42 ` Ingo Molnar
2011-05-13 12:19 ` Ingo Molnar
2011-05-13 13:04 ` Ingo Molnar
2011-05-13 13:12 ` Ingo Molnar
2011-05-13 14:14 ` Paul E. McKenney
2011-05-13 15:07 ` Ingo Molnar
2011-05-13 16:26 ` Paul E. McKenney
2011-05-16 7:08 ` Ingo Molnar
2011-05-16 7:48 ` Paul E. McKenney
2011-05-16 11:51 ` Ingo Molnar
2011-05-16 12:23 ` Ingo Molnar
2011-05-16 14:30 ` Ingo Molnar
2011-05-16 21:33 ` Paul E. McKenney
2011-05-16 22:07 ` Paul E. McKenney
2011-05-16 21:24 ` Paul E. McKenney
2011-05-16 23:52 ` Frederic Weisbecker
2011-05-17 2:40 ` Frederic Weisbecker
2011-05-17 7:53 ` Paul E. McKenney
2011-05-17 12:43 ` Frederic Weisbecker
2011-05-17 22:21 ` Paul E. McKenney
2011-05-18 21:10 ` Yinghai Lu
2011-05-18 23:13 ` Frederic Weisbecker
2011-05-19 4:33 ` Yinghai Lu
2011-05-19 14:47 ` Frederic Weisbecker
2011-05-19 19:51 ` Yinghai Lu
2011-05-19 21:15 ` Frederic Weisbecker
2011-05-19 21:45 ` Yinghai Lu
2011-05-20 0:09 ` [PATCH] rcu: Fix unpaired rcu_irq_enter() from locking selftests Frederic Weisbecker
2011-05-20 8:36 ` Ingo Molnar
2011-05-20 15:12 ` Paul E. McKenney
2011-05-20 15:11 ` Paul E. McKenney
2011-05-20 0:14 ` [GIT PULL rcu/next] rcu commits for 2.6.40 Frederic Weisbecker
2011-05-13 14:40 ` Ingo Molnar
2011-05-13 16:38 ` Paul E. McKenney
2011-05-16 7:10 ` Ingo Molnar
2011-05-13 21:08 ` Yinghai Lu
2011-05-14 14:26 ` Paul E. McKenney
2011-05-14 15:31 ` Paul E. McKenney
2011-05-14 18:34 ` Paul E. McKenney
2011-05-15 3:59 ` Yinghai Lu
2011-05-15 4:14 ` Yinghai Lu
2011-05-15 5:41 ` Yinghai Lu
2011-05-15 5:49 ` Yinghai Lu
2011-05-15 6:04 ` Paul E. McKenney
2011-05-15 6:59 ` Paul E. McKenney
2011-05-16 7:08 ` Paul E. McKenney
2011-05-16 7:39 ` Ingo Molnar
2011-05-15 6:01 ` Paul E. McKenney
2011-05-15 22:01 ` Frederic Weisbecker
2011-05-16 5:56 ` Paul E. McKenney
2011-05-16 22:40 ` Frederic Weisbecker
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.