Re: [Bug #12650] Strange load average and ksoftirqd behavior with 2.6.29-rc2-git1

From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Damien Wyart <damien.wyart@free.fr>
Cc: "Ingo Molnar" <mingo@elte.hu>,
	"Peter Zijlstra" <a.p.zijlstra@chello.nl>,
	"Mike Galbraith" <efault@gmx.de>,
	"Frédéric Weisbecker" <fweisbec@gmail.com>,
	"Rafael J. Wysocki" <rjw@sisk.pl>,
	"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
	"Kernel Testers List" <kernel-testers@vger.kernel.org>
Subject: Re: [Bug #12650] Strange load average and ksoftirqd behavior with 2.6.29-rc2-git1
Date: Tue, 17 Feb 2009 08:05:04 -0800	[thread overview]
Message-ID: <20090217160504.GF6761@linux.vnet.ibm.com> (raw)
In-Reply-To: <20090217153925.GA2308@localhost.localdomain>

On Tue, Feb 17, 2009 at 04:39:25PM +0100, Damien Wyart wrote:
> Hello Paul,
> 
> > > There's no strong reason - we've been back and forth about RCU in
> > > the dynticks code. Mind sending a test patch for Damien to try?
> 
> > But of course!  ;-)
> 
> With this patch, the problem goes away and system activity seems normal,
> both on the P4 with high load and on the recent laptop.
> 
> Btw, could you explain briefly why, without this patch, a kernel
> enabling classical RCU doesn't show the ksoftirqd problem at all?

Classic RCU's rcu_pending() can afford to be much more conservative,
due to the easy availability of global information.  In contrast,
Hierarchical RCU will invoke the softirq in cases where the information
required to make an exact decision is off somewhere else in the tree.

There is also a possibility that Hierarchical RCU is for some reason
failing to detect that its rcu_check_callbacks() is being invoked from
the idle loop -- which would need to be fixed if it is really happening.
But given that the two rcu_check_callbacks() implementations are nearly
identical, I cannot see how this is happening.  (Wouldn't be the first
time that I failed to see how something was happening, though!)

							Thanx, Paul

> Damien
> 
> > The following patch removes the call to rcu_pending() and
> > rcu_check_callbacks() from the x86 32-bit idle loop in order to
> > reduce the softirq load on idle systems.
> 
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > ---
> 
> >  process_32.c |    3 ---
> >  1 file changed, 3 deletions(-)
> 
> > diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
> > index a546f55..bd4da2a 100644
> > --- a/arch/x86/kernel/process_32.c
> > +++ b/arch/x86/kernel/process_32.c
> > @@ -104,9 +104,6 @@ void cpu_idle(void)
> >  			check_pgt_cache();
> >  			rmb();
> 
> > -			if (rcu_pending(cpu))
> > -				rcu_check_callbacks(cpu, 0);
> > -
> >  			if (cpu_is_offline(cpu))
> >  				play_dead();
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/