From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
LKML <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@elte.hu>,
Peter Zijlstra <peterz@infradead.org>,
Rusty Russell <rusty@rustcorp.com.au>,
Namhyung Kim <namhyung@kernel.org>
Subject: Re: [Patch 0/7] Per cpu thread hotplug infrastructure - V3
Date: Fri, 20 Jul 2012 10:53:03 -0700 [thread overview]
Message-ID: <20120720175303.GA26784@linux.vnet.ibm.com> (raw)
In-Reply-To: <5009728E.7080902@linux.vnet.ibm.com>
On Fri, Jul 20, 2012 at 08:30:30PM +0530, Srivatsa S. Bhat wrote:
> On 07/20/2012 08:05 PM, Paul E. McKenney wrote:
> > On Fri, Jul 20, 2012 at 06:47:30PM +0530, Srivatsa S. Bhat wrote:
> >> On 07/19/2012 05:24 AM, Paul E. McKenney wrote:
> >>> On Wed, Jul 18, 2012 at 11:06:52PM +0530, Srivatsa S. Bhat wrote:
> >>>> On 07/16/2012 08:52 PM, Paul E. McKenney wrote:
> >>>>> On Mon, Jul 16, 2012 at 10:42:34AM -0000, Thomas Gleixner wrote:
> >>>>>> The following series implements the infrastructure for parking and
> >>>>>> unparking kernel threads to avoid the full teardown and fork on cpu
> >>>>>> hotplug operations along with management infrastructure for hotplug
> >>>>>> and users.
> >>>>>>
> >>>>>> Changes vs. V2:
> >>>>>>
> >>>>>> Use callbacks for all functionality. Thanks to Rusty for pointing
> >>>>>> that out. It makes the use sites nice and simple and keeps all the
> >>>>>> code which would be duplicated otherwise on the core.
> >>>>>
> >>>>> Hello, Thomas,
> >>>>>
> >>>>> What version should I apply this patchset to? I tried v3.5-rc7, but
> >>>>> got lots of warnings (one shown below) and the watchdog patch did not
> >>>>> apply.
> >>>>>
> >>>>
> >>>> Hi Paul,
> >>>>
> >>>> This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip
> >>>> tree.
> >>>
> >>> Thank you, Srivatsa, works much better. Still get "scheduling while
> >>> atomic", looking into that.
> >>>
> >>
> >> Got a chance to run this patchset now.. Even I am getting "scheduling while
> >> atomic" messages like shown below.. Hmmm...
> >
> > Here is what little I have done so far (lots of completing demands on time
> > this week, but I should have a goodly block of time to focus on this today):
> >
> > 1. The failure is from the softirq modifications. Reverting that
> > commit gets rid of the failures.
> >
> > 2. As one would expect, CONFIG_PREEMPT=n kernels do not have the
> > problem, which of course indicates a preempt_disable() imbalance.
>
> Right..
Except that the imbalance is not in softirq like I was thinking, but
rather in smpboot. See patch below, which clears this up for me.
Thanx, Paul
> > 3. I was unable to spot the problem by inspection, but this is not
> > too surprising given the high level of distraction this week.
> >
> > 4. Instrumentation shows that preempt_count() grows slowly with
> > time, but with the upper bits zero. This confirms the
> > preempt_disable imbalance.
> >
> > 5. I am currently placing WARN_ONCE() calls in the code to track
> > this down. When I do find it, I fully expect to feel very stupid
> > about my efforts on #3 above. ;-)
> >
>
> Hehe :-) I'll also see if I can dig out the problem..
smpboot.c | 4 ++--
softirq.c | 3 ++-
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index 1c1458f..b2545c8 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -148,12 +148,12 @@ static int smpboot_thread_fn(void *data)
}
if (!ht->thread_should_run(td->cpu)) {
- schedule_preempt_disabled();
+ preempt_enable();
+ schedule();
} else {
set_current_state(TASK_RUNNING);
preempt_enable();
ht->thread_fn(td->cpu);
- preempt_disable();
}
}
}
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 82ca065..090e1b9 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -744,9 +744,10 @@ static void run_ksoftirqd(unsigned int cpu)
local_irq_disable();
if (local_softirq_pending()) {
__do_softirq();
+ rcu_note_context_switch(cpu);
local_irq_enable();
cond_resched();
- rcu_note_context_switch(cpu);
+ return;
}
local_irq_enable();
}
next prev parent reply other threads:[~2012-07-20 17:53 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-07-16 10:42 [Patch 0/7] Per cpu thread hotplug infrastructure - V3 Thomas Gleixner
2012-07-16 10:42 ` [Patch 1/7] rcu: Yield simpler Thomas Gleixner
2012-08-13 15:07 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2012-07-16 10:42 ` [Patch 3/7] smpboot: Provide infrastructure for percpu hotplug threads Thomas Gleixner
2012-07-21 9:26 ` Srivatsa S. Bhat
2012-07-21 18:01 ` Srivatsa S. Bhat
2012-07-21 17:12 ` Paul E. McKenney
2012-08-13 15:10 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2012-09-19 21:47 ` [Patch 3/7] " Sasha Levin
2012-10-12 1:39 ` Sasha Levin
2012-07-16 10:42 ` [Patch 2/7] kthread: Implement park/unpark facility Thomas Gleixner
2012-07-21 9:31 ` Srivatsa S. Bhat
2012-08-13 15:08 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2012-07-16 10:42 ` [Patch 4/7] softirq: Use hotplug thread infrastructure Thomas Gleixner
2012-07-21 17:21 ` Paul E. McKenney
2012-07-23 21:15 ` Paul E. McKenney
2012-07-25 14:21 ` JoonSoo Kim
2012-08-13 15:12 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2012-07-16 10:42 ` [Patch 6/7] rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread Thomas Gleixner
2012-07-16 16:59 ` Paul E. McKenney
2012-08-13 15:13 ` [tip:smp/hotplug] " tip-bot for Paul E. McKenney
2012-07-16 10:42 ` [Patch 5/7] watchdog: Use hotplug thread infrastructure Thomas Gleixner
2012-08-13 15:13 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2012-08-14 7:20 ` viresh kumar
2012-08-14 8:42 ` Thomas Gleixner
2012-07-16 10:42 ` [Patch 7/7] infiniband: ehca: " Thomas Gleixner
2012-07-17 0:27 ` Rusty Russell
2012-08-13 15:14 ` [tip:smp/hotplug] infiniband: Ehca: " tip-bot for Thomas Gleixner
2012-07-16 15:22 ` [Patch 0/7] Per cpu thread hotplug infrastructure - V3 Paul E. McKenney
2012-07-18 17:36 ` Srivatsa S. Bhat
2012-07-18 23:54 ` Paul E. McKenney
2012-07-20 13:17 ` Srivatsa S. Bhat
2012-07-20 14:35 ` Paul E. McKenney
2012-07-20 15:00 ` Srivatsa S. Bhat
2012-07-20 17:53 ` Paul E. McKenney [this message]
2012-07-20 18:28 ` Srivatsa S. Bhat
2012-07-25 12:25 ` Srivatsa S. Bhat
2012-07-25 14:25 ` JoonSoo Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120720175303.GA26784@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=rusty@rustcorp.com.au \
--cc=srivatsa.bhat@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).