Re: [RFC][PATCH 0/8] sched,idle: need resched polling rework

[Andy Lutomirski <luto@amacapital.net>]

On Fri, Apr 11, 2014 at 6:42 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> A while ago both Mike and Andy complained that we still get pointless wakeup
> IPIs, we had a few patches back and forth but eventually more or less agreed
> and then nothing... :-)
>
> So here's a number of patches that implement something near what we left off
> with.
>
> Its only been compile/boot tested on x86_64, I've no actually looked at the IPI
> numbers yet.
>

Looks generally good and it should be a nice cleanup.

That being said, I think that this addresses once one of the two major
issues.  While the race you're fixing is more interesting, I think its
impact is dwarfed by the fact that ttwu_queue_remote completely
ignores polling.  (NB: I haven't actually tested this patch set, but I
did try to instrument this stuff awhile ago.)

To fix this, presumably the wake-from-idle path needs a
sched_ttwu_pending call, and ttwu_queue_remote could use resched_task.
 sched_ttwu_pending could benefit from a straightforward optimization:
it doesn't need rq->lock if llist is empty.

If you're not planning on trying to fix that, I can try to write up a
patch in the next day or two.

Even with all of this fixed, what happens when ttwu_queue_remote is
called with a task that has lower priority than whatever is currently
running on the targeted cpu?  I think the result is an IPI that serves
very little purpose other than avoiding taking a spinlock in the
waking thread.  This may be a bad tradeoff.  I doubt that this matters
for my particular workload, though.

--Andy