Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE

[Josef Bacik <jbacik@fb.com>]

On 06/03/2015 10:24 AM, Peter Zijlstra wrote:
> On Wed, 2015-06-03 at 10:12 -0400, Rik van Riel wrote:
>
>> There is a policy vs mechanism thing here. Ingo and Peter
>> are worried about the overhead in the mechanism of finding
>> an idle CPU.  Your measurements show that the policy of
>> finding an idle CPU is the correct one.
>
> For his workload; I'm sure I can find a workload where it hurts.
>
> In fact, I'm fairly sure Mike knows one from the top of his head, seeing
> how he's the one playing about trying to shrink that idle search :-)
>

So the perf bench sched microbenchmarks are a pretty good analog for our 
workload.  I run

perf bench sched messaging -g 100 -l 10000
perf bench sched pipe

5 times and average the results to get an answer, really the messaging 
one is closest one and the one I look at.  I get like 56 seconds of 
runtime on plain 4.0 and 47 seconds patched, it's how I check my little 
experiments before doing the full real workload.

I don't want to tune the scheduler just for our workload, but the 
microbenchmarks we have are also showing the same performance 
improvements.  I would be super interested in workloads where this patch 
doesn't help so we could integrate that workload into perf sched bench 
to make us more confident in making policy changes in the scheduler.  So 
Mike if you have something specific in mind please elaborate and I'm 
happy to do the legwork to get it into perf bench and to test things 
until we're happy.

In the meantime I really want to get this fixed for us, I do not want to 
pull some weird old patch around for the next year until we rebase again 
next year, and then do this whole dance again.  What would be the way 
forward for getting this fixed now?  Do I need to hide it behind a 
sysctl or config option?  Thanks,

Josef