Re: [rfc patch] sched/fair: Use instantaneous load for fork/exec balancing

From: Mike Galbraith <umgwanakikbuti@gmail.com>
To: Matt Fleming <matt@codeblueprint.co.uk>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Yuyang Du <yuyang.du@intel.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Mel Gorman <mgorman@techsingularity.net>
Subject: Re: [rfc patch] sched/fair: Use instantaneous load for fork/exec balancing
Date: Wed, 06 Jul 2016 14:21:32 +0200	[thread overview]
Message-ID: <1467807692.3630.25.camel@gmail.com> (raw)
In-Reply-To: <20160706114543.GQ8415@codeblueprint.co.uk>

On Wed, 2016-07-06 at 12:45 +0100, Matt Fleming wrote:
> On Mon, 04 Jul, at 07:43:14PM, Mike Galbraith wrote:
> > On Mon, 2016-07-04 at 16:04 +0100, Matt Fleming wrote:
> > 
> > > But we can optimise the special case of dequeueing the last entity and
> > > reset ::runnable_load_avg early, which gives a performance improvement
> > > to workloads that trigger the load balancer, such as fork-heavy
> > > applications when SD_BALANCE_FORK is set, because it gives a more up
> > > to date view of how busy the cpu is.
> > 
> > Begs the question: what's so special about this case vs any other
> > dequeue/enqueue?
>  
> All that makes this special is that this is the behaviour seen when
> running hackbench - initial heavy forking by some master task which
> eventually wakes everyone up. So you get this huge sequence of "fork,
> enqueue, run, dequeue". Yes, it's a complete hack.

I'm a bit concerned that poking holes in the logic to make hackbench a
bit happier will eradicate the calming effect that avg/aging business
has on load balancing, inflicting harm on real world loads.  That would
be a bad trade.

> > I've given up on this as being a waste of time.  Either you serialize
> > everything box wide (not!) and can then make truly accurate evaluations
> > of state, or you're making an educated guess based upon what once was.
> > 
> > The only place I've seen where using the average consistently has
> > issues is with a longish period periodic load (schbench).
> 
> I'm open to any suggestion that restores performance to that seen
> before commit 0905f04eb21f, whether or not that involves changing how
> load averages are used.

None here.  That hackbench was fond of that dead bug is just too bad,
as Peter seldom resurrects bugs once swatted :)  FWIW, I took a peek at
distribution on my little desktop box while fiddling, and while it was
not a pretty flat line, it wasn't a stock market crash graph either.

	-Mike