Re: [RFC PATCH] sched/fair: Introduce SIS_PAIR to wakeup task on local idle core first

From: Chen Yu <yu.c.chen@intel.com>
To: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Mel Gorman <mgorman@techsingularity.net>,
	Tim Chen <tim.c.chen@intel.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	"Steven Rostedt" <rostedt@goodmis.org>,
	K Prateek Nayak <kprateek.nayak@amd.com>,
	Abel Wu <wuyun.abel@bytedance.com>,
	Yicong Yang <yangyicong@hisilicon.com>,
	"Gautham R . Shenoy" <gautham.shenoy@amd.com>,
	Len Brown <len.brown@intel.com>, Chen Yu <yu.chen.surf@gmail.com>,
	Arjan Van De Ven <arjan.van.de.ven@intel.com>,
	Aaron Lu <aaron.lu@intel.com>, Barry Song <baohua@kernel.org>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH] sched/fair: Introduce SIS_PAIR to wakeup task on local idle core first
Date: Thu, 18 May 2023 11:41:55 +0800	[thread overview]
Message-ID: <ZGWeg6UaZ3WJ6ykI@chenyu5-mobl1> (raw)
In-Reply-To: <a2a4cd5b398390dcf01b800c964b80c6eba89d18.camel@gmx.de>

On 2023-05-17 at 21:52:21 +0200, Mike Galbraith wrote:
> On Thu, 2023-05-18 at 00:57 +0800, Chen Yu wrote:
> > >
> > I'm thinking of two directions based on current patch:
> >
> > 1. Check the task duration, if it is a high speed ping-pong pair, let the
> >    wakee search for an idle SMT sibling on current core.
> >
> >    This strategy give the best overall performance improvement, but
> >    the short task duration tweak based on online CPU number would be
> >    an obstacle.
> 
> Duration is pretty useless, as it says nothing about concurrency.
> Taking the 500us metric as an example, one pipe ping-pong can meet
> that, and toss up to nearly 50% of throughput out the window if you
> stack based only on duration.
> 
> > Or
> >
> > 2. Honors the idle core.
> >    That is to say, if there is an idle core in the system, choose that
> >    idle core first. Otherwise, fall back to searching for an idle smt
> >    sibling rather than choosing a idle CPU in a random half-busy core.
> >
> >    This strategy could partially mitigate the C2C overhead, and not
> >    breaking the idle-core-first strategy. So I had a try on it, with
> >    above change, I did see some improvement when the system is around
> >    half busy(afterall, the idle_has_core has to be false):
> 
> If mitigation is the goal, and until the next iteration of socket
> growth that's not a waste of effort, continuing to honor idle core is
> the only option that has a ghost of a chance.
> 
> That said, I don't like the waker/wakee have met heuristic much either,
> because tasks waking one another before can just as well mean they met
> at a sleeping lock, it does not necessarily imply latency bound IPC.
>
Yes, for a sleeping lock case, it does not matter whether it is woken up
on sibling idle, or an idle CPU on another half-busy core. But for the
pair sharing data, it could bring benefit. 
> I haven't met a heuristic I like, and that includes the ones I invent.
> The smarter you try to make them, the more precious fast path cycles
> they eat, and there's a never ending supply of holes in the damn things
> that want plugging.  A prime example was the SIS_CURRENT heuristic self
> destructing in my box, rendering that patch a not quite free noop :)
>
Yes.. SIS_CURRENT is not a universal win.


thanks,
Chenyu