Re: [PATCH 1/1] sched: Consider CPU contention in frequency & load-balance busiest CPU selection

From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Qais Yousef <qyousef@layalina.io>,
	Kajetan Puchalski <kajetan.puchalski@arm.com>,
	Morten Rasmussen <morten.rasmussen@arm.com>,
	Vincent Donnefort <vdonnefort@google.com>,
	Quentin Perret <qperret@google.com>,
	Abhijeet Dharmapurikar <adharmap@quicinc.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/1] sched: Consider CPU contention in frequency & load-balance busiest CPU selection
Date: Wed, 3 May 2023 19:13:17 +0200	[thread overview]
Message-ID: <4c77a01e-8ff3-f415-ffff-01c8d79a8bc7@arm.com> (raw)
In-Reply-To: <20230429145829.GB1495785@hirez.programming.kicks-ass.net>

On 29/04/2023 16:58, Peter Zijlstra wrote:
> On Thu, Apr 06, 2023 at 05:50:30PM +0200, Dietmar Eggemann wrote:
>> Use new cpu_boosted_util_cfs() instead of cpu_util_cfs().
>>
>> The former returns max(util_avg, runnable_avg) capped by max CPU
>> capacity. CPU contention is thereby considered through runnable_avg.
>>
>> The change in load-balance only affects migration type `migrate_util`.
> 
> But why, and how does it affect? That is, isn't this Changelog a wee bit
> sparse?

Absolutely. 

I have compelling test data based on JankbenchX on Pixel6 for 
sugov_get_util() case I will share with v2.

But for the find_busiest_queue() (lb migration_type = migrate_util) case 
it is tricky to create a test env.

`migrate_util` only operates in DIE or NUMA SD (!SD_SHARE_PKG_RESOURCES) 
and the system should not be overloaded (spare capacity on the local 
group).

perf bench sched messaging with a small number of tasks compared to CPU 
number shows some improvement.

E.g. Ampere Altra with 160 CPUs, SDs = {MC, DIE, NUMA} and 1 group = 40 
tasks shows some improvement:

perf stat --null --repeat 10 -- perf bench sched messaging -t -g 1 -l 2000

0.4869 +- 0.0173 seconds time elapsed  ( +-  3.55% ) -> 0.4377 +- 0.0147 seconds time elapsed  ( +-  3.36% )

If I put more tasks onto the machine, the conditions to go into 
`migrate_util` lb vanish so there is no difference.

Also if I test on an 8 CPUs system, SDs = {MC, DIE} and 1 group = 40 
tasks the conditions to do migrate_util lb are only true for a short
moment of the beginning of the test so it does not have much implication
on the score.

[...]