Re: [PATCH V2 0/3] Introduce Thermal Pressure

From: Thara Gopinath <thara.gopinath@linaro.org>
To: Ionela Voinescu <ionela.voinescu@arm.com>,
	mingo@redhat.com, peterz@infradead.org, rui.zhang@intel.com
Cc: linux-kernel@vger.kernel.org, amit.kachhap@gmail.com,
	viresh.kumar@linaro.org, javi.merino@kernel.org,
	edubezval@gmail.com, daniel.lezcano@linaro.org,
	vincent.guittot@linaro.org, nicolas.dechesne@linaro.org,
	bjorn.andersson@linaro.org, dietmar.eggemann@arm.com
Subject: Re: [PATCH V2 0/3] Introduce Thermal Pressure
Date: Fri, 26 Apr 2019 07:50:21 -0400	[thread overview]
Message-ID: <5CC2F07D.1080603@linaro.org> (raw)
In-Reply-To: <b777c6da-0fe2-96d2-240d-96b065a3f18d@arm.com>

On 04/24/2019 11:57 AM, Ionela Voinescu wrote:
> Hi Thara,
> 
> The idea and the results look promising. I'm trying to understand better
> the cause of the improvements so I've added below some questions that
> would help me out with this.

Hi Ionela,

Thanks for the review.

> 
> 
>> Regarding testing, basic build, boot and sanity testing have been
>> performed on hikey960 mainline kernel with debian file system.
>> Further, aobench (An occlusion renderer for benchmarking realworld
>> floating point performance), dhrystone and hackbench test have been
>> run with the thermal pressure algorithm. During testing, due to
>> constraints of step wise governor in dealing with big little systems,
>> cpu cooling was disabled on little core, the idea being that
>> big core will heat up and cpu cooling device will throttle the
>> frequency of the big cores there by limiting the maximum available
>> capacity and the scheduler will spread out tasks to little cores as well.
>> Finally, this patch series has been boot tested on db410C running v5.1-rc4
>> kernel.
>>
> 
> Did you try using IPA as well? It is better equipped to deal with
> big-LITTLE systems and it's more probable IPA will be used for these
> systems, where your solution will have the biggest impact as well.
> The difference will be that you'll have both the big cluster and the
> LITTLE cluster capped in different proportions depending on their
> utilization and their efficiency.

No. I did not use IPA simply because it was not enabled in mainline. I
agree it is better equipped  to deal with big-little systems. The idea
to remove cpu cooling on little cluster was to in some (not the
cleanest) manner to mimic this. But I agree that IPA testing is possibly
the next step.Any help in this regard is appreciated.

> 
>> During the course of development various methods of capturing
>> and reflecting thermal pressure were implemented.
>>
>> The first method to be evaluated was to convert the
>> capped max frequency into capacity and have the scheduler use the
>> instantaneous value when updating cpu_capacity.
>> This method is referenced as "Instantaneous Thermal Pressure" in the
>> test results below. 
>>
>> The next two methods employs different methods of averaging the
>> thermal pressure before applying it when updating cpu_capacity.
>> The first of these methods re-used the PELT algorithm already present
>> in the kernel that does the averaging of rt and dl load and utilization.
>> This method is referenced as "Thermal Pressure Averaging using PELT fmwk"
>> in the test results below.
>>
>> The final method employs an averaging algorithm that collects and
>> decays thermal pressure based on the decay period. In this method,
>> the decay period is configurable. This method is referenced as
>> "Thermal Pressure Averaging non-PELT Algo. Decay : XXX ms" in the
>> test results below.
>>
>> The test results below shows 3-5% improvement in performance when
>> using the third solution compared to the default system today where
>> scheduler is unware of cpu capacity limitations due to thermal events.
>>
> 
> Did you happen to record the amount of capping imposed on the big cores
> when these results were obtained? Did you find scenarios where the
> capacity of the bigs resulted in being lower than the capacity of the
> LITTLEs (capacity inversion)?
> This is one case where we'll see a big impact in considering thermal
> pressure.

I think I saw capacity inversion in some scenarios. I did not
particularly capture them.

> 
> Also, given that these are more or less sustained workloads, I'm
> wondering if there is any effect on workloads running on an uncapped
> system following capping. I would image such a test being composed of a
> single threaded period (no capping) followed by a multi-threaded period
> (with capping), continued in a loop. It might be interesting to have
> something like this as well, as part of your test coverage

I do not understand this. There is either capping for a workload or no
capping. There is no sysctl entry to turn on or off capping.

Regards
Thara
> 
> 
> Thanks,
> Ionela.
> 

-- 
Regards
Thara