[PATCH v7 0/3] CPUFreq: Add support for opp-sharing cpus

[PATCH v7 0/3] CPUFreq: Add support for opp-sharing cpus

[Nicola Mazzucato]

Hi Viresh,

In this V7 posting I have reworked the CPUFreq scmi driver in a different
way as suggested in v6.
Essentially I believe it is more efficient to keep the support for opp-shared
in the _init() stage, rather than moving everything to _probe(), storing
whatever is required and reuse it only once in _init().
The reasons for this are:
- scmi-cpufreq has no init cases that would require a deferred probe.
- therefore, moving the all the cpus initialisation to probe, whilst possible,
  will result in a waste of memory, since we need to store cpumask and
  freq_table only for them to be reused just once later on at init.
- it does not appear to be functional justification for moving the init code
  to probe, which results in unnecessary overhead for both coding and review.
- this change is much smaller and only one patch required (1 file changed,
  52 insertions, 20 deletions) compared to a version where we first move init
  code to _probe and later add support for opp-v2 (2 patches, 1 file changed,
  194+34 insertions, 44+9 deletions, version with a linked list).
- this v7 implementation is much easier to maintain.

Many thanks,
Nicola

[v7]
  * Bring back common stuff for CPUs from _init stage to _probe
  * Remove patch "scmi-cpufreq: Move CPU initialisation to probe"

This v7 is based on Linux 5.11-rc6

[v6]
  * Remove deferred probe, not occurring
  * Move common stuff for CPUs from _init stage to _probe

This V6 is rebased on next-20210111

[v5]
  * Rework documentation of opp-shared within OPP node
  * Register EM only for the first CPU within cpumask in driver
  * Add check for nr_opp in driver before registering EM
  * Add comments on both dev_pm_opp_get_opp_count in driver
  * Remove redundant ret=0 in driver

This v5 is rebased on top of:
next-20201208 + Lukasz Luba's patches [1]

[v4]
  * Remove unconditional set of opp_table->shared_opp to exclusive
  * Add implementation for scmi-cpufreq
  * Change subject

These patches are on top of:
next-20201201 + Lukasz Luba's patches (waiting for Rafael) [1]

[v3]
  * Remove proposal for new 'cpu-performance-dependencies' as we instead
    can reuse the opp table.
  * Update documentation for devicetree/bindings/opp
  * Minor changes within opp to support empty opp table
  * Rework the RFC by adding a second proposal

[v2]
  * Fix errors when running make dt_binding_check
  * Improve commit message description for the dt-binding
  * Add RFC for implementation in cpufreq-core and one of its
    drivers.

Nicola Mazzucato (2):
  scmi-cpufreq: Remove deferred probe
  scmi-cpufreq: Get opp_shared_cpus from opp-v2 for EM

Sudeep Holla (1):
  cpufreq: blacklist Arm Vexpress platforms in cpufreq-dt-platdev

 drivers/cpufreq/cpufreq-dt-platdev.c |  2 +
 drivers/cpufreq/scmi-cpufreq.c       | 70 +++++++++++++++++++++-------
 2 files changed, 54 insertions(+), 18 deletions(-)

-- 
2.27.0

[PATCH v7 1/3] scmi-cpufreq: Remove deferred probe

[Nicola Mazzucato]

The current implementation of the scmi_cpufreq_init() function returns
-EPROBE_DEFER when the OPP table is not populated. In practice the
cpufreq core cannot handle this error code.
Therefore, fix the return value and clarify the error message.

Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
Signed-off-by: Nicola Mazzucato <nicola.mazzucato@arm.com>
---
 drivers/cpufreq/scmi-cpufreq.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c
index 491a0a24fb1e..34bf2eb8d465 100644
--- a/drivers/cpufreq/scmi-cpufreq.c
+++ b/drivers/cpufreq/scmi-cpufreq.c
@@ -155,9 +155,11 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
 
 	nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
 	if (nr_opp <= 0) {
-		dev_dbg(cpu_dev, "OPP table is not ready, deferring probe\n");
-		ret = -EPROBE_DEFER;
-		goto out_free_opp;
+		dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
+			__func__, ret);
+
+		ret = -ENODEV;
+		goto out_free_priv;
 	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
-- 
2.27.0

[PATCH v7 2/3] scmi-cpufreq: Get opp_shared_cpus from opp-v2 for EM

[Nicola Mazzucato]

By design, SCMI performance domains define the granularity of
performance controls, they do not describe any underlying hardware
dependencies (although they may match in many cases).

It is therefore possible to have some platforms where hardware may have
the ability to control CPU performance at different granularity and choose
to describe fine-grained performance control through SCMI.

In such situations, the energy model would be provided with inaccurate
information based on controls, while it still needs to know the
performance boundaries.

To restore correct functionality, retrieve information of CPUs under the
same performance domain from operating-points-v2 in DT, and pass it on to
EM.

Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Ionela Voinescu <ionela.voinescu@arm.com>
Signed-off-by: Nicola Mazzucato <nicola.mazzucato@arm.com>
---
 drivers/cpufreq/scmi-cpufreq.c | 72 ++++++++++++++++++++++++----------
 1 file changed, 52 insertions(+), 20 deletions(-)

diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c
index 34bf2eb8d465..fc9866511f01 100644
--- a/drivers/cpufreq/scmi-cpufreq.c
+++ b/drivers/cpufreq/scmi-cpufreq.c
@@ -126,6 +126,7 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
 	struct scmi_data *priv;
 	struct cpufreq_frequency_table *freq_table;
 	struct em_data_callback em_cb = EM_DATA_CB(scmi_get_cpu_power);
+	cpumask_var_t opp_shared_cpus;
 	bool power_scale_mw;
 
 	cpu_dev = get_cpu_device(policy->cpu);
@@ -134,32 +135,62 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
 		return -ENODEV;
 	}
 
-	ret = handle->perf_ops->device_opps_add(handle, cpu_dev);
-	if (ret) {
-		dev_warn(cpu_dev, "failed to add opps to the device\n");
-		return ret;
-	}
+	if (!zalloc_cpumask_var(&opp_shared_cpus, GFP_KERNEL))
+		ret = -ENOMEM;
 
+	/* Obtain CPUs that share SCMI performance controls */
 	ret = scmi_get_sharing_cpus(cpu_dev, policy->cpus);
 	if (ret) {
 		dev_warn(cpu_dev, "failed to get sharing cpumask\n");
-		return ret;
+		goto out_free_cpumask;
 	}
 
-	ret = dev_pm_opp_set_sharing_cpus(cpu_dev, policy->cpus);
-	if (ret) {
-		dev_err(cpu_dev, "%s: failed to mark OPPs as shared: %d\n",
-			__func__, ret);
-		return ret;
+	/*
+	 * Obtain CPUs that share performance levels.
+	 * The OPP 'sharing cpus' info may come from DT through an empty opp
+	 * table and opp-shared.
+	 */
+	ret = dev_pm_opp_of_get_sharing_cpus(cpu_dev, opp_shared_cpus);
+	if (ret || !cpumask_weight(opp_shared_cpus)) {
+		/*
+		 * Either opp-table is not set or no opp-shared was found.
+		 * Use the CPU mask from SCMI to designate CPUs sharing an OPP
+		 * table.
+		 */
+		cpumask_copy(opp_shared_cpus, policy->cpus);
 	}
 
+	/*
+	 * Add OPPs only on those CPUs for which we haven't already done so.
+	 */
 	nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
 	if (nr_opp <= 0) {
-		dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
-			__func__, ret);
-
-		ret = -ENODEV;
-		goto out_free_priv;
+		ret = handle->perf_ops->device_opps_add(handle, cpu_dev);
+		if (ret) {
+			dev_warn(cpu_dev, "failed to add opps to the device\n");
+			goto out_free_cpumask;
+		}
+
+		nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
+		if (nr_opp <= 0) {
+			dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
+				__func__, ret);
+
+			ret = -ENODEV;
+			goto out_free_opp;
+		}
+
+		ret = dev_pm_opp_set_sharing_cpus(cpu_dev, opp_shared_cpus);
+		if (ret) {
+			dev_err(cpu_dev, "%s: failed to mark OPPs as shared: %d\n",
+				__func__, ret);
+
+			goto out_free_opp;
+		}
+
+		power_scale_mw = handle->perf_ops->power_scale_mw_get(handle);
+		em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb,
+					    opp_shared_cpus, power_scale_mw);
 	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
@@ -192,17 +223,18 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
 	policy->fast_switch_possible =
 		handle->perf_ops->fast_switch_possible(handle, cpu_dev);
 
-	power_scale_mw = handle->perf_ops->power_scale_mw_get(handle);
-	em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb, policy->cpus,
-				    power_scale_mw);
-
+	free_cpumask_var(opp_shared_cpus);
 	return 0;
 
 out_free_priv:
 	kfree(priv);
+
 out_free_opp:
 	dev_pm_opp_remove_all_dynamic(cpu_dev);
 
+out_free_cpumask:
+	free_cpumask_var(opp_shared_cpus);
+
 	return ret;
 }
 
-- 
2.27.0

[PATCH v7 3/3] cpufreq: blacklist Arm Vexpress platforms in cpufreq-dt-platdev

[Nicola Mazzucato]

From: Sudeep Holla <sudeep.holla@arm.com>

Add "arm,vexpress" to cpufreq-dt-platdev blacklist since the actual
scaling is handled by the firmware cpufreq drivers(scpi, scmi and
vexpress-spc).

Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
---
 drivers/cpufreq/cpufreq-dt-platdev.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/cpufreq/cpufreq-dt-platdev.c b/drivers/cpufreq/cpufreq-dt-platdev.c
index bd2db0188cbb..91e6a0c10dbf 100644
--- a/drivers/cpufreq/cpufreq-dt-platdev.c
+++ b/drivers/cpufreq/cpufreq-dt-platdev.c
@@ -103,6 +103,8 @@ static const struct of_device_id whitelist[] __initconst = {
 static const struct of_device_id blacklist[] __initconst = {
 	{ .compatible = "allwinner,sun50i-h6", },
 
+	{ .compatible = "arm,vexpress", },
+
 	{ .compatible = "calxeda,highbank", },
 	{ .compatible = "calxeda,ecx-2000", },
 
-- 
2.27.0

Re: [PATCH v7 1/3] scmi-cpufreq: Remove deferred probe

[Viresh Kumar]

On 15-02-21, 07:51, Nicola Mazzucato wrote:
> The current implementation of the scmi_cpufreq_init() function returns
> -EPROBE_DEFER when the OPP table is not populated. In practice the
> cpufreq core cannot handle this error code.
> Therefore, fix the return value and clarify the error message.
> 
> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Signed-off-by: Nicola Mazzucato <nicola.mazzucato@arm.com>
> ---
>  drivers/cpufreq/scmi-cpufreq.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c
> index 491a0a24fb1e..34bf2eb8d465 100644
> --- a/drivers/cpufreq/scmi-cpufreq.c
> +++ b/drivers/cpufreq/scmi-cpufreq.c
> @@ -155,9 +155,11 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
>  
>  	nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
>  	if (nr_opp <= 0) {
> -		dev_dbg(cpu_dev, "OPP table is not ready, deferring probe\n");
> -		ret = -EPROBE_DEFER;
> -		goto out_free_opp;

Why change goto label as well ?

> +		dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
> +			__func__, ret);
> +
> +		ret = -ENODEV;
> +		goto out_free_priv;
>  	}
>  
>  	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
> -- 
> 2.27.0

-- 
viresh

Re: [PATCH v7 2/3] scmi-cpufreq: Get opp_shared_cpus from opp-v2 for EM

[Viresh Kumar]

On 15-02-21, 07:51, Nicola Mazzucato wrote:
> +	/*
> +	 * Add OPPs only on those CPUs for which we haven't already done so.
> +	 */
>  	nr_opp = dev_pm_opp_get_opp_count(cpu_dev);

Please add a more detailed comment here explaining why you expect OPPs
to be present here in advance. i.e. you _may_ have policy per CPU even
though OPP core says OPPs are shared.. It is not straight forward to
catch otherwise.

>  	if (nr_opp <= 0) {
> -		dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
> -			__func__, ret);
> -
> -		ret = -ENODEV;
> -		goto out_free_priv;
> +		ret = handle->perf_ops->device_opps_add(handle, cpu_dev);
> +		if (ret) {
> +			dev_warn(cpu_dev, "failed to add opps to the device\n");
> +			goto out_free_cpumask;
> +		}
> +
> +		nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
> +		if (nr_opp <= 0) {
> +			dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
> +				__func__, ret);
> +
> +			ret = -ENODEV;
> +			goto out_free_opp;
> +		}
> +
> +		ret = dev_pm_opp_set_sharing_cpus(cpu_dev, opp_shared_cpus);
> +		if (ret) {
> +			dev_err(cpu_dev, "%s: failed to mark OPPs as shared: %d\n",
> +				__func__, ret);
> +
> +			goto out_free_opp;
> +		}
> +
> +		power_scale_mw = handle->perf_ops->power_scale_mw_get(handle);
> +		em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb,
> +					    opp_shared_cpus, power_scale_mw);
>  	}

-- 
viresh

Re: [PATCH v7 1/3] scmi-cpufreq: Remove deferred probe

[Nicola Mazzucato]

Hi Viresh,


On 2/18/21 10:35 AM, Viresh Kumar wrote:
> On 15-02-21, 07:51, Nicola Mazzucato wrote:
>> The current implementation of the scmi_cpufreq_init() function returns
>> -EPROBE_DEFER when the OPP table is not populated. In practice the
>> cpufreq core cannot handle this error code.
>> Therefore, fix the return value and clarify the error message.
>>
>> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
>> Signed-off-by: Nicola Mazzucato <nicola.mazzucato@arm.com>
>> ---
>>  drivers/cpufreq/scmi-cpufreq.c | 8 +++++---
>>  1 file changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c
>> index 491a0a24fb1e..34bf2eb8d465 100644
>> --- a/drivers/cpufreq/scmi-cpufreq.c
>> +++ b/drivers/cpufreq/scmi-cpufreq.c
>> @@ -155,9 +155,11 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
>>  
>>  	nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
>>  	if (nr_opp <= 0) {
>> -		dev_dbg(cpu_dev, "OPP table is not ready, deferring probe\n");
>> -		ret = -EPROBE_DEFER;
>> -		goto out_free_opp;
> 
> Why change goto label as well ?

oops! thanks for spotting this :)

> 
>> +		dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
>> +			__func__, ret);
>> +
>> +		ret = -ENODEV;
>> +		goto out_free_priv;
>>  	}
>>  
>>  	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
>> -- 
>> 2.27.0
> 

Re: [PATCH v7 2/3] scmi-cpufreq: Get opp_shared_cpus from opp-v2 for EM

[Nicola Mazzucato]

Hi Viresh,


On 2/18/21 11:00 AM, Viresh Kumar wrote:
> On 15-02-21, 07:51, Nicola Mazzucato wrote:
>> +	/*
>> +	 * Add OPPs only on those CPUs for which we haven't already done so.
>> +	 */
>>  	nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
> 
> Please add a more detailed comment here explaining why you expect OPPs
> to be present here in advance. i.e. you _may_ have policy per CPU even
> though OPP core says OPPs are shared.. It is not straight forward to
> catch otherwise.

Sure, I'll put more details, thanks.

> 
>>  	if (nr_opp <= 0) {
>> -		dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
>> -			__func__, ret);
>> -
>> -		ret = -ENODEV;
>> -		goto out_free_priv;
>> +		ret = handle->perf_ops->device_opps_add(handle, cpu_dev);
>> +		if (ret) {
>> +			dev_warn(cpu_dev, "failed to add opps to the device\n");
>> +			goto out_free_cpumask;
>> +		}
>> +
>> +		nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
>> +		if (nr_opp <= 0) {
>> +			dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
>> +				__func__, ret);
>> +
>> +			ret = -ENODEV;
>> +			goto out_free_opp;
>> +		}
>> +
>> +		ret = dev_pm_opp_set_sharing_cpus(cpu_dev, opp_shared_cpus);
>> +		if (ret) {
>> +			dev_err(cpu_dev, "%s: failed to mark OPPs as shared: %d\n",
>> +				__func__, ret);
>> +
>> +			goto out_free_opp;
>> +		}
>> +
>> +		power_scale_mw = handle->perf_ops->power_scale_mw_get(handle);
>> +		em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb,
>> +					    opp_shared_cpus, power_scale_mw);
>>  	}
>