[PATCH RESEND v2 0/3] condition EAS enablement on FI support

[PATCH RESEND v2 0/3] condition EAS enablement on FI support

[Ionela Voinescu]

Given the maturity gained by cpufreq-based Frequency Invariance (FI)
support following the patches at [1], this series conditions Energy
Aware Scheduling (EAS) enablement on a frequency invariant system.

Currently, EAS can be enabled on a system without FI support, leading
to incorrect (energy-wise) task placements. As no warning is emitted,
it could take some debugging effort to track the behavior back to the
lack of FI support; this series changes that by disabling EAS
(and advertising it) when FI support is missing.

The series is structured as follows:
 - 1/3 - create function that can rebuild the scheduling and EAS'
   performance domains if EAS' initial conditions change
 - 2/3 - arm64: rebuild scheduling and performance domains in the
         case of late, counter-driven FI initialisation.
 - 3/3 - condition EAS enablement on FI support

RESEND v2: rebase and retest on v5.10-rc1

v1 -> v2:
 - v1 can be found at [2]
 - 1/2 - moved now "static inline" rebuild_sched_domains_energy() stub
   in sched/topology.h header, as suggested by Quentin.
 - Changed the order of 2/3 and 3/3 to maintain bisection as suggested
   by Quentin.
 - 3/3 - place the warning under sched_debug() as per Quentin and
   Dietmar's recommendations.

[1] Most recent version at:
https://lore.kernel.org/lkml/20200901205549.30096-1-ionela.voinescu@arm.com/
[2] https://lore.kernel.org/lkml/20200924123937.20938-1-ionela.voinescu@arm.com/

Many thanks,
Ionela.

Ionela Voinescu (3):
  sched/topology,schedutil: wrap sched domains rebuild
  arm64: rebuild sched domains on invariance status changes
  sched/topology: condition EAS enablement on FIE support

 arch/arm64/kernel/topology.c     | 10 ++++++++++
 include/linux/sched/topology.h   |  8 ++++++++
 kernel/sched/cpufreq_schedutil.c |  9 +--------
 kernel/sched/topology.c          | 27 ++++++++++++++++++++-------
 4 files changed, 39 insertions(+), 15 deletions(-)


base-commit: 3650b228f83adda7e5ee532e2b90429c03f7b9ec
-- 
2.17.1

[PATCH RESEND v2 1/3] sched/topology,schedutil: wrap sched domains rebuild

[Ionela Voinescu]

Add the rebuild_sched_domains_energy() function to wrap the functionality
that rebuilds the scheduling domains if any of the Energy Aware Scheduling
(EAS) initialisation conditions change. This functionality is used when
schedutil is added or removed or when EAS is enabled or disabled
through the sched_energy_aware sysctl.

Therefore, create a single function that is used in both these cases and
that can be later reused.

Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Acked-by: Quentin Perret <qperret@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
---
 include/linux/sched/topology.h   |  8 ++++++++
 kernel/sched/cpufreq_schedutil.c |  9 +--------
 kernel/sched/topology.c          | 18 +++++++++++-------
 3 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 9ef7bf686a9f..8f0f778b7c91 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -225,6 +225,14 @@ static inline bool cpus_share_cache(int this_cpu, int that_cpu)
 
 #endif	/* !CONFIG_SMP */
 
+#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_CPU_FREQ_GOV_SCHEDUTIL)
+extern void rebuild_sched_domains_energy(void);
+#else
+static inline void rebuild_sched_domains_energy(void)
+{
+}
+#endif
+
 #ifndef arch_scale_cpu_capacity
 /**
  * arch_scale_cpu_capacity - get the capacity scale factor of a given CPU.
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index e254745a82cb..37b303890336 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -899,16 +899,9 @@ struct cpufreq_governor *cpufreq_default_governor(void)
 cpufreq_governor_init(schedutil_gov);
 
 #ifdef CONFIG_ENERGY_MODEL
-extern bool sched_energy_update;
-extern struct mutex sched_energy_mutex;
-
 static void rebuild_sd_workfn(struct work_struct *work)
 {
-	mutex_lock(&sched_energy_mutex);
-	sched_energy_update = true;
-	rebuild_sched_domains();
-	sched_energy_update = false;
-	mutex_unlock(&sched_energy_mutex);
+	rebuild_sched_domains_energy();
 }
 static DECLARE_WORK(rebuild_sd_work, rebuild_sd_workfn);
 
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index dd7770226086..270bafb73506 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -211,6 +211,15 @@ unsigned int sysctl_sched_energy_aware = 1;
 DEFINE_MUTEX(sched_energy_mutex);
 bool sched_energy_update;
 
+void rebuild_sched_domains_energy(void)
+{
+	mutex_lock(&sched_energy_mutex);
+	sched_energy_update = true;
+	rebuild_sched_domains();
+	sched_energy_update = false;
+	mutex_unlock(&sched_energy_mutex);
+}
+
 #ifdef CONFIG_PROC_SYSCTL
 int sched_energy_aware_handler(struct ctl_table *table, int write,
 		void *buffer, size_t *lenp, loff_t *ppos)
@@ -223,13 +232,8 @@ int sched_energy_aware_handler(struct ctl_table *table, int write,
 	ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
 	if (!ret && write) {
 		state = static_branch_unlikely(&sched_energy_present);
-		if (state != sysctl_sched_energy_aware) {
-			mutex_lock(&sched_energy_mutex);
-			sched_energy_update = 1;
-			rebuild_sched_domains();
-			sched_energy_update = 0;
-			mutex_unlock(&sched_energy_mutex);
-		}
+		if (state != sysctl_sched_energy_aware)
+			rebuild_sched_domains_energy();
 	}
 
 	return ret;
-- 
2.17.1

Re: [PATCH RESEND v2 1/3] sched/topology,schedutil: wrap sched domains rebuild

[Rafael J. Wysocki]

On Tue, Oct 27, 2020 at 7:08 PM Ionela Voinescu <ionela.voinescu@arm.com> wrote:
>
> Add the rebuild_sched_domains_energy() function to wrap the functionality
> that rebuilds the scheduling domains if any of the Energy Aware Scheduling
> (EAS) initialisation conditions change. This functionality is used when
> schedutil is added or removed or when EAS is enabled or disabled
> through the sched_energy_aware sysctl.
>
> Therefore, create a single function that is used in both these cases and
> that can be later reused.
>
> Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Acked-by: Quentin Perret <qperret@google.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
> Cc: Viresh Kumar <viresh.kumar@linaro.org>

For the schedutil part:

Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

and I'm assuming the patch to be taken care of by Peter.

> ---
>  include/linux/sched/topology.h   |  8 ++++++++
>  kernel/sched/cpufreq_schedutil.c |  9 +--------
>  kernel/sched/topology.c          | 18 +++++++++++-------
>  3 files changed, 20 insertions(+), 15 deletions(-)
>
> diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
> index 9ef7bf686a9f..8f0f778b7c91 100644
> --- a/include/linux/sched/topology.h
> +++ b/include/linux/sched/topology.h
> @@ -225,6 +225,14 @@ static inline bool cpus_share_cache(int this_cpu, int that_cpu)
>
>  #endif /* !CONFIG_SMP */
>
> +#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_CPU_FREQ_GOV_SCHEDUTIL)
> +extern void rebuild_sched_domains_energy(void);
> +#else
> +static inline void rebuild_sched_domains_energy(void)
> +{
> +}
> +#endif
> +
>  #ifndef arch_scale_cpu_capacity
>  /**
>   * arch_scale_cpu_capacity - get the capacity scale factor of a given CPU.
> diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
> index e254745a82cb..37b303890336 100644
> --- a/kernel/sched/cpufreq_schedutil.c
> +++ b/kernel/sched/cpufreq_schedutil.c
> @@ -899,16 +899,9 @@ struct cpufreq_governor *cpufreq_default_governor(void)
>  cpufreq_governor_init(schedutil_gov);
>
>  #ifdef CONFIG_ENERGY_MODEL
> -extern bool sched_energy_update;
> -extern struct mutex sched_energy_mutex;
> -
>  static void rebuild_sd_workfn(struct work_struct *work)
>  {
> -       mutex_lock(&sched_energy_mutex);
> -       sched_energy_update = true;
> -       rebuild_sched_domains();
> -       sched_energy_update = false;
> -       mutex_unlock(&sched_energy_mutex);
> +       rebuild_sched_domains_energy();
>  }
>  static DECLARE_WORK(rebuild_sd_work, rebuild_sd_workfn);
>
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index dd7770226086..270bafb73506 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -211,6 +211,15 @@ unsigned int sysctl_sched_energy_aware = 1;
>  DEFINE_MUTEX(sched_energy_mutex);
>  bool sched_energy_update;
>
> +void rebuild_sched_domains_energy(void)
> +{
> +       mutex_lock(&sched_energy_mutex);
> +       sched_energy_update = true;
> +       rebuild_sched_domains();
> +       sched_energy_update = false;
> +       mutex_unlock(&sched_energy_mutex);
> +}
> +
>  #ifdef CONFIG_PROC_SYSCTL
>  int sched_energy_aware_handler(struct ctl_table *table, int write,
>                 void *buffer, size_t *lenp, loff_t *ppos)
> @@ -223,13 +232,8 @@ int sched_energy_aware_handler(struct ctl_table *table, int write,
>         ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
>         if (!ret && write) {
>                 state = static_branch_unlikely(&sched_energy_present);
> -               if (state != sysctl_sched_energy_aware) {
> -                       mutex_lock(&sched_energy_mutex);
> -                       sched_energy_update = 1;
> -                       rebuild_sched_domains();
> -                       sched_energy_update = 0;
> -                       mutex_unlock(&sched_energy_mutex);
> -               }
> +               if (state != sysctl_sched_energy_aware)
> +                       rebuild_sched_domains_energy();
>         }
>
>         return ret;
> --
> 2.17.1
>

Re: [PATCH RESEND v2 1/3] sched/topology,schedutil: wrap sched domains rebuild

[Ionela Voinescu]

Hi Rafael,

On Friday 30 Oct 2020 at 16:26:32 (+0100), Rafael J. Wysocki wrote:
> On Tue, Oct 27, 2020 at 7:08 PM Ionela Voinescu <ionela.voinescu@arm.com> wrote:
> >
> > Add the rebuild_sched_domains_energy() function to wrap the functionality
> > that rebuilds the scheduling domains if any of the Energy Aware Scheduling
> > (EAS) initialisation conditions change. This functionality is used when
> > schedutil is added or removed or when EAS is enabled or disabled
> > through the sched_energy_aware sysctl.
> >
> > Therefore, create a single function that is used in both these cases and
> > that can be later reused.
> >
> > Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
> > Acked-by: Quentin Perret <qperret@google.com>
> > Cc: Ingo Molnar <mingo@redhat.com>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
> > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> 
> For the schedutil part:
> 
> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> 

Thank you for taking a look!

> and I'm assuming the patch to be taken care of by Peter.

I'm hoping that as well. I'll ask some of the arm64 folk to take another
look over 2/3 to make sure that's covered.

Many thanks,
Ionela.

[tip: sched/core] sched/topology,schedutil: Wrap sched domains rebuild

[tip-bot2 for Ionela Voinescu]

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     31f6a8c0a471be7d7d05c93eac50fcb729e79b9d
Gitweb:        https://git.kernel.org/tip/31f6a8c0a471be7d7d05c93eac50fcb729e79b9d
Author:        Ionela Voinescu <ionela.voinescu@arm.com>
AuthorDate:    Tue, 27 Oct 2020 18:07:11 
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Thu, 19 Nov 2020 11:25:47 +01:00

sched/topology,schedutil: Wrap sched domains rebuild

Add the rebuild_sched_domains_energy() function to wrap the functionality
that rebuilds the scheduling domains if any of the Energy Aware Scheduling
(EAS) initialisation conditions change. This functionality is used when
schedutil is added or removed or when EAS is enabled or disabled
through the sched_energy_aware sysctl.

Therefore, create a single function that is used in both these cases and
that can be later reused.

Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Quentin Perret <qperret@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Link: https://lkml.kernel.org/r/20201027180713.7642-2-ionela.voinescu@arm.com
---
 include/linux/sched/topology.h   |  8 ++++++++
 kernel/sched/cpufreq_schedutil.c |  9 +--------
 kernel/sched/topology.c          | 18 +++++++++++-------
 3 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 9ef7bf6..8f0f778 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -225,6 +225,14 @@ static inline bool cpus_share_cache(int this_cpu, int that_cpu)
 
 #endif	/* !CONFIG_SMP */
 
+#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_CPU_FREQ_GOV_SCHEDUTIL)
+extern void rebuild_sched_domains_energy(void);
+#else
+static inline void rebuild_sched_domains_energy(void)
+{
+}
+#endif
+
 #ifndef arch_scale_cpu_capacity
 /**
  * arch_scale_cpu_capacity - get the capacity scale factor of a given CPU.
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index e254745..37b3038 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -899,16 +899,9 @@ struct cpufreq_governor *cpufreq_default_governor(void)
 cpufreq_governor_init(schedutil_gov);
 
 #ifdef CONFIG_ENERGY_MODEL
-extern bool sched_energy_update;
-extern struct mutex sched_energy_mutex;
-
 static void rebuild_sd_workfn(struct work_struct *work)
 {
-	mutex_lock(&sched_energy_mutex);
-	sched_energy_update = true;
-	rebuild_sched_domains();
-	sched_energy_update = false;
-	mutex_unlock(&sched_energy_mutex);
+	rebuild_sched_domains_energy();
 }
 static DECLARE_WORK(rebuild_sd_work, rebuild_sd_workfn);
 
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index b296c1c..04d9ebf 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -211,6 +211,15 @@ unsigned int sysctl_sched_energy_aware = 1;
 DEFINE_MUTEX(sched_energy_mutex);
 bool sched_energy_update;
 
+void rebuild_sched_domains_energy(void)
+{
+	mutex_lock(&sched_energy_mutex);
+	sched_energy_update = true;
+	rebuild_sched_domains();
+	sched_energy_update = false;
+	mutex_unlock(&sched_energy_mutex);
+}
+
 #ifdef CONFIG_PROC_SYSCTL
 int sched_energy_aware_handler(struct ctl_table *table, int write,
 		void *buffer, size_t *lenp, loff_t *ppos)
@@ -223,13 +232,8 @@ int sched_energy_aware_handler(struct ctl_table *table, int write,
 	ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
 	if (!ret && write) {
 		state = static_branch_unlikely(&sched_energy_present);
-		if (state != sysctl_sched_energy_aware) {
-			mutex_lock(&sched_energy_mutex);
-			sched_energy_update = 1;
-			rebuild_sched_domains();
-			sched_energy_update = 0;
-			mutex_unlock(&sched_energy_mutex);
-		}
+		if (state != sysctl_sched_energy_aware)
+			rebuild_sched_domains_energy();
 	}
 
 	return ret;

[PATCH RESEND v2 2/3] arm64: rebuild sched domains on invariance status changes

[Ionela Voinescu]

Task scheduler behavior depends on frequency invariance (FI) support and
the resulting invariant load tracking signals. For example, in order to
make accurate predictions across CPUs for all performance states, Energy
Aware Scheduling (EAS) needs frequency-invariant load tracking signals
and therefore it has a direct dependency on FI. This dependency is known,
but EAS enablement is not yet conditioned on the presence of FI during
the built of the scheduling domain hierarchy.

Before this is done, the following must be considered: while
arch_scale_freq_invariant() will see changes in FI support and could
be used to condition the use of EAS, it could return different values
during system initialisation.

For arm64, such a scenario will happen for a system that does not support
cpufreq driven FI, but does support counter-driven FI. For such a system,
arch_scale_freq_invariant() will return false if called before counter
based FI initialisation, but change its status to true after it.
If EAS becomes explicitly dependent on FI this would affect the task
scheduler behavior which builds its scheduling domain hierarchy well
before the late counter-based FI init. During that process, EAS would be
disabled due to its dependency on FI.

Two points of future early calls to arch_scale_freq_invariant() which
would determine EAS enablement are:
 - (1) drivers/base/arch_topology.c:126 <<update_topology_flags_workfn>>
		rebuild_sched_domains();
       This will happen after CPU capacity initialisation.
 - (2) kernel/sched/cpufreq_schedutil.c:917 <<rebuild_sd_workfn>>
		rebuild_sched_domains_energy();
		-->rebuild_sched_domains();
       This will happen during sched_cpufreq_governor_change() for the
       schedutil cpufreq governor.

Therefore, before enforcing the presence of FI support for the use of EAS,
ensure the following: if there is a change in FI support status after
counter init, use the existing rebuild_sched_domains_energy() function to
trigger a rebuild of the scheduling and performance domains that in turn
will determine the enablement of EAS.

Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
---
 arch/arm64/kernel/topology.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 543c67cae02f..2a9b69fdabc9 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -213,6 +213,7 @@ static DEFINE_STATIC_KEY_FALSE(amu_fie_key);
 
 static int __init init_amu_fie(void)
 {
+	bool invariance_status = topology_scale_freq_invariant();
 	cpumask_var_t valid_cpus;
 	bool have_policy = false;
 	int ret = 0;
@@ -255,6 +256,15 @@ static int __init init_amu_fie(void)
 	if (!topology_scale_freq_invariant())
 		static_branch_disable(&amu_fie_key);
 
+	/*
+	 * Task scheduler behavior depends on frequency invariance support,
+	 * either cpufreq or counter driven. If the support status changes as
+	 * a result of counter initialisation and use, retrigger the build of
+	 * scheduling domains to ensure the information is propagated properly.
+	 */
+	if (invariance_status != topology_scale_freq_invariant())
+		rebuild_sched_domains_energy();
+
 free_valid_mask:
 	free_cpumask_var(valid_cpus);
 
-- 
2.17.1

Re: [PATCH RESEND v2 2/3] arm64: rebuild sched domains on invariance status changes

[Catalin Marinas]

On Tue, Oct 27, 2020 at 06:07:12PM +0000, Ionela Voinescu wrote:
> Task scheduler behavior depends on frequency invariance (FI) support and
> the resulting invariant load tracking signals. For example, in order to
> make accurate predictions across CPUs for all performance states, Energy
> Aware Scheduling (EAS) needs frequency-invariant load tracking signals
> and therefore it has a direct dependency on FI. This dependency is known,
> but EAS enablement is not yet conditioned on the presence of FI during
> the built of the scheduling domain hierarchy.
> 
> Before this is done, the following must be considered: while
> arch_scale_freq_invariant() will see changes in FI support and could
> be used to condition the use of EAS, it could return different values
> during system initialisation.
> 
> For arm64, such a scenario will happen for a system that does not support
> cpufreq driven FI, but does support counter-driven FI. For such a system,
> arch_scale_freq_invariant() will return false if called before counter
> based FI initialisation, but change its status to true after it.
> If EAS becomes explicitly dependent on FI this would affect the task
> scheduler behavior which builds its scheduling domain hierarchy well
> before the late counter-based FI init. During that process, EAS would be
> disabled due to its dependency on FI.
> 
> Two points of future early calls to arch_scale_freq_invariant() which
> would determine EAS enablement are:
>  - (1) drivers/base/arch_topology.c:126 <<update_topology_flags_workfn>>
> 		rebuild_sched_domains();
>        This will happen after CPU capacity initialisation.
>  - (2) kernel/sched/cpufreq_schedutil.c:917 <<rebuild_sd_workfn>>
> 		rebuild_sched_domains_energy();
> 		-->rebuild_sched_domains();
>        This will happen during sched_cpufreq_governor_change() for the
>        schedutil cpufreq governor.
> 
> Therefore, before enforcing the presence of FI support for the use of EAS,
> ensure the following: if there is a change in FI support status after
> counter init, use the existing rebuild_sched_domains_energy() function to
> trigger a rebuild of the scheduling and performance domains that in turn
> will determine the enablement of EAS.
> 
> Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

[tip: sched/core] arm64: Rebuild sched domains on invariance status changes

[tip-bot2 for Ionela Voinescu]

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     ecec9e86d1a366f97c827ab4a8134ec06ccf031a
Gitweb:        https://git.kernel.org/tip/ecec9e86d1a366f97c827ab4a8134ec06ccf031a
Author:        Ionela Voinescu <ionela.voinescu@arm.com>
AuthorDate:    Tue, 27 Oct 2020 18:07:12 
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Thu, 19 Nov 2020 11:25:47 +01:00

arm64: Rebuild sched domains on invariance status changes

Task scheduler behavior depends on frequency invariance (FI) support and
the resulting invariant load tracking signals. For example, in order to
make accurate predictions across CPUs for all performance states, Energy
Aware Scheduling (EAS) needs frequency-invariant load tracking signals
and therefore it has a direct dependency on FI. This dependency is known,
but EAS enablement is not yet conditioned on the presence of FI during
the built of the scheduling domain hierarchy.

Before this is done, the following must be considered: while
arch_scale_freq_invariant() will see changes in FI support and could
be used to condition the use of EAS, it could return different values
during system initialisation.

For arm64, such a scenario will happen for a system that does not support
cpufreq driven FI, but does support counter-driven FI. For such a system,
arch_scale_freq_invariant() will return false if called before counter
based FI initialisation, but change its status to true after it.
If EAS becomes explicitly dependent on FI this would affect the task
scheduler behavior which builds its scheduling domain hierarchy well
before the late counter-based FI init. During that process, EAS would be
disabled due to its dependency on FI.

Two points of future early calls to arch_scale_freq_invariant() which
would determine EAS enablement are:
 - (1) drivers/base/arch_topology.c:126 <<update_topology_flags_workfn>>
		rebuild_sched_domains();
       This will happen after CPU capacity initialisation.
 - (2) kernel/sched/cpufreq_schedutil.c:917 <<rebuild_sd_workfn>>
		rebuild_sched_domains_energy();
		-->rebuild_sched_domains();
       This will happen during sched_cpufreq_governor_change() for the
       schedutil cpufreq governor.

Therefore, before enforcing the presence of FI support for the use of EAS,
ensure the following: if there is a change in FI support status after
counter init, use the existing rebuild_sched_domains_energy() function to
trigger a rebuild of the scheduling and performance domains that in turn
will determine the enablement of EAS.

Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lkml.kernel.org/r/20201027180713.7642-3-ionela.voinescu@arm.com
---
 arch/arm64/kernel/topology.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 543c67c..2a9b69f 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -213,6 +213,7 @@ static DEFINE_STATIC_KEY_FALSE(amu_fie_key);
 
 static int __init init_amu_fie(void)
 {
+	bool invariance_status = topology_scale_freq_invariant();
 	cpumask_var_t valid_cpus;
 	bool have_policy = false;
 	int ret = 0;
@@ -255,6 +256,15 @@ static int __init init_amu_fie(void)
 	if (!topology_scale_freq_invariant())
 		static_branch_disable(&amu_fie_key);
 
+	/*
+	 * Task scheduler behavior depends on frequency invariance support,
+	 * either cpufreq or counter driven. If the support status changes as
+	 * a result of counter initialisation and use, retrigger the build of
+	 * scheduling domains to ensure the information is propagated properly.
+	 */
+	if (invariance_status != topology_scale_freq_invariant())
+		rebuild_sched_domains_energy();
+
 free_valid_mask:
 	free_cpumask_var(valid_cpus);
 

[PATCH RESEND v2 3/3] sched/topology: condition EAS enablement on FIE support

[Ionela Voinescu]

In order to make accurate predictions across CPUs and for all performance
states, Energy Aware Scheduling (EAS) needs frequency-invariant load
tracking signals.

EAS task placement aims to minimize energy consumption, and does so in
part by limiting the search space to only CPUs with the highest spare
capacity (CPU capacity - CPU utilization) in their performance domain.
Those candidates are the placement choices that will keep frequency at
its lowest possible and therefore save the most energy.

But without frequency invariance, a CPU's utilization is relative to the
CPU's current performance level, and not relative to its maximum
performance level, which determines its capacity. As a result, it will
fail to correctly indicate any potential spare capacity obtained by an
increase in a CPU's performance level. Therefore, a non-invariant
utilization signal would render the EAS task placement logic invalid.

Now that we properly report support for the Frequency Invariance Engine
(FIE) through arch_scale_freq_invariant() for arm and arm64 systems,
while also ensuring a re-evaluation of the EAS use conditions for
possible invariance status change, we can assert this is the case when
initializing EAS. Warn and bail out otherwise.

Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Suggested-by: Quentin Perret <qperret@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
---
 kernel/sched/topology.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 270bafb73506..5a1580ef1264 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -328,6 +328,7 @@ static void sched_energy_set(bool has_eas)
  *    3. no SMT is detected.
  *    4. the EM complexity is low enough to keep scheduling overheads low;
  *    5. schedutil is driving the frequency of all CPUs of the rd;
+ *    6. frequency invariance support is present;
  *
  * The complexity of the Energy Model is defined as:
  *
@@ -376,6 +377,14 @@ static bool build_perf_domains(const struct cpumask *cpu_map)
 		goto free;
 	}
 
+	if (!arch_scale_freq_invariant()) {
+		if (sched_debug()) {
+			pr_warn("rd %*pbl: Disabling EAS: frequency-invariant load tracking not yet supported",
+				cpumask_pr_args(cpu_map));
+		}
+		goto free;
+	}
+
 	for_each_cpu(i, cpu_map) {
 		/* Skip already covered CPUs. */
 		if (find_pd(pd, i))
-- 
2.17.1

[tip: sched/core] sched/topology: Condition EAS enablement on FIE support

[tip-bot2 for Ionela Voinescu]

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     fa50e2b452c60cff9f4000de5b372a61d6695c26
Gitweb:        https://git.kernel.org/tip/fa50e2b452c60cff9f4000de5b372a61d6695c26
Author:        Ionela Voinescu <ionela.voinescu@arm.com>
AuthorDate:    Tue, 27 Oct 2020 18:07:13 
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Thu, 19 Nov 2020 11:25:47 +01:00

sched/topology: Condition EAS enablement on FIE support

In order to make accurate predictions across CPUs and for all performance
states, Energy Aware Scheduling (EAS) needs frequency-invariant load
tracking signals.

EAS task placement aims to minimize energy consumption, and does so in
part by limiting the search space to only CPUs with the highest spare
capacity (CPU capacity - CPU utilization) in their performance domain.
Those candidates are the placement choices that will keep frequency at
its lowest possible and therefore save the most energy.

But without frequency invariance, a CPU's utilization is relative to the
CPU's current performance level, and not relative to its maximum
performance level, which determines its capacity. As a result, it will
fail to correctly indicate any potential spare capacity obtained by an
increase in a CPU's performance level. Therefore, a non-invariant
utilization signal would render the EAS task placement logic invalid.

Now that we properly report support for the Frequency Invariance Engine
(FIE) through arch_scale_freq_invariant() for arm and arm64 systems,
while also ensuring a re-evaluation of the EAS use conditions for
possible invariance status change, we can assert this is the case when
initializing EAS. Warn and bail out otherwise.

Suggested-by: Quentin Perret <qperret@google.com>
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20201027180713.7642-4-ionela.voinescu@arm.com
---
 kernel/sched/topology.c |  9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 04d9ebf..5d3675c 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -328,6 +328,7 @@ static void sched_energy_set(bool has_eas)
  *    3. no SMT is detected.
  *    4. the EM complexity is low enough to keep scheduling overheads low;
  *    5. schedutil is driving the frequency of all CPUs of the rd;
+ *    6. frequency invariance support is present;
  *
  * The complexity of the Energy Model is defined as:
  *
@@ -376,6 +377,14 @@ static bool build_perf_domains(const struct cpumask *cpu_map)
 		goto free;
 	}
 
+	if (!arch_scale_freq_invariant()) {
+		if (sched_debug()) {
+			pr_warn("rd %*pbl: Disabling EAS: frequency-invariant load tracking not yet supported",
+				cpumask_pr_args(cpu_map));
+		}
+		goto free;
+	}
+
 	for_each_cpu(i, cpu_map) {
 		/* Skip already covered CPUs. */
 		if (find_pd(pd, i))

Re: [PATCH RESEND v2 0/3] condition EAS enablement on FI support

[Ionela Voinescu]

Hi guys,

On Tuesday 27 Oct 2020 at 18:07:10 (+0000), Ionela Voinescu wrote:
> Given the maturity gained by cpufreq-based Frequency Invariance (FI)
> support following the patches at [1], this series conditions Energy
> Aware Scheduling (EAS) enablement on a frequency invariant system.
> 
> Currently, EAS can be enabled on a system without FI support, leading
> to incorrect (energy-wise) task placements. As no warning is emitted,
> it could take some debugging effort to track the behavior back to the
> lack of FI support; this series changes that by disabling EAS
> (and advertising it) when FI support is missing.
> 
> The series is structured as follows:
>  - 1/3 - create function that can rebuild the scheduling and EAS'
>    performance domains if EAS' initial conditions change
>  - 2/3 - arm64: rebuild scheduling and performance domains in the
>          case of late, counter-driven FI initialisation.
>  - 3/3 - condition EAS enablement on FI support
> 
> RESEND v2: rebase and retest on v5.10-rc1
> 

This still applies nicely on v5.10-rc4.

Peter, given that 1/3 and 2/3 are acked, do you think there's anything
else that needs to be done for this?

Thank you,
Ionela.

> v1 -> v2:
>  - v1 can be found at [2]
>  - 1/2 - moved now "static inline" rebuild_sched_domains_energy() stub
>    in sched/topology.h header, as suggested by Quentin.
>  - Changed the order of 2/3 and 3/3 to maintain bisection as suggested
>    by Quentin.
>  - 3/3 - place the warning under sched_debug() as per Quentin and
>    Dietmar's recommendations.
> 
> [1] Most recent version at:
> https://lore.kernel.org/lkml/20200901205549.30096-1-ionela.voinescu@arm.com/
> [2] https://lore.kernel.org/lkml/20200924123937.20938-1-ionela.voinescu@arm.com/
> 
> Many thanks,
> Ionela.
> 
> Ionela Voinescu (3):
>   sched/topology,schedutil: wrap sched domains rebuild
>   arm64: rebuild sched domains on invariance status changes
>   sched/topology: condition EAS enablement on FIE support
> 
>  arch/arm64/kernel/topology.c     | 10 ++++++++++
>  include/linux/sched/topology.h   |  8 ++++++++
>  kernel/sched/cpufreq_schedutil.c |  9 +--------
>  kernel/sched/topology.c          | 27 ++++++++++++++++++++-------
>  4 files changed, 39 insertions(+), 15 deletions(-)
> 
> 
> base-commit: 3650b228f83adda7e5ee532e2b90429c03f7b9ec
> -- 
> 2.17.1
> 

Re: [PATCH RESEND v2 0/3] condition EAS enablement on FI support

[Peter Zijlstra]

On Wed, Nov 18, 2020 at 11:42:34AM +0000, Ionela Voinescu wrote:
> Hi guys,
> 
> On Tuesday 27 Oct 2020 at 18:07:10 (+0000), Ionela Voinescu wrote:
> > Given the maturity gained by cpufreq-based Frequency Invariance (FI)
> > support following the patches at [1], this series conditions Energy
> > Aware Scheduling (EAS) enablement on a frequency invariant system.
> > 
> > Currently, EAS can be enabled on a system without FI support, leading
> > to incorrect (energy-wise) task placements. As no warning is emitted,
> > it could take some debugging effort to track the behavior back to the
> > lack of FI support; this series changes that by disabling EAS
> > (and advertising it) when FI support is missing.
> > 
> > The series is structured as follows:
> >  - 1/3 - create function that can rebuild the scheduling and EAS'
> >    performance domains if EAS' initial conditions change
> >  - 2/3 - arm64: rebuild scheduling and performance domains in the
> >          case of late, counter-driven FI initialisation.
> >  - 3/3 - condition EAS enablement on FI support
> > 
> > RESEND v2: rebase and retest on v5.10-rc1
> > 
> 
> This still applies nicely on v5.10-rc4.
> 
> Peter, given that 1/3 and 2/3 are acked, do you think there's anything
> else that needs to be done for this?

I'll take it, thanks!