[PATCH V6 0/4] cpufreq: cppc: Add support for frequency invariance

[PATCH V6 0/4] cpufreq: cppc: Add support for frequency invariance

[Viresh Kumar]

Hello,

CPPC cpufreq driver is used for ARM servers and this patch series tries
to provide counter-based frequency invariance support for them in the
absence for architecture specific counters (like AMUs).

This is tested by:
- Vincent Guittot on ThunderX2.
- Ionela Voinescu on Juno R2.
- /me with hacks on Hikey, as I don't have access to the right hardware.

This is based of 5.12-rc2. I will merge these via the arm-cpufreq tree
directly.

Changes since V5:
- New patch to rename freq_scale to arch_freq_scale (Will Deacon).
- Separate patch to export arch_freq_scale and helpers (Will Deacon).
- Some improvements in the last patch like commit log, moving more stuff
  to policy init, new fie_disabled flag, etc. (Ionela Voinescu).
- Added Reviewed/Acked/Tested-by tags.

Changes since V4:
- Move some code to policy specific initialization for cppc driver.
- Initialize kthread specific stuff only once in cppc driver.
- Added a kerneldoc comment in cppc driver and improved changelog as
  well.

Changes since V3:
- rebuild_sched_domains_energy() stuff moved from arm64 to drivers/base.
- Added Reviewed/Tested-by Ionela for the first patch.
- Remove unused max_freq field from structure in cppc driver.
- s/cppc_f_i/cppc_freq_inv.
- Fix an per-cpu access, there was a bug in earlier version.
- Create a single kthread which can run on any CPU and takes care of
  work from all the CPUs.
- Do the whole FIE thing under a new CONFIG option for cppc driver.
- Few minor improvements.

Changes since V2:
- Not sending as an RFC anymore.
- Several renames, reordering of code in 1/2 based on Ionela's comments.
- Several rebase changes for 2/2.
- The freq_scale calculations are optimized a bit.
- Better overall commenting and commit logs.

Changes since V1:
- The interface for setting the callbacks is improved, so different
  parts looking to provide their callbacks don't need to think about
  each other.

- Moved to per-cpu storage for storing the callback related data, AMU
  counters have higher priority with this.

--
Viresh

Viresh Kumar (4):
  arch_topology: Rename freq_scale as arch_freq_scale
  arch_topology: Allow multiple entities to provide sched_freq_tick()
    callback
  arch_topology: Export arch_freq_scale and helpers
  cpufreq: CPPC: Add support for frequency invariance

 arch/arm64/include/asm/topology.h |  10 +-
 arch/arm64/kernel/topology.c      | 109 +++++--------
 drivers/base/arch_topology.c      |  89 ++++++++++-
 drivers/cpufreq/Kconfig.arm       |  10 ++
 drivers/cpufreq/cppc_cpufreq.c    | 245 ++++++++++++++++++++++++++++--
 include/linux/arch_topology.h     |  19 ++-
 kernel/sched/core.c               |   1 +
 7 files changed, 385 insertions(+), 98 deletions(-)


base-commit: a38fd8748464831584a19438cbb3082b5a2dab15
-- 
2.25.0.rc1.19.g042ed3e048af

[PATCH V6 1/4] arch_topology: Rename freq_scale as arch_freq_scale

[Viresh Kumar]

Rename freq_scale to a less generic name, as it will get exported soon
for modules. Since x86 already names its own implementation of this as
arch_freq_scale, lets stick to that.

Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
 arch/arm64/kernel/topology.c  | 6 +++---
 drivers/base/arch_topology.c  | 4 ++--
 include/linux/arch_topology.h | 4 ++--
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index e08a4126453a..aa8d893619ed 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -256,8 +256,8 @@ static int init_amu_fie_callback(struct notifier_block *nb, unsigned long val,
 	 * initialized AMU support and enabled invariance. The AMU counters will
 	 * keep on working just fine in the absence of the cpufreq driver, and
 	 * for the CPUs for which there are no counters available, the last set
-	 * value of freq_scale will remain valid as that is the frequency those
-	 * CPUs are running at.
+	 * value of arch_freq_scale will remain valid as that is the frequency
+	 * those CPUs are running at.
 	 */
 
 	return 0;
@@ -327,7 +327,7 @@ void topology_scale_freq_tick(void)
 			  const_cnt - prev_const_cnt);
 
 	scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE);
-	this_cpu_write(freq_scale, (unsigned long)scale);
+	this_cpu_write(arch_freq_scale, (unsigned long)scale);
 }
 
 #ifdef CONFIG_ACPI_CPPC_LIB
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index de8587cc119e..2a1cecbde0a4 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -31,7 +31,7 @@ __weak bool arch_freq_counters_available(const struct cpumask *cpus)
 {
 	return false;
 }
-DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE;
+DEFINE_PER_CPU(unsigned long, arch_freq_scale) = SCHED_CAPACITY_SCALE;
 
 void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
 			     unsigned long max_freq)
@@ -53,7 +53,7 @@ void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
 	scale = (cur_freq << SCHED_CAPACITY_SHIFT) / max_freq;
 
 	for_each_cpu(i, cpus)
-		per_cpu(freq_scale, i) = scale;
+		per_cpu(arch_freq_scale, i) = scale;
 }
 
 DEFINE_PER_CPU(unsigned long, cpu_scale) = SCHED_CAPACITY_SCALE;
diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
index 0f6cd6b73a61..583af517f123 100644
--- a/include/linux/arch_topology.h
+++ b/include/linux/arch_topology.h
@@ -23,11 +23,11 @@ static inline unsigned long topology_get_cpu_scale(int cpu)
 
 void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity);
 
-DECLARE_PER_CPU(unsigned long, freq_scale);
+DECLARE_PER_CPU(unsigned long, arch_freq_scale);
 
 static inline unsigned long topology_get_freq_scale(int cpu)
 {
-	return per_cpu(freq_scale, cpu);
+	return per_cpu(arch_freq_scale, cpu);
 }
 
 void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
-- 
2.25.0.rc1.19.g042ed3e048af

[PATCH V6 2/4] arch_topology: Allow multiple entities to provide sched_freq_tick() callback

[Viresh Kumar]

This patch attempts to make it generic enough so other parts of the
kernel can also provide their own implementation of scale_freq_tick()
callback, which is called by the scheduler periodically to update the
per-cpu arch_freq_scale variable.

The implementations now need to provide 'struct scale_freq_data' for the
CPUs for which they have hardware counters available, and a callback
gets registered for each possible CPU in a per-cpu variable.

The arch specific (or ARM AMU) counters are updated to adapt to this and
they take the highest priority if they are available, i.e. they will be
used instead of CPPC based counters for example.

The special code to rebuild the sched domains, in case invariance status
change for the system, is moved out of arm64 specific code and is added
to arch_topology.c.

Note that this also defines SCALE_FREQ_SOURCE_CPUFREQ but doesn't use it
and it is added to show that cpufreq is also acts as source of
information for FIE and will be used by default if no other counters are
supported for a platform.

Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Ionela Voinescu <ionela.voinescu@arm.com>
Acked-by: Will Deacon <will@kernel.org> # for arm64
Tested-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
 arch/arm64/include/asm/topology.h |  10 +--
 arch/arm64/kernel/topology.c      | 105 +++++++++++-------------------
 drivers/base/arch_topology.c      |  82 +++++++++++++++++++++--
 include/linux/arch_topology.h     |  14 +++-
 4 files changed, 131 insertions(+), 80 deletions(-)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index 3b8dca4eb08d..ec2db3419c41 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -17,17 +17,9 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
-void topology_scale_freq_tick(void);
-
-#ifdef CONFIG_ARM64_AMU_EXTN
-/*
- * Replace task scheduler's default counter-based
- * frequency-invariance scale factor setting.
- */
-#define arch_scale_freq_tick topology_scale_freq_tick
-#endif /* CONFIG_ARM64_AMU_EXTN */
 
 /* Replace task scheduler's default frequency-invariant accounting */
+#define arch_scale_freq_tick topology_scale_freq_tick
 #define arch_set_freq_scale topology_set_freq_scale
 #define arch_scale_freq_capacity topology_get_freq_scale
 #define arch_scale_freq_invariant topology_scale_freq_invariant
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index aa8d893619ed..4dd14a6620c1 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -199,12 +199,47 @@ static int freq_inv_set_max_ratio(int cpu, u64 max_rate, u64 ref_rate)
 	return 0;
 }
 
-static DEFINE_STATIC_KEY_FALSE(amu_fie_key);
-#define amu_freq_invariant() static_branch_unlikely(&amu_fie_key)
+static void amu_scale_freq_tick(void)
+{
+	u64 prev_core_cnt, prev_const_cnt;
+	u64 core_cnt, const_cnt, scale;
+
+	prev_const_cnt = this_cpu_read(arch_const_cycles_prev);
+	prev_core_cnt = this_cpu_read(arch_core_cycles_prev);
+
+	update_freq_counters_refs();
+
+	const_cnt = this_cpu_read(arch_const_cycles_prev);
+	core_cnt = this_cpu_read(arch_core_cycles_prev);
+
+	if (unlikely(core_cnt <= prev_core_cnt ||
+		     const_cnt <= prev_const_cnt))
+		return;
+
+	/*
+	 *	    /\core    arch_max_freq_scale
+	 * scale =  ------- * --------------------
+	 *	    /\const   SCHED_CAPACITY_SCALE
+	 *
+	 * See validate_cpu_freq_invariance_counters() for details on
+	 * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT.
+	 */
+	scale = core_cnt - prev_core_cnt;
+	scale *= this_cpu_read(arch_max_freq_scale);
+	scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT,
+			  const_cnt - prev_const_cnt);
+
+	scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE);
+	this_cpu_write(arch_freq_scale, (unsigned long)scale);
+}
+
+static struct scale_freq_data amu_sfd = {
+	.source = SCALE_FREQ_SOURCE_ARCH,
+	.set_freq_scale = amu_scale_freq_tick,
+};
 
 static void amu_fie_setup(const struct cpumask *cpus)
 {
-	bool invariant;
 	int cpu;
 
 	/* We are already set since the last insmod of cpufreq driver */
@@ -221,25 +256,10 @@ static void amu_fie_setup(const struct cpumask *cpus)
 
 	cpumask_or(amu_fie_cpus, amu_fie_cpus, cpus);
 
-	invariant = topology_scale_freq_invariant();
-
-	/* We aren't fully invariant yet */
-	if (!invariant && !cpumask_equal(amu_fie_cpus, cpu_present_mask))
-		return;
-
-	static_branch_enable(&amu_fie_key);
+	topology_set_scale_freq_source(&amu_sfd, amu_fie_cpus);
 
 	pr_debug("CPUs[%*pbl]: counters will be used for FIE.",
 		 cpumask_pr_args(cpus));
-
-	/*
-	 * Task scheduler behavior depends on frequency invariance support,
-	 * either cpufreq or counter driven. If the support status changes as
-	 * a result of counter initialisation and use, retrigger the build of
-	 * scheduling domains to ensure the information is propagated properly.
-	 */
-	if (!invariant)
-		rebuild_sched_domains_energy();
 }
 
 static int init_amu_fie_callback(struct notifier_block *nb, unsigned long val,
@@ -283,53 +303,6 @@ static int __init init_amu_fie(void)
 }
 core_initcall(init_amu_fie);
 
-bool arch_freq_counters_available(const struct cpumask *cpus)
-{
-	return amu_freq_invariant() &&
-	       cpumask_subset(cpus, amu_fie_cpus);
-}
-
-void topology_scale_freq_tick(void)
-{
-	u64 prev_core_cnt, prev_const_cnt;
-	u64 core_cnt, const_cnt, scale;
-	int cpu = smp_processor_id();
-
-	if (!amu_freq_invariant())
-		return;
-
-	if (!cpumask_test_cpu(cpu, amu_fie_cpus))
-		return;
-
-	prev_const_cnt = this_cpu_read(arch_const_cycles_prev);
-	prev_core_cnt = this_cpu_read(arch_core_cycles_prev);
-
-	update_freq_counters_refs();
-
-	const_cnt = this_cpu_read(arch_const_cycles_prev);
-	core_cnt = this_cpu_read(arch_core_cycles_prev);
-
-	if (unlikely(core_cnt <= prev_core_cnt ||
-		     const_cnt <= prev_const_cnt))
-		return;
-
-	/*
-	 *	    /\core    arch_max_freq_scale
-	 * scale =  ------- * --------------------
-	 *	    /\const   SCHED_CAPACITY_SCALE
-	 *
-	 * See validate_cpu_freq_invariance_counters() for details on
-	 * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT.
-	 */
-	scale = core_cnt - prev_core_cnt;
-	scale *= this_cpu_read(arch_max_freq_scale);
-	scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT,
-			  const_cnt - prev_const_cnt);
-
-	scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE);
-	this_cpu_write(arch_freq_scale, (unsigned long)scale);
-}
-
 #ifdef CONFIG_ACPI_CPPC_LIB
 #include <acpi/cppc_acpi.h>
 
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 2a1cecbde0a4..ebcd2ea3091f 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -21,16 +21,90 @@
 #include <linux/sched.h>
 #include <linux/smp.h>
 
+static DEFINE_PER_CPU(struct scale_freq_data *, sft_data);
+static struct cpumask scale_freq_counters_mask;
+static bool scale_freq_invariant;
+
+static bool supports_scale_freq_counters(const struct cpumask *cpus)
+{
+	return cpumask_subset(cpus, &scale_freq_counters_mask);
+}
+
 bool topology_scale_freq_invariant(void)
 {
 	return cpufreq_supports_freq_invariance() ||
-	       arch_freq_counters_available(cpu_online_mask);
+	       supports_scale_freq_counters(cpu_online_mask);
 }
 
-__weak bool arch_freq_counters_available(const struct cpumask *cpus)
+static void update_scale_freq_invariant(bool status)
 {
-	return false;
+	if (scale_freq_invariant == status)
+		return;
+
+	/*
+	 * Task scheduler behavior depends on frequency invariance support,
+	 * either cpufreq or counter driven. If the support status changes as
+	 * a result of counter initialisation and use, retrigger the build of
+	 * scheduling domains to ensure the information is propagated properly.
+	 */
+	if (topology_scale_freq_invariant() == status) {
+		scale_freq_invariant = status;
+		rebuild_sched_domains_energy();
+	}
 }
+
+void topology_set_scale_freq_source(struct scale_freq_data *data,
+				    const struct cpumask *cpus)
+{
+	struct scale_freq_data *sfd;
+	int cpu;
+
+	/*
+	 * Avoid calling rebuild_sched_domains() unnecessarily if FIE is
+	 * supported by cpufreq.
+	 */
+	if (cpumask_empty(&scale_freq_counters_mask))
+		scale_freq_invariant = topology_scale_freq_invariant();
+
+	for_each_cpu(cpu, cpus) {
+		sfd = per_cpu(sft_data, cpu);
+
+		/* Use ARCH provided counters whenever possible */
+		if (!sfd || sfd->source != SCALE_FREQ_SOURCE_ARCH) {
+			per_cpu(sft_data, cpu) = data;
+			cpumask_set_cpu(cpu, &scale_freq_counters_mask);
+		}
+	}
+
+	update_scale_freq_invariant(true);
+}
+
+void topology_clear_scale_freq_source(enum scale_freq_source source,
+				      const struct cpumask *cpus)
+{
+	struct scale_freq_data *sfd;
+	int cpu;
+
+	for_each_cpu(cpu, cpus) {
+		sfd = per_cpu(sft_data, cpu);
+
+		if (sfd && sfd->source == source) {
+			per_cpu(sft_data, cpu) = NULL;
+			cpumask_clear_cpu(cpu, &scale_freq_counters_mask);
+		}
+	}
+
+	update_scale_freq_invariant(false);
+}
+
+void topology_scale_freq_tick(void)
+{
+	struct scale_freq_data *sfd = *this_cpu_ptr(&sft_data);
+
+	if (sfd)
+		sfd->set_freq_scale();
+}
+
 DEFINE_PER_CPU(unsigned long, arch_freq_scale) = SCHED_CAPACITY_SCALE;
 
 void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
@@ -47,7 +121,7 @@ void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
 	 * want to update the scale factor with information from CPUFREQ.
 	 * Instead the scale factor will be updated from arch_scale_freq_tick.
 	 */
-	if (arch_freq_counters_available(cpus))
+	if (supports_scale_freq_counters(cpus))
 		return;
 
 	scale = (cur_freq << SCHED_CAPACITY_SHIFT) / max_freq;
diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
index 583af517f123..11e555cfaecb 100644
--- a/include/linux/arch_topology.h
+++ b/include/linux/arch_topology.h
@@ -34,7 +34,19 @@ void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
 			     unsigned long max_freq);
 bool topology_scale_freq_invariant(void);
 
-bool arch_freq_counters_available(const struct cpumask *cpus);
+enum scale_freq_source {
+	SCALE_FREQ_SOURCE_CPUFREQ = 0,
+	SCALE_FREQ_SOURCE_ARCH,
+};
+
+struct scale_freq_data {
+	enum scale_freq_source source;
+	void (*set_freq_scale)(void);
+};
+
+void topology_scale_freq_tick(void);
+void topology_set_scale_freq_source(struct scale_freq_data *data, const struct cpumask *cpus);
+void topology_clear_scale_freq_source(enum scale_freq_source source, const struct cpumask *cpus);
 
 DECLARE_PER_CPU(unsigned long, thermal_pressure);
 
-- 
2.25.0.rc1.19.g042ed3e048af

[PATCH V6 3/4] arch_topology: Export arch_freq_scale and helpers

[Viresh Kumar]

It is possible now for other parts of the kernel to provide their own
implementation of sched_freq_tick() and they can very well be modules
themselves (like CPPC cpufreq driver, which is going to use these in a
later commit).

Export arch_freq_scale and topology_{set|clear}_scale_freq_source().

Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
 drivers/base/arch_topology.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index ebcd2ea3091f..995e52b9eca4 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -78,6 +78,7 @@ void topology_set_scale_freq_source(struct scale_freq_data *data,
 
 	update_scale_freq_invariant(true);
 }
+EXPORT_SYMBOL_GPL(topology_set_scale_freq_source);
 
 void topology_clear_scale_freq_source(enum scale_freq_source source,
 				      const struct cpumask *cpus)
@@ -96,6 +97,7 @@ void topology_clear_scale_freq_source(enum scale_freq_source source,
 
 	update_scale_freq_invariant(false);
 }
+EXPORT_SYMBOL_GPL(topology_clear_scale_freq_source);
 
 void topology_scale_freq_tick(void)
 {
@@ -106,6 +108,7 @@ void topology_scale_freq_tick(void)
 }
 
 DEFINE_PER_CPU(unsigned long, arch_freq_scale) = SCHED_CAPACITY_SCALE;
+EXPORT_SYMBOL_GPL(arch_freq_scale);
 
 void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
 			     unsigned long max_freq)
-- 
2.25.0.rc1.19.g042ed3e048af

[PATCH V6 4/4] cpufreq: CPPC: Add support for frequency invariance

[Viresh Kumar]

The Frequency Invariance Engine (FIE) is providing a frequency scaling
correction factor that helps achieve more accurate load-tracking.

Normally, this scaling factor can be obtained directly with the help of
the cpufreq drivers as they know the exact frequency the hardware is
running at. But that isn't the case for CPPC cpufreq driver.

Another way of obtaining that is using the arch specific counter
support, which is already present in kernel, but that hardware is
optional for platforms.

This patch updates the CPPC driver to register itself with the topology
core to provide its own implementation (cppc_scale_freq_tick()) of
topology_scale_freq_tick() which gets called by the scheduler on every
tick. Note that the arch specific counters have higher priority than
CPPC counters, if available, though the CPPC driver doesn't need to have
any special handling for that.

On an invocation of cppc_scale_freq_tick(), we schedule an irq work
(since we reach here from hard-irq context), which then schedules a
normal work item and cppc_scale_freq_workfn() updates the per_cpu
arch_freq_scale variable based on the counter updates since the last
tick.

To allow platforms to disable this CPPC counter-based frequency
invariance support, this is all done under CONFIG_ACPI_CPPC_CPUFREQ_FIE,
which is enabled by default.

This also exports sched_setattr_nocheck() as the CPPC driver can be
built as a module.

Cc: linux-acpi@vger.kernel.org
Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
 drivers/cpufreq/Kconfig.arm    |  10 ++
 drivers/cpufreq/cppc_cpufreq.c | 245 +++++++++++++++++++++++++++++++--
 include/linux/arch_topology.h  |   1 +
 kernel/sched/core.c            |   1 +
 4 files changed, 245 insertions(+), 12 deletions(-)

diff --git a/drivers/cpufreq/Kconfig.arm b/drivers/cpufreq/Kconfig.arm
index e65e0a43be64..a5c5f70acfc9 100644
--- a/drivers/cpufreq/Kconfig.arm
+++ b/drivers/cpufreq/Kconfig.arm
@@ -19,6 +19,16 @@ config ACPI_CPPC_CPUFREQ
 
 	  If in doubt, say N.
 
+config ACPI_CPPC_CPUFREQ_FIE
+	bool "Frequency Invariance support for CPPC cpufreq driver"
+	depends on ACPI_CPPC_CPUFREQ && GENERIC_ARCH_TOPOLOGY
+	default y
+	help
+	  This extends frequency invariance support in the CPPC cpufreq driver,
+	  by using CPPC delivered and reference performance counters.
+
+	  If in doubt, say N.
+
 config ARM_ALLWINNER_SUN50I_CPUFREQ_NVMEM
 	tristate "Allwinner nvmem based SUN50I CPUFreq driver"
 	depends on ARCH_SUNXI
diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
index 8a482c434ea6..b8e1b8ea628c 100644
--- a/drivers/cpufreq/cppc_cpufreq.c
+++ b/drivers/cpufreq/cppc_cpufreq.c
@@ -10,14 +10,18 @@
 
 #define pr_fmt(fmt)	"CPPC Cpufreq:"	fmt
 
+#include <linux/arch_topology.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/delay.h>
 #include <linux/cpu.h>
 #include <linux/cpufreq.h>
 #include <linux/dmi.h>
+#include <linux/irq_work.h>
+#include <linux/kthread.h>
 #include <linux/time.h>
 #include <linux/vmalloc.h>
+#include <uapi/linux/sched/types.h>
 
 #include <asm/unaligned.h>
 
@@ -57,6 +61,204 @@ static struct cppc_workaround_oem_info wa_info[] = {
 	}
 };
 
+#ifdef CONFIG_ACPI_CPPC_CPUFREQ_FIE
+
+/* Frequency invariance support */
+struct cppc_freq_invariance {
+	int cpu;
+	struct irq_work irq_work;
+	struct kthread_work work;
+	struct cppc_perf_fb_ctrs prev_perf_fb_ctrs;
+	struct cppc_cpudata *cpu_data;
+};
+
+static DEFINE_PER_CPU(struct cppc_freq_invariance, cppc_freq_inv);
+static struct kthread_worker *kworker_fie;
+static bool fie_disabled;
+
+static struct cpufreq_driver cppc_cpufreq_driver;
+static unsigned int hisi_cppc_cpufreq_get_rate(unsigned int cpu);
+static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
+				 struct cppc_perf_fb_ctrs fb_ctrs_t0,
+				 struct cppc_perf_fb_ctrs fb_ctrs_t1);
+
+/**
+ * cppc_scale_freq_workfn - CPPC arch_freq_scale updater for frequency invariance
+ * @work: The work item.
+ *
+ * The CPPC driver register itself with the topology core to provide its own
+ * implementation (cppc_scale_freq_tick()) of topology_scale_freq_tick() which
+ * gets called by the scheduler on every tick.
+ *
+ * Note that the arch specific counters have higher priority than CPPC counters,
+ * if available, though the CPPC driver doesn't need to have any special
+ * handling for that.
+ *
+ * On an invocation of cppc_scale_freq_tick(), we schedule an irq work (since we
+ * reach here from hard-irq context), which then schedules a normal work item
+ * and cppc_scale_freq_workfn() updates the per_cpu arch_freq_scale variable
+ * based on the counter updates since the last tick.
+ */
+static void cppc_scale_freq_workfn(struct kthread_work *work)
+{
+	struct cppc_freq_invariance *cppc_fi;
+	struct cppc_perf_fb_ctrs fb_ctrs = {0};
+	struct cppc_cpudata *cpu_data;
+	unsigned long local_freq_scale;
+	u64 perf;
+
+	cppc_fi = container_of(work, struct cppc_freq_invariance, work);
+	cpu_data = cppc_fi->cpu_data;
+
+	if (cppc_get_perf_ctrs(cppc_fi->cpu, &fb_ctrs)) {
+		pr_warn("%s: failed to read perf counters\n", __func__);
+		return;
+	}
+
+	cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
+	perf = cppc_perf_from_fbctrs(cpu_data, cppc_fi->prev_perf_fb_ctrs,
+				     fb_ctrs);
+
+	perf <<= SCHED_CAPACITY_SHIFT;
+	local_freq_scale = div64_u64(perf, cpu_data->perf_caps.highest_perf);
+	if (WARN_ON(local_freq_scale > 1024))
+		local_freq_scale = 1024;
+
+	per_cpu(arch_freq_scale, cppc_fi->cpu) = local_freq_scale;
+}
+
+static void cppc_irq_work(struct irq_work *irq_work)
+{
+	struct cppc_freq_invariance *cppc_fi;
+
+	cppc_fi = container_of(irq_work, struct cppc_freq_invariance, irq_work);
+	kthread_queue_work(kworker_fie, &cppc_fi->work);
+}
+
+static void cppc_scale_freq_tick(void)
+{
+	struct cppc_freq_invariance *cppc_fi = &per_cpu(cppc_freq_inv, smp_processor_id());
+
+	/*
+	 * cppc_get_perf_ctrs() can potentially sleep, call that from the right
+	 * context.
+	 */
+	irq_work_queue(&cppc_fi->irq_work);
+}
+
+static struct scale_freq_data cppc_sftd = {
+	.source = SCALE_FREQ_SOURCE_CPPC,
+	.set_freq_scale = cppc_scale_freq_tick,
+};
+
+static void cppc_freq_invariance_policy_init(struct cpufreq_policy *policy,
+					     struct cppc_cpudata *cpu_data)
+{
+	struct cppc_perf_fb_ctrs fb_ctrs = {0};
+	struct cppc_freq_invariance *cppc_fi;
+	int i, ret;
+
+	if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
+		return;
+
+	if (fie_disabled)
+		return;
+
+	for_each_cpu(i, policy->cpus) {
+		cppc_fi = &per_cpu(cppc_freq_inv, i);
+		cppc_fi->cpu = i;
+		cppc_fi->cpu_data = cpu_data;
+		kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
+		init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
+
+		ret = cppc_get_perf_ctrs(i, &fb_ctrs);
+		if (ret) {
+			pr_warn("%s: failed to read perf counters: %d\n",
+				__func__, ret);
+			fie_disabled = true;
+		} else {
+			cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
+		}
+	}
+}
+
+static void __init cppc_freq_invariance_init(void)
+{
+	struct sched_attr attr = {
+		.size		= sizeof(struct sched_attr),
+		.sched_policy	= SCHED_DEADLINE,
+		.sched_nice	= 0,
+		.sched_priority	= 0,
+		/*
+		 * Fake (unused) bandwidth; workaround to "fix"
+		 * priority inheritance.
+		 */
+		.sched_runtime	= 1000000,
+		.sched_deadline = 10000000,
+		.sched_period	= 10000000,
+	};
+	int ret;
+
+	if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
+		return;
+
+	if (fie_disabled)
+		return;
+
+	kworker_fie = kthread_create_worker(0, "cppc_fie");
+	if (IS_ERR(kworker_fie))
+		return;
+
+	ret = sched_setattr_nocheck(kworker_fie->task, &attr);
+	if (ret) {
+		pr_warn("%s: failed to set SCHED_DEADLINE: %d\n", __func__,
+			ret);
+		kthread_destroy_worker(kworker_fie);
+		return;
+	}
+
+	/* Register for freq-invariance */
+	topology_set_scale_freq_source(&cppc_sftd, cpu_present_mask);
+}
+
+static void cppc_freq_invariance_exit(void)
+{
+	struct cppc_freq_invariance *cppc_fi;
+	int i;
+
+	if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
+		return;
+
+	if (fie_disabled)
+		return;
+
+	topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, cpu_present_mask);
+
+	for_each_possible_cpu(i) {
+		cppc_fi = &per_cpu(cppc_freq_inv, i);
+		irq_work_sync(&cppc_fi->irq_work);
+	}
+
+	kthread_destroy_worker(kworker_fie);
+	kworker_fie = NULL;
+}
+
+#else
+static inline void
+cppc_freq_invariance_policy_init(struct cpufreq_policy *policy,
+				 struct cppc_cpudata *cpu_data)
+{
+}
+
+static inline void cppc_freq_invariance_init(void)
+{
+}
+
+static inline void cppc_freq_invariance_exit(void)
+{
+}
+#endif /* CONFIG_ACPI_CPPC_CPUFREQ_FIE */
+
 /* Callback function used to retrieve the max frequency from DMI */
 static void cppc_find_dmi_mhz(const struct dmi_header *dm, void *private)
 {
@@ -355,9 +557,12 @@ static int cppc_cpufreq_cpu_init(struct cpufreq_policy *policy)
 	cpu_data->perf_ctrls.desired_perf =  caps->highest_perf;
 
 	ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
-	if (ret)
+	if (ret) {
 		pr_debug("Err setting perf value:%d on CPU:%d. ret:%d\n",
 			 caps->highest_perf, cpu, ret);
+	} else {
+		cppc_freq_invariance_policy_init(policy, cpu_data);
+	}
 
 	return ret;
 }
@@ -370,12 +575,12 @@ static inline u64 get_delta(u64 t1, u64 t0)
 	return (u32)t1 - (u32)t0;
 }
 
-static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
-				     struct cppc_perf_fb_ctrs fb_ctrs_t0,
-				     struct cppc_perf_fb_ctrs fb_ctrs_t1)
+static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
+				 struct cppc_perf_fb_ctrs fb_ctrs_t0,
+				 struct cppc_perf_fb_ctrs fb_ctrs_t1)
 {
 	u64 delta_reference, delta_delivered;
-	u64 reference_perf, delivered_perf;
+	u64 reference_perf;
 
 	reference_perf = fb_ctrs_t0.reference_perf;
 
@@ -384,12 +589,21 @@ static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
 	delta_delivered = get_delta(fb_ctrs_t1.delivered,
 				    fb_ctrs_t0.delivered);
 
-	/* Check to avoid divide-by zero */
-	if (delta_reference || delta_delivered)
-		delivered_perf = (reference_perf * delta_delivered) /
-					delta_reference;
-	else
-		delivered_perf = cpu_data->perf_ctrls.desired_perf;
+	/* Check to avoid divide-by zero and invalid delivered_perf */
+	if (!delta_reference || !delta_delivered)
+		return cpu_data->perf_ctrls.desired_perf;
+
+	return (reference_perf * delta_delivered) / delta_reference;
+}
+
+static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
+				     struct cppc_perf_fb_ctrs fb_ctrs_t0,
+				     struct cppc_perf_fb_ctrs fb_ctrs_t1)
+{
+	u64 delivered_perf;
+
+	delivered_perf = cppc_perf_from_fbctrs(cpu_data, fb_ctrs_t0,
+					       fb_ctrs_t1);
 
 	return cppc_cpufreq_perf_to_khz(cpu_data, delivered_perf);
 }
@@ -514,6 +728,8 @@ static void cppc_check_hisi_workaround(void)
 
 static int __init cppc_cpufreq_init(void)
 {
+	int ret;
+
 	if ((acpi_disabled) || !acpi_cpc_valid())
 		return -ENODEV;
 
@@ -521,7 +737,11 @@ static int __init cppc_cpufreq_init(void)
 
 	cppc_check_hisi_workaround();
 
-	return cpufreq_register_driver(&cppc_cpufreq_driver);
+	ret = cpufreq_register_driver(&cppc_cpufreq_driver);
+	if (!ret)
+		cppc_freq_invariance_init();
+
+	return ret;
 }
 
 static inline void free_cpu_data(void)
@@ -538,6 +758,7 @@ static inline void free_cpu_data(void)
 
 static void __exit cppc_cpufreq_exit(void)
 {
+	cppc_freq_invariance_exit();
 	cpufreq_unregister_driver(&cppc_cpufreq_driver);
 
 	free_cpu_data();
diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
index 11e555cfaecb..f180240dc95f 100644
--- a/include/linux/arch_topology.h
+++ b/include/linux/arch_topology.h
@@ -37,6 +37,7 @@ bool topology_scale_freq_invariant(void);
 enum scale_freq_source {
 	SCALE_FREQ_SOURCE_CPUFREQ = 0,
 	SCALE_FREQ_SOURCE_ARCH,
+	SCALE_FREQ_SOURCE_CPPC,
 };
 
 struct scale_freq_data {
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ca2bb629595f..3adedc7b1725 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6386,6 +6386,7 @@ int sched_setattr_nocheck(struct task_struct *p, const struct sched_attr *attr)
 {
 	return __sched_setscheduler(p, attr, false, true);
 }
+EXPORT_SYMBOL_GPL(sched_setattr_nocheck);
 
 /**
  * sched_setscheduler_nocheck - change the scheduling policy and/or RT priority of a thread from kernelspace.
-- 
2.25.0.rc1.19.g042ed3e048af

Re: [PATCH V6 3/4] arch_topology: Export arch_freq_scale and helpers

[Viresh Kumar]

On 10-03-21, 10:53, Viresh Kumar wrote:
> It is possible now for other parts of the kernel to provide their own
> implementation of sched_freq_tick() and they can very well be modules
> themselves (like CPPC cpufreq driver, which is going to use these in a
> later commit).
> 
> Export arch_freq_scale and topology_{set|clear}_scale_freq_source().
> 
> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Tested-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Tested-by: Vincent Guittot <vincent.guittot@linaro.org>
> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
> ---
>  drivers/base/arch_topology.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> index ebcd2ea3091f..995e52b9eca4 100644
> --- a/drivers/base/arch_topology.c
> +++ b/drivers/base/arch_topology.c
> @@ -78,6 +78,7 @@ void topology_set_scale_freq_source(struct scale_freq_data *data,
>  
>  	update_scale_freq_invariant(true);
>  }
> +EXPORT_SYMBOL_GPL(topology_set_scale_freq_source);
>  
>  void topology_clear_scale_freq_source(enum scale_freq_source source,
>  				      const struct cpumask *cpus)
> @@ -96,6 +97,7 @@ void topology_clear_scale_freq_source(enum scale_freq_source source,
>  
>  	update_scale_freq_invariant(false);
>  }
> +EXPORT_SYMBOL_GPL(topology_clear_scale_freq_source);
>  
>  void topology_scale_freq_tick(void)
>  {
> @@ -106,6 +108,7 @@ void topology_scale_freq_tick(void)
>  }
>  
>  DEFINE_PER_CPU(unsigned long, arch_freq_scale) = SCHED_CAPACITY_SCALE;
> +EXPORT_SYMBOL_GPL(arch_freq_scale);

Did minor change here after testing bot complaint of sparse warning.

-EXPORT_SYMBOL_GPL(arch_freq_scale);
+EXPORT_PER_CPU_SYMBOL_GPL(arch_freq_scale);

-- 
viresh

Re: [PATCH V6 4/4] cpufreq: CPPC: Add support for frequency invariance

[Rafael J. Wysocki]

On Wed, Mar 10, 2021 at 6:24 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> The Frequency Invariance Engine (FIE) is providing a frequency scaling
> correction factor that helps achieve more accurate load-tracking.
>
> Normally, this scaling factor can be obtained directly with the help of
> the cpufreq drivers as they know the exact frequency the hardware is
> running at. But that isn't the case for CPPC cpufreq driver.
>
> Another way of obtaining that is using the arch specific counter
> support, which is already present in kernel, but that hardware is
> optional for platforms.
>
> This patch updates the CPPC driver to register itself with the topology
> core to provide its own implementation (cppc_scale_freq_tick()) of
> topology_scale_freq_tick() which gets called by the scheduler on every
> tick. Note that the arch specific counters have higher priority than
> CPPC counters, if available, though the CPPC driver doesn't need to have
> any special handling for that.
>
> On an invocation of cppc_scale_freq_tick(), we schedule an irq work
> (since we reach here from hard-irq context), which then schedules a
> normal work item and cppc_scale_freq_workfn() updates the per_cpu
> arch_freq_scale variable based on the counter updates since the last
> tick.
>
> To allow platforms to disable this CPPC counter-based frequency
> invariance support, this is all done under CONFIG_ACPI_CPPC_CPUFREQ_FIE,
> which is enabled by default.
>
> This also exports sched_setattr_nocheck() as the CPPC driver can be
> built as a module.
>
> Cc: linux-acpi@vger.kernel.org
> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Tested-by: Ionela Voinescu <ionela.voinescu@arm.com>
> Tested-by: Vincent Guittot <vincent.guittot@linaro.org>
> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

Sorry for the delay.

Acked-by: Rafael J. Wysocki <rafael@kernel.org>

and I'm assuming that either you or the sched guys will take care of it.

> ---
>  drivers/cpufreq/Kconfig.arm    |  10 ++
>  drivers/cpufreq/cppc_cpufreq.c | 245 +++++++++++++++++++++++++++++++--
>  include/linux/arch_topology.h  |   1 +
>  kernel/sched/core.c            |   1 +
>  4 files changed, 245 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/cpufreq/Kconfig.arm b/drivers/cpufreq/Kconfig.arm
> index e65e0a43be64..a5c5f70acfc9 100644
> --- a/drivers/cpufreq/Kconfig.arm
> +++ b/drivers/cpufreq/Kconfig.arm
> @@ -19,6 +19,16 @@ config ACPI_CPPC_CPUFREQ
>
>           If in doubt, say N.
>
> +config ACPI_CPPC_CPUFREQ_FIE
> +       bool "Frequency Invariance support for CPPC cpufreq driver"
> +       depends on ACPI_CPPC_CPUFREQ && GENERIC_ARCH_TOPOLOGY
> +       default y
> +       help
> +         This extends frequency invariance support in the CPPC cpufreq driver,
> +         by using CPPC delivered and reference performance counters.
> +
> +         If in doubt, say N.
> +
>  config ARM_ALLWINNER_SUN50I_CPUFREQ_NVMEM
>         tristate "Allwinner nvmem based SUN50I CPUFreq driver"
>         depends on ARCH_SUNXI
> diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
> index 8a482c434ea6..b8e1b8ea628c 100644
> --- a/drivers/cpufreq/cppc_cpufreq.c
> +++ b/drivers/cpufreq/cppc_cpufreq.c
> @@ -10,14 +10,18 @@
>
>  #define pr_fmt(fmt)    "CPPC Cpufreq:" fmt
>
> +#include <linux/arch_topology.h>
>  #include <linux/kernel.h>
>  #include <linux/module.h>
>  #include <linux/delay.h>
>  #include <linux/cpu.h>
>  #include <linux/cpufreq.h>
>  #include <linux/dmi.h>
> +#include <linux/irq_work.h>
> +#include <linux/kthread.h>
>  #include <linux/time.h>
>  #include <linux/vmalloc.h>
> +#include <uapi/linux/sched/types.h>
>
>  #include <asm/unaligned.h>
>
> @@ -57,6 +61,204 @@ static struct cppc_workaround_oem_info wa_info[] = {
>         }
>  };
>
> +#ifdef CONFIG_ACPI_CPPC_CPUFREQ_FIE
> +
> +/* Frequency invariance support */
> +struct cppc_freq_invariance {
> +       int cpu;
> +       struct irq_work irq_work;
> +       struct kthread_work work;
> +       struct cppc_perf_fb_ctrs prev_perf_fb_ctrs;
> +       struct cppc_cpudata *cpu_data;
> +};
> +
> +static DEFINE_PER_CPU(struct cppc_freq_invariance, cppc_freq_inv);
> +static struct kthread_worker *kworker_fie;
> +static bool fie_disabled;
> +
> +static struct cpufreq_driver cppc_cpufreq_driver;
> +static unsigned int hisi_cppc_cpufreq_get_rate(unsigned int cpu);
> +static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
> +                                struct cppc_perf_fb_ctrs fb_ctrs_t0,
> +                                struct cppc_perf_fb_ctrs fb_ctrs_t1);
> +
> +/**
> + * cppc_scale_freq_workfn - CPPC arch_freq_scale updater for frequency invariance
> + * @work: The work item.
> + *
> + * The CPPC driver register itself with the topology core to provide its own
> + * implementation (cppc_scale_freq_tick()) of topology_scale_freq_tick() which
> + * gets called by the scheduler on every tick.
> + *
> + * Note that the arch specific counters have higher priority than CPPC counters,
> + * if available, though the CPPC driver doesn't need to have any special
> + * handling for that.
> + *
> + * On an invocation of cppc_scale_freq_tick(), we schedule an irq work (since we
> + * reach here from hard-irq context), which then schedules a normal work item
> + * and cppc_scale_freq_workfn() updates the per_cpu arch_freq_scale variable
> + * based on the counter updates since the last tick.
> + */
> +static void cppc_scale_freq_workfn(struct kthread_work *work)
> +{
> +       struct cppc_freq_invariance *cppc_fi;
> +       struct cppc_perf_fb_ctrs fb_ctrs = {0};
> +       struct cppc_cpudata *cpu_data;
> +       unsigned long local_freq_scale;
> +       u64 perf;
> +
> +       cppc_fi = container_of(work, struct cppc_freq_invariance, work);
> +       cpu_data = cppc_fi->cpu_data;
> +
> +       if (cppc_get_perf_ctrs(cppc_fi->cpu, &fb_ctrs)) {
> +               pr_warn("%s: failed to read perf counters\n", __func__);
> +               return;
> +       }
> +
> +       cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
> +       perf = cppc_perf_from_fbctrs(cpu_data, cppc_fi->prev_perf_fb_ctrs,
> +                                    fb_ctrs);
> +
> +       perf <<= SCHED_CAPACITY_SHIFT;
> +       local_freq_scale = div64_u64(perf, cpu_data->perf_caps.highest_perf);
> +       if (WARN_ON(local_freq_scale > 1024))
> +               local_freq_scale = 1024;
> +
> +       per_cpu(arch_freq_scale, cppc_fi->cpu) = local_freq_scale;
> +}
> +
> +static void cppc_irq_work(struct irq_work *irq_work)
> +{
> +       struct cppc_freq_invariance *cppc_fi;
> +
> +       cppc_fi = container_of(irq_work, struct cppc_freq_invariance, irq_work);
> +       kthread_queue_work(kworker_fie, &cppc_fi->work);
> +}
> +
> +static void cppc_scale_freq_tick(void)
> +{
> +       struct cppc_freq_invariance *cppc_fi = &per_cpu(cppc_freq_inv, smp_processor_id());
> +
> +       /*
> +        * cppc_get_perf_ctrs() can potentially sleep, call that from the right
> +        * context.
> +        */
> +       irq_work_queue(&cppc_fi->irq_work);
> +}
> +
> +static struct scale_freq_data cppc_sftd = {
> +       .source = SCALE_FREQ_SOURCE_CPPC,
> +       .set_freq_scale = cppc_scale_freq_tick,
> +};
> +
> +static void cppc_freq_invariance_policy_init(struct cpufreq_policy *policy,
> +                                            struct cppc_cpudata *cpu_data)
> +{
> +       struct cppc_perf_fb_ctrs fb_ctrs = {0};
> +       struct cppc_freq_invariance *cppc_fi;
> +       int i, ret;
> +
> +       if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
> +               return;
> +
> +       if (fie_disabled)
> +               return;
> +
> +       for_each_cpu(i, policy->cpus) {
> +               cppc_fi = &per_cpu(cppc_freq_inv, i);
> +               cppc_fi->cpu = i;
> +               cppc_fi->cpu_data = cpu_data;
> +               kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
> +               init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
> +
> +               ret = cppc_get_perf_ctrs(i, &fb_ctrs);
> +               if (ret) {
> +                       pr_warn("%s: failed to read perf counters: %d\n",
> +                               __func__, ret);
> +                       fie_disabled = true;
> +               } else {
> +                       cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
> +               }
> +       }
> +}
> +
> +static void __init cppc_freq_invariance_init(void)
> +{
> +       struct sched_attr attr = {
> +               .size           = sizeof(struct sched_attr),
> +               .sched_policy   = SCHED_DEADLINE,
> +               .sched_nice     = 0,
> +               .sched_priority = 0,
> +               /*
> +                * Fake (unused) bandwidth; workaround to "fix"
> +                * priority inheritance.
> +                */
> +               .sched_runtime  = 1000000,
> +               .sched_deadline = 10000000,
> +               .sched_period   = 10000000,
> +       };
> +       int ret;
> +
> +       if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
> +               return;
> +
> +       if (fie_disabled)
> +               return;
> +
> +       kworker_fie = kthread_create_worker(0, "cppc_fie");
> +       if (IS_ERR(kworker_fie))
> +               return;
> +
> +       ret = sched_setattr_nocheck(kworker_fie->task, &attr);
> +       if (ret) {
> +               pr_warn("%s: failed to set SCHED_DEADLINE: %d\n", __func__,
> +                       ret);
> +               kthread_destroy_worker(kworker_fie);
> +               return;
> +       }
> +
> +       /* Register for freq-invariance */
> +       topology_set_scale_freq_source(&cppc_sftd, cpu_present_mask);
> +}
> +
> +static void cppc_freq_invariance_exit(void)
> +{
> +       struct cppc_freq_invariance *cppc_fi;
> +       int i;
> +
> +       if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
> +               return;
> +
> +       if (fie_disabled)
> +               return;
> +
> +       topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, cpu_present_mask);
> +
> +       for_each_possible_cpu(i) {
> +               cppc_fi = &per_cpu(cppc_freq_inv, i);
> +               irq_work_sync(&cppc_fi->irq_work);
> +       }
> +
> +       kthread_destroy_worker(kworker_fie);
> +       kworker_fie = NULL;
> +}
> +
> +#else
> +static inline void
> +cppc_freq_invariance_policy_init(struct cpufreq_policy *policy,
> +                                struct cppc_cpudata *cpu_data)
> +{
> +}
> +
> +static inline void cppc_freq_invariance_init(void)
> +{
> +}
> +
> +static inline void cppc_freq_invariance_exit(void)
> +{
> +}
> +#endif /* CONFIG_ACPI_CPPC_CPUFREQ_FIE */
> +
>  /* Callback function used to retrieve the max frequency from DMI */
>  static void cppc_find_dmi_mhz(const struct dmi_header *dm, void *private)
>  {
> @@ -355,9 +557,12 @@ static int cppc_cpufreq_cpu_init(struct cpufreq_policy *policy)
>         cpu_data->perf_ctrls.desired_perf =  caps->highest_perf;
>
>         ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
> -       if (ret)
> +       if (ret) {
>                 pr_debug("Err setting perf value:%d on CPU:%d. ret:%d\n",
>                          caps->highest_perf, cpu, ret);
> +       } else {
> +               cppc_freq_invariance_policy_init(policy, cpu_data);
> +       }
>
>         return ret;
>  }
> @@ -370,12 +575,12 @@ static inline u64 get_delta(u64 t1, u64 t0)
>         return (u32)t1 - (u32)t0;
>  }
>
> -static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
> -                                    struct cppc_perf_fb_ctrs fb_ctrs_t0,
> -                                    struct cppc_perf_fb_ctrs fb_ctrs_t1)
> +static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
> +                                struct cppc_perf_fb_ctrs fb_ctrs_t0,
> +                                struct cppc_perf_fb_ctrs fb_ctrs_t1)
>  {
>         u64 delta_reference, delta_delivered;
> -       u64 reference_perf, delivered_perf;
> +       u64 reference_perf;
>
>         reference_perf = fb_ctrs_t0.reference_perf;
>
> @@ -384,12 +589,21 @@ static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
>         delta_delivered = get_delta(fb_ctrs_t1.delivered,
>                                     fb_ctrs_t0.delivered);
>
> -       /* Check to avoid divide-by zero */
> -       if (delta_reference || delta_delivered)
> -               delivered_perf = (reference_perf * delta_delivered) /
> -                                       delta_reference;
> -       else
> -               delivered_perf = cpu_data->perf_ctrls.desired_perf;
> +       /* Check to avoid divide-by zero and invalid delivered_perf */
> +       if (!delta_reference || !delta_delivered)
> +               return cpu_data->perf_ctrls.desired_perf;
> +
> +       return (reference_perf * delta_delivered) / delta_reference;
> +}
> +
> +static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
> +                                    struct cppc_perf_fb_ctrs fb_ctrs_t0,
> +                                    struct cppc_perf_fb_ctrs fb_ctrs_t1)
> +{
> +       u64 delivered_perf;
> +
> +       delivered_perf = cppc_perf_from_fbctrs(cpu_data, fb_ctrs_t0,
> +                                              fb_ctrs_t1);
>
>         return cppc_cpufreq_perf_to_khz(cpu_data, delivered_perf);
>  }
> @@ -514,6 +728,8 @@ static void cppc_check_hisi_workaround(void)
>
>  static int __init cppc_cpufreq_init(void)
>  {
> +       int ret;
> +
>         if ((acpi_disabled) || !acpi_cpc_valid())
>                 return -ENODEV;
>
> @@ -521,7 +737,11 @@ static int __init cppc_cpufreq_init(void)
>
>         cppc_check_hisi_workaround();
>
> -       return cpufreq_register_driver(&cppc_cpufreq_driver);
> +       ret = cpufreq_register_driver(&cppc_cpufreq_driver);
> +       if (!ret)
> +               cppc_freq_invariance_init();
> +
> +       return ret;
>  }
>
>  static inline void free_cpu_data(void)
> @@ -538,6 +758,7 @@ static inline void free_cpu_data(void)
>
>  static void __exit cppc_cpufreq_exit(void)
>  {
> +       cppc_freq_invariance_exit();
>         cpufreq_unregister_driver(&cppc_cpufreq_driver);
>
>         free_cpu_data();
> diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
> index 11e555cfaecb..f180240dc95f 100644
> --- a/include/linux/arch_topology.h
> +++ b/include/linux/arch_topology.h
> @@ -37,6 +37,7 @@ bool topology_scale_freq_invariant(void);
>  enum scale_freq_source {
>         SCALE_FREQ_SOURCE_CPUFREQ = 0,
>         SCALE_FREQ_SOURCE_ARCH,
> +       SCALE_FREQ_SOURCE_CPPC,
>  };
>
>  struct scale_freq_data {
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index ca2bb629595f..3adedc7b1725 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -6386,6 +6386,7 @@ int sched_setattr_nocheck(struct task_struct *p, const struct sched_attr *attr)
>  {
>         return __sched_setscheduler(p, attr, false, true);
>  }
> +EXPORT_SYMBOL_GPL(sched_setattr_nocheck);
>
>  /**
>   * sched_setscheduler_nocheck - change the scheduling policy and/or RT priority of a thread from kernelspace.
> --
> 2.25.0.rc1.19.g042ed3e048af
>

Re: [PATCH V6 4/4] cpufreq: CPPC: Add support for frequency invariance

[Viresh Kumar]

On 19-03-21, 17:20, Rafael J. Wysocki wrote:
> Sorry for the delay.
> 
> Acked-by: Rafael J. Wysocki <rafael@kernel.org>

Thanks.

> and I'm assuming that either you or the sched guys will take care of it.

Yeah, I have already queued this up.

-- 
viresh

Re: [PATCH V6 1/4] arch_topology: Rename freq_scale as arch_freq_scale

[Will Deacon]

On Wed, Mar 10, 2021 at 10:53:23AM +0530, Viresh Kumar wrote:
> Rename freq_scale to a less generic name, as it will get exported soon
> for modules. Since x86 already names its own implementation of this as
> arch_freq_scale, lets stick to that.
> 
> Suggested-by: Will Deacon <will@kernel.org>
> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
> ---
>  arch/arm64/kernel/topology.c  | 6 +++---
>  drivers/base/arch_topology.c  | 4 ++--
>  include/linux/arch_topology.h | 4 ++--
>  3 files changed, 7 insertions(+), 7 deletions(-)

Acked-by: Will Deacon <will@kernel.org>

Will

Re: [PATCH V6 0/4] cpufreq: cppc: Add support for frequency invariance

[Qian Cai]

On 3/10/2021 12:23 AM, Viresh Kumar wrote:
> Hello,
> 
> CPPC cpufreq driver is used for ARM servers and this patch series tries
> to provide counter-based frequency invariance support for them in the
> absence for architecture specific counters (like AMUs).
> 
> This is tested by:
> - Vincent Guittot on ThunderX2.
> - Ionela Voinescu on Juno R2.
> - /me with hacks on Hikey, as I don't have access to the right hardware.

Git blame pointed out this series for a use-after-free during CPU offline/online. Any thoughts?

cppc_scale_freq_workfn+0x2e8/0x368:
cppc_perf_from_fbctrs at /usr/src/linux-next/drivers/cpufreq/cppc_cpufreq.c:584
(inlined by) cppc_scale_freq_workfn at /usr/src/linux-next/drivers/cpufreq/cppc_cpufreq.c:119

[ 9334.586904][  T694] BUG: KASAN: use-after-free in cppc_scale_freq_workfn+0x2e8/0x368 [cppc_cpufreq]
[ 9334.595966][  T694] Read of size 4 at addr ffff0008d52557b4 by task cppc_fie/694
[ 9334.603360][  T694]
[ 9334.605543][  T694] CPU: 2 PID: 694 Comm: cppc_fie Tainted: G        W         5.13.0-rc5-next-20210609+ #19
[ 9334.615368][  T694] Hardware name: MiTAC RAPTOR EV-883832-X3-0001/RAPTOR, BIOS 1.6 06/28/2020
[ 9334.623888][  T694] Call trace:
[ 9334.627025][  T694]  dump_backtrace+0x0/0x3b8
[ 9334.631385][  T694]  show_stack+0x20/0x30
[ 9334.635394][  T694]  dump_stack_lvl+0x144/0x190
[ 9334.639925][  T694]  print_address_description.constprop.0+0x74/0x3c8
[ 9334.646368][  T694]  kasan_report+0x1f0/0x208
[ 9334.650724][  T694]  __asan_report_load4_noabort+0x34/0x60
[ 9334.656208][  T694]  cppc_scale_freq_workfn+0x2e8/0x368 [cppc_cpufreq]
[ 9334.662740][  T694]  kthread_worker_fn+0x2f0/0xda0
[ 9334.667532][  T694]  kthread+0x3ac/0x460
[ 9334.671453][  T694]  ret_from_fork+0x10/0x18
[ 9334.675722][  T694]
[ 9334.677904][  T694] Allocated by task 607:
[ 9334.681996][  T694]  kasan_save_stack+0x28/0x58
[ 9334.686525][  T694]  94]  cppc_cpufre7164][  T694]  cpufreq_add_dev+0x164/0x1b8
[ 9334.711779][  T694]  subsys_interface_register+0x218/0x360
[ 9334.717265][  T694]  cpufreq_register_driver+0x2a4/0x4c0
[ 9334.722577][  T694]  0xffff80000af902a4
[ 9334.726412][  T694]  do_one_initcall+0x170/0xb98
[ 9334.731029][  T694]  do_init_module+0x18c/0x648
[ 9334.735559][  T694]  load_module+0x2618/0x3240
[ 9334.740001][  T694]  __do_sys_finit_module+0x118/0x1a8
[ 9334.745138][  T694]  __arm64_sys_finit_module+0x74/0xa8
[ 9334.750360][  T694]  invoke_syscall.constprop.0+0xdc/0x1d8
[ 9334.755846][  T694]  do_el0_svc+0x1f8/0x298
[ 9334.760028][  T694]  el0_svc+0x20/0x30
[ 9334.763775][  T694]  el0t_64_sync_handler+0xb0/0xb8
[ 9334.768651][  T694]  el0t_64_sync+0x178/0x17c
[ 9334.773006][  T694]
[ 9334.775187][  T694] The buggy address belongs to the object at ffff0008d5255780
[ 9334.775187][  T694]  which belongs to the cache kmalloc-128 of size 128
[ 9334.789089][  T694] The buggy address is located 52 bytes inside of
[ 9334.789089][  T694]  128-byte region [ffff0008d5255780, ffff0008d5255800)
[ 9334.802125][  T694] The buggy address belongs to the page:
[ 9334.807606][  T694] page:ffffffc002354940 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff0008d5255780 pfn:0x95525
[ 9334.818907][  T694] flags: 0x7ffff800000200(slab|node=0|zone=0|lastcpupid=0xfffff)
[ 9334.826480][  T694] raw: 007ffff800000200 ffff000012900448 ffffffc00240f8c8 ffff000012910580
[ 9334.834915][  T694] raw: ffff0008d5255780 0000000000aa00a9 00000001ffffffff 0000000000000000
[ 9334.843347][  T694] page dumped because: kasan: bad access detected
[ 9334.849609][  T694]
[ 9334.851790][  T694] Memory state around the buggy address:
[ 9334.857272][  T694]  ffff0008d5255680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 9334.865184][  T694]  ffff0008d5255700: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 9334.873096][  T694] >ffff0008d5255780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 9334.881007][  T694]                                      ^
[ 9334.886489][  T694]  ffff0008d5255800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 9334.894401][  T694]  ffff0008d5255880: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 9334.902312][  T694] ==================================================================
[ 9334.910223][  T694] Disabling lock debugging due to kernel taint
[ 9334.916638][T62553] CPU1: shutdown
[ 9334.920044][T62553] psci: CPU1 killed (polled 0 ms)
[ 9335.175037][   T22] IRQ 326: no longer affine to CPU2
[ 9335.180136][   T22] IRQ 382: no longer affine to CPU2
[ 9335.185320][T62553] CPU2: shutdown
[ 9335.188751][T62553] psci: CPU2 killed (polled 0 ms)
[ 9335.469684][   T27] IRQ 327: no longer affine to CPU3
[ 9335.474792][   T27] IRQ 361: no longer affine to CPU3
[ 9335.479938][T62553] CPU3: shutdown
[ 9335.483344][T62553] psci: CPU3 killed (polled 0 ms)
[ 9335.797240][   T32] IRQ 15: no longer affine to CPU4
[ 9335.802343][   T32] IRQ 334: no longer affine to CPU4
[ 9335.807554][T62553] CPU4: shutdown
[ 9335.810973][T62553] psci: CPU4 killed (polled 0 ms)
[ 9336.064091][T62553] CPU5: shutdown
[ 9336.067529][T62553] psci: CPU5 killed (polled 0 ms)
[ 9336.346263][T62553] CPU6: shutdown
[ 9336.349668][T62553] psci: CPU6 killed (polled 0 ms)
[ 9336.586727][T62553] CPU7: shutdown
[ 9336.590140][T62553] psci: CPU7 killed (polled 0 ms)
[ 9336.846866][T62553] CPU8: shutdown
[ 9336.850273][T62553] psci: CPU8 killed (polled 0 ms)
[ 9357.773249][T62671] loop0: detected capacity change from 0 to 8
[ 9471.525541][  T191] INFO: task cpuhp/9:56 blocked for more than 122 seconds.
[ 9471.532603][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
[ 9471.540291][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 9471.548837][  T191] task:cpuhp/9         state:D stack:58816 pid:   56 ppid:     2 flags:0x00000008
[ 9471.557924][  T191] Call trace:
[ 9471.561063][  T191]  __switch_to+0x184/0x400
[ 9471.565338][  T191]  __schedule+0x744/0x1930
[ 9471.569632][  T191]  schedule+0x1d0/0x3e8
[ 9471.573641][  T191]  schedule_timeout+0x188/0x1f8
[ 9471.578380][  T191]  wait_for_completion+0x15c/0x270
[ 9471.583348][  T191]  kthread_flush_work+0x15c/0x248
[ 9471.588274][  T191]  __kthread_cancel_work_sync+0x1a0/0x230
[ 9471.593851][  T191]  kthread_cancel_work_sync+0x1c/0x28
[ 9471.599114][  T191]  sugov_stop+0x104/0x148
[ 9471.603302][  T191]  cpufreq_stop_governor+0x78/0x138
[ 9471.608390][  T191]  cpufreq_offline+0x7c/0x748
[ 9471.612924][  T191]  cpuhp_cpufreq_offline+0x18/0x28
[ 9471.617924][  T191]  cpuhp_invoke_callback+0x54c/0x2be0
[ 9471.623153][  T191]  cpuhp_thread_fun+0x204/0x588
[ 9471.627892][  T191]  smpboot_thread_fn+0x3c8/0xbf8
[ 9471.632687][  T191]  kthread+0x3ac/0x460
[ 9471.636646][  T191]  ret_from_fork+0x10/0x18
[ 9471.640998][  T191] INFO: task irqbalance:940 blocked for more than 122 seconds.
[ 9471.648437][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
[ 9471.656127][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 9471.664650][  T191] task:irqbalance      state:D stack:55696 pid:  940 ppid:     1 flags:0x00000800
[ 9471.673753][  T191] Call trace:
[ 9471.676931][  T191]  __switch_to+0x184/0x400
[ 9471.681207][  T191]  __schedule+0x744/0x1930
[ 9471.685513][  T191]  schedule+0x1d0/0x3e8
[ 9471.689526][  T191]  schedule_preempt_disabled+0x80/0x110
[ 9471.694925][  T191]  __mutex_lock+0x618/0x1330
[ 9471.699401][  T191]  mutex_lock_nested+0x6c/0xc0
[ 9471.704024][  T191]  online_show+0x34/0xa8
[ 9471.708167][  T191]  dev_attr_show+0x50/0xc8
[ 9471.712442][  T191]  sysfs_kf_seq_show+0x164/0x368
[ 9471.717271][  T191]  kernfs_seq_show+0x130/0x198
[ 9471.721893][  T191]  seq_read_iter+0x344/0xd50
[ 9471.726371][  T191]  kernfs_fop_read_iter+0x32c/0x4a8
[ 9471.731426][  T191]  new_sync_read+0x2bc/0x4e8
[ 9471.735905][  T191]  vfs_read+0x18c/0x340
[ 9471.739918][  T191]  ksys_read+0xf8/0x1e0
[ 9471.743927][  T191]  __arm64_sys_read+0x74/0xa8
[ 9471.748479][  T191]  invoke_syscall.constprop.0+0xdc/0x1d8
[ 9471.753967][  T191]  do_el0_svc+0x1f8/0x298
[ 9471.758168][  T191]  el0_svc+0x20/0x30
[ 9471.761916][  T191]  el0t_64_sync_handler+0xb0/0xb8
[ 9471.766828][  T191]  el0t_64_sync+0x178/0x17c
[ 9471.771209][  T191] INFO: task kworker/9:2:27306 blocked for more than 123 seconds.
[ 9471.778895][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
[ 9471.786574][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 9471.795094][  T191] task:kworker/9:2     state:D stack:57552 pid:27306 ppid:     2 flags:0x00000008
[ 9471.804179][  T191] Workqueue: events cpuset_hotplug_workfn
[ 9471.809782][  T191] Call trace:
[ 9471.812919][  T191]  __switch_to+0x184/0x400
[ 9471.817208][  T191]  __schedule+0x744/0x1930
[ 9471.821479][  T191]  schedule+0x1d0/0x3e8
[ 9471.825512][  T191]  percpu_rwsem_wait+0x1a4/0x320
[ 9471.830303][  T191]  __percpu_down_read+0xb0/0x148
[ 9471.835089][  T191]  cpus_read_lock+0x2b4/0x308
[ 9471.839635][  T191]  rebuild_sched_domains+0x24/0x50
[ 9471.844598][  T191]  cpuset_hotplug_workfn+0x21c/0x11e0
[ 9471.849839][  T191]  process_one_work+0x7e4/0x1998
[ 9471.854629][  T191]  worker_thread+0x334/0xad0
[ 9471.859089][  T191]  kthread+0x3ac/0x460
[ 9471.863011][  T191]  ret_from_fork+0x10/0x18
[ 9471.867318][  T191] INFO: task kworker/31:1:49260 blocked for more than 123 seconds.
[ 9471.875058][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
[ 9471.882732][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 9471.891280][  T191] task:kworker/31:1    state:D stack:60640 pid:49260 ppid:     2 flags:0x00000008
[ 9471.900366][  T191] Workqueue: events vmstat_shepherd
[ 9471.905425][  T191] Call trace:
[ 9471.908582][  T191]  __switch_to+0x184/0x400
[ 9471.912853][  T191]  __schedule+0x744/0x1930
[ 9471.917142][  T191]  schedule+0x1d0/0x3e8
[ 9471.921150][  T191]  percpu_rwsem_wait+0x1a4/0x320
[ 9471.925960][  T191]  __percpu_down_read+0xb0/0x148
[ 9471.930751][  T191]  cpus_read_lock+0x2b4/0x308
[ 9471.935277][  T191]  vmstat_shepherd+0x5c/0x1a8
[ 9471.939828][  T191]  process_one_work+0x7e4/0x1998
[ 9471.944618][  T191]  worker_thread+0x334/0xad0
[ 9471.949084][  T191]  kthread+0x3ac/0x460
[ 9471.953007][  T191]  ret_from_fork+0x10/0x18
[ 9471.957305][  T191] INFO: task cpuhotplug04.sh:62553 blocked for more than 123 seconds.
[ 9471.965306][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
[ 9471.972991][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 9471.981538][  T191] task:cpuhotplug04.sh state:D stack:55936 pid:62553 ppid:  7301 flags:0x00000000
[ 9471.990625][  T191] Call trace:
[ 9471.993764][  T191]  __switch_to+0x184/0x400
[ 9471.998053][  T191]  __schedule+0x744/0x1930
[ 9472.002322][  T191]  schedule+0x1d0/0x3e8
[ 9472.006352][  T191]  schedule_timeout+0x188/0x1f8
[ 9472.011056][  T191]  wait_for_completion+0x15c/0x270
[ 9472.016038][  T191]  __cpuhp_kick_ap+0x158/0x1a8
[ 9472.020656][  T191]  cpuhp_kick_ap+0x200/0x7f8
[ 9472.025095][  T191]  cpuhp_kick_ap_work+0x1f0/0xc98
[ 9472.029990][  T191]  _cpu_down.constprop.0+0x348/0x1118
[ 9472.035214][  T191]  cpu_down+0x50/0x80
[ 9472.039065][  T191]  cpu_device_down+0x4c/0x68
[ 9472.043507][  T191]  cpu_subsys_offline+0x18/0x28
[ 9472.048229][  T191]  device_offline+0x154/0x1e0
[ 9472.052757][  T191]  online_store+0xa4/0x118
[ 9472.057042][  T191]  dev_attr_store+0x44/0x78
[ 9472.061397][  T191]  sysfs_kf_write+0xe8/0x138
[ 9472.065867][  T191]  kernfs_fop_write_iter+0x26c/0x3d0
[ 9472.071006][  T191]  new_sync_write+0x2bc/0x4f8
[ 9472.075554][  T191]  vfs_write+0x718/0xc88
[ 9472.079649][  T191]  ksys_write+0xf8/0x1e0
[ 9472.083742][  T191]  __arm64_sys_write+0x74/0xa8
[ 9472.088374][  T191]  invoke_syscall.constprop.0+0xdc/0x1d8
[ 9472.093859][  T191]  do_el0_svc+0xe4/0x298
[ 9472.097971][  T191]  el0_svc+0x20/0x30
[ 9472.101718][  T191]  el0t_64_sync_handler+0xb0/0xb8
[ 9472.106610][  T191]  el0t_64_sync+0x178/0x17c
[ 9472.110972][  T191] INFO: lockdep is turned off.
[ 9594.405358][  T191] INFO: task cpuhp/9:56 blocked for more than 245 seconds.
[ 9594.412416][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
[ 9594.420107][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 9594.428651][  T191] task:cpuhp/9         state:D stack:58816 pid:   56 ppid:     2 flags:0x00000008
[ 9594.437737][  T191] Call trace:
[ 9594.440877][  T191]  __switch_to+0x184/0x400
[ 9594.445148][  T191]  __schedule+0x744/0x1930
[ 9594.449438][  T191]  schedule+0x1d0/0x3e8
[ 9594.453447][  T191]  schedule_timeout+0x188/0x1f8
[ 9594.458169][  T191]  wait_for_completion+0x15c/0x270
[ 9594.463133][  T191]  kthread_flush_work+0x15c/0x248
[ 9594.468038][  T191]  __kthread_cancel_work_sync+0x1a0/0x230
[ 9594.473611][  T191]  kthread_cancel_work_sync+0x1c/0x28
[ 9594.478853][  T191]  sugov_stop+0x104/0x148
[ 9594.483036][  T191]  cpufreq_stop_governor+0x78/0x138
[ 9594.488106][  T191]  cpufreq_offline+0x7c/0x748
[ 9594.492634][  T191]  cpuhp_cpufreq_offline+0x18/0x28
[ 9594.497616][  T191]  cpuhp_invoke_callback+0x54c/0x2be0
[ 9594.502840][  T191]  cpuhp_thread_fun+0x204/0x588
[ 9594.507559][  T191]  smpboot_thread_fn+0x3c8/0xbf8
[ 9594.512349][  T191]  kthread+0x3ac/0x460
[ 9594.516291][  T191]  ret_from_fork+0x10/0x18


> 
> This is based of 5.12-rc2. I will merge these via the arm-cpufreq tree
> directly.
> 
> Changes since V5:
> - New patch to rename freq_scale to arch_freq_scale (Will Deacon).
> - Separate patch to export arch_freq_scale and helpers (Will Deacon).
> - Some improvements in the last patch like commit log, moving more stuff
>   to policy init, new fie_disabled flag, etc. (Ionela Voinescu).
> - Added Reviewed/Acked/Tested-by tags.
> 
> Changes since V4:
> - Move some code to policy specific initialization for cppc driver.
> - Initialize kthread specific stuff only once in cppc driver.
> - Added a kerneldoc comment in cppc driver and improved changelog as
>   well.
> 
> Changes since V3:
> - rebuild_sched_domains_energy() stuff moved from arm64 to drivers/base.
> - Added Reviewed/Tested-by Ionela for the first patch.
> - Remove unused max_freq field from structure in cppc driver.
> - s/cppc_f_i/cppc_freq_inv.
> - Fix an per-cpu access, there was a bug in earlier version.
> - Create a single kthread which can run on any CPU and takes care of
>   work from all the CPUs.
> - Do the whole FIE thing under a new CONFIG option for cppc driver.
> - Few minor improvements.
> 
> Changes since V2:
> - Not sending as an RFC anymore.
> - Several renames, reordering of code in 1/2 based on Ionela's comments.
> - Several rebase changes for 2/2.
> - The freq_scale calculations are optimized a bit.
> - Better overall commenting and commit logs.
> 
> Changes since V1:
> - The interface for setting the callbacks is improved, so different
>   parts looking to provide their callbacks don't need to think about
>   each other.
> 
> - Moved to per-cpu storage for storing the callback related data, AMU
>   counters have higher priority with this.
> 
> --
> Viresh
> 
> Viresh Kumar (4):
>   arch_topology: Rename freq_scale as arch_freq_scale
>   arch_topology: Allow multiple entities to provide sched_freq_tick()
>     callback
>   arch_topology: Export arch_freq_scale and helpers
>   cpufreq: CPPC: Add support for frequency invariance
> 
>  arch/arm64/include/asm/topology.h |  10 +-
>  arch/arm64/kernel/topology.c      | 109 +++++--------
>  drivers/base/arch_topology.c      |  89 ++++++++++-
>  drivers/cpufreq/Kconfig.arm       |  10 ++
>  drivers/cpufreq/cppc_cpufreq.c    | 245 ++++++++++++++++++++++++++++--
>  include/linux/arch_topology.h     |  19 ++-
>  kernel/sched/core.c               |   1 +
>  7 files changed, 385 insertions(+), 98 deletions(-)
> 
> 
> base-commit: a38fd8748464831584a19438cbb3082b5a2dab15
> 

Re: [PATCH V6 0/4] cpufreq: cppc: Add support for frequency invariance

[Qian Cai]

On 6/9/2021 11:46 AM, Qian Cai wrote:
> 
> 
> On 3/10/2021 12:23 AM, Viresh Kumar wrote:
>> Hello,
>>
>> CPPC cpufreq driver is used for ARM servers and this patch series tries
>> to provide counter-based frequency invariance support for them in the
>> absence for architecture specific counters (like AMUs).
>>
>> This is tested by:
>> - Vincent Guittot on ThunderX2.
>> - Ionela Voinescu on Juno R2.
>> - /me with hacks on Hikey, as I don't have access to the right hardware.
> 
> Git blame pointed out this series for a use-after-free during CPU offline/online. Any thoughts?

FYI, it could trigger a linked-list corruption instead below sometimes.

Reverting only the 4th patch (cpufreq: CPPC: Add support for frequency invariance) alone fixed the use-after-free. The corruption seems gone as well. 

[  258.585467][  T704] WARNING: CPU: 1 PID: 704 at drivers/cpufreq/cppc_cpufreq.c:124 cppc_scale_freq_workfn+0x2d0/0x368 [cppc_cpufreq]
[  258.593303][   C13] list_add corruption. prev->next should be next (ffff0008d88e6e48), but was ffff009b676bed30. (prev=ffff009b676bed30).

[  259.155595][   C13] Call trace:
[  259.158727][   C13]  __list_add_valid+0xf0/0xf8
[  259.163251][   C13]  kthread_insert_work+0x114/0x5c8
[  259.168209][   C13]  kthread_queue_work+0xf8/0x138
[  259.172993][   C13]  cppc_irq_work+0x24/0x30 [cppc_cpufreq]
[  259.178566][   C13]  irq_work_single+0xec/0x1e0
[  259.183091][   C13]  irq_work_run_list+0x88/0xb8
[  259.187701][   C13]  irq_work_run+0x28/0x48
[  259.191876][   C13]  ipi_handler+0x3dc/0x7a0
[  259.196140][   C13]  handle_percpu_devid_irq+0x240/0xb70
[  259.201447][   C13]  handle_domain_irq+0xe4/0x168
[  259.206144][   C13]  gic_handle_irq+0x5c/0x150
[  259.210582][   C13]  do_interrupt_handler+0x58/0xa8
[  259.215453][   C13]  el1_interrupt+0x30/0x48
[  259.219717][   C13]  el1h_64_irq_handler+0x18/0x28
[  259.224502][   C13]  el1h_64_irq+0x74/0x78
[  259.228590][   C13]  _stext+0x264/0x1114
[  259.232504][   C13]  irq_exit+0x53c/0x610
[  259.236507][   C13]  handle_domain_irq+0xe8/0x168
[  259.241203][   C13]  gic_handle_irq+0x5c/0x150
[  259.245640][   C13]  call_on_irq_stack+0x28/0x50
[  259.250250][   C13]  do_interrupt_handler+0x80/0xa8
[  259.255120][   C13]  el1_interrupt+0x30/0x48
[  259.259383][   C13]  el1h_64_irq_handler+0x18/0x28
[  259.264167][   C13]  el1h_64_irq+0x74/0x78
[  259.268255][   C13]  cpuidle_enter_state+0x2a4/0xff0
[  259.273214][   C13]  cpuidle_enter+0x60/0xb0
[  259.277476][   C13]  call_cpuidle+0x44/0xc0
[  259.281652][   C13]  do_idle+0x36c/0x420
[  259.285568][   C13]  cpu_startup_entry+0x30/0x88
[  259.290177][   C13]  secondary_start_kernel+0x28c/0x2d8
[  259.295395][   C13]  __secondary_switched+0x90/0x94
[  259.300269][   C13] Code: aa0103e3 91020000 aa1303e1 941b5a28 (d4210000)
[  259.307049][   C13] ---[ end trace dabb4391495b1abe ]---
[  259.312353][   C13] Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt

> 
> cppc_scale_freq_workfn+0x2e8/0x368:
> cppc_perf_from_fbctrs at /usr/src/linux-next/drivers/cpufreq/cppc_cpufreq.c:584
> (inlined by) cppc_scale_freq_workfn at /usr/src/linux-next/drivers/cpufreq/cppc_cpufreq.c:119
> 
> [ 9334.586904][  T694] BUG: KASAN: use-after-free in cppc_scale_freq_workfn+0x2e8/0x368 [cppc_cpufreq]
> [ 9334.595966][  T694] Read of size 4 at addr ffff0008d52557b4 by task cppc_fie/694
> [ 9334.603360][  T694]
> [ 9334.605543][  T694] CPU: 2 PID: 694 Comm: cppc_fie Tainted: G        W         5.13.0-rc5-next-20210609+ #19
> [ 9334.615368][  T694] Hardware name: MiTAC RAPTOR EV-883832-X3-0001/RAPTOR, BIOS 1.6 06/28/2020
> [ 9334.623888][  T694] Call trace:
> [ 9334.627025][  T694]  dump_backtrace+0x0/0x3b8
> [ 9334.631385][  T694]  show_stack+0x20/0x30
> [ 9334.635394][  T694]  dump_stack_lvl+0x144/0x190
> [ 9334.639925][  T694]  print_address_description.constprop.0+0x74/0x3c8
> [ 9334.646368][  T694]  kasan_report+0x1f0/0x208
> [ 9334.650724][  T694]  __asan_report_load4_noabort+0x34/0x60
> [ 9334.656208][  T694]  cppc_scale_freq_workfn+0x2e8/0x368 [cppc_cpufreq]
> [ 9334.662740][  T694]  kthread_worker_fn+0x2f0/0xda0
> [ 9334.667532][  T694]  kthread+0x3ac/0x460
> [ 9334.671453][  T694]  ret_from_fork+0x10/0x18
> [ 9334.675722][  T694]
> [ 9334.677904][  T694] Allocated by task 607:
> [ 9334.681996][  T694]  kasan_save_stack+0x28/0x58
> [ 9334.686525][  T694]  94]  cppc_cpufre7164][  T694]  cpufreq_add_dev+0x164/0x1b8
> [ 9334.711779][  T694]  subsys_interface_register+0x218/0x360
> [ 9334.717265][  T694]  cpufreq_register_driver+0x2a4/0x4c0
> [ 9334.722577][  T694]  0xffff80000af902a4
> [ 9334.726412][  T694]  do_one_initcall+0x170/0xb98
> [ 9334.731029][  T694]  do_init_module+0x18c/0x648
> [ 9334.735559][  T694]  load_module+0x2618/0x3240
> [ 9334.740001][  T694]  __do_sys_finit_module+0x118/0x1a8
> [ 9334.745138][  T694]  __arm64_sys_finit_module+0x74/0xa8
> [ 9334.750360][  T694]  invoke_syscall.constprop.0+0xdc/0x1d8
> [ 9334.755846][  T694]  do_el0_svc+0x1f8/0x298
> [ 9334.760028][  T694]  el0_svc+0x20/0x30
> [ 9334.763775][  T694]  el0t_64_sync_handler+0xb0/0xb8
> [ 9334.768651][  T694]  el0t_64_sync+0x178/0x17c
> [ 9334.773006][  T694]
> [ 9334.775187][  T694] The buggy address belongs to the object at ffff0008d5255780
> [ 9334.775187][  T694]  which belongs to the cache kmalloc-128 of size 128
> [ 9334.789089][  T694] The buggy address is located 52 bytes inside of
> [ 9334.789089][  T694]  128-byte region [ffff0008d5255780, ffff0008d5255800)
> [ 9334.802125][  T694] The buggy address belongs to the page:
> [ 9334.807606][  T694] page:ffffffc002354940 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff0008d5255780 pfn:0x95525
> [ 9334.818907][  T694] flags: 0x7ffff800000200(slab|node=0|zone=0|lastcpupid=0xfffff)
> [ 9334.826480][  T694] raw: 007ffff800000200 ffff000012900448 ffffffc00240f8c8 ffff000012910580
> [ 9334.834915][  T694] raw: ffff0008d5255780 0000000000aa00a9 00000001ffffffff 0000000000000000
> [ 9334.843347][  T694] page dumped because: kasan: bad access detected
> [ 9334.849609][  T694]
> [ 9334.851790][  T694] Memory state around the buggy address:
> [ 9334.857272][  T694]  ffff0008d5255680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> [ 9334.865184][  T694]  ffff0008d5255700: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> [ 9334.873096][  T694] >ffff0008d5255780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [ 9334.881007][  T694]                                      ^
> [ 9334.886489][  T694]  ffff0008d5255800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> [ 9334.894401][  T694]  ffff0008d5255880: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> [ 9334.902312][  T694] ==================================================================
> [ 9334.910223][  T694] Disabling lock debugging due to kernel taint
> [ 9334.916638][T62553] CPU1: shutdown
> [ 9334.920044][T62553] psci: CPU1 killed (polled 0 ms)
> [ 9335.175037][   T22] IRQ 326: no longer affine to CPU2
> [ 9335.180136][   T22] IRQ 382: no longer affine to CPU2
> [ 9335.185320][T62553] CPU2: shutdown
> [ 9335.188751][T62553] psci: CPU2 killed (polled 0 ms)
> [ 9335.469684][   T27] IRQ 327: no longer affine to CPU3
> [ 9335.474792][   T27] IRQ 361: no longer affine to CPU3
> [ 9335.479938][T62553] CPU3: shutdown
> [ 9335.483344][T62553] psci: CPU3 killed (polled 0 ms)
> [ 9335.797240][   T32] IRQ 15: no longer affine to CPU4
> [ 9335.802343][   T32] IRQ 334: no longer affine to CPU4
> [ 9335.807554][T62553] CPU4: shutdown
> [ 9335.810973][T62553] psci: CPU4 killed (polled 0 ms)
> [ 9336.064091][T62553] CPU5: shutdown
> [ 9336.067529][T62553] psci: CPU5 killed (polled 0 ms)
> [ 9336.346263][T62553] CPU6: shutdown
> [ 9336.349668][T62553] psci: CPU6 killed (polled 0 ms)
> [ 9336.586727][T62553] CPU7: shutdown
> [ 9336.590140][T62553] psci: CPU7 killed (polled 0 ms)
> [ 9336.846866][T62553] CPU8: shutdown
> [ 9336.850273][T62553] psci: CPU8 killed (polled 0 ms)
> [ 9357.773249][T62671] loop0: detected capacity change from 0 to 8
> [ 9471.525541][  T191] INFO: task cpuhp/9:56 blocked for more than 122 seconds.
> [ 9471.532603][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
> [ 9471.540291][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 9471.548837][  T191] task:cpuhp/9         state:D stack:58816 pid:   56 ppid:     2 flags:0x00000008
> [ 9471.557924][  T191] Call trace:
> [ 9471.561063][  T191]  __switch_to+0x184/0x400
> [ 9471.565338][  T191]  __schedule+0x744/0x1930
> [ 9471.569632][  T191]  schedule+0x1d0/0x3e8
> [ 9471.573641][  T191]  schedule_timeout+0x188/0x1f8
> [ 9471.578380][  T191]  wait_for_completion+0x15c/0x270
> [ 9471.583348][  T191]  kthread_flush_work+0x15c/0x248
> [ 9471.588274][  T191]  __kthread_cancel_work_sync+0x1a0/0x230
> [ 9471.593851][  T191]  kthread_cancel_work_sync+0x1c/0x28
> [ 9471.599114][  T191]  sugov_stop+0x104/0x148
> [ 9471.603302][  T191]  cpufreq_stop_governor+0x78/0x138
> [ 9471.608390][  T191]  cpufreq_offline+0x7c/0x748
> [ 9471.612924][  T191]  cpuhp_cpufreq_offline+0x18/0x28
> [ 9471.617924][  T191]  cpuhp_invoke_callback+0x54c/0x2be0
> [ 9471.623153][  T191]  cpuhp_thread_fun+0x204/0x588
> [ 9471.627892][  T191]  smpboot_thread_fn+0x3c8/0xbf8
> [ 9471.632687][  T191]  kthread+0x3ac/0x460
> [ 9471.636646][  T191]  ret_from_fork+0x10/0x18
> [ 9471.640998][  T191] INFO: task irqbalance:940 blocked for more than 122 seconds.
> [ 9471.648437][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
> [ 9471.656127][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 9471.664650][  T191] task:irqbalance      state:D stack:55696 pid:  940 ppid:     1 flags:0x00000800
> [ 9471.673753][  T191] Call trace:
> [ 9471.676931][  T191]  __switch_to+0x184/0x400
> [ 9471.681207][  T191]  __schedule+0x744/0x1930
> [ 9471.685513][  T191]  schedule+0x1d0/0x3e8
> [ 9471.689526][  T191]  schedule_preempt_disabled+0x80/0x110
> [ 9471.694925][  T191]  __mutex_lock+0x618/0x1330
> [ 9471.699401][  T191]  mutex_lock_nested+0x6c/0xc0
> [ 9471.704024][  T191]  online_show+0x34/0xa8
> [ 9471.708167][  T191]  dev_attr_show+0x50/0xc8
> [ 9471.712442][  T191]  sysfs_kf_seq_show+0x164/0x368
> [ 9471.717271][  T191]  kernfs_seq_show+0x130/0x198
> [ 9471.721893][  T191]  seq_read_iter+0x344/0xd50
> [ 9471.726371][  T191]  kernfs_fop_read_iter+0x32c/0x4a8
> [ 9471.731426][  T191]  new_sync_read+0x2bc/0x4e8
> [ 9471.735905][  T191]  vfs_read+0x18c/0x340
> [ 9471.739918][  T191]  ksys_read+0xf8/0x1e0
> [ 9471.743927][  T191]  __arm64_sys_read+0x74/0xa8
> [ 9471.748479][  T191]  invoke_syscall.constprop.0+0xdc/0x1d8
> [ 9471.753967][  T191]  do_el0_svc+0x1f8/0x298
> [ 9471.758168][  T191]  el0_svc+0x20/0x30
> [ 9471.761916][  T191]  el0t_64_sync_handler+0xb0/0xb8
> [ 9471.766828][  T191]  el0t_64_sync+0x178/0x17c
> [ 9471.771209][  T191] INFO: task kworker/9:2:27306 blocked for more than 123 seconds.
> [ 9471.778895][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
> [ 9471.786574][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 9471.795094][  T191] task:kworker/9:2     state:D stack:57552 pid:27306 ppid:     2 flags:0x00000008
> [ 9471.804179][  T191] Workqueue: events cpuset_hotplug_workfn
> [ 9471.809782][  T191] Call trace:
> [ 9471.812919][  T191]  __switch_to+0x184/0x400
> [ 9471.817208][  T191]  __schedule+0x744/0x1930
> [ 9471.821479][  T191]  schedule+0x1d0/0x3e8
> [ 9471.825512][  T191]  percpu_rwsem_wait+0x1a4/0x320
> [ 9471.830303][  T191]  __percpu_down_read+0xb0/0x148
> [ 9471.835089][  T191]  cpus_read_lock+0x2b4/0x308
> [ 9471.839635][  T191]  rebuild_sched_domains+0x24/0x50
> [ 9471.844598][  T191]  cpuset_hotplug_workfn+0x21c/0x11e0
> [ 9471.849839][  T191]  process_one_work+0x7e4/0x1998
> [ 9471.854629][  T191]  worker_thread+0x334/0xad0
> [ 9471.859089][  T191]  kthread+0x3ac/0x460
> [ 9471.863011][  T191]  ret_from_fork+0x10/0x18
> [ 9471.867318][  T191] INFO: task kworker/31:1:49260 blocked for more than 123 seconds.
> [ 9471.875058][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
> [ 9471.882732][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 9471.891280][  T191] task:kworker/31:1    state:D stack:60640 pid:49260 ppid:     2 flags:0x00000008
> [ 9471.900366][  T191] Workqueue: events vmstat_shepherd
> [ 9471.905425][  T191] Call trace:
> [ 9471.908582][  T191]  __switch_to+0x184/0x400
> [ 9471.912853][  T191]  __schedule+0x744/0x1930
> [ 9471.917142][  T191]  schedule+0x1d0/0x3e8
> [ 9471.921150][  T191]  percpu_rwsem_wait+0x1a4/0x320
> [ 9471.925960][  T191]  __percpu_down_read+0xb0/0x148
> [ 9471.930751][  T191]  cpus_read_lock+0x2b4/0x308
> [ 9471.935277][  T191]  vmstat_shepherd+0x5c/0x1a8
> [ 9471.939828][  T191]  process_one_work+0x7e4/0x1998
> [ 9471.944618][  T191]  worker_thread+0x334/0xad0
> [ 9471.949084][  T191]  kthread+0x3ac/0x460
> [ 9471.953007][  T191]  ret_from_fork+0x10/0x18
> [ 9471.957305][  T191] INFO: task cpuhotplug04.sh:62553 blocked for more than 123 seconds.
> [ 9471.965306][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
> [ 9471.972991][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 9471.981538][  T191] task:cpuhotplug04.sh state:D stack:55936 pid:62553 ppid:  7301 flags:0x00000000
> [ 9471.990625][  T191] Call trace:
> [ 9471.993764][  T191]  __switch_to+0x184/0x400
> [ 9471.998053][  T191]  __schedule+0x744/0x1930
> [ 9472.002322][  T191]  schedule+0x1d0/0x3e8
> [ 9472.006352][  T191]  schedule_timeout+0x188/0x1f8
> [ 9472.011056][  T191]  wait_for_completion+0x15c/0x270
> [ 9472.016038][  T191]  __cpuhp_kick_ap+0x158/0x1a8
> [ 9472.020656][  T191]  cpuhp_kick_ap+0x200/0x7f8
> [ 9472.025095][  T191]  cpuhp_kick_ap_work+0x1f0/0xc98
> [ 9472.029990][  T191]  _cpu_down.constprop.0+0x348/0x1118
> [ 9472.035214][  T191]  cpu_down+0x50/0x80
> [ 9472.039065][  T191]  cpu_device_down+0x4c/0x68
> [ 9472.043507][  T191]  cpu_subsys_offline+0x18/0x28
> [ 9472.048229][  T191]  device_offline+0x154/0x1e0
> [ 9472.052757][  T191]  online_store+0xa4/0x118
> [ 9472.057042][  T191]  dev_attr_store+0x44/0x78
> [ 9472.061397][  T191]  sysfs_kf_write+0xe8/0x138
> [ 9472.065867][  T191]  kernfs_fop_write_iter+0x26c/0x3d0
> [ 9472.071006][  T191]  new_sync_write+0x2bc/0x4f8
> [ 9472.075554][  T191]  vfs_write+0x718/0xc88
> [ 9472.079649][  T191]  ksys_write+0xf8/0x1e0
> [ 9472.083742][  T191]  __arm64_sys_write+0x74/0xa8
> [ 9472.088374][  T191]  invoke_syscall.constprop.0+0xdc/0x1d8
> [ 9472.093859][  T191]  do_el0_svc+0xe4/0x298
> [ 9472.097971][  T191]  el0_svc+0x20/0x30
> [ 9472.101718][  T191]  el0t_64_sync_handler+0xb0/0xb8
> [ 9472.106610][  T191]  el0t_64_sync+0x178/0x17c
> [ 9472.110972][  T191] INFO: lockdep is turned off.
> [ 9594.405358][  T191] INFO: task cpuhp/9:56 blocked for more than 245 seconds.
> [ 9594.412416][  T191]       Tainted: G    B   W         5.13.0-rc5-next-20210609+ #19
> [ 9594.420107][  T191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 9594.428651][  T191] task:cpuhp/9         state:D stack:58816 pid:   56 ppid:     2 flags:0x00000008
> [ 9594.437737][  T191] Call trace:
> [ 9594.440877][  T191]  __switch_to+0x184/0x400
> [ 9594.445148][  T191]  __schedule+0x744/0x1930
> [ 9594.449438][  T191]  schedule+0x1d0/0x3e8
> [ 9594.453447][  T191]  schedule_timeout+0x188/0x1f8
> [ 9594.458169][  T191]  wait_for_completion+0x15c/0x270
> [ 9594.463133][  T191]  kthread_flush_work+0x15c/0x248
> [ 9594.468038][  T191]  __kthread_cancel_work_sync+0x1a0/0x230
> [ 9594.473611][  T191]  kthread_cancel_work_sync+0x1c/0x28
> [ 9594.478853][  T191]  sugov_stop+0x104/0x148
> [ 9594.483036][  T191]  cpufreq_stop_governor+0x78/0x138
> [ 9594.488106][  T191]  cpufreq_offline+0x7c/0x748
> [ 9594.492634][  T191]  cpuhp_cpufreq_offline+0x18/0x28
> [ 9594.497616][  T191]  cpuhp_invoke_callback+0x54c/0x2be0
> [ 9594.502840][  T191]  cpuhp_thread_fun+0x204/0x588
> [ 9594.507559][  T191]  smpboot_thread_fn+0x3c8/0xbf8
> [ 9594.512349][  T191]  kthread+0x3ac/0x460
> [ 9594.516291][  T191]  ret_from_fork+0x10/0x18
> 
> 
>>
>> This is based of 5.12-rc2. I will merge these via the arm-cpufreq tree
>> directly.
>>
>> Changes since V5:
>> - New patch to rename freq_scale to arch_freq_scale (Will Deacon).
>> - Separate patch to export arch_freq_scale and helpers (Will Deacon).
>> - Some improvements in the last patch like commit log, moving more stuff
>>   to policy init, new fie_disabled flag, etc. (Ionela Voinescu).
>> - Added Reviewed/Acked/Tested-by tags.
>>
>> Changes since V4:
>> - Move some code to policy specific initialization for cppc driver.
>> - Initialize kthread specific stuff only once in cppc driver.
>> - Added a kerneldoc comment in cppc driver and improved changelog as
>>   well.
>>
>> Changes since V3:
>> - rebuild_sched_domains_energy() stuff moved from arm64 to drivers/base.
>> - Added Reviewed/Tested-by Ionela for the first patch.
>> - Remove unused max_freq field from structure in cppc driver.
>> - s/cppc_f_i/cppc_freq_inv.
>> - Fix an per-cpu access, there was a bug in earlier version.
>> - Create a single kthread which can run on any CPU and takes care of
>>   work from all the CPUs.
>> - Do the whole FIE thing under a new CONFIG option for cppc driver.
>> - Few minor improvements.
>>
>> Changes since V2:
>> - Not sending as an RFC anymore.
>> - Several renames, reordering of code in 1/2 based on Ionela's comments.
>> - Several rebase changes for 2/2.
>> - The freq_scale calculations are optimized a bit.
>> - Better overall commenting and commit logs.
>>
>> Changes since V1:
>> - The interface for setting the callbacks is improved, so different
>>   parts looking to provide their callbacks don't need to think about
>>   each other.
>>
>> - Moved to per-cpu storage for storing the callback related data, AMU
>>   counters have higher priority with this.
>>
>> --
>> Viresh
>>
>> Viresh Kumar (4):
>>   arch_topology: Rename freq_scale as arch_freq_scale
>>   arch_topology: Allow multiple entities to provide sched_freq_tick()
>>     callback
>>   arch_topology: Export arch_freq_scale and helpers
>>   cpufreq: CPPC: Add support for frequency invariance
>>
>>  arch/arm64/include/asm/topology.h |  10 +-
>>  arch/arm64/kernel/topology.c      | 109 +++++--------
>>  drivers/base/arch_topology.c      |  89 ++++++++++-
>>  drivers/cpufreq/Kconfig.arm       |  10 ++
>>  drivers/cpufreq/cppc_cpufreq.c    | 245 ++++++++++++++++++++++++++++--
>>  include/linux/arch_topology.h     |  19 ++-
>>  kernel/sched/core.c               |   1 +
>>  7 files changed, 385 insertions(+), 98 deletions(-)
>>
>>
>> base-commit: a38fd8748464831584a19438cbb3082b5a2dab15
>>

Re: [PATCH V6 0/4] cpufreq: cppc: Add support for frequency invariance

[Viresh Kumar]

On 09-06-21, 11:46, Qian Cai wrote:
> Git blame pointed out this series for a use-after-free during CPU offline/online. Any thoughts?

Thanks for reporting the problem here Qian. I have sent you a patchset
to fix these issues, please see if that works fine or not. It isn't
tested by me, as I didn't have the right setup for it.

-- 
viresh