[PATCHv2 0/8] cpufreq support for Marvell Armada XP

[PATCHv2 0/8] cpufreq support for Marvell Armada XP

[Thomas Petazzoni]

Mike, Viresh, Rafael, Jason, Gregory, Andrew, Sebastian,

This series of commits adds cpufreq support for the Marvell Armada XP
processor.

Changes since v1
================

 - Rework the patch series to use the generalized cpufreq-cpu0
   (renamed cpufreq-generic) driver instead of having an Armada XP
   specific cpufreq driver. This was suggested by Viresh
   Kumar. Basically, it only involved adding a "clock-latency"
   property in the DT, and changing the PMSU code to register the two
   OPPs supported by each CPU, and registering the "cpufreq-generic"
   platform device instead of the "armadaxp-cpufreq" one.

Details of the patches
======================

 - The core clock framework, to add a new clk_set_rate() notifier. See
   below for the explanation. This should go through Mike Turquette's
   tree. This is PATCH 01/08.

 - Improvements to the existing Armada XP cpu-clock driver. This
   should also go through Mike Turquette's tree, probably thanks to a
   pull request to be sent by the mvebu platform maintainers. This is
   PATCH 02/08.

 - Extension of the Marvell EBU PMSU code in arch/arm/mach-mvebu/ to
   implement a clock notifier needed for the dynamic frequency change
   procedure. This patch should go through the mvebu maintainers
   tree. This is PATCH 04/08. Note that this patch has a build
   dependency on PATCH 01/08, while all other patches don't have any
   other build dependency.

 - Misc other small changes (defconfigs, Device Tree, SMP
   initialization code) that are specific to the mvebu platform, also
   to be handled by the mvebu maintainers. Those are patches 3, 5, 6,
   7 and 8.

The only tricky part in this series is the addition of the
APPLY_RATE_CHANGE clock notifier. All the details about why a new
clock notifier is needed are exposed in the commit log of the first
patch, "clk: add an APPLY_RATE_CHANGE notifier event during
clk_set_rate()". Please read carefully this commit log to understand
the reasons for this proposed clock notifier.

Basically, the issue is that the procedure to change the CPU clock
frequency involves touching registers managed by the CPU clock driver,
but also registers from the PMSU which is used for a wide range of
power management activities, and is therefore handled by platform
specific code in mach-mvebu.

Jason, this patch series is based on 3.16-rc3, but it applies fine
even with mvebu/fixes and mvebu/soc merged (which contain some PMSU
changes), so I haven't based the patch series on those branches. To
_work_, the code needs the new cpufreq-generic driver which is pending
in Viresh Kumar's tree for 3.17, but there is no build dependency.

Thanks,

Thomas

Thomas Petazzoni (8):
  clk: add an APPLY_RATE_CHANGE notifier event during clk_set_rate()
  clk: mvebu: extend clk-cpu for dynamic frequency scaling
  ARM: mvebu: ensure CPU clocks are enabled
  ARM: mvebu: extend PMSU code to support dynamic frequency scaling
  ARM: mvebu: update Armada XP DT for dynamic frequency scaling
  ARM: mvebu: allow enabling of cpufreq on Armada XP
  ARM: mvebu: update mvebu_v7_defconfig with cpufreq support
  ARM: configs: add cpufreq-generic in multi_v7_defconfig

 .../devicetree/bindings/clock/mvebu-cpu-clock.txt  |   5 +-
 arch/arm/boot/dts/armada-xp-mv78230.dtsi           |   2 +
 arch/arm/boot/dts/armada-xp-mv78260.dtsi           |   2 +
 arch/arm/boot/dts/armada-xp-mv78460.dtsi           |   4 +
 arch/arm/boot/dts/armada-xp.dtsi                   |   2 +-
 arch/arm/configs/multi_v7_defconfig                |   1 +
 arch/arm/configs/mvebu_v7_defconfig                |   2 +
 arch/arm/mach-mvebu/Kconfig                        |   1 +
 arch/arm/mach-mvebu/platsmp.c                      |   1 +
 arch/arm/mach-mvebu/pmsu.c                         | 184 +++++++++++++++++++++
 drivers/clk/clk.c                                  |   3 +
 drivers/clk/mvebu/clk-cpu.c                        |  79 ++++++++-
 include/linux/clk.h                                |   4 +
 13 files changed, 282 insertions(+), 8 deletions(-)

-- 
2.0.0

[PATCHv2 1/8] clk: add an APPLY_RATE_CHANGE notifier event during clk_set_rate()

[Thomas Petazzoni]

The current clk_set_rate() logic allows notifiers to be notified of
three different events:

 * PRE_RATE_CHANGE: before the clock driver ->set_rate() function is
   called.
 * ABORT_RATE_CHANGE: called if the rate change failed
 * POST_RATE_CHANGE: called after the rate change has been
   successfully done, but after ->recalc_rate() has been called, and
   only if the rate of the clock has actually changed.

This commit adds an additional APPLY_RATE_CHANGE, which we require on
Armada XP to implement dynamic frequency scaling of the CPU
clocks. The problem is as follows.

On Armada XP, there are three hardware blocks involved in the dynamic
frequency scaling of the CPU clocks:

 - The CPU clocks hardware block itself, whose registers are already
   "managed" by drivers/clk/mvebu/clk-cpu.c (compatible string:
   marvell,armada-xp-cpu-clock). The driver currently only supports
   changing the rate of the CPU clock when the clock is off (i.e
   before we boot the secondary CPUs).

 - The PMU DFS registers, which are used to configure the target
   frequency for a dynamic rate change. Those registers are relatively
   well isolated from other PMU registers, so they can also be
   "managed" by the drivers/clk/mvebu/clk-cpu.c.

 - The PMSU registers, which are used to actually trigger the dynamic
   frequency change procedure, which consists in programming a bunch
   of PMSU registers, entering the idle state on the CPU we want to
   change the frequency, and then again reprogram a bunch of PMSU
   registers.

The procedure to change the frequency is:

 1/ Reset some clocks using the CPU clocks hardware block.
 2/ Define the target frequency using the PMU DFS registers.
 3/ Do the actual frequency change using the PMSU registers.

However, the PMSU registers are already "managed" by a driver in
arch/arm/mach-mvebu/pmsu.c, and the code there is needed for a wide
variety of power management activities: booting of secondary CPUs, CPU
hotplug, cpuidle, cpufreq, and soon suspend/resume. The same registers
in the PMSU are used for several of those activities.

So, what we need to do is to have steps (1) and (2) above done in the
CPU clocks driver, and then step (3) done through a clk notifier.

However, the current POST_RATE_CHANGE notifier is called too late
(after ->recalc_rate) and only if the rate has changed. So it works
fine for a pure notification of a frequency change, but it doesn't
work if the notified code is actually involved in the frequency
change, which is exactly the case we have here. Until the sequence
that uses the PMSU registers has been executed, the rate of the CPU
clock has not changed.

In order to solve this problem, we propose to add an APPLY_RATE_CHANGE
notifier event, which gets called right after ->set_rate(), but before
->recalc_rate(), and therefore regardless of whether there was an
actualy frequency change or not.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/clk/clk.c   | 3 +++
 include/linux/clk.h | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index 8b73ede..afa559b 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -1525,6 +1525,9 @@ static void clk_change_rate(struct clk *clk)
 	if (!skip_set_rate && clk->ops->set_rate)
 		clk->ops->set_rate(clk->hw, clk->new_rate, best_parent_rate);
 
+	if (clk->notifier_count)
+		__clk_notify(clk, APPLY_RATE_CHANGE, old_rate, clk->new_rate);
+
 	clk->rate = clk_recalc(clk, best_parent_rate);
 
 	if (clk->notifier_count && old_rate != clk->rate)
diff --git a/include/linux/clk.h b/include/linux/clk.h
index fb5e097..9ea9f5f 100644
--- a/include/linux/clk.h
+++ b/include/linux/clk.h
@@ -39,10 +39,14 @@ struct clk;
  * POST_RATE_CHANGE - called after the clk rate change has successfully
  *     completed.  Callbacks must always return NOTIFY_DONE or NOTIFY_OK.
  *
+ * APPLY_RATE_CHANGE - called right after calling ->set_rate(), but
+ *     before recalculating the rate.
+ *
  */
 #define PRE_RATE_CHANGE			BIT(0)
 #define POST_RATE_CHANGE		BIT(1)
 #define ABORT_RATE_CHANGE		BIT(2)
+#define APPLY_RATE_CHANGE		BIT(3)
 
 /**
  * struct clk_notifier - associate a clk with a notifier
-- 
2.0.0

[PATCHv2 2/8] clk: mvebu: extend clk-cpu for dynamic frequency scaling

[Thomas Petazzoni]

This commit extends the existing clk-cpu driver used on Marvell Armada
XP platforms to support the dynamic frequency scaling of the CPU
clock. Non-dynamic frequency change was already supported (and used
before secondary CPUs are started), but the dynamic frequency change
requires a completely different procedure.

In order to achieve this, the clk_cpu_set_rate() function is reworked
to handle two separate cases:

 - The case where the clock is enabled, which is the new dynamic
   frequency change code, implemented in clk_cpu_on_set_rate(). This
   part will be used for cpufreq activities.

 - The case where the clock is disabled, which is the existing
   frequency change code, moved in clk_cpu_off_set_rate(). This part
   is already used to set the clock frequency of the secondary CPUs
   before starting them.

In order to implement the dynamic frequency change function, we need
to access the PMU DFS registers, which are outside the currently
mapped "Clock Complex" registers, so a new area of registers is now
mapped. This affects the Device Tree binding, but we are careful to do
it in a backward-compatible way (by allowing the second pair of
registers to be non-existent, and in this case, ensuring
clk_cpu_on_set_rate() returns an error).

Note that technically speaking, the clk_cpu_on_set_rate() does not do
the entire procedure needed to change the frequency dynamically, as it
involves touching a number of PMSU registers. This is done through a
clock notifier registered by the PMSU driver in followup commits.

Cc: <devicetree@vger.kernel.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../devicetree/bindings/clock/mvebu-cpu-clock.txt  |  5 +-
 drivers/clk/mvebu/clk-cpu.c                        | 79 ++++++++++++++++++++--
 2 files changed, 77 insertions(+), 7 deletions(-)

diff --git a/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt b/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt
index feb8301..99c2146 100644
--- a/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt
+++ b/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt
@@ -3,14 +3,15 @@ Device Tree Clock bindings for cpu clock of Marvell EBU platforms
 Required properties:
 - compatible : shall be one of the following:
 	"marvell,armada-xp-cpu-clock" - cpu clocks for Armada XP
-- reg : Address and length of the clock complex register set
+- reg : Address and length of the clock complex register set, followed
+        by address and length of the PMU DFS registers
 - #clock-cells : should be set to 1.
 - clocks : shall be the input parent clock phandle for the clock.
 
 cpuclk: clock-complex at d0018700 {
 	#clock-cells = <1>;
 	compatible = "marvell,armada-xp-cpu-clock";
-	reg = <0xd0018700 0xA0>;
+	reg = <0xd0018700 0xA0>, <0x1c054 0x10>;
 	clocks = <&coreclk 1>;
 }
 
diff --git a/drivers/clk/mvebu/clk-cpu.c b/drivers/clk/mvebu/clk-cpu.c
index 8ebf757..b931e72 100644
--- a/drivers/clk/mvebu/clk-cpu.c
+++ b/drivers/clk/mvebu/clk-cpu.c
@@ -16,10 +16,18 @@
 #include <linux/io.h>
 #include <linux/of.h>
 #include <linux/delay.h>
+#include <asm/smp_plat.h>
 
-#define SYS_CTRL_CLK_DIVIDER_CTRL_OFFSET    0x0
-#define SYS_CTRL_CLK_DIVIDER_VALUE_OFFSET   0xC
-#define SYS_CTRL_CLK_DIVIDER_MASK	    0x3F
+#define SYS_CTRL_CLK_DIVIDER_CTRL_OFFSET               0x0
+#define   SYS_CTRL_CLK_DIVIDER_CTRL_RESET_ALL          0xff
+#define   SYS_CTRL_CLK_DIVIDER_CTRL_RESET_SHIFT        8
+#define SYS_CTRL_CLK_DIVIDER_CTRL2_OFFSET              0x8
+#define   SYS_CTRL_CLK_DIVIDER_CTRL2_NBCLK_RATIO_SHIFT 16
+#define SYS_CTRL_CLK_DIVIDER_VALUE_OFFSET              0xC
+#define SYS_CTRL_CLK_DIVIDER_MASK                      0x3F
+
+#define PMU_DFS_RATIO_SHIFT 16
+#define PMU_DFS_RATIO_MASK  0x3F
 
 #define MAX_CPU	    4
 struct cpu_clk {
@@ -28,6 +36,7 @@ struct cpu_clk {
 	const char *clk_name;
 	const char *parent_name;
 	void __iomem *reg_base;
+	void __iomem *pmu_dfs;
 };
 
 static struct clk **clks;
@@ -62,8 +71,9 @@ static long clk_cpu_round_rate(struct clk_hw *hwclk, unsigned long rate,
 	return *parent_rate / div;
 }
 
-static int clk_cpu_set_rate(struct clk_hw *hwclk, unsigned long rate,
-			    unsigned long parent_rate)
+static int clk_cpu_off_set_rate(struct clk_hw *hwclk, unsigned long rate,
+				unsigned long parent_rate)
+
 {
 	struct cpu_clk *cpuclk = to_cpu_clk(hwclk);
 	u32 reg, div;
@@ -95,6 +105,58 @@ static int clk_cpu_set_rate(struct clk_hw *hwclk, unsigned long rate,
 	return 0;
 }
 
+static int clk_cpu_on_set_rate(struct clk_hw *hwclk, unsigned long rate,
+			       unsigned long parent_rate)
+{
+	u32 reg;
+	unsigned long fabric_div, target_div, cur_rate;
+	struct cpu_clk *cpuclk = to_cpu_clk(hwclk);
+
+	/*
+	 * PMU DFS registers are not mapped, Device Tree does not
+	 * describes them. We cannot change the frequency dynamically.
+	 */
+	if (!cpuclk->pmu_dfs)
+		return -ENODEV;
+
+	cur_rate = __clk_get_rate(hwclk->clk);
+
+	reg = readl(cpuclk->reg_base + SYS_CTRL_CLK_DIVIDER_CTRL2_OFFSET);
+	fabric_div = (reg >> SYS_CTRL_CLK_DIVIDER_CTRL2_NBCLK_RATIO_SHIFT) &
+		SYS_CTRL_CLK_DIVIDER_MASK;
+
+	/* Frequency is going up */
+	if (rate == 2 * cur_rate)
+		target_div = fabric_div / 2;
+	/* Frequency is going down */
+	else
+		target_div = fabric_div;
+
+	if (target_div == 0)
+		target_div = 1;
+
+	reg = readl(cpuclk->pmu_dfs);
+	reg &= ~(PMU_DFS_RATIO_MASK << PMU_DFS_RATIO_SHIFT);
+	reg |= (target_div << PMU_DFS_RATIO_SHIFT);
+	writel(reg, cpuclk->pmu_dfs);
+
+	reg = readl(cpuclk->reg_base + SYS_CTRL_CLK_DIVIDER_CTRL_OFFSET);
+	reg |= (SYS_CTRL_CLK_DIVIDER_CTRL_RESET_ALL <<
+		SYS_CTRL_CLK_DIVIDER_CTRL_RESET_SHIFT);
+	writel(reg, cpuclk->reg_base + SYS_CTRL_CLK_DIVIDER_CTRL_OFFSET);
+
+	return 0;
+}
+
+static int clk_cpu_set_rate(struct clk_hw *hwclk, unsigned long rate,
+			    unsigned long parent_rate)
+{
+	if (__clk_is_enabled(hwclk->clk))
+		return clk_cpu_on_set_rate(hwclk, rate, parent_rate);
+	else
+		return clk_cpu_off_set_rate(hwclk, rate, parent_rate);
+}
+
 static const struct clk_ops cpu_ops = {
 	.recalc_rate = clk_cpu_recalc_rate,
 	.round_rate = clk_cpu_round_rate,
@@ -105,6 +167,7 @@ static void __init of_cpu_clk_setup(struct device_node *node)
 {
 	struct cpu_clk *cpuclk;
 	void __iomem *clock_complex_base = of_iomap(node, 0);
+	void __iomem *pmu_dfs_base = of_iomap(node, 1);
 	int ncpus = 0;
 	struct device_node *dn;
 
@@ -114,6 +177,10 @@ static void __init of_cpu_clk_setup(struct device_node *node)
 		return;
 	}
 
+	if (pmu_dfs_base == NULL)
+		pr_warn("%s: pmu-dfs base register not set, dynamic frequency scaling not available\n",
+			__func__);
+
 	for_each_node_by_type(dn, "cpu")
 		ncpus++;
 
@@ -146,6 +213,8 @@ static void __init of_cpu_clk_setup(struct device_node *node)
 		cpuclk[cpu].clk_name = clk_name;
 		cpuclk[cpu].cpu = cpu;
 		cpuclk[cpu].reg_base = clock_complex_base;
+		if (pmu_dfs_base)
+			cpuclk[cpu].pmu_dfs = pmu_dfs_base + 4 * cpu;
 		cpuclk[cpu].hw.init = &init;
 
 		init.name = cpuclk[cpu].clk_name;
-- 
2.0.0

[PATCHv2 3/8] ARM: mvebu: ensure CPU clocks are enabled

[Thomas Petazzoni]

In the Armada XP SMP support code, we are reading the clock frequency
of the booting CPU, and use that to assign the same frequency to the
other CPUs, and we do this while the clocks are disabled.

However, the CPU clocks are in fact never prepared/enabled, and to
support cpufreq, we now have two code paths to change the frequency of
the CPU clocks in the CPU clock driver: one when the clock is enabled
(dynamic frequency scaling), one when the clock is disabled (adjusting
the CPU frequency before starting the CPU). In order for this to work,
the CPU clocks now have to be prepared and enabled after the initial
synchronization of the clock frequencies is done, so that all future
rate changes of the CPU clocks will trigger a dynamic frequency
scaling transition.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/platsmp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/mach-mvebu/platsmp.c b/arch/arm/mach-mvebu/platsmp.c
index 88b976b3..4880b0f 100644
--- a/arch/arm/mach-mvebu/platsmp.c
+++ b/arch/arm/mach-mvebu/platsmp.c
@@ -67,6 +67,7 @@ static void __init set_secondary_cpus_clock(void)
 		if (!cpu_clk)
 			return;
 		clk_set_rate(cpu_clk, rate);
+		clk_prepare_enable(cpu_clk);
 	}
 }
 
-- 
2.0.0

[PATCHv2 4/8] ARM: mvebu: extend PMSU code to support dynamic frequency scaling

[Thomas Petazzoni]

This commit adds the necessary code in the Marvell EBU PMSU driver to
support dynamic frequency scaling. In essence, what this new code does
is that it:

 * registers the frequency operating points supported by the CPU;

 * registers a clock notifier of the CPU clocks. The notifier function
   listens to the newly introduced APPLY_RATE_CHANGE event, and uses
   that to finalize the frequency transition by doing the part of the
   procedure that involves the PMSU;

 * registers a platform device for the cpufreq-generic driver, which
   will take care of the CPU frequency transitions.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/pmsu.c | 184 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 184 insertions(+)

diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
index 53a55c8..9257b16 100644
--- a/arch/arm/mach-mvebu/pmsu.c
+++ b/arch/arm/mach-mvebu/pmsu.c
@@ -18,20 +18,26 @@
 
 #define pr_fmt(fmt) "mvebu-pmsu: " fmt
 
+#include <linux/clk.h>
 #include <linux/cpu_pm.h>
+#include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <linux/of_address.h>
+#include <linux/of_device.h>
 #include <linux/io.h>
 #include <linux/platform_device.h>
+#include <linux/pm_opp.h>
 #include <linux/smp.h>
 #include <linux/resource.h>
+#include <linux/slab.h>
 #include <asm/cacheflush.h>
 #include <asm/cp15.h>
 #include <asm/smp_plat.h>
 #include <asm/suspend.h>
 #include <asm/tlbflush.h>
 #include "common.h"
+#include "armada-370-xp.h"
 
 static void __iomem *pmsu_mp_base;
 
@@ -57,6 +63,10 @@ static void __iomem *pmsu_mp_base;
 #define PMSU_STATUS_AND_MASK_IRQ_MASK		BIT(24)
 #define PMSU_STATUS_AND_MASK_FIQ_MASK		BIT(25)
 
+#define PMSU_EVENT_STATUS_AND_MASK(cpu)     ((cpu * 0x100) + 0x120)
+#define PMSU_EVENT_STATUS_AND_MASK_DFS_DONE        BIT(1)
+#define PMSU_EVENT_STATUS_AND_MASK_DFS_DONE_MASK   BIT(17)
+
 #define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu) ((cpu * 0x100) + 0x124)
 
 /* PMSU fabric registers */
@@ -296,3 +306,177 @@ int __init armada_370_xp_cpu_pm_init(void)
 
 arch_initcall(armada_370_xp_cpu_pm_init);
 early_initcall(armada_370_xp_pmsu_init);
+
+static void armada_xp_cpufreq_clk_set_local(void *data)
+{
+	u32 reg;
+	u32 cpu = smp_processor_id();
+	unsigned long flags;
+
+	local_irq_save(flags);
+
+	/* Prepare to enter idle */
+	reg = readl(pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
+	reg |= PMSU_STATUS_AND_MASK_CPU_IDLE_WAIT |
+	       PMSU_STATUS_AND_MASK_IRQ_MASK     |
+	       PMSU_STATUS_AND_MASK_FIQ_MASK;
+	writel(reg, pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
+
+	/* Request the DFS transition */
+	reg = readl(pmsu_mp_base + PMSU_CONTROL_AND_CONFIG(cpu));
+	reg |= PMSU_CONTROL_AND_CONFIG_DFS_REQ;
+	writel(reg, pmsu_mp_base + PMSU_CONTROL_AND_CONFIG(cpu));
+
+	/* The fact of entering idle will trigger the DFS transition */
+	wfi();
+
+	/*
+	 * We're back from idle, the DFS transition has completed,
+	 * clear the idle wait indication.
+	 */
+	reg = readl(pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
+	reg &= ~PMSU_STATUS_AND_MASK_CPU_IDLE_WAIT;
+	writel(reg, pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
+
+	local_irq_restore(flags);
+}
+
+struct armada_xp_cpufreq_notifier_block {
+	struct notifier_block nb;
+	int cpu;
+};
+
+static int armada_xp_cpufreq_clk_notify(struct notifier_block *self,
+					unsigned long action, void *data)
+{
+	struct armada_xp_cpufreq_notifier_block *nb =
+		container_of(self, struct armada_xp_cpufreq_notifier_block, nb);
+	unsigned long timeout;
+	int cpu = cpu_logical_map(nb->cpu);
+	u32 reg;
+
+	if (action != APPLY_RATE_CHANGE)
+		return 0;
+
+	/* Clear any previous DFS DONE event */
+	reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+	reg &= ~PMSU_EVENT_STATUS_AND_MASK_DFS_DONE;
+	writel(reg, pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+
+	/* Mask the DFS done interrupt, since we are going to poll */
+	reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+	reg |= PMSU_EVENT_STATUS_AND_MASK_DFS_DONE_MASK;
+	writel(reg, pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+
+	/* Trigger the DFS on the appropriate CPU */
+	smp_call_function_single(get_logical_index(cpu),
+				 armada_xp_cpufreq_clk_set_local, NULL, false);
+
+	/* Poll until the DFS done event is generated */
+	timeout = jiffies + HZ;
+	while (time_before(jiffies, timeout)) {
+		reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+		if (reg & PMSU_EVENT_STATUS_AND_MASK_DFS_DONE)
+			break;
+		udelay(10);
+	}
+
+	/* Restore the DFS mask to its original state */
+	reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+	reg &= ~BIT(17);
+	writel(reg, pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
+
+	return NOTIFY_DONE;
+}
+
+static int __init armada_xp_pmsu_cpufreq_init(void)
+{
+	struct device_node *np;
+	struct resource res;
+	int ret, cpu;
+
+	if (!of_machine_is_compatible("marvell,armadaxp"))
+		return 0;
+
+	/*
+	 * In order to have proper cpufreq handling, we need to ensure
+	 * that the Device Tree description of the CPU clock includes
+	 * the definition of the PMU DFS registers. If not, we do not
+	 * register the clock notifier and the cpufreq driver. This
+	 * piece of code is only for compatibility with old Device
+	 * Trees.
+	 */
+	np = of_find_compatible_node(NULL, NULL, "marvell,armada-xp-cpu-clock");
+	if (!np)
+		return 0;
+
+	ret = of_address_to_resource(np, 1, &res);
+	if (ret) {
+		pr_warn(FW_WARN "not enabling cpufreq, deprecated armada-xp-cpu-clock binding\n");
+		of_node_put(np);
+		return 0;
+	}
+
+	of_node_put(np);
+
+	/*
+	 * For each CPU, this loop registers the operating points
+	 * supported (which are the nominal CPU frequency and half of
+	 * it), and registers the clock notifier that will take care
+	 * of doing the PMSU part of a frequency transition.
+	 */
+	for_each_possible_cpu(cpu) {
+		struct clk *clk;
+		struct device *cpu_dev;
+		struct armada_xp_cpufreq_notifier_block *nbs;
+		int ret;
+
+		cpu_dev = get_cpu_device(cpu);
+		if (!cpu_dev) {
+			pr_err("Cannot get CPU %d\n", cpu);
+			continue;
+		}
+
+		clk = clk_get(cpu_dev, 0);
+		if (!clk) {
+			pr_err("Cannot get clock for CPU %d\n", cpu);
+			return -ENODEV;
+		}
+
+		ret = dev_pm_opp_add(cpu_dev, clk_get_rate(clk), 0);
+		if (ret) {
+			clk_put(clk);
+			return ret;
+		}
+
+		ret = dev_pm_opp_add(cpu_dev, clk_get_rate(clk) / 2, 0);
+		if (ret) {
+			clk_put(clk);
+			return ret;
+		}
+
+		nbs = kzalloc(sizeof(struct armada_xp_cpufreq_notifier_block),
+			      GFP_KERNEL);
+		if (!nbs) {
+			pr_err("Cannot allocate memory for cpufreq notifier\n");
+			clk_put(clk);
+			return -ENOMEM;
+		}
+
+		nbs->nb.notifier_call = armada_xp_cpufreq_clk_notify;
+		nbs->cpu = cpu;
+
+		ret = clk_notifier_register(clk, &nbs->nb);
+		if (ret) {
+			pr_err("Cannot register clock notifier\n");
+			kfree(nbs);
+			clk_put(clk);
+			return ret;
+		}
+	}
+
+	platform_device_register_simple("cpufreq-generic", -1, NULL, 0);
+	return 0;
+}
+
+device_initcall(armada_xp_pmsu_cpufreq_init);
-- 
2.0.0

[PATCHv2 5/8] ARM: mvebu: update Armada XP DT for dynamic frequency scaling

[Thomas Petazzoni]

In order to support dynamic frequency scaling:

 * the cpuclk Device Tree node needs to be updated to describe a
   second set of registers describing the PMU DFS registers.

 * the clock-latency property of the CPUs must be filled, otherwise
   the ondemand and conservative cpufreq governors refuse to work. The
   latency is high because the cost of a frequency transition is quite
   high on those CPUs.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-mv78230.dtsi | 2 ++
 arch/arm/boot/dts/armada-xp-mv78260.dtsi | 2 ++
 arch/arm/boot/dts/armada-xp-mv78460.dtsi | 4 ++++
 arch/arm/boot/dts/armada-xp.dtsi         | 2 +-
 4 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/armada-xp-mv78230.dtsi b/arch/arm/boot/dts/armada-xp-mv78230.dtsi
index 1257ff1..2592e1c 100644
--- a/arch/arm/boot/dts/armada-xp-mv78230.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78230.dtsi
@@ -34,6 +34,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <0>;
 			clocks = <&cpuclk 0>;
+			clock-latency = <1000000>;
 		};
 
 		cpu at 1 {
@@ -41,6 +42,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <1>;
 			clocks = <&cpuclk 1>;
+			clock-latency = <1000000>;
 		};
 	};
 
diff --git a/arch/arm/boot/dts/armada-xp-mv78260.dtsi b/arch/arm/boot/dts/armada-xp-mv78260.dtsi
index 3396b25..480e237 100644
--- a/arch/arm/boot/dts/armada-xp-mv78260.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78260.dtsi
@@ -36,6 +36,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <0>;
 			clocks = <&cpuclk 0>;
+			clock-latency = <1000000>;
 		};
 
 		cpu at 1 {
@@ -43,6 +44,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <1>;
 			clocks = <&cpuclk 1>;
+			clock-latency = <1000000>;
 		};
 	};
 
diff --git a/arch/arm/boot/dts/armada-xp-mv78460.dtsi b/arch/arm/boot/dts/armada-xp-mv78460.dtsi
index 6da84bf..2c7b1fe 100644
--- a/arch/arm/boot/dts/armada-xp-mv78460.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78460.dtsi
@@ -37,6 +37,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <0>;
 			clocks = <&cpuclk 0>;
+			clock-latency = <1000000>;
 		};
 
 		cpu at 1 {
@@ -44,6 +45,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <1>;
 			clocks = <&cpuclk 1>;
+			clock-latency = <1000000>;
 		};
 
 		cpu at 2 {
@@ -51,6 +53,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <2>;
 			clocks = <&cpuclk 2>;
+			clock-latency = <1000000>;
 		};
 
 		cpu at 3 {
@@ -58,6 +61,7 @@
 			compatible = "marvell,sheeva-v7";
 			reg = <3>;
 			clocks = <&cpuclk 3>;
+			clock-latency = <1000000>;
 		};
 	};
 
diff --git a/arch/arm/boot/dts/armada-xp.dtsi b/arch/arm/boot/dts/armada-xp.dtsi
index 5902e83..bff9f6c 100644
--- a/arch/arm/boot/dts/armada-xp.dtsi
+++ b/arch/arm/boot/dts/armada-xp.dtsi
@@ -99,7 +99,7 @@
 			cpuclk: clock-complex at 18700 {
 				#clock-cells = <1>;
 				compatible = "marvell,armada-xp-cpu-clock";
-				reg = <0x18700 0xA0>;
+				reg = <0x18700 0xA0>, <0x1c054 0x10>;
 				clocks = <&coreclk 1>;
 			};
 
-- 
2.0.0

[PATCHv2 6/8] ARM: mvebu: allow enabling of cpufreq on Armada XP

[Thomas Petazzoni]

Now that all the pieces to enable cpufreq on Armada XP are in place,
we can make MACH_ARMADA_XP select ARCH_HAS_CPUFREQ to allow the usage
of dynamic frequency scaling on this platform.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
index b9bc599..1ff60ef 100644
--- a/arch/arm/mach-mvebu/Kconfig
+++ b/arch/arm/mach-mvebu/Kconfig
@@ -66,6 +66,7 @@ config MACH_ARMADA_XP
 	select CPU_PJ4B
 	select MACH_MVEBU_V7
 	select PINCTRL_ARMADA_XP
+	select ARCH_HAS_CPUFREQ
 	help
 	  Say 'Y' here if you want your kernel to support boards based
 	  on the Marvell Armada XP SoC with device tree.
-- 
2.0.0

[PATCHv2 7/8] ARM: mvebu: update mvebu_v7_defconfig with cpufreq support

[Thomas Petazzoni]

Now that the Armada XP supports dynamic CPU frequency scaling, it
makes sense to enable the cpufreq subsystem in mvebu_v7_defconfig, as
well as the cpufreq-generic driver used on Armada XP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/configs/mvebu_v7_defconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/configs/mvebu_v7_defconfig b/arch/arm/configs/mvebu_v7_defconfig
index b0bfefa..be2b5bc 100644
--- a/arch/arm/configs/mvebu_v7_defconfig
+++ b/arch/arm/configs/mvebu_v7_defconfig
@@ -29,6 +29,8 @@ CONFIG_ZBOOT_ROM_TEXT=0x0
 CONFIG_ZBOOT_ROM_BSS=0x0
 CONFIG_ARM_APPENDED_DTB=y
 CONFIG_ARM_ATAG_DTB_COMPAT=y
+CONFIG_CPU_FREQ=y
+CONFIG_CPUFREQ_GENERIC=y
 CONFIG_VFP=y
 CONFIG_NET=y
 CONFIG_INET=y
-- 
2.0.0

[PATCHv2 8/8] ARM: configs: add cpufreq-generic in multi_v7_defconfig

[Thomas Petazzoni]

The cpufreq-generic driver is going to be used by more and more
platforms to support dynamic frequency scaling of the CPU. This commit
enables this driver by default in multi_v7_defconfig.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/configs/multi_v7_defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v7_defconfig
index be1a345..80c1f60 100644
--- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -82,6 +82,7 @@ CONFIG_KEXEC=y
 CONFIG_CPU_FREQ=y
 CONFIG_CPU_FREQ_STAT_DETAILS=y
 CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
+CONFIG_CPUFREQ_GENERIC=y
 CONFIG_CPU_IDLE=y
 CONFIG_NET=y
 CONFIG_PACKET=y
-- 
2.0.0

[PATCHv2 1/8] clk: add an APPLY_RATE_CHANGE notifier event during clk_set_rate()

[Stephen Boyd]

On 07/07/14 07:51, Thomas Petazzoni wrote:
> The current clk_set_rate() logic allows notifiers to be notified of
> three different events:
>
>  * PRE_RATE_CHANGE: before the clock driver ->set_rate() function is
>    called.
>  * ABORT_RATE_CHANGE: called if the rate change failed
>  * POST_RATE_CHANGE: called after the rate change has been
>    successfully done, but after ->recalc_rate() has been called, and
>    only if the rate of the clock has actually changed.
>
> This commit adds an additional APPLY_RATE_CHANGE, which we require on
> Armada XP to implement dynamic frequency scaling of the CPU
> clocks. The problem is as follows.
>
> On Armada XP, there are three hardware blocks involved in the dynamic
> frequency scaling of the CPU clocks:
>
>  - The CPU clocks hardware block itself, whose registers are already
>    "managed" by drivers/clk/mvebu/clk-cpu.c (compatible string:
>    marvell,armada-xp-cpu-clock). The driver currently only supports
>    changing the rate of the CPU clock when the clock is off (i.e
>    before we boot the secondary CPUs).
>
>  - The PMU DFS registers, which are used to configure the target
>    frequency for a dynamic rate change. Those registers are relatively
>    well isolated from other PMU registers, so they can also be
>    "managed" by the drivers/clk/mvebu/clk-cpu.c.
>
>  - The PMSU registers, which are used to actually trigger the dynamic
>    frequency change procedure, which consists in programming a bunch
>    of PMSU registers, entering the idle state on the CPU we want to
>    change the frequency, and then again reprogram a bunch of PMSU
>    registers.
>
> The procedure to change the frequency is:
>
>  1/ Reset some clocks using the CPU clocks hardware block.
>  2/ Define the target frequency using the PMU DFS registers.
>  3/ Do the actual frequency change using the PMSU registers.
>
> However, the PMSU registers are already "managed" by a driver in
> arch/arm/mach-mvebu/pmsu.c, and the code there is needed for a wide
> variety of power management activities: booting of secondary CPUs, CPU
> hotplug, cpuidle, cpufreq, and soon suspend/resume. The same registers
> in the PMSU are used for several of those activities.
>
> So, what we need to do is to have steps (1) and (2) above done in the
> CPU clocks driver, and then step (3) done through a clk notifier.
>
> However, the current POST_RATE_CHANGE notifier is called too late
> (after ->recalc_rate) and only if the rate has changed. So it works
> fine for a pure notification of a frequency change, but it doesn't
> work if the notified code is actually involved in the frequency
> change, which is exactly the case we have here. Until the sequence
> that uses the PMSU registers has been executed, the rate of the CPU
> clock has not changed.
>
> In order to solve this problem, we propose to add an APPLY_RATE_CHANGE
> notifier event, which gets called right after ->set_rate(), but before
> ->recalc_rate(), and therefore regardless of whether there was an
> actualy frequency change or not.

Is there any reason why we can't call the pmsu code (part #3) directly
from the cpu clock driver? It seems like if we just called the
.set_rate() op we wouldn't actually have changed the clock's rate. That
doesn't seem very intuitive and it really makes the code flow hard to
follow.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCHv2 1/8] clk: add an APPLY_RATE_CHANGE notifier event during clk_set_rate()

[Thomas Petazzoni]

Dear Stephen Boyd,

On Mon, 07 Jul 2014 16:44:18 -0700, Stephen Boyd wrote:

> > In order to solve this problem, we propose to add an APPLY_RATE_CHANGE
> > notifier event, which gets called right after ->set_rate(), but before
> > ->recalc_rate(), and therefore regardless of whether there was an
> > actualy frequency change or not.
> 
> Is there any reason why we can't call the pmsu code (part #3) directly
> from the cpu clock driver? It seems like if we just called the
> .set_rate() op we wouldn't actually have changed the clock's rate. That
> doesn't seem very intuitive and it really makes the code flow hard to
> follow.

Right, but what solution would you propose to achieve that? These days,
a direct call from drivers/ code to arch/arm/mach-<foo>/ code is
frowned upon, no? (The code handling the PMSU is in
arch/arm/mach-mvebu/pmsu.c).

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCHv2 4/8] ARM: mvebu: extend PMSU code to support dynamic frequency scaling

[Ezequiel Garcia]

On 07 Jul 04:51 PM, Thomas Petazzoni wrote:
> This commit adds the necessary code in the Marvell EBU PMSU driver to
> support dynamic frequency scaling. In essence, what this new code does
> is that it:
> 
>  * registers the frequency operating points supported by the CPU;
> 
>  * registers a clock notifier of the CPU clocks. The notifier function
>    listens to the newly introduced APPLY_RATE_CHANGE event, and uses
>    that to finalize the frequency transition by doing the part of the
>    procedure that involves the PMSU;
> 
>  * registers a platform device for the cpufreq-generic driver, which
>    will take care of the CPU frequency transitions.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  arch/arm/mach-mvebu/pmsu.c | 184 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 184 insertions(+)
> 
> diff --git a/arch/arm/mach-mvebu/pmsu.c b/arch/arm/mach-mvebu/pmsu.c
> index 53a55c8..9257b16 100644
> --- a/arch/arm/mach-mvebu/pmsu.c
> +++ b/arch/arm/mach-mvebu/pmsu.c
> @@ -18,20 +18,26 @@
>  
>  #define pr_fmt(fmt) "mvebu-pmsu: " fmt
>  
> +#include <linux/clk.h>
>  #include <linux/cpu_pm.h>
> +#include <linux/delay.h>
>  #include <linux/kernel.h>
>  #include <linux/init.h>
>  #include <linux/of_address.h>
> +#include <linux/of_device.h>
>  #include <linux/io.h>
>  #include <linux/platform_device.h>
> +#include <linux/pm_opp.h>
>  #include <linux/smp.h>
>  #include <linux/resource.h>
> +#include <linux/slab.h>
>  #include <asm/cacheflush.h>
>  #include <asm/cp15.h>
>  #include <asm/smp_plat.h>
>  #include <asm/suspend.h>
>  #include <asm/tlbflush.h>
>  #include "common.h"
> +#include "armada-370-xp.h"
>  
>  static void __iomem *pmsu_mp_base;
>  
> @@ -57,6 +63,10 @@ static void __iomem *pmsu_mp_base;
>  #define PMSU_STATUS_AND_MASK_IRQ_MASK		BIT(24)
>  #define PMSU_STATUS_AND_MASK_FIQ_MASK		BIT(25)
>  
> +#define PMSU_EVENT_STATUS_AND_MASK(cpu)     ((cpu * 0x100) + 0x120)
> +#define PMSU_EVENT_STATUS_AND_MASK_DFS_DONE        BIT(1)
> +#define PMSU_EVENT_STATUS_AND_MASK_DFS_DONE_MASK   BIT(17)
> +
>  #define PMSU_BOOT_ADDR_REDIRECT_OFFSET(cpu) ((cpu * 0x100) + 0x124)
>  
>  /* PMSU fabric registers */
> @@ -296,3 +306,177 @@ int __init armada_370_xp_cpu_pm_init(void)
>  
>  arch_initcall(armada_370_xp_cpu_pm_init);
>  early_initcall(armada_370_xp_pmsu_init);
> +
> +static void armada_xp_cpufreq_clk_set_local(void *data)
> +{
> +	u32 reg;
> +	u32 cpu = smp_processor_id();
> +	unsigned long flags;
> +
> +	local_irq_save(flags);
> +
> +	/* Prepare to enter idle */
> +	reg = readl(pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
> +	reg |= PMSU_STATUS_AND_MASK_CPU_IDLE_WAIT |
> +	       PMSU_STATUS_AND_MASK_IRQ_MASK     |
> +	       PMSU_STATUS_AND_MASK_FIQ_MASK;
> +	writel(reg, pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
> +
> +	/* Request the DFS transition */
> +	reg = readl(pmsu_mp_base + PMSU_CONTROL_AND_CONFIG(cpu));
> +	reg |= PMSU_CONTROL_AND_CONFIG_DFS_REQ;
> +	writel(reg, pmsu_mp_base + PMSU_CONTROL_AND_CONFIG(cpu));
> +
> +	/* The fact of entering idle will trigger the DFS transition */
> +	wfi();
> +
> +	/*
> +	 * We're back from idle, the DFS transition has completed,
> +	 * clear the idle wait indication.
> +	 */
> +	reg = readl(pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
> +	reg &= ~PMSU_STATUS_AND_MASK_CPU_IDLE_WAIT;
> +	writel(reg, pmsu_mp_base + PMSU_STATUS_AND_MASK(cpu));
> +
> +	local_irq_restore(flags);
> +}
> +
> +struct armada_xp_cpufreq_notifier_block {
> +	struct notifier_block nb;
> +	int cpu;
> +};
> +
> +static int armada_xp_cpufreq_clk_notify(struct notifier_block *self,
> +					unsigned long action, void *data)
> +{
> +	struct armada_xp_cpufreq_notifier_block *nb =
> +		container_of(self, struct armada_xp_cpufreq_notifier_block, nb);
> +	unsigned long timeout;
> +	int cpu = cpu_logical_map(nb->cpu);
> +	u32 reg;
> +
> +	if (action != APPLY_RATE_CHANGE)
> +		return 0;

Maybe NOTIFY_DONE should be used here?

> +
> +	/* Clear any previous DFS DONE event */
> +	reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +	reg &= ~PMSU_EVENT_STATUS_AND_MASK_DFS_DONE;
> +	writel(reg, pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +
> +	/* Mask the DFS done interrupt, since we are going to poll */
> +	reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +	reg |= PMSU_EVENT_STATUS_AND_MASK_DFS_DONE_MASK;
> +	writel(reg, pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +
> +	/* Trigger the DFS on the appropriate CPU */
> +	smp_call_function_single(get_logical_index(cpu),
> +				 armada_xp_cpufreq_clk_set_local, NULL, false);
> +
> +	/* Poll until the DFS done event is generated */
> +	timeout = jiffies + HZ;
> +	while (time_before(jiffies, timeout)) {
> +		reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +		if (reg & PMSU_EVENT_STATUS_AND_MASK_DFS_DONE)
> +			break;
> +		udelay(10);
> +	}
> +
> +	/* Restore the DFS mask to its original state */
> +	reg = readl(pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +	reg &= ~BIT(17);
> +	writel(reg, pmsu_mp_base + PMSU_EVENT_STATUS_AND_MASK(cpu));
> +
> +	return NOTIFY_DONE;

I think NOTIFY_OK should be used here, not sure.

> +}
> +
> +static int __init armada_xp_pmsu_cpufreq_init(void)
> +{
> +	struct device_node *np;
> +	struct resource res;
> +	int ret, cpu;
> +
> +	if (!of_machine_is_compatible("marvell,armadaxp"))
> +		return 0;
> +
> +	/*
> +	 * In order to have proper cpufreq handling, we need to ensure
> +	 * that the Device Tree description of the CPU clock includes
> +	 * the definition of the PMU DFS registers. If not, we do not
> +	 * register the clock notifier and the cpufreq driver. This
> +	 * piece of code is only for compatibility with old Device
> +	 * Trees.
> +	 */
> +	np = of_find_compatible_node(NULL, NULL, "marvell,armada-xp-cpu-clock");
> +	if (!np)
> +		return 0;
> +
> +	ret = of_address_to_resource(np, 1, &res);
> +	if (ret) {
> +		pr_warn(FW_WARN "not enabling cpufreq, deprecated armada-xp-cpu-clock binding\n");
> +		of_node_put(np);
> +		return 0;
> +	}
> +
> +	of_node_put(np);
> +
> +	/*
> +	 * For each CPU, this loop registers the operating points
> +	 * supported (which are the nominal CPU frequency and half of
> +	 * it), and registers the clock notifier that will take care
> +	 * of doing the PMSU part of a frequency transition.
> +	 */
> +	for_each_possible_cpu(cpu) {
> +		struct clk *clk;
> +		struct device *cpu_dev;
> +		struct armada_xp_cpufreq_notifier_block *nbs;
> +		int ret;
> +
> +		cpu_dev = get_cpu_device(cpu);
> +		if (!cpu_dev) {
> +			pr_err("Cannot get CPU %d\n", cpu);
> +			continue;
> +		}
> +
> +		clk = clk_get(cpu_dev, 0);
> +		if (!clk) {
> +			pr_err("Cannot get clock for CPU %d\n", cpu);
> +			return -ENODEV;

You are leaking the notifier block here.

> +		}
> +
> +		ret = dev_pm_opp_add(cpu_dev, clk_get_rate(clk), 0);
> +		if (ret) {
> +			clk_put(clk);
> +			return ret;

Ditto.

> +		}
> +
> +		ret = dev_pm_opp_add(cpu_dev, clk_get_rate(clk) / 2, 0);
> +		if (ret) {
> +			clk_put(clk);
> +			return ret;

Ditto.

> +		}
> +
> +		nbs = kzalloc(sizeof(struct armada_xp_cpufreq_notifier_block),
> +			      GFP_KERNEL);
> +		if (!nbs) {
> +			pr_err("Cannot allocate memory for cpufreq notifier\n");

You don't need out of memory messages, to error will be really verbose by
itself.

> +			clk_put(clk);
> +			return -ENOMEM;

And here you leak as well.

> +		}
> +
> +		nbs->nb.notifier_call = armada_xp_cpufreq_clk_notify;

I'd say this is the notifier and use "armada_xp_cpufreq_clk_notifier",
but it's just a nitpick.

> +		nbs->cpu = cpu;
> +
> +		ret = clk_notifier_register(clk, &nbs->nb);
> +		if (ret) {
> +			pr_err("Cannot register clock notifier\n");
> +			kfree(nbs);
> +			clk_put(clk);
> +			return ret;

Ditto.

> +		}
> +	}
> +
> +	platform_device_register_simple("cpufreq-generic", -1, NULL, 0);
> +	return 0;
> +}
> +
> +device_initcall(armada_xp_pmsu_cpufreq_init);
> -- 
> 2.0.0
> 

Reviewed-by: Ezequiel Garcia <ezequiel.garcia@free-electrons.com>

-- 
Ezequiel Garc?a, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com

[PATCHv2 1/8] clk: add an APPLY_RATE_CHANGE notifier event during clk_set_rate()

[Stephen Boyd]

On 07/08/14 00:15, Thomas Petazzoni wrote:
> Dear Stephen Boyd,
>
> On Mon, 07 Jul 2014 16:44:18 -0700, Stephen Boyd wrote:
>
>>> In order to solve this problem, we propose to add an APPLY_RATE_CHANGE
>>> notifier event, which gets called right after ->set_rate(), but before
>>> ->recalc_rate(), and therefore regardless of whether there was an
>>> actualy frequency change or not.
>> Is there any reason why we can't call the pmsu code (part #3) directly
>> from the cpu clock driver? It seems like if we just called the
>> .set_rate() op we wouldn't actually have changed the clock's rate. That
>> doesn't seem very intuitive and it really makes the code flow hard to
>> follow.
> Right, but what solution would you propose to achieve that? These days,
> a direct call from drivers/ code to arch/arm/mach-<foo>/ code is
> frowned upon, no? (The code handling the PMSU is in
> arch/arm/mach-mvebu/pmsu.c).
>

I don't see a problem with having an include file in include/linux/ for
this, maybe others do though. If it actually is a problem then perhaps
moving the pmsu.c code into drivers/ is the right solution.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

[PATCHv2 1/8] clk: add an APPLY_RATE_CHANGE notifier event during clk_set_rate()

[Thomas Petazzoni]

Dear Stephen Boyd,

On Tue, 08 Jul 2014 13:18:54 -0700, Stephen Boyd wrote:

> >>> In order to solve this problem, we propose to add an APPLY_RATE_CHANGE
> >>> notifier event, which gets called right after ->set_rate(), but before
> >>> ->recalc_rate(), and therefore regardless of whether there was an
> >>> actualy frequency change or not.
> >> Is there any reason why we can't call the pmsu code (part #3) directly
> >> from the cpu clock driver? It seems like if we just called the
> >> .set_rate() op we wouldn't actually have changed the clock's rate. That
> >> doesn't seem very intuitive and it really makes the code flow hard to
> >> follow.
> > Right, but what solution would you propose to achieve that? These days,
> > a direct call from drivers/ code to arch/arm/mach-<foo>/ code is
> > frowned upon, no? (The code handling the PMSU is in
> > arch/arm/mach-mvebu/pmsu.c).
> 
> I don't see a problem with having an include file in include/linux/ for
> this, maybe others do though.

I'm fine with that as well, but I believe one of the policy was to
avoid having too much drivers/ stuff call into arch/arm/ stuff.

> If it actually is a problem then perhaps
> moving the pmsu.c code into drivers/ is the right solution.

Yes, but where would it belong? The PMSU hardware block is tightly
linked to SMP initialization (which means it should be up and running
very early), CPU hotplug, cpuidle, cpufreq and suspend/resume. It means
that it interacts with a lot of different subsystems.

Maybe the solution of adding a direct call from
drivers/clk/mvebu/clk-cpu.c to arch/arm/mach-mvebu/pmsu.c is the
easiest and most explicit solution.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com