[PATCH v2 0/4] OMAP4: CPUidle: Add coupled idle support

[PATCH v2 0/4] OMAP4: CPUidle: Add coupled idle support

[Santosh Shilimkar]

Kevin,

Now Colin's v3 [1] is apearing in Len Brown's next branch, I have rebased
OMAP4 support against it. I need to pick up couple of fixes [2] posted on
top of v3 [1] version and arm-soc 'omap/cpuidle-cleanup' branch lined up
for 3.5.

This series adds the coupled cpuidle support for OMAP4 and also removes
the hard dependency of off-lining CPU1 to trigger deeper CPU C-states.
It has been build tested for all OMAP builds and functionally tested
on OMAP4430 SDP.

I have put together a branch with:
1) Len Brown's 'next'
	: git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux 
2) ARM-SOC 'omap/cpuidle-cleanup'
	: git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
3) Two couple_idle_fixes picked up from [2]

And same is available:

git://gitorious.org/omap-sw-develoment/linux-omap-dev.git
for_3.5/omap4_couple_idle

Kevin Hilman (1):
  ARM: OMAP4: CPUidle: add synchronization for coupled idle states

Santosh Shilimkar (3):
  ARM: OMAP: timer: allow gp timer clock-event to be used on both cpus
  ARM: OMAP4: CPUidle: Use coupled cpuidle states to implement SMP
    cpuidle.
  ARM: OMAP4: CPUidle: Open broadcast clock-event device.

 arch/arm/mach-omap2/Kconfig       |    1 +
 arch/arm/mach-omap2/cpuidle44xx.c |  141 +++++++++++++++++++++++++------------
 arch/arm/mach-omap2/timer.c       |    4 +-
 3 files changed, 101 insertions(+), 45 deletions(-)

Regards
Santosh

[1] [PATCHv3 0/4] coupled cpuidle state support
 https://lkml.org/lkml/2012/5/19/24

[2] [PATCH 0/2] bug fixes for coupled cpuidle
 https://lkml.org/lkml/2012/5/18/379

-- 
1.7.5.4

[PATCH v2 0/4] OMAP4: CPUidle: Add coupled idle support

[Santosh Shilimkar]

Kevin,

Now Colin's v3 [1] is apearing in Len Brown's next branch, I have rebased
OMAP4 support against it. I need to pick up couple of fixes [2] posted on
top of v3 [1] version and arm-soc 'omap/cpuidle-cleanup' branch lined up
for 3.5.

This series adds the coupled cpuidle support for OMAP4 and also removes
the hard dependency of off-lining CPU1 to trigger deeper CPU C-states.
It has been build tested for all OMAP builds and functionally tested
on OMAP4430 SDP.

I have put together a branch with:
1) Len Brown's 'next'
	: git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux 
2) ARM-SOC 'omap/cpuidle-cleanup'
	: git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
3) Two couple_idle_fixes picked up from [2]

And same is available:

git://gitorious.org/omap-sw-develoment/linux-omap-dev.git
for_3.5/omap4_couple_idle

Kevin Hilman (1):
  ARM: OMAP4: CPUidle: add synchronization for coupled idle states

Santosh Shilimkar (3):
  ARM: OMAP: timer: allow gp timer clock-event to be used on both cpus
  ARM: OMAP4: CPUidle: Use coupled cpuidle states to implement SMP
    cpuidle.
  ARM: OMAP4: CPUidle: Open broadcast clock-event device.

 arch/arm/mach-omap2/Kconfig       |    1 +
 arch/arm/mach-omap2/cpuidle44xx.c |  141 +++++++++++++++++++++++++------------
 arch/arm/mach-omap2/timer.c       |    4 +-
 3 files changed, 101 insertions(+), 45 deletions(-)

Regards
Santosh

[1] [PATCHv3 0/4] coupled cpuidle state support
 https://lkml.org/lkml/2012/5/19/24

[2] [PATCH 0/2] bug fixes for coupled cpuidle
 https://lkml.org/lkml/2012/5/18/379

-- 
1.7.5.4

[PATCH v2 1/4] ARM: OMAP: timer: allow gp timer clock-event to be used on both cpus

[Santosh Shilimkar]

For coupled cpuidle to work when both cpus are active, it needs a global timer
that can handle events for both cpus.  This timer is used as the broadcast
clock-event when the per-cpu timer hardware stop in low power states.
Set the cpumask of clockevent_gpt to all cpus, set the rating correctly, and
set the irq to allow the clockevent core to determine the affinity of the
timer.

Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/timer.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-omap2/timer.c b/arch/arm/mach-omap2/timer.c
index c512bac..4212f2f 100644
--- a/arch/arm/mach-omap2/timer.c
+++ b/arch/arm/mach-omap2/timer.c
@@ -135,6 +135,7 @@ static struct clock_event_device clockevent_gpt = {
 	.name		= "gp timer",
 	.features       = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT,
 	.shift		= 32,
+	.rating		= 300,
 	.set_next_event	= omap2_gp_timer_set_next_event,
 	.set_mode	= omap2_gp_timer_set_mode,
 };
@@ -226,7 +227,8 @@ static void __init omap2_gp_clockevent_init(int gptimer_id,
 		clockevent_delta2ns(3, &clockevent_gpt);
 		/* Timer internal resynch latency. */
 
-	clockevent_gpt.cpumask = cpumask_of(0);
+	clockevent_gpt.cpumask = cpu_possible_mask;
+	clockevent_gpt.irq = omap_dm_timer_get_irq(&clkev);
 	clockevents_register_device(&clockevent_gpt);
 
 	pr_info("OMAP clockevent source: GPTIMER%d at %lu Hz\n",
-- 
1.7.5.4

[PATCH v2 1/4] ARM: OMAP: timer: allow gp timer clock-event to be used on both cpus

[Santosh Shilimkar]

For coupled cpuidle to work when both cpus are active, it needs a global timer
that can handle events for both cpus.  This timer is used as the broadcast
clock-event when the per-cpu timer hardware stop in low power states.
Set the cpumask of clockevent_gpt to all cpus, set the rating correctly, and
set the irq to allow the clockevent core to determine the affinity of the
timer.

Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/timer.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-omap2/timer.c b/arch/arm/mach-omap2/timer.c
index c512bac..4212f2f 100644
--- a/arch/arm/mach-omap2/timer.c
+++ b/arch/arm/mach-omap2/timer.c
@@ -135,6 +135,7 @@ static struct clock_event_device clockevent_gpt = {
 	.name		= "gp timer",
 	.features       = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT,
 	.shift		= 32,
+	.rating		= 300,
 	.set_next_event	= omap2_gp_timer_set_next_event,
 	.set_mode	= omap2_gp_timer_set_mode,
 };
@@ -226,7 +227,8 @@ static void __init omap2_gp_clockevent_init(int gptimer_id,
 		clockevent_delta2ns(3, &clockevent_gpt);
 		/* Timer internal resynch latency. */
 
-	clockevent_gpt.cpumask = cpumask_of(0);
+	clockevent_gpt.cpumask = cpu_possible_mask;
+	clockevent_gpt.irq = omap_dm_timer_get_irq(&clkev);
 	clockevents_register_device(&clockevent_gpt);
 
 	pr_info("OMAP clockevent source: GPTIMER%d at %lu Hz\n",
-- 
1.7.5.4

[PATCH v2 2/4] ARM: OMAP4: CPUidle: Use coupled cpuidle states to implement SMP cpuidle.

[Santosh Shilimkar]

OMAP4 CPUDILE driver is converted mainly based on notes from the
coupled cpuidle patch series.

The changes include :
- Register both CPUs and C-states to cpuidle driver.
- Set struct cpuidle_device.coupled_cpus
- Set struct cpuidle_device.safe_state to non coupled state.
- Set CPUIDLE_FLAG_COUPLED in struct cpuidle_state.flags for each
  state that affects multiple cpus.
- Separate ->enter hooks for coupled & simple idle.
- CPU0 wait loop for CPU1 power transition.
- CPU1 wakeup mechanism for the idle exit.
- Enabling ARCH_NEEDS_CPU_IDLE_COUPLED for OMAP4.

Thanks to Kevin Hilman and Colin Cross on the suggestions/fixes
on the intermediate version of this patch.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/Kconfig       |    1 +
 arch/arm/mach-omap2/cpuidle44xx.c |  112 ++++++++++++++++++++++---------------
 2 files changed, 67 insertions(+), 46 deletions(-)

diff --git a/arch/arm/mach-omap2/Kconfig b/arch/arm/mach-omap2/Kconfig
index 8141b76..8a12c40 100644
--- a/arch/arm/mach-omap2/Kconfig
+++ b/arch/arm/mach-omap2/Kconfig
@@ -54,6 +54,7 @@ config ARCH_OMAP4
 	select PM_OPP if PM
 	select USB_ARCH_HAS_EHCI if USB_SUPPORT
 	select ARM_CPU_SUSPEND if PM
+	select ARCH_NEEDS_CPU_IDLE_COUPLED
 
 comment "OMAP Core Type"
 	depends on ARCH_OMAP2
diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index be1617c..25655eb 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -21,6 +21,7 @@
 #include "common.h"
 #include "pm.h"
 #include "prm.h"
+#include "clockdomain.h"
 
 #ifdef CONFIG_CPU_IDLE
 
@@ -49,10 +50,11 @@ static struct omap4_idle_statedata omap4_idle_data[] = {
 	},
 };
 
-static struct powerdomain *mpu_pd, *cpu0_pd, *cpu1_pd;
+static struct powerdomain *mpu_pd, *cpu_pd[NR_CPUS];
+static struct clockdomain *cpu_clkdm[NR_CPUS];
 
 /**
- * omap4_enter_idle - Programs OMAP4 to enter the specified state
+ * omap4_enter_idle_coupled_[simple/coupled] - OMAP4 cpuidle entry functions
  * @dev: cpuidle device
  * @drv: cpuidle driver
  * @index: the index of state to be entered
@@ -61,60 +63,71 @@ static struct powerdomain *mpu_pd, *cpu0_pd, *cpu1_pd;
  * specified low power state selected by the governor.
  * Returns the amount of time spent in the low power state.
  */
-static int omap4_enter_idle(struct cpuidle_device *dev,
+static int omap4_enter_idle_simple(struct cpuidle_device *dev,
+			struct cpuidle_driver *drv,
+			int index)
+{
+	local_fiq_disable();
+	omap_do_wfi();
+	local_fiq_enable();
+
+	return index;
+}
+
+static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 			struct cpuidle_driver *drv,
 			int index)
 {
 	struct omap4_idle_statedata *cx = &omap4_idle_data[index];
-	u32 cpu1_state;
 	int cpu_id = smp_processor_id();
 
 	local_fiq_disable();
 
 	/*
-	 * CPU0 has to stay ON (i.e in C1) until CPU1 is OFF state.
+	 * CPU0 has to wait and stay ON until CPU1 is OFF state.
 	 * This is necessary to honour hardware recommondation
 	 * of triggeing all the possible low power modes once CPU1 is
 	 * out of coherency and in OFF mode.
-	 * Update dev->last_state so that governor stats reflects right
-	 * data.
 	 */
-	cpu1_state = pwrdm_read_pwrst(cpu1_pd);
-	if (cpu1_state != PWRDM_POWER_OFF) {
-		index = drv->safe_state_index;
-		cx = &omap4_idle_data[index];
+	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
+		while (pwrdm_read_pwrst(cpu_pd[1]) != PWRDM_POWER_OFF)
+			cpu_relax();
 	}
 
-	if (index > 0)
-		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
 
 	/*
 	 * Call idle CPU PM enter notifier chain so that
 	 * VFP and per CPU interrupt context is saved.
 	 */
-	if (cx->cpu_state == PWRDM_POWER_OFF)
-		cpu_pm_enter();
-
-	pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
-	omap_set_pwrdm_state(mpu_pd, cx->mpu_state);
-
-	/*
-	 * Call idle CPU cluster PM enter notifier chain
-	 * to save GIC and wakeupgen context.
-	 */
-	if ((cx->mpu_state == PWRDM_POWER_RET) &&
-		(cx->mpu_logic_state == PWRDM_POWER_OFF))
-			cpu_cluster_pm_enter();
+	cpu_pm_enter();
+
+	if (dev->cpu == 0) {
+		pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
+		omap_set_pwrdm_state(mpu_pd, cx->mpu_state);
+
+		/*
+		 * Call idle CPU cluster PM enter notifier chain
+		 * to save GIC and wakeupgen context.
+		 */
+		if ((cx->mpu_state == PWRDM_POWER_RET) &&
+			(cx->mpu_logic_state == PWRDM_POWER_OFF))
+				cpu_cluster_pm_enter();
+	}
 
 	omap4_enter_lowpower(dev->cpu, cx->cpu_state);
 
+	/* Wakeup CPU1 only if it is not offlined */
+	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
+		clkdm_wakeup(cpu_clkdm[1]);
+		clkdm_allow_idle(cpu_clkdm[1]);
+	}
+
 	/*
 	 * Call idle CPU PM exit notifier chain to restore
-	 * VFP and per CPU IRQ context. Only CPU0 state is
-	 * considered since CPU1 is managed by CPU hotplug.
+	 * VFP and per CPU IRQ context.
 	 */
-	if (pwrdm_read_prev_pwrst(cpu0_pd) == PWRDM_POWER_OFF)
-		cpu_pm_exit();
+	cpu_pm_exit();
 
 	/*
 	 * Call idle CPU cluster PM exit notifier chain
@@ -123,8 +136,7 @@ static int omap4_enter_idle(struct cpuidle_device *dev,
 	if (omap4_mpuss_read_prev_context_state())
 		cpu_cluster_pm_exit();
 
-	if (index > 0)
-		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
 
 	local_fiq_enable();
 
@@ -143,7 +155,7 @@ struct cpuidle_driver omap4_idle_driver = {
 			.exit_latency = 2 + 2,
 			.target_residency = 5,
 			.flags = CPUIDLE_FLAG_TIME_VALID,
-			.enter = omap4_enter_idle,
+			.enter = omap4_enter_idle_simple,
 			.name = "C1",
 			.desc = "MPUSS ON"
 		},
@@ -151,8 +163,8 @@ struct cpuidle_driver omap4_idle_driver = {
                         /* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */
 			.exit_latency = 328 + 440,
 			.target_residency = 960,
-			.flags = CPUIDLE_FLAG_TIME_VALID,
-			.enter = omap4_enter_idle,
+			.flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED,
+			.enter = omap4_enter_idle_coupled,
 			.name = "C2",
 			.desc = "MPUSS CSWR",
 		},
@@ -160,8 +172,8 @@ struct cpuidle_driver omap4_idle_driver = {
 			/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */
 			.exit_latency = 460 + 518,
 			.target_residency = 1100,
-			.flags = CPUIDLE_FLAG_TIME_VALID,
-			.enter = omap4_enter_idle,
+			.flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED,
+			.enter = omap4_enter_idle_coupled,
 			.name = "C3",
 			.desc = "MPUSS OSWR",
 		},
@@ -182,19 +194,27 @@ int __init omap4_idle_init(void)
 	unsigned int cpu_id = 0;
 
 	mpu_pd = pwrdm_lookup("mpu_pwrdm");
-	cpu0_pd = pwrdm_lookup("cpu0_pwrdm");
-	cpu1_pd = pwrdm_lookup("cpu1_pwrdm");
-	if ((!mpu_pd) || (!cpu0_pd) || (!cpu1_pd))
+	cpu_pd[0] = pwrdm_lookup("cpu0_pwrdm");
+	cpu_pd[1] = pwrdm_lookup("cpu1_pwrdm");
+	if ((!mpu_pd) || (!cpu_pd[0]) || (!cpu_pd[1]))
+		return -ENODEV;
+
+	cpu_clkdm[0] = clkdm_lookup("mpu0_clkdm");
+	cpu_clkdm[1] = clkdm_lookup("mpu1_clkdm");
+	if (!cpu_clkdm[0] || !cpu_clkdm[1])
 		return -ENODEV;
 
-	dev = &per_cpu(omap4_idle_dev, cpu_id);
-	dev->cpu = cpu_id;
+	for_each_cpu(cpu_id, cpu_online_mask) {
+		dev = &per_cpu(omap4_idle_dev, cpu_id);
+		dev->cpu = cpu_id;
+		dev->coupled_cpus = *cpu_online_mask;
 
-	cpuidle_register_driver(&omap4_idle_driver);
+		cpuidle_register_driver(&omap4_idle_driver);
 
-	if (cpuidle_register_device(dev)) {
-		pr_err("%s: CPUidle register device failed\n", __func__);
-		return -EIO;
+		if (cpuidle_register_device(dev)) {
+			pr_err("%s: CPUidle register failed\n", __func__);
+			return -EIO;
+		}
 	}
 
 	return 0;
-- 
1.7.5.4

[PATCH v2 2/4] ARM: OMAP4: CPUidle: Use coupled cpuidle states to implement SMP cpuidle.

[Santosh Shilimkar]

OMAP4 CPUDILE driver is converted mainly based on notes from the
coupled cpuidle patch series.

The changes include :
- Register both CPUs and C-states to cpuidle driver.
- Set struct cpuidle_device.coupled_cpus
- Set struct cpuidle_device.safe_state to non coupled state.
- Set CPUIDLE_FLAG_COUPLED in struct cpuidle_state.flags for each
  state that affects multiple cpus.
- Separate ->enter hooks for coupled & simple idle.
- CPU0 wait loop for CPU1 power transition.
- CPU1 wakeup mechanism for the idle exit.
- Enabling ARCH_NEEDS_CPU_IDLE_COUPLED for OMAP4.

Thanks to Kevin Hilman and Colin Cross on the suggestions/fixes
on the intermediate version of this patch.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/Kconfig       |    1 +
 arch/arm/mach-omap2/cpuidle44xx.c |  112 ++++++++++++++++++++++---------------
 2 files changed, 67 insertions(+), 46 deletions(-)

diff --git a/arch/arm/mach-omap2/Kconfig b/arch/arm/mach-omap2/Kconfig
index 8141b76..8a12c40 100644
--- a/arch/arm/mach-omap2/Kconfig
+++ b/arch/arm/mach-omap2/Kconfig
@@ -54,6 +54,7 @@ config ARCH_OMAP4
 	select PM_OPP if PM
 	select USB_ARCH_HAS_EHCI if USB_SUPPORT
 	select ARM_CPU_SUSPEND if PM
+	select ARCH_NEEDS_CPU_IDLE_COUPLED
 
 comment "OMAP Core Type"
 	depends on ARCH_OMAP2
diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index be1617c..25655eb 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -21,6 +21,7 @@
 #include "common.h"
 #include "pm.h"
 #include "prm.h"
+#include "clockdomain.h"
 
 #ifdef CONFIG_CPU_IDLE
 
@@ -49,10 +50,11 @@ static struct omap4_idle_statedata omap4_idle_data[] = {
 	},
 };
 
-static struct powerdomain *mpu_pd, *cpu0_pd, *cpu1_pd;
+static struct powerdomain *mpu_pd, *cpu_pd[NR_CPUS];
+static struct clockdomain *cpu_clkdm[NR_CPUS];
 
 /**
- * omap4_enter_idle - Programs OMAP4 to enter the specified state
+ * omap4_enter_idle_coupled_[simple/coupled] - OMAP4 cpuidle entry functions
  * @dev: cpuidle device
  * @drv: cpuidle driver
  * @index: the index of state to be entered
@@ -61,60 +63,71 @@ static struct powerdomain *mpu_pd, *cpu0_pd, *cpu1_pd;
  * specified low power state selected by the governor.
  * Returns the amount of time spent in the low power state.
  */
-static int omap4_enter_idle(struct cpuidle_device *dev,
+static int omap4_enter_idle_simple(struct cpuidle_device *dev,
+			struct cpuidle_driver *drv,
+			int index)
+{
+	local_fiq_disable();
+	omap_do_wfi();
+	local_fiq_enable();
+
+	return index;
+}
+
+static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 			struct cpuidle_driver *drv,
 			int index)
 {
 	struct omap4_idle_statedata *cx = &omap4_idle_data[index];
-	u32 cpu1_state;
 	int cpu_id = smp_processor_id();
 
 	local_fiq_disable();
 
 	/*
-	 * CPU0 has to stay ON (i.e in C1) until CPU1 is OFF state.
+	 * CPU0 has to wait and stay ON until CPU1 is OFF state.
 	 * This is necessary to honour hardware recommondation
 	 * of triggeing all the possible low power modes once CPU1 is
 	 * out of coherency and in OFF mode.
-	 * Update dev->last_state so that governor stats reflects right
-	 * data.
 	 */
-	cpu1_state = pwrdm_read_pwrst(cpu1_pd);
-	if (cpu1_state != PWRDM_POWER_OFF) {
-		index = drv->safe_state_index;
-		cx = &omap4_idle_data[index];
+	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
+		while (pwrdm_read_pwrst(cpu_pd[1]) != PWRDM_POWER_OFF)
+			cpu_relax();
 	}
 
-	if (index > 0)
-		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
 
 	/*
 	 * Call idle CPU PM enter notifier chain so that
 	 * VFP and per CPU interrupt context is saved.
 	 */
-	if (cx->cpu_state == PWRDM_POWER_OFF)
-		cpu_pm_enter();
-
-	pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
-	omap_set_pwrdm_state(mpu_pd, cx->mpu_state);
-
-	/*
-	 * Call idle CPU cluster PM enter notifier chain
-	 * to save GIC and wakeupgen context.
-	 */
-	if ((cx->mpu_state == PWRDM_POWER_RET) &&
-		(cx->mpu_logic_state == PWRDM_POWER_OFF))
-			cpu_cluster_pm_enter();
+	cpu_pm_enter();
+
+	if (dev->cpu == 0) {
+		pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
+		omap_set_pwrdm_state(mpu_pd, cx->mpu_state);
+
+		/*
+		 * Call idle CPU cluster PM enter notifier chain
+		 * to save GIC and wakeupgen context.
+		 */
+		if ((cx->mpu_state == PWRDM_POWER_RET) &&
+			(cx->mpu_logic_state == PWRDM_POWER_OFF))
+				cpu_cluster_pm_enter();
+	}
 
 	omap4_enter_lowpower(dev->cpu, cx->cpu_state);
 
+	/* Wakeup CPU1 only if it is not offlined */
+	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
+		clkdm_wakeup(cpu_clkdm[1]);
+		clkdm_allow_idle(cpu_clkdm[1]);
+	}
+
 	/*
 	 * Call idle CPU PM exit notifier chain to restore
-	 * VFP and per CPU IRQ context. Only CPU0 state is
-	 * considered since CPU1 is managed by CPU hotplug.
+	 * VFP and per CPU IRQ context.
 	 */
-	if (pwrdm_read_prev_pwrst(cpu0_pd) == PWRDM_POWER_OFF)
-		cpu_pm_exit();
+	cpu_pm_exit();
 
 	/*
 	 * Call idle CPU cluster PM exit notifier chain
@@ -123,8 +136,7 @@ static int omap4_enter_idle(struct cpuidle_device *dev,
 	if (omap4_mpuss_read_prev_context_state())
 		cpu_cluster_pm_exit();
 
-	if (index > 0)
-		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
 
 	local_fiq_enable();
 
@@ -143,7 +155,7 @@ struct cpuidle_driver omap4_idle_driver = {
 			.exit_latency = 2 + 2,
 			.target_residency = 5,
 			.flags = CPUIDLE_FLAG_TIME_VALID,
-			.enter = omap4_enter_idle,
+			.enter = omap4_enter_idle_simple,
 			.name = "C1",
 			.desc = "MPUSS ON"
 		},
@@ -151,8 +163,8 @@ struct cpuidle_driver omap4_idle_driver = {
                         /* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */
 			.exit_latency = 328 + 440,
 			.target_residency = 960,
-			.flags = CPUIDLE_FLAG_TIME_VALID,
-			.enter = omap4_enter_idle,
+			.flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED,
+			.enter = omap4_enter_idle_coupled,
 			.name = "C2",
 			.desc = "MPUSS CSWR",
 		},
@@ -160,8 +172,8 @@ struct cpuidle_driver omap4_idle_driver = {
 			/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */
 			.exit_latency = 460 + 518,
 			.target_residency = 1100,
-			.flags = CPUIDLE_FLAG_TIME_VALID,
-			.enter = omap4_enter_idle,
+			.flags = CPUIDLE_FLAG_TIME_VALID | CPUIDLE_FLAG_COUPLED,
+			.enter = omap4_enter_idle_coupled,
 			.name = "C3",
 			.desc = "MPUSS OSWR",
 		},
@@ -182,19 +194,27 @@ int __init omap4_idle_init(void)
 	unsigned int cpu_id = 0;
 
 	mpu_pd = pwrdm_lookup("mpu_pwrdm");
-	cpu0_pd = pwrdm_lookup("cpu0_pwrdm");
-	cpu1_pd = pwrdm_lookup("cpu1_pwrdm");
-	if ((!mpu_pd) || (!cpu0_pd) || (!cpu1_pd))
+	cpu_pd[0] = pwrdm_lookup("cpu0_pwrdm");
+	cpu_pd[1] = pwrdm_lookup("cpu1_pwrdm");
+	if ((!mpu_pd) || (!cpu_pd[0]) || (!cpu_pd[1]))
+		return -ENODEV;
+
+	cpu_clkdm[0] = clkdm_lookup("mpu0_clkdm");
+	cpu_clkdm[1] = clkdm_lookup("mpu1_clkdm");
+	if (!cpu_clkdm[0] || !cpu_clkdm[1])
 		return -ENODEV;
 
-	dev = &per_cpu(omap4_idle_dev, cpu_id);
-	dev->cpu = cpu_id;
+	for_each_cpu(cpu_id, cpu_online_mask) {
+		dev = &per_cpu(omap4_idle_dev, cpu_id);
+		dev->cpu = cpu_id;
+		dev->coupled_cpus = *cpu_online_mask;
 
-	cpuidle_register_driver(&omap4_idle_driver);
+		cpuidle_register_driver(&omap4_idle_driver);
 
-	if (cpuidle_register_device(dev)) {
-		pr_err("%s: CPUidle register device failed\n", __func__);
-		return -EIO;
+		if (cpuidle_register_device(dev)) {
+			pr_err("%s: CPUidle register failed\n", __func__);
+			return -EIO;
+		}
 	}
 
 	return 0;
-- 
1.7.5.4

[PATCH v2 3/4] ARM: OMAP4: CPUidle: add synchronization for coupled idle states

[Santosh Shilimkar]

From: Kevin Hilman <khilman@ti.com>

With coupled idle states, a failure for any CPU to hit a low power
state must be coordinated such that all CPUs abort.

On OMAP4, when entering a coupled state, CPU0 has to wait for CPU1 to
enter its low power state before it can enter its low power state.

This is implemented by letting CPU0 wait for the CPU1 powerdomain to
hit off.  However, there are conditions where CPU1 might abort/fail
and not hit off while CPU0 is waiting for it.  For example, a CPU1
wakeup or a failed attempt to hit off due to hardware conditions.

To avoid the deadlock where CPU0 would continually wait for CPU1 to
hit off-mode, this patch adds a flag to signal when each CPU has come
out of its low-power state.  CPU0 then checks whether CPU1 has hit off
*or* has already completed its attempt to hit off.  If the latter,
CPU0 must abort its attempt to hit a low-power state so the coupled
state enter method can return.

In addition, cpuidle_coupled_parallel_barrier() is used to ensure the
clearing of the 'done' flag is synchronized on all CPUs.

Signed-off-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/cpuidle44xx.c |   22 +++++++++++++++++++++-
 1 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index 25655eb..eb93e45 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -53,6 +53,9 @@ static struct omap4_idle_statedata omap4_idle_data[] = {
 static struct powerdomain *mpu_pd, *cpu_pd[NR_CPUS];
 static struct clockdomain *cpu_clkdm[NR_CPUS];
 
+static atomic_t abort_barrier;
+static bool cpu_done[NR_CPUS];
+
 /**
  * omap4_enter_idle_coupled_[simple/coupled] - OMAP4 cpuidle entry functions
  * @dev: cpuidle device
@@ -90,8 +93,20 @@ static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 	 * out of coherency and in OFF mode.
 	 */
 	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
-		while (pwrdm_read_pwrst(cpu_pd[1]) != PWRDM_POWER_OFF)
+		while (pwrdm_read_pwrst(cpu_pd[1]) != PWRDM_POWER_OFF) {
 			cpu_relax();
+
+			/*
+			 * CPU1 could have already entered & exited idle
+			 * without hitting off because of a wakeup
+			 * or a failed attempt to hit off mode.  Check for
+			 * that here, otherwise we could spin forever
+			 * waiting for CPU1 off.
+			 */
+			if (cpu_done[1])
+			    goto fail;
+
+		}
 	}
 
 	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
@@ -116,6 +131,7 @@ static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 	}
 
 	omap4_enter_lowpower(dev->cpu, cx->cpu_state);
+	cpu_done[dev->cpu] = true;
 
 	/* Wakeup CPU1 only if it is not offlined */
 	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
@@ -138,6 +154,10 @@ static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 
 	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
 
+fail:
+	cpuidle_coupled_parallel_barrier(dev, &abort_barrier);
+	cpu_done[dev->cpu] = false;
+
 	local_fiq_enable();
 
 	return index;
-- 
1.7.5.4

[PATCH v2 3/4] ARM: OMAP4: CPUidle: add synchronization for coupled idle states

[Santosh Shilimkar]

From: Kevin Hilman <khilman@ti.com>

With coupled idle states, a failure for any CPU to hit a low power
state must be coordinated such that all CPUs abort.

On OMAP4, when entering a coupled state, CPU0 has to wait for CPU1 to
enter its low power state before it can enter its low power state.

This is implemented by letting CPU0 wait for the CPU1 powerdomain to
hit off.  However, there are conditions where CPU1 might abort/fail
and not hit off while CPU0 is waiting for it.  For example, a CPU1
wakeup or a failed attempt to hit off due to hardware conditions.

To avoid the deadlock where CPU0 would continually wait for CPU1 to
hit off-mode, this patch adds a flag to signal when each CPU has come
out of its low-power state.  CPU0 then checks whether CPU1 has hit off
*or* has already completed its attempt to hit off.  If the latter,
CPU0 must abort its attempt to hit a low-power state so the coupled
state enter method can return.

In addition, cpuidle_coupled_parallel_barrier() is used to ensure the
clearing of the 'done' flag is synchronized on all CPUs.

Signed-off-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/cpuidle44xx.c |   22 +++++++++++++++++++++-
 1 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index 25655eb..eb93e45 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -53,6 +53,9 @@ static struct omap4_idle_statedata omap4_idle_data[] = {
 static struct powerdomain *mpu_pd, *cpu_pd[NR_CPUS];
 static struct clockdomain *cpu_clkdm[NR_CPUS];
 
+static atomic_t abort_barrier;
+static bool cpu_done[NR_CPUS];
+
 /**
  * omap4_enter_idle_coupled_[simple/coupled] - OMAP4 cpuidle entry functions
  * @dev: cpuidle device
@@ -90,8 +93,20 @@ static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 	 * out of coherency and in OFF mode.
 	 */
 	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
-		while (pwrdm_read_pwrst(cpu_pd[1]) != PWRDM_POWER_OFF)
+		while (pwrdm_read_pwrst(cpu_pd[1]) != PWRDM_POWER_OFF) {
 			cpu_relax();
+
+			/*
+			 * CPU1 could have already entered & exited idle
+			 * without hitting off because of a wakeup
+			 * or a failed attempt to hit off mode.  Check for
+			 * that here, otherwise we could spin forever
+			 * waiting for CPU1 off.
+			 */
+			if (cpu_done[1])
+			    goto fail;
+
+		}
 	}
 
 	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
@@ -116,6 +131,7 @@ static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 	}
 
 	omap4_enter_lowpower(dev->cpu, cx->cpu_state);
+	cpu_done[dev->cpu] = true;
 
 	/* Wakeup CPU1 only if it is not offlined */
 	if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
@@ -138,6 +154,10 @@ static int omap4_enter_idle_coupled(struct cpuidle_device *dev,
 
 	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
 
+fail:
+	cpuidle_coupled_parallel_barrier(dev, &abort_barrier);
+	cpu_done[dev->cpu] = false;
+
 	local_fiq_enable();
 
 	return index;
-- 
1.7.5.4

[PATCH v2 4/4] ARM: OMAP4: CPUidle: Open broadcast clock-event device.

[Santosh Shilimkar]

OMAP4 idle driver uses CLOCK_EVT_NOTIFY_BROADCAST_[ENTER/EXIT]
for broadcast clock events. But _ENTER/_EXIT doesn't really open
broadcast clock events and to explicitly setup the broadcast device,
CLOCK_EVT_NOTIFY_BROADCAST_ON should be used.

Add the missing CLOCK_EVT_NOTIFY_BROADCAST_ON clockevent notifications.
This will setup the broadcast timer in either periodic/oneshot modes
correctly. Recent clockevent infrastructure change 77b0d60 {leave the
broadcast device in shutdown mode when not needed} exposed this bug
leading to boot hangs in oneshot mode. Prior to this, periodic broadcast
mode was also broken. This change fixes both the periodic/oneshot broadcast
modes.

Discussion thread :
	https://lkml.org/lkml/2012/4/9/13

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/cpuidle44xx.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index eb93e45..45e6a54 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -202,6 +202,16 @@ struct cpuidle_driver omap4_idle_driver = {
 	.safe_state_index = 0,
 };
 
+/*
+ * For each cpu, setup the broadcast timer because local timers
+ * stops for the states above C1.
+ */
+static void omap_setup_broadcast_timer(void *arg)
+{
+	int cpu = smp_processor_id();
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ON, &cpu);
+}
+
 /**
  * omap4_idle_init - Init routine for OMAP4 idle
  *
@@ -224,6 +234,9 @@ int __init omap4_idle_init(void)
 	if (!cpu_clkdm[0] || !cpu_clkdm[1])
 		return -ENODEV;
 
+	/* Configure the broadcast timer on each cpu */
+	on_each_cpu(omap_setup_broadcast_timer, NULL, 1);
+
 	for_each_cpu(cpu_id, cpu_online_mask) {
 		dev = &per_cpu(omap4_idle_dev, cpu_id);
 		dev->cpu = cpu_id;
-- 
1.7.5.4

[PATCH v2 4/4] ARM: OMAP4: CPUidle: Open broadcast clock-event device.

[Santosh Shilimkar]

OMAP4 idle driver uses CLOCK_EVT_NOTIFY_BROADCAST_[ENTER/EXIT]
for broadcast clock events. But _ENTER/_EXIT doesn't really open
broadcast clock events and to explicitly setup the broadcast device,
CLOCK_EVT_NOTIFY_BROADCAST_ON should be used.

Add the missing CLOCK_EVT_NOTIFY_BROADCAST_ON clockevent notifications.
This will setup the broadcast timer in either periodic/oneshot modes
correctly. Recent clockevent infrastructure change 77b0d60 {leave the
broadcast device in shutdown mode when not needed} exposed this bug
leading to boot hangs in oneshot mode. Prior to this, periodic broadcast
mode was also broken. This change fixes both the periodic/oneshot broadcast
modes.

Discussion thread :
	https://lkml.org/lkml/2012/4/9/13

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/mach-omap2/cpuidle44xx.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index eb93e45..45e6a54 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -202,6 +202,16 @@ struct cpuidle_driver omap4_idle_driver = {
 	.safe_state_index = 0,
 };
 
+/*
+ * For each cpu, setup the broadcast timer because local timers
+ * stops for the states above C1.
+ */
+static void omap_setup_broadcast_timer(void *arg)
+{
+	int cpu = smp_processor_id();
+	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ON, &cpu);
+}
+
 /**
  * omap4_idle_init - Init routine for OMAP4 idle
  *
@@ -224,6 +234,9 @@ int __init omap4_idle_init(void)
 	if (!cpu_clkdm[0] || !cpu_clkdm[1])
 		return -ENODEV;
 
+	/* Configure the broadcast timer on each cpu */
+	on_each_cpu(omap_setup_broadcast_timer, NULL, 1);
+
 	for_each_cpu(cpu_id, cpu_online_mask) {
 		dev = &per_cpu(omap4_idle_dev, cpu_id);
 		dev->cpu = cpu_id;
-- 
1.7.5.4

Re: [PATCH v2 0/4] OMAP4: CPUidle: Add coupled idle support

[Kevin Hilman]

Hi Santosh,

Santosh Shilimkar <santosh.shilimkar@ti.com> writes:

> Kevin,
>
> Now Colin's v3 [1] is apearing in Len Brown's next branch, I have rebased
> OMAP4 support against it. I need to pick up couple of fixes [2] posted on
> top of v3 [1] version and arm-soc 'omap/cpuidle-cleanup' branch lined up
> for 3.5.

Great.

> This series adds the coupled cpuidle support for OMAP4 and also removes
> the hard dependency of off-lining CPU1 to trigger deeper CPU C-states.
> It has been build tested for all OMAP builds and functionally tested
> on OMAP4430 SDP.
>
> I have put together a branch with:
> 1) Len Brown's 'next'
> 	: git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux 
> 2) ARM-SOC 'omap/cpuidle-cleanup'
> 	: git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
> 3) Two couple_idle_fixes picked up from [2]
>
> And same is available:
>
> git://gitorious.org/omap-sw-develoment/linux-omap-dev.git
> for_3.5/omap4_couple_idle

Excellent, thanks for pulling this together with the dependencies.

Linus has now merged the arm-soc/pm branch containing the OMAP CPUidle
cleanup from Daniel.  Also, I just ping'd Len on the status of Colin's
fixes.

As soon as all the depenencies are queued/merged I'll rebase this
on the right mainline commit and get this queued up for v3.6.

Thanks for keeping on top of this,

Kevin

[PATCH v2 0/4] OMAP4: CPUidle: Add coupled idle support

[Kevin Hilman]

Hi Santosh,

Santosh Shilimkar <santosh.shilimkar@ti.com> writes:

> Kevin,
>
> Now Colin's v3 [1] is apearing in Len Brown's next branch, I have rebased
> OMAP4 support against it. I need to pick up couple of fixes [2] posted on
> top of v3 [1] version and arm-soc 'omap/cpuidle-cleanup' branch lined up
> for 3.5.

Great.

> This series adds the coupled cpuidle support for OMAP4 and also removes
> the hard dependency of off-lining CPU1 to trigger deeper CPU C-states.
> It has been build tested for all OMAP builds and functionally tested
> on OMAP4430 SDP.
>
> I have put together a branch with:
> 1) Len Brown's 'next'
> 	: git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux 
> 2) ARM-SOC 'omap/cpuidle-cleanup'
> 	: git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
> 3) Two couple_idle_fixes picked up from [2]
>
> And same is available:
>
> git://gitorious.org/omap-sw-develoment/linux-omap-dev.git
> for_3.5/omap4_couple_idle

Excellent, thanks for pulling this together with the dependencies.

Linus has now merged the arm-soc/pm branch containing the OMAP CPUidle
cleanup from Daniel.  Also, I just ping'd Len on the status of Colin's
fixes.

As soon as all the depenencies are queued/merged I'll rebase this
on the right mainline commit and get this queued up for v3.6.

Thanks for keeping on top of this,

Kevin