[PATCH v2 0/5] CPU PM notifiers

[PATCH v2 0/5] CPU PM notifiers

[Santosh Shilimkar]

Updates V2:
- The CPU PM notifiers are moved to generic level instead of keeping
  it ARM specific as per Rafael's comment.
- To avoid every driver duplicating the sys-core ops and cpu pm notfiers,
  the CPU PM common notifier is registering it in one place so that
  suspend is taken care.
- CPU COMPLEX events are renamed to more appropriate CPU CLUSTER.
- GIC distributor is not disabled to avoid the live locks. This is
  observed with CPUIDLE cases where at time CPU cluster fails to
  transition to low power states. As such there was no need to
  disable distributor in first place since on cluster reset, it
  will get disabled anyways.
- In VFP code, syscore ops are dropped in favour of CPU PM
  notifiers since S2R is already handled in core CPU PM notifiers.
- The last patch is an independent patch but can be pushed along with
  the other GIC changes done here. 
 
V1: [http://lwn.net/Articles/447259/]

This patch set tries to address concerns with platform pm code
calling into the driver for every block in the Cortex A9s
during idle, hotplug, and suspend.  The first patch adds cpu pm
notifiers that can be called by platform code, the second uses
the notifier to save and restore the GIC state, and the third
saves the VFP state.

The notifiers are used for two types of events, CPU PM events and
CPU cluster PM events.  CPU PM events are used to save and restore
per-cpu context when a single CPU is preparing to enter or has
just exited a low power state.  For example, the VFP saves the
last thread context, and the GIC saves banked CPU registers.

CPU cluster events are used after all the CPUs in a power domain
have been prepared for the low power state.  The GIC uses these
events to save global register state.

L2 cache is not covered by this patch set, as the determination
of when the L2 is reset and when it is retained is
platform-specific, and most of the APIs necessary are already


The series is tested with OMAP4 with S2R and CPUIDLE. 

The following changes since commit c6a389f123b9f68d605bb7e0f9b32ec1e3e14132:

  Linux 3.1-rc4 (2011-08-28 21:16:01 -0700)

Colin Cross (5):
      cpu_pm: Add cpu power management notifiers
      cpu_pm: call notifiers during suspend
      ARM: gic: Use cpu pm notifiers to save gic state
      ARM: vfp: Use cpu pm notifiers to save vfp state
      ARM: gic: Allow gic arch extensions to provide irqchip flags

 arch/arm/common/gic.c               |  188 +++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/hardware/gic.h |    8 ++
 arch/arm/vfp/vfpmodule.c            |   29 ++++--
 include/linux/cpu_pm.h              |  123 +++++++++++++++++++++++
 kernel/Makefile                     |    1 +
 kernel/cpu_pm.c                     |  148 +++++++++++++++++++++++++++
 kernel/power/Kconfig                |    4 +
 7 files changed, 493 insertions(+), 8 deletions(-)
 create mode 100644 include/linux/cpu_pm.h
 create mode 100644 kernel/cpu_pm.c

[PATCH v2 1/5] cpu_pm: Add cpu power management notifiers

[Santosh Shilimkar]

From: Colin Cross <ccross@android.com>

During some CPU power modes entered during idle, hotplug and
suspend, peripherals located in the CPU power domain, such as
the GIC, localtimers, and VFP, may be powered down.  Add a
notifier chain that allows drivers for those peripherals to
be notified before and after they may be reset.

Signed-off-by: Colin Cross <ccross@android.com>
[santosh.shilimkar at ti.com: Rebased against 3.1-rc3]
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
---
 include/linux/cpu_pm.h |  123 ++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/Makefile        |    1 +
 kernel/cpu_pm.c        |  115 ++++++++++++++++++++++++++++++++++++++++++++
 kernel/power/Kconfig   |    4 ++
 4 files changed, 243 insertions(+), 0 deletions(-)
 create mode 100644 include/linux/cpu_pm.h
 create mode 100644 kernel/cpu_pm.c

diff --git a/include/linux/cpu_pm.h b/include/linux/cpu_pm.h
new file mode 100644
index 0000000..3c880cb
--- /dev/null
+++ b/include/linux/cpu_pm.h
@@ -0,0 +1,123 @@
+/*
+ * Copyright (C) 2011 Google, Inc.
+ *
+ * Author:
+ *	Colin Cross <ccross@android.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _LINUX_CPU_PM_H
+#define _LINUX_CPU_PM_H
+
+#include <linux/kernel.h>
+#include <linux/notifier.h>
+
+/*
+ * When a CPU goes to a low power state that turns off power to the CPU's
+ * power domain, the contents of some blocks (floating point coprocessors,
+ * interrutp controllers, caches, timers) in the same power domain can
+ * be lost.  The cpm_pm notifiers provide a method for platform idle, suspend,
+ * and hotplug implementations to notify the drivers for these blocks that
+ * they may be reset.
+ *
+ * All cpu_pm notifications must be called with interrupts disabled.
+ *
+ * The notifications are split into two classes, CPU notifications and CPU
+ * cluster notifications.
+ *
+ * CPU notifications apply to a single CPU, and must be called on the affected
+ * CPU.  They are used to save per-cpu context for affected blocks.
+ *
+ * CPU cluster notifications apply to all CPUs in a single power domain. They
+ * are used to save any global context for affected blocks, and must be called
+ * after all the CPUs in the power domain have been notified of the low power
+ * state.
+ *
+ */
+
+/*
+ * Event codes passed as unsigned long val to notifier calls
+ */
+enum cpu_pm_event {
+	/* A single cpu is entering a low power state */
+	CPU_PM_ENTER,
+
+	/* A single cpu failed to enter a low power state */
+	CPU_PM_ENTER_FAILED,
+
+	/* A single cpu is exiting a low power state */
+	CPU_PM_EXIT,
+
+	/* A cpu power domain is entering a low power state */
+	CPU_CLUSTER_PM_ENTER,
+
+	/* A cpu power domain failed to enter a low power state */
+	CPU_CLUSTER_PM_ENTER_FAILED,
+
+	/* A cpu power domain is exiting a low power state */
+	CPU_CLUSTER_PM_EXIT,
+};
+
+int cpu_pm_register_notifier(struct notifier_block *nb);
+int cpu_pm_unregister_notifier(struct notifier_block *nb);
+
+/*
+ * cpm_pm_enter
+ *
+ * Notifies listeners that a single cpu is entering a low power state that may
+ * cause some blocks in the same power domain as the cpu to reset.
+ *
+ * Must be called on the affected cpu with interrupts disabled.  Platform is
+ * responsible for ensuring that cpu_pm_enter is not called twice on the same
+ * cpu before cpu_pm_exit is called.
+ */
+int cpu_pm_enter(void);
+
+/*
+ * cpm_pm_exit
+ *
+ * Notifies listeners that a single cpu is exiting a low power state that may
+ * have caused some blocks in the same power domain as the cpu to reset.
+ *
+ * Must be called on the affected cpu with interrupts disabled.
+ */
+int cpu_pm_exit(void);
+
+/*
+ * cpm_cluster_pm_enter
+ *
+ * Notifies listeners that all cpus in a power domain are entering a low power
+ * state that may cause some blocks in the same power domain to reset.
+ *
+ * Must be called after cpu_pm_enter has been called on all cpus in the power
+ * domain, and before cpu_pm_exit has been called on any cpu in the power
+ * domain.
+ *
+ * Must be called with interrupts disabled.
+ */
+int cpu_cluster_pm_enter(void);
+
+/*
+ * cpm_pm_enter
+ *
+ * Notifies listeners that a single cpu is entering a low power state that may
+ * cause some blocks in the same power domain as the cpu to reset.
+ *
+ * Must be called after cpu_pm_enter has been called on all cpus in the power
+ * domain, and before cpu_pm_exit has been called on any cpu in the power
+ * domain.
+ *
+ * Must be called with interrupts disabled.
+ */
+int cpu_cluster_pm_exit(void);
+
+#endif
diff --git a/kernel/Makefile b/kernel/Makefile
index eca595e..988cb3d 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -101,6 +101,7 @@ obj-$(CONFIG_RING_BUFFER) += trace/
 obj-$(CONFIG_TRACEPOINTS) += trace/
 obj-$(CONFIG_SMP) += sched_cpupri.o
 obj-$(CONFIG_IRQ_WORK) += irq_work.o
+obj-$(CONFIG_CPU_PM) += cpu_pm.o
 
 obj-$(CONFIG_PERF_EVENTS) += events/
 
diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c
new file mode 100644
index 0000000..54aa892
--- /dev/null
+++ b/kernel/cpu_pm.c
@@ -0,0 +1,115 @@
+/*
+ * Copyright (C) 2011 Google, Inc.
+ *
+ * Author:
+ *	Colin Cross <ccross@android.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/cpu_pm.h>
+#include <linux/module.h>
+#include <linux/notifier.h>
+#include <linux/spinlock.h>
+
+static DEFINE_RWLOCK(cpu_pm_notifier_lock);
+static RAW_NOTIFIER_HEAD(cpu_pm_notifier_chain);
+
+int cpu_pm_register_notifier(struct notifier_block *nb)
+{
+	unsigned long flags;
+	int ret;
+
+	write_lock_irqsave(&cpu_pm_notifier_lock, flags);
+	ret = raw_notifier_chain_register(&cpu_pm_notifier_chain, nb);
+	write_unlock_irqrestore(&cpu_pm_notifier_lock, flags);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_register_notifier);
+
+int cpu_pm_unregister_notifier(struct notifier_block *nb)
+{
+	unsigned long flags;
+	int ret;
+
+	write_lock_irqsave(&cpu_pm_notifier_lock, flags);
+	ret = raw_notifier_chain_unregister(&cpu_pm_notifier_chain, nb);
+	write_unlock_irqrestore(&cpu_pm_notifier_lock, flags);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_unregister_notifier);
+
+static int cpu_pm_notify(enum cpu_pm_event event, int nr_to_call, int *nr_calls)
+{
+	int ret;
+
+	ret = __raw_notifier_call_chain(&cpu_pm_notifier_chain, event, NULL,
+		nr_to_call, nr_calls);
+
+	return notifier_to_errno(ret);
+}
+
+int cpu_pm_enter(void)
+{
+	int nr_calls;
+	int ret = 0;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_PM_ENTER, -1, &nr_calls);
+	if (ret)
+		cpu_pm_notify(CPU_PM_ENTER_FAILED, nr_calls - 1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_enter);
+
+int cpu_pm_exit(void)
+{
+	int ret;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_PM_EXIT, -1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_exit);
+
+int cpu_cluster_pm_enter(void)
+{
+	int nr_calls;
+	int ret = 0;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_CLUSTER_PM_ENTER, -1, &nr_calls);
+	if (ret)
+		cpu_pm_notify(CPU_CLUSTER_PM_ENTER_FAILED, nr_calls - 1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_cluster_pm_enter);
+
+int cpu_cluster_pm_exit(void)
+{
+	int ret;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_CLUSTER_PM_EXIT, -1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_cluster_pm_exit);
diff --git a/kernel/power/Kconfig b/kernel/power/Kconfig
index 3744c59..f7456b4 100644
--- a/kernel/power/Kconfig
+++ b/kernel/power/Kconfig
@@ -235,3 +235,7 @@ config PM_GENERIC_DOMAINS
 config PM_GENERIC_DOMAINS_RUNTIME
 	def_bool y
 	depends on PM_RUNTIME && PM_GENERIC_DOMAINS
+
+config CPU_PM
+	def_bool y
+	depends on SUSPEND || CPU_IDLE
-- 
1.7.4.1

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Santosh Shilimkar]

From: Colin Cross <ccross@android.com>

Implements syscore_ops in cpu_pm to call the cpu and
cpu cluster notifiers during suspend and resume,
allowing drivers receiving the notifications to
avoid implementing syscore_ops.

Signed-off-by: Colin Cross <ccross@android.com>
[santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 kernel/cpu_pm.c |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+), 0 deletions(-)

diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c
index 54aa892..3d115d0 100644
--- a/kernel/cpu_pm.c
+++ b/kernel/cpu_pm.c
@@ -20,6 +20,7 @@
 #include <linux/module.h>
 #include <linux/notifier.h>
 #include <linux/spinlock.h>
+#include <linux/syscore_ops.h>
 
 static DEFINE_RWLOCK(cpu_pm_notifier_lock);
 static RAW_NOTIFIER_HEAD(cpu_pm_notifier_chain);
@@ -113,3 +114,35 @@ int cpu_cluster_pm_exit(void)
 	return ret;
 }
 EXPORT_SYMBOL_GPL(cpu_cluster_pm_exit);
+
+#ifdef CONFIG_PM
+static int cpu_pm_suspend(void)
+{
+	int ret;
+
+	ret = cpu_pm_enter();
+	if (ret)
+		return ret;
+
+	ret = cpu_cluster_pm_enter();
+	return ret;
+}
+
+static void cpu_pm_resume(void)
+{
+	cpu_cluster_pm_exit();
+	cpu_pm_exit();
+}
+
+static struct syscore_ops cpu_pm_syscore_ops = {
+	.suspend = cpu_pm_suspend,
+	.resume = cpu_pm_resume,
+};
+
+static int cpu_pm_init(void)
+{
+	register_syscore_ops(&cpu_pm_syscore_ops);
+	return 0;
+}
+core_initcall(cpu_pm_init);
+#endif
-- 
1.7.4.1

[PATCH v2 3/5] ARM: gic: Use cpu pm notifiers to save gic state

[Santosh Shilimkar]

From: Colin Cross <ccross@android.com>

When the cpu is powered down in a low power mode, the gic cpu
interface may be reset, and when the cpu cluster is powered
down, the gic distributor may also be reset.

This patch uses CPU_PM_ENTER and CPU_PM_EXIT notifiers to save
and restore the gic cpu interface registers, and the
CPU_CLUSTER_PM_ENTER and CPU_CLUSTER_PM_EXIT notifiers to save
and restore the gic distributor registers.

Original-author: Gary King <gking@nvidia.com>
Signed-off-by: Colin Cross <ccross@android.com>
[santosh.shilimkar at ti.com: Rebased against 3.1-rc4, removed gic dist. disable]
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/common/gic.c               |  187 +++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/hardware/gic.h |    8 ++
 2 files changed, 195 insertions(+), 0 deletions(-)

diff --git a/arch/arm/common/gic.c b/arch/arm/common/gic.c
index 3227ca9..66c7c48 100644
--- a/arch/arm/common/gic.c
+++ b/arch/arm/common/gic.c
@@ -26,6 +26,7 @@
 #include <linux/kernel.h>
 #include <linux/list.h>
 #include <linux/smp.h>
+#include <linux/cpu_pm.h>
 #include <linux/cpumask.h>
 #include <linux/io.h>
 
@@ -276,6 +277,8 @@ static void __init gic_dist_init(struct gic_chip_data *gic,
 	if (gic_irqs > 1020)
 		gic_irqs = 1020;
 
+	gic->gic_irqs = gic_irqs;
+
 	/*
 	 * Set all global interrupts to be level triggered, active low.
 	 */
@@ -343,6 +346,189 @@ static void __cpuinit gic_cpu_init(struct gic_chip_data *gic)
 	writel_relaxed(1, base + GIC_CPU_CTRL);
 }
 
+#ifdef CONFIG_CPU_PM
+/*
+ * Saves the GIC distributor registers during suspend or idle.  Must be called
+ * with interrupts disabled but before powering down the GIC.  After calling
+ * this function, no interrupts will be delivered by the GIC, and another
+ * platform-specific wakeup source must be enabled.
+ */
+static void gic_dist_save(unsigned int gic_nr)
+{
+	unsigned int gic_irqs;
+	void __iomem *dist_base;
+	int i;
+
+	if (gic_nr >= MAX_GIC_NR)
+		BUG();
+
+	gic_irqs = gic_data[gic_nr].gic_irqs;
+	dist_base = gic_data[gic_nr].dist_base;
+
+	if (!dist_base)
+		return;
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 16); i++)
+		gic_data[gic_nr].saved_spi_conf[i] =
+			readl_relaxed(dist_base + GIC_DIST_CONFIG + i * 4);
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 4); i++)
+		gic_data[gic_nr].saved_spi_target[i] =
+			readl_relaxed(dist_base + GIC_DIST_TARGET + i * 4);
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 32); i++)
+		gic_data[gic_nr].saved_spi_enable[i] =
+			readl_relaxed(dist_base + GIC_DIST_ENABLE_SET + i * 4);
+}
+
+/*
+ * Restores the GIC distributor registers during resume or when coming out of
+ * idle.  Must be called before enabling interrupts.  If a level interrupt
+ * that occured while the GIC was suspended is still present, it will be
+ * handled normally, but any edge interrupts that occured will not be seen by
+ * the GIC and need to be handled by the platform-specific wakeup source.
+ */
+static void gic_dist_restore(unsigned int gic_nr)
+{
+	unsigned int gic_irqs;
+	unsigned int i;
+	void __iomem *dist_base;
+
+	if (gic_nr >= MAX_GIC_NR)
+		BUG();
+
+	gic_irqs = gic_data[gic_nr].gic_irqs;
+	dist_base = gic_data[gic_nr].dist_base;
+
+	if (!dist_base)
+		return;
+
+	writel_relaxed(0, dist_base + GIC_DIST_CTRL);
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 16); i++)
+		writel_relaxed(gic_data[gic_nr].saved_spi_conf[i],
+			dist_base + GIC_DIST_CONFIG + i * 4);
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 4); i++)
+		writel_relaxed(0xa0a0a0a0,
+			dist_base + GIC_DIST_PRI + i * 4);
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 4); i++)
+		writel_relaxed(gic_data[gic_nr].saved_spi_target[i],
+			dist_base + GIC_DIST_TARGET + i * 4);
+
+	for (i = 0; i < DIV_ROUND_UP(gic_irqs, 32); i++)
+		writel_relaxed(gic_data[gic_nr].saved_spi_enable[i],
+			dist_base + GIC_DIST_ENABLE_SET + i * 4);
+
+	writel_relaxed(1, dist_base + GIC_DIST_CTRL);
+}
+
+static void gic_cpu_save(unsigned int gic_nr)
+{
+	int i;
+	u32 *ptr;
+	void __iomem *dist_base;
+	void __iomem *cpu_base;
+
+	if (gic_nr >= MAX_GIC_NR)
+		BUG();
+
+	dist_base = gic_data[gic_nr].dist_base;
+	cpu_base = gic_data[gic_nr].cpu_base;
+
+	if (!dist_base || !cpu_base)
+		return;
+
+	ptr = __this_cpu_ptr(gic_data[gic_nr].saved_ppi_enable);
+	for (i = 0; i < DIV_ROUND_UP(32, 32); i++)
+		ptr[i] = readl_relaxed(dist_base + GIC_DIST_ENABLE_SET + i * 4);
+
+	ptr = __this_cpu_ptr(gic_data[gic_nr].saved_ppi_conf);
+	for (i = 0; i < DIV_ROUND_UP(32, 16); i++)
+		ptr[i] = readl_relaxed(dist_base + GIC_DIST_CONFIG + i * 4);
+
+}
+
+static void gic_cpu_restore(unsigned int gic_nr)
+{
+	int i;
+	u32 *ptr;
+	void __iomem *dist_base;
+	void __iomem *cpu_base;
+
+	if (gic_nr >= MAX_GIC_NR)
+		BUG();
+
+	dist_base = gic_data[gic_nr].dist_base;
+	cpu_base = gic_data[gic_nr].cpu_base;
+
+	if (!dist_base || !cpu_base)
+		return;
+
+	ptr = __this_cpu_ptr(gic_data[gic_nr].saved_ppi_enable);
+	for (i = 0; i < DIV_ROUND_UP(32, 32); i++)
+		writel_relaxed(ptr[i], dist_base + GIC_DIST_ENABLE_SET + i * 4);
+
+	ptr = __this_cpu_ptr(gic_data[gic_nr].saved_ppi_conf);
+	for (i = 0; i < DIV_ROUND_UP(32, 16); i++)
+		writel_relaxed(ptr[i], dist_base + GIC_DIST_CONFIG + i * 4);
+
+	for (i = 0; i < DIV_ROUND_UP(32, 4); i++)
+		writel_relaxed(0xa0a0a0a0, dist_base + GIC_DIST_PRI + i * 4);
+
+	writel_relaxed(0xf0, cpu_base + GIC_CPU_PRIMASK);
+	writel_relaxed(1, cpu_base + GIC_CPU_CTRL);
+}
+
+static int gic_notifier(struct notifier_block *self, unsigned long cmd,	void *v)
+{
+	int i;
+
+	for (i = 0; i < MAX_GIC_NR; i++) {
+		switch (cmd) {
+		case CPU_PM_ENTER:
+			gic_cpu_save(i);
+			break;
+		case CPU_PM_ENTER_FAILED:
+		case CPU_PM_EXIT:
+			gic_cpu_restore(i);
+			break;
+		case CPU_CLUSTER_PM_ENTER:
+			gic_dist_save(i);
+			break;
+		case CPU_CLUSTER_PM_ENTER_FAILED:
+		case CPU_CLUSTER_PM_EXIT:
+			gic_dist_restore(i);
+			break;
+		}
+	}
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block gic_notifier_block = {
+	.notifier_call = gic_notifier,
+};
+
+static void __init gic_pm_init(struct gic_chip_data *gic)
+{
+	gic->saved_ppi_enable = __alloc_percpu(DIV_ROUND_UP(32, 32) * 4,
+		sizeof(u32));
+	BUG_ON(!gic->saved_ppi_enable);
+
+	gic->saved_ppi_conf = __alloc_percpu(DIV_ROUND_UP(32, 16) * 4,
+		sizeof(u32));
+	BUG_ON(!gic->saved_ppi_conf);
+
+	cpu_pm_register_notifier(&gic_notifier_block);
+}
+#else
+static void __init gic_pm_init(struct gic_chip_data *gic)
+{
+}
+#endif
+
 void __init gic_init(unsigned int gic_nr, unsigned int irq_start,
 	void __iomem *dist_base, void __iomem *cpu_base)
 {
@@ -360,6 +546,7 @@ void __init gic_init(unsigned int gic_nr, unsigned int irq_start,
 
 	gic_dist_init(gic, irq_start);
 	gic_cpu_init(gic);
+	gic_pm_init(gic);
 }
 
 void __cpuinit gic_secondary_init(unsigned int gic_nr)
diff --git a/arch/arm/include/asm/hardware/gic.h b/arch/arm/include/asm/hardware/gic.h
index 435d3f8..c562705 100644
--- a/arch/arm/include/asm/hardware/gic.h
+++ b/arch/arm/include/asm/hardware/gic.h
@@ -46,6 +46,14 @@ struct gic_chip_data {
 	unsigned int irq_offset;
 	void __iomem *dist_base;
 	void __iomem *cpu_base;
+#ifdef CONFIG_CPU_PM
+	u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
+	u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
+	u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
+	u32 __percpu *saved_ppi_enable;
+	u32 __percpu *saved_ppi_conf;
+#endif
+	unsigned int gic_irqs;
 };
 #endif
 
-- 
1.7.4.1

[PATCH v2 4/5] ARM: vfp: Use cpu pm notifiers to save vfp state

[Santosh Shilimkar]

From: Colin Cross <ccross@android.com>

When the cpu is powered down in a low power mode, the vfp
registers may be reset.

This patch uses CPU_PM_ENTER and CPU_PM_EXIT notifiers to save
and restore the cpu's vfp registers.

Signed-off-by: Colin Cross <ccross@android.com>
[santosh.shilimkar at ti.com: Rebased against 3.1-rc4, resolved the conflicts]
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/vfp/vfpmodule.c |   29 +++++++++++++++++++++--------
 1 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c
index 79bcb43..fe4b60b 100644
--- a/arch/arm/vfp/vfpmodule.c
+++ b/arch/arm/vfp/vfpmodule.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/types.h>
 #include <linux/cpu.h>
+#include <linux/cpu_pm.h>
 #include <linux/kernel.h>
 #include <linux/notifier.h>
 #include <linux/signal.h>
@@ -436,9 +437,7 @@ static void vfp_enable(void *unused)
 	set_copro_access(access | CPACC_FULL(10) | CPACC_FULL(11));
 }
 
-#ifdef CONFIG_PM
-#include <linux/syscore_ops.h>
-
+#ifdef CONFIG_CPU_PM
 static int vfp_pm_suspend(void)
 {
 	struct thread_info *ti = current_thread_info();
@@ -468,19 +467,33 @@ static void vfp_pm_resume(void)
 	fmxr(FPEXC, fmrx(FPEXC) & ~FPEXC_EN);
 }
 
-static struct syscore_ops vfp_pm_syscore_ops = {
-	.suspend	= vfp_pm_suspend,
-	.resume		= vfp_pm_resume,
+static int vfp_cpu_pm_notifier(struct notifier_block *self, unsigned long cmd,
+	void *v)
+{
+	switch (cmd) {
+	case CPU_PM_ENTER:
+		vfp_pm_suspend();
+		break;
+	case CPU_PM_ENTER_FAILED:
+	case CPU_PM_EXIT:
+		vfp_pm_resume();
+		break;
+	}
+	return NOTIFY_OK;
+}
+
+static struct notifier_block vfp_cpu_pm_notifier_block = {
+	.notifier_call = vfp_cpu_pm_notifier,
 };
 
 static void vfp_pm_init(void)
 {
-	register_syscore_ops(&vfp_pm_syscore_ops);
+	cpu_pm_register_notifier(&vfp_cpu_pm_notifier_block);
 }
 
 #else
 static inline void vfp_pm_init(void) { }
-#endif /* CONFIG_PM */
+#endif /* CONFIG_CPU_PM */
 
 /*
  * Ensure that the VFP state stored in 'thread->vfpstate' is up to date
-- 
1.7.4.1

[PATCH v2 5/5] ARM: gic: Allow gic arch extensions to provide irqchip flags

[Santosh Shilimkar]

From: Colin Cross <ccross@android.com>

Tegra can benefit from the IRQCHIP_MASK_ON_SUSPEND flag, allow it
to be passed to the gic irq chip.

Signed-off-by: Colin Cross <ccross@android.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 arch/arm/common/gic.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/arm/common/gic.c b/arch/arm/common/gic.c
index 66c7c48..734db99 100644
--- a/arch/arm/common/gic.c
+++ b/arch/arm/common/gic.c
@@ -544,6 +544,7 @@ void __init gic_init(unsigned int gic_nr, unsigned int irq_start,
 	if (gic_nr == 0)
 		gic_cpu_base_addr = cpu_base;
 
+	gic_chip.flags |= gic_arch_extn.flags;
 	gic_dist_init(gic, irq_start);
 	gic_cpu_init(gic);
 	gic_pm_init(gic);
-- 
1.7.4.1

[PATCH v2 0/5] CPU PM notifiers

[Shawn Guo]

On Sat, Sep 03, 2011 at 08:09:10PM +0530, Santosh Shilimkar wrote:
> Updates V2:
> - The CPU PM notifiers are moved to generic level instead of keeping
>   it ARM specific as per Rafael's comment.
> - To avoid every driver duplicating the sys-core ops and cpu pm notfiers,
>   the CPU PM common notifier is registering it in one place so that
>   suspend is taken care.
> - CPU COMPLEX events are renamed to more appropriate CPU CLUSTER.
> - GIC distributor is not disabled to avoid the live locks. This is
>   observed with CPUIDLE cases where at time CPU cluster fails to
>   transition to low power states. As such there was no need to
>   disable distributor in first place since on cluster reset, it
>   will get disabled anyways.
> - In VFP code, syscore ops are dropped in favour of CPU PM
>   notifiers since S2R is already handled in core CPU PM notifiers.
> - The last patch is an independent patch but can be pushed along with
>   the other GIC changes done here. 
>  
> V1: [http://lwn.net/Articles/447259/]
> 
> This patch set tries to address concerns with platform pm code
> calling into the driver for every block in the Cortex A9s
> during idle, hotplug, and suspend.  The first patch adds cpu pm
> notifiers that can be called by platform code, the second uses
> the notifier to save and restore the GIC state, and the third
> saves the VFP state.
> 
> The notifiers are used for two types of events, CPU PM events and
> CPU cluster PM events.  CPU PM events are used to save and restore
> per-cpu context when a single CPU is preparing to enter or has
> just exited a low power state.  For example, the VFP saves the
> last thread context, and the GIC saves banked CPU registers.
> 
> CPU cluster events are used after all the CPUs in a power domain
> have been prepared for the low power state.  The GIC uses these
> events to save global register state.
> 
> L2 cache is not covered by this patch set, as the determination
> of when the L2 is reset and when it is retained is
> platform-specific, and most of the APIs necessary are already
> 
> 
> The series is tested with OMAP4 with S2R and CPUIDLE. 
> 
> The following changes since commit c6a389f123b9f68d605bb7e0f9b32ec1e3e14132:
> 
>   Linux 3.1-rc4 (2011-08-28 21:16:01 -0700)
> 
> Colin Cross (5):
>       cpu_pm: Add cpu power management notifiers
>       cpu_pm: call notifiers during suspend
>       ARM: gic: Use cpu pm notifiers to save gic state
>       ARM: vfp: Use cpu pm notifiers to save vfp state
>       ARM: gic: Allow gic arch extensions to provide irqchip flags
> 
Really appreciate the patches.  Now platform code needs to do nothing
to have GIC back to work across suspend/resume cycle.

On i.MX6Q:

Tested-and-Acked-by: Shawn Guo <shawn.guo@linaro.org>

-- 
Regards,
Shawn

[PATCH v2 0/5] CPU PM notifiers

[Santosh]

On Tuesday 06 September 2011 08:04 AM, Shawn Guo wrote:
> On Sat, Sep 03, 2011 at 08:09:10PM +0530, Santosh Shilimkar wrote:
>> Updates V2:
>> - The CPU PM notifiers are moved to generic level instead of keeping
>>    it ARM specific as per Rafael's comment.
>> - To avoid every driver duplicating the sys-core ops and cpu pm notfiers,
>>    the CPU PM common notifier is registering it in one place so that
>>    suspend is taken care.
>> - CPU COMPLEX events are renamed to more appropriate CPU CLUSTER.
>> - GIC distributor is not disabled to avoid the live locks. This is
>>    observed with CPUIDLE cases where at time CPU cluster fails to
>>    transition to low power states. As such there was no need to
>>    disable distributor in first place since on cluster reset, it
>>    will get disabled anyways.
>> - In VFP code, syscore ops are dropped in favour of CPU PM
>>    notifiers since S2R is already handled in core CPU PM notifiers.
>> - The last patch is an independent patch but can be pushed along with
>>    the other GIC changes done here.
>>
>> V1: [http://lwn.net/Articles/447259/]
>>
>> This patch set tries to address concerns with platform pm code
>> calling into the driver for every block in the Cortex A9s
>> during idle, hotplug, and suspend.  The first patch adds cpu pm
>> notifiers that can be called by platform code, the second uses
>> the notifier to save and restore the GIC state, and the third
>> saves the VFP state.
>>
>> The notifiers are used for two types of events, CPU PM events and
>> CPU cluster PM events.  CPU PM events are used to save and restore
>> per-cpu context when a single CPU is preparing to enter or has
>> just exited a low power state.  For example, the VFP saves the
>> last thread context, and the GIC saves banked CPU registers.
>>
>> CPU cluster events are used after all the CPUs in a power domain
>> have been prepared for the low power state.  The GIC uses these
>> events to save global register state.
>>
>> L2 cache is not covered by this patch set, as the determination
>> of when the L2 is reset and when it is retained is
>> platform-specific, and most of the APIs necessary are already
>>
>>
>> The series is tested with OMAP4 with S2R and CPUIDLE.
>>
>> The following changes since commit c6a389f123b9f68d605bb7e0f9b32ec1e3e14132:
>>
>>    Linux 3.1-rc4 (2011-08-28 21:16:01 -0700)
>>
>> Colin Cross (5):
>>        cpu_pm: Add cpu power management notifiers
>>        cpu_pm: call notifiers during suspend
>>        ARM: gic: Use cpu pm notifiers to save gic state
>>        ARM: vfp: Use cpu pm notifiers to save vfp state
>>        ARM: gic: Allow gic arch extensions to provide irqchip flags
>>
> Really appreciate the patches.  Now platform code needs to do nothing
> to have GIC back to work across suspend/resume cycle.
>
Yep.

> On i.MX6Q:
>
> Tested-and-Acked-by: Shawn Guo<shawn.guo@linaro.org>
>
Thanks for testing.

Regards
Santosh

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Kevin Hilman]

Santosh Shilimkar <santosh.shilimkar@ti.com> writes:

> From: Colin Cross <ccross@android.com>
>
> Implements syscore_ops in cpu_pm to call the cpu and
> cpu cluster notifiers during suspend and resume,
> allowing drivers receiving the notifications to
> avoid implementing syscore_ops.
>
> Signed-off-by: Colin Cross <ccross@android.com>
> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>

I don't think using syscore_ops is right here.  The platform code should
decide where in its own suspend path the notifiers should be triggered.

The reason is because while the syscore_ops run late in the suspend
path, they still run before some platform-specific decisions about the
low-power states are made.  That means that any notifiers that need to
use information about the target low-power state (e.g. whether context
will be lost or not) cannot do so since that information has not yet
been decided until the platform_suspend_ops->enter() runs.

Basically, I think the cpu_*_pm_enter() calls should be called by
platform-specific code, not by common code.

Kevin

> ---
>  kernel/cpu_pm.c |   33 +++++++++++++++++++++++++++++++++
>  1 files changed, 33 insertions(+), 0 deletions(-)
>
> diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c
> index 54aa892..3d115d0 100644
> --- a/kernel/cpu_pm.c
> +++ b/kernel/cpu_pm.c
> @@ -20,6 +20,7 @@
>  #include <linux/module.h>
>  #include <linux/notifier.h>
>  #include <linux/spinlock.h>
> +#include <linux/syscore_ops.h>
>  
>  static DEFINE_RWLOCK(cpu_pm_notifier_lock);
>  static RAW_NOTIFIER_HEAD(cpu_pm_notifier_chain);
> @@ -113,3 +114,35 @@ int cpu_cluster_pm_exit(void)
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(cpu_cluster_pm_exit);
> +
> +#ifdef CONFIG_PM
> +static int cpu_pm_suspend(void)
> +{
> +	int ret;
> +
> +	ret = cpu_pm_enter();
> +	if (ret)
> +		return ret;
> +
> +	ret = cpu_cluster_pm_enter();
> +	return ret;
> +}
> +
> +static void cpu_pm_resume(void)
> +{
> +	cpu_cluster_pm_exit();
> +	cpu_pm_exit();
> +}
> +
> +static struct syscore_ops cpu_pm_syscore_ops = {
> +	.suspend = cpu_pm_suspend,
> +	.resume = cpu_pm_resume,
> +};
> +
> +static int cpu_pm_init(void)
> +{
> +	register_syscore_ops(&cpu_pm_syscore_ops);
> +	return 0;
> +}
> +core_initcall(cpu_pm_init);
> +#endif

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Santosh]

On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
> Santosh Shilimkar<santosh.shilimkar@ti.com>  writes:
>
>> From: Colin Cross<ccross@android.com>
>>
>> Implements syscore_ops in cpu_pm to call the cpu and
>> cpu cluster notifiers during suspend and resume,
>> allowing drivers receiving the notifications to
>> avoid implementing syscore_ops.
>>
>> Signed-off-by: Colin Cross<ccross@android.com>
>> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
>> Signed-off-by: Santosh Shilimkar<santosh.shilimkar@ti.com>
>
> I don't think using syscore_ops is right here.  The platform code should
> decide where in its own suspend path the notifiers should be triggered.
>
> The reason is because while the syscore_ops run late in the suspend
> path, they still run before some platform-specific decisions about the
> low-power states are made.  That means that any notifiers that need to
> use information about the target low-power state (e.g. whether context
> will be lost or not) cannot do so since that information has not yet
> been decided until the platform_suspend_ops->enter() runs.
>
Initially I thought the same but in general S2R, platform doesn't
support multiple states like CPUIDLE. On OMAP, we do have a debug
option to choose the state but on real product, it's always the
deepest supported state is used. So the driver saving the
full context for S2R, should be fine.

Ofcourse for CPUIDLE, the notifier call chain decisions are left
with platform CPUIDLE drivers since there can be multiple low
power states and the context save/restore has to be done based
on low power states.

The advantage with this is, the platform code is clean from the
notfiers calls. CPUIDLE driver needs to call the different notifier
events based on C-states and that perfectly works.

I liked this simplification for the S2R. Down side is in S2R if you
don't plan to hit deepest state, drivers end up saving full context
which is fine I guess.

> Basically, I think the cpu_*_pm_enter() calls should be called by
> platform-specific code, not by common code.
>
Sure and that's the case for CPUIDLE already. The change was
done only for S2R based on above points.

Regards
Santosh

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Kevin Hilman]

Santosh <santosh.shilimkar@ti.com> writes:

> On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
>> Santosh Shilimkar<santosh.shilimkar@ti.com>  writes:
>>
>>> From: Colin Cross<ccross@android.com>
>>>
>>> Implements syscore_ops in cpu_pm to call the cpu and
>>> cpu cluster notifiers during suspend and resume,
>>> allowing drivers receiving the notifications to
>>> avoid implementing syscore_ops.
>>>
>>> Signed-off-by: Colin Cross<ccross@android.com>
>>> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
>>> Signed-off-by: Santosh Shilimkar<santosh.shilimkar@ti.com>
>>
>> I don't think using syscore_ops is right here.  The platform code should
>> decide where in its own suspend path the notifiers should be triggered.
>>
>> The reason is because while the syscore_ops run late in the suspend
>> path, they still run before some platform-specific decisions about the
>> low-power states are made.  That means that any notifiers that need to
>> use information about the target low-power state (e.g. whether context
>> will be lost or not) cannot do so since that information has not yet
>> been decided until the platform_suspend_ops->enter() runs.
>>
> Initially I thought the same but in general S2R, platform doesn't
> support multiple states like CPUIDLE. On OMAP, we do have a debug
> option to choose the state but on real product, it's always the
> deepest supported state is used. So the driver saving the
> full context for S2R, should be fine.
>
> Ofcourse for CPUIDLE, the notifier call chain decisions are left
> with platform CPUIDLE drivers since there can be multiple low
> power states and the context save/restore has to be done based
> on low power states.
>
> The advantage with this is, the platform code is clean from the
> notfiers calls. CPUIDLE driver needs to call the different notifier
> events based on C-states and that perfectly works.
>
> I liked this simplification for the S2R. Down side is in S2R if you
> don't plan to hit deepest state, drivers end up saving full context
> which is fine I guess.

That's not the downside I'm worried about.

If you have a driver that has a notifier, presumably it has something it
wants to do to prepare for suspend *and* for idle, and you'd only want a
single notifier callback in the driver to be used for both.  That
callback would look something like:

   start_preparing_for_suspend();

   if (next_state == OFF)
      save_context();

   finish_preparing_for_suspend();


The problem with the current cpu_*_pm_enter() calls in syscore_ops is
that they happen before the next states are programmed, so during
suspend the 'if (next_state == off)' above would never be true, but
during idle it might be.

Kevin

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Santosh]

On Thursday 08 September 2011 07:31 PM, Kevin Hilman wrote:
> Santosh<santosh.shilimkar@ti.com>  writes:
>
>> On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
>>> Santosh Shilimkar<santosh.shilimkar@ti.com>   writes:
>>>
>>>> From: Colin Cross<ccross@android.com>
>>>>
>>>> Implements syscore_ops in cpu_pm to call the cpu and
>>>> cpu cluster notifiers during suspend and resume,
>>>> allowing drivers receiving the notifications to
>>>> avoid implementing syscore_ops.
>>>>
>>>> Signed-off-by: Colin Cross<ccross@android.com>
>>>> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
>>>> Signed-off-by: Santosh Shilimkar<santosh.shilimkar@ti.com>
>>>
>>> I don't think using syscore_ops is right here.  The platform code should
>>> decide where in its own suspend path the notifiers should be triggered.
>>>
>>> The reason is because while the syscore_ops run late in the suspend
>>> path, they still run before some platform-specific decisions about the
>>> low-power states are made.  That means that any notifiers that need to
>>> use information about the target low-power state (e.g. whether context
>>> will be lost or not) cannot do so since that information has not yet
>>> been decided until the platform_suspend_ops->enter() runs.
>>>
>> Initially I thought the same but in general S2R, platform doesn't
>> support multiple states like CPUIDLE. On OMAP, we do have a debug
>> option to choose the state but on real product, it's always the
>> deepest supported state is used. So the driver saving the
>> full context for S2R, should be fine.
>>
>> Ofcourse for CPUIDLE, the notifier call chain decisions are left
>> with platform CPUIDLE drivers since there can be multiple low
>> power states and the context save/restore has to be done based
>> on low power states.
>>
>> The advantage with this is, the platform code is clean from the
>> notfiers calls. CPUIDLE driver needs to call the different notifier
>> events based on C-states and that perfectly works.
>>
>> I liked this simplification for the S2R. Down side is in S2R if you
>> don't plan to hit deepest state, drivers end up saving full context
>> which is fine I guess.
>
> That's not the downside I'm worried about.
>
> If you have a driver that has a notifier, presumably it has something it
> wants to do to prepare for suspend *and* for idle, and you'd only want a
> single notifier callback in the driver to be used for both.  That
> callback would look something like:
>
>     start_preparing_for_suspend();
>
>     if (next_state == OFF)
>        save_context();
>
>     finish_preparing_for_suspend();
>
>
> The problem with the current cpu_*_pm_enter() calls in syscore_ops is
> that they happen before the next states are programmed, so during
> suspend the 'if (next_state == off)' above would never be true, but
> during idle it might be.
>
Point taken and now I agree with you.

I am going to drop this patch unless and until somebody shouts at me.
What that means is platform code need to take care of suspend case
as well which is trivial change.

Thanks for bringing up the point.

Regards
Santosh

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Kevin Hilman]

Santosh <santosh.shilimkar@ti.com> writes:

> On Thursday 08 September 2011 07:31 PM, Kevin Hilman wrote:
>> Santosh<santosh.shilimkar@ti.com>  writes:
>>
>>> On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
>>>> Santosh Shilimkar<santosh.shilimkar@ti.com>   writes:
>>>>
>>>>> From: Colin Cross<ccross@android.com>
>>>>>
>>>>> Implements syscore_ops in cpu_pm to call the cpu and
>>>>> cpu cluster notifiers during suspend and resume,
>>>>> allowing drivers receiving the notifications to
>>>>> avoid implementing syscore_ops.
>>>>>
>>>>> Signed-off-by: Colin Cross<ccross@android.com>
>>>>> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
>>>>> Signed-off-by: Santosh Shilimkar<santosh.shilimkar@ti.com>
>>>>
>>>> I don't think using syscore_ops is right here.  The platform code should
>>>> decide where in its own suspend path the notifiers should be triggered.
>>>>
>>>> The reason is because while the syscore_ops run late in the suspend
>>>> path, they still run before some platform-specific decisions about the
>>>> low-power states are made.  That means that any notifiers that need to
>>>> use information about the target low-power state (e.g. whether context
>>>> will be lost or not) cannot do so since that information has not yet
>>>> been decided until the platform_suspend_ops->enter() runs.
>>>>
>>> Initially I thought the same but in general S2R, platform doesn't
>>> support multiple states like CPUIDLE. On OMAP, we do have a debug
>>> option to choose the state but on real product, it's always the
>>> deepest supported state is used. So the driver saving the
>>> full context for S2R, should be fine.
>>>
>>> Ofcourse for CPUIDLE, the notifier call chain decisions are left
>>> with platform CPUIDLE drivers since there can be multiple low
>>> power states and the context save/restore has to be done based
>>> on low power states.
>>>
>>> The advantage with this is, the platform code is clean from the
>>> notfiers calls. CPUIDLE driver needs to call the different notifier
>>> events based on C-states and that perfectly works.
>>>
>>> I liked this simplification for the S2R. Down side is in S2R if you
>>> don't plan to hit deepest state, drivers end up saving full context
>>> which is fine I guess.
>>
>> That's not the downside I'm worried about.
>>
>> If you have a driver that has a notifier, presumably it has something it
>> wants to do to prepare for suspend *and* for idle, and you'd only want a
>> single notifier callback in the driver to be used for both.  That
>> callback would look something like:
>>
>>     start_preparing_for_suspend();
>>
>>     if (next_state == OFF)
>>        save_context();
>>
>>     finish_preparing_for_suspend();
>>
>>
>> The problem with the current cpu_*_pm_enter() calls in syscore_ops is
>> that they happen before the next states are programmed, so during
>> suspend the 'if (next_state == off)' above would never be true, but
>> during idle it might be.
>>
> Point taken and now I agree with you.
>
> I am going to drop this patch unless and until somebody shouts at me.
> What that means is platform code need to take care of suspend case
> as well which is trivial change.

Agreed.

Thanks,

Kevin

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Colin Cross]

On Thu, Sep 8, 2011 at 7:01 AM, Kevin Hilman <khilman@ti.com> wrote:
> Santosh <santosh.shilimkar@ti.com> writes:
>
>> On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
>>> Santosh Shilimkar<santosh.shilimkar@ti.com> ?writes:
>>>
>>>> From: Colin Cross<ccross@android.com>
>>>>
>>>> Implements syscore_ops in cpu_pm to call the cpu and
>>>> cpu cluster notifiers during suspend and resume,
>>>> allowing drivers receiving the notifications to
>>>> avoid implementing syscore_ops.
>>>>
>>>> Signed-off-by: Colin Cross<ccross@android.com>
>>>> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
>>>> Signed-off-by: Santosh Shilimkar<santosh.shilimkar@ti.com>
>>>
>>> I don't think using syscore_ops is right here. ?The platform code should
>>> decide where in its own suspend path the notifiers should be triggered.
>>>
>>> The reason is because while the syscore_ops run late in the suspend
>>> path, they still run before some platform-specific decisions about the
>>> low-power states are made. ?That means that any notifiers that need to
>>> use information about the target low-power state (e.g. whether context
>>> will be lost or not) cannot do so since that information has not yet
>>> been decided until the platform_suspend_ops->enter() runs.
>>>
>> Initially I thought the same but in general S2R, platform doesn't
>> support multiple states like CPUIDLE. On OMAP, we do have a debug
>> option to choose the state but on real product, it's always the
>> deepest supported state is used. So the driver saving the
>> full context for S2R, should be fine.
>>
>> Ofcourse for CPUIDLE, the notifier call chain decisions are left
>> with platform CPUIDLE drivers since there can be multiple low
>> power states and the context save/restore has to be done based
>> on low power states.
>>
>> The advantage with this is, the platform code is clean from the
>> notfiers calls. CPUIDLE driver needs to call the different notifier
>> events based on C-states and that perfectly works.
>>
>> I liked this simplification for the S2R. Down side is in S2R if you
>> don't plan to hit deepest state, drivers end up saving full context
>> which is fine I guess.
>
> That's not the downside I'm worried about.
>
> If you have a driver that has a notifier, presumably it has something it
> wants to do to prepare for suspend *and* for idle, and you'd only want a
> single notifier callback in the driver to be used for both. ?That
> callback would look something like:
>
> ? start_preparing_for_suspend();
>
> ? if (next_state == OFF)
> ? ? ?save_context();
>
> ? finish_preparing_for_suspend();
>
>
> The problem with the current cpu_*_pm_enter() calls in syscore_ops is
> that they happen before the next states are programmed, so during
> suspend the 'if (next_state == off)' above would never be true, but
> during idle it might be.

These notifiers are designed for drivers that are tightly coupled to
the cpu, and shared across multiple architectures (mostly GIC and
VFP).  In practice, all of these devices are off in every suspend
state, because nobody leaves the CPU on in suspend.

The (next_state == OFF) api you refer to would have to be something
architecture specific, since the power state handling is very
different on every platform, so it's not something that would ever be
included in drivers that I imagined would be using these notifiers.

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Kevin Hilman]

Colin Cross <ccross@android.com> writes:

> On Thu, Sep 8, 2011 at 7:01 AM, Kevin Hilman <khilman@ti.com> wrote:
>> Santosh <santosh.shilimkar@ti.com> writes:
>>
>>> On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
>>>> Santosh Shilimkar<santosh.shilimkar@ti.com> ?writes:
>>>>
>>>>> From: Colin Cross<ccross@android.com>
>>>>>
>>>>> Implements syscore_ops in cpu_pm to call the cpu and
>>>>> cpu cluster notifiers during suspend and resume,
>>>>> allowing drivers receiving the notifications to
>>>>> avoid implementing syscore_ops.
>>>>>
>>>>> Signed-off-by: Colin Cross<ccross@android.com>
>>>>> [santosh.shilimkar at ti.com: Rebased against 3.1-rc4]
>>>>> Signed-off-by: Santosh Shilimkar<santosh.shilimkar@ti.com>
>>>>
>>>> I don't think using syscore_ops is right here. ?The platform code should
>>>> decide where in its own suspend path the notifiers should be triggered.
>>>>
>>>> The reason is because while the syscore_ops run late in the suspend
>>>> path, they still run before some platform-specific decisions about the
>>>> low-power states are made. ?That means that any notifiers that need to
>>>> use information about the target low-power state (e.g. whether context
>>>> will be lost or not) cannot do so since that information has not yet
>>>> been decided until the platform_suspend_ops->enter() runs.
>>>>
>>> Initially I thought the same but in general S2R, platform doesn't
>>> support multiple states like CPUIDLE. On OMAP, we do have a debug
>>> option to choose the state but on real product, it's always the
>>> deepest supported state is used. So the driver saving the
>>> full context for S2R, should be fine.
>>>
>>> Ofcourse for CPUIDLE, the notifier call chain decisions are left
>>> with platform CPUIDLE drivers since there can be multiple low
>>> power states and the context save/restore has to be done based
>>> on low power states.
>>>
>>> The advantage with this is, the platform code is clean from the
>>> notfiers calls. CPUIDLE driver needs to call the different notifier
>>> events based on C-states and that perfectly works.
>>>
>>> I liked this simplification for the S2R. Down side is in S2R if you
>>> don't plan to hit deepest state, drivers end up saving full context
>>> which is fine I guess.
>>
>> That's not the downside I'm worried about.
>>
>> If you have a driver that has a notifier, presumably it has something it
>> wants to do to prepare for suspend *and* for idle, and you'd only want a
>> single notifier callback in the driver to be used for both. ?That
>> callback would look something like:
>>
>> ? start_preparing_for_suspend();
>>
>> ? if (next_state == OFF)
>> ? ? ?save_context();
>>
>> ? finish_preparing_for_suspend();
>>
>>
>> The problem with the current cpu_*_pm_enter() calls in syscore_ops is
>> that they happen before the next states are programmed, so during
>> suspend the 'if (next_state == off)' above would never be true, but
>> during idle it might be.
>
> These notifiers are designed for drivers that are tightly coupled to
> the cpu, and shared across multiple architectures (mostly GIC and
> VFP).  

That is certainly the initial intended usage, and I understand that
design, but they are useful for much more.

Specifically, consider devices whose power transitions need to be
tightly coupled with the CPU, but are in different power domains.
Notifiers for these devices may need to be coordinated with
platform-specific events.

Also, it's not only about context save for off-mode.  Some of these
tightly-coupled devices have other work to do besides context
save/restore, and CPU PM notifiers are useful there.  A dumb example off
the top of my head: pins (e.g. GPIOs), that need to be mux'd into safe
mode to avoid glitches when coming back from off.  (admittedly, this is
broken HW, but we all know broken HW is part of life.)

> In practice, all of these devices are off in every suspend
> state, because nobody leaves the CPU on in suspend.

Sure, but you might leave other power domains on (or in retention)
during suspend, and these domains might contain some of the devices
whose power transitions are coupled with CPU transitions and thus using
CPU PM notifiers.

Also, so far we've only talked about suspend, but the CPU (and other
power domains) might also go off during idle.  The approach in $SUBJECT
patch addresses suspend but not idle, which means it's up to
platform-specific code to trigger the notifiers for idle.  I think it
should be the same for suspend.

> The (next_state == OFF) api you refer to would have to be something
> architecture specific, since the power state handling is very
> different on every platform, so it's not something that would ever be
> included in drivers that I imagined would be using these notifiers.

Sure, but you created something so useful that it can be used in other
areas than you initially imagined. :)   Thanks!

I wouldn't imagine arch-specific being used in those generic drivers
either, but in addition to the drivers you imagined, I'm already trying
to the notifiers in drivers that are platform-specific.  I only imagine
using the "next state" type of checking in platform-specific code, not
in generic ones.

Note however that I'm certainly not arguing that the notifiers should
not be called in suspend.  I'm only arguing that it should be up to
platform-specific code when to call them because of possible
platform-specific pre-requisites in platform-specific notifier
callbacks.

IMO, there are 2 options.

1) leave it up to platform-specific code when to trigger the notifiers
   for *both* suspend and idle PM transitions

2) trigger the notifiers in arch-independent code for both suspend and
   idle *but* provide a way that platform-specific code can disable
   them in favor of using platform-specific trigger points.

If most platforms really don't care, then maybe (2) would be the
better approach.  That's fine with me as long as there's a way to
disable them so platform-specific ones can be used.

Kevin

[PATCH v2 2/5] cpu_pm: call notifiers during suspend

[Santosh]

On Friday 09 September 2011 02:21 AM, Kevin Hilman wrote:
> Colin Cross<ccross@android.com>  writes:
>
>> On Thu, Sep 8, 2011 at 7:01 AM, Kevin Hilman<khilman@ti.com>  wrote:
>>> Santosh<santosh.shilimkar@ti.com>  writes:
>>>
>>>> On Thursday 08 September 2011 01:32 AM, Kevin Hilman wrote:
>>>>> Santosh Shilimkar<santosh.shilimkar@ti.com>    writes:
>>>>>
>>>>>> From: Colin Cross<ccross@android.com>
[...]

>>
>> These notifiers are designed for drivers that are tightly coupled to
>> the cpu, and shared across multiple architectures (mostly GIC and
>> VFP).
>
> That is certainly the initial intended usage, and I understand that
> design, but they are useful for much more.
>
> Specifically, consider devices whose power transitions need to be
> tightly coupled with the CPU, but are in different power domains.
> Notifiers for these devices may need to be coordinated with
> platform-specific events.
>
> Also, it's not only about context save for off-mode.  Some of these
> tightly-coupled devices have other work to do besides context
> save/restore, and CPU PM notifiers are useful there.  A dumb example off
> the top of my head: pins (e.g. GPIOs), that need to be mux'd into safe
> mode to avoid glitches when coming back from off.  (admittedly, this is
> broken HW, but we all know broken HW is part of life.)
>
>> In practice, all of these devices are off in every suspend
>> state, because nobody leaves the CPU on in suspend.
>
> Sure, but you might leave other power domains on (or in retention)
> during suspend, and these domains might contain some of the devices
> whose power transitions are coupled with CPU transitions and thus using
> CPU PM notifiers.
>
> Also, so far we've only talked about suspend, but the CPU (and other
> power domains) might also go off during idle.  The approach in $SUBJECT
> patch addresses suspend but not idle, which means it's up to
> platform-specific code to trigger the notifiers for idle.  I think it
> should be the same for suspend.
>
>> The (next_state == OFF) api you refer to would have to be something
>> architecture specific, since the power state handling is very
>> different on every platform, so it's not something that would ever be
>> included in drivers that I imagined would be using these notifiers.
>
> Sure, but you created something so useful that it can be used in other
> areas than you initially imagined. :)   Thanks!
>
> I wouldn't imagine arch-specific being used in those generic drivers
> either, but in addition to the drivers you imagined, I'm already trying
> to the notifiers in drivers that are platform-specific.  I only imagine
> using the "next state" type of checking in platform-specific code, not
> in generic ones.
>
> Note however that I'm certainly not arguing that the notifiers should
> not be called in suspend.  I'm only arguing that it should be up to
> platform-specific code when to call them because of possible
> platform-specific pre-requisites in platform-specific notifier
> callbacks.
>
> IMO, there are 2 options.
>
> 1) leave it up to platform-specific code when to trigger the notifiers
>     for *both* suspend and idle PM transitions
>
> 2) trigger the notifiers in arch-independent code for both suspend and
>     idle *but* provide a way that platform-specific code can disable
>     them in favor of using platform-specific trigger points.
>
> If most platforms really don't care, then maybe (2) would be the
> better approach.  That's fine with me as long as there's a way to
> disable them so platform-specific ones can be used.
>
I think, we should keep the notifiers simple and option 1 the one
thing we should consider. It's just easy to take care of IDLE and
suspend together.

Regards
Santosh

[PATCH v2 0/5] CPU PM notifiers

[Kevin Hilman]

Santosh Shilimkar <santosh.shilimkar@ti.com> writes:

[...]

> This patch set tries to address concerns with platform pm code
> calling into the driver for every block in the Cortex A9s
> during idle, hotplug, and suspend.  The first patch adds cpu pm
> notifiers that can be called by platform code, the second uses
> the notifier to save and restore the GIC state, and the third
> saves the VFP state.
>
> The notifiers are used for two types of events, CPU PM events and
> CPU cluster PM events.  CPU PM events are used to save and restore
> per-cpu context when a single CPU is preparing to enter or has
> just exited a low power state.  For example, the VFP saves the
> last thread context, and the GIC saves banked CPU registers.
>
> CPU cluster events are used after all the CPUs in a power domain
> have been prepared for the low power state.  The GIC uses these
> events to save global register state.

Stepping back from my earlier objections, I think I had a fundamental
misunderstanding about what these notifiers should be used for.

The current assumptions/goals seem to be

1) used only for devices in the same power domain as the CPU (cluster)
2) use only for one specific power state of the CPU (cluster): off.

For awhile now, we've been discussing how to better coordinate CPU PM
transitions (CPUidle) with non-CPU PM transitions (runtime PM) for
devices that are tightly coupled to the CPU, but not necessarily in the
same powerdomain.

I was assuming (and hoping) that CPU PM notifiers could be used to do
that, but the more I think about it, I don't think we can achieve the
current CPU PM goals and the coordination with runtime PM with this
series.

I think it's more likely that we'll need to do some work with Rafael's
new PM domains to make that work correctly.

So, I'll retract my objections to this series, and feel free to add

Reviewed-by: Kevin Hilman <khilman@ti.com>

Kevin

[PATCH v2 1/5] cpu_pm: Add cpu power management notifiers

[Andrew Morton]

On Sat, 3 Sep 2011 20:09:11 +0530
Santosh Shilimkar <santosh.shilimkar@ti.com> wrote:

> From: Colin Cross <ccross@android.com>
> 
> During some CPU power modes entered during idle, hotplug and
> suspend, peripherals located in the CPU power domain, such as
> the GIC, localtimers, and VFP, may be powered down.  Add a
> notifier chain that allows drivers for those peripherals to
> be notified before and after they may be reset.

Have you identified which indivudual you hope/expect to merge this into
mainline?

The code is presumably and hopefully applicable to architectures other
than ARM, yes?  Can you suggest likely candidate architectures so we
can go off and bug the relevant maintainers to review it?

>
> ...
>
> +/*
> + * When a CPU goes to a low power state that turns off power to the CPU's
> + * power domain, the contents of some blocks (floating point coprocessors,
> + * interrutp controllers, caches, timers) in the same power domain can

s/interrutp/interrupt/

> + * be lost.  The cpm_pm notifiers provide a method for platform idle, suspend,
> + * and hotplug implementations to notify the drivers for these blocks that
> + * they may be reset.
> + *
> + * All cpu_pm notifications must be called with interrupts disabled.
> + *
> + * The notifications are split into two classes, CPU notifications and CPU

s/,/:/

> + * cluster notifications.
> + *
> + * CPU notifications apply to a single CPU, and must be called on the affected

s/,// ;)

> + * CPU.  They are used to save per-cpu context for affected blocks.
> + *
> + * CPU cluster notifications apply to all CPUs in a single power domain. They
> + * are used to save any global context for affected blocks, and must be called
> + * after all the CPUs in the power domain have been notified of the low power
> + * state.
> + *

Remove this line.

> + */
> +
>
> ...
>
> +/*
> + * cpm_pm_enter
> + *
> + * Notifies listeners that a single cpu is entering a low power state that may
> + * cause some blocks in the same power domain as the cpu to reset.
> + *
> + * Must be called on the affected cpu with interrupts disabled.  Platform is
> + * responsible for ensuring that cpu_pm_enter is not called twice on the same
> + * cpu before cpu_pm_exit is called.
> + */

It's unconventional to put the documentation over the declarations in the
.h file.  It's not a *bad* idea per-se, but we generally don't do it. 
People will look at the definition in .c for the documentation and it
if isn't there, some will assume that documentation doesn't exist.

Plus: I don't know about others, but I don't configure ctags to lead me
to declarations.  So finding the documentation for cpm_pm_enter() is a
single keystroke if it's in the .c file, and a big PITA if it is in the
.h file.

Also, this documentation could trivially be converted into kerneldoc
format - you may as well do this?

> +int cpu_pm_enter(void);

An actual design question: the interface assumes that CPU PM is a
boolean state: on or off.  "a CPU goes to a low power state that turns
off power to the CPU's power domain".

Will that always be true for all CPUs?  Or should the interface have
the capability of notifying clients of multi-level power state
transitions?

> +
> +/*
> + * cpm_pm_exit
> + *
> + * Notifies listeners that a single cpu is exiting a low power state that may
> + * have caused some blocks in the same power domain as the cpu to reset.
> + *
> + * Must be called on the affected cpu with interrupts disabled.

It's unobvious (to little old me) why all these things need to be
called under local_irq_disable().  I suggest the addition of a code
comment and changelog update so that others are not similarly
mystified.

> + */
> +int cpu_pm_exit(void);
>
> ...
>
> +int cpu_cluster_pm_enter(void)
> +{
> +	int nr_calls;
> +	int ret = 0;
> +
> +	read_lock(&cpu_pm_notifier_lock);
> +	ret = cpu_pm_notify(CPU_CLUSTER_PM_ENTER, -1, &nr_calls);
> +	if (ret)
> +		cpu_pm_notify(CPU_CLUSTER_PM_ENTER_FAILED, nr_calls - 1, NULL);

What's going on with nr_calls?  Avoiding calling the most recently
registered callback?  It is unclear why.  Some explanation here would
be good.

> +	read_unlock(&cpu_pm_notifier_lock);
> +
> +	return ret;
> +}
>
> ...
>
> --- a/kernel/power/Kconfig
> +++ b/kernel/power/Kconfig
> @@ -235,3 +235,7 @@ config PM_GENERIC_DOMAINS
>  config PM_GENERIC_DOMAINS_RUNTIME
>  	def_bool y
>  	depends on PM_RUNTIME && PM_GENERIC_DOMAINS
> +
> +config CPU_PM
> +	def_bool y
> +	depends on SUSPEND || CPU_IDLE

This will unconditionally include kernel/cpu_pm.o in x86 kernels, and
it's all dead code.  Fix, please!

[PATCH v2 1/5] cpu_pm: Add cpu power management notifiers

[Santosh]

On Saturday 10 September 2011 04:26 AM, Andrew Morton wrote:
> On Sat, 3 Sep 2011 20:09:11 +0530
> Santosh Shilimkar<santosh.shilimkar@ti.com>  wrote:
>
>> From: Colin Cross<ccross@android.com>
>>
>> During some CPU power modes entered during idle, hotplug and
>> suspend, peripherals located in the CPU power domain, such as
>> the GIC, localtimers, and VFP, may be powered down.  Add a
>> notifier chain that allows drivers for those peripherals to
>> be notified before and after they may be reset.
>
> Have you identified which indivudual you hope/expect to merge this into
> mainline?
>
> The code is presumably and hopefully applicable to architectures other
> than ARM, yes?  Can you suggest likely candidate architectures so we
> can go off and bug the relevant maintainers to review it?
>
I was planning to send the pull request to Russell.

>>
>> ...
>>
>> +/*
>> + * When a CPU goes to a low power state that turns off power to the CPU's
>> + * power domain, the contents of some blocks (floating point coprocessors,
>> + * interrutp controllers, caches, timers) in the same power domain can
>
> s/interrutp/interrupt/
ok.
>
>> + * be lost.  The cpm_pm notifiers provide a method for platform idle, suspend,
>> + * and hotplug implementations to notify the drivers for these blocks that
>> + * they may be reset.
>> + *
>> + * All cpu_pm notifications must be called with interrupts disabled.
>> + *
>> + * The notifications are split into two classes, CPU notifications and CPU
>
> s/,/:/
ok
>
>> + * cluster notifications.
>> + *
>> + * CPU notifications apply to a single CPU, and must be called on the affected
>
> s/,// ;)
>
ok
>> + * CPU.  They are used to save per-cpu context for affected blocks.
>> + *
>> + * CPU cluster notifications apply to all CPUs in a single power domain. They
>> + * are used to save any global context for affected blocks, and must be called
>> + * after all the CPUs in the power domain have been notified of the low power
>> + * state.
>> + *
>
> Remove this line.
>
ok.

>> + */
>> +
>>
>> ...
>>
>> +/*
>> + * cpm_pm_enter
>> + *
>> + * Notifies listeners that a single cpu is entering a low power state that may
>> + * cause some blocks in the same power domain as the cpu to reset.
>> + *
>> + * Must be called on the affected cpu with interrupts disabled.  Platform is
>> + * responsible for ensuring that cpu_pm_enter is not called twice on the same
>> + * cpu before cpu_pm_exit is called.
>> + */
>
> It's unconventional to put the documentation over the declarations in the
> .h file.  It's not a *bad* idea per-se, but we generally don't do it.
> People will look at the definition in .c for the documentation and it
> if isn't there, some will assume that documentation doesn't exist.
>
> Plus: I don't know about others, but I don't configure ctags to lead me
> to declarations.  So finding the documentation for cpm_pm_enter() is a
> single keystroke if it's in the .c file, and a big PITA if it is in the
> .h file.
>
Will move that to C file.

> Also, this documentation could trivially be converted into kerneldoc
> format - you may as well do this?
>
ok

>> +int cpu_pm_enter(void);
>
> An actual design question: the interface assumes that CPU PM is a
> boolean state: on or off.  "a CPU goes to a low power state that turns
> off power to the CPU's power domain".
>
> Will that always be true for all CPUs?  Or should the interface have
> the capability of notifying clients of multi-level power state
> transitions?
>
Yes. Those are CPU cluster events. We already use them for
interrupt controller which looses power only when CPU cluster
looses power.

>> +
>> +/*
>> + * cpm_pm_exit
>> + *
>> + * Notifies listeners that a single cpu is exiting a low power state that may
>> + * have caused some blocks in the same power domain as the cpu to reset.
>> + *
>> + * Must be called on the affected cpu with interrupts disabled.
>
> It's unobvious (to little old me) why all these things need to be
> called under local_irq_disable().  I suggest the addition of a code
> comment and changelog update so that others are not similarly
> mystified.
>
These notifiers are used in CPUIDLE and suspend code. We were
aslo disabling the interrupt controller. Will add more description
to it.

>> + */
>> +int cpu_pm_exit(void);
>>
>> ...
>>
>> +int cpu_cluster_pm_enter(void)
>> +{
>> +	int nr_calls;
>> +	int ret = 0;
>> +
>> +	read_lock(&cpu_pm_notifier_lock);
>> +	ret = cpu_pm_notify(CPU_CLUSTER_PM_ENTER, -1,&nr_calls);
>> +	if (ret)
>> +		cpu_pm_notify(CPU_CLUSTER_PM_ENTER_FAILED, nr_calls - 1, NULL);
>
> What's going on with nr_calls?  Avoiding calling the most recently
> registered callback?  It is unclear why.  Some explanation here would
> be good.
>
ok.

>> +	read_unlock(&cpu_pm_notifier_lock);
>> +
>> +	return ret;
>> +}
>>
>> ...
>>
>> --- a/kernel/power/Kconfig
>> +++ b/kernel/power/Kconfig
>> @@ -235,3 +235,7 @@ config PM_GENERIC_DOMAINS
>>   config PM_GENERIC_DOMAINS_RUNTIME
>>   	def_bool y
>>   	depends on PM_RUNTIME&&  PM_GENERIC_DOMAINS
>> +
>> +config CPU_PM
>> +	def_bool y
>> +	depends on SUSPEND || CPU_IDLE
>
> This will unconditionally include kernel/cpu_pm.o in x86 kernels, and
> it's all dead code.  Fix, please!
The idea was to make it not depend on any arch. I can make this default
n and then enabled it on ARCH_ARM. Same things needs to be done on other
arch's whoever wants to use it.

Regards
Santsoh

[PATCH v2 0/5] CPU PM notifiers

[Santosh]

On Friday 09 September 2011 11:30 PM, Kevin Hilman wrote:
> Santosh Shilimkar<santosh.shilimkar@ti.com>  writes:
>
> [...]
>
>> This patch set tries to address concerns with platform pm code
>> calling into the driver for every block in the Cortex A9s
>> during idle, hotplug, and suspend.  The first patch adds cpu pm
>> notifiers that can be called by platform code, the second uses
>> the notifier to save and restore the GIC state, and the third
>> saves the VFP state.
>>
>> The notifiers are used for two types of events, CPU PM events and
>> CPU cluster PM events.  CPU PM events are used to save and restore
>> per-cpu context when a single CPU is preparing to enter or has
>> just exited a low power state.  For example, the VFP saves the
>> last thread context, and the GIC saves banked CPU registers.
>>
>> CPU cluster events are used after all the CPUs in a power domain
>> have been prepared for the low power state.  The GIC uses these
>> events to save global register state.
>
> Stepping back from my earlier objections, I think I had a fundamental
> misunderstanding about what these notifiers should be used for.
>
> The current assumptions/goals seem to be
>
> 1) used only for devices in the same power domain as the CPU (cluster)
> 2) use only for one specific power state of the CPU (cluster): off.
>
> For awhile now, we've been discussing how to better coordinate CPU PM
> transitions (CPUidle) with non-CPU PM transitions (runtime PM) for
> devices that are tightly coupled to the CPU, but not necessarily in the
> same powerdomain.
>
> I was assuming (and hoping) that CPU PM notifiers could be used to do
> that, but the more I think about it, I don't think we can achieve the
> current CPU PM goals and the coordination with runtime PM with this
> series.
>
> I think it's more likely that we'll need to do some work with Rafael's
> new PM domains to make that work correctly.
>
> So, I'll retract my objections to this series, and feel free to add
>
> Reviewed-by: Kevin Hilman<khilman@ti.com>
>
Will add.
Thanks for the review Kevin.

Regards
Santosh

Regards

[PATCH v2 1/5] cpu_pm: Add cpu power management notifiers

[Santosh]

Andrew,

On Saturday 10 September 2011 04:26 AM, Andrew Morton wrote:
> On Sat, 3 Sep 2011 20:09:11 +0530
> Santosh Shilimkar<santosh.shilimkar@ti.com>  wrote:

[...]

>
> Have you identified which indivudual you hope/expect to merge this into
> mainline?
>
> The code is presumably and hopefully applicable to architectures other
> than ARM, yes?  Can you suggest likely candidate architectures so we
> can go off and bug the relevant maintainers to review it?
>
Sorry I missed above question in last email. Am not too sure but any
arch which has CPU local timers, interrupt controller, Floating point
co-processor etc, should be able to make use of it. So it's applicable
to any architecture, but whether they want to use it, i don't know.

As you noticed most of the ARM machine's are adapting to it.
Along with generic patch, there are couple ARM drivers adapting
to make use of it. That was the reason I was pushing this series
via Russell's tree.

Regards
Santosh

[PATCH v2 1/5] cpu_pm: Add cpu power management notifiers

[Santosh]

On Saturday 10 September 2011 03:01 PM, Santosh wrote:
> Andrew,
>
> On Saturday 10 September 2011 04:26 AM, Andrew Morton wrote:
>> On Sat, 3 Sep 2011 20:09:11 +0530
>> Santosh Shilimkar<santosh.shilimkar@ti.com> wrote:
>
> [...]
>
>>
>> Have you identified which indivudual you hope/expect to merge this into
>> mainline?
>>
>> The code is presumably and hopefully applicable to architectures other
>> than ARM, yes? Can you suggest likely candidate architectures so we
>> can go off and bug the relevant maintainers to review it?
>>
> Sorry I missed above question in last email. Am not too sure but any
> arch which has CPU local timers, interrupt controller, Floating point
> co-processor etc, should be able to make use of it. So it's applicable
> to any architecture, but whether they want to use it, i don't know.
>
> As you noticed most of the ARM machine's are adapting to it.
> Along with generic patch, there are couple ARM drivers adapting
> to make use of it. That was the reason I was pushing this series
> via Russell's tree.
>
Here is an updated patch addressing your comments.
Attaching the same in case mailer eats spaces.

 From 6a0ef5d42da459d9eba0a5f396e6d9a95d3f94ff Mon Sep 17 00:00:00 2001
From: Colin Cross <ccross@android.com>
Date: Thu, 10 Feb 2011 02:04:45 -0800
Subject: [PATCH 1/5] cpu_pm: Add cpu power management notifiers

During some CPU power modes entered during idle, hotplug and
suspend, peripherals located in the CPU power domain, such as
the GIC, localtimers, and VFP, may be powered down.  Add a
notifier chain that allows drivers for those peripherals to
be notified before and after they may be reset.

Notified drivers can include VFP co-processor, interrupt controller
and it's PM extensions, local CPU timers context save/restore which
shouldn't be interrupted. Hence CPU PM event APIs  must be called
with interrupts disabled.

Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Kevin Hilman <khilman@ti.com>
Tested-and-Acked-by: Shawn Guo <shawn.guo@linaro.org>
Tested-by: Kevin Hilman <khilman@ti.com>
---
  include/linux/cpu_pm.h |  109 ++++++++++++++++++++++++++
  kernel/Makefile        |    1 +
  kernel/cpu_pm.c        |  200 
++++++++++++++++++++++++++++++++++++++++++++++++
  kernel/power/Kconfig   |    4 +
  4 files changed, 314 insertions(+), 0 deletions(-)
  create mode 100644 include/linux/cpu_pm.h
  create mode 100644 kernel/cpu_pm.c

diff --git a/include/linux/cpu_pm.h b/include/linux/cpu_pm.h
new file mode 100644
index 0000000..455b233
--- /dev/null
+++ b/include/linux/cpu_pm.h
@@ -0,0 +1,109 @@
+/*
+ * Copyright (C) 2011 Google, Inc.
+ *
+ * Author:
+ *	Colin Cross <ccross@android.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _LINUX_CPU_PM_H
+#define _LINUX_CPU_PM_H
+
+#include <linux/kernel.h>
+#include <linux/notifier.h>
+
+/*
+ * When a CPU goes to a low power state that turns off power to the CPU's
+ * power domain, the contents of some blocks (floating point coprocessors,
+ * interrupt controllers, caches, timers) in the same power domain can
+ * be lost.  The cpm_pm notifiers provide a method for platform idle, 
suspend,
+ * and hotplug implementations to notify the drivers for these blocks that
+ * they may be reset.
+ *
+ * All cpu_pm notifications must be called with interrupts disabled.
+ *
+ * The notifications are split into two classes: CPU notifications and CPU
+ * cluster notifications.
+ *
+ * CPU notifications apply to a single CPU and must be called on the 
affected
+ * CPU.  They are used to save per-cpu context for affected blocks.
+ *
+ * CPU cluster notifications apply to all CPUs in a single power 
domain. They
+ * are used to save any global context for affected blocks, and must be 
called
+ * after all the CPUs in the power domain have been notified of the low 
power
+ * state.
+ */
+
+/*
+ * Event codes passed as unsigned long val to notifier calls
+ */
+enum cpu_pm_event {
+	/* A single cpu is entering a low power state */
+	CPU_PM_ENTER,
+
+	/* A single cpu failed to enter a low power state */
+	CPU_PM_ENTER_FAILED,
+
+	/* A single cpu is exiting a low power state */
+	CPU_PM_EXIT,
+
+	/* A cpu power domain is entering a low power state */
+	CPU_CLUSTER_PM_ENTER,
+
+	/* A cpu power domain failed to enter a low power state */
+	CPU_CLUSTER_PM_ENTER_FAILED,
+
+	/* A cpu power domain is exiting a low power state */
+	CPU_CLUSTER_PM_EXIT,
+};
+
+#ifdef CONFIG_CPU_PM
+int cpu_pm_register_notifier(struct notifier_block *nb);
+int cpu_pm_unregister_notifier(struct notifier_block *nb);
+int cpu_pm_enter(void);
+int cpu_pm_exit(void);
+int cpu_cluster_pm_enter(void);
+int cpu_cluster_pm_exit(void);
+
+#else
+
+static inline int cpu_pm_register_notifier(struct notifier_block *nb)
+{
+	return 0;
+}
+
+static inline int cpu_pm_unregister_notifier(struct notifier_block *nb)
+{
+	return 0;
+}
+
+static inline int cpu_pm_enter(void)
+{
+	return 0;
+}
+
+static inline int cpu_pm_exit(void)
+{
+	return 0;
+}
+
+static inline int cpu_cluster_pm_enter(void)
+{
+	return 0;
+}
+
+static inline int cpu_cluster_pm_exit(void)
+{
+	return 0;
+}
+#endif
+#endif
diff --git a/kernel/Makefile b/kernel/Makefile
index eca595e..988cb3d 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -101,6 +101,7 @@ obj-$(CONFIG_RING_BUFFER) += trace/
  obj-$(CONFIG_TRACEPOINTS) += trace/
  obj-$(CONFIG_SMP) += sched_cpupri.o
  obj-$(CONFIG_IRQ_WORK) += irq_work.o
+obj-$(CONFIG_CPU_PM) += cpu_pm.o

  obj-$(CONFIG_PERF_EVENTS) += events/

diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c
new file mode 100644
index 0000000..4d1ff4a
--- /dev/null
+++ b/kernel/cpu_pm.c
@@ -0,0 +1,200 @@
+/*
+ * Copyright (C) 2011 Google, Inc.
+ *
+ * Author:
+ *	Colin Cross <ccross@android.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/cpu_pm.h>
+#include <linux/module.h>
+#include <linux/notifier.h>
+#include <linux/spinlock.h>
+
+static DEFINE_RWLOCK(cpu_pm_notifier_lock);
+static RAW_NOTIFIER_HEAD(cpu_pm_notifier_chain);
+
+static int cpu_pm_notify(enum cpu_pm_event event, int nr_to_call, int 
*nr_calls)
+{
+	int ret;
+
+	ret = __raw_notifier_call_chain(&cpu_pm_notifier_chain, event, NULL,
+		nr_to_call, nr_calls);
+
+	return notifier_to_errno(ret);
+}
+
+/**
+ * cpu_pm_register_notifier - register a driver with cpu_pm
+ * @nb: notifier block to register
+ *
+ * Add a driver to a list of drivers that are notified about
+ * CPU and CPU cluster low power entry and exit.
+ *
+ * This function may sleep, and has the same return conditions as
+ * raw_notifier_chain_register.
+ */
+int cpu_pm_register_notifier(struct notifier_block *nb)
+{
+	unsigned long flags;
+	int ret;
+
+	write_lock_irqsave(&cpu_pm_notifier_lock, flags);
+	ret = raw_notifier_chain_register(&cpu_pm_notifier_chain, nb);
+	write_unlock_irqrestore(&cpu_pm_notifier_lock, flags);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_register_notifier);
+
+/**
+ * cpu_pm_unregister_notifier - unregister a driver with cpu_pm
+ * @nb: notifier block to be unregistered
+ *
+ * Remove a driver from the CPU PM notifier list.
+ *
+ * This function may sleep, and has the same return conditions as
+ * raw_notifier_chain_unregister.
+ */
+int cpu_pm_unregister_notifier(struct notifier_block *nb)
+{
+	unsigned long flags;
+	int ret;
+
+	write_lock_irqsave(&cpu_pm_notifier_lock, flags);
+	ret = raw_notifier_chain_unregister(&cpu_pm_notifier_chain, nb);
+	write_unlock_irqrestore(&cpu_pm_notifier_lock, flags);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_unregister_notifier);
+
+/**
+ * cpm_pm_enter - CPU low power entry notifier
+ *
+ * Notifies listeners that a single CPU is entering a low power state 
that may
+ * cause some blocks in the same power domain as the cpu to reset.
+ *
+ * Must be called on the affected CPU with interrupts disabled. 
Platform is
+ * responsible for ensuring that cpu_pm_enter is not called twice on 
the same
+ * CPU before cpu_pm_exit is called. Notified drivers can include VFP
+ * co-processor, interrupt controller and it's PM extensions, local CPU
+ * timers context save/restore which shouldn't be interrupted. Hence it
+ * must be called with interrupts disabled.
+ *
+ * Return conditions are same as __raw_notifier_call_chain.
+ */
+int cpu_pm_enter(void)
+{
+	int nr_calls;
+	int ret = 0;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_PM_ENTER, -1, &nr_calls);
+	if (ret)
+		/*
+		 * Inform listeners (nr_calls - 1) about failure of CPU PM
+		 * PM entry who are notified earlier to prepare for it.
+		 */
+		cpu_pm_notify(CPU_PM_ENTER_FAILED, nr_calls - 1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_enter);
+
+/**
+ * cpm_pm_exit - CPU low power exit notifier
+ *
+ * Notifies listeners that a single CPU is exiting a low power state 
that may
+ * have caused some blocks in the same power domain as the cpu to reset.
+ *
+ * Notified drivers can include VFP co-processor, interrupt controller
+ * and it's PM extensions, local CPU timers context save/restore which
+ * shouldn't be interrupted. Hence it must be called with interrupts 
disabled.
+ *
+ * Return conditions are same as __raw_notifier_call_chain.
+ */
+int cpu_pm_exit(void)
+{
+	int ret;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_PM_EXIT, -1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_pm_exit);
+
+/**
+ * cpm_cluster_pm_enter - CPU cluster low power entry notifier
+ *
+ * Notifies listeners that all cpus in a power domain are entering a 
low power
+ * state that may cause some blocks in the same power domain to reset.
+ *
+ * Must be called after cpu_pm_enter has been called on all cpus in the 
power
+ * domain, and before cpu_pm_exit has been called on any cpu in the power
+ * domain. Notified drivers can include VFP co-processor, interrupt 
controller
+ * and it's PM extensions, local CPU timers context save/restore which
+ * shouldn't be interrupted. Hence it must be called with interrupts 
disabled.
+ *
+ * Must be called with interrupts disabled.
+ *
+ * Return conditions are same as __raw_notifier_call_chain.
+ */
+int cpu_cluster_pm_enter(void)
+{
+	int nr_calls;
+	int ret = 0;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_CLUSTER_PM_ENTER, -1, &nr_calls);
+	if (ret)
+		/*
+		 * Inform listeners (nr_calls - 1) about failure of CPU cluster
+		 * PM entry who are notified earlier to prepare for it.
+		 */
+		cpu_pm_notify(CPU_CLUSTER_PM_ENTER_FAILED, nr_calls - 1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_cluster_pm_enter);
+
+/**
+ * cpm_cluster_pm_exit - CPU cluster low power exit notifier
+ *
+ * Notifies listeners that all cpus in a power domain are exiting form a
+ * low power state that may have caused some blocks in the same power 
domain
+ * to reset.
+ *
+ * Must be called after cpu_pm_exit has been called on all cpus in the 
power
+ * domain, and before cpu_pm_exit has been called on any cpu in the power
+ * domain. Notified drivers can include VFP co-processor, interrupt 
controller
+ * and it's PM extensions, local CPU timers context save/restore which
+ * shouldn't be interrupted. Hence it must be called with interrupts 
disabled.
+ *
+ * Return conditions are same as __raw_notifier_call_chain.
+ */
+int cpu_cluster_pm_exit(void)
+{
+	int ret;
+
+	read_lock(&cpu_pm_notifier_lock);
+	ret = cpu_pm_notify(CPU_CLUSTER_PM_EXIT, -1, NULL);
+	read_unlock(&cpu_pm_notifier_lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(cpu_cluster_pm_exit);
diff --git a/kernel/power/Kconfig b/kernel/power/Kconfig
index 3744c59..80a8597 100644
--- a/kernel/power/Kconfig
+++ b/kernel/power/Kconfig
@@ -235,3 +235,7 @@ config PM_GENERIC_DOMAINS
  config PM_GENERIC_DOMAINS_RUNTIME
  	def_bool y
  	depends on PM_RUNTIME && PM_GENERIC_DOMAINS
+
+config CPU_PM
+	bool
+	depends on SUSPEND || CPU_IDLE
-- 
1.7.4.1

Regards
Santosh

[PATCH v2 1/5] cpu_pm: Add cpu power management notifiers

[Santosh]

( CC'ing linux-arch at vger.kernel.org)

On Monday 12 September 2011 10:32 AM, Santosh wrote:
> On Saturday 10 September 2011 03:01 PM, Santosh wrote:
>> Andrew,
>>
>> On Saturday 10 September 2011 04:26 AM, Andrew Morton wrote:
>>> On Sat, 3 Sep 2011 20:09:11 +0530
>>> Santosh Shilimkar<santosh.shilimkar@ti.com> wrote:
>>
>> [...]
>>
>>>
>>> Have you identified which indivudual you hope/expect to merge this into
>>> mainline?
>>>
>>> The code is presumably and hopefully applicable to architectures other
>>> than ARM, yes? Can you suggest likely candidate architectures so we
>>> can go off and bug the relevant maintainers to review it?
>>>
>> Sorry I missed above question in last email. Am not too sure but any
>> arch which has CPU local timers, interrupt controller, Floating point
>> co-processor etc, should be able to make use of it. So it's applicable
>> to any architecture, but whether they want to use it, i don't know.
>>
>> As you noticed most of the ARM machine's are adapting to it.
>> Along with generic patch, there are couple ARM drivers adapting
>> to make use of it. That was the reason I was pushing this series
>> via Russell's tree.
>>
> Here is an updated patch addressing your comments.
> Attaching the same in case mailer eats spaces.
>
>  From 6a0ef5d42da459d9eba0a5f396e6d9a95d3f94ff Mon Sep 17 00:00:00 2001
> From: Colin Cross <ccross@android.com>
> Date: Thu, 10 Feb 2011 02:04:45 -0800
> Subject: [PATCH 1/5] cpu_pm: Add cpu power management notifiers
>
> During some CPU power modes entered during idle, hotplug and
> suspend, peripherals located in the CPU power domain, such as
> the GIC, localtimers, and VFP, may be powered down. Add a
> notifier chain that allows drivers for those peripherals to
> be notified before and after they may be reset.
>
> Notified drivers can include VFP co-processor, interrupt controller
> and it's PM extensions, local CPU timers context save/restore which
> shouldn't be interrupted. Hence CPU PM event APIs must be called
> with interrupts disabled.
>
> Signed-off-by: Colin Cross <ccross@android.com>
> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
> Reviewed-by: Kevin Hilman <khilman@ti.com>
> Tested-and-Acked-by: Shawn Guo <shawn.guo@linaro.org>
> Tested-by: Kevin Hilman <khilman@ti.com>
> ---
> include/linux/cpu_pm.h | 109 ++++++++++++++++++++++++++
> kernel/Makefile | 1 +
> kernel/cpu_pm.c | 200 ++++++++++++++++++++++++++++++++++++++++++++++++
> kernel/power/Kconfig | 4 +
> 4 files changed, 314 insertions(+), 0 deletions(-)
> create mode 100644 include/linux/cpu_pm.h
> create mode 100644 kernel/cpu_pm.c
>
> diff --git a/include/linux/cpu_pm.h b/include/linux/cpu_pm.h
> new file mode 100644
> index 0000000..455b233
> --- /dev/null
> +++ b/include/linux/cpu_pm.h
> @@ -0,0 +1,109 @@
> +/*
> + * Copyright (C) 2011 Google, Inc.
> + *
> + * Author:
> + * Colin Cross <ccross@android.com>
> + *
> + * This software is licensed under the terms of the GNU General Public
> + * License version 2, as published by the Free Software Foundation, and
> + * may be copied, distributed, and modified under those terms.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#ifndef _LINUX_CPU_PM_H
> +#define _LINUX_CPU_PM_H
> +
> +#include <linux/kernel.h>
> +#include <linux/notifier.h>
> +
> +/*
> + * When a CPU goes to a low power state that turns off power to the CPU's
> + * power domain, the contents of some blocks (floating point coprocessors,
> + * interrupt controllers, caches, timers) in the same power domain can
> + * be lost. The cpm_pm notifiers provide a method for platform idle,
> suspend,
> + * and hotplug implementations to notify the drivers for these blocks that
> + * they may be reset.
> + *
> + * All cpu_pm notifications must be called with interrupts disabled.
> + *
> + * The notifications are split into two classes: CPU notifications and CPU
> + * cluster notifications.
> + *
> + * CPU notifications apply to a single CPU and must be called on the
> affected
> + * CPU. They are used to save per-cpu context for affected blocks.
> + *
> + * CPU cluster notifications apply to all CPUs in a single power
> domain. They
> + * are used to save any global context for affected blocks, and must be
> called
> + * after all the CPUs in the power domain have been notified of the low
> power
> + * state.
> + */
> +
> +/*
> + * Event codes passed as unsigned long val to notifier calls
> + */
> +enum cpu_pm_event {
> + /* A single cpu is entering a low power state */
> + CPU_PM_ENTER,
> +
> + /* A single cpu failed to enter a low power state */
> + CPU_PM_ENTER_FAILED,
> +
> + /* A single cpu is exiting a low power state */
> + CPU_PM_EXIT,
> +
> + /* A cpu power domain is entering a low power state */
> + CPU_CLUSTER_PM_ENTER,
> +
> + /* A cpu power domain failed to enter a low power state */
> + CPU_CLUSTER_PM_ENTER_FAILED,
> +
> + /* A cpu power domain is exiting a low power state */
> + CPU_CLUSTER_PM_EXIT,
> +};
> +
> +#ifdef CONFIG_CPU_PM
> +int cpu_pm_register_notifier(struct notifier_block *nb);
> +int cpu_pm_unregister_notifier(struct notifier_block *nb);
> +int cpu_pm_enter(void);
> +int cpu_pm_exit(void);
> +int cpu_cluster_pm_enter(void);
> +int cpu_cluster_pm_exit(void);
> +
> +#else
> +
> +static inline int cpu_pm_register_notifier(struct notifier_block *nb)
> +{
> + return 0;
> +}
> +
> +static inline int cpu_pm_unregister_notifier(struct notifier_block *nb)
> +{
> + return 0;
> +}
> +
> +static inline int cpu_pm_enter(void)
> +{
> + return 0;
> +}
> +
> +static inline int cpu_pm_exit(void)
> +{
> + return 0;
> +}
> +
> +static inline int cpu_cluster_pm_enter(void)
> +{
> + return 0;
> +}
> +
> +static inline int cpu_cluster_pm_exit(void)
> +{
> + return 0;
> +}
> +#endif
> +#endif
> diff --git a/kernel/Makefile b/kernel/Makefile
> index eca595e..988cb3d 100644
> --- a/kernel/Makefile
> +++ b/kernel/Makefile
> @@ -101,6 +101,7 @@ obj-$(CONFIG_RING_BUFFER) += trace/
> obj-$(CONFIG_TRACEPOINTS) += trace/
> obj-$(CONFIG_SMP) += sched_cpupri.o
> obj-$(CONFIG_IRQ_WORK) += irq_work.o
> +obj-$(CONFIG_CPU_PM) += cpu_pm.o
>
> obj-$(CONFIG_PERF_EVENTS) += events/
>
> diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c
> new file mode 100644
> index 0000000..4d1ff4a
> --- /dev/null
> +++ b/kernel/cpu_pm.c
> @@ -0,0 +1,200 @@
> +/*
> + * Copyright (C) 2011 Google, Inc.
> + *
> + * Author:
> + * Colin Cross <ccross@android.com>
> + *
> + * This software is licensed under the terms of the GNU General Public
> + * License version 2, as published by the Free Software Foundation, and
> + * may be copied, distributed, and modified under those terms.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/cpu_pm.h>
> +#include <linux/module.h>
> +#include <linux/notifier.h>
> +#include <linux/spinlock.h>
> +
> +static DEFINE_RWLOCK(cpu_pm_notifier_lock);
> +static RAW_NOTIFIER_HEAD(cpu_pm_notifier_chain);
> +
> +static int cpu_pm_notify(enum cpu_pm_event event, int nr_to_call, int
> *nr_calls)
> +{
> + int ret;
> +
> + ret = __raw_notifier_call_chain(&cpu_pm_notifier_chain, event, NULL,
> + nr_to_call, nr_calls);
> +
> + return notifier_to_errno(ret);
> +}
> +
> +/**
> + * cpu_pm_register_notifier - register a driver with cpu_pm
> + * @nb: notifier block to register
> + *
> + * Add a driver to a list of drivers that are notified about
> + * CPU and CPU cluster low power entry and exit.
> + *
> + * This function may sleep, and has the same return conditions as
> + * raw_notifier_chain_register.
> + */
> +int cpu_pm_register_notifier(struct notifier_block *nb)
> +{
> + unsigned long flags;
> + int ret;
> +
> + write_lock_irqsave(&cpu_pm_notifier_lock, flags);
> + ret = raw_notifier_chain_register(&cpu_pm_notifier_chain, nb);
> + write_unlock_irqrestore(&cpu_pm_notifier_lock, flags);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(cpu_pm_register_notifier);
> +
> +/**
> + * cpu_pm_unregister_notifier - unregister a driver with cpu_pm
> + * @nb: notifier block to be unregistered
> + *
> + * Remove a driver from the CPU PM notifier list.
> + *
> + * This function may sleep, and has the same return conditions as
> + * raw_notifier_chain_unregister.
> + */
> +int cpu_pm_unregister_notifier(struct notifier_block *nb)
> +{
> + unsigned long flags;
> + int ret;
> +
> + write_lock_irqsave(&cpu_pm_notifier_lock, flags);
> + ret = raw_notifier_chain_unregister(&cpu_pm_notifier_chain, nb);
> + write_unlock_irqrestore(&cpu_pm_notifier_lock, flags);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(cpu_pm_unregister_notifier);
> +
> +/**
> + * cpm_pm_enter - CPU low power entry notifier
> + *
> + * Notifies listeners that a single CPU is entering a low power state
> that may
> + * cause some blocks in the same power domain as the cpu to reset.
> + *
> + * Must be called on the affected CPU with interrupts disabled.
> Platform is
> + * responsible for ensuring that cpu_pm_enter is not called twice on
> the same
> + * CPU before cpu_pm_exit is called. Notified drivers can include VFP
> + * co-processor, interrupt controller and it's PM extensions, local CPU
> + * timers context save/restore which shouldn't be interrupted. Hence it
> + * must be called with interrupts disabled.
> + *
> + * Return conditions are same as __raw_notifier_call_chain.
> + */
> +int cpu_pm_enter(void)
> +{
> + int nr_calls;
> + int ret = 0;
> +
> + read_lock(&cpu_pm_notifier_lock);
> + ret = cpu_pm_notify(CPU_PM_ENTER, -1, &nr_calls);
> + if (ret)
> + /*
> + * Inform listeners (nr_calls - 1) about failure of CPU PM
> + * PM entry who are notified earlier to prepare for it.
> + */
> + cpu_pm_notify(CPU_PM_ENTER_FAILED, nr_calls - 1, NULL);
> + read_unlock(&cpu_pm_notifier_lock);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(cpu_pm_enter);
> +
> +/**
> + * cpm_pm_exit - CPU low power exit notifier
> + *
> + * Notifies listeners that a single CPU is exiting a low power state
> that may
> + * have caused some blocks in the same power domain as the cpu to reset.
> + *
> + * Notified drivers can include VFP co-processor, interrupt controller
> + * and it's PM extensions, local CPU timers context save/restore which
> + * shouldn't be interrupted. Hence it must be called with interrupts
> disabled.
> + *
> + * Return conditions are same as __raw_notifier_call_chain.
> + */
> +int cpu_pm_exit(void)
> +{
> + int ret;
> +
> + read_lock(&cpu_pm_notifier_lock);
> + ret = cpu_pm_notify(CPU_PM_EXIT, -1, NULL);
> + read_unlock(&cpu_pm_notifier_lock);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(cpu_pm_exit);
> +
> +/**
> + * cpm_cluster_pm_enter - CPU cluster low power entry notifier
> + *
> + * Notifies listeners that all cpus in a power domain are entering a
> low power
> + * state that may cause some blocks in the same power domain to reset.
> + *
> + * Must be called after cpu_pm_enter has been called on all cpus in the
> power
> + * domain, and before cpu_pm_exit has been called on any cpu in the power
> + * domain. Notified drivers can include VFP co-processor, interrupt
> controller
> + * and it's PM extensions, local CPU timers context save/restore which
> + * shouldn't be interrupted. Hence it must be called with interrupts
> disabled.
> + *
> + * Must be called with interrupts disabled.
> + *
> + * Return conditions are same as __raw_notifier_call_chain.
> + */
> +int cpu_cluster_pm_enter(void)
> +{
> + int nr_calls;
> + int ret = 0;
> +
> + read_lock(&cpu_pm_notifier_lock);
> + ret = cpu_pm_notify(CPU_CLUSTER_PM_ENTER, -1, &nr_calls);
> + if (ret)
> + /*
> + * Inform listeners (nr_calls - 1) about failure of CPU cluster
> + * PM entry who are notified earlier to prepare for it.
> + */
> + cpu_pm_notify(CPU_CLUSTER_PM_ENTER_FAILED, nr_calls - 1, NULL);
> + read_unlock(&cpu_pm_notifier_lock);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(cpu_cluster_pm_enter);
> +
> +/**
> + * cpm_cluster_pm_exit - CPU cluster low power exit notifier
> + *
> + * Notifies listeners that all cpus in a power domain are exiting form a
> + * low power state that may have caused some blocks in the same power
> domain
> + * to reset.
> + *
> + * Must be called after cpu_pm_exit has been called on all cpus in the
> power
> + * domain, and before cpu_pm_exit has been called on any cpu in the power
> + * domain. Notified drivers can include VFP co-processor, interrupt
> controller
> + * and it's PM extensions, local CPU timers context save/restore which
> + * shouldn't be interrupted. Hence it must be called with interrupts
> disabled.
> + *
> + * Return conditions are same as __raw_notifier_call_chain.
> + */
> +int cpu_cluster_pm_exit(void)
> +{
> + int ret;
> +
> + read_lock(&cpu_pm_notifier_lock);
> + ret = cpu_pm_notify(CPU_CLUSTER_PM_EXIT, -1, NULL);
> + read_unlock(&cpu_pm_notifier_lock);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(cpu_cluster_pm_exit);
> diff --git a/kernel/power/Kconfig b/kernel/power/Kconfig
> index 3744c59..80a8597 100644
> --- a/kernel/power/Kconfig
> +++ b/kernel/power/Kconfig
> @@ -235,3 +235,7 @@ config PM_GENERIC_DOMAINS
> config PM_GENERIC_DOMAINS_RUNTIME
> def_bool y
> depends on PM_RUNTIME && PM_GENERIC_DOMAINS
> +
> +config CPU_PM
> + bool
> + depends on SUSPEND || CPU_IDLE

[PATCH v2 0/5] CPU PM notifiers

[Santosh]

On Saturday 03 September 2011 08:09 PM, Santosh Shilimkar wrote:
> Updates V2:
> - The CPU PM notifiers are moved to generic level instead of keeping
>   it ARM specific as per Rafael's comment.
> - To avoid every driver duplicating the sys-core ops and cpu pm notfiers,
>   the CPU PM common notifier is registering it in one place so that
>   suspend is taken care.
> - CPU COMPLEX events are renamed to more appropriate CPU CLUSTER.
> - GIC distributor is not disabled to avoid the live locks. This is
>   observed with CPUIDLE cases where at time CPU cluster fails to
>   transition to low power states. As such there was no need to
>   disable distributor in first place since on cluster reset, it
>   will get disabled anyways.
> - In VFP code, syscore ops are dropped in favour of CPU PM
>   notifiers since S2R is already handled in core CPU PM notifiers.
> - The last patch is an independent patch but can be pushed along with
>   the other GIC changes done here. 
>  
> V1: [http://lwn.net/Articles/447259/]
> 
> This patch set tries to address concerns with platform pm code
> calling into the driver for every block in the Cortex A9s
> during idle, hotplug, and suspend.  The first patch adds cpu pm
> notifiers that can be called by platform code, the second uses
> the notifier to save and restore the GIC state, and the third
> saves the VFP state.
> 
> The notifiers are used for two types of events, CPU PM events and
> CPU cluster PM events.  CPU PM events are used to save and restore
> per-cpu context when a single CPU is preparing to enter or has
> just exited a low power state.  For example, the VFP saves the
> last thread context, and the GIC saves banked CPU registers.
> 
> CPU cluster events are used after all the CPUs in a power domain
> have been prepared for the low power state.  The GIC uses these
> events to save global register state.
> 
> L2 cache is not covered by this patch set, as the determination
> of when the L2 is reset and when it is retained is
> platform-specific, and most of the APIs necessary are already
> 
> 
> The series is tested with OMAP4 with S2R and CPUIDLE. 
> 
> The following changes since commit c6a389f123b9f68d605bb7e0f9b32ec1e3e14132:
> 
>   Linux 3.1-rc4 (2011-08-28 21:16:01 -0700)
> 
> Colin Cross (5):
>       cpu_pm: Add cpu power management notifiers
>       cpu_pm: call notifiers during suspend
>       ARM: gic: Use cpu pm notifiers to save gic state
>       ARM: vfp: Use cpu pm notifiers to save vfp state
>       ARM: gic: Allow gic arch extensions to provide irqchip flags
> 

Thanks all for the reviews and testing of this series.
I am going to add these patches as part of my pull request
to Russell for the 3.2.

Regards
Santosh