[PATCH v3 00/10] Add tegra30 support for secondary cores

[PATCH v3 00/10] Add tegra30 support for secondary cores

[Peter De Schrijver]

This patchset introduces support for secondary cores on tegra30. It also
introduces some functions for the flowcontroller and adds basic support for
tegra30 powerdomains.

---

This patchset requires http://www.spinics.net/lists/arm-kernel/msg158016.html
for the cortex A9 erratum #743622 workaround

Changes in v3:

 * split out chip id changes in several patches
 * remove reset handler debugging code
 * remove unused size declartion for reset SRAM code
 * remove workaround for cortex A9 erratum #743622. This will be handled by
   generic code.

Changes in v2:

 * moved chip id reading to fuse.c and cleanup code
 * don't set bit 1 of SB_CTRL on Tegra20
 * removed unused variables in tegra_cpu_reset_handler_init()

Peter De Schrijver (10):
  ARM: tegra: cleanup use of chipid register
  ARM: tegra: export Tegra chipid
  ARM: tegra: initialize Tegra chipid early
  ARM: tegra: functions to access the flowcontroller
  ARM: tegra: rework Tegra secondary CPU core bringup
  ARM: tegra: prepare powergate.c for multiple variants
  ARM: tegra: export tegra_powergate_is_powered()
  ARM: tegra: add support for Tegra30 powerdomains
  ARM: tegra: support for Tegra30 CPU powerdomains
  ARM: tegra: support for secondary cores on Tegra30

 arch/arm/mach-tegra/Makefile                 |    2 +
 arch/arm/mach-tegra/common.c                 |    4 +
 arch/arm/mach-tegra/flowctrl.c               |   62 ++++++++++
 arch/arm/mach-tegra/flowctrl.h               |    5 +
 arch/arm/mach-tegra/fuse.c                   |   18 ++-
 arch/arm/mach-tegra/fuse.h                   |    4 +
 arch/arm/mach-tegra/headsmp.S                |  166 ++++++++++++++++++++++++--
 arch/arm/mach-tegra/include/mach/iomap.h     |    3 +
 arch/arm/mach-tegra/include/mach/powergate.h |   15 ++-
 arch/arm/mach-tegra/platsmp.c                |  137 +++++++++++++++-------
 arch/arm/mach-tegra/powergate.c              |   53 +++++++-
 arch/arm/mach-tegra/reset.c                  |   84 +++++++++++++
 arch/arm/mach-tegra/reset.h                  |   50 ++++++++
 13 files changed, 540 insertions(+), 63 deletions(-)
 create mode 100644 arch/arm/mach-tegra/flowctrl.c
 create mode 100644 arch/arm/mach-tegra/reset.c
 create mode 100644 arch/arm/mach-tegra/reset.h

-- 
1.7.4.1

[PATCH v3 01/10] ARM: tegra: cleanup use of chipid register

[Peter De Schrijver]

The chipid register contains information about the Tegra variant and revision.
We want differentiate between Tegra variants for powergating and secondary
core bringup. This patch cleans up the reading and decoding of this register.
In subsequent patches the variant will exported as a global variable.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/fuse.c |   18 +++++++++++-------
 arch/arm/mach-tegra/fuse.h |    3 +++
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/arch/arm/mach-tegra/fuse.c b/arch/arm/mach-tegra/fuse.c
index adfa429..032a936 100644
--- a/arch/arm/mach-tegra/fuse.c
+++ b/arch/arm/mach-tegra/fuse.c
@@ -34,6 +34,7 @@
 int tegra_sku_id;
 int tegra_cpu_process_id;
 int tegra_core_process_id;
+static int tegra_chip_id;
 enum tegra_revision tegra_revision;
 
 /* The BCT to use at boot is specified by board straps that can be read
@@ -66,12 +67,9 @@ static inline bool get_spare_fuse(int bit)
 	return tegra_fuse_readl(FUSE_SPARE_BIT + bit * 4);
 }
 
-static enum tegra_revision tegra_get_revision(void)
+static enum tegra_revision tegra_get_revision(u32 id)
 {
-	void __iomem *chip_id = IO_ADDRESS(TEGRA_APB_MISC_BASE) + 0x804;
-	u32 id = readl(chip_id);
 	u32 minor_rev = (id >> 16) & 0xf;
-	u32 chipid = (id >> 8) & 0xff;
 
 	switch (minor_rev) {
 	case 1:
@@ -79,7 +77,8 @@ static enum tegra_revision tegra_get_revision(void)
 	case 2:
 		return TEGRA_REVISION_A02;
 	case 3:
-		if (chipid == 0x20 && (get_spare_fuse(18) || get_spare_fuse(19)))
+		if (tegra_chip_id == TEGRA20 &&
+			(get_spare_fuse(18) || get_spare_fuse(19)))
 			return TEGRA_REVISION_A03p;
 		else
 			return TEGRA_REVISION_A03;
@@ -92,6 +91,8 @@ static enum tegra_revision tegra_get_revision(void)
 
 void tegra_init_fuse(void)
 {
+	u32 id;
+
 	u32 reg = readl(IO_TO_VIRT(TEGRA_CLK_RESET_BASE + 0x48));
 	reg |= 1 << 28;
 	writel(reg, IO_TO_VIRT(TEGRA_CLK_RESET_BASE + 0x48));
@@ -108,10 +109,13 @@ void tegra_init_fuse(void)
 	reg = tegra_apb_readl(TEGRA_APB_MISC_BASE + STRAP_OPT);
 	tegra_bct_strapping = (reg & RAM_ID_MASK) >> RAM_CODE_SHIFT;
 
-	tegra_revision = tegra_get_revision();
+	id = readl_relaxed(IO_ADDRESS(TEGRA_APB_MISC_BASE) + 0x804);
+	tegra_chip_id = (id >> 8) & 0xff;
+
+	tegra_revision = tegra_get_revision(id);
 
 	pr_info("Tegra Revision: %s SKU: %d CPU Process: %d Core Process: %d\n",
-		tegra_revision_name[tegra_get_revision()],
+		tegra_revision_name[tegra_revision],
 		tegra_sku_id, tegra_cpu_process_id,
 		tegra_core_process_id);
 }
diff --git a/arch/arm/mach-tegra/fuse.h b/arch/arm/mach-tegra/fuse.h
index d65d2ab..c554572 100644
--- a/arch/arm/mach-tegra/fuse.h
+++ b/arch/arm/mach-tegra/fuse.h
@@ -35,6 +35,9 @@ enum tegra_revision {
 #define SKU_ID_AP25E	27
 #define SKU_ID_T25E	28
 
+#define TEGRA20		0x20
+#define TEGRA30		0x30
+
 extern int tegra_sku_id;
 extern int tegra_cpu_process_id;
 extern int tegra_core_process_id;
-- 
1.7.4.1

[PATCH v3 02/10] ARM: tegra: export Tegra chipid

[Peter De Schrijver]

The powergating and reset handling code needs to differentiate between Tegra
variants. Therefore we export the chipid here.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/fuse.c |    2 +-
 arch/arm/mach-tegra/fuse.h |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-tegra/fuse.c b/arch/arm/mach-tegra/fuse.c
index 032a936..3298880 100644
--- a/arch/arm/mach-tegra/fuse.c
+++ b/arch/arm/mach-tegra/fuse.c
@@ -34,7 +34,7 @@
 int tegra_sku_id;
 int tegra_cpu_process_id;
 int tegra_core_process_id;
-static int tegra_chip_id;
+int tegra_chip_id;
 enum tegra_revision tegra_revision;
 
 /* The BCT to use at boot is specified by board straps that can be read
diff --git a/arch/arm/mach-tegra/fuse.h b/arch/arm/mach-tegra/fuse.h
index c554572..d2107b2 100644
--- a/arch/arm/mach-tegra/fuse.h
+++ b/arch/arm/mach-tegra/fuse.h
@@ -41,6 +41,7 @@ enum tegra_revision {
 extern int tegra_sku_id;
 extern int tegra_cpu_process_id;
 extern int tegra_core_process_id;
+extern int tegra_chip_id;
 extern enum tegra_revision tegra_revision;
 
 extern int tegra_bct_strapping;
-- 
1.7.4.1

[PATCH v3 03/10] ARM: tegra: initialize Tegra chipid early

[Peter De Schrijver]

Secondary core bringup relies on the Tegra chipid to distinguish between
Tegra variants. Therefore this data needs to be available early on.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/common.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-tegra/common.c b/arch/arm/mach-tegra/common.c
index e2a2c1f..bdc5d5e 100644
--- a/arch/arm/mach-tegra/common.c
+++ b/arch/arm/mach-tegra/common.c
@@ -125,6 +125,7 @@ void __init tegra20_init_early(void)
 #ifdef CONFIG_ARCH_TEGRA_3x_SOC
 void __init tegra30_init_early(void)
 {
+	tegra_init_fuse();
 	tegra30_init_clocks();
 	tegra_init_cache(0x441, 0x551);
 	tegra_pmc_init();
-- 
1.7.4.1

[PATCH v3 04/10] ARM: tegra: functions to access the flowcontroller

[Peter De Schrijver]

Introduce some functions to write to the flowcontroller registers. The
flowcontroller controls CPU sleepstates and wakeup.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/Makefile   |    1 +
 arch/arm/mach-tegra/flowctrl.c |   62 ++++++++++++++++++++++++++++++++++++++++
 arch/arm/mach-tegra/flowctrl.h |    5 +++
 3 files changed, 68 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm/mach-tegra/flowctrl.c

diff --git a/arch/arm/mach-tegra/Makefile b/arch/arm/mach-tegra/Makefile
index 231c67d..9710547 100644
--- a/arch/arm/mach-tegra/Makefile
+++ b/arch/arm/mach-tegra/Makefile
@@ -9,6 +9,7 @@ obj-y					+= fuse.o
 obj-y					+= cpuidle.o
 obj-y					+= sleep.o
 obj-y					+= pmc.o
+obj-y					+= flowctrl.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= powergate.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)         += tegra2_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_2x_SOC)		+= tegra2_emc.o
diff --git a/arch/arm/mach-tegra/flowctrl.c b/arch/arm/mach-tegra/flowctrl.c
new file mode 100644
index 0000000..fef66a7
--- /dev/null
+++ b/arch/arm/mach-tegra/flowctrl.c
@@ -0,0 +1,62 @@
+/*
+ * arch/arm/mach-tegra/flowctrl.c
+ *
+ * functions and macros to control the flowcontroller
+ *
+ * Copyright (c) 2010-2012, NVIDIA Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/io.h>
+
+#include <mach/iomap.h>
+
+#include "flowctrl.h"
+
+u8 flowctrl_offset_halt_cpu[] = {
+	FLOW_CTRL_HALT_CPU0_EVENTS,
+	FLOW_CTRL_HALT_CPU1_EVENTS,
+	FLOW_CTRL_HALT_CPU1_EVENTS + 8,
+	FLOW_CTRL_HALT_CPU1_EVENTS + 16,
+};
+
+u8 flowctrl_offset_cpu_csr[] = {
+	FLOW_CTRL_CPU0_CSR,
+	FLOW_CTRL_CPU1_CSR,
+	FLOW_CTRL_CPU1_CSR + 8,
+	FLOW_CTRL_CPU1_CSR + 16,
+};
+
+static void flowctrl_update(u8 offset, u32 value)
+{
+	void __iomem *addr = IO_ADDRESS(TEGRA_FLOW_CTRL_BASE) + offset;
+
+	writel(value, addr);
+
+	/* ensure the update has reached the flow controller */
+	wmb();
+	readl_relaxed(addr);
+}
+
+void flowctrl_write_cpu_csr(unsigned int cpuid, u32 value)
+{
+	return flowctrl_update(flowctrl_offset_halt_cpu[cpuid], value);
+}
+
+void flowctrl_write_cpu_halt(unsigned int cpuid, u32 value)
+{
+	return flowctrl_update(flowctrl_offset_cpu_csr[cpuid], value);
+}
diff --git a/arch/arm/mach-tegra/flowctrl.h b/arch/arm/mach-tegra/flowctrl.h
index 74c6efb..1942817 100644
--- a/arch/arm/mach-tegra/flowctrl.h
+++ b/arch/arm/mach-tegra/flowctrl.h
@@ -34,4 +34,9 @@
 #define FLOW_CTRL_HALT_CPU1_EVENTS	0x14
 #define FLOW_CTRL_CPU1_CSR		0x18
 
+#ifndef __ASSEMBLY__
+void flowctrl_write_cpu_csr(unsigned int cpuid, u32 value);
+void flowctrl_write_cpu_halt(unsigned int cpuid, u32 value);
+#endif
+
 #endif
-- 
1.7.4.1

[PATCH v3 05/10] ARM: tegra: rework Tegra secondary CPU core bringup

[Peter De Schrijver]

Prepare the Tegra secondary CPU core bringup code for other Tegra variants.
The reset handler is also generalized to allow for future introduction of
powersaving modes which turn off the CPU cores.

Based on work by:

Scott Williams <scwilliams@nvidia.com>
Chris Johnson <cwj@nvidia.com>
Colin Cross <ccross@android.com>

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/Makefile             |    1 +
 arch/arm/mach-tegra/headsmp.S            |  134 ++++++++++++++++++++++++++++--
 arch/arm/mach-tegra/include/mach/iomap.h |    3 +
 arch/arm/mach-tegra/platsmp.c            |   97 ++++++++++++----------
 arch/arm/mach-tegra/reset.c              |   84 +++++++++++++++++++
 arch/arm/mach-tegra/reset.h              |   50 +++++++++++
 6 files changed, 316 insertions(+), 53 deletions(-)
 create mode 100644 arch/arm/mach-tegra/reset.c
 create mode 100644 arch/arm/mach-tegra/reset.h

diff --git a/arch/arm/mach-tegra/Makefile b/arch/arm/mach-tegra/Makefile
index 9710547..3ad1990 100644
--- a/arch/arm/mach-tegra/Makefile
+++ b/arch/arm/mach-tegra/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= board-dt-tegra30.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= tegra30_clocks.o
 obj-$(CONFIG_ARCH_TEGRA_3x_SOC)		+= board-cardhu-pinmux.o
 obj-$(CONFIG_SMP)                       += platsmp.o localtimer.o headsmp.o
+obj-$(CONFIG_SMP)                       += reset.o
 obj-$(CONFIG_HOTPLUG_CPU)               += hotplug.o
 obj-$(CONFIG_TEGRA_SYSTEM_DMA)		+= dma.o apbio.o
 obj-$(CONFIG_CPU_FREQ)                  += cpu-tegra.o
diff --git a/arch/arm/mach-tegra/headsmp.S b/arch/arm/mach-tegra/headsmp.S
index b5349b2..7973f1c 100644
--- a/arch/arm/mach-tegra/headsmp.S
+++ b/arch/arm/mach-tegra/headsmp.S
@@ -1,6 +1,23 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 
+#include <asm/cache.h>
+
+#include <mach/iomap.h>
+
+#include "flowctrl.h"
+#include "reset.h"
+
+#define APB_MISC_GP_HIDREV	0x804
+#define PMC_SCRATCH41	0x140
+
+#define RESET_DATA(x)	((TEGRA_RESET_##x)*4)
+
+	.macro mov32, reg, val
+	movw	\reg, #:lower16:\val
+	movt	\reg, #:upper16:\val
+	.endm
+
         .section ".text.head", "ax"
 	__CPUINIT
 
@@ -47,15 +64,116 @@ ENTRY(v7_invalidate_l1)
         mov     pc, lr
 ENDPROC(v7_invalidate_l1)
 
+
 ENTRY(tegra_secondary_startup)
-	msr	cpsr_fsxc, #0xd3
         bl      v7_invalidate_l1
-	mrc	p15, 0, r0, c0, c0, 5
-        and	r0, r0, #15
-        ldr     r1, =0x6000f100
-        str     r0, [r1]
-1:      ldr     r2, [r1]
-        cmp     r0, r2
-        beq     1b
+	mov32	r0, 0xC5ACCE55
+	mcr	p14, 0, r0, c7, c12, 6
         b       secondary_startup
 ENDPROC(tegra_secondary_startup)
+
+	.align L1_CACHE_SHIFT
+ENTRY(__tegra_cpu_reset_handler_start)
+
+/*
+ * __tegra_cpu_reset_handler:
+ *
+ * Common handler for all CPU reset events.
+ *
+ * Register usage within the reset handler:
+ *
+ *      R7  = CPU present (to the OS) mask
+ *      R8  = CPU in LP1 state mask
+ *      R9  = CPU in LP2 state mask
+ *      R10 = CPU number
+ *      R11 = CPU mask
+ *      R12 = pointer to reset handler data
+ *
+ * NOTE: This code is copied to IRAM. All code and data accesses
+ *       must be position-independent.
+ */
+
+	.align L1_CACHE_SHIFT
+ENTRY(__tegra_cpu_reset_handler)
+
+	cpsid	aif, 0x13			@ SVC mode, interrupts disabled
+	mrc	p15, 0, r10, c0, c0, 5		@ MPIDR
+	and	r10, r10, #0x3			@ R10 = CPU number
+	mov	r11, #1
+	mov	r11, r11, lsl r10  		@ R11 = CPU mask
+	adr	r12, __tegra_cpu_reset_handler_data
+
+#ifdef CONFIG_SMP
+	/* Does the OS know about this CPU? */
+	ldr	r7, [r12, #RESET_DATA(MASK_PRESENT)]
+	tst	r7, r11 			@ if !present
+	bleq	__die				@ CPU not present (to OS)
+#endif
+
+#ifdef CONFIG_ARCH_TEGRA_2x_SOC
+	/* Are we on Tegra20? */
+	mov32	r6, TEGRA_APB_MISC_BASE
+	ldr	r0, [r6, #APB_MISC_GP_HIDREV]
+	and	r0, r0, #0xff00
+	cmp	r0, #(0x20 << 8)
+	bne	1f
+	/* If not CPU0, don't let CPU0 reset CPU1 now that CPU1 is coming up. */
+	mov32	r6, TEGRA_PMC_BASE
+	mov	r0, #0
+	cmp	r10, #0
+	strne	r0, [r6, #PMC_SCRATCH41]
+1:
+#endif
+
+#ifdef CONFIG_SMP
+	/*
+	 * Can only be secondary boot (initial or hotplug) but CPU 0
+	 * cannot be here.
+	 */
+	cmp	r10, #0
+	bleq	__die				@ CPU0 cannot be here
+	ldr	lr, [r12, #RESET_DATA(STARTUP_SECONDARY)]
+	cmp	lr, #0
+	bleq	__die				@ no secondary startup handler
+	bx	lr
+#endif
+
+/*
+ * We don't know why the CPU reset. Just kill it.
+ * The LR register will contain the address we died at + 4.
+ */
+
+__die:
+	sub	lr, lr, #4
+	mov32	r7, TEGRA_PMC_BASE
+	str	lr, [r7, #PMC_SCRATCH41]
+
+	mov32	r7, TEGRA_CLK_RESET_BASE
+
+	/* Are we on Tegra20? */
+	mov32	r6, TEGRA_APB_MISC_BASE
+	ldr	r0, [r6, #APB_MISC_GP_HIDREV]
+	and	r0, r0, #0xff00
+	cmp	r0, #(0x20 << 8)
+	bne	1f
+
+#ifdef CONFIG_ARCH_TEGRA_2x_SOC
+	mov32	r0, 0x1111
+	mov	r1, r0, lsl r10
+	str	r1, [r7, #0x340]		@ CLK_RST_CPU_CMPLX_SET
+#endif
+1:
+	/* If the CPU still isn't dead, just spin here. */
+	b	.
+ENDPROC(__tegra_cpu_reset_handler)
+
+	.align L1_CACHE_SHIFT
+	.type	__tegra_cpu_reset_handler_data, %object
+	.globl	__tegra_cpu_reset_handler_data
+__tegra_cpu_reset_handler_data:
+	.rept	TEGRA_RESET_DATA_SIZE
+	.long	0
+	.endr
+	.align L1_CACHE_SHIFT
+
+ENTRY(__tegra_cpu_reset_handler_end)
diff --git a/arch/arm/mach-tegra/include/mach/iomap.h b/arch/arm/mach-tegra/include/mach/iomap.h
index 67644c9..cff672a 100644
--- a/arch/arm/mach-tegra/include/mach/iomap.h
+++ b/arch/arm/mach-tegra/include/mach/iomap.h
@@ -113,6 +113,9 @@
 #define TEGRA_AHB_GIZMO_BASE		0x6000C004
 #define TEGRA_AHB_GIZMO_SIZE		0x10C
 
+#define TEGRA_SB_BASE			0x6000C200
+#define TEGRA_SB_SIZE			256
+
 #define TEGRA_STATMON_BASE		0x6000C400
 #define TEGRA_STATMON_SIZE		SZ_1K
 
diff --git a/arch/arm/mach-tegra/platsmp.c b/arch/arm/mach-tegra/platsmp.c
index 7d2b5d0..79a241a 100644
--- a/arch/arm/mach-tegra/platsmp.c
+++ b/arch/arm/mach-tegra/platsmp.c
@@ -26,18 +26,26 @@
 
 #include <mach/iomap.h>
 
+#include "fuse.h"
+#include "flowctrl.h"
+#include "reset.h"
+
 extern void tegra_secondary_startup(void);
 
-static DEFINE_SPINLOCK(boot_lock);
 static void __iomem *scu_base = IO_ADDRESS(TEGRA_ARM_PERIF_BASE);
 
 #define EVP_CPU_RESET_VECTOR \
 	(IO_ADDRESS(TEGRA_EXCEPTION_VECTORS_BASE) + 0x100)
 #define CLK_RST_CONTROLLER_CLK_CPU_CMPLX \
 	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x4c)
+#define CLK_RST_CONTROLLER_RST_CPU_CMPLX_SET \
+	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x340)
 #define CLK_RST_CONTROLLER_RST_CPU_CMPLX_CLR \
 	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x344)
 
+#define CPU_CLOCK(cpu)	(0x1<<(8+cpu))
+#define CPU_RESET(cpu)	(0x1111ul<<(cpu))
+
 void __cpuinit platform_secondary_init(unsigned int cpu)
 {
 	/*
@@ -47,63 +55,62 @@ void __cpuinit platform_secondary_init(unsigned int cpu)
 	 */
 	gic_secondary_init(0);
 
-	/*
-	 * Synchronise with the boot thread.
-	 */
-	spin_lock(&boot_lock);
-	spin_unlock(&boot_lock);
 }
 
-int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
+static int tegra20_power_up_cpu(unsigned int cpu)
 {
-	unsigned long old_boot_vector;
-	unsigned long boot_vector;
-	unsigned long timeout;
 	u32 reg;
 
-	/*
-	 * set synchronisation state between this boot processor
-	 * and the secondary one
-	 */
-	spin_lock(&boot_lock);
-
-
-	/* set the reset vector to point to the secondary_startup routine */
-
-	boot_vector = virt_to_phys(tegra_secondary_startup);
-	old_boot_vector = readl(EVP_CPU_RESET_VECTOR);
-	writel(boot_vector, EVP_CPU_RESET_VECTOR);
-
-	/* enable cpu clock on cpu1 */
+	/* Enable the CPU clock. */
+	reg = readl(CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
+	writel(reg & ~CPU_CLOCK(cpu), CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
+	barrier();
 	reg = readl(CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
-	writel(reg & ~(1<<9), CLK_RST_CONTROLLER_CLK_CPU_CMPLX);
-
-	reg = (1<<13) | (1<<9) | (1<<5) | (1<<1);
-	writel(reg, CLK_RST_CONTROLLER_RST_CPU_CMPLX_CLR);
 
-	smp_wmb();
-	flush_cache_all();
+	/* Clear flow controller CSR. */
+	flowctrl_write_cpu_csr(cpu, 0);
 
-	/* unhalt the cpu */
-	writel(0, IO_ADDRESS(TEGRA_FLOW_CTRL_BASE) + 0x14);
+	return 0;
+}
 
-	timeout = jiffies + (1 * HZ);
-	while (time_before(jiffies, timeout)) {
-		if (readl(EVP_CPU_RESET_VECTOR) != boot_vector)
-			break;
-		udelay(10);
-	}
+int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
+{
+	int status;
 
-	/* put the old boot vector back */
-	writel(old_boot_vector, EVP_CPU_RESET_VECTOR);
+	/* Force the CPU into reset. The CPU must remain in reset when the
+	 * flow controller state is cleared (which will cause the flow
+	 * controller to stop driving reset if the CPU has been power-gated
+	 * via the flow controller). This will have no effect on first boot
+	 * of the CPU since it should already be in reset.
+	 */
+	writel(CPU_RESET(cpu), CLK_RST_CONTROLLER_RST_CPU_CMPLX_SET);
+	dmb();
 
 	/*
-	 * now the secondary core is starting up let it run its
-	 * calibrations, then wait for it to finish
+	 * Unhalt the CPU. If the flow controller was used to power-gate the
+	 * CPU this will cause the flow controller to stop driving reset.
+	 * The CPU will remain in reset because the clock and reset block
+	 * is now driving reset.
 	 */
-	spin_unlock(&boot_lock);
+	flowctrl_write_cpu_halt(cpu, 0);
+
+	switch (tegra_chip_id) {
+	case TEGRA20:
+		status = tegra20_power_up_cpu(cpu);
+		break;
+	default:
+		status = -EINVAL;
+		break;
+	}
 
-	return 0;
+	if (status)
+		goto done;
+
+	/* Take the CPU out of reset. */
+	writel(CPU_RESET(cpu), CLK_RST_CONTROLLER_RST_CPU_CMPLX_CLR);
+	wmb();
+done:
+	return status;
 }
 
 /*
@@ -128,6 +135,6 @@ void __init smp_init_cpus(void)
 
 void __init platform_smp_prepare_cpus(unsigned int max_cpus)
 {
-
+	tegra_cpu_reset_handler_init();
 	scu_enable(scu_base);
 }
diff --git a/arch/arm/mach-tegra/reset.c b/arch/arm/mach-tegra/reset.c
new file mode 100644
index 0000000..4d6a2ee
--- /dev/null
+++ b/arch/arm/mach-tegra/reset.c
@@ -0,0 +1,84 @@
+/*
+ * arch/arm/mach-tegra/reset.c
+ *
+ * Copyright (C) 2011,2012 NVIDIA Corporation.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/cpumask.h>
+#include <linux/bitops.h>
+
+#include <asm/cacheflush.h>
+#include <asm/hardware/cache-l2x0.h>
+
+#include <mach/iomap.h>
+#include <mach/irammap.h>
+
+#include "reset.h"
+#include "fuse.h"
+
+#define TEGRA_IRAM_RESET_BASE (TEGRA_IRAM_BASE + \
+				TEGRA_IRAM_RESET_HANDLER_OFFSET)
+
+static bool is_enabled;
+
+static void tegra_cpu_reset_handler_enable(void)
+{
+	void __iomem *iram_base = IO_ADDRESS(TEGRA_IRAM_RESET_BASE);
+	void __iomem *evp_cpu_reset =
+		IO_ADDRESS(TEGRA_EXCEPTION_VECTORS_BASE + 0x100);
+	void __iomem *sb_ctrl = IO_ADDRESS(TEGRA_SB_BASE);
+	u32 reg;
+
+	BUG_ON(is_enabled);
+	BUG_ON(tegra_cpu_reset_handler_size > TEGRA_IRAM_RESET_HANDLER_SIZE);
+
+	memcpy(iram_base, (void *)__tegra_cpu_reset_handler_start,
+			tegra_cpu_reset_handler_size);
+
+	/*
+	 * NOTE: This must be the one and only write to the EVP CPU reset
+	 *       vector in the entire system.
+	 */
+	writel(TEGRA_IRAM_RESET_BASE + tegra_cpu_reset_handler_offset,
+			evp_cpu_reset);
+	wmb();
+	reg = readl(evp_cpu_reset);
+
+	/*
+	 * Prevent further modifications to the physical reset vector.
+	 *  NOTE: Has no effect on chips prior to Tegra30.
+	 */
+	if (tegra_chip_id != TEGRA20) {
+		reg = readl(sb_ctrl);
+		reg |= 2;
+		writel(reg, sb_ctrl);
+		wmb();
+	}
+
+	is_enabled = true;
+}
+
+void __init tegra_cpu_reset_handler_init(void)
+{
+
+#ifdef CONFIG_SMP
+	__tegra_cpu_reset_handler_data[TEGRA_RESET_MASK_PRESENT] =
+		*((u32 *)cpu_present_mask);
+	__tegra_cpu_reset_handler_data[TEGRA_RESET_STARTUP_SECONDARY] =
+		virt_to_phys((void *)tegra_secondary_startup);
+#endif
+
+	tegra_cpu_reset_handler_enable();
+}
diff --git a/arch/arm/mach-tegra/reset.h b/arch/arm/mach-tegra/reset.h
new file mode 100644
index 0000000..de88bf8
--- /dev/null
+++ b/arch/arm/mach-tegra/reset.h
@@ -0,0 +1,50 @@
+/*
+ * arch/arm/mach-tegra/reset.h
+ *
+ * CPU reset dispatcher.
+ *
+ * Copyright (c) 2011, NVIDIA Corporation.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef __MACH_TEGRA_RESET_H
+#define __MACH_TEGRA_RESET_H
+
+#define TEGRA_RESET_MASK_PRESENT	0
+#define TEGRA_RESET_MASK_LP1		1
+#define TEGRA_RESET_MASK_LP2		2
+#define TEGRA_RESET_STARTUP_SECONDARY	3
+#define TEGRA_RESET_STARTUP_LP2		4
+#define TEGRA_RESET_STARTUP_LP1		5
+#define TEGRA_RESET_DATA_SIZE		6
+
+#ifndef __ASSEMBLY__
+
+extern unsigned long __tegra_cpu_reset_handler_data[TEGRA_RESET_DATA_SIZE];
+
+void __tegra_cpu_reset_handler_start(void);
+void __tegra_cpu_reset_handler(void);
+void __tegra_cpu_reset_handler_end(void);
+void tegra_secondary_startup(void);
+
+#define tegra_cpu_reset_handler_offset \
+		((u32)__tegra_cpu_reset_handler - \
+		 (u32)__tegra_cpu_reset_handler_start)
+
+#define tegra_cpu_reset_handler_size \
+		(__tegra_cpu_reset_handler_end - \
+		 __tegra_cpu_reset_handler_start)
+
+void __init tegra_cpu_reset_handler_init(void);
+
+#endif
+#endif
-- 
1.7.4.1

[PATCH v3 06/10] ARM: tegra: prepare powergate.c for multiple variants

[Peter De Schrijver]

Prepare the powergating code for other Tegra variants which have a different
number of powerdomains.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/include/mach/powergate.h |    1 -
 arch/arm/mach-tegra/powergate.c              |   33 +++++++++++++++++++++----
 2 files changed, 27 insertions(+), 7 deletions(-)

diff --git a/arch/arm/mach-tegra/include/mach/powergate.h b/arch/arm/mach-tegra/include/mach/powergate.h
index 39c396d..36846dc 100644
--- a/arch/arm/mach-tegra/include/mach/powergate.h
+++ b/arch/arm/mach-tegra/include/mach/powergate.h
@@ -27,7 +27,6 @@
 #define TEGRA_POWERGATE_VDEC	4
 #define TEGRA_POWERGATE_L2	5
 #define TEGRA_POWERGATE_MPE	6
-#define TEGRA_NUM_POWERGATE	7
 
 int tegra_powergate_power_on(int id);
 int tegra_powergate_power_off(int id);
diff --git a/arch/arm/mach-tegra/powergate.c b/arch/arm/mach-tegra/powergate.c
index 9483064..9b828f2 100644
--- a/arch/arm/mach-tegra/powergate.c
+++ b/arch/arm/mach-tegra/powergate.c
@@ -31,6 +31,8 @@
 #include <mach/iomap.h>
 #include <mach/powergate.h>
 
+#include "fuse.h"
+
 #define PWRGATE_TOGGLE		0x30
 #define  PWRGATE_TOGGLE_START	(1 << 8)
 
@@ -38,6 +40,8 @@
 
 #define PWRGATE_STATUS		0x38
 
+static int tegra_num_powerdomains;
+
 static DEFINE_SPINLOCK(tegra_powergate_lock);
 
 static void __iomem *pmc = IO_ADDRESS(TEGRA_PMC_BASE);
@@ -75,7 +79,7 @@ static int tegra_powergate_set(int id, bool new_state)
 
 int tegra_powergate_power_on(int id)
 {
-	if (id < 0 || id >= TEGRA_NUM_POWERGATE)
+	if (id < 0 || id >= tegra_num_powerdomains)
 		return -EINVAL;
 
 	return tegra_powergate_set(id, true);
@@ -83,17 +87,18 @@ int tegra_powergate_power_on(int id)
 
 int tegra_powergate_power_off(int id)
 {
-	if (id < 0 || id >= TEGRA_NUM_POWERGATE)
+	if (id < 0 || id >= tegra_num_powerdomains)
 		return -EINVAL;
 
 	return tegra_powergate_set(id, false);
 }
 
-static bool tegra_powergate_is_powered(int id)
+static int tegra_powergate_is_powered(int id)
 {
 	u32 status;
 
-	WARN_ON(id < 0 || id >= TEGRA_NUM_POWERGATE);
+	if (id < 0 || id >= tegra_num_powerdomains)
+		return -EINVAL;
 
 	status = pmc_read(PWRGATE_STATUS) & (1 << id);
 	return !!status;
@@ -103,7 +108,7 @@ int tegra_powergate_remove_clamping(int id)
 {
 	u32 mask;
 
-	if (id < 0 || id >= TEGRA_NUM_POWERGATE)
+	if (id < 0 || id >= tegra_num_powerdomains)
 		return -EINVAL;
 
 	/*
@@ -156,6 +161,22 @@ err_power:
 	return ret;
 }
 
+int __init tegra_powergate_init(void)
+{
+	switch (tegra_chip_id) {
+	case TEGRA20:
+		tegra_num_powerdomains = 7;
+		break;
+	default:
+		/* Unknown Tegra variant. Disable powergating */
+		tegra_num_powerdomains = 0;
+		break;
+	}
+
+	return 0;
+}
+arch_initcall(tegra_powergate_init);
+
 #ifdef CONFIG_DEBUG_FS
 
 static const char * const powergate_name[] = {
@@ -175,7 +196,7 @@ static int powergate_show(struct seq_file *s, void *data)
 	seq_printf(s, " powergate powered\n");
 	seq_printf(s, "------------------\n");
 
-	for (i = 0; i < TEGRA_NUM_POWERGATE; i++)
+	for (i = 0; i < tegra_num_powerdomains; i++)
 		seq_printf(s, " %9s %7s\n", powergate_name[i],
 			tegra_powergate_is_powered(i) ? "yes" : "no");
 	return 0;
-- 
1.7.4.1

[PATCH v3 07/10] ARM: tegra: export tegra_powergate_is_powered()

[Peter De Schrijver]

Export tegra_powergate_is_powered(). This function will be used by the Tegra30
code to bringup secondary CPU cores.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/include/mach/powergate.h |    1 +
 arch/arm/mach-tegra/powergate.c              |    2 +-
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-tegra/include/mach/powergate.h b/arch/arm/mach-tegra/include/mach/powergate.h
index 36846dc..0ec8ce1 100644
--- a/arch/arm/mach-tegra/include/mach/powergate.h
+++ b/arch/arm/mach-tegra/include/mach/powergate.h
@@ -28,6 +28,7 @@
 #define TEGRA_POWERGATE_L2	5
 #define TEGRA_POWERGATE_MPE	6
 
+int tegra_powergate_is_powered(int id);
 int tegra_powergate_power_on(int id);
 int tegra_powergate_power_off(int id);
 int tegra_powergate_remove_clamping(int id);
diff --git a/arch/arm/mach-tegra/powergate.c b/arch/arm/mach-tegra/powergate.c
index 9b828f2..984bbdf 100644
--- a/arch/arm/mach-tegra/powergate.c
+++ b/arch/arm/mach-tegra/powergate.c
@@ -93,7 +93,7 @@ int tegra_powergate_power_off(int id)
 	return tegra_powergate_set(id, false);
 }
 
-static int tegra_powergate_is_powered(int id)
+int tegra_powergate_is_powered(int id)
 {
 	u32 status;
 
-- 
1.7.4.1

[PATCH v3 08/10] ARM: tegra: add support for Tegra30 powerdomains

[Peter De Schrijver]

Add support for the new powerdomains in Tegra30 such as extra CPU cores and
the SATA domain.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/include/mach/powergate.h |   10 ++++++++++
 arch/arm/mach-tegra/powergate.c              |    3 +++
 2 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-tegra/include/mach/powergate.h b/arch/arm/mach-tegra/include/mach/powergate.h
index 0ec8ce1..ca41186 100644
--- a/arch/arm/mach-tegra/include/mach/powergate.h
+++ b/arch/arm/mach-tegra/include/mach/powergate.h
@@ -27,6 +27,16 @@
 #define TEGRA_POWERGATE_VDEC	4
 #define TEGRA_POWERGATE_L2	5
 #define TEGRA_POWERGATE_MPE	6
+#define TEGRA_POWERGATE_HEG	7
+#define TEGRA_POWERGATE_SATA	8
+#define TEGRA_POWERGATE_CPU1	9
+#define TEGRA_POWERGATE_CPU2	10
+#define TEGRA_POWERGATE_CPU3	11
+#define TEGRA_POWERGATE_CELP	12
+#define TEGRA_POWERGATE_3D1	13
+
+#define TEGRA_POWERGATE_CPU0	TEGRA_POWERGATE_CPU
+#define TEGRA_POWERGATE_3D0	TEGRA_POWERGATE_3D
 
 int tegra_powergate_is_powered(int id);
 int tegra_powergate_power_on(int id);
diff --git a/arch/arm/mach-tegra/powergate.c b/arch/arm/mach-tegra/powergate.c
index 984bbdf..7120ad7 100644
--- a/arch/arm/mach-tegra/powergate.c
+++ b/arch/arm/mach-tegra/powergate.c
@@ -167,6 +167,9 @@ int __init tegra_powergate_init(void)
 	case TEGRA20:
 		tegra_num_powerdomains = 7;
 		break;
+	case TEGRA30:
+		tegra_num_powerdomains = 14;
+		break;
 	default:
 		/* Unknown Tegra variant. Disable powergating */
 		tegra_num_powerdomains = 0;
-- 
1.7.4.1

[PATCH v3 09/10] ARM: tegra: support for Tegra30 CPU powerdomains

[Peter De Schrijver]

Secondary CPU powerdomains can be powergated on Tegra30. Add the necessary
functions to do this. This will be used to boot the secondary CPUs later on.

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/common.c                 |    3 +++
 arch/arm/mach-tegra/include/mach/powergate.h |    3 +++
 arch/arm/mach-tegra/powergate.c              |   19 ++++++++++++++++++-
 3 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-tegra/common.c b/arch/arm/mach-tegra/common.c
index bdc5d5e..68815ce 100644
--- a/arch/arm/mach-tegra/common.c
+++ b/arch/arm/mach-tegra/common.c
@@ -27,6 +27,7 @@
 #include <asm/hardware/gic.h>
 
 #include <mach/iomap.h>
+#include <mach/powergate.h>
 
 #include "board.h"
 #include "clock.h"
@@ -120,6 +121,7 @@ void __init tegra20_init_early(void)
 	tegra_clk_init_from_table(tegra20_clk_init_table);
 	tegra_init_cache(0x331, 0x441);
 	tegra_pmc_init();
+	tegra_powergate_init();
 }
 #endif
 #ifdef CONFIG_ARCH_TEGRA_3x_SOC
@@ -129,5 +131,6 @@ void __init tegra30_init_early(void)
 	tegra30_init_clocks();
 	tegra_init_cache(0x441, 0x551);
 	tegra_pmc_init();
+	tegra_powergate_init();
 }
 #endif
diff --git a/arch/arm/mach-tegra/include/mach/powergate.h b/arch/arm/mach-tegra/include/mach/powergate.h
index ca41186..4752b1a 100644
--- a/arch/arm/mach-tegra/include/mach/powergate.h
+++ b/arch/arm/mach-tegra/include/mach/powergate.h
@@ -38,6 +38,9 @@
 #define TEGRA_POWERGATE_CPU0	TEGRA_POWERGATE_CPU
 #define TEGRA_POWERGATE_3D0	TEGRA_POWERGATE_3D
 
+int  __init tegra_powergate_init(void);
+
+int tegra_cpu_powergate_id(int cpuid);
 int tegra_powergate_is_powered(int id);
 int tegra_powergate_power_on(int id);
 int tegra_powergate_power_off(int id);
diff --git a/arch/arm/mach-tegra/powergate.c b/arch/arm/mach-tegra/powergate.c
index 7120ad7..c238699 100644
--- a/arch/arm/mach-tegra/powergate.c
+++ b/arch/arm/mach-tegra/powergate.c
@@ -41,6 +41,14 @@
 #define PWRGATE_STATUS		0x38
 
 static int tegra_num_powerdomains;
+static int tegra_num_cpu_domains;
+static u8 *tegra_cpu_domains;
+static u8 tegra30_cpu_domains[] = {
+	TEGRA_POWERGATE_CPU0,
+	TEGRA_POWERGATE_CPU1,
+	TEGRA_POWERGATE_CPU2,
+	TEGRA_POWERGATE_CPU3,
+};
 
 static DEFINE_SPINLOCK(tegra_powergate_lock);
 
@@ -161,6 +169,14 @@ err_power:
 	return ret;
 }
 
+int tegra_cpu_powergate_id(int cpuid)
+{
+	if (cpuid > 0 && cpuid < tegra_num_cpu_domains)
+		return tegra_cpu_domains[cpuid];
+
+	return -EINVAL;
+}
+
 int __init tegra_powergate_init(void)
 {
 	switch (tegra_chip_id) {
@@ -169,6 +185,8 @@ int __init tegra_powergate_init(void)
 		break;
 	case TEGRA30:
 		tegra_num_powerdomains = 14;
+		tegra_num_cpu_domains = 4;
+		tegra_cpu_domains = tegra30_cpu_domains;
 		break;
 	default:
 		/* Unknown Tegra variant. Disable powergating */
@@ -178,7 +196,6 @@ int __init tegra_powergate_init(void)
 
 	return 0;
 }
-arch_initcall(tegra_powergate_init);
 
 #ifdef CONFIG_DEBUG_FS
 
-- 
1.7.4.1

[PATCH v3 10/10] ARM: tegra: support for secondary cores on Tegra30

[Peter De Schrijver]

Add support for bringing up secondary cores on Tegra30. On Tegra30 secondary
CPU cores are powergated, so we need to turn on the domains before we can bring
the CPU cores online. Bringing secondary cores online happens early during the
sytem boot, so we call powergating initialization from platform early_init
function.

Based on work by:

Scott Williams <scwilliams@nvidia.com>
Colin Cross <ccross@android.com>
Alex Frid <afrid@nvidia.com>

Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
---
 arch/arm/mach-tegra/headsmp.S |   32 ++++++++++++++++++++++++++
 arch/arm/mach-tegra/platsmp.c |   50 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 81 insertions(+), 1 deletions(-)

diff --git a/arch/arm/mach-tegra/headsmp.S b/arch/arm/mach-tegra/headsmp.S
index 7973f1c..b63a4f7 100644
--- a/arch/arm/mach-tegra/headsmp.S
+++ b/arch/arm/mach-tegra/headsmp.S
@@ -163,6 +163,38 @@ __die:
 	str	r1, [r7, #0x340]		@ CLK_RST_CPU_CMPLX_SET
 #endif
 1:
+#ifdef CONFIG_ARCH_TEGRA_3x_SOC
+	mov32	r6, TEGRA_FLOW_CTRL_BASE
+
+	cmp	r10, #0
+	moveq	r1, #FLOW_CTRL_HALT_CPU0_EVENTS
+	moveq	r2, #FLOW_CTRL_CPU0_CSR
+	movne	r1, r10, lsl #3
+	addne	r2, r1, #(FLOW_CTRL_CPU1_CSR-8)
+	addne	r1, r1, #(FLOW_CTRL_HALT_CPU1_EVENTS-8)
+
+	/* Clear CPU "event" and "interrupt" flags and power gate
+	   it when halting but not before it is in the "WFI" state. */
+	ldr	r0, [r6, +r2]
+	orr	r0, r0, #FLOW_CTRL_CSR_INTR_FLAG | FLOW_CTRL_CSR_EVENT_FLAG
+	orr	r0, r0, #FLOW_CTRL_CSR_ENABLE
+	str	r0, [r6, +r2]
+
+	/* Unconditionally halt this CPU */
+	mov	r0, #FLOW_CTRL_WAITEVENT
+	str	r0, [r6, +r1]
+	ldr	r0, [r6, +r1]			@ memory barrier
+
+	dsb
+	isb
+	wfi					@ CPU should be power gated here
+
+	/* If the CPU didn't power gate above just kill it's clock. */
+
+	mov	r0, r11, lsl #8
+	str	r0, [r7, #348]			@ CLK_CPU_CMPLX_SET
+#endif
+
 	/* If the CPU still isn't dead, just spin here. */
 	b	.
 ENDPROC(__tegra_cpu_reset_handler)
diff --git a/arch/arm/mach-tegra/platsmp.c b/arch/arm/mach-tegra/platsmp.c
index 79a241a..1a208db 100644
--- a/arch/arm/mach-tegra/platsmp.c
+++ b/arch/arm/mach-tegra/platsmp.c
@@ -24,7 +24,9 @@
 #include <asm/mach-types.h>
 #include <asm/smp_scu.h>
 
+#include <mach/clk.h>
 #include <mach/iomap.h>
+#include <mach/powergate.h>
 
 #include "fuse.h"
 #include "flowctrl.h"
@@ -42,6 +44,8 @@ static void __iomem *scu_base = IO_ADDRESS(TEGRA_ARM_PERIF_BASE);
 	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x340)
 #define CLK_RST_CONTROLLER_RST_CPU_CMPLX_CLR \
 	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x344)
+#define CLK_RST_CONTROLLER_CLK_CPU_CMPLX_CLR \
+	(IO_ADDRESS(TEGRA_CLK_RESET_BASE) + 0x34c)
 
 #define CPU_CLOCK(cpu)	(0x1<<(8+cpu))
 #define CPU_RESET(cpu)	(0x1111ul<<(cpu))
@@ -73,11 +77,52 @@ static int tegra20_power_up_cpu(unsigned int cpu)
 	return 0;
 }
 
+static int tegra30_power_up_cpu(unsigned int cpu)
+{
+	u32 reg;
+	int ret, pwrgateid;
+	unsigned long timeout;
+
+	pwrgateid = tegra_cpu_powergate_id(cpu);
+	if (pwrgateid < 0)
+		return pwrgateid;
+
+	/* If this is the first boot, toggle powergates directly. */
+	if (!tegra_powergate_is_powered(pwrgateid)) {
+		ret = tegra_powergate_power_on(pwrgateid);
+		if (ret)
+			return ret;
+
+		/* Wait for the power to come up. */
+		timeout = jiffies + 10*HZ;
+		while (tegra_powergate_is_powered(pwrgateid)) {
+			if (time_after(jiffies, timeout))
+				return -ETIMEDOUT;
+			udelay(10);
+		}
+	}
+
+	/* CPU partition is powered. Enable the CPU clock. */
+	writel(CPU_CLOCK(cpu), CLK_RST_CONTROLLER_CLK_CPU_CMPLX_CLR);
+	reg = readl(CLK_RST_CONTROLLER_CLK_CPU_CMPLX_CLR);
+	udelay(10);
+
+	/* Remove I/O clamps. */
+	ret = tegra_powergate_remove_clamping(pwrgateid);
+	udelay(10);
+
+	/* Clear flow controller CSR. */
+	flowctrl_write_cpu_csr(cpu, 0);
+
+	return 0;
+}
+
 int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 {
 	int status;
 
-	/* Force the CPU into reset. The CPU must remain in reset when the
+	/*
+	 * Force the CPU into reset. The CPU must remain in reset when the
 	 * flow controller state is cleared (which will cause the flow
 	 * controller to stop driving reset if the CPU has been power-gated
 	 * via the flow controller). This will have no effect on first boot
@@ -98,6 +143,9 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
 	case TEGRA20:
 		status = tegra20_power_up_cpu(cpu);
 		break;
+	case TEGRA30:
+		status = tegra30_power_up_cpu(cpu);
+		break;
 	default:
 		status = -EINVAL;
 		break;
-- 
1.7.4.1

RE: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Stephen Warren]

Peter De Schrijver wrote at Thursday, February 09, 2012 4:48 PM:
> This patchset introduces support for secondary cores on tegra30. It also
> introduces some functions for the flowcontroller and adds basic support for
> tegra30 powerdomains.

The code looks fine to me. I do notice the following when running
"shutdown -h now":

[   27.762124] SMP: failed to stop secondary CPUs

So, this patch series makes the secondary cores boot OK, but not shut
down it appears. Is that expected? It's probably something we can fix
with a following on patch rather than re-spinning this series though?

-- 
nvpublic

Re: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Peter De Schrijver]

On Wed, Feb 15, 2012 at 10:27:43PM +0100, Stephen Warren wrote:
> Peter De Schrijver wrote at Thursday, February 09, 2012 4:48 PM:
> > This patchset introduces support for secondary cores on tegra30. It also
> > introduces some functions for the flowcontroller and adds basic support for
> > tegra30 powerdomains.
> 
> The code looks fine to me. I do notice the following when running
> "shutdown -h now":
> 
> [   27.762124] SMP: failed to stop secondary CPUs
> 
> So, this patch series makes the secondary cores boot OK, but not shut
> down it appears. Is that expected? It's probably something we can fix
> with a following on patch rather than re-spinning this series though?
> 

This seems to be a bug indeed. I will look at it next week when I'm back in
finland.

Cheers,

Peter.

Re: [PATCH v3 05/10] ARM: tegra: rework Tegra secondary CPU core bringup

[Colin Cross]

On Thu, Feb 9, 2012 at 3:47 PM, Peter De Schrijver
<pdeschrijver@nvidia.com> wrote:
> Prepare the Tegra secondary CPU core bringup code for other Tegra variants.
> The reset handler is also generalized to allow for future introduction of
> powersaving modes which turn off the CPU cores.
>
> Based on work by:
>
> Scott Williams <scwilliams@nvidia.com>
> Chris Johnson <cwj@nvidia.com>
> Colin Cross <ccross@android.com>
>
> Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
> ---
<snip>

> diff --git a/arch/arm/mach-tegra/headsmp.S b/arch/arm/mach-tegra/headsmp.S
> index b5349b2..7973f1c 100644
> --- a/arch/arm/mach-tegra/headsmp.S
> +++ b/arch/arm/mach-tegra/headsmp.S

<snip>

> @@ -47,15 +64,116 @@ ENTRY(v7_invalidate_l1)
>         mov     pc, lr
>  ENDPROC(v7_invalidate_l1)
>
> +
>  ENTRY(tegra_secondary_startup)
> -       msr     cpsr_fsxc, #0xd3
>         bl      v7_invalidate_l1
> -       mrc     p15, 0, r0, c0, c0, 5
> -        and    r0, r0, #15
> -        ldr     r1, =0x6000f100
> -        str     r0, [r1]
> -1:      ldr     r2, [r1]
> -        cmp     r0, r2
> -        beq     1b
> +       mov32   r0, 0xC5ACCE55
> +       mcr     p14, 0, r0, c7, c12, 6

One minor nit, you should comment that this is enabling coresight.

Re: [PATCH v3 02/10] ARM: tegra: export Tegra chipid

[Colin Cross]

On Thu, Feb 9, 2012 at 3:47 PM, Peter De Schrijver
<pdeschrijver@nvidia.com> wrote:
> The powergating and reset handling code needs to differentiate between Tegra
> variants. Therefore we export the chipid here.
>
> Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
> ---
>  arch/arm/mach-tegra/fuse.c |    2 +-
>  arch/arm/mach-tegra/fuse.h |    1 +
>  2 files changed, 2 insertions(+), 1 deletions(-)
>
> diff --git a/arch/arm/mach-tegra/fuse.c b/arch/arm/mach-tegra/fuse.c
> index 032a936..3298880 100644
> --- a/arch/arm/mach-tegra/fuse.c
> +++ b/arch/arm/mach-tegra/fuse.c
> @@ -34,7 +34,7 @@
>  int tegra_sku_id;
>  int tegra_cpu_process_id;
>  int tegra_core_process_id;
> -static int tegra_chip_id;
> +int tegra_chip_id;
>  enum tegra_revision tegra_revision;
>
>  /* The BCT to use at boot is specified by board straps that can be read
> diff --git a/arch/arm/mach-tegra/fuse.h b/arch/arm/mach-tegra/fuse.h
> index c554572..d2107b2 100644
> --- a/arch/arm/mach-tegra/fuse.h
> +++ b/arch/arm/mach-tegra/fuse.h
> @@ -41,6 +41,7 @@ enum tegra_revision {
>  extern int tegra_sku_id;
>  extern int tegra_cpu_process_id;
>  extern int tegra_core_process_id;
> +extern int tegra_chip_id;
>  extern enum tegra_revision tegra_revision;
>
>  extern int tegra_bct_strapping;

Other platforms use accessors for this (cpu_is_xxx(), although
soc_is_xxx() is probably more appropriate).

Re: [PATCH v3 02/10] ARM: tegra: export Tegra chipid

[Olof Johansson]

Hi,

On Mon, Feb 20, 2012 at 3:13 PM, Colin Cross <ccross@android.com> wrote:

> Other platforms use accessors for this (cpu_is_xxx(), although
> soc_is_xxx() is probably more appropriate).

I have previously objected to additions of said functions on tegra,
since they tend to start crawling into drivers as runtime tests
instead of providing more appropriate detection at probe time.

I can be persuaded differently but I haven't yet seen a strong case
for changing my mind. :)


-Olof

Re: [PATCH v3 02/10] ARM: tegra: export Tegra chipid

[Colin Cross]

On Mon, Feb 20, 2012 at 3:30 PM, Olof Johansson <olof@lixom.net> wrote:
> Hi,
>
> On Mon, Feb 20, 2012 at 3:13 PM, Colin Cross <ccross@android.com> wrote:
>
>> Other platforms use accessors for this (cpu_is_xxx(), although
>> soc_is_xxx() is probably more appropriate).
>
> I have previously objected to additions of said functions on tegra,
> since they tend to start crawling into drivers as runtime tests
> instead of providing more appropriate detection at probe time.
>
> I can be persuaded differently but I haven't yet seen a strong case
> for changing my mind. :)

soc_is_xxx() is better than spreading tegra_chip_id == XXX all over
the place, but keeping them out of drivers completely is even better.

Re: [PATCH v3 05/10] ARM: tegra: rework Tegra secondary CPU core bringup

[Olof Johansson]

Hi,

On Mon, Feb 20, 2012 at 03:04:57PM -0800, Colin Cross wrote:
> On Thu, Feb 9, 2012 at 3:47 PM, Peter De Schrijver
> <pdeschrijver@nvidia.com> wrote:
> > Prepare the Tegra secondary CPU core bringup code for other Tegra variants.
> > The reset handler is also generalized to allow for future introduction of
> > powersaving modes which turn off the CPU cores.
> >
> > Based on work by:
> >
> > Scott Williams <scwilliams@nvidia.com>
> > Chris Johnson <cwj@nvidia.com>
> > Colin Cross <ccross@android.com>
> >
> > Signed-off-by: Peter De Schrijver <pdeschrijver@nvidia.com>
> > ---
> <snip>
> 
> > diff --git a/arch/arm/mach-tegra/headsmp.S b/arch/arm/mach-tegra/headsmp.S
> > index b5349b2..7973f1c 100644
> > --- a/arch/arm/mach-tegra/headsmp.S
> > +++ b/arch/arm/mach-tegra/headsmp.S
> 
> <snip>
> 
> > @@ -47,15 +64,116 @@ ENTRY(v7_invalidate_l1)
> >         mov     pc, lr
> >  ENDPROC(v7_invalidate_l1)
> >
> > +
> >  ENTRY(tegra_secondary_startup)
> > -       msr     cpsr_fsxc, #0xd3
> >         bl      v7_invalidate_l1
> > -       mrc     p15, 0, r0, c0, c0, 5
> > -        and    r0, r0, #15
> > -        ldr     r1, =0x6000f100
> > -        str     r0, [r1]
> > -1:      ldr     r2, [r1]
> > -        cmp     r0, r2
> > -        beq     1b
> > +       mov32   r0, 0xC5ACCE55
> > +       mcr     p14, 0, r0, c7, c12, 6
> 
> One minor nit, you should comment that this is enabling coresight.

Since it's trivial, I did it when I applied the patch here instead of having
Peter respin or add it separately.


-Olof

Re: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Olof Johansson]

On Fri, Feb 10, 2012 at 01:47:40AM +0200, Peter De Schrijver wrote:
> This patchset introduces support for secondary cores on tegra30. It also
> introduces some functions for the flowcontroller and adds basic support for
> tegra30 powerdomains.

Thanks, I've applied this series in a new for-3.4/t3-smp branch: It depends and
pulls in both for-3.4/soc and for-3.4/soc-drivers, but I didn't want to just
add it on top of soc-drivers on the same topic.

I added the comment to headsmp.S that Colin had a nit about, besides that it
was just a couple of minor trivial context conflicts that I fixed up.

Please follow up with separate patches for the shutdown issues, etc.


Thanks!


-Olof

Re: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Peter De Schrijver]

[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]

On Wed, Feb 15, 2012 at 10:27:43PM +0100, Stephen Warren wrote:
> Peter De Schrijver wrote at Thursday, February 09, 2012 4:48 PM:
> > This patchset introduces support for secondary cores on tegra30. It also
> > introduces some functions for the flowcontroller and adds basic support for
> > tegra30 powerdomains.
> 
> The code looks fine to me. I do notice the following when running
> "shutdown -h now":
> 
> [   27.762124] SMP: failed to stop secondary CPUs
> 
> So, this patch series makes the secondary cores boot OK, but not shut
> down it appears. Is that expected? It's probably something we can fix
> with a following on patch rather than re-spinning this series though?
> 

I spent some more time looking at this. It appears that nr_online_cpus isn't
always updated properly. I added an extra pr_warn() when the failure happens
and I got:

[   22.369815] Restarting system.
[   23.506792] SMP: failed to stop secondary CPUs
[   23.511535] running on CPU: 3, still online: 2, mask: 8

I also did some function tracing and all CPUs besides the one initiating the
halt reach platform_cpu_kill() about 50us after sending the IPIs to stop them.
The set_cpu_online() call in ipi_cpu_stop() should update nr_online_cpus, but
somehow this doesn't always happen. Any insight on what might be going on?

Instrumentation patch attached.

Cheers,

Peter.


[-- Attachment #2: 0001-ARM-debugging-smp_send_stop.patch --]
[-- Type: text/x-diff, Size: 1189 bytes --]

>From d871feda574bd9f626f175be403cabd308ca18c0 Mon Sep 17 00:00:00 2001
From: Peter De Schrijver <pdeschrijver@nvidia.com>
Date: Wed, 29 Feb 2012 17:37:27 +0200
Subject: [PATCH] ARM: debugging smp_send_stop
X-NVConfidentiality: public

Change-Id: Ia4f33b054f2eccf173a3d977686e1577e2146b92
---
 arch/arm/kernel/smp.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 3e43c5f..e0e5eea 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -570,6 +570,7 @@ void smp_send_reschedule(int cpu)
 void smp_send_stop(void)
 {
 	unsigned long timeout;
+	int cpu = smp_processor_id();
 
 	if (num_online_cpus() > 1) {
 		cpumask_t mask = cpu_online_map;
@@ -583,8 +584,12 @@ void smp_send_stop(void)
 	while (num_online_cpus() > 1 && timeout--)
 		udelay(1);
 
-	if (num_online_cpus() > 1)
+	if (num_online_cpus() > 1) {
+		char cpu_mask[10];
 		pr_warning("SMP: failed to stop secondary CPUs\n");
+		cpumask_scnprintf(cpu_mask, sizeof(cpu_mask), cpu_online_mask);
+		pr_warn("running on CPU: %d, still online: %d, mask: %s\n", cpu, num_online_cpus(), cpu_mask);
+	}
 }
 
 /*
-- 
1.7.7.rc0.72.g4b5ea.dirty

RE: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Stephen Warren]

Peter De Schrijver wrote at Wednesday, February 29, 2012 8:42 AM:
> On Wed, Feb 15, 2012 at 10:27:43PM +0100, Stephen Warren wrote:
> > Peter De Schrijver wrote at Thursday, February 09, 2012 4:48 PM:
> > > This patchset introduces support for secondary cores on tegra30. It also
> > > introduces some functions for the flowcontroller and adds basic support for
> > > tegra30 powerdomains.
> >
> > The code looks fine to me. I do notice the following when running
> > "shutdown -h now":
> >
> > [   27.762124] SMP: failed to stop secondary CPUs

I can't seem to reproduce this at all on next-20120229 anymore. What
kind of repro frequency are you seeing, and does it get fixed if you
update to that baseline?

(I tried 20 times with shutdown, and 15 of them I reverted all the CPU
and cache errata in case they affected the issue at all)

-- 
nvpublic

Re: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Peter De Schrijver]

On Wed, Feb 29, 2012 at 11:38:16PM +0100, Stephen Warren wrote:
> Peter De Schrijver wrote at Wednesday, February 29, 2012 8:42 AM:
> > On Wed, Feb 15, 2012 at 10:27:43PM +0100, Stephen Warren wrote:
> > > Peter De Schrijver wrote at Thursday, February 09, 2012 4:48 PM:
> > > > This patchset introduces support for secondary cores on tegra30. It also
> > > > introduces some functions for the flowcontroller and adds basic support for
> > > > tegra30 powerdomains.
> > >
> > > The code looks fine to me. I do notice the following when running
> > > "shutdown -h now":
> > >
> > > [   27.762124] SMP: failed to stop secondary CPUs
> 
> I can't seem to reproduce this at all on next-20120229 anymore. What
> kind of repro frequency are you seeing, and does it get fixed if you
> update to that baseline?
> 

I see it abount once every 3 reboots. I have a an expect script which
automates the test. I haven't tried with next-20120229 yet.

> (I tried 20 times with shutdown, and 15 of them I reverted all the CPU
> and cache errata in case they affected the issue at all)

Cheers,

Peter.

Re: [PATCH v3 00/10] Add tegra30 support for secondary cores

[Peter De Schrijver]

On Thu, Mar 01, 2012 at 12:42:11PM +0100, Peter De Schrijver wrote:
> On Wed, Feb 29, 2012 at 11:38:16PM +0100, Stephen Warren wrote:
> > Peter De Schrijver wrote at Wednesday, February 29, 2012 8:42 AM:
> > > On Wed, Feb 15, 2012 at 10:27:43PM +0100, Stephen Warren wrote:
> > > > Peter De Schrijver wrote at Thursday, February 09, 2012 4:48 PM:
> > > > > This patchset introduces support for secondary cores on tegra30. It also
> > > > > introduces some functions for the flowcontroller and adds basic support for
> > > > > tegra30 powerdomains.
> > > >
> > > > The code looks fine to me. I do notice the following when running
> > > > "shutdown -h now":
> > > >
> > > > [   27.762124] SMP: failed to stop secondary CPUs
> > 
> > I can't seem to reproduce this at all on next-20120229 anymore. What
> > kind of repro frequency are you seeing, and does it get fixed if you
> > update to that baseline?
> > 
> 
> I see it abount once every 3 reboots. I have a an expect script which
> automates the test. I haven't tried with next-20120229 yet.
> 
> > (I tried 20 times with shutdown, and 15 of them I reverted all the CPU
> > and cache errata in case they affected the issue at all)

Tried it with next-20120229 and it works fine here. I also noticed
next-20120229 does not have https://lkml.org/lkml/2012/1/31/444
which was part of the earlier tree I used for testing. So num_online_cpus()
recalculates the number of online CPUs everytime instead of trying to only
recalculate it when there is a change.

Cheers,

Peter.